Skip to main content
Select required features before the first apply so their credentials, permissions, quotas, and network access are ready. Each HCL snippet shows module inputs; merge it into module "runlayer_infrastructure". Examples enable features and do not represent the defaults.

Bedrock and Anthropic

On this ECS deployment path, Runlayer Assistant uses Bedrock in the deployment account by default (model global.anthropic.claude-opus-4-8). Having “some Anthropic model” enabled in the console is not enough — that specific model (or its base) must be available. To run it on one of your own providers instead (for example an OpenAI-compatible gateway), an admin picks it under Runlayer Assistant model in Settings → AI Providers. Module v28.1.x+ attaches the model-access IAM actions to the ECS task role (GetFoundationModelAvailability, PutUseCaseForModelAccess, Marketplace subscribe, etc.). You do not add those by hand unless an SCP / boundary strips them.
If Assistant chat returns does not have access to the Anthropic model yet, see troubleshooting.

Agent runtime

Runlayer Assistant, Security Investigations, and Procedures run on the agent runtime (Bedrock AgentCore). Without it, Assistant chat fails with No agent runtime is provisioned for this deployment.
The runtime is independent of custom Agents, a separate entitlement Runlayer enables for your workspace. enable_runlayer_agents only adds infrastructure custom agents use (the agent event-trigger consumer) and requires the runtime. AgentCore calls your application URL. The default PUBLIC network mode reaches it over the internet; with WAF IP allowlisting, use VPC mode and PrivateLink. Before applying, check the runtime’s egress and AWS permissions; the apply host also needs the AgentCore prerequisites in the module README. Sandbox options are under Agent Sandbox.

Distribution configuration

The deployment guide configures Distribution with all three inputs below. Obtain a registered reader key from Runlayer before the first apply:
The module stores the key in Secrets Manager and injects the URL as non-secret runtime configuration. It rejects flagd without both reader inputs. The application default remains WorkOS when the provider input is omitted; setting the key and URL alone does not switch providers. Existing WorkOS deployments can stage the reader credentials before explicitly switching to flagd. If Runlayer has registered your account for self-managed Distribution registration, also pass the IDs Runlayer support provides and the module manages your registry manifest as a Terraform resource:
Rotate keys with zero gap by setting distribution_api_key_previous to the outgoing key, applying, waiting for ECS services to reach steady state, then clearing it and applying again. Set distribution_status = "disabled" to revoke access without destroying the stack; move openfeature_provider off flagd first.

Optional feature flags

Enable only the features you intend to operate. ToolGuard needs GPU quota and bootstrap egress. With WAF IP allowlisting, configure AgentCore VPC mode and PrivateLink before enabling the agent runtime.

Which optional services need EC2 or a GPU

The application services run on ECS Fargate. Three optional ML services run on EC2 instead, and all three are off by default: Rev always runs on g6f.large. ToolGuard defaults to g6f.large, and runlayer_tool_guard_instance_type selects another single-GPU g6f, g6 or g6e size. Check GPU quota and availability of the chosen types in your private subnets’ AZs before you turn either on. Runlayer Deploy is always enabled by the ECS module.
The audit-log consumer deploys start-before-stop, like SIEM export below: ECS stops an old task only once the replacement’s health check reports that it holds its share of the shards, so each shard is unread for a few seconds per deploy (records wait in the stream). A backend image without this readiness signal passes the health check only after a 4-minute fallback, so each deploy of such an image takes that much longer. Saved Bedrock providers for Slack monitors. The agent-event-trigger consumer uses its dedicated ECS task role to assume the saved provider role. The module grants sts:AssumeRole only on arn:aws:iam::*:role/runlayer-bedrock-* and arn:aws:iam::*:role/RunlayerBedrock*. The saved role must trust the caller’s AWS account/identity with the saved external ID and grant the required model access. When both BEDROCK_ENABLED and BEDROCK_ANTHROPIC_MODELS_ENABLED are enabled, the consumer also receives nonstreaming bedrock:InvokeModel access to Claude foundation models and the deployment account’s Claude inference profiles for the legacy platform fallback. Upgrade and apply the Terraform module alongside the backend/consumer image; updating the image alone does not install this permission. After IAM propagation, verify an authorized monitor canary and its dead-letter alarm.

ToolGuard capacity and scaling

ToolGuard scale-out speed. A new ToolGuard GPU host installs NVIDIA GRID drivers at boot and takes 10–15 minutes before it serves traffic, so a scale-out decision is only as fast as the spare hosts already running. That boot path also needs outbound HTTPS to Ubuntu, NVIDIA GRID S3, Docker, and related hosts — see Self-Hosted Egress Requirements → ToolGuard. runlayer_tool_guard_scaling_profile trades idle GPU cost for time-to-capacity: CPU targets are a percent of a g6f.large task’s CPU reservation; on a larger runlayer_tool_guard_instance_type the same CPU is a smaller percent (balanced 35% becomes 15% on g6.xlarge). balanced and aggressive keep one to two extra g6f.large instances running at all times (billed even when idle) so the next task starts immediately. Scale-in waits 15 minutes on every profile. The dashboard’s Tasks: Desired vs Running panel shows how long scale-outs wait on new hosts. The runlayer-tool-guard-scale-out-lag alarm fires only when desired tasks exceed running tasks for 20 consecutive minutes — longer than a cold GPU host boot — so a routine scale-out does not trip it; when it does fire, new hosts are not arriving (check the Auto Scaling group and capacity provider). GPU signal on aggressive. CPU is a weak load signal on a GPU inference host, so aggressive adds a second target-tracking policy on fleet-average GPU utilization (target 60%). The policy tracks the existing RunlayerToolGuard/GPUUtilization metric (Environment dimension, Average statistic): every host whose ToolGuard answers /health 200 publishes one sample per minute, so the per-minute average is the mean across serving GPUs. Hosts that are not serving (capacity-provider spares, hosts still installing drivers or loading the model, hosts in the recycle drain) publish nothing and are left out, so the spare pool cannot hold the average under the target while serving GPUs are busy. No per-host series is added, so CloudWatch metric cardinality is unchanged. Scaling takes the higher of the CPU and GPU decisions, so the GPU policy never forces the fleet below what the CPU or predictive policies request. Hosts publish GPU metrics from their boot script every 60 seconds, so changes to the ToolGuard launch template trigger an ASG rolling instance refresh that replaces the hosts: tasks restart on new hosts as old hosts drain (each restart reloads the model, so expect a capacity dip per batch). Until the refresh replaces them, hosts from the previous launch template still publish every 5 minutes and so land in only one of every five 1-minute averages. GPU alarms. On every profile, with enable_runlayer_tool_guard_alarms (default on), two alarms watch the same GPU utilization series, which CPU-based scaling and the high-cpu alarm do not see. With today’s models a ToolGuard host runs out of CPU well before its GPU, so GPU is rarely the limit.
  • runlayer-tool-guard-high-gpu fires when fleet-average GPU utilization stays above 70% for 15 minutes (three 5-minute averages): the whole fleet is GPU-bound, usually after a model or batch change. It does not see one busy host among idle ones (one of two hosts at 100% averages about 50%).
  • runlayer-tool-guard-gpu-pegged fires when the busiest host reads above 90% in 10 of 15 minutes: a GPU held by a stuck kernel or process, or a model or batch change that made serving GPU-bound. The metric has no host dimension, so run nvidia-smi on each host. Replace a single pegged host; on a fleet at its minimum, raise the minimum by one first and let the new host serve, since a host takes up to 15 minutes to boot. If several hosts are high, roll back a recent model or batch change, or treat it like high-gpu.
Predict latency alarm. runlayer-tool-guard-predict-p95 fires when the p95 of every successful predict request, fleet-wide and server-side, is above a threshold in 12 of the last 15 minutes (minutes with fewer than 10 requests don’t count): the fleet is past its latency knee. Short bursts of concurrent requests slow only the minute they land in, so they don’t fire it. The threshold defaults by runlayer_tool_guard_instance_type: 400 ms on g6f.large (and on types not yet measured), 150 ms on g6.xlarge; during an instance type cutover, the higher of the two. Set runlayer_tool_guard_predict_p95_alarm_threshold_ms if your busy-hour p95 sits near the default. When it fires, check whether the service is at its task ceiling (raise runlayer_tool_guard_max_count) or a scale-out is stuck (scale-out-lag); after a model or batch change at normal load, roll that change back. high-gpu, gpu-pegged and predict-p95 are non-paging alerts. On ALARM they notify the ToolGuard alerts topic and, with enable_monitoring, the topics in critical_alerts_extra_sns_topic_arns. On OK they notify only those extra topics, like the ToolGuard alarms that page. GPU query timeout alarm. A host whose GPU or driver is wedged stops publishing GPU utilization, so high-gpu, gpu-pegged and GPU scaling cannot see it. Its nvidia-smi read gives up after 10 seconds and publishes RunlayerToolGuard/GPUQueryTimeouts instead, and runlayer-tool-guard-gpu-query-timeout fires when that happens in 3 of 5 minutes, counted across the fleet. It pages like gpu-errors. The same metric with an InstanceId dimension names the host; terminate it. The alarm deliberately pages instead of replacing the host automatically: a driver fault on every host would otherwise replace the whole fleet into the same fault. Predictive scaling. Because a reactive scale-out cannot beat the 10–15 minute host boot, the module also creates an ECS predictive scaling policy on the ToolGuard service. It forecasts the next 48 hours of demand hourly from up to 14 days of ToolGuard predict request history and raises the task count runlayer_tool_guard_predictive_scheduling_buffer_seconds (default 1200) before each forecast hour, so GPU hosts are booting before the load arrives. conservative runs it in forecast_only mode (forecasts are recorded but never acted on); balanced and aggressive run forecast_and_scale. Override the profile default with runlayer_tool_guard_predictive_scaling_mode (disabled, forecast_only, forecast_and_scale). The first forecast appears after 24 hours of metrics; review it in the ECS console (service → Service auto scaling) or with aws application-autoscaling get-predictive-scaling-forecast, then flip forecast_only → forecast_and_scale once the curve matches your traffic. Predictive scaling only scales out; the CPU policy still handles scale-in and unforecast bursts. Every policy stops below runlayer_tool_guard_max_count, which reserves ceil(25%) of it, at least one host (8 allows 6 tasks, 4 allows 3, 2 allows 1), because each task needs a whole GPU host and a rolling deploy starts each replacement task before it stops the old one. runlayer_tool_guard_max_count stays the GPU host limit. In regions where ECS predictive scaling is unavailable (for example il-central-1) the policy is skipped automatically; the runlayer_tool_guard_predictive_scaling_mode_effective output reports the mode actually applied. Why the load is floored. The forecast learns daily and weekly patterns from the load metric, so a few days with scanning switched off would teach it that those weekdays are idle and under-provision them for the next two weeks. Before forecasting, weekday hours (Mon–Fri UTC) below runlayer_tool_guard_predictive_load_floor_fraction (default 0.5) of the window’s mean hourly request count are raised to that floor, so off days look like quiet business days instead of zero; weekends forecast from actual traffic (runlayer_tool_guard_predictive_load_floor_weekdays_only = false floors every day, 0 disables the floor). Capacity is sized so each task receives runlayer_tool_guard_predictive_target_requests_per_task_per_minute. Left null (default), it is the per-task rate measured for runlayer_tool_guard_instance_type: 50 per minute on g6f.large (3000 per hour), 250 on g6.xlarge, and 50 on other, unmeasured types (which over-provisions a faster host). On g6f.large, 50 keeps ToolGuard predict p95 near its floor; latency rises steeply once a task serves more than roughly twice that rate. 250 on g6.xlarge keeps the same headroom: a benchmark showed a g6.xlarge task serving at least five times what a g6f.large task does. While runlayer_tool_guard_cutover is set, the lower of the two types’ rates applies. Raise the target to trade latency for fewer hosts, lower it for more headroom. Running tasks are read from Container Insights (RunningTaskCount) when it is enabled, otherwise from the GPU Auto Scaling group’s in-service instance count (one task per host). The Auto Scaling group count also includes the spare hosts balanced and aggressive keep warm (roughly 20% and 30% of the fleet), so without Container Insights requests per task read low by that share and the forecast under-provisions. Pair forecast_and_scale with container_insights = "enabled"; terraform plan warns when it is not.

Agent Sandbox (Optional)

Procedures also need the agent sandbox: their code is validated and test-run inside it. Without the agent runtime (enable_runlayer_agentcore_runtime = true), creating or validating a Procedure returns 503. Select a network configuration that allows AgentCore to reach the application before applying.
With the egress proxy enabled, every sandbox request to the public internet is authenticated per run and checked against the agent’s network access policy. Runlayer traffic stays on PrivateLink and is never proxied.
Network access policies are enforced only by this ECS / AgentCore sandbox runtime. The in-cluster Kubernetes agent sandbox (sandboxMode: k8s) does not enforce them yet: an agent’s policy is stored but has no effect there.
By default, Agent Sandbox routes LLM calls through the backend Anthropic proxy. To use OpenAI or an OpenAI-compatible gateway for custom agents, add a provider in Settings → AI Providers (no Terraform change needed). This Terraform path provisions the ECS / AgentCore sandbox. In-cluster kubernetes-sigs Agent Sandbox (sandboxMode: k8s, gVisor or Kata on the nodes) is operator/GKE — see Runlayer Operator — K8s Agent Sandbox. Runlayer Assistant runs on Bedrock in this account by default, so it works without configuring a provider. Admins can move it to one of the workspace’s providers under Runlayer Assistant model in Settings → AI Providers. Model access is handled as described above; this module exposes it as bedrock_auto_model_access, bedrock_model_access_company_name, bedrock_model_access_company_website, bedrock_model_access_industry and bedrock_model_access_intended_users. Backend egress for WorkOS AuthKit, Bedrock, and AgentCore is listed in Self-Hosted Egress Requirements.

LLM gateway

Microsoft Teams

Optional. Off unless all five values are set — the module variable is the client secret, the rest ride additional_backend_env_vars. Provisioning creates a per-agent Entra app registration and Azure Bot Service resource in your Entra tenant and Azure subscription, so a self-managed install needs its own multi-tenant app registration (Graph application permissions Application.ReadWrite.All, AppCatalog.ReadWrite.All, TeamsAppInstallation.ReadWriteForUser.All, admin-consented), an Azure resource group whose IAM grants that app Azure Bot Service Contributor, and the Microsoft.BotService provider registered on the subscription.
MS_TEAMS_APP_URL defaults to APP_URL (https://<domain_name>). Register https://<domain_name>/api/v1/integrations/ms-teams/admin-consent-callback as a Redirect URI (Authentication → Web) on that app registration, or admin consent fails with AADSTS500113. Each tenant that installs an agent’s bot must also enable Upload custom apps in its Teams Admin Center setup policy; without it, publishing to the tenant’s app catalog returns 403 no matter how many times it is retried. domain_name is effectively immutable once agents are provisioned: each agent’s Azure Bot resource stores the messaging endpoint captured at creation time and is never rewritten, so changing the domain orphans existing bots until each is deleted and recreated.

AWS Cloud Discovery

On by default. Admins connect AWS accounts from AI Inventory → Sources; each account creates a read-only RunlayerDiscovery-… role from a CloudFormation stack that trusts only this deployment’s ECS task role, with an external ID per connection. The module adds a private template bucket in your deployment’s Region, a grant on the task role to assume those roles, and the backend settings that point at both. Customers create the stack in the same Region, so service control policies there must allow CloudFormation.
Only the ECS module wires this. On EKS, set it up by hand with the AWS_CLOUD_DISCOVERY_* keys in the operator guide; GKE has no AWS identity, so AWS Cloud Discovery isn’t available there. Customer stacks bake in the task role’s ARN, so keep project and environment unchanged once accounts are connected. With restricted egress, also allow sts.us-east-1.amazonaws.com, account.us-east-1.amazonaws.com, and bedrock.<region>.amazonaws.com plus bedrock-agentcore.<region>.amazonaws.com for every Region your customers’ accounts enable.

iMessage channel

Optional, off by default. Lets people text an agent from Messages; the user side is in iMessage. The stack doesn’t run any Apple hardware: Sendblue rents a phone number (“line”) that sends and receives iMessage (with SMS fallback) through its API. The stack has one line, Runlayer’s, configured in Terraform. One agent answers it at a time, so one workspace on the stack uses iMessage at a time: the first workspace whose admin turns it on holds the number until it turns it off. On a stack several workspaces share, agree on which one before you enable the channel.
enable_imessage_channel sets IMESSAGE_CHANNEL_ENABLED, which shows the Text message card on agent profiles in Go. The apply stores the key pair and webhook secret in one Secrets Manager secret and hands them, with SENDBLUE_FROM_NUMBER, to the backend service and the shared worker only. The plan fails unless all four sendblue_* values are set with the flag.

Before you apply

  • A Sendblue account with a dedicated line. Sign up at sendblue.com (or npx @sendblue/cli setup). The free plan shares one number across customers and only reaches contacts you verify first; for an agent anyone in the organization can text, use a dedicated line (Sendblue’s AI Agent plan is inbound-first: people text the line, the agent replies). Copy the line’s number and the API key pair (dashboard, or sendblue show-keys).
  • Who can reach it. Anyone can text the line. Runlayer ignores texts from people who haven’t linked their number to a Runlayer account, apart from one link reply every 10 minutes.
  • Data path. Texts and the agent’s replies pass through Sendblue (SOC 2 Type II). Review that with your security team the same way as any other messaging integration.

Point Sendblue at the stack

In the line’s Sendblue account (Developer → Webhooks), add a receive webhook with the imessage_channel_sendblue_webhook_url output, https://<domain_name>/api/v1/integrations/imessage/sendblue/receive, and sendblue_webhook_secret as its secret. Sendblue sends the secret in the sb-signing-secret header; the backend rejects any call without the right one. Sendblue webhooks are per account. If the account has other lines, texts to them reach the webhook too and are ignored: only texts to sendblue_from_number start runs. With WAF IP allowlisting on, the webhook must still reach https://<domain_name>: allow Sendblue’s webhook egress IPs (ask Sendblue support) in waf_allowlist_ipv4_cidrs, or texts get 403s and nothing happens. The backend also needs outbound HTTPS to api.sendblue.co.

Give the line a name and photo

Optional. Without a contact profile, Messages shows the line as a bare number. Set one once with Sendblue’s contact sharing API and the line’s key pair. The photo must be a public JPEG or PNG URL:
The backend asks Sendblue to share it with a linked phone number whenever that number’s text starts a run, and Sendblue shares it with each chat at most once a day. Until the profile is set, Sendblue refuses each share with a 409, logged as imessage_sendblue_name_share_failed; texts and replies are unaffected.

Operating it

  • Status. The agent page shows the number and Runlayer’s line, or No line. It does not call Sendblue, so a wrong key shows up as failed replies, not on that badge.
  • Replies. Each reply is queued in the database, then sent right away. One person’s replies go out one at a time and in order. If Sendblue is down, rate-limits, or rejects the keys, the worker’s imessage_outbox_sweep (every minute) retries it, up to 5 attempts over about 9 minutes, and the replies after it wait. A message Sendblue declines (for example, the person opted out) is not retried, and the rest of that answer is dropped. A reply still unsent after a day (the line’s config was missing) is dropped too. Logs: imessage_sendblue_send_failed.
  • Read receipts and typing. A text the agent takes is marked read, and the agent shows typing while its run works. Both are best effort: failures are logged (imessage_sendblue_read_receipt_failed, imessage_sendblue_typing_failed) and change nothing else.
  • What “sent” means. A reply counts as sent once Sendblue accepts it. If Sendblue’s response is lost, the reply still counts as sent (Sendblue may have queued it) and is never resent, because Sendblue takes no idempotency key. Delivery to the phone is Sendblue’s from there. Failures after that (for example, an unreachable phone) show in Sendblue’s dashboard, matched by the message handle stored on the reply when one came back. If a run’s answer never reached the queue (the server failed right after the run finished), the sweep queues it within a couple of minutes.
  • Rotating keys. Rotate in Sendblue, update the variables and apply; a new secret version rolls the backend and the worker onto it. Texts Sendblue sends with an old webhook secret get 401s, and Sendblue retries each one up to 3 times, so update both sides together.
  • Offboarding an organization: turn iMessage off on the agent that answers, with the API call in Turn it on for your workspace. To retire the channel, set enable_imessage_channel = false, apply, and delete the webhook in Sendblue.
Kubernetes deployments set the same backend settings directly: IMESSAGE_CHANNEL_ENABLED, SENDBLUE_FROM_NUMBER and the secrets SENDBLUE_API_KEY_ID, SENDBLUE_API_SECRET_KEY and SENDBLUE_WEBHOOK_SECRET on both the backend and the worker.

Audit SIEM, sessions, and hook events

Audit logs and session events use separate S3 exports. Enable either or both for ingestion into your SIEM:
  • Audit logs include tool activity, policy decisions, security events, and administrative changes. Set enable_audit_log_siem_export = true. With audit_log_write_target = "aurora", the worker exports through an outbox; audit-log Kinesis is optional. S3 audit archiving is a separate retention feature and does not enable SIEM export. See Audit-log export.
  • Session events include captured prompts, reasoning, tool calls, and scanner verdicts. The hook-events pipeline is on by default (leave enable_hook_events_pipeline unset); uncomment session_siem_export_kinesis_consumer_config below to enable the dedicated export consumer. Enabling the hook-events pipeline alone does not enable session export. See Session export.
For self-hosted ECS, configure these settings in your Terraform module. To write to existing buckets, uncomment the corresponding *_external_s3_bucket settings and replace the example names. Both exports can use the same bucket with separate *_s3_prefix values. Without an external destination, Terraform creates a bucket for each enabled export in your deployment’s AWS account. Direct writes use your deployment’s ECS task role. To write through another IAM role, set *_assume_role_arn together with the required *_external_id and *_external_s3_bucket for that export. Configure destination bucket and KMS policies, plus your SIEM’s S3 reader permissions, notifications, and schemas, separately. Optionally set hook_events_alert_email to receive pipeline-health emails through Amazon SNS. Alerts fire when the hook-events consumer Lambda’s error rate exceeds 20% for two consecutive five-minute periods, or its processing lag exceeds five minutes in three consecutive five-minute periods. Recovery notifications are also sent. Confirm the SNS subscription email to receive notifications; leaving the setting unset disables these email alerts. These alarms monitor the hook-events consumer Lambda, which is provisioned only when ML data export (data_export_bucket_arn), the Sparkle worker, or enable_hook_events_parquet_archive is set — with only session SIEM export on, no Lambda and none of these email alarms are created. The session SIEM exporter is a separate ECS consumer. Session export starts with new events and does not backfill earlier sessions. It contains unredacted content; session privacy settings do not apply to the export, so restrict access to the destination bucket.
If your deployment already uses audit_log_write_target = "kinesis", keep its existing audit-ingestion configuration. To export directly from that stream, also enable the independent SIEM export consumer below, alongside enable_audit_log_siem_export = true. This exports normal streamed events without routing them through the database outbox. Set both audit prefix variables to the same value so stream exports and outbox fallback exports use the same destination prefix:
The consumer deploys start-before-stop: a replacement task starts first, and ECS stops the old task only once the replacement’s health check reports that it holds its share of the shards. The remaining tasks claim the shards an old task releases within a few seconds, so an upgrade pauses exports for a few seconds per shard. A backend image without this readiness signal passes the health check only after a 4-minute fallback, so each deploy of such an image takes that much longer. Delivery is at-least-once, so a deploy can export a few records a second time; deduplicate on audit_log_id downstream.

Binary packages and downloads

AI Watch snapshot signing

Only for AI Watch Local Proxy. Generate the deployment key and get a delegation from Runlayer as described in AI Watch snapshot delegation, then pass both:
Tasks read both only at start. An apply that changes either rolls backend-service, so no manual redeploy is needed.
  • Renew before the delegation’s not_after (at most 90 days): replace the file with the one Runlayer sends and apply.
  • Rotate the key by changing both inputs in the same apply: a delegation names one key, so a new key needs new delegations.

Customer OpenTelemetry

Export metrics from the backend, and/or run an OTEL collector sidecar that ships to your collector:

Secrets, Sentry, and catalog

Deployment identity

For Sentry Relay credentials, follow Sentry Relay Credentials. For software upgrades and rollback, follow Updates.