Overview
Deploy Runlayer on AWS ECS (Elastic Container Service) with Fargate using Terraform. The stack runs entirely in your AWS account: VPC, ALB, Aurora, Redis, Secrets Manager, and ECS services. How Runlayer is delivered:terraform init or image pulls return AccessDenied.
Prerequisites
Install Tools
hashicorp/random below 3.9, run terraform init -upgrade before planning or applying the upgraded module.AWS Requirements
- AWS account with permission to create VPC, ECS, RDS, ElastiCache, ALB, IAM, Secrets Manager, S3, CloudWatch, and Lambda resources
- Domain name you control (for DNS records and SSL)
- Adequate service quotas (VPC, RDS, ECS; GPU quota if enabling ToolGuard)
- AWS provider
>= 6.64.0, < 7.0.0(module pin; see ECS Terraform inputs) - Outbound internet from private subnets (NAT or VPC endpoints) so ECS can pull images from ECR and reach WorkOS, AuthKit, catalog, OAuth Broker, and optional ToolGuard/Bedrock hosts — full hostname list: Self-Hosted Egress Requirements
- Anthropic first-time-use form submitted for the AWS account (see below) — this module sets
BEDROCK_ENABLED = true, and Runlayer Assistant always routes through Bedrock
BEDROCK_ENABLED=true (Runlayer Assistant platform inference and optional Bedrock agent models). With BEDROCK_ENABLED=false, Runlayer Assistant binds an org AI provider instead; skip this step unless you also enable Bedrock inference elsewhere (for example BEDROCK_ANTHROPIC_MODELS_ENABLED for custom agents).The backend does both for you at startup and on login, in the account the Bedrock calls come from, using the model-access IAM permissions your deployment grants. Four settings describe the deployment on the form — BEDROCK_MODEL_ACCESS_COMPANY_NAME (defaults to Runlayer, the operator of the software making the calls), BEDROCK_MODEL_ACCESS_COMPANY_WEBSITE, BEDROCK_MODEL_ACCESS_INDUSTRY and BEDROCK_MODEL_ACCESS_INTENDED_USERS. Override them if you run this install yourself. They are never derived from your data: the form is submit-once, so a guessed answer would be permanent.Set BEDROCK_AUTO_MODEL_ACCESS=false to turn this off and do it by hand instead: Bedrock console → Model catalog → any Anthropic model → Submit use case details. Do the same if your deployment role predates the model-access permissions — the backend logs exactly which actions it is missing. AWS accepts the form once per account, or once at your AWS Organization’s management account, which member accounts inherit.Deploying outside the US? Also set RUNLAYER_ASSISTANT_BEDROCK_MODEL to your geography’s inference profile, for example au.anthropic.claude-opus-4-6-v1 in ap-southeast-2. The risk-categorization explainer follows BEDROCK_REGION on its own and uses Claude Haiku 4.5 outside the US, so the Anthropic form above covers it; only set RISK_CATEGORIZATION_MODEL if you want a different model, and pick one served in your Region. If Bedrock rejects the primary model it falls back to Amazon Nova Lite, which needs no access request.Access takes up to 15 minutes to propagate. Until it does — or if this is skipped entirely — Bedrock-backed agents fail with Model use case details have not been submitted for this account; see troubleshooting for the CLI checks and Region caveats.Values from Runlayer
auth_client_id— WorkOS/Auth client IDauth_api_key— treat as a secret; pass viaTF_VAR_auth_api_key, never commit- Module version to pin (for example
v19.12.1) - Confirmation that your account is allowlisted for S3 module downloads and ECR image pulls
Quick Start
1. Create a working directory with a root module that references the published archive:aws and aws.route53 into the module, even when DNS lives in the same account.
Private connection provisioning
ECS private connections reconcile asynchronously. Existing Deploy PrivateLink connections upgrade to Terraform revision2, adding connection-owned access
security groups while retaining the existing endpoint and legacy security group.
Wait for reconciliation before creating or redeploying an MCP deployment that uses
the connection. Existing running deployments retain their legacy access rules;
redeployment switches them to access-SG attachment. Each deployment supports up
to four private connections.
RUNLAYER_PRIVATE_CONNECTION_TERRAFORM_TIMEOUT_SECONDS defaults to 1800 and
must be a positive finite number. Set it through additional_backend_env_vars
to override the shared deadline for Terraform init, apply/destroy, and output
commands. It does not change ordinary MCP deployment timeouts. Failed
reconciliation preserves the connection’s prior usable state and is retried
automatically up to 3 times, at least 15 minutes apart; if it still fails,
contact Runlayer support to inspect and recover the upgrade before retrying the
deployment.
Private CA certificates
To trust MCP servers or HTTPS proxies whose certificates use your private CA, pass the CA certificate bundle to the ECS module:Distribution configuration
WorkOS remains the default feature-flag provider. You can stage Distribution reader access without changing evaluation by passing the reader credential and API URL:openfeature_provider = "flagd"; the module rejects flagd without both reader
inputs.
Hook-body compression (gzip-hooks, zstd-hooks) is on by default for
Distribution-backed deployments. WorkOS-provider deployments opt in until they
move to Distribution; FF_DISABLE wins if both name a flag:
distribution_api_key_previous to the
outgoing key, applying, waiting for ECS services to reach steady state, then
clearing it and applying again. Set distribution_status = "disabled" to revoke
access without destroying the stack; move openfeature_provider off flagd first.
Delivery details
Terraform module (S3)
Module archives are published to:source form:
source, then:
Container images (ECR)
By default the module pulls Runlayer images from CustomerDistribution ECR:app_version. Override it, or supply your own image map:
- URIs must be public ECR (
public.ecr.aws/...) or private ECR (account.dkr.ecr.region.amazonaws.com/...) - Every URI must include an explicit tag or
@sha256:digest — no implicitlatest app_versionandecr_repositoriesare mutually exclusive
Configuration options
Every customer-configurable module input (defaults, types, validation) is listed in the ECS Terraform inputs reference. The accordions below show common HCL examples. Use placeholders only (acme, 123456789012, client_REPLACE_ME, var.*) — never commit real API keys or customer-specific secrets.
Naming constraints
These values are used in AWS resource names (ElastiCache replication group IDs, secrets, IAM, etc.) and are validated by the module:<project>-<environment>-…. Example: project = "acme", environment = "staging" → Redis replication group acme-staging-redis.
Required inputs
Only these five have no default (must be set):project and environment explicitly even though they have defaults — they drive almost every resource name.
Smart production-ready defaults include:
- Database: Aurora PostgreSQL Serverless v2, private subnets, automated backups
- Cache: ElastiCache Redis in private subnets
- Security: Public ALB by default; private DB/cache; Secrets Manager for app secrets
- SSL: Automatic wildcard certificate lookup; set
enable_acm_dns_validation = trueto create + validate a cert via Route53 - Scaling: Backend default 4096 CPU / 8192 MB; frontend 256 CPU / 512 MB; auto-scaling enabled
- Network: Multi-AZ VPC with public/private subnets (IPv6 on by default; set
enable_ipv6 = falsefor IPv4-only) - Monitoring: CloudWatch logs for all services
Optional configuration
Theadditional_tags input accepts a string map and defaults to {}. It applies
deployment metadata to directly managed workload resources. Resource identity tags
take precedence. No partner attribution is enabled by default. Verify product code,
resource eligibility and existing tag ownership before opt-in. See the module README
for the coverage available in your published module version.
Database Configuration
Database Configuration
db.serverless). There is no provisioned instance-class input (db.r6g.large, etc.). Size the cluster with Aurora Capacity Units (ACUs):external_database instead.Bring Your Own Database / Cache
Bring Your Own Database / Cache
POSTGRES_* env vars and Secrets Manager entries for PLATFORM_DB_PASSWORD and REDIS_URL. Ensure private subnets can reach your endpoints. For Kubernetes, see the Runlayer Operator database and Redis contract.Security Configuration
Security Configuration
waf_webhook_allowlist_ipv4_cidrs and waf_webhook_allowlist_ipv6_cidrs
default to []. They admit the specified request-origin IPs only for POST
agent webhook invocation: /api/v1/agents/{uuid}/webhooks/{token} and the
legacy /api/v1/assistants/{uuid}/webhooks/{token}, with an optional trailing
slash. Paths are URL-decoded and normalized before matching. Webhook management
routes and other methods receive no additional access; existing general
allowlists and public-path exemptions still apply.Nonempty lists require waf_enable_ip_allowlisting = true; nonempty IPv6 lists
also require enable_ipv6 = true. Invalid configurations fail planning; empty
lists remain valid when either feature is disabled. General allowlist CIDRs
remain required when allowlisting is enabled. Entries must be valid IPv4 CIDRs
with prefix lengths 1–32 or IPv6 CIDRs with prefix lengths 1–128.
The caller maintains these ranges; the module does not fetch vendor IP lists.
Managed WAF rules and webhook authentication still apply. These lists cover
invocation routes across agents, so review the admitted ranges, webhook
credentials/signing, and execution identity with your security team before
activation. ALB security groups must also permit the integration’s source IPs.Dual ALB Setup (Split-Horizon DNS)
Dual ALB Setup (Split-Horizon DNS)
- Public DNS → public ALB
- Private DNS → internal ALB
- ECS services register with both ALBs
- WAF applies only to the public ALB
VPC Peering for Cross-VPC Connectivity
VPC Peering for Cross-VPC Connectivity
existing_vpc_id).Service Scaling
Service Scaling
services_configurations map (partial maps are not merged):min_capacity / max_capacity control the running fleet: Application Auto Scaling owns each service’s live desired_count, and Terraform ignores it after the service is created, so desired_count only seeds a brand-new service. To change the floor of an existing service, raise min_capacity (or backend_min_capacity).Reserve backend.priority + 1 for the /mcp alias listener rule (module validation enforces this).Network Configuration
Network Configuration
CreateLoadBalancer to fail with A load balancer cannot be attached to multiple subnets in the same Availability Zone. Subnet size can be small (for example /26) as long as the ALB has free IPs (~8+). Multiple ALBs may share the same public subnets.For topology options, see Networking.Monitoring & Alerting
Monitoring & Alerting
Redis / ElastiCache
Redis / ElastiCache
external_redis is set):PrivateLink and tunnels
PrivateLink and tunnels
privatelink_service_name for consumer VPC endpoints.Upgrade the module with the application. Normally omit tunnel_image_tag so
the module selects the matching router, endpoint, and connector bundle.Enable tunnels only with a compatible Runlayer application release. Upgrade the
application before the Terraform module. When rolling back, restore the previous
module configuration before rolling back the application.Existing Deploy PrivateLink remains supported.When waf_enable_ip_allowlisting = true, set runlayer_agentcore_network_mode = "VPC" (with AgentCore enabled) so AgentCore reaches APP_URL over PrivateLink instead of the public ALB. See Self-Hosted Egress Requirements → AgentCore with WAF.Dual ALB resolver / cross-account DNS
Dual ALB resolver / cross-account DNS
LLM gateway
LLM gateway
Customer OpenTelemetry
Customer OpenTelemetry
ECS services, workers, and migrations
ECS services, workers, and migrations
Secrets, Sentry, and catalog
Secrets, Sentry, and catalog
Microsoft Teams
Microsoft Teams
additional_backend_env_vars.Provisioning creates a per-agent Entra app registration and Azure Bot Service
resource in your Entra tenant and Azure subscription, so a self-managed
install needs its own multi-tenant app registration (Graph application
permissions Application.ReadWrite.All, AppCatalog.ReadWrite.All,
TeamsAppInstallation.ReadWriteForUser.All, admin-consented), an Azure
resource group whose IAM grants that app Azure Bot Service Contributor, and
the Microsoft.BotService provider registered on the subscription.MS_TEAMS_APP_URL defaults to APP_URL (https://<domain_name>). Register
https://<domain_name>/api/v1/integrations/ms-teams/admin-consent-callback as a
Redirect URI (Authentication → Web) on that app registration, or admin consent
fails with AADSTS500113.Each tenant that installs an agent’s bot must also enable Upload custom apps
in its Teams Admin Center setup policy; without it, publishing to the tenant’s
app catalog returns 403 no matter how many times it is retried.domain_name is effectively immutable once agents are provisioned: each agent’s
Azure Bot resource stores the messaging endpoint captured at creation time and
is never rewritten, so changing the domain orphans existing bots until each is
deleted and recreated.Bedrock and Anthropic
Bedrock and Anthropic
global.anthropic.claude-opus-4-8). It ignores Settings → AI Providers.
Having “some Anthropic model” enabled in the console is not enough — that
specific model (or its base) must be available.Module v28.1.x+ attaches the model-access IAM actions to the ECS task role
(GetFoundationModelAvailability, PutUseCaseForModelAccess, Marketplace
subscribe, etc.). You do not add those by hand unless an SCP / boundary strips
them.does not have access to the Anthropic model yet,
see troubleshooting.Optional feature flags
Optional feature flags
sts:AssumeRole only on arn:aws:iam::*:role/runlayer-bedrock-* and
arn:aws:iam::*:role/RunlayerBedrock*. The saved role must trust the caller’s AWS
account/identity with the saved external ID and grant the required model access.
When both BEDROCK_ENABLED and BEDROCK_ANTHROPIC_MODELS_ENABLED are enabled,
the consumer also receives nonstreaming bedrock:InvokeModel access to Claude
foundation models and the deployment account’s Claude inference profiles for
the legacy platform fallback.
Upgrade and apply the Terraform module alongside the backend/consumer image;
updating the image alone does not install this permission. After IAM propagation,
verify an authorized monitor canary and its dead-letter alarm.ToolGuard scale-out speed. A new ToolGuard GPU host installs NVIDIA GRID
drivers at boot and takes 10–15 minutes before it serves traffic, so a
scale-out decision is only as fast as the spare hosts already running.
That boot path also needs outbound HTTPS to Ubuntu, NVIDIA GRID S3, Docker,
and related hosts — see Self-Hosted Egress Requirements → ToolGuard.
runlayer_tool_guard_scaling_profile trades idle GPU cost for
time-to-capacity:balanced and aggressive keep one to two extra g6f.large instances
running at all times (billed even when idle) so the next task starts
immediately. Scale-in waits 15 minutes on every profile. The dashboard’s
Tasks: Desired vs Running panel shows how long scale-outs wait on new
hosts. The runlayer-tool-guard-scale-out-lag alarm fires only when desired
tasks exceed running tasks for 20 consecutive minutes — longer than a cold
GPU host boot — so a routine scale-out does not trip it; when it does fire,
new hosts are not arriving (check the Auto Scaling group and capacity
provider).GPU signal on aggressive. CPU is a weak load signal on a GPU inference
host, so aggressive adds a second target-tracking policy on fleet-average
GPU utilization (target 60%). The policy tracks the existing
RunlayerToolGuard/GPUUtilization metric (Environment dimension,
Average statistic): every host whose ToolGuard answers /health 200
publishes one sample per minute, so the per-minute average is the mean across
serving GPUs. Hosts that are not serving (capacity-provider spares, hosts
still installing drivers or loading the model, hosts in the recycle drain)
publish nothing and are left out, so the spare pool cannot hold the average
under the target while serving GPUs are busy. No per-host series is added, so
CloudWatch metric cardinality is unchanged.
Scaling takes the higher of the CPU and GPU decisions, so the GPU policy
never forces the fleet below what the CPU or predictive policies request.
Hosts publish GPU metrics from their boot script every 60 seconds (was every
5 minutes), so upgrading to this version changes the ToolGuard launch
template on every profile and the ASG’s rolling instance refresh replaces the
hosts: tasks restart on new hosts as old hosts drain (each restart reloads the
model, so expect a capacity dip per batch). Until the refresh replaces them,
hosts from the previous launch template still publish every 5 minutes and so
land in only one of every five 1-minute averages.Predictive scaling. Because a reactive scale-out cannot beat the 10–15
minute host boot, the module also creates an ECS predictive scaling policy
on the ToolGuard service. It forecasts the next 48 hours of demand hourly
from up to 14 days of ToolGuard predict request history and raises the
task count runlayer_tool_guard_predictive_scheduling_buffer_seconds
(default 1200) before each forecast hour, so GPU hosts are booting before
the load arrives. conservative runs it in forecast_only mode (forecasts
are recorded but never acted on); balanced and aggressive run
forecast_and_scale. Override the profile default with
runlayer_tool_guard_predictive_scaling_mode (disabled, forecast_only,
forecast_and_scale). The first forecast appears after 24 hours of metrics;
review it in the ECS console (service → Service auto scaling) or with
aws application-autoscaling get-predictive-scaling-forecast, then flip
forecast_only → forecast_and_scale once the curve matches your traffic.
Predictive scaling only scales out and never raises the task count above
runlayer_tool_guard_max_count; the CPU policy still handles scale-in and
unforecast bursts. In regions where ECS predictive scaling is unavailable
(for example il-central-1) the policy is skipped automatically; the
runlayer_tool_guard_predictive_scaling_mode_effective output reports the
mode actually applied.Why the load is floored. The forecast learns daily and weekly patterns
from the load metric, so a few days with scanning switched off would teach
it that those weekdays are idle and under-provision them for the next two
weeks. Before forecasting, weekday hours (Mon–Fri UTC) below
runlayer_tool_guard_predictive_load_floor_fraction (default 0.5) of the
window’s mean hourly request count are raised to that floor, so off days
look like quiet business days instead of zero; weekends forecast from actual
traffic (runlayer_tool_guard_predictive_load_floor_weekdays_only = false
floors every day, 0 disables the floor). Capacity is sized so each task
receives runlayer_tool_guard_predictive_target_requests_per_task_per_minute
(default 50 per minute, i.e. 3000 per hour). The default keeps per-task
load in the range where ToolGuard
predict p95 stays near its floor; latency rises steeply once a task serves more
than roughly twice that rate. Raise the target to trade latency for fewer hosts,
lower it for more headroom.Running tasks are read from Container Insights (RunningTaskCount) when it
is enabled, otherwise from the GPU Auto Scaling group’s in-service instance
count (one task per host). The Auto Scaling group count also includes the
spare hosts balanced and aggressive keep warm (roughly 20% and 30% of
the fleet), so without Container Insights requests per task read low by that
share and the forecast under-provisions. Pair forecast_and_scale with
container_insights = "enabled"; terraform plan warns when it is not.Audit SIEM, sessions, and hook events
Audit SIEM, sessions, and hook events
Binary packages and downloads
Binary packages and downloads
Deployment identity
Deployment identity
Agent Sandbox (Optional)
Agent Sandbox (Optional)
enable_runlayer_agents with AgentCore),
creating or validating a Procedure returns 503.sandboxMode: k8s, gVisor or Kata on the nodes) is operator/GKE — see Runlayer Operator — K8s Agent Sandbox.Runlayer Assistant always runs on Bedrock in this account — it ignores Settings → AI Providers by design, so it works without configuring a provider. Model access is handled as described above; this module exposes it as bedrock_auto_model_access, bedrock_model_access_company_name, bedrock_model_access_company_website, bedrock_model_access_industry and bedrock_model_access_intended_users.Backend egress for WorkOS AuthKit, Bedrock, and AgentCore is listed in Self-Hosted Egress Requirements.Secrets management
All application secrets live in AWS Secrets Manager in your account. The module creates two primary secrets (plus an optional LLM gateway secret):SECRET_KEYandMASTER_SALTare isolated in the restricted secret so only privileged task roles can read them.- The app secret and restricted secret share the same random name suffix to avoid recreate collisions during Secrets Manager recovery windows.
- Prefer
TF_VAR_*/ a secrets backend for sensitive inputs — do not commit API keys to git. - After deploy, rotate non-key values with Secrets Manager (then force a new ECS deployment so tasks pick up the new version). Rotate
SECRET_KEY/MASTER_SALTviasecret_key/master_saltinstead — Terraform reverts out-of-band edits to the restricted secret on the next apply.
Module outputs (attributes)
After apply, useful outputs include:Deployment
Automated database setup
Database initialization is automated during apply: What happens automatically:- Migration task waits for PostgreSQL, then runs
alembic upgrade head - Backend services only roll out after migrations succeed (Terraform waits up to 30 minutes by default)
- Setup activity is logged to CloudWatch under
anysource-backend-migration-logs-<environment> - Re-running
terraform applyafter interruption is safe (idempotent + advisory lock)
Verify
Architecture
Cost estimation
Common use cases
Private enterprise deployment
High availability production
Development environment
Troubleshooting
terraform init cannot download the module
terraform init cannot download the module
s3:GetObject on runlayer-infrastructure-modules and that Runlayer allowlisted your account or IAM principal. Contact support@runlayer.com if access was not provisioned during onboarding.ECS tasks cannot pull images
ECS tasks cannot pull images
- Confirm ECR pull access to
088332244652(CustomerDistribution) - Private subnets need NAT or ECR VPC endpoints
- Check ECS service events for
CannotPullContainerError/AccessDeniedException
Login works but agent runs fail / ToolGuard EMPTY CAPACITY PROVIDER
Login works but agent runs fail / ToolGuard EMPTY CAPACITY PROVIDER
SSL Certificate Issues
SSL Certificate Issues
enable_acm_dns_validation = true):aws.route53 provider assume_role can write validation records.Application Not Accessible
Application Not Accessible
Migration task failures
Migration task failures
anysource-backend-migration-logs-<environment>. Re-running terraform apply is safe — the migration uses an advisory lock and rolls back on failure.High Costs
High Costs
- Use
environment = "development"for testing - Reduce database
min_capacity/max_capacity - Lower service
min_capacity/max_capacityand CPU/memory - Set shorter
backup_retention - Consider
single_nat_gateway = truefor non-prod