Skip to main content
Each release sets a default Runlayer app version; see the Runlayer changelog for what changed in the app.
November 20, 2026
Default Runlayer app version: 6.4.0. See the Runlayer changelog.Improvements
  • Tenant key vault with one switch: a self-hosted deployment now turns the tenant key vault on with tenant_key_vault_enabled = true, once Runlayer confirms the deployment is onboarded. The module finds the vault secrets by the name of its task execution role, so no ARN is exchanged with Runlayer. The task execution role keeps its grants on the provisioned secrets, so ECS can still roll a failed deployment back to the task definition from before the vault. If you pass distribution_api_key today, leave it (and mcp_catalog_api_key) set on the apply that turns the vault on, for the same rollback, and remove them in a later apply.
  • tenant_key_vault_kms_key_arn is deprecated: it is ignored and will be removed in a later release. Remove it from your module call.
  • tenant_key_vault_secret_arn is deprecated: the tenant key vault now keeps each key in a secret of its own, named after your task execution role, so there is no single secret left to name. The input is ignored and will be removed in a later release. Remove it from your module call.
November 19, 2026
Default Runlayer app version: 6.3.0. See the Runlayer changelog.Improvements
  • Session search follows the indexer: activity and security-finding filters on the Sessions page, and the agent session search tools, are now available only when enable_session_search_indexer = true and session_search_index_mode = "write". Other deployments keep text search and session id lookup. Nothing to change; to force a value, set SESSIONS_SEARCH_INDEX_READS_ENABLED in additional_backend_env_vars.
  • Tunnel diagnostics now report how long authenticated CONNECT streams stay open and why they close. The Runlayer backend also reports connection timings and failures by phase, from trust lookup through CONNECT admission.
  • Tunnel image security updates include fixes in Go and OpenTelemetry.
  • Aurora replica lag alarm: with enable_monitoring = true, each Aurora instance gets an alarm on the critical-alerts topic when its replica lag stays above rds_replica_lag_threshold_ms (default 1000) for 5 minutes. Replica reads are stale while it fires. Nothing to change.
Bug fixes
  • Tunnel connector readiness keeps healthy targets available while slow, successful TCP readiness checks wait for their scheduled turn after startup.
November 17, 2026
Default Runlayer app version: 6.2.0. See the Runlayer changelog.No upgrade notes for this release.
November 17, 2026
Default Runlayer app version: 6.1.0. See the Runlayer changelog.Breaking changes
  • Who is affected: only roots that never set enable_hook_events_pipeline. In each case below, the simplest fix is to set enable_hook_events_pipeline = false before you bump the module.
  • Inline Runlayer Deploy policy: with runlayer_deploy_use_managed_policy = false, terraform plan fails. Either turn the pipeline off, or remove that setting.
  • Digest-pinned backend image with the Parquet Lambda: if ecr_repositories.backend uses a digest (@sha256:...) and you set data_export_bucket_arn, enable_sparkle_worker = true or enable_hook_events_parquet_archive = true, terraform plan fails until you also set hook_events_consumer_image_uri. Either set it, or turn the pipeline off.
Action required
  • Sessions are also stored in Postgres and S3 by default: if you never set enable_hook_events_pipeline, the next apply creates the hook-events pipeline (Kinesis stream, S3 buckets, session materializer and compactor ECS services, and their alarms), which stores new sessions in Postgres and S3 as well as Redis. Sessions are still read from Redis. A later release reads them from Postgres and S3, so keep the pipeline on; set enable_hook_events_pipeline = false only if you must skip it.
  • Session retention: Redis keeps sessions for 7 days, but the Postgres rows and the session-payloads S3 bucket have no expiry, so captured prompts, reasoning and tool calls are kept indefinitely. Raw hook events in the Parquet bucket expire after a year. Records the session materializer or the cold-analyzers consumer fail to process are kept for 3 years in their dead-letter buckets. Check this with your compliance team before you apply; setting enable_hook_events_pipeline = false before the bump keeps the 7-day behavior.
  • Other services follow the pipeline: the cold-analyzers consumer (an always-on ECS service) also starts; set enable_cold_analyzers_consumer = false to skip it. If you set data_export_bucket_arn, data-export sync and the hook-events Parquet consumer Lambda also start; the Lambda also starts with enable_sparkle_worker = true or enable_hook_events_parquet_archive = true.
  • Cost: at default sizes the pipeline and the cold-analyzers consumer add about $90 a month in us-east-1, plus usage that grows with session traffic.
Improvements
  • AWS Cloud Discovery on by default: the apply adds a private template bucket in your deployment’s Region, a bucket policy, and an sts:AssumeRole grant on RunlayerDiscovery-* roles to the ECS task role’s managed policy, so admins can connect AWS accounts from AI Inventory. Set enable_aws_cloud_discovery = false to opt out. Run an app version from this release or later outside us-east-1. If your egress is restricted, allow sts.us-east-1.amazonaws.com, account.us-east-1.amazonaws.com, and bedrock and bedrock-agentcore in each Region your customers’ accounts use.
  • AI Watch policy snapshot signing (opt-in): set ai_watch_snapshot_signing_key and ai_watch_snapshot_delegations with the values Runlayer provides. Off while both are empty.
  • Valkey 9.1 for the managed cache: new deployments should set redis_engine = "valkey" (added in v49.0.0). On an existing deployment, setting it upgrades the cache in place and keeps endpoints; AWS keeps the data only on a best-effort basis. AWS warns that a single-node group, the default, may be unavailable for the whole upgrade. Setting redis again makes Terraform recreate the groups empty.
  • Per-key tenant secrets no longer need the JSON secret: with provisioned_tenant_secret_layout = "per_key" the module stops looking up restricted/<project>-<env>-provisioned, so the plan no longer needs that secret to exist. The task execution role keeps a grant on it by name, so a rollback to an earlier task definition still starts while the secret exists. Nothing changes on the default "json" layout or with tenant_key_vault_secret_arn.
  • Alarm for a load balancer target group with no healthy targets: with enable_monitoring on, the apply adds a <project>-<environment>-alb-<service>-no-healthy-hosts alarm for each public ALB target group. It fires when a service has had no healthy target for 10 minutes and alerts through the critical-alerts topic when critical_alerts_enabled is on. New alarms wait up to an hour before they first evaluate.
November 16, 2026
Default Runlayer app version: 6.0.0. See the Runlayer changelog.Breaking changes
  • The WorkOS, Sentry and Resend keys come only from the provisioned tenant secret: before bumping, make sure the Secrets Manager secret restricted/<project>-<env_short>-provisioned exists in the deployment account (env_short is prod for environment = "production", otherwise the environment name) with a JSON version carrying all six keys, an empty string for any you do not use: AUTH_API_KEY (your current auth_api_key value), RESEND_API_KEY, SENTRY_DSN, SENTRY_RELAY_PUBLIC_KEY, SENTRY_RELAY_SECRET_KEY and SENTRY_RELAY_ID (the Sentry values Runlayer gave you). Runlayer creates and fills it for managed deployments; a plan fails until it exists.
  • auth_api_key is removed: delete it from the module call and from the CI secret store that fed it, then rotate the WorkOS key with Runlayer, since copies remain in earlier state versions.
  • tenant_secret_source accepts only "provisioned": "input" fails the plan. The variable is now a no-op default; remove it from the module call before the next major, which drops it.
  • Relay without credentials: Terraform no longer checks whether Relay credentials exist. Set sentry_relay_enabled = false on a deployment whose provisioned secret has no SENTRY_RELAY_* values, or the Relay task fails to start. sentry_dsn only gates frontend Sentry; set it to the DSN the secret carries when you want frontend Sentry.
Action required
  • Stateless roots apply after their primary: a stateless root (region_role = "stateless") now reads the keys from the primary’s secret through central_state.provisioned_tenant_secret_arn and fails the plan while that is null. Apply the primary on this version first, then refresh central_state and apply each stateless root.
  • Upgrade from input mode in a quiet window: the first apply rewrites the app secret without the WorkOS and Sentry keys while earlier task revisions still reference them, so an old task that restarts during the rollout fails to start. Apply when you can wait for the new deployment to become healthy. To back out, return to module version 48 with tenant_secret_source = "provisioned" set and apply again: it reads the same provisioned secret and needs no auth_api_key. Earlier versions have no provisioned mode and need auth_api_key again.
  • Browser Sentry DSN is no longer read from WorkOS Vault: if browser Sentry was turned on through the optional frontend_sentry_dsn field of the Vault item, set the module’s frontend_sentry_dsn input to that public DSN before bumping, together with sentry_dsn. Without it, browser error and trace reporting stops.
  • Hook-events lag email removed: with hook_events_alert_email set, <project>-<environment>-hook-events-consumer-iterator-age (over 5 minutes behind) is deleted. Lag now alerts only through <project>-<environment>-hook-events-parquet-lambda-iterator-age, which needs enable_monitoring and critical_alerts_enabled and fires over 15 minutes behind. To keep a lag alert, set both to true.
Improvements
  • Tunnel collector autoscaling: collectors now scale on native ECS average CPU, targeting 60% by default, even with central observability off. The default ceiling is the larger of four and tunnel_otel_collector_desired_count, which now sets the runtime floor; set tunnel_otel_collector_max_capacity equal to that floor for fixed capacity, or tune the target with tunnel_otel_collector_cpu_target.
  • Model shed-rate alarms stop flipping on bursty load: every ToolGuard, topic-cpu and guard-cpu <project>-<service>-<model>-shed-rate-<env> alarm now pages when 503 load sheds pass runlayer_model_error_rate_threshold_percent in 3 of the last 4 ten-minute windows, and holds its state while the rate hovers near that line. Windows need at least twice runlayer_model_error_rate_min_requests requests to count.
  • Legacy topic-cpu alarms removed: while keep_legacy_runlayer_topic_cpu is true (the default) or enable_runlayer_guard_cpu_split is false, the apply deletes the runlayer-topic-cpu-no-tasks, -high-memory, -unhealthy and -fast-unhealthy alarms. With the split on, the guard-cpu stack alarms cover the same failures. With the split off, the legacy stack keeps only its per-model alarms (error rate, shed rate, model load failures).
Bug fixes
  • ToolGuard alarms no longer fire for other deployments: with several deployments in one AWS account and region, each deployment’s ToolGuard rotation-aborted, cuda-oom, gpu-errors and worker-recycles alarms fired on any of them. Each deployment now reads its own counts.
November 12, 2026
Default Runlayer app version: 5.0.0. See the Runlayer changelog.Improvements
  • Missing Runlayer Plugin alarm: with enable_monitoring on, the apply adds <project>-<env>-runlayer-plugin-row-missing, which fires when the Runlayer Plugin is missing from the database. It notifies critical_alerts_extra_sns_topic_arns only and never pages; its description says how to restore the plugin. No new variables.
  • ToolGuard predict-p95 alarm ignores short bursts: it now fires when predict p95 is above the threshold in 12 of the last 15 minutes, instead of in three 5-minute windows in a row, so brief bursts of concurrent scans no longer fire it. A value you set in runlayer_tool_guard_predict_p95_alarm_threshold_ms still applies; you don’t need to change it.
  • ToolGuard replaces a stuck worker: with ToolGuard enabled and more than one task running, a task that logs ROTATION_ABORTED and has been running for at least 30 minutes is stopped so ECS starts a fresh one. A replacement counts only once it is healthy in the load balancer. An abort that has to wait is remembered and retried, one task at a time. A younger task, or the last healthy task, is left alone. The rotation-aborted alarm still pages. Kubernetes is not covered.
  • Quieter token metering alarms: with token_metering_task_config.enabled, token-metering-lag fires only when every run in an hour is over 2 hours and now goes to critical_alerts_extra_sns_topic_arns instead of the critical alerts topic; run-duration fires on 3 slow runs in 8 ticks, tail-read-mismatch on mismatches in 2 of 6 hours, and quarantined-sessions on more than 2 in 6 hours.
  • ToolGuard replaces a task that stopped answering: with ToolGuard enabled, a task at least 30 minutes old whose container check fails and that has logged nothing for 10 minutes has its host drained and marked Unhealthy in Auto Scaling, then is stopped, so the replacement starts on a new machine. At most one host per 30 minutes, none while an earlier one is unfinished, and none while every task is quiet. Hosts in Standby or Detaching are left alone. Kubernetes is not covered.
  • Two new ToolGuard alarms: with enable_runlayer_tool_guard_alarms on, the apply adds <project>-runlayer-tool-guard-silent-task-replaced-<env> (fires once per host replaced this way) and <project>-runlayer-tool-guard-restart-errors-<env> (the replacement function failed five minutes in a row, or a host replacement has been stuck for an hour). Both follow the existing actionable ToolGuard alarm routing. No new variables.
Bug fixes
  • Worker autoscaling no longer stops tasks mid-job: the worker now scales out on CPU/memory but never scales in, so it holds its peak size up to worker_config.max_capacity. To pin or shrink it, set worker_config.desired_count, min_capacity and max_capacity to the same value.
  • Removing an EC2-backed guard or ToolGuard stack no longer hangs: turning off a stack that runs on its own EC2 hosts (topic CPU, guard CPU, ToolGuard, rev) could fail the apply after 20 minutes with its ECS service stuck in DRAINING. The hosts now keep their IAM permissions until they are gone, starting with the first apply after this upgrade; a stack turned off in that same apply is removed the old way.
  • Dropping the legacy topic CPU stack no longer hangs: with keep_legacy_runlayer_topic_cpu = false, the apply no longer times out waiting for the legacy ECS service to drain. If you already dropped that stack, your next plan creates its unused instance role, instance profile and two policy attachments again. That’s expected and needs no action.
November 10, 2026
Default Runlayer app version: 4.2.0. See the Runlayer changelog.Improvements
  • Portable tunnel metrics: Tunnel applications publish OpenTelemetry utilization. The shared collector formats the existing CloudWatch metrics independently of optional central forwarding, using its existing image and HTTPS access. Collector defaults remain one 0.25-vCPU / 512-MiB task.
  • Tunnel endpoint scaling: Compatible endpoints default to 256 concurrent streams. Ranged bounds enable automatic scaling at 80% average stream usage with 300-second cooldowns. Busy endpoints retain task protection until their streams finish naturally; idle endpoints can scale down. Upgrades keep their 90-second drain after healthy replacements arrive. Equal bounds retain fixed capacity. See the rollout and canary guidance in ECS configuration.
  • Autoscaled audit-log consumers also scale on CPU: when audit_log_kinesis_consumer_config.max_capacity is above min_capacity and cpu is 1024 or below, the consumer now tracks its busiest task’s CPU at 75%, which also owns scale-in; lag still adds tasks, and <project>-<environment>-audit-log-kinesis-consumer-lag-scale-in is removed. Keep max_capacity at or below the stream’s open shard count. A fixed fleet (the default), or one with cpu above 1024, is unchanged.
  • Audit-log consumer CPU alarm follows autoscaling: when audit_log_kinesis_consumer_config.max_capacity is above min_capacity and cpu is 1024 or below, the consumer’s paging 80% CPU alarm is replaced by <project>-<environment>-audit-log-kinesis-consumer-cpu-saturated: the busiest task above 90% for 15 minutes, sent to the non-paging Slack relay only (<project>-<environment>-audit-log-kinesis-iterator-age-high still pages). Otherwise the consumer keeps the 80% alarm.
  • Two new token metering alarms: with token_metering_task_config.enabled, enable_monitoring and critical_alerts_enabled on, the apply adds <project>-<env>-token-metering-tail-read-mismatch and <project>-<env>-token-metering-tail-fallback-ratio. Both notify critical_alerts_extra_sns_topic_arns only and never page. No new variables.
  • GPU driver compiler: ECS ToolGuard and Rev hosts now explicitly build the NVIDIA GRID driver with GCC 12. Applying this update rolls enabled ToolGuard and Rev GPU hosts through their existing instance refresh; benchmark hosts use the new bootstrap on their next launch.
Bug fixes
  • Token metering no longer runs out of memory on large sessions: with token_metering_task_config.enabled = true, the meter and nightly coverage tasks now default to 5 GiB (token_metering_task_config.memory, was 4 GiB), which the default 1 GiB max_archive_read_bytes needs. If you set memory yourself, keep it, minus otel_collector_config.memory (default 512 MiB) when the OTel collector sidecar is enabled, at least 2.6x max_archive_read_bytes + 1.9 GiB.
  • GPU hosts join ECS on newer Ubuntu kernels: ToolGuard and Rev hosts now install NVIDIA GRID driver 580.178.04, which builds on the latest Ubuntu 22.04 AWS kernel. Applying rolls enabled ToolGuard and Rev GPU hosts through their existing instance refresh.
November 9, 2026
Default Runlayer app version: 4.0.0. See the Runlayer changelog.Action required
  • ToolGuard and Rev host refresh: with ToolGuard enabled, the apply changes the ToolGuard launch template, so the ASG replaces every ToolGuard GPU host in a rolling refresh, and each batch reloads the model. With Rev enabled (enable_runlayer_rev), Rev hosts boot from the same user-data script, so the apply also replaces every Rev GPU host the same way. Apply in a quiet window.
Improvements
  • Token metering coverage check runs as its own scheduled task: with token_metering_task_config.enabled, the apply adds a nightly Fargate task (01:45 UTC, same size as the metering task) and two non-paging alarms: token-metering-coverage-failed and token-metering-coverage-heartbeat (no run reported in 26 hours). The heartbeat shows ALARM from about an hour after the upgrade until the first nightly run reports; that is expected. The check no longer runs on the worker service.
  • Audit SIEM export pauses for seconds, not minutes, during upgrades: with audit_log_siem_export_kinesis_consumer_config.enabled, the consumer starts its replacement task before stopping the old one, and the new task claims the shards within one 30-second discovery interval. Delivery stays at-least-once: a deploy can export a few records twice.
  • Pinned old backend images: if you pin app_version to a backend older than 1.28.775 (before shard leasing), every deploy exports the records written during its overlap twice for as long as the pin stays. Nothing is lost; unpin, or move to a newer backend, to stop the duplicates.
  • Audit-log stream shard count can be pinned: set audit_log_kinesis_shard_count = N to make the audit-log stream PROVISIONED with N shards, giving lease-sharing consumers (audit-log, audit SIEM export) more shards than on-demand mode creates at your load; a plan then fails if either consumer’s max_capacity exceeds N. Unset (null), the stream stays ON_DEMAND. Requires Runlayer backend 1.28.2313 or later (the first release with the ENG-6765 fix, #19256), or SIEM export skips the first records of new child shards. Size N for peak writes too, and read the module README (“Audit-log stream shard count”) before changing it.
  • ToolGuard GPU alarms send recovery to the relay topics only: <project>-runlayer-tool-guard-high-gpu-<environment> and <project>-runlayer-tool-guard-gpu-pegged-<environment> still notify the ToolGuard alerts topic on ALARM, but on OK they now notify only the topics in critical_alerts_extra_sns_topic_arns (with enable_monitoring), like the ToolGuard alarms that page. Subscribers of the ToolGuard alerts topic no longer get their OK messages.
  • Paging alarms for the ToolGuard prompt guard model: with enable_runlayer_tool_guard = true, runlayer_tool_guard_models now includes runlayer-user-prompt-guard by default, so the apply adds its <project>-tool-guard-runlayer-user-prompt-guard-{error-rate,shed-rate,error-burst,load-failure}-<environment> alarms and their log metric filters. Like the other ToolGuard model alarms, they notify the ToolGuard alerts topic and the critical alerts topic. enable_runlayer_tool_guard_alarms = false turns them off.
  • Predict p95 dashboard panel follows runlayer_tool_guard_models: the ToolGuard “Predict p95, request-weighted” panel now charts only the models in that list. If you set it yourself, add runlayer-user-prompt-guard to keep that model on the panel and to get its alarms.
  • ToolGuard predict-latency alarm: with ToolGuard on, the apply adds a log metric filter on the ToolGuard log group and its metric tool-guard.PredictDurationUs, even with enable_runlayer_tool_guard_alarms off. With alarms on it also adds <project>-runlayer-tool-guard-predict-p95-<environment>, which fires when the p95 of successful predict requests stays above a threshold for 15 minutes: 400 ms on g6f.large, 150 ms on g6.xlarge, and 400 ms (the slowest measured type’s) on other instance types. It does not page: on ALARM it notifies the ToolGuard alerts topic and, with enable_monitoring, the topics in critical_alerts_extra_sns_topic_arns; on OK, only those extra topics. If your normal busy-hour p95 sits near the default, set runlayer_tool_guard_predict_p95_alarm_threshold_ms.
  • New paging alarm for a wedged ToolGuard GPU: with ToolGuard alarms on (enable_runlayer_tool_guard_alarms, default true), the apply adds <project>-runlayer-tool-guard-gpu-query-timeout-<environment>. It fires when a host’s nvidia-smi gets no answer within 10 seconds in 3 of 5 minutes, which means its GPU or driver is wedged and its GPU metrics have stopped. It notifies the same topics as runlayer-tool-guard-gpu-errors. Rev hosts get no such alarm; they publish their GPU metrics to the RunlayerRev namespace.
  • WorkOS, Sentry and Resend keys can live only in your Secrets Manager: tenant_secret_source = "provisioned" makes tasks read AUTH_API_KEY, SENTRY_DSN, SENTRY_RELAY_* and the new RESEND_API_KEY from a secret restricted/<project>-<env>-provisioned that Runlayer creates and fills in your account, instead of from the auth_api_key input and the WorkOS Vault fetch; auth_api_key is then left unset. Terraform only reads the secret. Contact Runlayer to schedule the switch; until then nothing changes, and "input" stays the default. New outputs provisioned_tenant_secret_name and provisioned_tenant_secret_arn name the secret (null in input mode).
November 6, 2026
Default Runlayer app version: 3.2.0. See the Runlayer changelog.Improvements
  • ToolGuard predictive scaling sizes g6.xlarge for its speed: on g6.xlarge with runlayer_tool_guard_predictive_target_requests_per_task_per_minute unset, the apply retunes the predictive policy in place from 50 to 250 requests per task per minute (during runlayer_tool_guard_cutover, the lower of the two types’). With balanced or aggressive, predictive scaling then launches about a fifth as many tasks; with conservative (default), only the recorded forecast changes. A value you set still wins: set it to 50 to keep today’s sizing.
  • Hook-events Parquet Lambda runs only when something reads its output: with enable_hook_events_pipeline = true, the apply now deletes the <project>-<environment>-hook-events-consumer Lambda, its event source mapping, IAM role and log group, unless data_export_bucket_arn is set or enable_sparkle_worker = true. If you set hook_events_alert_email, it also deletes the <project>-<environment>-hook-events-alerts SNS topic, its email subscription, and the Lambda’s <project>-<environment>-hook-events-consumer-failure-rate and <project>-<environment>-hook-events-consumer-iterator-age email alarms. This frees each hook-events shard’s read budget for the session consumers. The stream and the hook-events bucket stay, with their objects. To keep the Lambda and its email alarms, set enable_hook_events_parquet_archive = true.
  • New paging alarm on Parquet Lambda lag: where the Lambda runs, <project>-<environment>-hook-events-parquet-lambda-iterator-age fires when the Lambda is more than 15 minutes behind the stream for 15 minutes.
  • Audit SIEM export can autoscale on CPU: set audit_log_siem_export_kinesis_consumer_config.max_capacity above min_capacity; see the module README “Audit SIEM export consumer scaling”. Unset, the fleet stays fixed at desired_count.
  • Scale events can duplicate a few exports: each one can export a few audit records twice; dedupe on audit_log_id.
  • Audit SIEM export CPU alarm follows autoscaling: when audit_log_siem_export_kinesis_consumer_config.max_capacity is above min_capacity, the consumer’s 80% CPU alarm is replaced by <project>-<environment>-audit-siem-export-kinesis-consumer-cpu-saturated: the busiest task above 90% for 15 minutes, sent to the non-paging Slack relay only (the SIEM export lag alarm still pages). A fixed fleet keeps the 80% alarm.
November 5, 2026
Default Runlayer app version: 3.1.0. See the Runlayer changelog.Action required
  • ToolGuard host refresh: Before upgrading with ToolGuard enabled, allow for a rolling GPU-host refresh and model reloads. Ubuntu hosts now use local Amazon Time Sync instead of public NTP pools.
Improvements
  • New ToolGuard GPU alarms: with ToolGuard alarms on (enable_runlayer_tool_guard_alarms, default true), the apply adds two alarms. <project>-runlayer-tool-guard-high-gpu-<environment> fires when fleet-average GPU utilization stays above 70% for 15 minutes (the whole fleet is GPU-bound). <project>-runlayer-tool-guard-gpu-pegged-<environment> fires when the busiest host reads above 90% in 10 of 15 minutes (one stuck host, or several held near full). Both notify the ToolGuard alerts topic and, with enable_monitoring, the topics in critical_alerts_extra_sns_topic_arns, on ALARM and OK. The other ToolGuard alarms send only ALARM to the ToolGuard alerts topic, so its subscribers will now also get OK messages from these two. Neither pages the critical-alerts topic.
November 4, 2026
Default Runlayer app version: 3.0.0. See the Runlayer changelog.Breaking changes
  • ALB 5XX alarm threshold is now a percentage: alb_5xx_alarm_threshold is removed. If you set it, remove it before upgrading. To tune the alarm, set alb_5xx_alarm_threshold_percent (default 2).
Improvements
  • Lag alarms for the other audit and session stream consumers: with enable_monitoring and critical_alerts_enabled, the agent event trigger and audit SIEM export each get a critical alarm when they fall behind the audit stream, and the session materializer gets a non-paging one when session_materializer_enhanced_fan_out is on. The agent event trigger and session materializer alarms are removed while that consumer is parked at zero tasks.
  • ALB 5XX alarm fires on error rate, not single errors: the <project>-alb-5xx-errors-<environment> alarm now fires when ALB-generated 5XX exceed alb_5xx_alarm_threshold_percent of requests in 2 of 3 periods. Periods with fewer than 50 requests never breach.
  • New non-paging ALB 5XX drip alarm: <project>-alb-5xx-drip-<environment> fires on more than 20 ALB-generated 5XX in 6 hours. It routes only to critical_alerts_extra_sns_topic_arns, never the paging topic, so a slow error drip under the rate alarm stays visible.
  • Parked deployments send no alarm notifications: with infra_parked = true, alarms stay in place but notify nobody. Delivery resumes when you unpark.
Bug fixes
  • Audit log archive heartbeat waits for the first run: a new deployment no longer pages <project>-<environment>-audit-log-archive-heartbeat before its first scheduled archive run. It still pages when runs stop or never complete.
  • Fewer false ToolGuard GPU alarms: runlayer-tool-guard-cuda-oom and runlayer-tool-guard-gpu-errors no longer fire on the brief GPU memory pressure of a ToolGuard worker restart, which clears on its own.
  • Idle-transaction alarms no longer fire on brief backend pauses: the default db_idle_in_txn_max_duration_threshold is now 30 seconds (was 3). Both idle-transaction alarms use it. A few seconds of idle time from a busy backend or a deploy no longer pages. If you set this input, keep your value or remove it to use the new default.
  • Fewer false model load-failure alarms: <service>-<model>-load-failure no longer fires when a ToolGuard worker restart briefly runs out of GPU memory. That case clears on its own while the old worker keeps serving.
November 3, 2026
Default Runlayer app version: 2.4.0. See the Runlayer changelog.Improvements
  • ToolGuard instance type changes keep the old fleet’s capacity: in the apply that changes runlayer_tool_guard_instance_type, set runlayer_tool_guard_cutover = { from_instance_type = "<old type>", from_max_count = <old runlayer_tool_guard_max_count> }. Running tasks are not scaled in before their replacements serve. Remove it when terraform plan warns that no old-type hosts remain. While set, plan needs autoscaling:DescribeAutoScalingGroups, ecs:DescribeServices and ec2:DescribeInstances. On a faster type, also set runlayer_tool_guard_predictive_target_requests_per_task_per_minute to its measured per-task rate; the default 50 is measured on g6f.large.
  • ToolGuard GPU Auto Scaling group gets a RunlayerToolGuardTaskCeiling tag: with enable_runlayer_tool_guard, the first apply adds it in place (no host replacement). A cutover plan reads it so it never lowers the task ceiling in force.
Bug fixes
  • ToolGuard CPU target and high-CPU alarm scale with the host’s CPU: with a runlayer_tool_guard_instance_type other than g6f.large, the scaling profile’s CPU target and the high-CPU alarm threshold are now a share of that host’s CPU, not of a g6f.large’s (on g6.xlarge, balanced goes from 35% to 15% and the alarm from 55% to 23.6%). The old values sat above what a ToolGuard task uses on those hosts. The new values aren’t yet benchmarked there. g6f.large is unchanged.
  • Worker autoscaling now scales: with worker_config.max_capacity above min_capacity, the worker now scales on CPU and memory between those bounds. Before this fix it stayed at min_capacity. The apply replaces the worker’s autoscaling target and its two policies.
  • No backend alarm on every deploy: the backend and frontend unhealthy-targets alarms now fire only when targets are unhealthy in 5 of the last 10 minutes. They no longer fire for the minute a new task warms up during a deploy or scale-out.
November 2, 2026
Default Runlayer app version: 2.3.0. See the Runlayer changelog.Action required
  • ToolGuard keeps 25% of its GPU hosts free for rollouts: with enable_runlayer_tool_guard, autoscaling now stops at runlayer_tool_guard_max_count minus ceil(25%), at least one host, so a rolling deploy no longer stalls at the ceiling (6 allows 4 tasks, 8 allows 6). A service above the new ceiling scales in to it on apply. To keep your current task ceiling, raise runlayer_tool_guard_max_count before upgrading (for 6 tasks, set 8). If runlayer_tool_guard_desired_count equals runlayer_tool_guard_max_count, the warm floor also drops to the new ceiling unless you raise the max.
October 30, 2026
Default Runlayer app version: 2.2.0. See the Runlayer changelog.No upgrade notes for this release.
October 30, 2026
Default Runlayer app version: 2.1.0. See the Runlayer changelog.Improvements
  • WAF no longer blocks attack text in prompt scans: the bad_inputs_count_uri_regex default adds /api/v1/hooks/prompt/pre, so AWS KnownBadInputs body matches there count instead of block and the prompt reaches Prompt Guard. If you set the variable, compare it with the new default.
  • Audit consumer lag alarm: With enable_monitoring and critical_alerts_enabled, the existing audit iterator-age alarm now monitors the audit consumer’s enhanced fan-out lag. No usable telemetry becomes INSUFFICIENT_DATA without notifications; the threshold and notification routes stay the same.
  • Disabled audit consumers: The alarm is removed when audit_log_kinesis_consumer_config.enabled = false or audit_log_kinesis_consumer_config.desired_count = 0, even if the audit stream remains enabled.
October 29, 2026
Default Runlayer app version: 1.42.0. See the Runlayer changelog.No upgrade notes for this release.
October 29, 2026
Default Runlayer app version: 1.41.0. See the Runlayer changelog.Breaking changes
  • Terraform or OpenTofu 1.12.0 or newer: before bumping, upgrade the CLI on the host that runs terraform apply; terraform init now rejects older versions. Terraform 1.11 already failed plan on recent releases.
Bug fixes
  • VPC peering routes stay in place across applies: with vpc_peering_connections set, applies no longer delete and re-add the peer routes. The first plan after upgrading reports aws_route.peering_private as no longer managed by Terraform; nothing is deleted.
October 27, 2026
Default Runlayer app version: 1.40.0. See the Runlayer changelog.No upgrade notes for this release.
October 26, 2026
Default Runlayer app version: 1.39.0. See the Runlayer changelog.Breaking changes
  • Slow-pool memory floors rise for the session-topic model: the next ToolGuard CPU image adds runlayer-session-topic, and every slow-pool worker loads every model, so terraform plan now requires runlayer_topic_cpu_memory >= runlayer_topic_cpu_gunicorn_workers x 3584 MiB (x 4096 MiB with runlayer_skill_guard_enabled, the default) while topic-cpu is enabled; the previous floors were 2048 / 2560. The default runlayer_topic_cpu_memory moves from 16384 to 20480. A pin at the old 16384 default still plans with the default 4 workers (4 x 4096 = 16384, sitting exactly on the floor). Configurations that fail after the bump: a pinned memory near the old floor, for example 4 workers x 8192, or 5 or 6 workers x 16384 with SkillGuard on (5 x 4096 = 20480, 6 x 4096 = 24576); 6 workers no longer fit the 20480 default either and need an explicit pin, and with the legacy runlayer-topic-cpu stack kept on the m7i.2xlarge that pin (24576) only fits beside a runlayer_topic_cpu_fast_memory of about 6656 or less (24576 + 6656 = 31232, under the ~31.4k MiB an m7i.2xlarge registers to ECS; the plan-time check only enforces 32768 - 1024 = 31744, a budget measured on an xlarge, so sizing up to it can plan and then fail placement). Fix by raising runlayer_topic_cpu_memory, lowering runlayer_topic_cpu_gunicorn_workers, or setting runlayer_skill_guard_enabled = false. On hosts smaller than the m7i.2xlarge default you already pin runlayer_topic_cpu_memory (the default never fit a 16 GiB host); that pin must now also clear the new per-worker floor, and an m7i.xlarge with the legacy stack kept (keep_legacy_runlayer_topic_cpu, the default) holds at most three slow workers (3 x 4096 = 12288 plus a 3072 MiB fast pool = 15360 = 16384 - 1024); four slow workers fit an xlarge only with SkillGuard off and the legacy stack dropped (4 x 3584 = 14336). A stack with enable_runlayer_topic_cpu = false keeps the old floor and is unaffected. The floors come from a measurement: one slow-pool worker with runlayer-session-topic loaded measures 3,011 MiB resident on the candidate image (2026-10-22, full roster, +375 MiB over the same roster without the model), so 3584 MiB is that figure plus 573 MiB of margin, and 4096 MiB adds the ~0.5 GiB SkillGuard scan peak on top with the same margin. Follow-up: Kubernetes deployments are not covered by this module; the operator still enforces 2048 MiB per worker and moves to the same floors before the session-topic image ships there.
  • enable_resolver_endpoint removed: delete it from your configuration; terraform plan fails while it is set. If you set it to true, the next apply deletes the Route53 Resolver inbound endpoint and its security group. If you still need private-zone resolution from outside the VPC (VPN or on-prem forwarders), create your own INBOUND aws_route53_resolver_endpoint in the VPC outside the module and re-point your forwarders to it before applying. The resolver_endpoint_ips output is also removed.
October 26, 2026
Default Runlayer app version: 1.38.0. See the Runlayer changelog.No upgrade notes for this release.
October 24, 2026
Default Runlayer app version: 1.37.1. See the Runlayer changelog.Bug fixes
  • Upgrading with the CPU guard stacks no longer fails terraform plan: with enable_runlayer_topic_cpu, bumping from 40.x to 41.0.0 failed with Error: Cycle naming the guard task definitions. Bump straight to this release instead; no other change is needed.
October 24, 2026
Default Runlayer app version: 1.37.0. See the Runlayer changelog.Breaking changes
  • The identity the CPU guard cutover wait runs as also needs ecs:DescribeServices: with enable_runlayer_topic_cpu and the default waits, scripts/guard_cpu_wait.sh now also polls the stack’s ECS service, as backend_migration_aws_role when that is set, otherwise as the apply host’s own credentials. Add ecs:DescribeServices next to elasticloadbalancing:DescribeTargetHealth on that identity before bumping; without it the apply fails before the backend has moved, and the error names the missing action. Setting both runlayer_guard_cpu_wait_for_steady_state and wait_for_ecs_steady_state to false skips the wait instead.
Action required
  • Schedule a tunnel interruption: with tunnels and runlayer_obs_otel_enabled = true, confirm existing-VPC private subnets can reach otlp.obs.runlayer.com on HTTPS before upgrading. The apply restarts tunnel routers and drops live tunnel sessions while clients reconnect.
  • Schedule later observability-secret seeding too: on a tunnel deployment, seeding runlayer/otlp-ingest enables collector wiring on the next apply, which restarts tunnel routers and drops live sessions while clients reconnect.
  • Keep the old telemetry proxy through the collector soak: apply, wait for the collector and tunnel connections to stabilize, then verify metrics and traces before removing the old proxy.
  • Disable export before rollback: set runlayer_obs_otel_enabled = false, apply, and wait for tunnel connections to reconcile before restoring the old module and application. This temporarily stops tunnel metrics and traces; application logs continue when runlayer_obs_otel_logs_enabled = true.
Improvements
  • Tunnel telemetry uses a private local relay: routers and endpoints send metrics and traces to a deployment-local collector. Set tunnel_otel_collector_desired_count = 2 for two replicas; logs remain in CloudWatch.
  • A failed first create of a CPU guard service recovers on the next apply: a guard-cpu-slow-path, guard-cpu-hot-path or runlayer-topic-cpu service left tainted by a failed steady-state wait is now replaced destroy-then-create, so a rerun completes; previously every rerun failed with Creation of service was not idempotent until the state was untainted by hand.
October 23, 2026
Default Runlayer app version: 1.35.0. See the Runlayer changelog.No upgrade notes for this release.
October 23, 2026
Default Runlayer app version: 1.34.0. See the Runlayer changelog.Breaking changes
  • The identity the cutover wait runs as needs elasticloadbalancing:DescribeTargetHealth: with enable_runlayer_topic_cpu and the default waits, every apply that points the backend at a CPU guard stack (the two new stacks on the cutover, the old one with the split off) runs scripts/guard_cpu_wait.sh on the apply host, which polls describe-target-health with the AWS CLI as backend_migration_aws_role when that is set, otherwise as the apply host’s own credentials; the aws provider’s assume_role does not reach it. The module README’s role list for backend_migration_aws_role previously named only ecs:RunTask, ecs:DescribeTasks, ecs:StopTask and iam:PassRole; a role scoped to that gets AccessDenied and the apply fails right after the new stacks exist, with the backend still on the old stack (nothing goes down, the script exits at once and names the missing action and the opt-out). Add elasticloadbalancing:DescribeTargetHealth to whichever identity runs the wait (the role when backend_migration_aws_role is set, otherwise the apply host’s own credentials) and make sure the apply host has the AWS CLI (the migration runner already needs it) before bumping. To defer the whole change instead, set enable_runlayer_guard_cpu_split = false and keep_runlayer_guard_cpu_split_stacks = false (the first alone still creates both new stacks and runs the wait on the old one), and both runlayer_guard_cpu_wait_for_steady_state and wait_for_ecs_steady_state to false until the IAM change lands.
  • AgentCore prune no longer uses apply-host uv / local-exec: with AgentCore enabled, version prune runs in a module Lambda (daily schedule plus invoke before each runtime update). Remove runlayer_agentcore_control_aws_role_arn if you set it for v38 — that input is gone. Apply hosts no longer need uv or AgentCore list/delete IAM; the apply role only needs lambda:InvokeFunction on the prune function (the same credentials that create it).
Action required
  • Two new CPU guard hosts are created beside the existing topic CPU stack: with enable_runlayer_topic_cpu, the apply adds guard-cpu-hot-path (runlayer_guard_cpu_hot_path_instance_type, default c7i.xlarge; serves the session tool scanner) and guard-cpu-slow-path (runlayer_topic_cpu_instance_type; topic_cascaded, session titling, tool removal, SkillGuard), each with its own cluster, ASG, NLB, log group and alarms, and points the backend at them. Nothing in the existing runlayer-topic-cpu stack is changed, replaced or removed, so there is no cutover window: the apply waits until both new services report steady state and their target groups have healthy targets (about 6 minutes for the hosts to boot; runlayer_guard_cpu_wait_for_steady_state, default true, or the module-wide wait_for_ecs_steady_state), and only then rolls the backend as on any deploy, while the old stack keeps running. Bounds: the steady-state wait by ecs_service_wait_timeout, the target health wait 15 minutes; its credentials are under Breaking changes above. Cost: a default deployment runs three CPU guard hosts instead of one until you drop the old stack (+1 m7i.2xlarge and +1 c7i.xlarge, about +12 vCPUs of On-Demand Standard quota), and one more than today after it; with runlayer_topic_cpu_desired_count = N the duplicate is N slow hosts, and the hot path follows the same N unless runlayer_guard_cpu_hot_path_desired_count is set. Budget the new hosts in cost and EC2 quotas, and the extra apply time in CI timeouts.
  • Drop the old stack once the new ones are verified: after the apply, confirm on the Grafana topic-cpu-health dashboard that scans and topic requests arrive on the new services (the ECS panels show the guard-cpu-* series next to the legacy runlayer-topic-cpu one), that SessionToolScannerConnectErrorCount stays at zero and that the runlayer-topic-cpu service is idle: its CPU series flat, and the <model>.RequestCount metrics in the RunlayerModels/<project>-<env> namespace (legacy pool only; the new pools publish under RunlayerModels/<project>-<env>/guard-cpu-slow-path and .../guard-cpu-hot-path) at zero. Then, in a later apply, never the apply that bumps the module, set keep_legacy_runlayer_topic_cpu = false: that plan destroys only the old cluster, ASG, NLB, log group (with its retained logs; export first if you need them), alerts SNS topic and alarms. In the same apply as the bump it would destroy the old NLB and service before the backend rolls, and the backend’s old tasks would keep calling them while the new hosts boot: SkillGuard imports fail closed and /pre scans fail open for that window. Fresh installs are the exception and can set it false from day one instead of getting an idle old host. The old stack and keep_legacy_runlayer_topic_cpu are removed in a future major release.
  • Subscribe to the new alerts topic before dropping the old stack: both new stacks alarm to <project>-guard-cpu-alerts-<env>, and, for as long as the old stack is kept, to the old <project>-runlayer-topic-cpu-alerts-<env> topic as well, so the subscriptions you have today page for the new hosts from the first apply on with nothing to do. The old topic and its subscriptions are untouched. Setting keep_legacy_runlayer_topic_cpu = false removes the old topic, after which the new stacks’ alarms fire to the new topic alone: add your email / PagerDuty subscriptions to it before that apply, or the new hosts alarm without paging from then on.
  • Dashboards and alarm hooks keyed on runlayer-topic-cpu: ECS metrics for live traffic now report under ServiceName guard-cpu-slow-path and guard-cpu-hot-path; the log groups are <project>-guard-cpu-slow-path-logs-<env> and <project>-guard-cpu-hot-path-logs-<env>. The shipped Grafana dashboard is updated; private dashboards, SIEM filters or alarm subscriptions matching the old names need the new ones. Model error-rate alarms are added as guard-cpu-slow-path/<model> and guard-cpu-hot-path/<model>, reading per-model metrics from their own namespaces (RunlayerModels/<project>-<env>/guard-cpu-slow-path and .../guard-cpu-hot-path) so they never sum with the legacy pool’s; the topic-cpu/<model> alarms and their RunlayerModels/<project>-<env> metrics stay until the old stack is dropped. Duration metric names and namespaces are unchanged.
  • Thread pin: the new slow path pins OMP_NUM_THREADS per worker from the same runlayer_topic_cpu_omp_threads input as the old slow pool (default null = the slow host’s vCPUs / runlayer_topic_cpu_gunicorn_workers, 2 on the m7i.2xlarge default, 1 on an m7i.xlarge with 4 workers), derived from the new stack’s own instance type. An explicit value whose threads x workers exceeds the host’s vCPUs fails the plan on both stacks. The old stack’s sizing inputs (runlayer_topic_cpu_fast_*, including runlayer_topic_cpu_fast_omp_threads) keep their meaning for the old stack until it is dropped; the hot path has its own runlayer_guard_cpu_hot_path_omp_threads.
  • Rollback is two applies, like the cutover: enable_runlayer_guard_cpu_split = false points the backend back at the old runlayer-topic-cpu stack, which is always kept while the split is off, and leaves both new stacks running, idle (keep_runlayer_guard_cpu_split_stacks, new, default true). Terraform destroys a removed stack before it rolls the backend off it, so an apply that did both would leave the backend’s tasks pointed at stacks being torn down for the length of the roll; keeping them is what makes the rollback safe. While the old stack is still running the move is immediate. After you have dropped it (keep_legacy_runlayer_topic_cpu = false), the same apply re-creates it first and, with the wait on, only moves the backend once its tasks are running and its targets are healthy (about 6 minutes for the host to boot; the old service’s steady-state wait runs under the provider’s default 20-minute timeout). Once the backend has rolled, set keep_runlayer_guard_cpu_split_stacks = false in a second apply to remove the new stacks; or set the split back to true to try again.
Improvements
  • The session tool scanner gets its own host: /pre scans no longer share a host with topic_cascaded, session titling, tool removal and SkillGuard. On a dedicated c7i.xlarge the scanner answers 64 % of a 70-concurrent burst inside its 0.8 s budget (27 % on the shared host) and 460/460 of a 20-burst at p95 423 ms; the slow path alone on its host brings topic_cascaded p50 from 1.5 s to 0.12 s with SkillGuard scanning concurrently.
  • The backend only moves onto a stack that answers: the backend task definition is only rolled to the new NLBs after both guard-cpu-* services exist and, with runlayer_guard_cpu_wait_for_steady_state (new, default true), report steady state and have healthy targets; on a rollback the same holds for the old stack. A /pre scan or SkillGuard import can never be sent to a target group whose host is still booting. Both new containers run the same plain /health check the topic CPU pools run today; as today, that check is 200 whenever the server is up, even if a model failed to load, so it does not prove the scanner or SkillGuard model is loaded. That is a pre-existing property of the serving image, unchanged by this release and tracked as a follow-up on the image. Moving the backend and removing the stack it left are always separate applies (keep_legacy_runlayer_topic_cpu, keep_runlayer_guard_cpu_split_stacks).
  • New hot-path inputs: runlayer_guard_cpu_hot_path_instance_type (c7i.xlarge), _desired_count (null: follows runlayer_topic_cpu_desired_count, so a deployment on two topic CPU hosts gets two scanner hosts, not one), _gunicorn_workers (4), _memory (6144), _omp_threads (1), _queue_max (22, per worker, DYNAMIC_BATCH_QUEUE_MAX), _expected_concurrent_callers (16), _models (["runlayer-dynamic-policy"], becomes the container’s MODEL_FILTER; must stay a subset of runlayer_topic_cpu_models and must include runlayer-dynamic-policy, the model the backend’s scanner requests, so a roster that would 404 every /pre is refused at plan). workers x threads must fit the instance’s vCPUs and memory must fit workers x 1024 and the instance; validation refuses otherwise.
  • Hot-path alarms: no-running-tasks, memory > 85 %, CPU > 70 % for 15 minutes, and unhealthy-targets alarms on the hot-path service, on the <project>-guard-cpu-alerts-<env> topic with the slow-path alarms (and on the old topic while the old stack is kept, see Action required).
  • Scanner connection errors are counted: a new SessionToolScannerConnectErrorCount metric (namespace RunlayerTopicCpu/<project>-<env>, from backend logs) sits next to the existing timeout count on the topic-cpu-health dashboard, so a scanner endpoint that is unreachable shows separately from one that is slow.
  • Aurora lock-wait logging: Set database_config.log_lock_waits = true to log waits exceeding the existing deadlock_timeout. The default leaves the parameter unspecified; set false to disable logging explicitly.
  • AgentCore prune uses a least-privilege Lambda role: the function can list/delete AgentCore runtime versions (plus companion DeleteAgentRuntimeEndpoint / DeleteWorkloadIdentity that versioned delete authorizes) and write its own logs. It also runs once a day on EventBridge. The function zip ships a botocore data overlay so runtime boto3 can pass agentRuntimeVersion.
  • Least-outstanding-requests routing for the backend: set backend_load_balancing_algorithm = "least_outstanding_requests" to send each backend request to the task with the fewest in flight, which lowers median latency under load. It turns off the backend’s 60s slow start when backend_health_scaling_enabled is on (AWS can’t combine the two), and makes the backend hold each 5xx briefly so a task that fails fast can’t draw most of the traffic. If you pin app_version, use a backend release that includes that hold. The default, round_robin, changes nothing.
  • Backend CPU target override: backend_cpu_target_value sets the backend’s CPU target-tracking target without replacing services_configurations. A lower target runs more tasks for lower latency; recheck rds_database_connections_threshold when you lower it.
  • Steadier backend scale-in: with backend_health_scaling_enabled = true, the quiet-fleet scale-in now waits until CPU is below 0.9 x 0.75 of the target (47.25% for the default 70%), not 90% of it, so removing a quarter of the tasks lands under the target instead of triggering an immediate scale-out. Busy fleets may keep slightly more tasks.
October 22, 2026
Default Runlayer app version: 1.33.0. See the Runlayer changelog.Improvements
  • Customer-managed Aurora encryption: optionally set rds_kms_key_arn before creating a cluster to use your own KMS key for database storage, backups, and snapshots. The default preserves existing key selection; changing an existing cluster’s key requires migration.
October 21, 2026
Default Runlayer app version: 1.32.0. See the Runlayer changelog.Action required
  • New session compactor service, ingest rolls after it: with enable_hook_events_pipeline, the apply creates session-materializer-compactor-service and waits for it to be healthy before it rolls the existing materializer service. The compactor runs tasks only while session_parquet_store_enabled is on. Bump the module together with or before app_version. If the store is already on when you bump and app_version is older than this module release, the apply fails on the compactor and leaves the materializer unchanged; raise app_version and apply again. If the store is off, the apply succeeds either way: before you turn the store on, raise app_version to this release or later. A mistake there is not caught by the apply (the compactor rolls at zero tasks); with monitoring on it shows up as the compactor’s running-tasks alarm (needs Container Insights) or busy dead-man alarm (needs critical alerts and autoscaling). Roll the module back only together with app_version.
  • Fixed-size materializer configs must pin capacity: the session_materializer_config.max_capacity default rises from 1 to 4, which turns on autoscaling (1–4 tasks) and makes desired_count seed-only. If you set desired_count (to 0 to stop ingest, or above 1 for a larger fixed fleet), also set min_capacity and max_capacity to the same value (for example both 1) before upgrading. Otherwise ingest restarts or resizes to 1–4 tasks on the upgrade.
Improvements
  • PrivateLink deployment access: Adds IAM read permissions for checking security-group rule capacity and waiting for ECS rollouts before removing obsolete access rules. Existing module inputs remain compatible.
  • Materializer sizing split: session_materializer_config now sizes ingest only (new defaults 1 vCPU / 2 GB, autoscaling 1–4 tasks on ingest lag). Compaction is sized by session_materializer_compactor_config (default 2 vCPU / 4 GB, 1–4 tasks on fold load; zero tasks while session_parquet_store_enabled is off). If you raised session_materializer_config for compaction, move those values to session_materializer_compactor_config.
  • Materializer deploys start new tasks first: ingest no longer stops before the new task starts. Upgrading from a module older than shard leasing, the first deploy may deliver some session events twice; nothing is lost.
October 21, 2026
Default Runlayer app version: 1.30.0. See the Runlayer changelog.Breaking changes
  • AgentCore prune needs deploy-account credentials on apply: with AgentCore enabled, prune fails if the apply-host credentials are not the deploy account (the aws provider account). Before bumping, set runlayer_agentcore_control_aws_role_arn when terraform starts in a build/parent account, or ensure ambient credentials are already the deploy account.
Improvements
  • AgentCore prune no longer needs a preinstalled uv: with AgentCore enabled, apply uses host uv when present, otherwise bootstraps a pinned uv into a temp dir (curl + Astral installer, which then fetches the binary from GitHub releases). Air-gapped hosts should put uv on PATH or set the installer’s download-base env (UV_INSTALLER_GITHUB_BASE_URL / UV_DOWNLOAD_URL); mirroring only install.sh is not enough. AgentCore control IAM from v34.0.0 is unchanged.
October 20, 2026
Default Runlayer app version: 1.29.0. See the Runlayer changelog.Improvements
  • Agent iMessage channel (opt-in): enable_imessage_channel = true lets people text an agent from Messages through Sendblue, with no Apple hardware in the stack. Off by default. Set it with the line’s sendblue_from_number, sendblue_api_key_id, sendblue_api_secret_key and sendblue_webhook_secret (the plan fails without all four), and add the imessage_channel_sendblue_webhook_url output as the line’s receive webhook in Sendblue. See ECS optional features → iMessage channel.
October 20, 2026
Default Runlayer app version: 1.28.1575. See the Runlayer changelog.Breaking changes
  • The agent runtime no longer needs enable_runlayer_agents: with enable_runlayer_agentcore_runtime = true and enable_runlayer_agents = false, the next apply creates the AgentCore runtime. Before bumping, give the apply host uv and credentials for the AgentCore control APIs and, with WAF IP allowlisting, set runlayer_agentcore_network_mode = "VPC" with PrivateLink; or set enable_runlayer_agentcore_runtime = false to keep it off.
Improvements
  • Runlayer Assistant without custom Agents: set enable_runlayer_agentcore_runtime = true to run Runlayer Assistant, Security Investigations and Procedures. enable_runlayer_agents is only needed for custom agents.
  • Two session-compaction alarms: deployments with enable_monitoring, critical_alerts_enabled and a running session materializer get a non-paging warning when a session sits at the fold quarantine breaker (CompactionQuarantinedSessions >= 1 for 15 min — it never self-heals), and a paging CRITICAL when, for 30 min straight, every sweep saw the oldest dirty session waiting more than 2h for compactor work (CompactionBacklogAgeSeconds, Minimum > 7200 — Minimum rides out the short ~24h blips that late-arriving events cause on a healthy queue). Both need the backend image this release ships: on older images the backlog-age gauge idles at ~86,400 s and the CRITICAL alarm stands in permanent breach, and the quarantine gauge is not emitted at all. If you pin an older app version, hold this module bump until you move the app forward.
October 17, 2026
Default Runlayer app version: 1.28.1480. See the Runlayer changelog.Action required
  • Check the topic-cpu slow pool’s memory per worker and image tag before bumping, or pin SkillGuard off: runlayer_skill_guard_enabled now defaults to true, so every deployment with enable_runlayer_topic_cpu = true scores skills with SkillGuard after the bump, and two validations fail plan for shapes that cannot run it. Memory: runlayer_topic_cpu_memory must be at least `runlayer_topic_cpu_gunicorn_workers
    • 2560MiB while the switch is on (SkillGuard scans peak at ~2.2 GiB per worker, measured 2026-10-16 across the fleet); the module default (4 workers, 16384 MiB) passes, but a slow pool left on the xlarge-era 8192 MiB with 4 workers does not. Image:runlayer_topic_cpu_image_tagmust berunlayer-multimodel-guard-v202607291645or newer (the first bundle with the SkillGuard model); the module default passes. For each failing input, either raise it to the default or setrunlayer_skill_guard_enabled = falseto keep the previous skill scorer and the 2048 MiB/worker floor. Once SkillGuard is on it becomes the source of each skill's risk level: skills it rates higher may be newly blocked under your skill policies, and file preview follows that level. The switch is carried by this module only; the backend default stays off, so Kubernetes (operator) and docker-compose deployments are unchanged until they setRUNLAYER_SKILL_GUARD_ENABLED` themselves.
Improvements
  • Stable public IPv4 block for the ALB: set enable_alb_public_ipv4_pool = true to give the public ALB its IPv4 addresses from one Amazon-provided block, output as alb_public_ipv4_cidr, that client networks can route or allowlist. It creates the account’s IPAM in region, so an account that already has one there can’t turn it on. Turning it on changes the ALB’s public IPv4 addresses once; turning it off releases the block, and a later re-enable gets a different CIDR.
October 16, 2026
Default Runlayer app version: 1.28.1446. See the Runlayer changelog.Improvements
  • runlayer_topic_cpu_fast_queue_max: new input (default 22, per worker) setting DYNAMIC_BATCH_QUEUE_MAX on the topic CPU fast-pool container. Both topic CPU containers run one image and therefore one serving profile, which carries the slow pool’s shallow cap; the apply now sets the fast pool’s cap explicitly so it stops inheriting that value. 22 is one CPU thread’s in-budget capacity for a /pre scan, so the container admits 22 x runlayer_topic_cpu_fast_gunicorn_workers concurrent scans (132 at the default 6) before shedding. 0 disables admission (rollback). No action required — the only visible change is a task definition revision.
  • runlayer_topic_cpu_expected_concurrent_callers (default 4), runlayer_topic_cpu_fast_expected_concurrent_callers (default 16), runlayer_tool_guard_expected_concurrent_callers (default 16): new inputs passed to each guard container as EXPECTED_CONCURRENT_CALLERS, the concurrency the pool is sized for. The serving image reports an admission cap below it at boot and in /health. Validation refuses a fast-pool cap below the declared fast-pool concurrency. No action required.
October 16, 2026
Default Runlayer app version: 1.28.1407. See the Runlayer changelog.No upgrade notes for this release.
October 15, 2026
Default Runlayer app version: 1.28.1396. See the Runlayer changelog.No upgrade notes for this release.
October 15, 2026
Default Runlayer app version: 1.28.1381. See the Runlayer changelog.No upgrade notes for this release.
October 13, 2026
Default Runlayer app version: 1.28.1271. See the Runlayer changelog.No upgrade notes for this release.
October 12, 2026
Default Runlayer app version: 1.28.1233. See the Runlayer changelog.No upgrade notes for this release.
October 10, 2026
Default Runlayer app version: 1.28.1158. See the Runlayer changelog.Improvements
  • WAF no longer blocks code in hook and upload bodies: on the routes in bad_inputs_count_uri_regex, AWS KnownBadInputs body matches now count instead of block, and header, path, query and method matches block.
  • More routes, any client: the bad_inputs_count_uri_regex default adds AI Watch hooks and scan inventory, local MCP pre/post and tool sync, skill/plugin batch uploads, and skill file sync and scoring, and bad_inputs_count_user_agent now defaults to "" (was "Runlayer CLI"). If you set either, compare it with the new default.
  • Renamed WAF rule: the count-mode rule is now AWS-AWSManagedRulesKnownBadInputsRuleSet-BodyCountRoutes (was ...-CountRunlayerCli); its metric name is unchanged and now also records blocked requests. Update WAF log queries that match the old rule name.
October 9, 2026
Default Runlayer app version: 1.28.1115. See the Runlayer changelog.Breaking changes
  • AgentCore needs more IAM on apply: with AgentCore enabled, the AWS credentials on the host that runs terraform apply (not a provider assume_role) need bedrock-agentcore:ListAgentRuntimes, bedrock-agentcore:ListAgentRuntimeVersions, bedrock-agentcore:ListAgentRuntimeEndpoints and bedrock-agentcore:DeleteAgentRuntime.
  • uv on the apply host: with AgentCore enabled, install uv where terraform apply runs.
Improvements
  • Redis connection alarms scale with your backend: they page on connections per backend task above redis_connections_per_backend_task_threshold (default 750), so scale-outs and deploys no longer page. They need Container Insights; with container_insights = "disabled" only the backstop below remains.
  • redis_curr_connections_threshold is now a raw backstop: default rises from 2000 to 20000. If you set it, keep it above your max backend tasks × ~500. The critical Redis cluster’s alarm still uses your value, or 2000 if you never set one; set critical_redis_curr_connections_threshold to split them.
  • Deploys no longer hit AgentCore’s version cap: with AgentCore enabled, each apply that updates the runtime deletes its oldest versions, keeping runlayer_agentcore_runtime_versions_to_keep (default 50).
October 7, 2026
Default Runlayer app version: 1.28.1078. See the Runlayer changelog.Improvements
  • Backend worker-death alarm: a new …-backend-worker-deaths alarm pages when more than two backend uvicorn workers die within 15 minutes (the WorkerDied metric in the Runlayer/Backend/<project>-<environment> namespace, read from the backend task log group). Repeated deaths usually mean task memory is at the container limit and the kernel is killing workers. Created with enable_monitoring and critical_alerts_enabled; nothing to configure.
October 7, 2026
Default Runlayer app version: 1.28.1073. See the Runlayer changelog.Breaking changes
  • Rev CPU variables removed: delete runlayer_rev_gunicorn_workers, runlayer_rev_memory and runlayer_rev_cpu_share from your configuration. Rev no longer runs in the topic CPU task.
Action required
  • Rev runs as its own GPU service: with enable_runlayer_rev = true, the apply removes the Rev container from the topic CPU task and creates a dedicated GPU ECS service (g6f.large, its own cluster, internal NLB on :8080, security group and alert topic). Rev is unavailable until the first GPU host boots, typically 10 to 15 minutes. Confirm g6f.large is available and within quota in your region before upgrading.
  • No-healthy-hosts alarm: the new alarm pages if no Rev host registers within 20 minutes of the first apply. Set enable_runlayer_rev_alarms to control it.
Improvements
  • runlayer_rev_desired_count: new input (default 1) for the Rev GPU service task count.

Earlier releases