Skip to main content
Runlayer ships test detections: a reserved MCP server and two reserved shell tokens that Enforce and Protect always block, the way antivirus products always flag the EICAR test file. Use them to confirm a device is enforcing after a rollout, or to check a device a user reports as “not blocking”, without allowlisting anything or creating a real violation. Each result shows in the agent’s reply, and Activity records the tests without creating a real finding (see Check Activity).

Before you start

  • The device runs AI Watch in Enforce or Protect with hooks installed. Check with aiwatch config show and aiwatch setup hooks check --mdm (see Troubleshooting).
  • The AI Watch package includes the enforce-test command. Run aiwatch enforce-test --help; if the command is missing, update AI Watch. The full CLI has the same command as runlayer enforce-test.
  • For the two shell tests, Full session scanning APIs and the client under Hook clients are enabled under Settings → Agent session monitoring. The MCP server test does not need them.
Test detections are on for every organization. To turn them off for yours, contact Runlayer support.

Get the setup for your client

Run this on the device:
It prints the test MCP server entry for Cursor, Claude Code, Codex, Gemini CLI, GitHub Copilot CLI and VS Code, using the path of the aiwatch binary you ran, followed by the prompts below. Add --json for machine-readable output.

Test 1: the test MCP server

Add the printed entry to your client’s MCP config and restart the client. For example:
On Windows the command is C:\Program Files\Runlayer\AIWatch\aiwatch.exe. Another path to aiwatch or runlayer works too, as long as the client runs it directly with only the arguments enforce-test mcp. On macOS and Linux that path can’t contain spaces or quotes. A server launched through another program, such as a shell or a script interpreter, is not the test server and gets your normal policy. In a new chat, ask the agent:
Call the runlayer_enforce_test tool.
Expected: the call is blocked with Runlayer Enforce test detection: this call was blocked as expected. Nothing was at risk. See Shadow AI → Activity. The client still starts the server and lists its one tool; Enforce blocks the call itself. If the tool answers instead, saying the test detection did not block the call, the device is not enforcing for that client. No allowlist entry or Activity override can allow this server, so a passing result never depends on your policy configuration.

Remote variant

To test remote MCP servers instead of local ones, or on a device whose AI Watch does not have the enforce-test command yet, add a remote server entry pointing at your Runlayer instance:
Cursor (.cursor/mcp.json)
Use the same prompt; the expected result is the same. The hosted server needs no sign-in and holds no data, and if a call reaches it, it answers that the test detection did not block it.

Test 2: blocked before a shell command runs

Ask the agent:
Run this shell command: echo RUNLAYER-ENFORCE-TEST-INPUT
Expected: the command is denied before it runs, with the same test detection message.

Test 3: blocked after a shell command runs

Ask the agent:
Run this shell command: echo RUNLAYER-ENFORCE-TEST-OUTPUT
Expected: the command runs, and its output is blocked before it reaches the agent. This token is never denied up front, so this test exercises the after-the-call path. The shell tests only fire for the client’s shell or terminal tool, and only when the command itself contains the token. Reading or editing a file that mentions a token, or printing it from a script, does nothing. If Runlayer could not fully scan a shell test’s call, for example while a scanner is unavailable, the call is still blocked, with the shorter reason Runlayer Enforce test detection, and its Activity row is not marked as a test. That still counts as a pass.
Some clients cannot block tool output after a call. On Grok CLI, Cline CLI, Devin CLI, and Windsurf, the agent still sees the output, so Test 3 cannot pass there. That matches how those clients handle every output block; see Endpoint modes.In Cursor, the shell tests run through Cursor’s tool hooks, which AI Watch installs for Enforce and Protect. Cursor’s afterShellExecution event never reaches Runlayer, so it plays no part in the result.

Check Activity

Open Shadow AI → Activity. Tests 1 and 2 each leave a Blocked row with the reason Runlayer Enforce test detection and a Test badge. Test 3 leaves the result row of the echo RUNLAYER-ENFORCE-TEST-OUTPUT command with a Test badge, and that row reads Allowed: result rows record that the command ran, not what happened to its output. For Test 3, the pass signal is the blocked output in the agent’s reply. Test rows cannot be allowed from Activity, and they are not real findings. On a device that uses an organization API key but that Runlayer can’t yet match to a user, the shell tests leave no row: Test 2 still blocks, with a message that leaves out the pointer to Activity, and Test 3 does not block.

Monitor mode and exempt users

  • In Monitor, nothing blocks. The test MCP tool answers that the test detection did not block it, and the two shell tests still leave rows with the Test badge, which confirms the hooks report to Runlayer.
  • For a user on Settings → Tool governance → Exempt users, the calls go through and each test row shows as an orange Allowed, with “Would have been blocked — enforcement disabled for this user” in its details. That is the expected result for an exempt user.

Verify built-in MCP server blocking

Built-in MCP servers, such as GitHub Copilot CLI’s github-mcp-server, have no test detection: the client ships them, so there is nothing to configure. Check that dimension by hand on a test device:
  1. Under Settings → Tool governance → Enforce → Allowed built-in MCP servers, turn off Use Runlayer safe defaults and remove the GitHub Copilot CLI / github-mcp-server row from the custom list. Save.
  2. In GitHub Copilot CLI, ask the agent to use a GitHub MCP tool, such as “List my GitHub repositories using the GitHub MCP server.” The call should be blocked.
  3. Restore your policy: turn Use Runlayer safe defaults back on (this discards the custom list), or re-add the row if you were already using a custom list.
For Protect, add the same built-in to the Protect denylist, test, then remove it. See Built-in MCP servers.

If a test is not blocked

Work through Hooks are installed but not blocking. The usual causes are a device still in Monitor, a client that was not restarted after hooks were installed, or an exempt user.