Skip to main content

​
Authentication and Configuration

Package-based AI Watch uses one organization API key for scans, hooks, sessions, and macOS updates. The key is written by the macOS Configuration Profile (OrgApiKey) or Windows MSI property (AIWATCH_ORG_API_KEY) and read by the installed aiwatch binary. Manual runlayer scan workflows can still use interactive login credentials or a stored organization API key. Legacy enrollment keys only apply to older non-package auto-provisioning flows. The AI Watch organization API key is tenant-scoped: it is not tied to a user, cannot impersonate one, and is blocked from policy-based access control. On the backend it is stored only as an HMAC-SHA256 hash — the plaintext is shown once at creation and cannot be retrieved later, so if it is lost, regenerate the configuration to mint a new one.

​
Interpreting Discovery Results

Cause: Detect classifies based on MCP config files found on disk, not actual tool calls. A user being attributed to a shadow MCP means a matching config was found on a device whose login username resolved to that user — not that they ever invoked any tool from the server.Common reasons a config gets attributed to a user who never used it:
  • Project-level config in a cloned repo. A .cursor/mcp.json, .vscode/mcp.json, .mcp.json, .codex/config.toml, or similar checked into a shared repo is discovered on every developer who clones it. The config is attributed to the OS user who ran the scan on that machine.
  • Stale or experimental config. The user added the server to a client config file once (e.g. trying it out, copy-pasting from a blog), never invoked it, and forgot it was still there.
  • Shared device. Another OS account on the same workstation configured the MCP, but the scan ran under a different OS account that resolves to this Runlayer user.
  • Username resolution mismatch. Tiered matching can link a device’s OS username to the wrong Runlayer user. See User and Identity Mapping, then fix via Settings → MDM Configuration → Unresolved usernames and re-analyze.
How the user (or admin) can verify the source themselves:
  1. Click the shadow artifact row in the Shadow page to open the detail drawer.
  2. The detail includes the device hostname, device id, the MCP server’s command / args / url, and any project paths where the config was found.
  3. If a project path is listed, the discovery came from a project-level config file. Open that repo and inspect the checked-in MCP config to confirm.
  4. If no project path is listed, the discovery came from a global client config. On the listed device, inspect the relevant client config (e.g. ~/Library/Application Support/Claude/claude_desktop_config.json, ~/.cursor/mcp.json, ~/.codex/config.toml).
  5. To clear the discovery, remove the entry from the offending config file (or uninstall the project) and the next scan will drop it from the inventory.
Cause: Detect assigns each discovered skill a risk level (High, Medium, Low, Minimal) by analyzing the skill’s instructions for risky patterns — prompt-injection phrasing, data-exfiltration, or unsafe-automation language. The risk level is a heuristic signal for triage, not a verdict that the skill is malicious. A legitimate internal skill can score High or Medium if its instructions resemble those patterns (for example, a sanctioned skill that reads files and posts to an API).What to do:
  1. Open the skill’s row in Shadow → Skills to read the analyzed instructions and see why it was flagged.
  2. If the skill is sanctioned, bring it under management by publishing it through Runlayer — managed skills are classified Managed instead of Shadow. See Responding to Discoveries.
  3. After the skill’s content or management status changes, re-analyze to refresh its classification and risk level.
Detect does not block skills — risk levels are for inventory and review only. Blocking applies to shadow MCP sources and local tool calls via Enforce.

​
Package Issues

Quick fixes:
  1. Confirm the package is installed:
  2. Confirm managed configuration is present:
  3. Trigger one scan manually from the device:
  4. Confirm network connectivity from the device to your Runlayer host.
Quick fixes:
  1. Confirm the bundled LaunchAgent exists:
  2. Check whether launchd loaded it for the logged-in user:
  3. Kickstart it once:
  4. Inspect the unified log:
Cause: The AI Watch Configuration Profile has not synced yet, or it did not write Host / OrgApiKey into managed preferences.Resolution:
  1. Check managed preferences:
  2. If the file is empty or missing, force an MDM sync from the device or wait for the next check-in.
  3. Confirm the profile targets the same device group as the .pkg.
Cause: The LaunchAgent has not loaded, no scan has run yet, or unified log history has already rolled off.Resolution:
  1. Check whether launchd loaded the agent for the logged-in user:
  2. If launchd returns Could not find service, re-deploy the .pkg; the postinstall may not have bootstrapped the agent.
  3. If the agent is loaded, wait one StartInterval tick (default 15 minutes) or seed output with:
  4. For durable logs, collect unified logs with log collect or forward them to your SIEM. macOS retains unified logs for roughly 24 hours to a few days depending on device log volume.
Cause: The Login Items profile has not applied yet, or the device is running macOS older than 13 where the payload is ignored harmlessly.Resolution:
  1. Confirm com.runlayer.aiwatch.loginitems.mobileconfig is deployed to the device.
  2. After MDM sync, verify Background Task Management state:
  3. Expect the agent to show Disposition: enabled allowed visible.
Cause: The .pkg is not signed, not notarized, or is not the official CI-built release artifact.Resolution:
  1. Verify the package signature and notarization:
  2. Expected output includes status: signed by a developer certificate issued by Apple and source=Notarized Developer ID.
  3. Official GitHub Release builds are signed and notarized by Anysource Inc. with Developer ID team AF2M8HC7A2.
Scans run from device-local Scheduled Tasks the MSI registers — there are no Intune Remediations to check.Quick fixes:
  1. Confirm the app installed aiwatch.exe under C:\Program Files\Runlayer\AIWatch\.
  2. Confirm the hidden \Runlayer task folder and the AIWatchScan task exist:
    The task-registration action gates on the OrgApiKey registry value — if there are no tasks, confirm Get-ItemProperty "HKLM:\Software\Runlayer\AIWatch" shows OrgApiKey, then repair/reinstall the app.
  3. Check the AIWatchScan task’s last result and the per-profile scan outcomes. The single SYSTEM task runs aiwatch.exe scan --all-users and scans each real profile in a child process; a never-signed-in user has no profile yet and is picked up on the next tick (logon / boot / 15 min):
  4. Check the task-registration lines (tagged [register]) in C:\ProgramData\Runlayer\Logs\scheduled-task.log.
  5. Run a one-off scan as the logged-in user:
Quick fixes:
  1. Confirm Enforce or Sessions is enabled. If both are disabled, AI Watch is Detect-only and no hooks are installed.
  2. Check hook config compliance:
  3. On macOS, confirm the bootstrap daemon is loaded:
  4. On Windows, confirm the device-local AIWatchHooks task (SYSTEM) is running: Get-ScheduledTaskInfo -TaskPath "\Runlayer\" -TaskName "AIWatchHooks" | Select-Object LastRunTime, LastTaskResult. Force a run with Start-ScheduledTask -TaskPath "\Runlayer\" -TaskName "AIWatchHooks", or repair/reinstall the app if it’s missing.
  5. Restart Cursor, VS Code, Claude Code, Codex, or Hermes after hook configs change.
Quick fixes:
  1. Confirm Enforcement is enabled:
  2. Enforcement=false, AIWATCH_ENFORCEMENT=0, or an absent enforcement key is monitoring-only. Hooks may still forward Sessions telemetry, but they do not block.
  3. Re-push the configuration with Enforcement=true / AIWATCH_ENFORCEMENT=1.
  4. Restart the AI client so it reloads hook config.
Symptoms: The scan discovers local config, then fails while submitting results:
Cause: Some enterprise proxies intercept HTTPS and re-sign traffic with a private root CA. The Runlayer CLI and packaged aiwatch binary trust the OS trust store automatically, but scans and hooks can fail if that private CA is not installed in system trust.Resolution:
  1. Prefer installing the corporate root CA into the OS trust store through MDM.
  2. For debugging or one-off deployments, pass a PEM bundle explicitly:
  3. For custom CLI workflows, set RUNLAYER_CA_BUNDLE. The CLI also honors SSL_CERT_FILE and REQUESTS_CA_BUNDLE as fallbacks.
  4. Ask the security team to exempt the Runlayer tenant hostname from TLS inspection, such as *.runlayer.com or customer-managed hostnames like runlayer.dev.company.com just like they do for other security products like endpoint detection and response (EDR).
Cause: uv tool install runlayer places the binary in ~/.local/bin (macOS/Linux) or %USERPROFILE%\.local\bin (Windows), which may not be in your shell’s PATH yet.Resolution:
  1. Restart your terminal (new shells pick up the updated PATH)
  2. Or run uv tool update-shell to add ~/.local/bin to your shell profile, then exec $SHELL -l (or open a new terminal)
  3. Verify: which runlayer (macOS/Linux) or where runlayer (Windows)
Symptoms: Scan completes successfully but doesn’t find MCP servers you know exist in directories like Desktop, Documents, or Application Support.Cause: macOS TCC (Transparency, Consent, and Control) is blocking access to protected directories.Resolution:
  1. Deploy the AI Watch PPPC profile to grant Full Disk Access to /usr/local/lib/runlayer/aiwatch/aiwatch.
  2. Ensure devices are enrolled via User-Approved MDM or Automated Device Enrollment
  3. Verify the profile is installed: profiles show -type configuration | grep aiwatch
  4. Check TCC denials:
    Look for Prompting policy ... Deny lines. TCC PPPC payloads are ignored on manually enrolled MDM devices.
  5. Verify the package binary is installed and signed correctly:
  6. Re-run the scan after the PPPC profile is applied
Cause: The PPPC profile’s CodeRequirement does not match the installed binary. The profile pins identifier com.runlayer.aiwatch and Developer ID team AF2M8HC7A2 (Anysource Inc.). Custom, local, or ad-hoc signed builds do not satisfy that requirement, so TCC falls back to deny.Resolution:
  1. Verify the installed binary:
  2. Expected output:
  3. If the team identifier or identifier do not match, confirm with your Runlayer account team that the .pkg you deployed is the official signed and notarized build from the GitHub Release.
Cause: The device still has artifacts from the old script-based macOS Detect rollout, such as /usr/local/bin/runlayer-scan, com.runlayer.scan, or com.runlayer.scan.pppc.Resolution: Follow Legacy macOS Detect Cleanup. For full package removal, use Remove AI Watch.

​
Log Locations and Checks

​
Exit Codes

A scan that finds no MCP servers, skills, or plugins exits 0 (nothing to submit). Servers, skills, and plugins are submitted independently, so a failure in one category does not block the others; if any found item cannot be stored the scan exits nonzero (2 or 3). Best-effort check-ins (detect, enforce, sessions) never change the exit code. Any nonzero scan exit surfaces in the AIWatchScan task’s LastTaskResult (Windows) / the launchd job log (macOS), so a scan that ran but persisted nothing is not reported as a silent green.

​
Verifying PPPC Profile Deployment

After deploying the AI Watch PPPC profile, verify it is installed on target devices:

​
Verifying Package Binary

Verify the package binary is properly installed and signed:
The signature should show Identifier=com.runlayer.aiwatch, which matches the AI Watch PPPC profile.

User and Identity Mapping

Resolve device usernames to Runlayer users

FAQ

Pre-deployment, security-review, and operational questions

Remove AI Watch

Remove package-based and legacy script-based deployments

​
Getting Help

If you continue to experience issues:
  1. Collect the relevant log files from the affected device
  2. Note the HTTP status codes and error messages
  3. Contact Runlayer support with the collected information