Skip to main content

​
Authentication and Configuration

Package-based AI Watch uses one organization API key for scans, hooks, sessions, and managed updates on every supported platform. The installed aiwatch binary reads it from the macOS Configuration Profile (OrgApiKey), the Windows MSI property (AIWATCH_ORG_API_KEY), or Linux’s root-only /etc/runlayer/aiwatch/credentials file (RUNLAYER_API_KEY). Manual runlayer scan workflows can still use interactive login credentials or a stored organization API key. Legacy enrollment keys only apply to older non-package auto-provisioning flows. The AI Watch organization API key is tenant-scoped: it is not tied to a user, cannot impersonate one, and does not grant dashboard or policy access. In Runlayer it is stored only as a one-way hash — the plaintext is shown once at creation and cannot be retrieved later, so if it is lost, regenerate the configuration to mint a new one.

​
Built-in diagnostic commands

AI Watch packages ship a small set of operator commands that cover most on-device checks. Use the installed binary path: /usr/local/bin/aiwatch on macOS, C:\Program Files\Runlayer\AIWatch\aiwatch.exe on Windows. aiwatch config show is the fastest way to answer “what settings is this device actually using?” — it shows the effective Mode, Sessions, and detection options after the settings sync, plus the tenant host and a redacted key suffix.

​
Interpreting Discovery Results

Cause: Detect classifies based on MCP config files found on disk, not actual tool calls. A user being attributed to a shadow MCP means a matching config was found on a device whose login username resolved to that user — not that they ever invoked any tool from the server.Common reasons a config gets attributed to a user who never used it:
  • Project-level config in a cloned repo. A .cursor/mcp.json, .vscode/mcp.json, .mcp.json, .codex/config.toml, or similar checked into a shared repo is discovered on every developer who clones it. The config is attributed to the OS user who ran the scan on that machine.
  • Stale or experimental config. The user added the server to a client config file once (e.g. trying it out, copy-pasting from a blog), never invoked it, and forgot it was still there.
  • Shared device. Another OS account on the same workstation configured the MCP, but the scan ran under a different OS account that resolves to this Runlayer user.
  • Username resolution mismatch. Username matching can link a device’s OS username to the wrong Runlayer user. See User and Identity Mapping, then fix via Settings → User mapping and re-analyze.
How the user (or admin) can verify the source themselves:
  1. Click the shadow artifact row on the Shadow AI page’s Details tab to open the detail drawer.
  2. The detail includes the device hostname, device id, the MCP server’s command / args / url, and any project paths where the config was found.
  3. If a project path is listed, the discovery came from a project-level config file. Open that repo and inspect the checked-in MCP config to confirm.
  4. If no project path is listed, the discovery came from a global client config. On the listed device, inspect the relevant client config (e.g. ~/Library/Application Support/Claude/claude_desktop_config.json, ~/.cursor/mcp.json, ~/.codex/config.toml).
  5. To clear the discovery, remove the entry from the offending config file (or uninstall the project) and the next scan will drop it from the inventory.
Cause: Detect assigns each discovered skill a risk level (High, Medium, Low, Minimal) by analyzing the skill’s instructions for risky patterns — prompt-injection phrasing, data-exfiltration, or unsafe-automation language. The risk level is a heuristic signal for triage, not a verdict that the skill is malicious. A legitimate internal skill can score High or Medium if its instructions resemble those patterns (for example, a sanctioned skill that reads files and posts to an API).What to do:
  1. Open the skill’s row in Shadow AI → Skills to read the analyzed instructions and see why it was flagged.
  2. If the skill is sanctioned, bring it under management by publishing it through Runlayer — managed skills are classified Managed instead of Shadow. See Responding to Discoveries.
  3. After the skill’s content or management status changes, re-analyze to refresh its classification and risk level.
Detect does not block skills — risk levels are for inventory and review only. Scanner Block/Mask actions and source governance apply to tool calls through Protect or Enforce.

​
Package Issues

Quick fixes:
  1. Confirm the package is installed:
  2. Confirm managed configuration is present:
  3. Trigger one scan manually from the device:
  4. Confirm network connectivity from the device to your Runlayer host.
Quick fixes:
  1. Confirm the bundled LaunchAgent exists:
  2. Check whether launchd loaded it for the logged-in user:
  3. Kickstart it once:
  4. Inspect the unified log:
Cause: The AI Watch Configuration Profile has not synced yet, or it did not write Host / OrgApiKey into managed preferences.Resolution:
  1. Check managed preferences:
  2. If the file is empty or missing, force an MDM sync from the device or wait for the next check-in.
  3. Confirm the profile targets the same device group as the .pkg.
Cause: The LaunchAgent has not loaded, no scan has run yet, or unified log history has already rolled off.Resolution:
  1. Check whether launchd loaded the agent for the logged-in user:
  2. If launchd returns Could not find service, re-deploy the .pkg; the postinstall may not have bootstrapped the agent.
  3. If the agent is loaded, wait one StartInterval tick (default 15 minutes) or seed output with:
  4. For durable logs, collect unified logs with log collect or forward them to your SIEM. macOS retains unified logs for roughly 24 hours to a few days depending on device log volume.
Cause: The Login Items payload has not applied yet, or the device is running macOS older than 13 where the payload is ignored harmlessly.Resolution:
  1. Confirm com.runlayer.aiwatch.mobileconfig (which carries the Login Items payload; com.runlayer.aiwatch.loginitems.mobileconfig on fleets still running the older split profiles) is deployed to the device.
  2. After MDM sync, verify Background Task Management state:
  3. Expect the agent to show Disposition: enabled allowed visible.
Cause: The .pkg is not signed, not notarized, or is not the official signed AI Watch package from Runlayer.Resolution:
  1. Verify the package signature and notarization:
  2. Expected output includes status: signed by a developer certificate issued by Apple and source=Notarized Developer ID.
  3. Official packages downloaded from Settings → MDM configuration are signed and notarized by Anysource Inc. with Developer ID team AF2M8HC7A2.
Scans run from device-local Scheduled Tasks the MSI registers — there are no Intune Remediations to check.Quick fixes:
  1. Confirm the app installed aiwatch.exe under C:\Program Files\Runlayer\AIWatch\.
  2. Confirm the hidden \Runlayer task folder and the AIWatchScan task exist:
    The task-registration action gates on the OrgApiKey registry value — if there are no tasks, confirm Get-ItemProperty "HKLM:\Software\Runlayer\AIWatch" shows OrgApiKey, then repair/reinstall the app.
  3. Check the AIWatchScan task’s last result and the per-profile scan outcomes. The single SYSTEM task runs aiwatch.exe scan --all-users and scans each real profile in a child process; a never-signed-in user has no profile yet and is picked up on the next tick (logon / boot / 15 min):
  4. Check the task-registration lines (tagged [register]) in C:\ProgramData\Runlayer\Logs\scheduled-task.log.
  5. Run a one-off scan as the logged-in user:
Quick fixes:
  1. Confirm Mode is Protect/Enforce, Sessions is enabled, or Track MCP tool usage without content is enabled. Monitor with Sessions disabled and that setting off is Detect-only and installs no hooks.
  2. Check hook config compliance:
  3. On macOS, confirm the bootstrap daemon is loaded:
  4. On Windows, confirm the device-local AIWatchHooks task (SYSTEM) is running: Get-ScheduledTaskInfo -TaskPath "\Runlayer\" -TaskName "AIWatchHooks" | Select-Object LastRunTime, LastTaskResult. Force a run with Start-ScheduledTask -TaskPath "\Runlayer\" -TaskName "AIWatchHooks", or repair/reinstall the app if it’s missing.
  5. Restart the AI client after hook configs change — Cursor, VS Code, Claude Code, GitHub Copilot CLI, Gemini CLI, Grok CLI, Cline CLI, Codex, Qwen Code, Hermes, Windsurf, and Goose all read hook configs only at process start.
Quick fixes:
  1. Confirm the deployment’s effective settings with aiwatch config show — it prints the resolved Mode, Sessions, and detection options, and whether a valid settings snapshot is cached. To inspect the raw snapshot instead:
  2. Mode=monitor is report-only. Protect applies scanner decisions plus fail-open, allow-unless-denied MCP source governance. Enforce adds local-tool controls and fail-closed, deny-by-default MCP source governance, while managed MCP calls scan at the gateway.
  3. Check whether the “missed” call was allowed by policy rather than missed by the hook:
    • The user is on the exemption list under Settings → Tool governance → Exempt users. Exempted users’ violations are still recorded and alerted, but the call proceeds.
    • The MCP server is managed through Runlayer or matches an allowlist entry, so source governance allows it by design.
    • For built-in tool blocking, the tool’s behavior type (shell/terminal, file read, file write) is not toggled on in the Enforce policy — types you have not enabled are allowed by default.
    • The matching scanner’s action is Alert rather than Block, which records a finding without stopping the call.
  4. Change Mode from Settings → AI Watch → Manage settings, then wait for the hourly hook reconcile or run aiwatch config sync to apply it immediately. Re-push MDM policy only when changing the fallback or another MDM-owned value.
  5. Restart the AI client so it reloads hook config.
Symptoms: Install browser extension is enabled in Manage settings → Browser extension, but the Runlayer extension is missing from chrome://extensions or about:addons.Resolution:
  1. The extension is installed by the hourly hook reconcile. Rerun it immediately as an administrator instead of waiting:
  2. Fully quit and reopen the browser — Chrome and Firefox apply managed extension policy at startup.
  3. Confirm the device runs AI Watch 0.29.6 or newer and appears in Shadow AI → Devices.
  4. See Browser extension for session-collection and blocking issues once the extension is installed.
Symptoms: The scan discovers local config, then fails while submitting results:
Cause: Some enterprise proxies intercept HTTPS and re-sign traffic with a private root CA. The Runlayer CLI and packaged aiwatch binary trust the OS trust store automatically, but scans and hooks can fail if that private CA is not installed in system trust.Resolution:
  1. Prefer installing the corporate root CA into the OS trust store through MDM.
  2. For debugging or one-off deployments, pass a PEM bundle explicitly:
  3. For custom CLI workflows, set RUNLAYER_CA_BUNDLE. The CLI also honors SSL_CERT_FILE and REQUESTS_CA_BUNDLE as fallbacks.
  4. Ask the security team to exempt the Runlayer tenant hostname from TLS inspection, such as *.runlayer.com or customer-managed hostnames like runlayer.dev.company.com just like they do for other security products like endpoint detection and response (EDR).
Cause: uv tool install runlayer places the binary in ~/.local/bin (macOS/Linux) or %USERPROFILE%\.local\bin (Windows), which may not be in your shell’s PATH yet.Resolution:
  1. Restart your terminal (new shells pick up the updated PATH)
  2. Or run uv tool update-shell to add ~/.local/bin to your shell profile, then exec $SHELL -l (or open a new terminal)
  3. Verify: which runlayer (macOS/Linux) or where runlayer (Windows)
Symptoms: Scan completes successfully but doesn’t find MCP servers you know exist in directories like Desktop, Documents, or Application Support.Cause: macOS TCC (Transparency, Consent, and Control) is blocking access to protected directories.Resolution:
  1. Deploy the AI Watch PPPC profile to grant Full Disk Access to /usr/local/lib/runlayer/aiwatch/aiwatch.
  2. Ensure devices are enrolled via User-Approved MDM or Automated Device Enrollment
  3. Verify the profile is installed: profiles show -type configuration | grep aiwatch
  4. Check TCC denials:
    Look for Prompting policy ... Deny lines. TCC PPPC payloads are ignored on manually enrolled MDM devices.
  5. Verify the package binary is installed and signed correctly:
  6. Re-run the scan after the PPPC profile is applied
Cause: The PPPC profile’s CodeRequirement does not match the installed binary. The profile pins identifier com.runlayer.aiwatch and Developer ID team AF2M8HC7A2 (Anysource Inc.). Custom, local, or ad-hoc signed builds do not satisfy that requirement, so TCC falls back to deny.Resolution:
  1. Verify the installed binary:
  2. Expected output:
  3. If the team identifier or identifier do not match, re-download the official signed and notarized .pkg from Settings → MDM configuration and re-deploy it. Contact your Runlayer account team if you suspect the deployed build was tampered with.
Cause: The device still has artifacts from the old script-based macOS Detect rollout, such as /usr/local/bin/runlayer-scan, com.runlayer.scan, or com.runlayer.scan.pppc.Resolution: Follow Legacy macOS Detect Cleanup. For full package removal, use Remove AI Watch.

​
Log Locations and Checks

​
Exit Codes

A scan that finds no MCP servers, skills, or plugins exits 0 (nothing to submit). Servers, skills, and plugins are submitted independently, so a failure in one category does not block the others; if any found item cannot be stored the scan exits nonzero (2 or 3). Any nonzero scan exit surfaces in the AIWatchScan task’s LastTaskResult (Windows) / the launchd job log (macOS), so a scan that ran but persisted nothing is not reported as a silent green.

​
Verifying PPPC Profile Deployment

After deploying the AI Watch PPPC profile, verify it is installed on target devices:

​
Verifying Package Binary

Verify the package binary is properly installed and signed:
The signature should show Identifier=com.runlayer.aiwatch, which matches the AI Watch PPPC profile.

User and Identity Mapping

Resolve device usernames to Runlayer users

FAQ

Pre-deployment, security-review, and operational questions

Remove AI Watch

Remove package-based and legacy script-based deployments

​
Getting Help

If you continue to experience issues:
  1. Collect the relevant log files from the affected device
  2. Note the HTTP status codes and error messages
  3. Contact Runlayer support with the collected information