Skip to main content
Detect scans devices for installed AI clients, MCP server configurations, skill artifacts, client-native agent definitions, and plugin artifacts. It is the always-on AI Watch feature: deploy AI Watch once, and Detect scans run from the package configuration. See User and Identity Mapping for how device usernames are attributed to Runlayer users, and Interpreting Discovery Results for how to verify a discovery.

​
How It Works

  1. Client and Artifact Discovery: The Runlayer CLI checks known installation signals, MCP configuration files, skill artifacts, client-native agent definitions, and installed plugin artifacts
  2. Runtime Process Discovery (optional): Detect polls the process table and listening sockets, then correlates running AI clients, MCP servers, and agents with configured artifacts
  3. Secure Submission: Client presence, configuration, skill, agent-definition, plugin, and redacted runtime metadata is securely submitted to Runlayer
  4. Classification: Runlayer classifies each discovery:
    • AI Clients are classified as:
      • Managed: Approved through your organization’s client allowlist
      • Shadow: Detected but not approved (flagged for review)
      • Risk: Client presence uses a Low baseline risk; review its device, user, version, and detection evidence for context
    • MCP Servers are classified as:
      • Managed: Running through Runlayer (approved and monitored)
      • Shadow: Configured outside Runlayer (flagged for review)
    • Skills are classified as:
      • Managed: Published and managed through Runlayer
      • Shadow: Installed outside organizational control (flagged for review)
      • Outdated: Previously managed but now out of date
    • Skill Risk Levels: Each discovered skill is assigned a risk level:
      • High: Skill contains risky instructions (prompt injection, data exfiltration patterns)
      • Medium: Skill has potentially risky characteristics that warrant review
      • Low: Skill has minor concerns but is unlikely to pose a threat
      • Minimal: Skill appears safe with no concerning patterns detected
  5. Review: Newly discovered shadow clients, servers, skills, agents, and plugins are flagged for review on the Shadow AI page

​
Security Team Workflow

  • Get visibility into supported AI clients, MCP servers, skills, and plugins across your organization
  • Identify which AI tools employees are using, even when no MCP server is configured
  • Build an inventory of shadow clients, integrations, and skills for risk assessment
  • Track trends in AI client, MCP, and skill adoption over time
When shadow MCP servers are discovered, evaluate:
  • Source: Is the MCP from a known vendor or unknown source?
  • Permissions: What data and systems can it access?
  • User context: Who configured it and for what purpose?
  • Network exposure: Does it connect to external endpoints?
When shadow skills are discovered, evaluate:
  • Risk level: Is the skill flagged as High or Medium risk?
  • Instructions: Does the skill contain prompts that could manipulate AI behavior?
  • Source: Is the skill from a trusted repository or an unknown source?
  • Scope: What actions does the skill instruct the AI to perform?
Based on risk assessment:
  • Low risk: Migrate to Runlayer-managed server or skill for visibility
  • Medium risk: Require user to submit for approval review
  • High risk: Take immediate action through MDM policy or direct intervention. If you suspect malicious activity, follow your incident response process, including credential rotation and forensic review as appropriate.

​
AI Client Detection

Detect inventories a supported AI client independently of its configured artifacts. A client can therefore appear in the Shadow AI inventory even when it has zero MCP servers, skills, or plugins. Detection evidence has three strengths:
  • Installed (app, cli, registry, npm_global, pip_global): an application bundle, executable, operating-system installation record, or validated allowlisted npm/Python package
  • Configuration (config): an exact known client configuration file
  • Weak trace (trace): a known parent or state directory exists, but no exact configuration file was found. Runlayer-created hook files are excluded so installing AI Watch hooks cannot create client presence by itself.
Depending on the operating system and client, installation signals include:
  • macOS: app bundles in /Applications or ~/Applications
  • Windows: uninstall-registry entries or known installation paths
  • Linux: binaries in known executable paths or desktop files
  • All platforms: known configuration or state paths
  • VS Code extensions: per-user installs plus built-in extensions from standard Visual Studio Code and Insiders installations and remote-server bundles. This includes GitHub Copilot as shipped with VS Code 1.116 and later; first-party VS Code platform extensions are excluded.
  • Global npm installs: known client packages installed globally through npm — including under custom or renamed npm prefixes and node_modules directories found by bounded project and hidden-space sweeps. Detect reads package metadata only — it never runs npm, Node, package scripts, or shims.
  • Python tool installs: known client packages installed as Python tools, identified by exact distribution metadata in active, pipx, uv tool, user-site, conventional system-site, and sweep-discovered virtual environments. Renaming the console script or relocating the environment does not change this identity. Detect never imports packages or runs Python tools.
  • Containers: client artifacts discovered inside running Docker or k3s containers. Known MCP images remain identifiable even if they are renamed locally.
  • Artifact evidence: a discovered MCP server, skill, agent definition, plugin, or extension also confirms that its client is present
Each Client row includes the available version and detection methods: app, cli, registry, npm_global, pip_global, container, config, trace, server, skill, plugin, or extension. Container and artifact methods (server, skill, plugin, extension) identify the client through discoveries made elsewhere in the same scan. New clients default to Shadow. Administrators can add an approved client to the organization allowlist from the Shadow AI Details tab or from the dedicated Settings → AI client allowlist page, marking it Managed and removing it from the shadow inventory. The AI client allowlist settings page provides a central view of all allowlisted clients and lets you add or remove clients without navigating through scan results.

​
Runtime process discovery

Runtime process discovery is an optional Detect channel, disabled by default. When enabled, each scan polls the local process table and listening sockets. It identifies supported AI clients, MCP server processes, and agent frameworks, including loopback MCP servers discovered from their listening ports. Detect correlates each runtime sighting with the artifacts discovered from static configuration. In Shadow AI → Details, the Liveness column and filter distinguish:
  • Running: a configured artifact was observed running during the selected date range
  • Configured only: configuration exists, but no running process was observed during the selected date range
  • Runtime only: a high-confidence running process has no matching configuration
Use Include managed to add managed artifacts to the configured-versus-running view. Runtime-only findings remain shadow artifacts for review. For managed fleets, enable Detect running processes from the deployment’s Manage settings dialog (Settings → AI Watch). macOS and Windows packages pick up the change from Runlayer without reinstalling. For a manual scan, pass --detect-processes. Process discovery is a point-in-time snapshot. It recognizes transient npx, uvx, pnpm dlx, and bunx MCP launchers while they are alive, but a command that starts and exits between scheduled scans is not observable. Renamed clients can still be recognized while running; installed extensions and home-relative MCP configurations remain the static backstop.

​
Supported Clients

For each OS, the tables below list the clients whose MCP, skill, and plugin artifacts Detect scans. Detect also reports the client itself when it finds an installation signal. A — means the client has no concept of that artifact type, so there is nothing to scan; see What — means below.

​
macOS

​
Windows

​
Linux

​
Presence-only clients

Detect also inventories these clients from installation signals even when no supported MCP configuration is present. Artifact support varies by client: Ollama, LM Studio, Jan, Cherry Studio, Msty, GPT4All, AnythingLLM, ChatWise, ChatGPT Desktop, Microsoft Copilot, Perplexity, Raycast, Replit Desktop, Continue, Amazon Q, Tabnine, Kiro, Trae, TraeWork, Qoder, Void, Aider, Amp, Crush, Droid, Qwen Code, OpenHands, Auggie, Grok CLI, Hermes, Smithery CLI, MCP Inspector, Everything MCP Server, Filesystem MCP Server, Memory MCP Server, Sequential Thinking MCP Server, Fetch MCP Server, Git MCP Server, Time MCP Server, and IntelliJ IDEA Community. Presence depends on an applicable signal on the device’s operating system. For example, OpenHands on Windows is detected through WSL.

​
What — means

A — indicates the client has no equivalent of that artifact type, so there is nothing for Detect to scan — it is not a gap in Detect coverage.
  • No skills convention (Gemini CLI, Antigravity): these clients do not define a SKILL.md directory layout. MCP servers and, where applicable, plugins/extensions are still scanned.
  • No plugin marketplace (Claude Desktop, Goose, Zed, Cline, Antigravity, Warp, Pi Coding Agent, Kilo Code): these clients do not have a separate plugin/extension format that bundles MCP servers, so there is no plugin layer beyond the MCP server config itself. For Goose and Zed, extensions are the MCP server and are surfaced through the MCP Servers column. Pi packages can ship extensions and skills but cannot declare MCP servers. Kilo Code marketplace entries are written straight into the MCP config files Detect already reads, so they surface through the MCP Servers column.
  • Known plugin gaps (Kimi Code, Devin CLI): Detect scans these clients’ MCP configuration and skills, but not their plugins or extensions.
  • Warp MCP servers are discovered from both Warp’s configuration file and servers added through Warp’s in-app gallery. Both feed the MCP Servers column; a server found through more than one source is counted once.
  • Kilo Code MCP servers are scanned from every config location Kilo reads at runtime — the current global (~/.config/kilo/) and project (kilo.json / kilo.jsonc / .kilo/) files, plus the older VS Code extension-store settings file, which Kilo still loads. All feed the MCP Servers column.
  • Pi Coding Agent has no built-in MCP support. Its mcp.json files are read by the community pi-mcp-adapter extension, so they only exist once a user installs it. Detect scans them wherever they appear.
  • Devin CLI is a separate surface from Devin Desktop. The CLI keeps servers in ~/.config/devin/mcp_config.json, while Devin Desktop (the renamed Windsurf) keeps them in ~/.codeium/windsurf/mcp_config.json and is reported under Windsurf. One device can legitimately show both.

​
Shadow Skills Discovery

Detect discovers skill artifacts alongside MCP server configurations in the same scan. Each discovered skill is classified and assigned a risk level based on its content. Skills found in a client-specific directory (such as ~/.claude/skills or ~/.codex/skills) are attributed to that client. Skills found in the shared .agents/skills directory are readable by several clients at once (VS Code, Goose, Zed, OpenCode, and others), so they are reported as shared rather than attributed to a single client.

​
Disguised skills in caches

Disguised-skill detection is off by default. For managed fleets, open Settings → AI Watch, select Manage settings for the deployment, and enable Detect disguised skills. For a manual scan, pass:
This phase checks selected browser and application caches, plus a bounded sweep of hidden home, cache, application-data, and temporary directories, for valid skill files. A file only counts when its content is a valid skill or matches a trusted hash — directory names alone never produce a finding. The scan is strictly limited by depth, file count, and total data read. Cache findings are deduplicated with other results. When a skill exists only in a cache, its Tool is shown as Browser cache. Enabling this option adds only the targeted cache and hidden-space check; it does not broaden the rest of the project scan. To turn it off, disable Detect disguised skills in the same dialog. Connected devices apply the change on their next settings sync. For manual scans, omit --detect-disguised-skills.

​
Skill Classification

​
Risk Levels

Skills flagged as High or Medium risk warrant immediate attention from security teams.

​
Shadow Plugins Discovery

Detect also discovers installed plugin artifacts alongside MCP servers and skills. Discovered plugins are classified using the same managed/shadow model: The Shadow AI page and exports surface only recognized AI marketplace extensions. Unrelated marketplace extensions may still be scanned and stored, but are excluded from these results even when Include managed is enabled. Plugin discoveries appear on the Shadow AI page alongside MCP server and skill data. Re-analysis also covers plugins — reclassifying them when catalog data changes.

​
Agent Discovery

Detect surfaces two agent-related artifact types on the Shadow AI Details tab:
  • Custom agent — a client-native agent definition file. Detect inventories these for Claude Code, Cursor, Codex, Gemini CLI, GitHub Copilot, OpenCode, and Goose. The same definition found in several places appears once, with each device, path, client, and container location visible as a separate sighting. Definition contents are parsed locally; Runlayer receives only bounded metadata, never the raw file.
  • Agent — an agent framework observed on the device, for example through runtime process discovery. Use the Type filter on the Details tab to review these alongside clients, MCP servers, skills, and plugins.

​
Run Detect

TLS trust: The runlayer CLI and MDM-deployed aiwatch binary verify HTTPS through the OS trust store — macOS Keychain (login + System), Windows certificate stores (Current User + Local Machine), and the Linux distro CA bundle (/etc/ssl/certs). Corporate root CAs pushed via MDM, GPO, or Intune are honored automatically with no extra config. If your corporate root is not installed system-wide and scans fail with CERTIFICATE_VERIFY_FAILED, see Troubleshooting for the CA bundle overrides.
For managed fleets, use Deploy AI Watch. Detect is always enabled in the AI Watch package; the MDM configuration wizard controls endpoint Mode and Sessions. This page keeps manual testing on the Runlayer CLI path.

​
Manual installation

For testing, one-off verification, or custom workflows, install the CLI and run a scan directly. Step 1 — Install the Runlayer CLI:
Restart your terminal after installing so ~/.local/bin (macOS/Linux) or %USERPROFILE%\.local\bin (Windows) is on PATH. Step 2 — Verify the CLI is on PATH:
If you see command not found, restart your terminal or run uv tool update-shell. See Troubleshooting for more details. Step 3 — Authenticate: You can authenticate interactively or with an organization API key.
Opens a browser for authentication. Credentials are stored in your OS keychain when available, with a fallback to ~/.runlayer/config.yaml.
Scans submitted with interactive login credentials will not be associated with any of the configuration cards on the Settings → MDM configuration page. Use an organization API key (Option B) if you need scan data linked to a specific configuration.
Step 4 — Run a scan:
The scan discovers installed AI clients, MCP server configurations, skill artifacts, agent definitions, and installed plugin artifacts in a single pass. Add --detect-processes to poll running processes and listening sockets; this channel is off by default. Add --detect-containers to scan MCP server configurations, skill artifacts, and agent definitions inside running Docker containers and k3s containers (on Linux, requires root), inventory running and stopped Docker containers plus local Docker images, and scan WSL projects on Windows. Add --detect-disguised-skills to run the default-off, explicit-cache probe described above.

​
Custom scan scheduling

Use these modular components when you need custom scheduling outside the managed AI Watch package. Install the CLI as a tool (if not already installed):
Example: Cron Job (macOS/Linux)
Example: Task Scheduler (Windows)

​
What the scan reads (and what it doesn’t)

Security and privacy reviewers usually want to know exactly what touches disk during a scan. Detect is a metadata-first configuration scanner, not a content crawler. Where it looks
  • The crawl is confined to the signed-in user’s home directory ($HOME on macOS/Linux, %USERPROFILE% on Windows). It does not walk system directories or other users’ homes.
  • Project-level discovery walks home to a bounded depth — default --project-depth 7, with a --project-timeout ceiling of 60 seconds. Pass --no-projects to skip project-level scanning entirely and only read the known global client config paths.
  • The scan caps its own CPU and memory footprint. Defaults are generous, so a normal scan is unaffected; the caps are tunable per deployment — see the tuning table for the MDM keys.
  • High-noise and cache directories are skipped: .git, venv / .venv, __pycache__, dist, build, target, vendor, Library/Caches, Library/Application Support, AppData, .Trash, tmp, .cache, .npm, .yarn, and plugin-install caches. Detect records bounded node_modules directory locations for npm package validation but does not descend into their dependency trees. The default-off disguised-skill phase is a narrow exception for its explicit allowlisted cache subtrees only.
  • Symlinks are not followed out of the home tree. On Windows, the SYSTEM all-users scan additionally drops reparse points and verifies each hit’s real path stays inside the profile.
When runtime process discovery is enabled
  • Detect reads process-table and local listening-socket metadata for classification and correlation. It does not read process memory or environment variables, capture packets, inspect network payloads, or proxy traffic.
  • Command arguments are scrubbed of credentials, secret-looking tokens, and usernames before submission, and bounded in length. The complete unredacted command line is retained only as a one-way hash used for correlation.
  • The process working directory is reduced to its scrubbed final project-directory name. For agent correlation, Detect may also submit a sanitized agent root path that preserves path shape while redacting home-directory usernames, URL credentials, and secret-looking segments. Raw command arguments and unredacted working directories are never submitted to Runlayer.
What it submits
  • AI clients: the normalized client identifier, display name, version when available, detection methods, and matched configuration or state paths
  • MCP servers: parsed configuration metadata only — the server command / args / url, the client name, and any project path where the config was found. Raw config files are not uploaded.
  • Skills and plugins: metadata plus bounded artifact text so Runlayer can risk-classify the skill or plugin. Text collection is capped at 1 MB per file and 5 MB per artifact; oversized files are skipped. Artifacts are deduplicated by fingerprint — once an artifact’s fingerprint is already known, later devices submit only the per-device installation record, not the file text again.
  • Agent definitions: parsed name and description, client, scope, redacted path, project path, and content hash. Raw definition content is not uploaded.
  • Runtime processes (when enabled): process kind, redacted executable and arguments, correlation hashes, listening ports and bind scope, confidence and detection source, a scrubbed project-directory name, and for agents a sanitized root path when available
Scope is defined by the home-directory boundary, not by a drive-type filter. Paths outside the home directory are never traversed, so a network share or removable drive mounted elsewhere on the system is not crawled — but there is no separate exclusion that skips a drive mounted under the home directory.

​
Viewing Results

After a scan, view discovered clients, servers, skills, agents, and plugins in the Runlayer dashboard:
  1. Navigate to Shadow AI
  2. Open the Details tab and use the Type filter to narrow to Clients, MCP servers, Skills, Plugins, Agents, or Custom agents. Each row identifies the artifact, user, and device. Open a row to review versions and detection evidence, or add a client to your organization’s allowlist.
  3. The Overview tab shows discovery trends, including the shadow vs. managed breakdown over time
  4. The Activity tab shows a filterable log of AI Watch activity, such as hook events and observed MCP tool calls
  5. The Connectors tab shows the most common shadow servers, who runs them, and which MCP clients are in use
  6. The Skills tab shows shadow skills by risk level, classification breakdown (Shadow, Managed, Outdated), and discovery trends over time
  7. The Devices tab shows AI Watch fleet mode and health, Sessions health, OS versions, and tool versions across enrolled devices

​
User Attribution

Detect resolves device usernames to Runlayer users so discoveries appear under the right owner. Some usernames remain unresolved because of shared machines, contractor naming, directory gaps, or ambiguous matches. See User and Identity Mapping for the full resolution flow and the Settings → User mapping workflow.

Endpoint modes

Monitor activity, apply Protect scanner and denylist decisions, or use Enforce deny-by-default governance

Deploy AI Watch

Install the package and configure endpoint Mode / Sessions

User and Identity Mapping

Resolve device usernames to Runlayer users

Responding to Discoveries

Security team response framework

Re-analyzing Classifications

Refresh server and skill classifications after changes

Troubleshooting

Common issues and solutions

FAQ

Pre-deployment, security-review, and operational questions