Anthropic Skill Scanners Miss Test File Execution Risk
Anthropic Skill scanners from Cisco, Snyk, and others pass malicious code bundled in test files because they only inspect the agent execution surface, not the developer toolchain. Gecko Security researcher Jeevan Jutla demonstrated that when developers install Skills via npx Skills add, test files like .test.ts execute with full filesystem and credential access through standard JavaScript test runners, bypassing all public scanners. This attack vector sits outside every scanner's detection model despite two major audits documenting widespread vulnerabilities in Anthropic Skills marketplaces.
TL;DR
- Malicious .test.ts files in Anthropic Skills execute with full local permissions through Jest, Vitest, and Mocha test runners, but no public scanner inspects them
- Gecko Security demonstrated the attack flow: installed Skills land in shared directories, propagate to teammates, and sit outside scanner detection surfaces entirely
- Two large-scale audits found 26.1% of 31,132 Skills contained vulnerabilities and 13.4% of 3,984 Skills had critical-level issues, but neither measured test file execution risk
- Cisco's AI Agent Security Scanner, Snyk Agent Scan, and VirusTotal Code Insight all share the same structural blind spot, targeting agent interaction layers rather than developer toolchain layers
Why It Matters
This reveals a fundamental mismatch between threat model and detection scope in AI agent security. Scanners are optimized to catch prompt injection and agent-layer attacks, but the Skill installation and execution model creates a separate attack surface through developer tooling that sits completely outside their purview. As Anthropic Skills become more widely adopted across teams, this gap becomes a systemic risk.
Business Impact
Teams deploying Anthropic Skills face credential theft and supply chain compromise through test files that execute silently during npm test or IDE auto-run, with no warning from any major scanner. For operators managing shared Skill repositories, this means malicious code can propagate to every teammate who clones the repo, with full access to deployment tokens and cloud credentials. Skill marketplace operators and tool vendors need to either expand scanner scope or document this limitation explicitly.
Key Implications
- Current Anthropic Skill scanners measure the wrong execution surface, creating false confidence in marketplace safety despite documented vulnerabilities
- Test file execution represents a trust-on-install attack vector similar to npm postinstall scripts and pytest plugins, but with higher blast radius due to shared team directories
- Disclosure of this gap occurred after two major audits, suggesting scanners may have other blind spots not yet documented by security researchers
What to Watch
Monitor whether Anthropic, Cisco, Snyk, and other scanner vendors update their tools to inspect bundled test files and other non-agent execution surfaces. Watch for whether Skill marketplace operators implement additional vetting or sandboxing. Track whether this vulnerability class appears in real-world Skill supply chain incidents, which would validate the practical risk.
Subscribe to the newsletter
The latest stories and analysis, delivered to your inbox.
Free. No spam. Unsubscribe any time.
