SafeClaw Policy Recipe: Test Automation Agent

This policy is for AI agents that run automated tests — executing test suites, reading source code for analysis, writing test reports, and generating test coverage data. The agent can run approved test commands and read the entire codebase but cannot modify source files, access production resources, or make network requests. Install SafeClaw with npx @authensor/safeclaw and paste this into safeclaw.config.yaml.

Use Case

A test automation agent runs unit tests, integration tests, and end-to-end tests. It reads source code and test files to understand what to test, executes test runners (Jest, Vitest, pytest, etc.), and writes coverage reports. It may also generate new test cases. The risks: a test agent with unrestricted shell access could execute arbitrary commands beyond testing, modify production source code, access production databases or APIs through test configuration, or exfiltrate code via network requests. This policy constrains the agent to test-related operations only.

The Policy

# safeclaw.config.yaml — Test Automation Agent
For: AI test runners, test generation agents, CI test stages
Install: npx @authensor/safeclaw
version: "1.0"
agent: test-agent
defaultAction: deny

rules:
  # --- FILE READ RULES ---

# Block reading credential files
  - id: deny-read-env
    action: file_read
    target: "*/.env"
    decision: deny
    description: "Block reading environment files with production secrets"

# Block reading production config
  - id: deny-read-prod-config
    action: file_read
    target: "*/production*"
    decision: deny
    description: "Block reading production configuration files"

# Block reading SSH keys
  - id: deny-read-ssh
    action: file_read
    target: "~/.ssh/**"
    decision: deny
    description: "Block reading SSH credentials"

# Allow reading all source code (needed to understand what to test)
  - id: allow-read-src
    action: file_read
    target: "./src/**"
    decision: allow
    description: "Allow reading source code for test analysis"

# Allow reading all test files
  - id: allow-read-tests
    action: file_read
    target: "./tests/**"
    decision: allow
    description: "Allow reading existing test files"

# Allow reading spec files (alternative test directory)
  - id: allow-read-spec
    action: file_read
    target: "./*/.{spec,test}.{ts,js,tsx,jsx,py}"
    decision: allow
    description: "Allow reading co-located test/spec files"

# Allow reading project config (test config, tsconfig, etc.)
  - id: allow-read-config
    action: file_read
    target: "./*.{json,yaml,yml,toml,js,ts,cjs,mjs}"
    decision: allow
    description: "Allow reading project and test configuration"

# Allow reading test fixtures
  - id: allow-read-fixtures
    action: file_read
    target: "./fixtures/**"
    decision: allow
    description: "Allow reading test fixture data"

# Allow reading mocks
  - id: allow-read-mocks
    action: file_read
    target: "./__mocks__/**"
    decision: allow
    description: "Allow reading mock modules"

# --- FILE WRITE RULES ---

# Block writing to source code (test agent should not modify source)
  - id: deny-write-src
    action: file_write
    target: "./src/**"
    decision: deny
    description: "Block modifying source code — test agent is non-destructive"

# Block writing outside the project
  - id: deny-write-outside
    action: file_write
    target: "/**"
    decision: deny
    description: "Block writing to absolute paths outside project"

# Allow writing new test files
  - id: allow-write-tests
    action: file_write
    target: "./tests/**"
    decision: allow
    description: "Allow writing new or updated test files"

# Allow writing co-located test files
  - id: allow-write-spec
    action: file_write
    target: "./*/.{spec,test}.{ts,js,tsx,jsx,py}"
    decision: allow
    description: "Allow writing co-located spec/test files"

# Allow writing coverage reports
  - id: allow-write-coverage
    action: file_write
    target: "./coverage/**"
    decision: allow
    description: "Allow writing test coverage output"

# Allow writing test result reports
  - id: allow-write-reports
    action: file_write
    target: "./reports/**"
    decision: allow
    description: "Allow writing test result reports (JUnit XML, etc.)"

# Allow writing test snapshots
  - id: allow-write-snapshots
    action: file_write
    target: ".//__snapshots__/"
    decision: allow
    description: "Allow writing Jest/Vitest snapshot files"

# --- SHELL EXEC RULES ---

# Block destructive commands
  - id: deny-rm-rf
    action: shell_exec
    target: "rm -rf *"
    decision: deny
    description: "Block recursive force-delete commands"

# Block sudo
  - id: deny-sudo
    action: shell_exec
    target: "sudo *"
    decision: deny
    description: "Block privilege escalation"

# Block curl/wget (no network tools)
  - id: deny-curl
    action: shell_exec
    target: "curl *"
    decision: deny
    description: "Block curl commands — no network from shell"

- id: deny-wget
    action: shell_exec
    target: "wget *"
    decision: deny
    description: "Block wget commands — no network from shell"

# Allow npm/yarn/pnpm test
  - id: allow-npm-test
    action: shell_exec
    target: "{npm,yarn,pnpm} test*"
    decision: allow
    description: "Allow running test suites"

# Allow npx for test runners
  - id: allow-npx-test
    action: shell_exec
    target: "npx {jest,vitest,mocha,playwright,cypress}*"
    decision: allow
    description: "Allow running test frameworks via npx"

# Allow pytest
  - id: allow-pytest
    action: shell_exec
    target: "pytest*"
    decision: allow
    description: "Allow running pytest"

# Allow installing test dependencies
  - id: allow-install
    action: shell_exec
    target: "{npm,yarn,pnpm} install*"
    decision: allow
    description: "Allow installing dependencies for test environment"

# Gate any other shell command
  - id: gate-other-shell
    action: shell_exec
    target: "*"
    decision: require_approval
    description: "All other shell commands require human approval"

# --- NETWORK RULES ---

# Allow npm registry for dependency installs
  - id: allow-npm-registry
    action: network
    target: "https://registry.npmjs.org/*"
    decision: allow
    description: "Allow npm registry for package downloads"

# Block all other network access
  - id: deny-network-all
    action: network
    target: "*"
    decision: deny
    description: "Block all other network — tests run locally"

What This Policy Allows

• Reading all source code, test files, fixtures, mocks, and project config
• Running npm test, npx jest, npx vitest, pytest and other test commands
• Writing new test files and co-located spec files
• Writing coverage reports, test result reports, and snapshot files
• Installing dependencies via npm/yarn/pnpm
• Network access to npm registry for package installs

What This Policy Blocks

• Modifying source code in ./src/ (test agent is non-destructive)
• Reading .env files, production config, or SSH keys
• Running rm -rf, sudo, curl, or wget commands
• All network requests except the npm registry
• Writing to any path outside test, coverage, and report directories
• Accessing any file outside the project directory

What Requires Approval

• Any shell command not in the explicit allow list
• This catches unexpected commands the agent may attempt (e.g., docker, python scripts/setup.py)
• Approvals are managed through the SafeClaw dashboard at safeclaw.onrender.com

Customization Guide

1. Add your test framework. If you use Karma, Ava, or a custom test runner, add a shell_exec allow rule for that command (e.g., target: "npx ava*"). Place it above the gate-other-shell catch-all so first-match-wins evaluation picks it up.

1. Allow source code writes for test generation. If the agent generates test files alongside source (e.g., creating src/utils/helpers.test.ts next to src/utils/helpers.ts), change the deny-write-src rule to specifically block non-test writes, or remove it and rely on the co-located test write rule.

1. Allow test database access. If integration tests need a local database, add a network allow rule for localhost or 127.0.0.1 on your database port (e.g., target: "http://localhost:5432/*"). Never allow network access to production database hosts.

Example Session

1. ALLOW — Agent reads source code to understand a module:

{
  "actionType": "file_read",
  "target": "./src/utils/validation.ts",
  "agentId": "test-agent",
  "decision": "ALLOW",
  "rule": "allow-read-src",
  "evaluationTime": "0.3ms"
}

2. ALLOW — Agent runs the test suite:

{
  "actionType": "shell_exec",
  "target": "npx vitest run --coverage",
  "agentId": "test-agent",
  "decision": "ALLOW",
  "rule": "allow-npx-test",
  "evaluationTime": "0.2ms"
}

3. ALLOW — Agent writes a new test file:

{
  "actionType": "file_write",
  "target": "./tests/utils/validation.test.ts",
  "agentId": "test-agent",
  "decision": "ALLOW",
  "rule": "allow-write-tests",
  "evaluationTime": "0.3ms"
}

4. DENY — Agent attempts to modify source code:

{
  "actionType": "file_write",
  "target": "./src/utils/validation.ts",
  "agentId": "test-agent",
  "decision": "DENY",
  "rule": "deny-write-src",
  "evaluationTime": "0.2ms"
}

5. DENY — Agent attempts to curl an external endpoint:

{
  "actionType": "shell_exec",
  "target": "curl https://api.production.com/data",
  "agentId": "test-agent",
  "decision": "DENY",
  "rule": "deny-curl",
  "evaluationTime": "0.2ms"
}

All evaluations are logged to SafeClaw's tamper-proof audit trail (SHA-256 hash chain). The audit trail provides a complete record of every test the agent ran and every action it was denied. Use simulation mode to verify this policy matches your test workflow before switching to enforce mode. SafeClaw evaluates in sub-millisecond time, has zero third-party dependencies, and is backed by 446 tests in TypeScript strict mode. Free tier keys are available at safeclaw.onrender.com with 7-day renewal, no credit card required.

Cross-References

• SafeClaw Policy Rule Syntax Reference
• SafeClaw Test Coverage Reference
• GitHub Actions CI Agent Integration Guide
• Pattern: Simulation Before Enforcement
• Use Case: Security Audit Workflow