Agent Task Queue

Local task queuing for AI agents. Prevents multiple agents from running expensive operations concurrently and thrashing your machine.

The Problem

When multiple AI agents work on the same machine, they independently trigger expensive operations. Running these concurrently causes:

• 5-minute builds stretching to 30+ minutes
• Memory thrashing and disk I/O saturation
• Machine unresponsiveness
• Agents unable to coordinate with each other

How It Works

Default: Global queue - All run_task calls share one queue.

# Agent A runs:
run_task("./gradlew test", working_directory="/project")

# Agent B runs (waits for A to finish, then executes):
run_task("./gradlew build", working_directory="/project")

Custom queues - Use queue_name to isolate workloads:

# These run in separate queues (can run in parallel):
run_task("./gradlew build", queue_name="android", ...)
run_task("npm run build", queue_name="web", ...)

Both agents block until their respective builds complete. The server handles sequencing automatically.

Demo: Two Agents, One Build Queue

Terminal A - First agent requests an Android build:

> Build the Android app

⏺ agent-task-queue - run_task (MCP)
  command: "./gradlew assembleDebug"
  working_directory: "/path/to/android-project"

  ⎿  "SUCCESS exit=0 192.6s output=/tmp/agent-task-queue/output/task_1.log"

⏺ Build completed successfully in 192.6s.

Terminal B - Second agent requests the same build (started 2 seconds after A):

> Build the Android app

⏺ agent-task-queue - run_task (MCP)
  command: "./gradlew assembleDebug"
  working_directory: "/path/to/android-project"

  ⎿  "SUCCESS exit=0 32.6s output=/tmp/agent-task-queue/output/task_2.log"

⏺ Build completed successfully in 32.6s.

What happened behind the scenes:

Time	Agent A	Agent B
0:00	Started build
0:02	Building...	Entered queue, waiting
3:12	Completed (192.6s)	Started build
3:45		Completed (32.6s)

Why this matters:

Without the queue, both builds would run simultaneously—fighting for CPU, memory, and disk I/O. Each build might take 5+ minutes, and your machine would be unresponsive.

With the queue:

• Agent B automatically waited for Agent A to finish
• Agent B's build was 6x faster (32s vs 193s) because Gradle reused cached artifacts
• Total time: 3:45 instead of 10+ minutes of thrashing
• Your machine stayed responsive throughout

Key Features

• FIFO Queuing: Strict first-in-first-out ordering
• No Queue Timeouts: MCP keeps connection alive while waiting in queue. The timeout_seconds parameter only applies to execution time—tasks can wait in queue indefinitely without timing out. (see Why MCP?)
• Environment Variables: Pass env_vars="ANDROID_SERIAL=emulator-5560"
• Multiple Queues: Isolate different workloads with queue_name
• Zombie Protection: Detects dead processes, kills orphans, clears stale locks
• Auto-Kill: Tasks running > 120 minutes are terminated

Installation

uvx agent-task-queue@latest

That's it. uvx runs the package directly from PyPI—no clone, no install, no virtual environment.

Agent Configuration

Agent Task Queue works with any AI coding tool that supports MCP. Add this config to your MCP client:

{
  "mcpServers": {
    "agent-task-queue": {
      "command": "uvx",
      "args": ["agent-task-queue@latest"]
    }
  }
}

MCP Client Configuration

Amp

Install via CLI:

amp mcp add agent-task-queue -- uvx agent-task-queue@latest

Or add to .amp/settings.json (workspace) or global settings. See Amp Manual for details.

Claude Code

Install via CLI (guide):

claude mcp add agent-task-queue -- uvx agent-task-queue@latest

Claude Desktop

Config file locations:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Linux: ~/.config/Claude/claude_desktop_config.json

Use the standard config above.

Cline

Open the MCP Servers panel > Configure > "Configure MCP Servers" to edit cline_mcp_settings.json. Use the standard config above.

See Cline MCP docs for details.

Copilot / VS Code

Requires VS Code 1.102+ with GitHub Copilot Chat extension.

Config file locations:

• Workspace: .vscode/mcp.json
• Global: Via Command Palette > "MCP: Open User Configuration"

{
  "servers": {
    "agent-task-queue": {
      "type": "stdio",
      "command": "uvx",
      "args": ["agent-task-queue@latest"]
    }
  }
}

See VS Code MCP docs for details.

Cursor

Go to Cursor Settings > MCP > + Add new global MCP server. Use the standard config above.

Config file locations:

• Global: ~/.cursor/mcp.json
• Project: .cursor/mcp.json

See Cursor MCP docs for details.

Firebender

Add to firebender.json in project root, or use Plugin Settings > MCP section. Use the standard config above.

See Firebender MCP docs for details.

Windsurf

Config file location: ~/.codeium/windsurf/mcp_config.json

Or use Windsurf Settings > Cascade > Manage MCPs. Use the standard config above.

See Windsurf MCP docs for details.

Usage

Agents use the run_task MCP tool for expensive operations:

Build Tools: gradle, bazel, make, cmake, mvn, cargo build, go build, npm/yarn/pnpm build

Container Operations: docker build, docker-compose, podman, kubectl, helm

Test Suites: pytest, jest, mocha, rspec

Note: Some agents automatically prefer MCP tools (Amp, Copilot, Windsurf). Others may need configuration to prefer run_task over built-in shell commands.

Tool Parameters

Parameter	Required	Description
command	Yes	Shell command to execute
working_directory	Yes	Absolute path to run from
queue_name	No	Queue identifier (default: "global")
timeout_seconds	No	Max execution time before kill (default: 1200). Queue wait time doesn't count.
env_vars	No	Environment variables: "KEY=val,KEY2=val2"

Example

run_task(
    command="./gradlew connectedAndroidTest",
    working_directory="/project",
    queue_name="android",
    env_vars="ANDROID_SERIAL=emulator-5560"
)

Agent Configuration Notes

Some agents need additional configuration to use the queue instead of built-in shell commands.

Agent	Extra Setup	Notes
Amp, Copilot, Windsurf	❌ None	Works out of the box
Claude Code, Cursor	✅ Required	Must remove Bash allowed rules
Cline, Firebender	⚠️ Maybe	Check agent docs

Important

Claude Code users: If you have allowed rules like Bash(gradle:*) or Bash(./gradlew:*), the agent will use Bash directly and bypass the queue entirely. You must remove these rules for the queue to work.

Check both settings.json and settings.local.json (project and global) for rules like:

• Bash(gradle:*), Bash(./gradlew:*), Bash(ANDROID_SERIAL=* ./gradlew:*)
• Bash(docker build:*), Bash(pytest:*), etc.

See Claude Code setup guide for the full fix.

Quick Agent Setup

After installing the MCP server, tell your agent:

"Configure agent-task-queue - use examples/<agent-name>/SETUP.md if available"

Available setup guides:

• Claude Code setup - 3-step configuration
• Other agents - Contributions welcome!

Configuration

The server supports the following command-line options:

Option	Default	Description
--data-dir	/tmp/agent-task-queue	Directory for database and logs
--max-log-size	5	Max metrics log size in MB before rotation
--max-output-files	50	Number of task output files to retain
--tail-lines	50	Lines of output to include on failure
--lock-timeout	120	Minutes before stale locks are cleared

Pass options via the args property in your MCP config:

{
  "mcpServers": {
    "agent-task-queue": {
      "command": "uvx",
      "args": [
        "agent-task-queue@latest",
        "--max-output-files=100",
        "--lock-timeout=60"
      ]
    }
  }
}

Run uvx agent-task-queue@latest --help to see all options.

Architecture

flowchart TD
    A[AI Agent<br/>Claude, Cursor, Windsurf, etc.] -->|MCP Protocol| B[task_queue.py<br/>FastMCP Server]
    B -->|Query/Update| C[(SQLite Queue<br/>/tmp/agent-task-queue/queue.db)]
    B -->|Execute| D[Subprocess<br/>gradle, docker, etc.]

    D -.->|stdout/stderr| B
    B -.->|blocks until complete| A

Loading

Data Directory

All data is stored in /tmp/agent-task-queue/ by default:

• queue.db - SQLite database for queue state
• agent-task-queue-logs.json - JSON metrics log (NDJSON format)

To use a different location, pass --data-dir=/path/to/data or set the TASK_QUEUE_DATA_DIR environment variable.

Database Schema

The queue state is stored in SQLite at /tmp/agent-task-queue/queue.db:

Column	Type	Description
id	INTEGER	Auto-incrementing primary key
queue_name	TEXT	Queue identifier (e.g., "global", "android")
status	TEXT	Task state: "waiting" or "running"
pid	INTEGER	MCP server process ID (for liveness check)
child_pid	INTEGER	Subprocess ID (for orphan cleanup)
created_at	TIMESTAMP	When task was queued
updated_at	TIMESTAMP	Last status change

Zombie Protection

If an agent crashes while a task is running:

1. The next task detects the dead parent process (via PID check)
2. It kills any orphaned child process (the actual build)
3. It clears the stale lock
4. Execution continues normally

Metrics Logging

All queue events are logged to agent-task-queue-logs.json in NDJSON format (one JSON object per line):

{"event":"task_queued","timestamp":"2025-12-12T16:01:34","task_id":8,"queue_name":"global","pid":23819}
{"event":"task_started","timestamp":"2025-12-12T16:01:34","task_id":8,"queue_name":"global","wait_time_seconds":0.0}
{"event":"task_completed","timestamp":"2025-12-12T16:02:05","task_id":8,"queue_name":"global","command":"./gradlew build","exit_code":0,"duration_seconds":31.2,"stdout_lines":45,"stderr_lines":2}

Events logged:

• task_queued - Task entered the queue
• task_started - Task acquired lock and began execution
• task_completed - Task finished (includes exit code and duration)
• task_timeout - Task killed after timeout
• task_error - Task failed with exception
• zombie_cleared - Stale lock was cleaned up

The log file rotates when it exceeds 5MB (keeps one backup as .json.1).

Task Output Logs

To reduce token usage, full command output is written to files instead of returned directly:

/tmp/agent-task-queue/output/
├── task_1.log
├── task_2.log
└── ...

On success, the tool returns a single line:

SUCCESS exit=0 31.2s output=/tmp/agent-task-queue/output/task_8.log

On failure, the last 50 lines of output are included:

FAILED exit=1 12.5s output=/tmp/agent-task-queue/output/task_9.log
[error output here]

Automatic cleanup: Old files are deleted when count exceeds 50 (configurable via MAX_OUTPUT_FILES).

Manual cleanup: Use the clear_task_logs tool to delete all output files.

CLI Tool

The atq command lets you inspect the queue.

Install CLI

uv tool install agent-task-queue

This installs both the MCP server and the atq CLI persistently.

Usage

atq list              # Show current queue
atq logs              # Show recent activity
atq logs -n 50        # Show last 50 entries
atq clear             # Clear stuck tasks
atq --data-dir PATH   # Use custom data directory

Respects TASK_QUEUE_DATA_DIR environment variable.

Note: Without installing, you can run one-off commands with:

uvx --from agent-task-queue atq list

Troubleshooting

Tasks stuck in queue

atq list    # Check queue status
atq clear   # Clear all tasks

"Database is locked" errors

ps aux | grep task_queue                  # Check for zombie processes
rm -rf /tmp/agent-task-queue/             # Delete and restart

Server not connecting

1. Ensure uvx is in your PATH (install uv if needed)
2. Test manually: uvx agent-task-queue@latest

Development

For contributors:

git clone https://github.com/block/agent-task-queue.git
cd agent-task-queue
uv sync                      # Install dependencies
uv run pytest -v             # Run tests
uv run python task_queue.py  # Run server locally

Platform Support

• macOS
• Linux

Why MCP Instead of a CLI Tool?

The first attempt at solving this problem was a file-based queue CLI that wrapped commands:

queue-cli ./gradlew build

The fatal flaw: AI tools have built-in shell timeouts (30s-120s). If a job waited in queue longer than the timeout, the agent gave up—even though the job would eventually run.

flowchart LR
    subgraph cli [CLI Approach]
        A1[Agent] --> B1[Shell]
        B1 --> C1[CLI]
        C1 --> D1[Queue]
        B1 -.-> |"⏱️ TIMEOUT!"| A1
    end

    subgraph mcp [MCP Approach]
        A2[Agent] --> |MCP Protocol| B2[Server]
        B2 --> C2[Queue]
        B2 -.-> |"✓ blocks until complete"| A2
    end

Loading

Why MCP solves this:

• The MCP server keeps the connection alive indefinitely
• The agent's tool call blocks until the task completes
• No timeout configuration needed—it "just works"
• The server manages the queue; the agent just waits

Aspect	CLI Wrapper	Agent Task Queue
Timeout handling	External workarounds	Solved by design
Queue storage	Filesystem	SQLite (WAL mode)
Integration	Wrap every command	Automatic tool selection
Agent compatibility	Varies by tool	Universal

License

Apache 2.0