MCP Workflows & Troubleshooting

This page covers the async execution contract, common workflows, Claude Code skills, troubleshooting, and security considerations for the RTILA engine MCP server.

MCP async workflow showing the three-step poll loop: start, poll status, get result

The Async Contract

Every browser tool returns immediately with a run_id. Page analysis, project runs, and the builder all take minutes, and most MCP clients kill a tool call at about 60 seconds. The engine never blocks by default.

The Three-Step Pattern

A browser tool call is always three steps, never one:

Step 1: Start the work
   rtila_analyze_page  { "url": "https://example.com" }
   → { "run_id": "mcp_k3f9x_4b1c2d3e", "status": "running", "url": "..." }

Step 2: Poll until finished (~3 seconds between polls)
   rtila_get_run_status { "run_id": "mcp_k3f9x_4b1c2d3e" }
   → { "status": "running", ... }
   → { "status": "running", ... }
   → { "status": "succeeded", "exit_code": 0, ... }

Step 3: Get the result (analyze tools only)
   rtila_get_run_result { "run_id": "mcp_k3f9x_4b1c2d3e" }
   → { "result_available": true, "result": { ...report... } }

Rules

Rule	Details
Poll until `status` is no longer `"running"`	Status is one of: `running`, `succeeded`, `failed`. A graceful cancellation also reports `succeeded`.
`rtila_get_run_result` returns `result: null` for project runs	This is expected. A project run’s outcome is its status; its output is rows in the database, readable via `rtila_get_project_data`.
`wait: true` blocks and returns the payload inline	Only use it when you know the client’s tool-call deadline covers the work. Analysis typically takes 90–180 seconds.
Read `stderr_tail` on failure	A failed run’s `stderr_tail` carries the engine’s own error classification. Read it before guessing.
Run records are ephemeral	They live in the server process, survive across tool calls in one session, but not across a server restart. Only the 50 most recent finished runs are retained. Poll promptly.
`rtila_cancel_run` terminates the worker	A client timeout on a `wait: true` call does the same thing automatically.

Status Values

Status	Meaning
`running`	The worker process is still executing.
`succeeded`	The worker completed successfully (exit code 0). Also used for graceful cancellation.
`failed`	The worker encountered an error. Check `stderr_tail` for the engine’s classification.

Common Workflows

Workflow 1: Analyze a Page

Goal: Understand a page’s structure before building a scraper.

1. rtila_analyze_page { "url": "https://example.com/products" }
   → { "run_id": "mcp_abc123", "status": "running" }

2. rtila_get_run_status { "run_id": "mcp_abc123" }     ← poll every ~3s
   → { "status": "succeeded", "exit_code": 0 }

3. rtila_get_run_result { "run_id": "mcp_abc123" }
   → { "result_available": true, "result": { ...analysis report... } }

Reading the analysis report — answer four questions:

Question	Where to look
Is the page blocked by a WAF?	`analysis.features.is_blocked_by_waf`
Is the data in an API response?	`networkRequests` — look for `xhr`/`fetch` calls returning JSON
What is the repeating unit?	The item selector (product card, table row, etc.)
What are the fields?	The distinct values inside one repeating unit

Workflow 2: Run a Project

Goal: Execute an existing scraping project and check the results.

1. rtila_list_projects { "filter": "name~\"Amazon\"" }
   → Find the project ID

2. rtila_get_project_config { "project_id": "abc123", "summary_only": true }
   → "3 URLs, 12 commands, 1 dataset"

3. rtila_run_project { "project_id": "abc123", "session_id": "my-session" }
   → { "run_id": "mcp_xyz789", "status": "running" }

4. rtila_get_run_status { "run_id": "mcp_xyz789" }     ← poll every ~3s
   → { "status": "succeeded", "exit_code": 0 }

5. rtila_get_project_data { "dataset": "products", "project_id": "abc123", "per_page": 10 }
   → Read the scraped rows

⚠️ Never run a project the user did not ask you to run. A run visits real URLs with the user’s proxy and CAPTCHA credentials.

Workflow 3: Debug a Failed Run

Goal: Understand why a run failed.

1. rtila_get_run_status { "run_id": "mcp_xyz789" }
   → { "status": "failed", "stderr_tail": "SELECTOR_ERROR: ..." }

2. Read stderr_tail — it contains the engine's own classification.

3. If stderr_tail is not enough:
   rtila_get_run_log { "run_id": "mcp_xyz789", "tail_chars": 20000 }

4. If you still need more:
   rtila_get_logs { "session_id": "my-session", "level": "ERROR" }

Common error classifications in stderr_tail:

What you see	What it means	Fix
`Required variable '<name>' is missing`	A required project variable has no value	Pass it in `variables`, or set it as a global variable
`Proxy connection failed` / `ERR_PROXY_CONNECTION_FAILED` / `407`	The proxy failed the pre-flight check	Fix or rotate the proxy, or disable it for this run
`CAPTCHA_LOOP_DETECTED`	Stuck in a CAPTCHA loop	Change the proxy, or disable CAPTCHA solving
`CAPTCHA_ERROR: Missing or invalid 2Captcha API key`	CAPTCHA provider key is missing or invalid	Check your 2Captcha API key and balance
`SELECTOR_ERROR`	An extraction selector no longer matches	Re-analyze the page and repair the selector
`WAF_BLOCK_ERROR`	The target is blocking the browser	Enable stealth features (proxies, humanoid, fingerprint alignment)
`PROFILE_LOCKED`	The browser profile directory is held by another process	Close other RTILA windows, then retry
`navigation timed out`	Slow or unreachable target	Raise `navigation_timeout` in the project settings

Workflow 4: Inspect Collected Data

Goal: Understand what a project scraped.

1. rtila_list_datasets { "project_id": "abc123" }
   → See dataset names, properties, selectors

2. rtila_get_project_data { "dataset": "products", "project_id": "abc123", "per_page": 10 }
   → Sample rows

3. Report totalItems alongside the sample so the user knows the scale.

Things to flag when inspecting data:

Observation	Likely cause
A column is null/empty in every row	The selector stopped matching — not that the site has no such data
A column looks like `{"a":1}` or `[1,2]`	Stringified JSON — the engine stores nested values as JSON-encoded strings
`succeeded` with zero rows	Selectors match nothing. Re-analyze and repair.
`failed` with some rows present	It collected partial data then broke. The partial data is real.

Skills for Claude Code

The RTILA repository includes Agent Skills that teach Claude Code how to drive the MCP server correctly. They encode the async polling loop, analysis report interpretation, run diagnosis, and dataset reading patterns.

Installing Skills

Copy the skill folders into your Claude Code skills directory:

Per project:

mkdir -p .claude/skills
cp -r skills/* .claude/skills/

For every project on your machine:

mkdir -p ~/.claude/skills
cp -r skills/* ~/.claude/skills/

Available Skills

Skill	Use it when
`rtila-engine`	Any RTILA work — the shared contract and tool selection guide.
`rtila-build-scraper`	“Scrape this page”, “build me an extractor”.
`rtila-run-project`	“Run my project”, “start the scraper”.
`rtila-debug-run`	“It failed”, “why did the run stop”.
`rtila-inspect-data`	“What did it collect”, “explain this dataset”.

What the Skills Encode

Skill	Key behaviors
rtila-engine	The three-step async contract, tool selection table, schema validation rules, integer type requirements, session_id requirements.
rtila-build-scraper	Analyze → read report → check WAF → check for API data → identify repeating unit → propose plan → confirm with user → run. Never guess selectors.
rtila-run-project	Resolve project → preflight check → launch → poll → report. Never run without user consent. Never pass `wait: true` for project runs.
rtila-debug-run	Read `stderr_tail` first → check error classification table → escalate to `rtila_get_run_log` → escalate to `rtila_get_logs` → distinguish “failed” from “no data”.
rtila-inspect-data	List datasets → sample rows (small page) → explain columns from selectors/transformations → flag empty columns and stringified JSON → report counts.

Troubleshooting

Symptom	Cause	Fix
Client shows no tools	Command path is wrong, or the process exited at startup	Run the command by hand in a terminal. A startup error appears on stderr.
`Server not initialized`	Protocol-level error; the client skipped the `initialize` handshake	Update your MCP client to the latest version.
Every data tool returns “database is not configured”	`--rest-api` is missing	Add `--rest-api http://127.0.0.1:8090` (or your actual database URL).
`'session_id' is required`	No session ID configured	Add `--session-id` at launch or pass `session_id` on each `rtila_run_project` call.
A tool call times out in the client	You passed `wait: true` and the work exceeded the client’s deadline	Drop `wait: true` and use the poll loop instead.
`Unknown run_id`	The server restarted since the run, or more than 50 runs finished since	Run records are ephemeral. Poll promptly.
`rtila_get_run_result` returns `result: null`	Normal for project runs	Use `rtila_get_run_status` for the outcome and `rtila_get_project_data` for the data.
Analysis “produced no JSON envelope”	The worker crashed before writing its report	The error details contain the worker’s stdout/stderr tails. Check them.
A run reports `failed`	The engine encountered an error	Read `stderr_tail` — it carries the engine’s own error classification.
`impure stdout lines` above zero in smoke test	Something in the command chain prints to stdout	Remove any wrapper script or shell echo. Never pipe the server through another program.
Workers survive after the client closes	The coordinator was not shut down gracefully	Ctrl-C the coordinator: it terminates every worker it started. Orphaned workers from a hard kill are reclaimed by the engine’s manifest self-heal on the next launch.
Connection refused on database URL	The database is not running	Start RTILA X (which starts the database) or start the database process manually.
`407 Proxy Authentication Required`	Proxy credentials are wrong	Check your proxy list format: `http://user:pass@host:port`. See Proxy Management.

Security

Consideration	Details
The server exposes your local database	It can read all projects, data, and logs. Treat the `RTILA_TOKEN` like a password.
The server can drive a real browser	It inherits your proxy, CAPTCHA, and browser profile settings. A project run visits real URLs.
Do not commit tokens	Use the `RTILA_TOKEN` environment variable, never hardcode tokens in config files that are version-controlled.
Loopback use only	The server is designed for local use. Do not expose `--rest-api` beyond your machine.
Data tools are read-only	Only browser tools mutate anything, and only through a project run or the builder.
`rtila_run_project` inherits project settings	It does not add any capability the desktop app does not already have. The project’s configured proxy, CAPTCHA provider, and browser profile are used.
Run records are ephemeral	They do not persist across server restarts. For audit purposes, use the database logs via `rtila_get_logs`.

Security Best Practices

Practice	Why It Matters
Use `RTILA_TOKEN` env var, not CLI args	Command-line args are visible in process listings (`ps`, Task Manager)
Use unique session IDs per client	Keeps run records and log attribution separate
Don’t expose the database port externally	The MCP server is designed for loopback use
Review project configs before running	`rtila_get_project_config` shows what URLs will be visited
Don’t run projects you didn’t create	A run visits real URLs with your credentials
Monitor `rtila_list_runs` for unexpected activity	Check for runs you didn’t initiate

MCP Server vs. AI Assistant Tools

RTILA X has two AI integration surfaces. Understanding the difference helps you choose the right one:

Aspect	AI Assistant (built-in)	Engine MCP Server
Where it runs	Inside the RTILA X desktop app	Standalone `rtila-engine` process
How to access	Chat interface in RTILA X	Any MCP client (Claude Code, Cursor, etc.)
Tools	`analyze_target_page`, `navigate_and_analyze`, `request_user_input`	17 tools: browser, run control, data
Data access	Through the AI chat	Direct database reads via data tools
Project execution	Via the Run button or AI	Via `rtila_run_project` tool
Async pattern	Handled internally	Explicit three-step poll loop
Best for	Building automations interactively	Integrating RTILA into external AI workflows

Tip: Use the AI Assistant for interactive automation building within RTILA X. Use the engine MCP server when you want an external AI tool (Claude Code, Cursor, etc.) to drive your RTILA automations programmatically.