Connect Firecrawl

How Firecrawl fits into Agent Barn

Firecrawl provides web search and page fetching for supported Agent runtimes. Agent Barn can supply its configuration in two ways:

1. A platform-wide default configured by the deployment administrator.
2. A per-Agent credential that overrides part or all of the platform default.

The platform default is useful when many Agents should use the same self-hosted Firecrawl service. A per-Agent override is useful when an Agent needs a separate Firecrawl Cloud project, quota, API key, or endpoint.

The runtime receives Firecrawl only when Agent Barn can resolve both a non-empty API key and a non-empty Base URL.

Choose a connection path

A per-Agent override is not a Shared Credential. It belongs to that Agent, and must be configured separately for other Agents.

Before you begin

The Agent Barn website may guide you through any required stop, update, and restart sequence.

Runtime behavior

Configuration resolution

Agent Barn resolves the effective Firecrawl configuration independently for the API key and Base URL.

The resolved API key and Base URL must both be non-empty.

There is no separate per-Agent disable switch in this credential model. If a platform default exists, removing the Agent override returns the Agent to that default.

Rotate or remove credentials

Rotate a platform key

1. Create or select the replacement Firecrawl key.
2. Update the Firecrawl server if it validates the shared key.
3. Update AGENT_FIRECRAWL_API_KEY in the Agent API deployment.
4. Apply the deployment.
5. Restart affected Agents.
6. Run both verification prompts.
7. Revoke the previous key.

Plan the rotation carefully when many Agents share the platform key, because they also share its availability and quota.

Rotate a per-Agent key

1. Stop the Agent if required.
2. Replace the Firecrawl API key in the Agent’s integration settings.
3. Confirm the correct Base URL.
4. Save and restart the Agent.
5. Run a search and fetch test.
6. Revoke the previous key.

Remove a per-Agent override

Remove the Firecrawl credential from the Agent and restart it. After removal:

• The Agent uses the platform Firecrawl configuration if one exists
• Firecrawl becomes unavailable if no complete platform configuration exists

Removing the override does not necessarily remove web access.

API reference

Add or replace a per-Agent credential

PATCH /api/v1/organizations/{organization_id}/agents/{agent_id}
Content-Type: application/json

{
  "secrets": [
    {
      "provider": "firecrawl",
      "content": {
        "api_key": "REDACTED",
        "base_url": "https://api.firecrawl.dev"
      }
    }
  ]
}

The provider identifier must be firecrawl.

To inherit the platform Base URL, submit an empty base_url:

{
  "secrets": [
    {
      "provider": "firecrawl",
      "content": {
        "api_key": "REDACTED",
        "base_url": ""
      }
    }
  ]
}

Only use this form when the platform Base URL is the intended endpoint.

Remove a per-Agent credential

{
  "removed_secret_providers": [
    "firecrawl"
  ]
}

Credential validation

Agent Barn currently has no registered validator for Firecrawl. Calling the validation endpoint:

POST /api/v1/organizations/{organization_id}/agents/{agent_id}/integrations/firecrawl/validate

may return HTTP 400 with:

{
  "detail": "No validator available for firecrawl"
}

Use an Agent runtime search or fetch test as the authoritative verification.

Limits and caching

Firecrawl Cloud applies the quotas, concurrency limits, and credit rules of the selected Firecrawl plan. Review the current Firecrawl documentation and your dashboard before enabling web-heavy workloads.

Consider that:

• Multiple Agents using one platform key share its quota
• Search and page-fetch operations can consume different amounts of capacity
• Concurrent Agent tasks can trigger provider rate limits
• HTTP 429 generally indicates rate or concurrency limiting
• HTTP 402 can indicate insufficient plan capacity or credits
• Runtime caching can cause a fetched page to remain unchanged temporarily
• The OpenClaw fetch configuration may reuse content for up to approximately 48 hours

For stricter quota or billing isolation, use separate per-Agent Firecrawl keys.

Troubleshooting

Security considerations

Follow these practices:

• Store Firecrawl keys only in Agent Barn’s secret management or deployment secret store
• Do not commit keys to the public agent-barn repository
• Use TLS for Firecrawl endpoints reached over untrusted networks
• Restrict access to self-hosted Firecrawl services at the network layer
• Consider whether Firecrawl can reach internal services from its deployment network
• Avoid giving web-enabled Agents unnecessary access to private network endpoints
• Rotate keys regularly, and immediately after suspected exposure
• Use separate keys when Agents require quota or billing isolation
• Remember that a platform-wide key increases the impact of a leak or quota exhaustion
• Review Firecrawl Cloud’s data-processing and retention terms before sending sensitive URLs or queries
• Limit Agent Barn access, because users who can manage Agent secrets can change the Firecrawl endpoint and key
• Do not paste credentials into prompts, conversations, logs, or support messages

Next steps

After Firecrawl is connected:

1. Verify both web search and page fetching.
2. Review Agent instructions for safe use of untrusted web content.
3. Monitor Firecrawl quota, latency, and error rates.
4. Decide whether the platform default or isolated Agent keys best fit your workload.
5. Continue to Use shared credentials for providers that support centrally managed Organization credentials.