Skip to content

Commit 1b10bf0

Browse files
committed
[Browser Run] Document crawl event subscriptions
1 parent 0fa94df commit 1b10bf0

4 files changed

Lines changed: 151 additions & 1 deletion

File tree

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
title: Subscribe to Browser Run crawl events
3+
description: Receive crawl lifecycle events through Cloudflare Queues.
4+
products:
5+
- browser-run
6+
- queues
7+
date: 2026-09-23
8+
---
9+
10+
import { PackageManagers } from "~/components";
11+
12+
[Browser Run crawl jobs](/browser-run/quick-actions/crawl-endpoint/) can publish lifecycle events to [Cloudflare Queues](/queues/). Subscribe to started, updated, and finished events to track progress or trigger downstream processing without polling.
13+
14+
To create an account-level subscription, run the following command:
15+
16+
<PackageManagers
17+
type="exec"
18+
pkg="wrangler"
19+
args="queues subscription create <QUEUE_NAME> --source browserRun --events crawl.started,crawl.updated,crawl.finished"
20+
/>
21+
22+
For payload examples, refer to the [Browser Run event schemas](/queues/event-subscriptions/events-schemas/#browser-run).

‎src/content/docs/browser-run/quick-actions/crawl-endpoint.mdx‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ Refer to [optional parameters](/browser-run/quick-actions/crawl-endpoint/#option
3737
There are two steps to using the `/crawl` endpoint:
3838

3939
1. [Initiate the crawl job](/browser-run/quick-actions/crawl-endpoint/#initiate-the-crawl-job) — A `POST` request where you initiate the crawl and receive a response with a job `id`.
40-
2. [Request results of the crawl job](/browser-run/quick-actions/crawl-endpoint/#request-results-of-the-crawl-job) — A `GET` request where you request the status or results of the crawl.
40+
2. [Request results of the crawl job](/browser-run/quick-actions/crawl-endpoint/#request-results-of-the-crawl-job) — Monitor the crawl with an event subscription or `GET` request. When it finishes, send a `GET` request for the results.
4141

4242
Crawl jobs have a maximum run time of seven days. If a job does not finish within this time, it will be cancelled due to timeout. Job results are available for 14 days after the job completes, after which the job data is deleted.
4343

@@ -85,6 +85,18 @@ The response includes a `status` field indicating the current state of the crawl
8585
- `errored` — The crawl job encountered an error.
8686
- `completed` — The crawl job finished successfully.
8787

88+
### Subscribe to lifecycle events
89+
90+
Use [Queues event subscriptions](/queues/event-subscriptions/events-schemas/#browser-run) as a push-based alternative to status polling. The linked reference includes setup instructions and payload schemas.
91+
92+
Browser Run is an account-level event source. A subscription receives events for all crawl jobs in your account:
93+
94+
- `crawl.started` — A crawl job starts.
95+
- `crawl.updated` — A crawled URL changes status.
96+
- `crawl.finished` — A crawl job finishes.
97+
98+
These events provide lifecycle and status information, not crawled page content. After `crawl.finished`, send a `GET` request with the job ID to fetch the full results.
99+
88100
### Polling for completion
89101

90102
Since crawl jobs run asynchronously, you can poll the endpoint periodically to check when the job finishes. Add `?limit=1` to the request URL so the response stays lightweight — you only need the job `status`, not the full set of crawled records.

‎src/content/docs/queues/event-subscriptions/events-schemas.mdx‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@ This page provides a comprehensive reference of available event sources and thei
2525

2626
<Render file="event-subscriptions/artifacts-events" product="queues" />
2727

28+
### Browser Run
29+
30+
<Render file="event-subscriptions/browser-run-events" product="queues" />
31+
2832
### Email Sending
2933

3034
<Render file="event-subscriptions/email-sending-events" product="queues" />
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
import { PackageManagers } from "~/components";
2+
3+
[Browser Run crawl](/browser-run/quick-actions/crawl-endpoint/) events are account-level. A subscription receives events for all crawl jobs in your account and does not require a source-specific selector.
4+
5+
#### Subscribe to events
6+
7+
##### Dashboard
8+
9+
Follow the [subscription creation procedure](/queues/event-subscriptions/manage-event-subscriptions/#create-subscription) and select **Browser Run** as the source.
10+
11+
##### Wrangler
12+
13+
To subscribe with Wrangler, run the following command:
14+
15+
<PackageManagers
16+
type="exec"
17+
pkg="wrangler"
18+
args="queues subscription create <QUEUE_NAME> --source browserRun --events crawl.started,crawl.updated,crawl.finished"
19+
/>
20+
21+
#### `crawl.started`
22+
23+
Triggered when a crawl job starts.
24+
25+
**Example:**
26+
27+
```json
28+
{
29+
"type": "cloudflare.browserRun.crawl.started",
30+
"source": {},
31+
"payload": {
32+
"jobId": "crawl-job-1234",
33+
"createdAt": "2026-09-23T14:00:00.000Z",
34+
"crawlConfig": {
35+
"url": "https://example.com",
36+
"limit": 100,
37+
"depth": 2,
38+
"render": true,
39+
"formats": ["markdown", "html"],
40+
"source": "all"
41+
}
42+
},
43+
"metadata": {
44+
"accountId": "00000000000000000000000000000000",
45+
"eventSubscriptionId": "11111111111111111111111111111111",
46+
"eventSchemaVersion": 1,
47+
"eventTimestamp": "2026-09-23T14:00:00.000Z"
48+
}
49+
}
50+
```
51+
52+
#### `crawl.updated`
53+
54+
Triggered when a crawled URL changes status.
55+
56+
**Example:**
57+
58+
```json
59+
{
60+
"type": "cloudflare.browserRun.crawl.updated",
61+
"source": {},
62+
"payload": {
63+
"jobId": "crawl-job-1234",
64+
"url": "https://example.com/docs/",
65+
"crawlStatus": "completed",
66+
"httpStatus": 200
67+
},
68+
"metadata": {
69+
"accountId": "00000000000000000000000000000000",
70+
"eventSubscriptionId": "11111111111111111111111111111111",
71+
"eventSchemaVersion": 1,
72+
"eventTimestamp": "2026-09-23T14:02:00.000Z"
73+
}
74+
}
75+
```
76+
77+
#### `crawl.finished`
78+
79+
Triggered when a crawl job finishes.
80+
81+
**Example:**
82+
83+
```json
84+
{
85+
"type": "cloudflare.browserRun.crawl.finished",
86+
"source": {},
87+
"payload": {
88+
"jobId": "crawl-job-1234",
89+
"jobStatus": "completed",
90+
"createdAt": "2026-09-23T14:00:00.000Z",
91+
"finishedAt": "2026-09-23T14:05:00.000Z",
92+
"total": 3,
93+
"completed": 2,
94+
"errored": 1,
95+
"skipped": 0,
96+
"crawlConfig": {
97+
"url": "https://example.com",
98+
"limit": 100,
99+
"depth": 2,
100+
"render": true,
101+
"formats": ["markdown", "html"],
102+
"source": "all"
103+
}
104+
},
105+
"metadata": {
106+
"accountId": "00000000000000000000000000000000",
107+
"eventSubscriptionId": "11111111111111111111111111111111",
108+
"eventSchemaVersion": 1,
109+
"eventTimestamp": "2026-09-23T14:05:00.000Z"
110+
}
111+
}
112+
```

0 commit comments

Comments
 (0)