Pipeline architecture

The request flow

Every upload is the same six-step handshake. The app server orchestrates metadata and authorization; the heavy bytes and heavy compute live elsewhere.

1. 1

   Claim an upload slotPOST /api/upload-url

   After authentication and a quick storage-quota pre-check, the server inserts a tasks row in D1 with a generated UUID, a storage key, and status uploading. It then returns a presigned R2 PUT URL valid for five minutes, along with the fileKey and taskId.

   Anonymous uploads from the homepage benchmark skip the quota check and are anonymous; signed-in uploads are tied to the real account.

2. 2

   Bytes go straight to R2

   The browser uploads the file with a direct XHR PUT to the presigned URL — the bytes never proxy through the application servers. The real byte progress from this request is the only segment driven by your actual transfer speed.

3. 3

   The upload-complete handshakePOST /api/upload-complete

   The server sends a HEAD request to R2 to learn the object’s real size and MIME type. It then enforces the plan’s per-file size limit and total storage quota; on either violation the R2 object is deleted and the task is marked failed.

   When the checks pass, the row moves to uploaded and the measured size and MIME type are persisted. The request then dispatches the Modal pre-check job, which is acknowledged with 202 Processing within seconds — the Worker never blocks on PDF analysis.

4. 4

   Cloud processing on Modal

   The server hands the fileKey, an HMAC-signed preview URL, a one-hour HMAC-signed CDN source-download URL and a signed progress-callback URL to a Modal-hosted pre-check service, so the workers never need R2 credentials.

   Modal runs the job in the background and calls back as it advances: Metadata done, xref/Pages done, then per-page Pixels progress. It parses the PDF, writes xref and page-tree artifacts back to R2, counts the pages, and spawns a parallel render-validation pass whose pixel-diff and SSIM results decide the Pixels stage.

5. 5

   Plan gates, then ready

   When the pre-check finishes, Modal posts the result to the callback endpoint. The page count is checked against the plan’s per-file page limit and monthly page quota; any failure marks the task failed with a machine-readable reason, the artifacts are deleted and the source file is kept so you can free the storage yourself.

   On success the row becomes ready: the artifact locations (pre_check_file) and page count are persisted, and the file is openable in the reader. Failures arrive through the same callback, so the dashboard only needs to poll the task list.

6. 6

   Signed, range-streamed preview/preview/pdf/{taskId}?token=…

   The preview token is a self-contained HMAC-SHA256 signature carrying the taskId and an expiry — no database lookup, constant-time verification. The reader exchanges it for time-limited HMAC-signed CDN URLs (cdn-stream-pdf.streampdf.com) for the source PDF and its artifacts.

   pdf.js then fetches only the byte ranges each page needs from the edge CDN backed by the private R2 bucket, which is why a large PDF opens in about a second and streams the rest as you scroll.

Task lifecycle

Each upload is one row in the D1 tasks table. Its status mirrors the pipeline above, and every transition is durable server-side — close the tab and the bar is rebuilt from this state when you return.

uploading

Task row created at upload-url; the bytes are still in flight from the browser.

uploaded

R2 object confirmed via HEAD; real size and MIME persisted; per-file size and total storage checks passed.

ready

Pre-check finished; page_count and artifact locations (xref, page tree) persisted — the file is openable.

failed

failure_reason is set and pre_check_file is cleared. The source object is deleted for size/storage violations and kept for quota or processing failures.

failure_reason codes

file_too_large

Object exceeds the plan’s per-file size limit.

storage_exceeded

Account storage quota was already exhausted.

too_many_pages

Page count exceeds the plan’s per-file page limit.

pages_quota_exceeded

Processing this file would exceed the plan’s monthly page quota.

processing_failed

The downstream pre-check or render pipeline errored or timed out.

Progress model

The segmented progress bar is a pure data model in app/dashboard/progress.ts — no React, no timers in the model itself — so it is unit-testable and reusable by any view. The key pieces:

STAGE_MIN_MS = 2000

Minimum wall-clock time each segment is shown, so fast files keep a readable bar. Elapsed time below it is padded for display only — processing is never delayed.

STAGE_HOLD_PERCENT = 95

The last stage fills to 95% and holds until the server confirms ready, preventing a false complete state.

createTaskProgress()

Builds the runtime for in-flight rows: session uploads follow real XHR progress; rows discovered after a refresh infer their stage from created_at.

stageWeights()

Segment widths derive from real per-stage durations with square-root compression and a minimum-width floor, so one very long stage cannot collapse the others.

createTerminalProgress()

Rebuilds canonical finished timelines for rows restored from history: each executed stage occupies one minimum duration — 8.0s on Free (four stages), 10.0s on paid plans (five).

startPixelRun() / pixelExecuted

Free-plan files keep pixelExecuted = false. After upgrading, a manual run rebuilds the frozen terminal state into “first four stages done, last stage running” and reuses the existing stage renderer.