PDF.js Source Walkthrough: BasePDFStream

Introduction

This installment examines one complete source file: src/shared/base_pdf_stream.js, all 240 lines of it.

The earlier network-stream article treated this file as one subsystem among fetch, XHR, Node.js, and push transports. Here the perspective is narrower: what contract does the base file itself establish, what state does it own, and which decisions remain the responsibility of each subclass?

The file exports three classes:

• BasePDFStream owns the byte source and creates readers;
• BasePDFStreamReader describes whole-document streaming and response metadata;
• BasePDFStreamRangeReader describes an independently requested byte interval.

None of these classes downloads data by itself. Their value is a common interface that lets the worker pull bytes without knowing whether the main thread obtained them through fetch, XMLHttpRequest, a local file stream, or application-supplied buffers.

Module Overview

The dependency graph is deliberately small: only assert and unreachable come from util.js. There is no networking, parser, or worker-message code in this module.

The central object relationship is:

BasePDFStream
├── at most one BasePDFStreamReader
└── zero or more BasePDFStreamRangeReader instances in a Set

“Full” and “range” are therefore not two methods on one cursor. They are separate reader objects with independent I/O state, coordinated only by the stream's small registry.

BasePDFStream.constructor(): Injecting the Implementation

constructor(source, PDFStreamReader, PDFStreamRangeReader) {
  if (
    (typeof PDFJSDev === "undefined" || PDFJSDev.test("TESTING")) &&
    this.constructor === BasePDFStream
  ) {
    unreachable("Cannot initialize BasePDFStream.");
  }
  this._source = source;
 
  this.#PDFStreamReader = PDFStreamReader;
  this.#PDFStreamRangeReader = PDFStreamRangeReader;
}

The base class does not select a transport. A concrete stream injects two reader constructors through super:

super(source, PDFFetchStreamReader, PDFFetchStreamRangeReader);

This is constructor injection without a registry or factory framework. BasePDFStream knows when readers may be created; the supplied classes know how their read() and cancel() methods work. The private fields prevent later replacement of those classes.

source is retained as _source, and subclasses read transport-specific options from it: URL, headers, range size, disableRange, disableStream, or a PDFDataRangeTransport. The base class deliberately gives this object no schema.

The direct-instantiation guard is active in testing or unbuilt code. It catches new BasePDFStream(...) while still permitting subclasses. The abstract method guards discussed below remain the more important protection against missing reader implementations.

BasePDFStream._progressiveDataLength: Measuring Full-Stream Coverage

get _progressiveDataLength() {
  return this._fullReader?._loaded ?? 0;
}

This getter answers only one question: how many bytes has the full reader reported loading so far?

Before getFullReader(), or before any bytes arrive, the value is zero. It is not the file length, the number of bytes consumed by the parser, or the sum of range traffic. Only the full reader's _loaded field participates.

That distinction explains its sole use: avoiding a range request whose entire interval is already covered by progressive loading.

BasePDFStream.getFullReader(): One Whole-Document Reader

getFullReader() {
  assert(
    !this._fullReader,
    "BasePDFStream.getFullReader can only be called once."
  );
  return (this._fullReader = new this.#PDFStreamReader(this));
}

The full reader is a singleton per stream. Construction and registration happen together, so later calls to _progressiveDataLength and cancelAllRequests() can reach the same object.

The assertion enforces a lifecycle rule rather than performing I/O:

1. the first call constructs the concrete full reader;
2. the instance is saved in _fullReader;
3. a second call fails instead of replacing or duplicating it.

Cancellation does not reset _fullReader, so cancelling a stream does not make a second full reader available. This matches the one-document-loading lifecycle of the callers.

Starting the request is a subclass decision made inside its reader constructor. The base factory merely instantiates it.

BasePDFStream.getRangeReader(): A Conditional Factory

getRangeReader(begin, end) {
  if (end <= this._progressiveDataLength) {
    return null;
  }
  const reader = new this.#PDFStreamRangeReader(this, begin, end);
  this._rangeReaders.add(reader);
  return reader;
}

Range intervals use the half-open form [begin, end) at this layer. Concrete transports translate that form to their own APIs, such as HTTP's inclusive end - 1.

The method has two mutually exclusive outcomes:

The test looks only at end. If begin is below _loaded but end is above it, a range reader is still created; the base class does not trim the already-loaded prefix.

Registration occurs only after successful construction. Unlike the full reader, range readers are explicitly multi-instance because the parser may need unrelated intervals concurrently.

The source comment states an additional timing contract: callers are expected to invoke this method after the full reader's headersReady promise resolves. The base method does not await or assert that ordering. It relies on the caller to establish range support and stream capabilities first.

Why null Is Part of the Contract

null does not mean that the PDF bytes are unavailable. It means that this particular range request is unnecessary because the full stream already covers them.

The real GetRangeReader handler in display/api.js handles the branch explicitly:

const rangeReader = this.#networkStream.getRangeReader(data.begin, data.end);
 
if (!rangeReader) {
  sink.close();
  return;
}

Closing this range sink does not discard the document data. The already-running full reader continues toward the worker, so a new request would duplicate work.

BasePDFStream.cancelAllRequests(): Fan-Out, Not Cleanup

cancelAllRequests(reason) {
  this._fullReader?.cancel(reason);
 
  // Always create a copy of the rangeReaders.
  for (const reader of new Set(this._rangeReaders)) {
    reader.cancel(reason);
  }
}

Cancellation has three precise semantics:

• the full reader is cancelled if it exists;
• every registered range reader receives the same reason;
• range iteration uses a copy of the Set.

The copy matters because concrete cancel() implementations may synchronously remove themselves from the original set through onClosed or onDone callbacks. Iterating a snapshot prevents that mutation from changing the cancellation pass.

This method neither awaits asynchronous shutdown nor clears _rangeReaders, and it does not reset _fullReader. It is a fan-out operation, not a destructor that restores the stream to its initial state.

The base class also does not remove readers at end-of-file. XHR and push streams arrange their own removal callbacks; fetch and Node.js leave that lifecycle detail to their implementations and the owning stream's lifetime.

BasePDFStreamReader.constructor(): Shared Full-Reader State

The full-reader base class owns metadata and progress state, not a concrete byte queue:

onProgress = null;
_contentLength = 0;
_filename = null;
_headersCapability = Promise.withResolvers();
_isRangeSupported = false;
_isStreamingSupported = false;
_loaded = 0;
_stream = null;
 
constructor(stream) {
  // Direct construction is rejected in testing/unbuilt code.
  this._stream = stream;
}

Each reader gets its own Promise.withResolvers() capability. The default values are intentionally conservative: unknown length is 0, missing filename is null, and optional range/streaming capabilities are false until a subclass establishes otherwise.

_stream is a back-reference to the owning BasePDFStream. It gives implementations access to _source, shared headers, response origin, and reader-registry callbacks without moving those transport details into the base class.

_loaded has a special dual role: implementations use it for progress accounting, and BasePDFStream reads it for range suppression.

headersReady and the Metadata Getters

get headersReady() {
  return this._headersCapability.promise;
}
 
get filename() {
  return this._filename;
}
 
get contentLength() {
  return this._contentLength;
}
 
get isRangeSupported() {
  return this._isRangeSupported;
}
 
get isStreamingSupported() {
  return this._isStreamingSupported;
}

These getters form a metadata barrier. Their documented values are defined after headersReady resolves:

headersReady may reject when response setup fails or a reader is cancelled. A subclass must resolve it only after writing the metadata it wants consumers to observe.

This promise does not mean that the document body has finished downloading. For fetch it usually means response headers have arrived; for the push transport it can resolve immediately from properties already supplied by PDFDataRangeTransport.

The worker-facing ReaderHeadersReady handler in display/api.js awaits this exact promise before returning contentLength, isStreamingSupported, and isRangeSupported. That caller is what turns the source comment about range timing into an actual ordering guarantee.

BasePDFStreamReader._callOnProgress(): One Notification Shape

_callOnProgress() {
  this.onProgress?.({ loaded: this._loaded, total: this._contentLength });
}

The helper is synchronous and optional. If no callback is installed, it does nothing. Otherwise it emits a fresh object containing the current _loaded and _contentLength values; it does not retain a progress history.

Fetch, Node.js, and push readers invoke this helper as data arrives. XHR instead forwards its native progress event directly, using the event's total when it is computable. Thus the base class standardizes the callback shape, not the mechanism that measures progress.

The documentation highlights progress as especially useful when streaming is unsupported: the consumer may be waiting for one complete response rather than receiving a steady sequence of body chunks.

BasePDFStreamReader.read() and cancel(): Abstract Full-Reader Operations

async read() {
  unreachable("Abstract method `read` called");
}
 
cancel(reason) {
  unreachable("Abstract method `cancel` called");
}

These are the two operations every full-reader subclass must implement.

The documented read() result mirrors a standard stream reader:

{ value, done: false } // value contains the next binary chunk
{ value, done: true }  // end of input

A pending read waits when no chunk is available. The documented cancellation contract resolves cancelled reads with done: true, while I/O failures reject the pending operation.

Despite the similar shape, BasePDFStreamReader is not itself a WHATWG ReadableStream or reader. It has no standard locking, queueing strategy, or tee(). The resemblance is intentional: display/api.js can place it directly under a MessageHandler sink's onPull callback.

There is also a concrete boundary beyond the base JSDoc. The main-thread GetReader and GetRangeReader handlers assert that each non-final value is an ArrayBuffer, wrap it in a Uint8Array, and transfer the buffer to the worker. Concrete main-thread readers therefore fulfill a stricter payload contract than “binary data” alone.

cancel(reason) initiates teardown for that reader. The base interface does not return a promise or prescribe whether cancellation is implemented through AbortController, xhr.abort(), a web-stream reader, or local state transitions.

BasePDFStreamRangeReader.constructor(): Only the Owner Is Shared

constructor(stream, begin, end) {
  if (
    (typeof PDFJSDev === "undefined" || PDFJSDev.test("TESTING")) &&
    this.constructor === BasePDFStreamRangeReader
  ) {
    unreachable("Cannot initialize BasePDFStreamRangeReader.");
  }
  this._stream = stream;
}

Although the constructor receives begin and end, it stores only _stream. That is not an omission in the concrete request path: the arguments remain available to every subclass constructor, which can use them immediately or retain them itself.

The implementations make different choices:

• fetch uses both values immediately to build a Range header;
• Node.js maps them to fs.createReadStream options;
• XHR passes them into its request helper;
• push mode stores _begin so incoming chunks can be matched to the correct reader, while end is used when requesting the range.

This is why the base class does not impose _begin and _end fields on everyone.

The range reader's read() and cancel(reason) have the same abstract contracts as the full reader. It intentionally has no headersReady, onProgress, content length, or capability getters: response-level metadata belongs to the one full reader, not to every fragment.

How fetch, XHR, Node.js, and Push Fulfill the Contract

The base classes are small because each transport supplies a different state machine behind the same methods.

fetch: Two Different Readiness Promises

The fetch full reader waits on the inherited _headersCapability; each range reader has its own _readCapability. Both gate read() until their underlying response.body reader exists.

The full reader also owns the progressive counters and response metadata. If streaming is disabled and range is supported, it cancels the initial body after using its headers, then resolves headersReady. Cancellation of that initial request is therefore a capability-negotiation outcome, not necessarily a failure of document loading.

XHR: Buffered Responses behind a Reader-Shaped API

PDFNetworkStreamReader never enables _isStreamingSupported. Its read() still returns the standard { value, done } shape, but internally it chooses among a cached chunk, a completed state, and a newly parked promise.

When range support is established, XHR aborts the full request and uses separate requests for fragments. Its full and range readers also demonstrate why cancellation must settle pending reads: their shared endRequests helper resolves parked capabilities with done: true.

XHR and push readers actively remove completed or cancelled range readers from _rangeReaders. That cleanup is transport-specific, not behavior supplied by the base registry.

Node.js: Filesystem Metadata Replaces HTTP Headers

PDFNodeStreamReader treats lstat(url) as its metadata phase. File size becomes contentLength; range support depends on disableRange and whether the file is larger than two range chunks. There is no content-disposition filename to extract.

Both full and range readers expose file data through web-compatible readers. As with fetch, the range endpoint changes from the base layer's exclusive end to the filesystem API's inclusive end - 1.

Push: The Application Becomes the Byte Source

PDFDataTransportStream reverses the I/O direction. The transport constructor seeds its queue and done state from initialData and progressiveDone; the full reader reads length and contentDispositionFilename from PDFDataRangeTransport, inherits that queued state through the stream, and resolves headersReady immediately.

A read() may therefore complete in three ways: shift an already queued chunk, observe _done, or wait in _requests until application code pushes more data. _enqueue() updates _loaded even when nobody is currently reading, which keeps _progressiveDataLength useful.

For ranges, the stream both constructs the reader and calls requestDataRange(begin, end). Incoming range chunks carry a begin offset, and the stream finds the registered reader whose _begin matches. Here the base Set is not merely a cancellation list; it is also the demultiplexing table.

Finally, push mode overrides cancelAllRequests(reason): after the base fan-out, it also calls pdfDataRangeTransport.abort(), propagating cancellation back to the application-owned source.

The Worker-Side Adapter Uses the Same Base

core/worker_stream.js adds another implementation, PDFWorkerStream. Its readers wrap GetReader and GetRangeReader in MessageHandler.sendWithStream(), then unwrap transferred Uint8Array values back to ArrayBuffer.

This closes the design loop: main-thread transports implement the base contract over real I/O, display/api.js pumps their reads into message streams, and the worker consumes the same full/range abstraction without knowing which transport produced the bytes.

Summary

base_pdf_stream.js is not a network implementation. It is the reader lifecycle and capability contract shared by every PDF byte source.

The most important invariants are:

1. One full reader per stream. getFullReader() registers it once; cancellation does not reopen the factory.
2. Range creation is conditional. getRangeReader() returns null when the full reader's _loaded already covers end, and callers must handle that successful suppression branch.
3. Metadata has a timing barrier. contentLength, filename, isRangeSupported, and isStreamingSupported are meaningful after headersReady resolves.
4. The reader API is stream-shaped but not a ReadableStream. Subclasses implement asynchronous read() and cancel(); main-thread consumers additionally require ArrayBuffer chunks.
5. The base owns coordination, not transport policy. Header validation, request initiation, buffering, range matching, and physical cancellation belong to fetch, XHR, Node.js, push, or worker adapters.
6. Cancellation is mutation-aware fan-out. Iterating a copied range set allows subclass callbacks to unregister readers safely, but the base method does not itself clear all lifecycle state.

Once these boundaries are clear, the concrete stream files become implementations of a precise contract rather than four unrelated networking recipes.

StreamPDF design principle: anything that can be handled ahead of time is absolutely not deferred to parse time; the server solves all performance problems once. The browser only fetches and renders the data it must.