PDF.js Source Walkthrough: brotli_stream

Introduction

This is another entry in the "Bytes and Streams" layer, covering the worker-side src/core/brotli_stream.js.

Earlier articles already took this chain apart once: decode_stream provides the output buffer and the read cursor, flate_stream turns a DEFLATE bit stream back into bytes, and predictor_stream turns row-predicted samples back into original rows. This article swaps in a different filter, but the skeleton is unchanged — BrotliStream is still a subclass of DecodeStream, still implements a single readBlock, and is still driven on demand by the layers above.

What differs is seniority. Flate, LZW and ASCII85 are all old faces from the PDF 1.x era, whereas Brotli only entered the standard as a filter with PDF 2.0. The spec writes it as /BrotliDecode, and the data itself is the Brotli byte stream defined by RFC 7932. It shows up rarely in real documents, because the vast majority of generation tools still prioritise compatibility with older readers; when it does appear, it usually means the producer was willing to trade a newer compression algorithm for a smaller file, for instance size-sensitive font subsets or high-resolution scans.

For the parser its position is exactly that of Flate: after decryption, before PDF semantic parsing. When /Filter /BrotliDecode appears in a dictionary, Parser.makeFilter wraps the raw bytes into a BrotliStream; what the upper layer reads is already decompressed bytes, and it never needs to know which compression algorithm was used.

brotli_stream.js is only 86 lines long, with no Huffman tables of its own and no bit buffer. The actual decompression is delegated to two external implementations: the browser's DecompressionStream, and BrotliDecode in external/brotli/decode.js. The file's responsibility is therefore narrow — choose between the two decoders, move bytes around, reuse data that has already been read, and fold the state back into the DecodeStream contract.

This article also does not intend to expand on the Brotli algorithm itself. The static dictionary, context modelling, window size and metablock layout all belong to RFC 7932 and external/brotli, and have no direct bearing on PDF's stream model. The questions genuinely worth discussing in a PDF.js context are: how does a decoder with no incremental interface fit into a stream framework that reads on demand, and how does a fast path that depends on platform capability fall back safely when it fails. Those two points are the subject of these 86 lines.

In one sentence: it is the adapter layer for the Brotli filter — first ask whether the browser can decompress it, and if not fall back to the bundled decoder, with both routes handing the upper layer the same bytes.

Module Overview

The file splits into five blocks:

Where it sits in the chain:

PDF bytes
  → Parser.readStream              (read the stream dictionary and the data section)
  → DecryptStream                  (optional: default stream cipher / /Crypt filter)
  → Parser.makeFilter              (sees /Filter /BrotliDecode)
  → BrotliStream(stream, length)   (this file)
  → DecompressionStream("brotli")  (native fast path)
  → BrotliDecode                   (synchronous fallback after failure)
  → upper layers: content streams / fonts / image data

Inheritance is still BaseStream → DecodeStream → BrotliStream. The base class provides buffer, bufferLength, eof, pos, ensureBuffer, getByte and getBytes, and specifies that a subclass only needs to implement readBlock: each call appends a segment of data to the output buffer, and sets eof to true when the source is exhausted. BrotliStream neither overrides the read interface nor introduces a second buffer; it only adds an "async first, sync later" branch on top of that contract.

Design note: All of this class's complexity comes from having two decoding paths. On success it hands out the whole block through a return value; on failure it downgrades itself into an ordinary synchronous stream. Both paths share the same dict, the same pos and the same buffer field, so the upper layer always faces a single stream object and never has to care which path this particular decode took.

The Parent Contract: When readBlock Is Called

BrotliStream has no read loop of its own; the driving power is in the parent class. Only by understanding this can you see why readBlock can read everything in one go without breaking the semantics of on-demand reading:

  getBytes(length, decoderOptions = null) {
    const pos = this.pos;
    let end;
 
    if (length) {
      this.ensureBuffer(pos + length);
      end = pos + length;
 
      while (!this.eof && this.bufferLength < end) {
        this.readBlock(decoderOptions);
      }
      const bufEnd = this.bufferLength;
      if (end > bufEnd) {
        end = bufEnd;
      }
    } else {
      while (!this.eof) {
        this.readBlock(decoderOptions);
      }
      end = this.bufferLength;
    }
 
    this.pos = end;
    return this.buffer.subarray(pos, end);
  }

When called with a length, the parent first ensures the buffer can hold the target position, then calls readBlock repeatedly until the buffer is long enough or the stream ends; when called without a length, it reads until eof and then returns a view from the current position to the end. getByte goes through the same small loop, only asking for one byte at a time.

Constructing a BrotliStream does not trigger any decoding by itself. readBlock runs the first time somebody wants bytes; and once it runs, it decodes the whole block. That produces a result which looks contradictory but is in fact reasonable: the stream is lazy in semantics and all-or-nothing in granularity. Semantic laziness guarantees that a stream nobody reads costs nothing; block decoding guarantees that after one call, any subsequent read of any length is just a pos move.

The decoderOptions argument passed to readBlock(decoderOptions) is never used inside BrotliStream. That matches its role too: decoderOptions serves image decoders such as JPX, while Brotli only does byte-to-byte conversion and has no tunable parameters. The parent class allows subclasses to ignore the parameter precisely so that pure filters like this do not have to fake options that do not exist.

Class Structure and Construction

The constructor Method

class BrotliStream extends DecodeStream {
  #isAsync = true;
 
  constructor(stream, maybeLength) {
    super(maybeLength);
 
    this.stream = stream;
    this.dict = stream.dict;
  }

The constructor does only three things.

First, it hands maybeLength to the parent. DecodeStream uses it to compute the starting point for growth of minBufferLength: it doubles from that value until it can hold the requested length. Note that it is only a buffer starting point; it takes no part in any length validation and is never treated as the length of the decode result. The /Length in a PDF dictionary is often wrong, sometimes even 0, and the constructor does not trust it.

Second, it stores the underlying stream. this.stream may be a Stream wrapped around a file slice, a ChunkedStream, or even another DecodeStream. This matters: filters compose, and if the /Filter array puts another decoder after Brotli, the outer decoder's underlying stream is the BrotliStream, whose asyncGetBytes will be asked for bytes first.

Third, it shares the dictionary. this.dict = stream.dict is a reference assignment, not a copy. The dictionary carries the object number, the encryption context and everything else later stages need; copying it not only wastes memory but also makes the two layers disagree about dictionary identity.

The private field #isAsync starts out true, meaning "the native path may still be attempted". It is not a global capability cache but per-instance state: within the same document, some Brotli streams succeed natively while others fall back, and they do not affect each other.

The code performs no header validation on the Brotli data at all. Flate has the two-byte zlib header and FCHECK, which can detect bad data early; Brotli has no fixed header — the first bytes are the compressed stream itself — so validation can only be deferred until the actual decode.

The Async Fast Path: asyncGetBytes

The Reused DecompressionStream Helper

Native decoding does not live in this file; it lives in the parent's asyncGetBytesFromDecompressionStream:

  async asyncGetBytesFromDecompressionStream(name) {
    this.stream.reset();
    const bytes = this.stream.isAsync
      ? await this.stream.asyncGetBytes()
      : this.stream.getBytes();
 
    try {
      const { readable, writable } = new DecompressionStream(name);

It first calls the underlying stream's reset() to move the read position back to the start; then, depending on whether the underlying stream supports async, it picks await this.stream.asyncGetBytes() or the synchronous this.stream.getBytes(). The former lets the whole async chain recurse: when one async stream sits on top of another, the outer one waits for the inner one to produce a result and then feeds it to its own decoder in one shot. The latter is a no-argument call, meaning "read to the end", leaving no residual bytes.

The bytes obtained are both the input to be decompressed and the asset to be reclaimed on failure. The function returns two fields, { decompressed, compressed }: on success the former is a Uint8Array, and on failure the former is null while the latter always keeps the compressed bytes that were already read. All native exceptions are swallowed inside this function and never reach the caller.

The format name passed in is "brotli". It is not among the first batch of formats Compression Streams supported (deflate, deflate-raw, gzip), and availability varies with browser version. When unsupported, new DecompressionStream("brotli") throws immediately, which lands exactly in the catch above and the flow naturally moves to the fallback. So no extra capability probe is needed here: one real attempt replaces one feature detection — success saves a check, and failure still leaves you with the bytes already read.

The asyncGetBytes Method

  async asyncGetBytes() {
    const { decompressed, compressed } =
      await this.asyncGetBytesFromDecompressionStream("brotli");
    if (decompressed) {
      return decompressed;
    }
    // DecompressionStream failed (for example because there are some extra
    // bytes after the end of the compressed data), so we fallback to our
    // decoder.
    // We already get the bytes from the underlying stream, so we just reuse
    // them to avoid get them again.
 
    this.#isAsync = false;
    this.stream = new Stream(
      compressed,
      0,
      compressed.length,
      this.stream.dict
    );
    this.reset();
    return null;
  }

The body splits into two halves: on success return the data, on failure remodel itself.

The success branch only checks whether decompressed is truthy. Note that this is a truthiness test, not a length test: even a zero-length decode result is a valid Uint8Array object and is equally truthy. A valid zero-length result and "decoding failed" are two different things, and using null for the latter keeps the semantics unambiguous.

The failure branch does four steps.

1. #isAsync = false. This is a one-way switch: once the native path is proven unusable, it is never attempted a second time. It simultaneously tells the upper layer "do not treat this stream as async any more".
2. Create Stream(compressed, 0, compressed.length, this.stream.dict). The start is 0 because Brotli has no header to skip; by contrast, FlateStream uses 2 on fallback, which skips exactly the zlib header. The dictionary is still inherited from the old stream, preserving object identity.
3. this.reset(). DecodeStream.reset only zeroes pos and does not clear the buffer. At this point the output buffer holds nothing yet, and the reset exists so that subsequent getBytes calls drive readBlock from the beginning.
4. return null. This is a sentinel value: it does not mean "no data", it means "the async path failed, please take the synchronous path".

compressed is the bytes just read in one go from the underlying stream, used directly to build the new Stream, avoiding a second request to the network or the file. The cost is that this compressed data cannot be released before the fallback completes; given that a Brotli stream needs whole-block decoding anyway, that cost is of the same order as the decoder's own memory requirement and adds no extra burden.

Design note: Failure is not an exception, it is a return value. Once an exception is thrown to the upper layer, the caller has to know Brotli implementation details to handle it correctly; returning null keeps the decision to take the other path inside the stream. #isAsync carries two meanings at once — capability switch and fallback record — and it only goes down, never up, so an accidental success will not switch it back to async.

Synchronous Fallback: readBlock

The readBlock Method

  readBlock() {
    // TODO: add some telemetry to measure how often we fallback here.
    // Get all bytes from the input stream
    const bytes = this.stream.getBytes();
    const decodedData = BrotliDecode(
      new Int8Array(bytes.buffer, bytes.byteOffset, bytes.length)
    );
 
    this.buffer = new Uint8Array(
      decodedData.buffer,
      decodedData.byteOffset,
      decodedData.length
    );
    this.bufferLength = this.buffer.length;
    this.eof = true;
  }

The first comment states a fact: the author considers the fallback a rare but worth-measuring path. Native decoding is faster, and a fallback means extra JavaScript decoding overhead; if some class of documents frequently ends up here, it indicates a systematic difference between the native implementation and what PDF generators produce.

this.stream.getBytes() takes no argument, meaning "read until the stream ends". This differs from how most decoders consume the underlying stream chunk by chunk: FlateStream takes it bit by bit through a bit buffer, RunLengthStream takes it run by run, whereas here everything must be taken at once. The reason is that BrotliDecode has no incremental interface, and Brotli's window references can reach very far back, so only the complete compressed stream can be decoded safely. The cost is that the fallback path's memory peak equals compressed data plus decompressed data; the benefit is a simple implementation with no state machine.

new Int8Array(bytes.buffer, bytes.byteOffset, bytes.length) merely creates another view, with no copy. All three arguments are essential: if only bytes.buffer were passed, then when bytes is a subarray of some larger buffer the decoder would receive unrelated bytes from that buffer as well, and the length would be wrong. BrotliDecode's signature requires an Int8Array while the underlying stream yields a Uint8Array, so a conversion is mandatory here — but only the type label changes.

The decode result is likewise wrapped back into a Uint8Array as a view: decodedData.buffer plus its own offset and length. That way, even if BrotliDecode returns a slice of some larger buffer, no surplus bytes are exposed, and there is no second copy.

The key line is this.buffer = .... Other text-oriented decoders usually grow with ensureBuffer and append via set; here the entire output buffer is replaced. There are three reasons: the output length is unknown before decoding, so it cannot be pre-sized; the decode result is already a complete standalone array, so appending would only add a full copy; and after replacement bufferLength can simply take the result length, so the read interface no longer has to track how much has been written. The side effect of replacement is that the old empty buffer is discarded, but at the moment the fallback happens it holds nothing anyway.

Finally eof is set to true. Brotli decodes in one whole block, so there cannot be another block; DecodeStream.getBytes's loop therefore stops immediately, and later reads only move pos over a fixed buffer.

Image Entry Point: getImageData

The getImageData Method

  async getImageData(length, _decoderOptions) {
    const data = await this.asyncGetBytes();
    if (!data) {
      return this.getBytes(length);
    }
    if (data.length <= length) {
      return data;
    }
    return data.subarray(0, length);
  }

This is a dedicated entry point for the image path, and its logic is exactly the same as FlateStream.getImageData.

It first attempts an async whole-block decode unconditionally. On failure, asyncGetBytes has already converted the stream to synchronous mode and returned null, so execution falls to this.getBytes(length): the DecodeStream loop drives readBlock and stops once length bytes are available. On success it receives the whole decompressed result, which still has to be trimmed to length.

length comes from the size declared in the image dictionary itself; the caller computes it in PDFImage.getImageBytes from Width, Height and the bit depth. It need not match the real length of the stream: the generator may have written /Length incorrectly, or left padding at the end of the data. Extra bytes are meaningless for pixel decoding, and subarray(0, length) simply cuts them off. If the decompressed result is shorter than length, it is returned as is, without zero padding — whether to pad is the business of the specific image decoder, and this layer does not decide it on its behalf.

subarray returns a view rather than a copy, sharing memory with the decode buffer. That sharing is safe here: PDFImage has a clear contract about bytes produced by a DecodeStream, treating them as self-contained and not modifiable by later stream reads; only image data whose underlying stream is a plain Stream needs an extra copy, so that the returned view does not disturb other reads.

One more prerequisite detail: PDFImage.getImageBytes calls this.image.reset() before invoking this, moving the stream's read position back to the start. The same image may be requested several times, and the semantics of getImageData are "take one complete copy from the beginning", not "continue from where the last read stopped".

The image path only lands on BrotliStream when Brotli is the last link in the filter chain, so the output is directly pixel data. If the chain still has an image codec such as DCT or JPX, PDFImage receives the outer image stream and Brotli is merely the byte source beneath it. That is also why the _decoderOptions parameter is unused here: Brotli is not responsible for image formats, and the options belong to the outer decoder.

State: #isAsync

The isAsync Method (getter)

  get isAsync() {
    return this.#isAsync;
  }

BaseStream.isAsync returns false by default, DecodeStream does not override it, and only subclasses with asynchronous decoding capability change it to reflect real state. BrotliStream starts at true because Brotli happens to have a native implementation; once it falls back, the value becomes permanently false.

This boolean is consumed in several places, in almost the same shape each time: check isAsync first, then await asyncGetBytes() if true, wrap a non-null result into an ordinary Stream, and continue synchronous parsing.

• PDFDocument.getContentStream uses it to pre-decompress a page content stream into a Stream.
• PartialEvaluator.getOperatorList performs the same replacement before interpreting drawing operators.
• PartialEvaluator.getTextContent prepares contiguous bytes for text extraction.
• Font loading and CMap parsing also do one async prefetch for font files and encoding streams.

The design result is this: when the native path succeeds, decoding finishes inside a single await in the Worker, after which every consumer holds an ordinary Stream with no adapter layer at all; when the native path fails, isAsync is already false, consumers skip the async attempt and keep using the BrotliStream, with readBlock decompressing on demand. In neither case is the same failing call attempted twice.

Worth noting is the meaning of null. An empty result new Uint8Array(0) is valid data and the caller wraps it as usual; only null means "async is unavailable, keep using this stream". A dedicated sentinel distinguishing empty from failed is more reliable than a length test.

The Two Paths Side by Side

The Timeline of a Single Read

Laying the two paths out as a timeline makes it clearer how the constraint that compressed bytes are read only once is enforced.

When native succeeds:

1. The upper layer gets the stream, sees that isAsync is true, and calls asyncGetBytes().
2. The parent helper first reset()s the underlying stream, then reads the compressed bytes in one go.
3. Those bytes are written to the writable of DecompressionStream("brotli"); the write does not block, and the readable side subsequently collects all chunks asynchronously.
4. The chunks are concatenated into one contiguous Uint8Array and returned as decompressed; the original bytes come back alongside as compressed.
5. asyncGetBytes returns the decompressed result directly. The upper layer wraps it into a Stream, and BrotliStream drops out of this parse's view.
6. All subsequent reads happen on an ordinary Stream and pass through no decoder at all.

On fallback:

1. Steps 2 and 3 are the same as above, except that construction or writing of the DecompressionStream throws.
2. The helper catches the exception and returns decompressed as null with compressed holding the original bytes.
3. asyncGetBytes replaces the underlying stream with a new Stream wrapping compressed, turns off #isAsync, resets the cursor, and returns null.
4. The upper layer sees null and decides to keep using the original BrotliStream.
5. The first real getBytes triggers readBlock, and BrotliDecode decodes the whole block in one go.
6. The output buffer is replaced wholesale, eof is set to true, and the caller receives a view from pos to the end.

The only differences between the two paths are who performs the decode and where the result lives: the native result lives in the return value, the fallback result lives in buffer. Reading the compressed bytes, inheriting the dictionary and the identity of the stream object are identical throughout.

There is also an easily overlooked detail: the async path is optional, not mandatory. Even if no caller ever awaits asyncGetBytes, calling getBytes directly on a BrotliStream that never tried the native path still decompresses correctly through the bundled decoder, and #isAsync remaining true merely means it has not been tried yet. The class's correctness does not depend on which path the caller picks; async is just an extra acceleration lane.

When the Async Path Is Not Taken

The following scenarios leave the stream synchronous:

• The caller only uses getBytes and never checks isAsync, for example some internal helper or test code.
• The upper layer has already decided, for other reasons, that this stream needs no prefetch, for example when a content stream is processed slice by slice.
• Native decoding already failed once, so #isAsync has been set to false.

All three cases end the same way: readBlock runs the first time bytes are needed, decoding happens on the Worker's synchronous stack, and the caller notices nothing — except that the wait for the first byte includes the whole Brotli decode.

Filter Chain and Edge Cases

The dispatch in Parser.makeFilter is a single line:

        case "BrotliDecode":
          return new BrotliStream(stream, maybeLength);

Comparing with the Flate branch reveals one difference: Flate and LZW wrap another PredictorStream when /DecodeParms is present, whereas Brotli returns directly with no predictor wrapper. In other words, this version does not apply row-predictor restoration to Brotli output. If a document really does attach predictor parameters to Brotli, the result will not be corrected. This is a matter of implementation scope, not a defect of the decoder.

A few edge cases deserve their own explanation.

• Zero length. When maybeLength === 0, makeFilter returns a NullStream first and never constructs a BrotliStream at all. The constructor therefore does not need to handle an empty stream.
• Trailing extra bytes. Native implementations usually require the compressed stream to end cleanly at its last byte, and trailing residue makes them raise an error. Incorrect /Length, padded data sections and concatenated filters are all common in PDFs, and this is exactly the fallback cause named in the code comment. The bundled decoder is more tolerant of trailing data, but it does not guarantee a successful result either.
• Truncated data. Both decoders may raise on missing data. The try in makeFilter only covers stream construction, not lazy decoding; the real throw happens in readBlock, where the caller decides whether to ignore or abort.
• Stacked filters. If Brotli is not the last link, the outer decoder recursively asks it for bytes through asyncGetBytesFromDecompressionStream. Async capability propagates up the chain, which is why isAsync is a stream property rather than the capability of some fixed type.

Together these edge cases show one thing: BrotliStream is responsible only for whether the compressed bytes are complete, not for whether the PDF generator is well-behaved. The former is judged by the two decoders, the latter is handled by the upper parser according to its fault-tolerance policy. With the responsibilities drawn that clearly, this file of under a hundred lines can hold both the native and bundled implementations without looking messy.

Comparison with FlateStream

The two share the same async skeleton: getImageData is written word for word identically, and the failure handling in asyncGetBytes differs only in the start offset. That consistency is no coincidence — once native-first, fall-back-on-failure is factored out into a parent helper, each filter only has to fill in its own format name and fallback details.

Conclusion

BrotliStream is PDF.js's Brotli adapter layer: upward it gives content streams, fonts and images decompressed bytes, and downward it stitches the browser's native capability and the bundled decoder into a path that is transparent to the upper layers.

Where It Sits in the Overall Chain

In one sentence: Parser recognises the filter, BrotliStream chooses the decoder, the native implementation handles the common case, and the bundled implementation covers the rest.

Five Design Principles

1. Platform capability first, semantic fallback. Use native browser decompression whenever possible; when the native implementation rejects the data — trailing extra bytes, unsupported format — swap back to the bundled decoder on the spot. The standard path aims for speed, the fallback path aims at getting through the file.
2. Read once, decode as one block. readBlock does not consume the underlying stream chunk by chunk but takes everything at once. That is dictated by the BrotliDecode interface, and it buys a simple implementation with no incremental state machine.
3. Reuse on failure. The fallback does not re-read the compressed data; it wraps the already-obtained compressed into a new Stream. The failure path costs one extra decode, not one extra I/O.
4. Failures are expressed as return values. null is a dedicated sentinel meaning async is unavailable; an empty array is still valid data. The upper layer can therefore switch paths without catching exceptions.
5. State only goes one way, and only once. #isAsync never returns from false to true, avoiding repeated native attempts on the same bad data; each instance records it independently, so other streams in the same document are unaffected.

With this, the less common Brotli filter is covered. Its relationship with Flate is not replacement but two implementations on the same skeleton: one carries a full bit-stream state machine, the other outsources the algorithm to the platform and external/. The next article returns to decrypt_stream to see why decryption must come before filters in encrypted documents, and how it uses 512-byte chunks plus a single lookahead to find the real last chunk.

StreamPDF design principle: anything that can be handled ahead of time is absolutely not deferred to parse time; the server solves all performance problems once. The browser only fetches and renders the data it must.