PDF.js Source Walkthrough: flate_stream

Introduction

This is an article in the "Bytes and Streams" layer, and its subject is the worker-side src/core/flate_stream.js.

Flate is the most frequent filter in PDF, and Predictor always follows it, applying row prediction to images and cross-reference streams. This article covers only the former; the latter is left for the next one.

Almost every heavyweight piece of data in a PDF hides behind a compressed stream: cross-reference streams, object streams, page content streams, embedded fonts, and image pixel data. The moment that /Filter /FlateDecode appears in a dictionary, Parser.makeFilter wraps the raw bytes in a FlateStream. One layer below, PredictorStream may wrap it again to handle row prediction for images and cross-reference streams.

So the place of this article is clear: it is the link in the "bytes to bytes" chain that sits closest to the compressed data. Its input is DEFLATE data with a zlib header; its output is the raw byte sequence, after decryption and before prediction. It understands no PDF syntax and cares nothing about image color spaces; it does one thing — turn a bit stream back into bytes.

flate_stream.js is 457 lines in total, about 120 of which are lookup constants. The copyright header at the top of the file is candid: this implementation is a JavaScript port of XPDF. So do not expect a "textbook DEFLATE lecture" when reading it; it is an engineering implementation polished for performance and fault tolerance: tables over loops, block-by-block processing, and partial results whenever the stream is truncated.

In one sentence, its job: turn a zlib-wrapped DEFLATE bit stream back into a byte buffer the caller can read sequentially.

Module Overview

The file splits into eight blocks:

Where it sits in the chain:

PDF bytes
  → Parser.makeFilter             (after reading /Filter /FlateDecode)
  → FlateStream(str, maybeLength) (this file: zlib header + DEFLATE decoding)
  → PredictorStream               (optional: /DecodeParms row prediction)
  → DecodeStream.getBytes         (pull on demand, readBlock loop)
  → upper layers: image data / font files / content streams / xref streams

Inheritance runs BaseStream → DecodeStream → FlateStream. DecodeStream provides buffer, bufferLength, eof, ensureBuffer, getByte and getBytes; FlateStream only has to implement readBlock, appending one chunk of decoded output to the tail of buffer per call. Upper layers trigger readBlock repeatedly by the length they need, so decoding is lazy: a stream nobody reads is never decompressed.

Design note: the granularity of readBlock is "one DEFLATE block", not "the whole stream". Once a block is decoded it returns and control goes back to DecodeStream.getBytes; if the caller only wants 64 bytes, decoding stops as soon as 64 bytes are available. That keeps first-screen rendering from waiting for an entire image to decompress.

The zlib Header: Validation and Identity

PDF's Flate data is not bare DEFLATE but the RFC 1950 zlib wrapper: a 2-byte header, the DEFLATE data, and a 4-byte Adler-32 trailer. The constructor handles only those two header bytes; the trailer checksum is ignored entirely.

The constructor method

  constructor(str, maybeLength) {
    super(maybeLength);
 
    this.stream = str;
    this.dict = str.dict;
 
    const cmf = str.getByte();
    const flg = str.getByte();
    if (cmf === -1 || flg === -1) {
      throw new FormatError(`Invalid header in flate stream: ${cmf}, ${flg}`);
    }
    if ((cmf & 0x0f) !== 0x08) {
      throw new FormatError(
        `Unknown compression method in flate stream: ${cmf}, ${flg}`
      );
    }
    if (((cmf << 8) + flg) % 31 !== 0) {
      throw new FormatError(`Bad FCHECK in flate stream: ${cmf}, ${flg}`);
    }
    if (flg & 0x20) {
      throw new FormatError(`FDICT bit set in flate stream: ${cmf}, ${flg}`);
    }
 
    this.codeSize = 0;
    this.codeBuf = 0;
  }

Those two bytes carry four layers of information:

A few details are worth spelling out.

First, str.getByte() returning -1 means the stream is over. If the header has only 1 byte, the second getByte returns -1, and -1 in a bitwise operation yields all ones; without an explicit check, the tests that follow would emit misleading error messages. So the code does one overall check first.

Second, (cmf & 0x0f) !== 0x08 checks only the low 4 bits. The zlib specification requires CINFO to be no greater than 7, but the high 4 bits are not validated here — the DEFLATE decoder does not depend on window size, it only uses distance codes, and the maximum distance is already 32768. A wrong CINFO at most means the encoder used a smaller window; decoding is unaffected.

Third, ((cmf << 8) + flg) % 31 !== 0 is FCHECK. The equivalent form is % 31 === 0 rather than recomputing FCHECK bit by bit. The most common legal headers are 0x78 0x9C, 0x78 0xDA and 0x78 0x01, corresponding to the default, maximum and fastest levels.

Fourth, flg & 0x20 is FDICT. A zlib stream with a preset dictionary needs that dictionary to decompress, and the PDF specification does not allow /FlateDecode to use one, so the code throws instead of reading a 4-byte dictionary ID.

The constructor clears the bit buffer at the end. codeSize is how many bits remain usable in the buffer, and codeBuf holds those bits. The underlying stream's position has already advanced by 2 bytes, which is exactly why asyncGetBytes rebuilds the stream starting at offset 2 when it falls back.

Design note: header validation happens in the constructor, not on the first readBlock. A corrupt stream therefore surfaces immediately inside makeFilter's try, and the caller can catch it with a PixelError or a NullStream instead of failing halfway through rendering.

Bit Reading and Code Table Decoding

The getBits method

DEFLATE is a bit stream: block headers are 3 bits, dynamic blocks interleave 5, 4 and 3 bits, and Huffman codes themselves are variable-length. getBits handles "take bits by count" and keeps the leftover that spans byte boundaries on the instance.

  getBits(bits) {
    const str = this.stream;
    let codeSize = this.codeSize;
    let codeBuf = this.codeBuf;
 
    let b;
    while (codeSize < bits) {
      if ((b = str.getByte()) === -1) {
        throw new FormatError("Bad encoding in flate stream");
      }
      codeBuf |= b << codeSize;
      codeSize += 8;
    }
    b = codeBuf & ((1 << bits) - 1);
    this.codeBuf = codeBuf >> bits;
    this.codeSize = codeSize -= bits;
 
    return b;
  }

Only three things deserve attention.

First, bytes enter low bits first. codeBuf |= b << codeSize places the newly read byte above the bits already buffered; extraction uses codeBuf & ((1 << bits) - 1) to take from the low end. Bit 0 of the buffer is therefore always "the next bit to consume". This matches DEFLATE's bit order — RFC 1951 specifies that all fields other than Huffman codes are least-significant-bit first. Huffman codes themselves are most-significant-bit first, and generateHuffmanTable irons out that difference with a single bit reversal.

Second, while (codeSize < bits) decides how many bytes to read. If the buffer already holds 5 bits and 13 are needed, one more byte is read to reach 13; surplus high bits stay in the buffer for next time. The number of byte reads is "just enough", not a fixed value.

Third, the state is written back immediately: codeBuf >> bits discards the consumed low bits and codeSize -= bits updates the count. Copying into locals and writing back at the end means this read/write pair does not touch this repeatedly — a real gain in a function called once per byte.

There is no need to worry about 32-bit overflow. The largest single request in readBlock is the 13 extra distance bits, plus at most 7 residual bits in the buffer, well under 20 bits; JavaScript bitwise operations work on 32-bit signed integers, so 20 bits is far inside the safe range. A real decoder never needs a "32 bits at once" call.

Design note: the bit buffer fully decouples "the position in the byte stream" from "the position in the bit stream". str.getByte() is called only when needed, and the caller sees a purely functional interface, getBits(bits), with byte boundaries completely transparent.

The getCode method

With a bit buffer in place, a variable-length Huffman code needs just one table lookup. getCode takes an "expanded table": table[0] is the array of code entries and table[1] is the maximum code length.

  getCode(table) {
    const str = this.stream;
    const codes = table[0];
    const maxLen = table[1];
    let codeSize = this.codeSize;
    let codeBuf = this.codeBuf;
 
    let b;
    while (codeSize < maxLen) {
      if ((b = str.getByte()) === -1) {
        // premature end of stream. code might however still be valid.
        // codeSize < codeLen check below guards against incomplete codeVal.
        break;
      }
      codeBuf |= b << codeSize;
      codeSize += 8;
    }
    const code = codes[codeBuf & ((1 << maxLen) - 1)];
    const codeLen = code >> 16;
    const codeVal = code & 0xffff;
    if (codeLen < 1 || codeSize < codeLen) {
      throw new FormatError("Bad encoding in flate stream");
    }
    this.codeBuf = codeBuf >> codeLen;
    this.codeSize = codeSize - codeLen;
    return codeVal;
  }

The key is that each entry is packed into a 32-bit integer: the high 16 bits hold the code length, the low 16 bits hold the symbol value. The lookup uses the lowest maxLen bits of the buffer as an index and hits in one step; after the hit, the entry tells getCode how many bits to actually consume (codeLen).

Why is it fine to "read ahead a full maxLen bits"? Because the table is laid out to the maximum code length: a code of length len fills every slot whose low len bits equal that code with the same symbol. So even if the current code is only 3 bits long, 3 bits in the buffer are enough to locate it; extra bits read in do not affect the match on the low 3 bits. Bits that were read but not consumed stay for next time via codeBuf >> codeLen.

if (codeSize < codeLen) guards the end of the stream: when getByte returns -1 the loop breaks, the buffer may hold only 2 bits, and an entry claiming 5 bits means the code is incomplete, which must be an error. The comment is explicit: codeSize < codeLen is the only guard.

codeLen < 1 covers another case: an illegal or over-subscribed code table leaves 0 in some slots, so decoding into such a slot yields 0 in the high 16 bits and is rejected outright. This check turns a "bad code table" into a clear runtime exception rather than a nonsensical symbol.

this.codeSize = codeSize - codeLen is written differently from codeSize -= bits in getBits, but the semantics are identical: write the remaining bit count back.

Design note: this is the classic "fixed-width index, variable-length consumption" structure. Decoding drops from "walk len steps down a tree" to "one array access", at the cost of 2 to the power maxLen integers per table. Dynamic blocks build two fresh tables each time, and in the worst case maxLen is 15, so one table holds 32768 entries, roughly 128 KB — a deliberate trade of memory for CPU. It still pays off in a browser, because what is saved is a branch misprediction per symbol.

The generateHuffmanTable method

The table getCode relies on is built on the spot by generateHuffmanTable. The input is an array of code lengths per symbol; the output is "the expanded table plus the maximum code length".

  generateHuffmanTable(lengths) {
    const n = lengths.length;
 
    // find max code length
    let maxLen = 0;
    let i;
    for (i = 0; i < n; ++i) {
      if (lengths[i] > maxLen) {
        maxLen = lengths[i];
      }
    }
 
    // build the table
    const size = 1 << maxLen;
    const codes = new Int32Array(size);
    for (
      let len = 1, code = 0, skip = 2;
      len <= maxLen;
      ++len, code <<= 1, skip <<= 1
    ) {
      for (let val = 0; val < n; ++val) {
        if (lengths[val] === len) {
          // bit-reverse the code
          let code2 = 0;
          let t = code;
          for (i = 0; i < len; ++i) {
            code2 = (code2 << 1) | (t & 1);
            t >>= 1;
          }
 
          // fill the table entries
          for (i = code2; i < size; i += skip) {
            codes[i] = (len << 16) | val;
          }
          ++code;
        }
      }
    }
 
    return [codes, maxLen];
  }

Read it in three steps.

Step one: find the maximum code length. The table size is 1 << maxLen, which fixes both memory use and lookup width. An empty length array yields maxLen = 0 and size = 1; the single slot holds 0, and getCode fails on codeLen < 1 — semantically, an empty table is an illegal table.

Step two: assign codewords in canonical Huffman order. The outer loop walks code lengths from 1 up to maxLen and the inner loop walks symbol values upward. Within one length, earlier symbols get smaller codes; moving to a longer length does code <<= 1, because longer codes append a 0 to the previous level's codes. That is exactly the canonical order defined by RFC 1951, with no extra sorting required.

skip and code shift left in lockstep: skip is 2^len, the stride at which each code appears in the maxLen-bit index space. Codes of length 1 appear every 2 slots, length 2 every 4 slots, and so on.

Step three: reverse the bits, then fill the table. This is the line easiest to overlook in the whole function:

code2 = (code2 << 1) | (t & 1);

Canonical Huffman codewords are written most-significant-bit first, while the bit buffer stores least-significant-bit first. Without the reversal, the low bits of codeBuf would correspond to the high bits of the code and the masked index would never match. After reversing, "prefix matching" becomes "suffix matching", so the fill can be written as a simple loop that starts at code2 and steps by skip.

A minimal example: three symbols with lengths [1, 2, 2]:

Symbol 0 has length 1, so it occupies two slots; symbols 1 and 2 occupy one each. At lookup time, whether the low 2 bits of the buffer are 00 or 10, the result is symbol 0, and codeLen then tells getCode to consume only 1 bit.

Note that there is no validation for "over-subscribed" or "incomplete" tables. A legal DEFLATE stream never contains such tables; an illegal one shows up at lookup time as a 0 entry, caught by codeLen < 1 in getCode. Validation is deliberately deferred to the point of use, so the hot table-building path pays nothing extra.

Design note: canonical assignment guarantees that "the array of code lengths" is the only input — the codewords themselves never need to be transmitted; the bit reversal adapts "prefix codes" to a "low-bit-first" bit buffer. Together they reduce variable-length decoding to one fixed-width array access.

Fixed Tables and Dynamic Code Tables

Fixed code tables: 288 precomputed codes

DEFLATE's fixed Huffman block uses a code table hard-coded by the specification. PDF.js does not generate it at runtime; the whole expanded table is written out at the top of the file as a constant.

const fixedLitCodeTab = [
  new Int32Array([
    0x70100, 0x80050, 0x80010, 0x80118, 0x70110, 0x80070, 0x80030, 0x900c0, 0x70108, 0x80060,
    0x80020, 0x900a0, 0x80000, 0x80080, 0x80040, 0x900e0, 0x70104, 0x80058, 0x80018, 0x90090,
    0x70114, 0x80078, 0x80038, 0x900d0, 0x7010c, 0x80068, 0x80028, 0x900b0, 0x80008, 0x80088,
    0x80048, 0x900f0, 0x70102, 0x80054, 0x80014, 0x8011c, 0x70112, 0x80074, 0x80034, 0x900c8,
    0x7010a, 0x80064, 0x80024, 0x900a8, 0x80004, 0x80084, 0x80044, 0x900e8, 0x70106, 0x8005c,
    0x8001c, 0x90098, 0x70116, 0x8007c, 0x8003c, 0x900d8, 0x7010e, 0x8006c, 0x8002c, 0x900b8,
    0x8000c, 0x8008c, 0x8004c, 0x900f8,
    // middle omitted, see the source for the full 512 entries
    0x7010f, 0x8006f, 0x8002f, 0x900bf, 0x8000f, 0x8008f, 0x8004f, 0x900ff,
  ]),
  9,
];

Reading those hex values with the "high 16 bits code length, low 16 bits symbol" rule makes them immediately clear:

The fixed literal table's specification is: 0 to 143 use 8 bits, 144 to 255 use 9 bits, 256 to 279 use 7 bits, and 280 to 287 use 8 bits. Its 512 entries are 2 to the power 9 slots, exactly covering the maximum code length 9. All 288 symbols occupy several slots each, and the lookup width is fixed at 9.

The distance table is smaller:

const fixedDistCodeTab = [
  new Int32Array([
    0x50000, 0x50010, 0x50008, 0x50018, 0x50004, 0x50014, 0x5000c, 0x5001c, 0x50002, 0x50012,
    0x5000a, 0x5001a, 0x50006, 0x50016, 0x5000e, 0x00000, 0x50001, 0x50011, 0x50009, 0x50019,
    0x50005, 0x50015, 0x5000d, 0x5001d, 0x50003, 0x50013, 0x5000b, 0x5001b, 0x50007, 0x50017,
    0x5000f, 0x00000,
  ]),
  5,
];

It has 32 entries and a maximum code length of 5, but only 30 valid distance symbols (0 to 29). Symbols 30 and 31 are reserved by the specification, and their slots are filled with 0x00000 — code length 0, symbol 0. If a dynamic or corrupt stream ever decodes into those slots, codeLen < 1 in getCode throws Bad encoding in flate stream.

The constant tables buy two things: each fixed block skips a table build, and the builder is not run pointlessly for fixed blocks. Fixed blocks are very common in small content streams (a Form XObject of a few hundred bytes, say), so the precomputation is well worth it.

Design note: each entry packs "code length" and "symbol" into one integer, avoiding two parallel arrays or an array of objects. A lookup yields a single 32-bit value, and getCode splits it with two bitwise operations, adding almost no overhead.

Dynamic code tables: the code-length code and three repeats

A dynamic Huffman block encodes the code table into the bit stream too. It first sends a "code-length code table", then uses that table to decode the real literal and distance code lengths.

    } else if (hdr === 2) {
      // compressed block, dynamic codes
      const numLitCodes = this.getBits(5) + 257;
      const numDistCodes = this.getBits(5) + 1;
      const numCodeLenCodes = this.getBits(4) + 4;
 
      // build the code lengths code table
      const codeLenCodeLengths = new Uint8Array(codeLenCodeMap.length);
 
      let i;
      for (i = 0; i < numCodeLenCodes; ++i) {
        codeLenCodeLengths[codeLenCodeMap[i]] = this.getBits(3);
      }
      const codeLenCodeTab = this.generateHuffmanTable(codeLenCodeLengths);

The three counts are encoded differently: the literal code count stores "actual count minus 257", ranging 257 to 288; the distance code count stores "minus 1", ranging 1 to 32; the code-length code count stores "minus 4", ranging 4 to 19. None of the upper bounds is a power of two, so the values cannot be taken with a plain bit field.

The code-length codes are not transmitted in symbol order 0, 1, 2 but according to a fixed permutation table:

const codeLenCodeMap = new Int32Array([
  16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15,
]);

The first three are the special codes 16, 17 and 18, followed by the rarely used 0, then descending and interleaved from 8. RFC 1951 defines this order, putting the most frequent code lengths first so that "transmitting only the first few entries" covers as much as possible.

codeLenCodeLengths has the length of the permutation table itself, 19, and writes are placed at codeLenCodeMap[i], so the array used for table building always has 19 entries; symbols that were never transmitted get code length 0.

Next, that table decodes the literal and distance code lengths, with three special codes handling run-length compression:

len = 0;
i = 0;
const codes = numLitCodes + numDistCodes;
const codeLengths = new Uint8Array(codes);
let bitsLength, bitsOffset, what;
while (i < codes) {
  const code = this.getCode(codeLenCodeTab);
  if (code === 16) {
    bitsLength = 2;
    bitsOffset = 3;
    what = len;
  } else if (code === 17) {
    bitsLength = 3;
    bitsOffset = 3;
    what = len = 0;
  } else if (code === 18) {
    bitsLength = 7;
    bitsOffset = 11;
    what = len = 0;
  } else {
    codeLengths[i++] = len = code;
    continue;
  }
 
  let repeatLength = this.getBits(bitsLength) + bitsOffset;
  while (repeatLength-- > 0) {
    codeLengths[i++] = what;
  }
}

len is "the last non-special code length" and survives across iterations. Code 16 copies it; codes 17 and 18 copy 0 and also reset len to 0 — an easily missed detail: after a run of 17s, a following 16 must copy 0, not the older length from before the 17s. The specification requires this, and the code does both things at once with what = len = 0.

The repeat count is "extra bits read plus the base value". For code 18, 7 bits give 0 to 127, and adding 11 gives 11 to 138. That way 32 all-zero code lengths can be expressed with a handful of symbols, keeping the header overhead of dynamic blocks in check.

while (repeatLength-- > 0) has no bounds check at all. If a corrupt stream declares a repeat count beyond the remaining slots, i runs past codes; since codeLengths is a Uint8Array, out-of-range assignments are silently discarded and memory is not corrupted, and the surrounding while (i < codes) exits once i has overshot. Relying on the typed array as a backstop is not pretty, but it is safe enough.

Right after the two tables are built, they are split at the boundary:

litCodeTable = this.generateHuffmanTable(codeLengths.subarray(0, numLitCodes));
distCodeTable = this.generateHuffmanTable(codeLengths.subarray(numLitCodes, codes));

subarray is a view, not a copy, so this step allocates nothing; it just hands two slices of the same buffer to the table builder.

Design note: the code-length code table has only 19 entries yet must decode up to 320 code lengths, so run-length encoding is the only way out. Keeping "the previous length" in len and requiring codes 17 and 18 to zero it is the finest semantic constraint in the format; implement it one line short and consecutive runs produce a wrong code table.

The readBlock Method

readBlock starts by reading 3 header bits: the lowest is BFINAL, the other two are BTYPE.

let buffer, hdr, len;
const str = this.stream;
// read block header
try {
  hdr = this.getBits(3);
} catch (ex) {
  this.#endsStreamOnError(ex.message);
  return;
}
if (hdr & 1) {
  this.eof = true;
}
hdr >>= 1;

The three block types

After the block header, BFINAL is handled first: if set, eof becomes true. There is a counter-intuitive ordering here — eof is set before the current block is decoded. This does not conflict because readBlock runs synchronously: after setting the flag it still finishes the block completely and writes it to the buffer, and only when it returns does DecodeStream.getBytes see eof and stop calling again. In other words the semantics of eof are "do not call readBlock again", not "stop right now".

Uncompressed blocks

When BTYPE is 0, the remaining bits are discarded, the position is aligned to a byte boundary, and a 4-byte length follows.

    if (hdr === 0) {
      // uncompressed block
      let b;
 
      if ((b = str.getByte()) === -1) {
        this.#endsStreamOnError("Bad block header in flate stream");
        return;
      }
      let blockLen = b;
      if ((b = str.getByte()) === -1) {
        this.#endsStreamOnError("Bad block header in flate stream");
        return;
      }
      blockLen |= b << 8;
      if ((b = str.getByte()) === -1) {
        this.#endsStreamOnError("Bad block header in flate stream");
        return;
      }
      let check = b;
      if ((b = str.getByte()) === -1) {
        this.#endsStreamOnError("Bad block header in flate stream");
        return;
      }
      check |= b << 8;
      if (check !== (~blockLen & 0xffff) && (blockLen !== 0 || check !== 0)) {
        // Ignoring error for bad "empty" block (see issue 1277)
        throw new FormatError("Bad uncompressed block length in flate stream");
      }
 
      this.codeBuf = 0;
      this.codeSize = 0;

Four nearly identical reads, each checking for -1. The two 16-bit little-endian integers are LEN and its complement NLEN. The specification requires NLEN to be the bitwise complement of LEN; the check here adds one exception: when both blockLen and check are 0 it is accepted. The comment points at issue 1277 — some generators write an empty uncompressed block at the end with NLEN not correctly inverted. Reporting an error for an empty block is pointless, so it is allowed through.

The byte-alignment action is just this.codeBuf = 0; this.codeSize = 0;. Reading the 3 header bits consumed one byte, leaving 5 bits in the buffer; clearing them discards exactly those 5 bits and lands on the start of the next byte. No separate skip call is needed.

      const bufferLength = this.bufferLength,
        end = bufferLength + blockLen;
      buffer = this.ensureBuffer(end);
      this.bufferLength = end;
 
      if (blockLen === 0) {
        if (str.peekByte() === -1) {
          this.eof = true;
        }
      } else {
        const block = str.getBytes(blockLen);
        buffer.set(block, bufferLength);
        if (block.length < blockLen) {
          this.eof = true;
        }
      }
      return;
    }

The output buffer is grown to end in one step and the whole block is copied over. str.getBytes(blockLen) may return fewer bytes than requested — when the underlying stream is truncated it returns a short array. In that case set writes only what was actually obtained and eof is then set, so the caller finishes with an incomplete tail instead of hitting an exception.

Empty blocks are handled more carefully: eof is set only when peekByte() returns -1. If more data follows, this was merely an empty block in the middle and decoding continues with the next block.

Design note: the uncompressed block does not go through "decode bit by bit" but a single set copy. What can be solved in one step is not turned into a loop — a choice that recurs throughout this implementation. Likewise, ensureBuffer(end) keeps the main decode loop from growing the buffer once per literal.

The main decode loop: literals, lengths and distances

Fixed and dynamic blocks differ only in where their tables come from and are identical once they reach the main loop. Before entering the loop the code takes a reference to the underlying buffer and its capacity:

    buffer = this.buffer;
    let limit = buffer ? buffer.length : 0;
    let pos = this.bufferLength;
    while (true) {
      let code1 = this.getCode(litCodeTable);
      if (code1 < 256) {
        if (pos + 1 >= limit) {
          buffer = this.ensureBuffer(pos + 1);
          limit = buffer.length;
        }
        buffer[pos++] = code1;
        continue;
      }
      if (code1 === 256) {
        this.bufferLength = pos;
        return;
      }

code1 falls into three ranges with three meanings: 0 to 255 are literals, 256 is end-of-block, and 257 to 285 are length symbols. ensureBuffer is only called when capacity is insufficient, and limit is updated with it; on the normal path a literal costs one array store and one increment.

The literal branch tests pos + 1 >= limit, not pos + 1 > limit. Because the write target is index pos, the code needs pos to be less than limit; using greater-or-equal also calls ensureBuffer when the write exactly fills the buffer, where it simply returns the same buffer — conservative but safe. Real growth happens when the requested size exceeds capacity, and it grows in powers of two.

On 256 it aligns bufferLength to pos and returns. Note that bufferLength is not updated per byte — the main loop advances a local pos and synchronises only at block end. That keeps bufferLength from being repeatedly walked through an object property inside the block.

Lengths and distances use two constant tables:

code1 -= 257;
code1 = lengthDecode[code1];
let code2 = code1 >> 16;
if (code2 > 0) {
  code2 = this.getBits(code2);
}
len = (code1 & 0xffff) + code2;
code1 = this.getCode(distCodeTable);
code1 = distDecode[code1];
code2 = code1 >> 16;
if (code2 > 0) {
  code2 = this.getBits(code2);
}
const dist = (code1 & 0xffff) + code2;

Both tables follow the same packing convention as the code tables: the high 16 bits are the extra bit count, the low 16 bits are the base value. The table entries are themselves constants:

const lengthDecode = new Int32Array([
  0x00003, 0x00004, 0x00005, 0x00006, 0x00007, 0x00008, 0x00009, 0x0000a, 0x1000b, 0x1000d, 0x1000f,
  0x10011, 0x20013, 0x20017, 0x2001b, 0x2001f, 0x30023, 0x3002b, 0x30033, 0x3003b, 0x40043, 0x40053,
  0x40063, 0x40073, 0x50083, 0x500a3, 0x500c3, 0x500e3, 0x00102, 0x00102, 0x00102,
]);
 
const distDecode = new Int32Array([
  0x00001, 0x00002, 0x00003, 0x00004, 0x10005, 0x10007, 0x20009, 0x2000d, 0x30011, 0x30019, 0x40021,
  0x40031, 0x50041, 0x50061, 0x60081, 0x600c1, 0x70101, 0x70181, 0x80201, 0x80301, 0x90401, 0x90601,
  0xa0801, 0xa0c01, 0xb1001, 0xb1801, 0xc2001, 0xc3001, 0xd4001, 0xd6001,
]);

lengthDecode has 31 entries, indexed by "symbol minus 257". The first entries are short codes for lengths 3 to 10 with 0 extra bits; the middle entries gain extra bits as the base value follows the specification upward; the last three are all 0x00102, meaning base 258 with 0 extra bits. In the specification only symbol 285 is 258, while 286 and 287 are reserved and unused. This table maps them all to 258 — a lenient "better to give a plausible value than to throw" treatment; a proper encoder never produces them, and if one appears, it does not bring a whole page down.

distDecode has 30 entries, base values from 1 up to 24577 and at most 13 extra bits. 0x10005 means "1 extra bit, base 5", i.e. a distance of 5 or 6; 0xd6001 means "13 extra bits, base 24577".

The two unpacking blocks have the same shape: take the high 16 bits as the extra bit count, call getBits when needed, and add the low 16-bit base. The if (code2 > 0) check matters — when the extra bit count is 0, getBits(0) must not be called. It would perform a pointless read and return 0; the result would be the same, but the bit buffer's semantics would be broken (getBits(0) reads a byte when codeSize is 0).

LZ77 back-reference copy

With len and dist in hand, the last piece is the copy:

      if (pos + len >= limit) {
        buffer = this.ensureBuffer(pos + len);
        limit = buffer.length;
      }
      for (let k = 0; k < len; ++k, ++pos) {
        buffer[pos] = buffer[pos - dist];
      }
    }

Only three lines, but dist may be smaller than len: DEFLATE allows extremes such as "distance 1, length 100" to encode a long run of repeated bytes. In that case the source position read during the copy may be a byte that was just written. Copying byte by byte, forward, from pos - dist to pos handles the overlap correctly — when writing byte k, pos - dist may already lie inside the region written by this copy, so the pattern keeps spreading forward. If set or memcpy moved len bytes in one go, the overlapping part would read stale, not-yet-overwritten data and the output would be wrong.

That is also why the buffer.set optimisation from the uncompressed block cannot be reused here: the two have different data dependencies.

ensureBuffer(pos + len) checks capacity before each copy so the whole write stays in bounds. The condition again uses greater-or-equal: when pos + len equals the current capacity, ensureBuffer is called once and returns the same buffer; this only keeps it consistent with the literal branch and avoids re-deriving boundary rules.

Design note: pos is the cursor within a block, bufferLength the cursor between blocks. Everything written inside a block touches only pos, avoiding property access, and is committed once at block end. Combined with the cached limit, the inner loop is left with nothing but array access and bitwise operations.

Fault Tolerance and the Async Fast Path

FlateStream faces bytes from arbitrary PDFs, and corruption, truncation and non-standard encoding are all common. Its strategy is not "always throw" but layered handling.

The #endsStreamOnError method

  #endsStreamOnError(err) {
    info(err);
    this.eof = true;
  }

The method does exactly two things: log a line and set eof. It is used in two places — a failure to read the block header, and a failure to read the full length field of an uncompressed block. Both mean "the data after this cannot be trusted, but what was decoded before is still usable", so it ends with EOF and lets the caller keep the partial result.

By contrast, getBits and getCode failures inside a dynamic block are not caught; the FormatError propagates all the way to the caller. The reason is that such failures happen where "the code table or the data is already scrambled", and continuing would only produce garbage bytes; a missing block header or an incomplete length field, on the other hand, usually just means the stream was truncated, and the prefix data is still trustworthy.

That split of "where tolerance is allowed and where it is not" is not arbitrary; it matches how DecodeStream.getBytes is consumed: callers usually request a fixed number of bytes based on /Length or image dimensions, and a short array is treated as "the stream ended early".

The asyncGetBytes method

Modern browsers offer DecompressionStream to decode zlib in native code, an order of magnitude faster than decoding bit by bit in JavaScript. asyncGetBytes tries the native path first and falls back on failure:

  async asyncGetBytes() {
    const { decompressed, compressed } =
      await this.asyncGetBytesFromDecompressionStream("deflate");
    if (decompressed) {
      return decompressed;
    }
    // DecompressionStream failed (for example because there are some extra
    // bytes after the end of the compressed data), so we fallback to our
    // decoder.
    // We already get the bytes from the underlying stream, so we just reuse
    // them to avoid get them again.
 
    this.#isAsync = false;
    this.stream = new Stream(
      compressed,
      2 /* = header size (see ctor) */,
      compressed.length,
      this.stream.dict
    );
    this.reset();
    return null;
  }

asyncGetBytesFromDecompressionStream lives in DecodeStream. It reads the underlying stream in full and feeds it to new DecompressionStream("deflate"). Here "deflate" means the zlib wrapper format, not raw DEFLATE (raw is called "deflate-raw" in the Web API), which matches PDF's Flate data exactly.

On failure the compressed bytes returned are the raw bytes already read, so the fallback path does not fetch them again from the network or the file. The new Stream starts at offset 2, skipping the zlib header the constructor already validated, and takes the compressed length; then reset() rewinds the output side, #isAsync is set to false, and null is returned meaning "nothing decoded, please take the synchronous path".

Why fall back instead of propagating the exception? The comment gives a concrete reason: the native implementation errors out when there are extra bytes after the compressed data ends. Wrong /Length values, chained filters and trailing leftovers are common in PDF, and browser implementations tolerate them to different degrees. Handing "standard decoding" to the platform and "non-standard but decodable" back to our own decoder is the division of labour here.

get isAsync merely exposes the private field:

  get isAsync() {
    return this.#isAsync;
  }

BaseStream.isAsync returns false by default; this override reports the real availability. The upper-level Evaluator and PDFDocument use it to decide whether to await stream.asyncGetBytes() first and wrap the whole decoded result in a Stream in one shot. For an image of several megabytes, this path saves the per-block decode overhead on the main thread.

The getImageData method

The image path has its own entry point:

  async getImageData(length, _decoderOptions) {
    const data = await this.asyncGetBytes();
    if (!data) {
      return this.getBytes(length);
    }
    if (data.length <= length) {
      return data;
    }
    return data.subarray(0, length);
  }

It first attempts the whole-buffer async decode. On failure asyncGetBytes returns null, and it falls back to the synchronous getBytes(length). On success it does not return the data as-is — length is the byte count the image declares it needs, so excess is trimmed, and short data is returned unchanged (an image decoder treats short data as an incomplete row).

subarray is a view, not a copy; the returned Uint8Array shares memory with the decode buffer. The caller only reads, so the sharing is safe.

Design note: the async and synchronous paths share one decoding state machine. On fallback the code does not construct a new FlateStream; it swaps the underlying stream in place and resets the cursor, reusing the bytes already read while preserving dict and object identity. When capability detection succeeds, use native code; when it fails, use our own. Both paths are completely transparent to the caller.

Conclusion

FlateStream is PDF.js's entropy decoding layer: it turns a zlib-wrapped DEFLATE bit stream back into a byte sequence for PredictorStream, image decoders, font parsers and the xref parser to consume.

Where It Sits in the Overall Chain

In one sentence: Flate turns "variable-length bits" back into "fixed-length bytes", and Predictor turns "predicted rows" back into "original rows".

Five Design Principles

1. Table lookup beats traversal. Huffman decoding, lengths and distances, and the fixed code tables are all expanded into arrays, leaving no loops or branch trees on the hot path. The price is memory; the payoff is O(1) per symbol.
2. The bit buffer sees requests, not bytes. getBits and getCode take "how many bits" as their only input, and the caller never worries about byte boundaries; leftovers that span bytes live on the instance, so byte reads are naturally minimal.
3. Fault tolerance has boundaries. A missing block header or an incomplete uncompressed length ends with eof and keeps the prefix; a scrambled code table or a reserved block type throws immediately. Each layer owns its own failures and does not spread corrupt data downstream.
4. Platform capability first, semantics as backstop. Use the native implementation when DecompressionStream is available; when the native implementation rejects something like trailing bytes, switch back to the in-house decoder in place and reuse the compressed bytes already read.
5. Align with LZ77's data dependencies. Overlapping copies must proceed byte by byte, forward — the format's semantics demand it; uncompressed blocks have no dependencies, so they are copied as a whole. The two copy styles differ not out of inconsistency but because the data structures differ.

At this point the loop "how compressed bytes become raw bytes again" is closed. The next article looks at PredictorStream, right next door: why the bytes Flate produces still need per-row differencing, and how the five PNG filters turn predicted rows back into pixels.

StreamPDF design principle: anything that can be handled ahead of time is absolutely not deferred to parse time; the server solves all performance problems once. The browser only fetches and renders the data it must.