PDF.js Source Walkthrough: lzw_stream

Introduction

This is an article in the "Bytes and Streams" layer, and its subject is the worker-side src/core/lzw_stream.js.

The "Bytes and Streams" layer is about PDF filters: how the name written under /Filter in a dictionary is turned by Parser.makeFilter into a DecodeStream that emits bytes on demand. Last time we took apart /FlateDecode; this article deals with LZW and ASCII85, both filter decoders of the same family, with the ASCII85 details in the companion article released alongside this one, so here we focus on LZW.

Filters can be chained. The array /Filter [/ASCII85Decode /FlateDecode] means "decode ASCII85 first, then hand the result to Flate"; the loop in makeFilter starts at the head of the array and wraps one layer at a time, and reads pull inward from the outermost layer, so the actual decoding order still follows the array order. The input of LZWStream is therefore not necessarily the raw bytes of the file; it may be the output of another decoding stream.

Flate is the default compression method of modern PDFs and appears in almost every document; LZW is the dictionary compression PDF inherited from TIFF and GIF, and today it only shows up in a few scanned documents, older generators and some image data. Both perform entropy decoding, but their mechanisms are completely different: Flate relies on Huffman tables plus LZ77 back-references, while LZW relies on a string dictionary that grows as decoding proceeds, and the code length itself changes — it starts at 9 bits and widens step by step to 12 bits as the dictionary expands.

LZW was formalised by Welch in 1984, and PDF 1.0 already listed it as a standard filter. It became widely known through the GIF patent disputes, but inside PDF it has always been a supporting player: the spec's /LZWDecode requires implementations to support /EarlyChange as well, while most modern generators switched to Flate long ago. PDF.js still implements it in full, because when you open a scan from the nineties this code may be the only path that can unpack the content stream.

The value of reading this file is not "knowing how to use LZW" but that it compresses three classic problems into 145 lines: how variable-length codes are read across byte boundaries, how the dictionary grows without allocating objects, and how decoding state survives across blocks. These three things recur in any streaming decoder.

In one sentence, its job is: turn a stream of variable-length codes packed at 9 to 12 bits back into contiguous bytes, through a 4096-entry dictionary it builds itself.

Module Overview

lzw_stream.js is 145 lines in total and contains a single class, LZWStream; the code splits into four blocks:

Where it sits in the chain:

PDF bytes
  → Parser.makeFilter              (reads /Filter /LZWDecode)
  → LZWStream(str, maybeLength, earlyChange)
  → PredictorStream                (optional: /DecodeParms row prediction)
  → DecodeStream.getBytes          (pull on demand, calls readBlock in a loop)
  → upper layers: image data / font files / inline streams

The inheritance chain is BaseStream → DecodeStream → LZWStream. DecodeStream provides buffer, bufferLength, eof, ensureBuffer, getByte and getBytes; LZWStream only has to implement one readBlock, appending a slice of decoded output to the tail of buffer each time. Upper layers trigger readBlock repeatedly for the length they need, so decoding is lazy: a stream nobody reads is never decoded.

LZWStream does not override getBytes, getByte or ensureBuffer, nor does it offer an asynchronous fast path like FlateStream. All the complexity is packed into readBlock, and the outside only sees "a decoding block that advances on demand". This division of labour keeps the minimum cost of adding a filter very low: supporting a new encoding means writing one more readBlock.

The contract of DecodeStream.getBytes is also worth spelling out: the caller passes a length, and it calls readBlock repeatedly from pos until bufferLength covers the target or eof is set; what it returns is a subarray, not a copy. LZWStream is only responsible for moving bufferLength forward and never moves pos. If you confuse these two cursors while reading the source, you will treat "already decoded" and "already consumed" as the same thing.

The cost falls on state management. DecodeStream only knows about buffer and eof, and has no concept of "code length" or "dictionary", so LZWStream has to guarantee on its own: the code length is consistent between blocks, the leftover bits in the bit buffer are not lost, and the dictionary is not rebuilt. lzwState exists for exactly these three things.

Design note: The granularity of readBlock is "512 codes", not "the whole stream" and certainly not "a fixed number of bytes". Because the number of bytes each LZW code expands to is not fixed, the only way to slice here is by code count; between blocks, lzwState carries the code length, nextCode, prevCode and the current sequence. Slicing bounds the work of a single call while preserving the dictionary across blocks.

LZW Code-Value Conventions

PDF's LZW (LZWDecode) shares its algorithm with TIFF LZW, but the packing differs: PDF mandates most significant bit first (MSB first), while TIFF is least significant bit first. That shows clearly in readBits.

The code-value space is cut into four ranges:

258 is the first code number that can be freely allocated. The dictionary initially holds only the 256 literal entries 0 to 255, and from 258 on every code read adds one entry, so nextCode starts at 258 and increments.

MSB-first has another practical consequence: a 9-bit code may straddle two bytes, with the high-order byte that arrives first on top and the later byte below. In readBits, the pair cachedData = (cachedData << 8) | c and cachedData >>> bitsCached — one going in, one coming out — serves exactly that order. If you copy a TIFF LSB-first implementation to read PDF data, you get a pile of seemingly random bytes, and the problem often only surfaces at the first dictionary code — the literal codes before it are only 8 bits wide, so a wrong bit order is invisible.

EarlyChange is the switch controlled by /DecodeParms in the PDF spec: it determines whether the encoder widens the code length one code early or only when needed. Parser.makeFilter reads it and defaults to 1:

        case "LZW":
        case "LZWDecode":
          let earlyChange = 1;
          if (params) {
            if (params.has("EarlyChange")) {
              earlyChange = params.get("EarlyChange");
            }
            return new PredictorStream(
              new LZWStream(stream, maybeLength, earlyChange),
              maybeLength,
              params
            );
          }
          return new LZWStream(stream, maybeLength, earlyChange);

Two things to note. First, earlyChange is a parameter the decoder must know: it does not change the dictionary contents, only the moment when the code length switches, and getting it wrong desynchronises the whole remainder from the widening point on. Second, whenever /DecodeParms is present, a PredictorStream is wrapped around the result unconditionally, even if it only carries EarlyChange and has no Predictor field at all; PredictorStream is lenient about missing parameters, so this layer does not alter any bytes.

Unlike Flate, LZWStream makes no adjustment to maybeLength and passes Length straight to DecodeStream as the initial buffer hint.

Dictionary Representation

The constructor

  constructor(str, maybeLength, earlyChange) {
    super(maybeLength);
 
    this.stream = str;
    this.dict = str.dict;
    this.cachedData = 0;
    this.bitsCached = 0;
 
    const maxLzwDictionarySize = 4096;
    const lzwState = {
      earlyChange,
      codeLength: 9,
      nextCode: 258,
      dictionaryValues: new Uint8Array(maxLzwDictionarySize),
      dictionaryLengths: new Uint16Array(maxLzwDictionarySize),
      dictionaryPrevCodes: new Uint16Array(maxLzwDictionarySize),
      currentSequence: new Uint8Array(maxLzwDictionarySize),
      currentSequenceLength: 0,
    };
    for (let i = 0; i < 256; ++i) {
      lzwState.dictionaryValues[i] = i;
      lzwState.dictionaryLengths[i] = 1;
    }
    this.lzwState = lzwState;
  }

The constructor does three things: keep the underlying stream, initialise the bit buffer, and build the dictionary.

The dictionary is not a Map and not an array of strings, but four fixed-length TypedArrays plus a length cursor. This is the single most important design choice in the implementation:

The indices of the first three arrays are code numbers, up to 4096 entries, matching the 12-bit code ceiling. TypedArrays are used instead of object arrays because every code on the LZW hot path reads and writes these tables: a fixed-length, contiguous, unboxed array is an order of magnitude faster than a Map or string concatenation, and it allocates no object per dictionary entry.

Initialisation only fills 0 to 255: the literal code i has value i and length 1, and the prefix code is left at 0 to mean "no prefix". currentSequence is all zeros at this point; it takes no part in initialisation and is only filled byte by byte during decoding.

One thing is easy to misread: dictionaryValues stores not the first byte of an entry but its last byte. This comes up again and again in the dictionary update and the backtracking reconstruction below, so remember the conclusion now: a new entry equals "the sequence of the previous code" plus "the first byte of the current sequence", so the first byte of the current sequence happens to be the last byte of the new entry.

prevCode is not initialised in lzwState and reads as undefined. That causes no problem, because readBlock decides "is there a previous sequence" from whether currentSequenceLength is greater than 0; on the first code that value is 0 and the dictionary update is skipped.

Reusing the sequence workspace

currentSequence is the only one of the four arrays that is overwritten repeatedly. Every decoded code fills it first and then copies the whole stretch into the output buffer. Its capacity of 4096 holds any dictionary entry: the first dynamic entry (258) is 2 bytes long, each new entry adds 1, and by 4095 the length is about 3839, still within 4096.

Reusing one workspace instead of allocating a new array per code is the second allocation-saving device on the hot path (the first being TypedArrays themselves). The KwKwK branch appends a single byte directly to this workspace, with no temporary variables; the table-lookup branch first locates the tail by length, then fills forward byte by byte toward the front. Neither branch produces garbage objects, which is the key to LZW decoding running steadily inside a Worker.

Bit Reading

The readBits method

  readBits(n) {
    let bitsCached = this.bitsCached;
    let cachedData = this.cachedData;
    while (bitsCached < n) {
      const c = this.stream.getByte();
      if (c === -1) {
        this.eof = true;
        return null;
      }
      cachedData = (cachedData << 8) | c;
      bitsCached += 8;
    }
    this.bitsCached = bitsCached -= n;
    this.cachedData = cachedData;
    return (cachedData >>> bitsCached) & ((1 << n) - 1);
  }

This is a standard MSB-first bit buffer:

1. If there are not enough bits, pull one byte from the underlying stream;
2. When appending a byte, shift the existing bits left by 8 and put the new byte in the lowest 8 bits;
3. When extracting, shift the buffer right to the "remaining bits" position, then mask off the low n bits;
4. The remaining bits stay in bitsCached and carry on across calls and blocks.

cachedData never trims the high bits it has already consumed; it keeps growing leftward with history bits. That is safe: a read only cares about the low window given by bitsCached, the high bits are residue and the mask discards them, and JS bitwise operations truncate to 32 bits so there is no overflow from having too many bits. Across blocks bitsCached is at most 7, and with the 8 bits appended in this call that covers a 12-bit code comfortably.

An example: the buffer already holds 5 bits (bitsCached is 5) and this call wants 9. The loop appends one byte, bitsCached becomes 13, and the low 13 bits of cachedData are "5 old bits + 8 new bits". The return value is (cachedData >>> 4) & 0x1ff, which takes the 9 bits starting from position 4 upwards; the remaining low 4 bits stay in the buffer for the next read. No per-bit shifting happens anywhere in this process — just one shift and one mask.

c === -1 means the underlying stream is exhausted, in which case eof is set and null is returned. Note that it returns null, not 0 and not an exception; how the caller handles that value is a detail expanded in the "Edge Cases" section later.

The bit buffer lives on the instance rather than in a local variable of readBlock because codes may straddle blocks: a block can end in the middle of a code, and the remaining bits must be left for the next readBlock. This is also why readBits cannot simply wrap getBits — DecodeStream has no bit-level interface.

The upper bound of the bit buffer

bitsCached drops by n after every call, so at most 11 bits are left between calls (after reading a 12-bit code). If that is not enough when entering the loop, a single 8-bit top-up suffices: 11 plus 8 is 19, more than enough for 12 bits. The while is written as a loop rather than an if because it does not assume callers only read up to 12 bits; even someone reading 20 bits is padded correctly (although the 32-bit mask would then break, and this file never does that).

1 << n becomes 0 when n is 32 and the mask breaks; at n of 12 the result is 4096 and everything is fine. This is another implicit precondition behind pinning the code-length ceiling at 12 — the width of bitwise arithmetic is sufficient.

Now the lifetime of cachedData: it is never cleared, only shifted left and filled with new bytes. JS bitwise operations automatically convert their operands to 32-bit signed integers, so shifting past 32 bits drops the high bits while the low bitsCached window stays intact. The cost is wasted bit width; the benefit is that no bookkeeping is needed beyond "how many bits are valid".

The Decoding Loop

The readBlock method

First the preparation before entering the loop:

  readBlock() {
    const blockSize = 512,
      decodedSizeDelta = blockSize;
    let estimatedDecodedSize = blockSize * 2;
    let i, j, q;
 
    const { lzwState } = this;
    const earlyChange = lzwState.earlyChange;
    let nextCode = lzwState.nextCode;
    const dictionaryValues = lzwState.dictionaryValues;
    const dictionaryLengths = lzwState.dictionaryLengths;
    const dictionaryPrevCodes = lzwState.dictionaryPrevCodes;
    let codeLength = lzwState.codeLength;
    let prevCode = lzwState.prevCode;
    const currentSequence = lzwState.currentSequence;
    let currentSequenceLength = lzwState.currentSequenceLength;
 
    let decodedLength = 0;
    let currentBufferLength = this.bufferLength;
    let buffer = this.ensureBuffer(this.bufferLength + estimatedDecodedSize);

All cross-block state is pulled into local variables first: code length, next code number, prefix code, the three arrays, the current sequence. This is not a matter of style but a hot-path optimisation — every code inside the loop uses these values, and going through this.lzwState to fetch the properties each time would add visible overhead. At the end of the loop they are written back in one go.

blockSize is set to 512 and estimatedDecodedSize is initially reserved as 1024. ensureBuffer grows in powers of two, so this estimate is only a hint that "at least this much is enough".

Each incoming code goes down one of four paths:

    for (i = 0; i < blockSize; i++) {
      const code = this.readBits(codeLength);
      const hasPrev = currentSequenceLength > 0;
      if (code < 256) {
        currentSequence[0] = code;
        currentSequenceLength = 1;
      } else if (code >= 258) {
        if (code < nextCode) {
          currentSequenceLength = dictionaryLengths[code];
          for (j = currentSequenceLength - 1, q = code; j >= 0; j--) {
            currentSequence[j] = dictionaryValues[q];
            q = dictionaryPrevCodes[q];
          }
        } else {
          currentSequence[currentSequenceLength++] = currentSequence[0];
        }
      } else if (code === 256) {
        codeLength = 9;
        nextCode = 258;
        currentSequenceLength = 0;
        continue;
      } else {
        this.eof = true;
        delete this.lzwState;
        break;
      }

Literals: when code < 256, the current sequence is that single byte and its length is set to 1. Note that currentSequence[0] is later used by the dictionary update as "the first byte of the current sequence", so it cannot be omitted here.

Dictionary entries: when code >= 258, first check code < nextCode. If it holds, the code has already been built, so look it up directly: take the length, then fill currentSequence from the tail forward along the prefix chain. In the loop j counts down from length minus 1 while q walks back from the code number along dictionaryPrevCodes, writing dictionaryValues[q] at the current tail position until it reaches a base code in 0 to 255. Because dictionaryValues holds the last byte of an entry, filling backwards assembles the byte sequence in the correct order.

The KwKwK special case: code === nextCode is LZW's classic boundary. It means "this code has not been built yet, but the encoder is already using it". A table lookup is impossible, but its content is determined: it equals the sequence of the previous code plus the first byte of the current sequence — and the current sequence has not been updated yet at this point, so it still holds exactly the result decoded from the previous code. Hence the single line currentSequence[currentSequenceLength++] = currentSequence[0] completes the append. The implementation uses a plain else as the fallback, so an illegal code greater than nextCode also lands here and likewise appends the first byte; a conforming encoder never produces such a code.

Clear: when code === 256, the code length, nextCode and the sequence length are reset, then continue runs. The continue is crucial: it skips both the dictionary update and the byte output that follow. The dictionary arrays are not zeroed, but once nextCode is back at 258 the old entries get overwritten one by one starting from 258; and the lookup is guarded by code < nextCode beforehand, so the stale 4096 entries cannot cause a misread. prevCode is not reset either, but currentSequenceLength of 0 makes the next hasPrev false, so the stale prefix is never used.

EOD: the final branch, the one other than 256 and 257, is in fact only 257. It sets eof, deletes this.lzwState to release the four arrays, and breaks out of the loop. One detail is worth noting: lzwState has already been destructured into locals at the top of the method, so delete this.lzwState only severs the reference from the instance to the object; the write-back lines outside the loop still operate on that "orphaned" object without error. What is actually released is the instance's reference, after which the arrays become collectible.

A decoding trace

Chaining those rules together, walk through five codes. The input codes are 65, 66, 258, 259, 66 in order:

Concatenated, the five outputs are ABABBAB.

This table illustrates three things. First, when 258 is created at step 2, its prefix is "the sequence A of the previous code 65", not the sequence of the current code; that is what prevCode is for. Second, when 258 is read at step 3 it has already been built at step 2, so 258 < nextCode holds and the lookup branch runs, walking the prefix chain from 258 back to 65 and filling AB from the tail. Third, the last byte of every new entry is the first byte of the current sequence: the last byte of 258 is B, of 259 is A, of 260 is B — which explains exactly why the code is written dictionaryValues[nextCode] = currentSequence[0].

If step 3 read 259 instead of 258 — that is, code === nextCode — KwKwK kicks in: 259 has not been built, but by the rules it must equal "the sequence AB of 258" plus "the first byte of the current sequence", and the current sequence at that moment is still B from the previous step, so appending gives ABB. The single-byte append in the implementation saves a full table lookup.

Dictionary maintenance happens after the code has been handled:

if (hasPrev) {
  dictionaryPrevCodes[nextCode] = prevCode;
  dictionaryLengths[nextCode] = dictionaryLengths[prevCode] + 1;
  dictionaryValues[nextCode] = currentSequence[0];
  nextCode++;
  codeLength =
    (nextCode + earlyChange) & (nextCode + earlyChange - 1)
      ? codeLength
      : Math.min(Math.log(nextCode + earlyChange) / 0.6931471805599453 + 1, 12) | 0;
}
prevCode = code;

A new entry has three parts: the prefix is prevCode, the length is the previous entry's length plus 1, and the value is the first byte of the current sequence. prevCode is the previously read code and currentSequence[0] is the first byte decoded from the current code; concatenating the two is exactly the standard LZW table-building rule.

hasPrev is computed before each code is read and means "did the previous step leave a valid sequence behind". The first code, and the first code after a Clear, have no previous sequence to build an entry from and must be skipped; otherwise an empty or stale prevCode would be used to fill the table.

Code-length growth and earlyChange

The line that updates the code length is the densest expression in the whole file. Broken down:

• (nextCode + earlyChange) & (nextCode + earlyChange - 1): checks whether the number is a power of two. A bitwise AND gives 0 exactly when only one bit is set, so the negated condition means "is a power of two".
• Math.log(x) / 0.6931471805599453 + 1: 0.693... is the literal value of Math.LN2, so this expression is log2(x) + 1, which when truncated gives the number of bits needed to represent x.
• Math.min(..., 12): code length is at most 12 bits, matching a 4096-entry dictionary.
• | 0: truncates the floating-point result to an integer.

earlyChange only affects when widening happens:

With earlyChange of 0 this is standard LZW: only when the next code number actually reaches 512, which no longer fits in 9 bits, does it switch to 10 bits. With 1 it is one code early: nextCode at 511 already switches to 10 bits, because the encoder widens early as well. Decoder and encoder must use the same value, otherwise every bit after the switch point is re-sliced incorrectly.

The same rule continues to apply at 11 and 12 bits: with earlyChange of 1 the switches happen at nextCode 1023 and 2047; with 0 they happen at 1024 and 2048. 12 bits is the end of the road; there are no 13-bit codes — the dictionary ceiling is 4096 entries, and Math.min pins the code length at 12 rather than adding another branch to the power-of-two test.

Looking at the three steps together: 9 bits represent 0 to 511, 10 bits represent 0 to 1023, 11 bits represent 0 to 2047, and 12 bits represent 0 to 4095. The switch point is exactly the moment when "the code number is about to exceed what the current width can represent", and earlyChange simply moves that moment one code earlier across the board.

There is one boundary that is not handled explicitly: the dictionary arrays are 4096 long with legal indices 0 to 4095. Should nextCode climb all the way to 4096, writing dictionaryPrevCodes[4096] would be out of range. TypedArrays silently discard out-of-range writes instead of throwing; and a conforming encoder emits a Clear before the dictionary fills up, so that line is in practice an implicit safeguard rather than a defect.

Output and buffer growth

Only after the code has been handled and the dictionary updated is the current sequence written into the output buffer:

      decodedLength += currentSequenceLength;
      if (estimatedDecodedSize < decodedLength) {
        do {
          estimatedDecodedSize += decodedSizeDelta;
        } while (estimatedDecodedSize < decodedLength);
        buffer = this.ensureBuffer(this.bufferLength + estimatedDecodedSize);
      }
      for (j = 0; j < currentSequenceLength; j++) {
        buffer[currentBufferLength++] = currentSequence[j];
      }
    }
    lzwState.nextCode = nextCode;
    lzwState.codeLength = codeLength;
    lzwState.prevCode = prevCode;
    lzwState.currentSequenceLength = currentSequenceLength;
 
    this.bufferLength = currentBufferLength;

decodedLength is the number of bytes this block has decoded so far; when the estimate is too small it is raised in steps of 512 until it covers that number, and then ensureBuffer grows the buffer. You cannot estimate this as "number of codes times a fixed length", because each code's length rises from 1 all the way to thousands of bytes; accumulating the real decoded length is the only reliable approach.

The growth request is written as this.bufferLength + estimatedDecodedSize, that is always relative to the start of the block; currentBufferLength is the in-block cursor that advances byte by byte. At the end of the block this.bufferLength is committed once, and the object property is not touched inside the block. This is the same pattern as the main loop in Flate.

The loop always runs 512 times unless EOD arrives early. Even if one code decodes to thousands of bytes, the block still ends at 512 codes; even if 512 codes decode to only a few dozen bytes, it does not read more. This "quantify by code count" slicing bounds the number of code reads in a single readBlock, while the output volume is determined by the data itself.

buffer is the reference returned by ensureBuffer; on growth that method has already written the new array back to this.buffer, and the local buffer exists only so the rest of the block's writes do not go through the property. That is why the block end does not need to commit buffer, only bufferLength.

Why 512

The number 512 has no meaning in the spec; it is the batch size this implementation settled on. Too small and readBlock is called more often, spreading the destructuring at the top and the write-back at the bottom over every code; too large and a single call may decode hundreds of KB, breaking the granularity of "decode on demand" — a caller that needs 64 bytes would be forced through a whole batch of codes. The initial estimate of 1024 and the growth step of 512 follow the same reasoning: neither call ensureBuffer per code nor let the reservation balloon.

Design note: The dictionary, code length, prefix code and current sequence are all cross-block state, yet inside readBlock they all degrade into local variables. Localisation brings two benefits: no property access in the hot loop, and the write-back happens at the end of the block, so the state is never observable from outside mid-block. The cost is a set of boilerplate destructuring and assignments at the top and bottom of the function, a deliberate readability price paid for performance.

Comparison: LZW and Flate

Both are subclasses of DecodeStream with identical interfaces, yet internally they take two different roads:

One intuitive difference is "state lifetime". Flate's Huffman tables belong to the current block and are discarded when it ends; LZW's dictionary belongs to the whole stream and keeps growing unless there is a Clear. This is also why LZW must hang the four arrays off the instance.

In compression ratio LZW is usually worse than Flate, especially on text and content streams with many repeating patterns; its advantages are a simple decoder and a fixed memory footprint (4096 entries), needing neither a Huffman table nor block boundaries. PDF.js keeps it mainly for compatibility with old documents, not because it is faster. The per-code cost is higher too: every LZW code goes through readBits, branch decisions and dictionary writes, whereas Flate's fixed-table path does one lookup and one array write.

One interesting detail is what drives them: Flate's readBlock is driven by the block header, LZW's by code count. The amount of data handled per call is decided by the compressed file in the former case and by an implementation constant (512) in the latter. That also explains the difference in when they set eof — Flate can foresee the end from BFINAL in the block header, while LZW only knows once it actually reads EOD or runs out of stream.

They also share ground: both rely only on DecodeStream's buffering protocol, neither parses PDF syntax, and neither cares whether the data ultimately becomes pixels, glyphs or a content stream. The filter layer's boundary is therefore very clean — input is bytes, output is bytes, and failures surface only as eof or an exception.

Edge Cases

Missing EOD. readBits returns null when the underlying stream is exhausted, and readBlock has no separate check for null. In JS, null converted for a numeric comparison becomes 0, so code < 256 holds, the literal branch runs, and 0 is written into the current sequence. The result: a truncated stream gets a number of zero bytes padded into its final block instead of stopping immediately. eof has already been set by then and upper layers will not call a new readBlock, so at most the current block (up to 512 codes) is affected. This is a lenient behaviour worth noticing when reading the source.

EOD arrives early. After delete this.lzwState the instance has no dictionary any more, and any later readBlock would fail at destructuring; but eof is already set, and the condition in DecodeStream.getBytes is "keep reading only while not at eof", so it never re-enters. The real benefit of the deletion is that the four arrays become collectible as early as possible.

Clear at any position. Clear does not require alignment to any boundary and can appear anywhere in the code stream. The implementation's continue skips this round's dictionary update and output, and the state reset is complete: code length, nextCode and sequence length all return to their initial values together.

The first code after Clear. It has no previous sequence and therefore builds no entry; from the second code on, the dictionary grows from 258 again. The encoder may emit Clear at any position, and the decoder need not wait for any alignment.

A full dictionary. As described above, once nextCode reaches 4096 the out-of-range write is ignored by the TypedArray, the code length stays at 12, and decoding can continue until the encoder sends a Clear. The spec requires the encoder to Clear before filling up, so a normal file never gets here.

Combination with a predictor. Parser.makeFilter always wraps LZWStream in a PredictorStream when /DecodeParms is present. So the output of LZWStream is not necessarily the final data: if the parameters contain Predictor, rows still have to be restored through PNG or TIFF prediction first. LZWStream is only responsible for turning the code stream into the correct byte sequence; row prediction belongs to another layer.

The semantics of eof. It is set at only two moments: when EOD is read, or when the underlying stream runs out. DecodeStream.getBytes stops calling readBlock after seeing eof, but bytes already written into buffer can still be taken by the current request. This matches Flate's "return after the block is done": eof means "stop decoding", not "everything decoded before is void".

Three common misreadings

First, treating dictionaryValues as the first byte of an entry. It holds the last byte; the first byte can only be obtained by walking the prefix chain back to a base code. Get that backwards and the reverse-fill loop makes no sense.

Second, treating earlyChange as a dictionary parameter. It changes no entry's content, only the timing of the code-length switch; when the two sides disagree the dictionary itself is still "correct", but the bit stream is sliced wrongly from the switch point on.

Third, assuming Clear wipes the four arrays. The implementation only resets codeLength, nextCode and the sequence length; old entries are invalidated by the code < nextCode guard and then overwritten from 258 by new ones. That saves a 4096-entry zeroing pass and avoids a large memory write on the hot path.

If truncation handling were to be made stricter, the minimal change would be to check code === null at the top of the loop and break; the existing implementation chooses "pad with zeros and finish", consistent with the lenient policy PDF.js applies in other filters: better to hand back a result with a tail than to fail a whole page render because of one missing EOD.

Conclusion

LZWStream is PDF.js's dictionary entropy decoding layer: it turns 9- to 12-bit variable-length codes back into a byte sequence and supplies it upward to PredictorStream, image decoders and font parsers.

Where It Sits in the Overall Chain

In one sentence: LZWStream translates "codes" into "sequences", and PredictorStream then restores "predicted rows" into "original rows".

From an engineering standpoint this is a textbook case of "fixed memory in exchange for determinism": the four 4096-entry arrays are allocated once at construction and no objects are requested during decoding; the bit buffer is two numbers; the output buffer grows block by block. For a Worker that may handle hundreds of documents at once, this predictable memory curve matters more than a maximal compression ratio.

Five Design Principles

1. Localise state, commit once at the block end. Code length, nextCode, prevCode and sequence length are all pulled into locals at the top of the function, the loop touches only locals, and everything is written back once at the end of the block. Cross-block continuity and hot-path performance hold at the same time.
2. A fixed-length array dictionary, not a map or strings. Three 4096-entry TypedArrays are addressed directly by code number, so both building and lookup are constant time and no object is allocated per entry. For streams with tens of thousands of codes this is an order-of-magnitude difference.
3. Fill backwards, output forwards. dictionaryValues stores only the last byte of each entry; reconstruction fills from length minus 1 downwards and the prefix chain naturally yields the sequence in order. Storing the last byte rather than the whole string compresses each entry to three numbers.
4. Code-length switching must match the encoder exactly. Widening happens only when (nextCode + earlyChange) is a power of two, and earlyChange defaults to 1. The parameter changes no dictionary content, only the bit-slicing point, and one wrong code desynchronises everything that follows.
5. Chunk by code count, grow by real length. LZW code length varies, so the block size can only count codes (512); the output buffer grows in steps based on the bytes actually decoded in this block. The two kinds of "quantity" are managed separately, so the estimate never underestimates.

Put the five together and they serve one goal: keep "bit slicing" and "byte reconstruction" out of each other's way. Code length handles slicing, the dictionary handles translation, the buffer handles output, and the three only hand off through lzwState. The shortest path through this implementation is therefore: confirm the code length, then look at nextCode, and only then look at dictionary contents.

This completes the loop "how dictionary compression turns back into original bytes". ASCII85 in the same layer takes another road: it performs no statistical compression and only turns five printable characters back into four bytes; that breakdown lives in the companion article released alongside this one.

StreamPDF design principle: anything that can be handled ahead of time is absolutely not deferred to parse time; the server solves all performance problems once. The browser only fetches and renders the data it must.