PDF.js Source Walkthrough: ascii_85_stream

Introduction

This is an article in the "Bytes and Streams" layer, and its subject is the worker-side src/core/ascii_85_stream.js.

Last time we took apart /FlateDecode, the layer that actually performs statistical compression; this article covers /ASCII85Decode from the same layer, and the LZW breakdown is in the companion article released alongside this one. All three are subclasses of DecodeStream and all are created from Parser.makeFilter, but their jobs differ: Flate and LZW perform entropy decoding, while ASCII85 only performs base conversion — turning five printable characters back into four bytes.

It does not compress data; it inflates it to 125% of the original size. So why does PDF support it at all? Because in early networks and mail gateways binary bytes were often escaped, truncated or corrupted, whereas ASCII85 uses only printable characters and can pass safely through plain-text channels. Today it appears mainly in inline images, files produced by older generators, and combinations such as "ASCII85 first, then Flate".

ASCII85 is also called Base85, but it is not the same thing as Base64. Base64 uses 64 characters to represent 6 bits, so four bytes need about 5.33 characters; ASCII85 uses 85 characters to represent about 6.4 bits, so four bytes need only five characters, making it more compact. PDF chose it because all 85 characters are printable and encoding and decoding involve only integer multiply-add, with no lookup table.

In one sentence, its job is: take every five characters in the range 0x21 to 0x75, assemble them base 85 into a 32-bit integer, then split that integer into four bytes in big-endian order.

Module Overview

ascii_85_stream.js is 95 lines in total and contains a single class, Ascii85Stream; the code splits into three blocks:

Where it sits in the chain:

PDF bytes
  → Parser.makeFilter              (reads /Filter /ASCII85Decode)
  → Ascii85Stream(str, maybeLength)
  → DecodeStream.getBytes          (pull on demand, calls readBlock in a loop)
  → FlateStream                    (optional: the next filter in the array)
  → upper layers: image data / content streams

The inheritance chain is BaseStream → DecodeStream → Ascii85Stream. DecodeStream provides buffer, bufferLength, eof, ensureBuffer, getByte and getBytes; Ascii85Stream only has to implement readBlock.

Unlike Flate and LZW, its readBlock handles exactly one group per call, appending at most 4 bytes to the buffer. Flate handles one DEFLATE block at a time, LZW 512 codes, while ASCII85 is "read five characters, write four bytes, return". When an upper layer wants a longer stretch of data, DecodeStream.getBytes calls it repeatedly.

readBlock does not check this.eof internally. This is not an oversight: DecodeStream.getByte and getBytes both test eof before calling, so the normal path never re-enters after the end. Calling readBlock directly would keep consuming the underlying stream, but that is not part of the public contract.

Design note: Making readBlock "one group" instead of "one big chunk" buys extreme simplicity: no group boundary has to be maintained inside a block, no "half a group across blocks" splicing is needed, and #input only needs 5 bytes. The cost is more calls, but ASCII85 is rare in PDF and its data volume is usually small, so the trade is reasonable.

ASCII85 Encoding Rules

The idea behind ASCII85 is direct: four bytes are 32 bits and can represent an integer from 0 to 4294967295; writing that integer in base 85 takes at most 5 digits, because 85 to the fifth power is 4437053125, already greater than 2 to the 32nd power. Adding 33 to each digit lands it in the printable range 0x21 to 0x75, that is ! to u.

Decoding does the reverse: subtract 33 from each of the 5 characters, accumulate them base 85 into an integer, then split the integer back into 4 bytes in big-endian order. Because 85 to the fifth power is slightly larger than 2 to the 32nd power, not every 5-character combination is legal; the largest legal one is s8W-!, which corresponds to 0xFFFFFFFF.

Taking s8W-! as an example, the five characters minus 33 are 82, 23, 54, 12 and 0:

Summing the five terms gives exactly 0xFFFFFFFF, which splits into four bytes of 0xFF. This also shows why 85 to the fifth power must exceed 2 to the 32nd power: otherwise some 32-bit values could not be expressed in five digits.

z is a shortcut provided by the spec: it stands for four 0x00 bytes, saving the five characters of !!!!!. The spec requires z to appear only at the start of a group and never after a partial group; this implementation recognises it only at the first non-whitespace position of a group, which matches the constraint exactly.

Whitespace is ignored entirely during decoding and may appear anywhere, including between the five characters. ~ is the terminator, written ~> in the spec; the decoder recognises only ~, and the > after it is not read by this class but left to the upstream inline-stream length scan.

Chaining with Other Filters

/Filter can be a single name or an array. An array means "decode in order": [/ASCII85Decode /FlateDecode] means strip the ASCII85 wrapper first, then hand the resulting bytes to Flate.

The loop in Parser.makeFilter starts at the first array element and wraps the current stream in a new filter:

stream = this.makeFilter(stream, filter.name, maybeLength, params, cipherTransform);
// After the first stream the `length` variable is invalid.
maybeLength = null;

Only the first layer receives the original Length. When ASCII85Stream sits in the first position of the array it gets maybeLength for the 0.8 estimate; a Flate further along receives null, because after one decoding layer the original length no longer represents the remaining bytes. This detail explains why length estimation is only valid for the first layer and every later filter has to rely on its own buffer-growth strategy.

It also shows up in inline images: if the data between BI and ID declares /ASCII85Decode, decoding starts here; ~> is recognised by the upstream length scan, and this class only decodes up to ~.

When ASCII85 appears alone, its output is the image or content stream data directly; when chained with Flate it is only the first step of "turn printable characters back into binary", and the real decompression is done by the next layer.

Construction and Length Estimation

The constructor

  constructor(str, maybeLength) {
    // Most streams increase in size when decoded, but Ascii85 streams
    // typically shrink by ~20%.
    if (maybeLength) {
      maybeLength *= 0.8;
    }
    super(maybeLength);
 
    this.stream = str;
    this.dict = str.dict;
  }

The constructor does only two things: scale the length hint by 0.8 and keep the underlying stream.

The scaling exists because the input-to-output ratio is fixed at 5 to 4: every 5 characters decode to 4 bytes, so multiplying the compressed data's Length by 0.8 gives a reasonable estimate of the output length. The value is only a hint; DecodeStream uses it to compute the initial buffer capacity and never uses it to limit the actual decoded length.

The DecodeStream constructor rounds the hint up to a power of two as minBufferLength:

After multiplying by 0.8 the value can be fractional, for example 127 becomes 101.6; that is fine: minBufferLength starts at 512 and grows by factors of 2, comparing magnitudes rather than requiring integers. The raw scaled value is also stored in _rawMinBufferLength for StreamsSequenceStream to estimate the initial capacity of a chained stream.

#input is a class field allocated as 5 bytes when the instance is created, and readBlock reuses it every time. It is a pure scratch workspace: every group overwrites the first 5 positions, so it never needs clearing.

Declaring it as a private class field rather than assigning it in the constructor has the same runtime effect; the difference is readability: the field sits at the top of the class body, so a reader sees at a glance that "this class has exactly one mutable group buffer". this.dict is copied straight from the underlying stream to keep dictionary identity along the filter chain; Ascii85Stream never reads any dictionary entry itself.

Decoding

The readBlock method

  readBlock() {
    const TILDA_CHAR = 0x7e; // '~'
    const Z_LOWER_CHAR = 0x7a; // 'z'
    const EOF = -1;
 
    const str = this.stream;
 
    let c = str.getByte();
    while (isWhiteSpace(c)) {
      c = str.getByte();
    }
 
    if (c === EOF || c === TILDA_CHAR) {
      this.eof = true;
      return;
    }
 
    const bufferLength = this.bufferLength;
    let buffer, i;
 
    // special code for z
    if (c === Z_LOWER_CHAR) {
      buffer = this.ensureBuffer(bufferLength + 4);
      buffer.fill(0, bufferLength, bufferLength + 4);
      this.bufferLength += 4;
    } else {
      const input = this.#input;
      input[0] = c;
      for (i = 1; i < 5; ++i) {
        c = str.getByte();
        while (isWhiteSpace(c)) {
          c = str.getByte();
        }
 
        input[i] = c;
 
        if (c === EOF || c === TILDA_CHAR) {
          break;
        }
      }
      buffer = this.ensureBuffer(bufferLength + i - 1);
      this.bufferLength += i - 1;
 
      // partial ending;
      if (i < 5) {
        input.fill(0x21 + 84, i, 5);
        this.eof = true;
      }
      let t = 0;
      for (i = 0; i < 5; ++i) {
        t = t * 85 + (input[i] - 0x21);
      }
 
      for (i = 3; i >= 0; --i) {
        buffer[bufferLength + i] = t & 0xff;
        t >>= 8;
      }
    }
  }

The method is not long, but it packs "skip whitespace", "recognise the terminator", "the z shortcut", "pad a partial group" and "assemble 32 bits" into a single function. Take it apart below.

Execution order

Following the code from top to bottom, a single readBlock has only six steps:

1. Read one byte, skipping whitespace;
2. Check whether it is -1 or ~; if so, set eof and return;
3. Check whether it is z; if so, take the zero-byte shortcut;
4. Otherwise put the first character into input[0], then read four more characters, skipping whitespace and stopping at the terminator;
5. Grow by the number of characters read, i; a partial group is padded with u and sets eof;
6. Accumulate base 85 into a 32-bit integer, then write four bytes in big-endian order.

These six steps have no nested loops and no cross-call state; the only member variable is #input, and the only buffer state modified is bufferLength. That is exactly why the method can be interrupted at any time — after readBlock returns, the next call starts fresh with a new group and does not need to remember "how many characters of the previous group were read".

The six steps are also the order in which to debug. Empty output: first check step 2 for an early ~ or end of stream; wrong byte count: first check whether i in step 5 is right; garbled content: then check whether step 4 missed a whitespace skip, or step 6 wrote the byte order backwards.

Whitespace and the terminator

The method reads one byte on entry and skips whitespace; if that byte is -1 or ~, it sets eof and returns immediately. Two boundaries are worth noting here:

• -1 means the underlying stream is exhausted. A truncated stream with no terminator ends up here too, with the same effect as hitting ~.
• ~ is only a stop signal and is never written into any group. The > after it is not consumed either — this class returns at ~ and leaves > in the underlying stream. That does not matter for a standalone stream; for an inline image, the upstream findASCII85DecodeInlineStreamEnd has already scanned past ~> while locating the length.

The whitespace test uses isWhiteSpace, which recognises only four characters: space 0x20, tab 0x09, line feed 0x0A and carriage return 0x0D. The PDF spec counts NUL, tab, line feed, form feed, carriage return and space as whitespace, but this helper in core_utils covers only four of them; for ASCII85 data, form feed and NUL are extremely rare.

The form feed character 0x0C is an interesting counterexample: if it really appeared in the data it would be treated as the number 0x0C - 0x21, that is a negative value, and would take part in the accumulation and produce a wrong result. In practice generators never write it that way; the discrepancy is more of a footnote for source readers.

The whitespace skip appears twice: once before the group starts and once for each character read inside the group. The second one cannot be dropped, because the spec allows whitespace between the five characters, for example splitting a group across two lines. Both use the same while (isWhiteSpace(c)) loop with identical logic, only at different positions.

The z shortcut branch

If the first non-whitespace character of a group is z, take the shortcut: write four 0x00 bytes straight into the buffer, advance bufferLength by 4, and the method ends.

The line buffer.fill(0, bufferLength, bufferLength + 4) cannot be omitted. ensureBuffer only guarantees capacity, not content; the buffer may be carried over from an earlier block and still hold old bytes. The four zeros must be written explicitly.

The z branch reads no further characters and does not check whether the next character is whitespace. That is correct: z is a complete group by itself, and whatever follows belongs to the next group, handled by the next readBlock.

The implementation recognises z only "at the start of a group". If z appears in the middle of the five characters it is treated as an ordinary character, minus 0x21 giving 89, outside the legal digit range of 0 to 84. This file performs no character validation, so such input quietly produces wrong bytes instead of throwing.

Incidentally, !!!!! can also represent four zero bytes; it just wastes four characters. z is a pure size optimisation. The spec restricts it to group boundaries because, if a decoder allowed it inside a group, it would have to insert a zero group halfway through the five-digit accumulation and the state would become far more complicated.

Five-character groups and partial groups

The ordinary-group branch first puts the first character into input[0], then reads indices 1 to 4. Every character read skips whitespace, and the loop breaks as soon as EOF or ~ is read. Note that input[i] = c is written before the check, so input may briefly contain 255 converted from ~ or -1; but whenever an early break happens, the padding afterwards overwrites those positions.

Once the group is complete (or ended early), the output length is computed as i - 1:

i < 5 means the group was not filled, usually because ~ or the end of the stream was met. In that case input is filled from index i to 4 with 0x21 + 84, that is 0x75, the character u, digit value 84. This is the padding defined by the spec: fill with the largest digit, decode, and take only the first i - 1 bytes — the bytes after that would be discarded anyway.

The padding expression input.fill(0x21 + 84, i, 5) writes "character offset 33" and "largest digit 84" together. It conveys the intent better than writing 0x75 directly: this is not an arbitrary byte but "base character plus largest digit".

A partial group is not an error state but part of normal encoding. ASCII85 encodes in 4-byte groups; a final group shorter than 4 bytes is zero-padded to full length, encoded to 5 characters, and then as many trailing characters as there were padding bytes are removed; the decoder sees fewer than five and thus knows how many were removed, and padding with u restores exactly what the encoder zero-padded. i - 1 is the true number of original bytes in that group.

If i equals 5, all five characters were read, no padding is needed, and eof is not set; the next readBlock goes on to the next group.

32-bit assembly and byte order

The assembly is a single line:

let t = 0;
for (i = 0; i < 5; ++i) {
  t = t * 85 + (input[i] - 0x21);
}

t starts at 0, and each digit multiplies it by 85 and adds the current value. After five digits it holds the 32-bit integer represented by the group. Because t is a normal JS number (double-precision floating point) it does not overflow early the way a 32-bit integer would during accumulation; even if the input is illegal and the result exceeds 0xFFFFFFFF it can still be computed first and truncated afterwards.

Output is big-endian: the high byte goes to index 0 and the low byte to index 3.

for (i = 3; i >= 0; --i) {
  buffer[bufferLength + i] = t & 0xff;
  t >>= 8;
}

t & 0xff extracts the lowest byte and t >>= 8 shifts right by eight. This is a signed right shift, so t is first converted to a 32-bit signed integer; for values with the top bit set the result is negative and the shift sign-extends. The low bytes are unaffected: & 0xff looks only at the lowest eight bits, and successive shifts never change the high bytes already written. So 0xFFFFFFFF is written correctly as four 0xFF bytes, and 0x80000000 as 0x80, 0x00, 0x00, 0x00.

A way to remember big-endian: the five ASCII85 characters are ordered "most significant first", the first character contributing the 85 to the 4th power term. So in the decoded integer the highest byte corresponds to the group where the first character lives, and when writing back to the buffer the highest byte must be written first.

Illegal combinations are not so elegant. For example, five u characters accumulate to 4437053124, beyond 0xFFFFFFFF; & 0xff and >> first truncate it to 32 bits and then output the low four bytes. The spec does not define such input and the implementation does not validate it, so this falls under "illegal input, undefined output".

ensureBuffer(bufferLength + i - 1) guarantees capacity for the "number of output bytes". But for a partial group the loop still writes four positions, from bufferLength + 0 to bufferLength + 3, of which the part beyond i - 1 may be out of range. Those writes are silently discarded by the TypedArray and are not counted into bufferLength anyway; and because a partial group always comes with eof, no later block ever reads those positions. The first i - 1 bytes that really matter are always within capacity.

Growth of the output buffer

ensureBuffer grows in powers of two, and readBlock asks only for bufferLength + 4 (complete group) or bufferLength + i - 1 (partial group). Because the initial capacity is at least 512, the first 128 complete groups never trigger a growth; after that each growth doubles, so the amortised cost per group stays low.

This also means the output buffer of an ASCII85 stream is usually a bit larger than the final data — growth always takes the next power of two. The extra capacity costs no additional allocation, it is just empty tail space in the Uint8Array; only the bytes before bufferLength are valid data.

Results for four input fragments

Applying the rules above to concrete bytes, compare the four inputs below:

The third row deserves a closer look: the four ! represent zeros, and after padding with u the integer is still 0, so all four output bytes are 0 but only the first three are taken. If the encoder had padded with three zero bytes, exactly the one byte corresponding to the last digit was removed, and the decoder recovers it precisely with i - 1. The fourth row shows that writing the same data as a full five characters yields four zero bytes, none missing.

Comparison: ASCII85, ASCIIHex and Flate

All three filters live on the same layer, yet their implementation granularity is completely different:

The comment in AsciiHexStream says "shrink by 50%" because every two characters decode to one byte; Ascii85Stream says "shrink by ~20%" because five characters decode to four bytes. In both cases the length estimate is only a hint, and the actual decoding logic is independent of maybeLength.

The three factors correspond exactly to the respective size ratios: Flate uses 1, ASCII85 0.8, ASCIIHex 0.5. They only affect the initial buffer, yet they can also be read backwards as a hint about whether "this filter makes data larger or smaller".

The granularity difference is more interesting. AsciiHexStream pulls 8000 bytes from the underlying stream at a time and decodes up to 4000; Ascii85Stream decodes a single group at a time. For the same one megabyte of data the latter calls readBlock about 250,000 times while the former needs only about 250. That is not an oversight but a consequence of the two encodings' complexity: ASCII85 needs the five digits of a group computed together and must also handle z and partial groups, so the author chose the most direct single-group implementation. ASCII85 usually only serves as a pre-encoding for Flate and the data volume is limited, so this implementation does not optimise further for throughput.

Another detail is how they end. AsciiHexStream sets eof and breaks on >, but it may already have decoded the first part of the same 8000-byte batch; Ascii85Stream handles only the current group on ~ and leaves the characters after it in the stream untouched. Neither relies on "reading to the end of the stream" to detect the end; the terminator itself is part of the protocol.

Boundaries and Error Tolerance

No character validation. The digit input[i] - 0x21 can be negative or greater than 84, and the implementation checks neither. A normal encoder never produces such characters; corrupted or malicious input yields meaningless bytes rather than an exception. This matches ASCII85's role in PDF: it is only an encoding layer, and the security boundary is not here.

Contrasting this with LZW makes it clearer. LZWStream has a fallback for illegal codes (append the first byte when the code is greater than nextCode), because the code space itself has structure and an out-of-range value can be given a reasonable meaning; ASCII85's character space has no such structure, any character may be a typo or data corruption, and forcing an interpretation would only hide the problem. Both sides choose "do not throw", but for different reasons: one can produce an approximate result, the other cannot even judge.

Missing terminator. For a stream without a ~, reading -1 breaks the inner loop, i is less than 5, and the padding and eof branch runs. The characters already read are still decoded and output, the effect being "give a result if possible". If -1 appears at the start of a group, eof is set and the method returns without producing any bytes.

Repeated terminators. Whatever follows ~ is not read by this class; eof is set the first time. Two ~ in a row also consume only the first.

Empty length hint. When maybeLength is 0, Parser.makeFilter returns a NullStream before construction and logs a warning. So the Ascii85Stream constructor practically never receives 0; even if it did, if (maybeLength) would skip the scaling and super(0) would use the default floor of 512.

Workspace residue. #input is not cleared between two readBlock calls. The fifth character of the previous group may still be in the array, but the next group always starts overwriting from input[0] and fills or pads all five, so the residue is never read.

No validation of z's position. As noted above, z is recognised only at the group start; inside a group it is treated as the digit 89. The spec forbids that form and the implementation chooses not to report an error.

Whitespace between ~>. When scanning the length of an inline stream, the upstream findASCII85DecodeInlineStreamEnd skips whitespace between ~ and > and tolerates a missing >. This class recognises only ~, so whitespace and > after it do not affect it. Each of the two layers handles the anomalies it can handle.

State after eof. Once set, the bytes in buffer can still be taken by the current request; readBlock neither clears the buffer nor rewinds pos. This follows the convention of Flate and LZW: eof only means "stop decoding", and data already decoded remains valid.

Length hint versus reality. maybeLength may be completely wrong, for example when a generator writes the pre-compression length as Length. It only affects the initial capacity; decoding advances by actual characters, and a wildly wrong value costs nothing more than a few extra ensureBuffer growths.

Conclusion

Ascii85Stream is PDF.js's printable-character restoration layer: it restores 5-character groups back into 4 bytes for the next filter or the final consumer.

It is usually not the only filter: alone, its output is directly usable; combined with Flate, it is only the first layer. The chain table below follows the common "ASCII85 first, then Flate" combination.

Where It Sits in the Overall Chain

In one sentence: Ascii85 restores "printable characters" into "bytes", and the Flate behind it then restores "compressed bytes" into "original bytes".

It is optional in the chain: a stream not ASCII85-encoded never passes through this class; when it is present, it usually sits first in the /Filter array, because ASCII85 is the outermost wrapper. That position is also why the 0.8 length estimate is only meaningful for it — every later filter receives maybeLength as null.

Five Design Principles

1. Fixed grouping, no cross-block state inside a group. A group is exactly 5 characters and at most 4 output bytes; readBlock returns once it is done and does not need cross-block dictionary or bit-buffer state like LZW.
2. z as a short circuit for zero groups. Four zero bytes need no base-85 arithmetic; they are written directly with fill. Recognition is limited to the group start, matching the spec constraint.
3. Whitespace and the terminator are handled at character level. Skipping whitespace and detecting ~ both happen where characters are read, the same logic inside and outside a group, with no pre-scan.
4. A partial group is padded with the largest digit, and only valid bytes are output. Padding with u is required by the spec; outputting i - 1 bytes is the grouping semantics. Together they make truncation and normal termination follow the same path.
5. The length estimate only affects the buffer. The 0.8 factor brings the initial capacity close to the real output, but the decoding logic never reads maybeLength, so a wrong estimate costs one extra growth and never affects correctness.

From an implementation standpoint this class has almost no "state": apart from bufferLength and eof, the only mutable thing is the five bytes of #input. That makes it easier to read than Flate and LZW, and easier to port to another language. The cost is low throughput, but for an implementation whose purpose is compatibility, that price is worth paying.

This completes the ASCII85 part as well. Like Flate and LZW, it only does "bytes in, bytes out"; what actually decides whether those bytes are pixels or text is left to the image decoder and content-stream interpreter further up.

StreamPDF design principle: anything that can be handled ahead of time is absolutely not deferred to parse time; the server solves all performance problems once. The browser only fetches and renders the data it must.