PDF.js Source Walkthrough: decrypt_stream

Introduction

This entry in the "Bytes and Streams" layer occupies a somewhat special position; its subject is src/core/decrypt_stream.js.

The Flate, Brotli and Predictor articles mentioned earlier all sit on the compression chain: the input is encoded bytes, the output is original bytes, and the goal is to restore the data itself. DecryptStream sits on a different chain — the decryption chain. It does not decompress and does not do row prediction; it only restores ciphertext bytes to plaintext through a cryptographic algorithm. And the restored plaintext often still wraps Flate or Brotli, which must then be handed to the filters that follow. So it is not a member of the compression chain but a gate in front of it.

The authority for this gate is Section 7.6 of the PDF specification. Streams and strings in encrypted documents are encrypted by default, and the standard security handler picks a concrete algorithm according to the revision and the encryption dictionary: earlier revisions use RC4, later ones introduce AES, and key lengths run from 40 bits all the way to 256 bits. The unit of encryption is the object, so every stream and every string is handled separately with a key derived from the object number and generation. CipherTransform in crypto.js is responsible for that derivation; the subject of this article is only the last step: take a ciphertext block, call the cipher object, and hand out a plaintext block.

The action that actually puts DecryptStream into the chain happens in Parser.readStream:

stream = stream.makeSubStream(startPos, length, dict);
const filter = dict.get('F', 'Filter');
// Streams that explicitly use `/Crypt` are decrypted in the filter chain,
// so avoid applying the default stream cipher up-front.
if (cipherTransform && !this.#hasCryptFilter(filter)) {
  stream = cipherTransform.createStream(stream, length);
}

In other words, default decryption happens before filters. Only when /Crypt is written explicitly in the /Filter array does the parser skip this step and let decryption take part in dispatch as one link of the filter chain. Both routes end up constructing a DecryptStream; the difference is only where it is constructed and which crypto filter is chosen.

decrypt_stream.js is only 55 lines: one class, one constant, three methods. It is one of the shortest streams in this series, yet it sets out in plain sight the key trade-offs of streaming decryption: how big a chunk is, how to know it is the last one, who receives the last-chunk flag, and at which layer stream identity should stop.

In one sentence: it decrypts the ciphertext stream in 512-byte slices and only marks the last chunk as final once it has confirmed that the end of the stream has been reached.

Module Overview

The file splits into five blocks:

Where it sits in the chain:

PDF bytes
  → Parser.readStream
  → ChunkedStream / Stream
  → CipherTransform.createStream          (crypto.js, chooses RC4 or AES)
  → DecryptStream(str, length, decrypt)   (this file)
  → Parser.makeFilter                     (Flate / Brotli / DCT ...)
  → upper layers: object streams / content streams / images / fonts

Inheritance is BaseStream → DecodeStream → DecryptStream. Strictly speaking decryption is not decoding, yet DecryptStream still chooses to hang under DecodeStream, because it needs exactly the same things: a growable output buffer, a read cursor, a lazily driven getBytes, and a readBlock implemented by the subclass. Reusing that skeleton is far cheaper than writing another read interface.

The contract the parent provides is short: readBlock appends a segment of data to the end of buffer each time, and sets eof to true when the source is exhausted; getBytes triggers it repeatedly according to the length the caller wants, and on return gives a zero-copy view from pos to the end. DecryptStream's implementation stays entirely within that contract, with no additional read methods.

Design note: Cryptographic algorithms have block boundaries of their own — AES is a fixed 16 bytes, RC4 is byte by byte — while the length the caller wants is arbitrary. DecryptStream separates the two with a fixed 512-byte chunk, and then relies on the cipher object's own residual buffer to absorb ragged tails. The stream layer only moves bytes; it never tries to make every read align with a cipher block.

Why Decryption Comes Before Filters

The order is not an implementation preference; the format dictates it.

Encryption in a PDF applies to the raw bytes of a stream object, that is, the actual string of data written between stream and endstream in the file; /Filter describes how that data should be decompressed or deserialised after decryption. The two describe two stages of the same piece of data. An encrypted Flate content stream is ciphertext in the file; after decryption you get DEFLATE data with a zlib header; only after FlateStream do you get page content. The order can only be decrypt first, decompress second.

The reverse does not work either. A filter needs to read features of the data itself to determine the format, such as the two-byte zlib header, JPEG marker segments, or the ASCII85 character set; but the beginning of ciphertext is an AES initialisation vector or random bytes produced by an RC4 keystream, and a filter can find no trustworthy structure in it. Putting decryption after filters would amount to asking a decoder to make a format judgement on a random byte stream, which is untenable in principle.

Parser.readStream therefore places default decryption before makeFilter:

1. Cut the raw data section designated by startPos and length out of the file.
2. If an encryption context exists and the filter chain has no /Crypt, first wrap a DecryptStream with CipherTransform.createStream.
3. Then wrap decoders layer by layer according to the /Filter array.
4. The upper layer reads from the outermost layer, while the actual data flow is ciphertext → plaintext → decompressed → prediction restored.

So why still allow /Crypt to appear in the filter chain? Because the specification lets a generator declare decryption explicitly as a filter, and use /Name in /DecodeParms to designate a named crypto filter, or Identity to say this section is not encrypted. Seeing /Crypt, the parser skips the default wrapping and leaves decryption to makeFilter's dispatch, avoiding decrypting the same data twice. Both routes end on DecryptStream; the difference is only who constructs it and which crypto filter is used.

This also explains why DecryptStream is worth inheriting from DecodeStream: it is not a fixed prefix to the filter chain but an ordinary node that can appear anywhere in the chain. As long as the read interface is the same, it works wherever it sits.

Construction and Chunk Granularity

The chunkSize Constant

const chunkSize = 512;

This number has no direct relationship with the 16-byte AES block, nor is it a read unit prescribed by the PDF specification. It is the I/O granularity chosen by this layer, and it serves three goals at once.

First, it bounds the temporary memory allocated per call. 512 bytes is small enough that even a multi-megabyte image stream will not occupy a large extra buffer because of one over-read; it is also large enough to amortise the per-call overhead of getBytes and decrypt.

Second, it stays friendly to cipher blocks. 512 is a multiple of 16, so in most cases each fetch lands exactly on an AES block boundary and the cipher object hardly ever needs to keep residue across calls. This is not a correctness requirement — decryptBlock would keep an incomplete block for the next call anyway — but it makes the hot path smoother.

Third, it leaves room for last chunk detection. To know whether the current chunk is the last one you have to look one chunk ahead; the smaller the chunk, the cheaper the lookahead and the shorter the latency for the first chunk of data.

DecodeStream's minBufferLength also defaults to 512. The numbers are the same but the meanings differ: one decides where to read from, the other decides how big the output buffer starts before doubling. They coincide by accident; do not treat them as the same concept when reading the source.

The constructor Method

  constructor(str, maybeLength, decrypt) {
    super(maybeLength);
 
    this.stream = str;
    this.dict = str.dict;
    this.decrypt = decrypt;
  }

The constructor stores only three things.

maybeLength is handed to the parent to compute the initial capacity of the output buffer. It comes from the stream dictionary's /Length and is the ciphertext length, not the plaintext length. Since the first 16 bytes of an AES stream are the initialisation vector and the last block also drops 1 to 16 bytes of padding, the plaintext is always shorter than this value. The parent treats it as a capacity hint rather than an exact length, which fits perfectly.

this.stream is the underlying ciphertext stream; it may be a Stream, a ChunkedStream, or even another stream layer. this.dict references the underlying stream's dictionary directly without copying: the object number, generation and encryption context all hang off the dictionary, and the plaintext stream produced after decryption still needs that information to identify itself.

this.decrypt is a closure that captures a freshly created cipher instance when crypto.js constructs it. This matters: cipher objects are stateful. RC4 maintains a 256-byte permutation table and two cursors, while AES maintains the key, the current IV and any residue shorter than one block. Every DecryptStream therefore must have its own cipher instance and must never share one across streams, otherwise the keystream position or the IV falls out of step and the output goes wrong from some block onward.

The Lifetime of a Cipher Instance

CipherTransform.createStream runs const cipher = new Cipher() every time. Even if two streams in the same document use the same object key, they receive two independent cipher objects. This is necessity, not waste: decryption is sequential, and cipher state can only correspond to one stream read from start to finish.

This constraint also determines how streams are cloned. DecodeStream.clone() first reads the current stream to eof, then wraps the output buffer into an ordinary Stream and returns it. For DecryptStream, cloning does not duplicate cipher state; it finishes in advance the part that has not been decrypted yet: from then on the clone and the original share the same plaintext bytes, the underlying ciphertext is exhausted, and the cipher object will receive no new input. If two consumers want to read the same encrypted object at the same time, the safe order is for one of them to read to the end and clone out an ordinary Stream, rather than letting two readers interleave driving the same cipher object; if a genuine read from the beginning is needed, the upper layer should re-fetch the object and construct a new DecryptStream.

The Main Loop: readBlock

The readBlock Method

  readBlock() {
    let chunk = this.#nextChunk ?? this.stream.getBytes(chunkSize);
    if (!chunk.length) {
      this.eof = true;
      return;
    }
    this.#nextChunk = this.stream.getBytes(chunkSize);
    const hasMoreData = this.#nextChunk.length > 0;
 
    const decrypt = this.decrypt;
    chunk = decrypt(chunk, !hasMoreData);
 
    const bufferLength = this.bufferLength,
      newLength = bufferLength + chunk.length,
      buffer = this.ensureBuffer(newLength);
    buffer.set(chunk, bufferLength);
    this.bufferLength = newLength;
  }

These dozen lines read in four steps.

Step one, take the current chunk. this.#nextChunk ?? this.stream.getBytes(chunkSize) is a nullish coalescing: if the previous round left a cached lookahead, use it directly; otherwise ask the underlying stream for 512 bytes. The first call necessarily takes the latter branch.

Step two, decide whether we are at the end. If the current chunk has length 0, the underlying stream is exhausted, so set eof and return. Note that no decryption happens here: an empty chunk is meaningless, and both AES and RC4 would just return empty output for empty input, so calling them would be pure waste. This is also the only place in the whole method that sets eof.

Step three, read the next chunk ahead of time. Before decrypting the current chunk, read the next one into #nextChunk and use whether its length is zero to obtain hasMoreData. This step is the core of the whole design: the crypto callback needs to know whether the current chunk is the last one, and that question can only be answered by reading the next chunk. The extra 512 bytes are not discarded — the next round takes them straight from #nextChunk — so the underlying stream is still consumed sequentially exactly once and the cursor never needs to be rewound.

Step four, decrypt and append. decrypt(chunk, !hasMoreData) passes the answer to "is this the last chunk" down as the second argument. The return value may be shorter than the input: the first AES block swallows the 16-byte IV and the last block drops the padding. Then ensureBuffer guarantees the output buffer is long enough, set writes the plaintext at the end, and bufferLength is updated accordingly.

The lookahead gives a verifiable invariant: every non-final chunk is called with false, the final chunk with true, and true appears exactly once. If the stream length is an exact multiple of 512, the last round takes the real final chunk and the lookahead gets an empty chunk, so the final chunk correctly carries true; if the length is not a multiple, the final chunk is itself shorter than 512 bytes and the lookahead likewise returns an empty chunk. Neither case needs an extra branch.

One more detail: on the first call, if AES receives fewer than 16 bytes, it buffers them and returns an empty array. Then chunk.length is 0, the append does nothing, bufferLength is unchanged, and the parent's loop keeps calling readBlock until a full block accumulates or the stream ends. The append logic needs no special handling for this zero-output case, since ensureBuffer and set are both safe with a zero length.

Why Not Read Everything at Once

If the whole ciphertext may eventually have to be decrypted, why chunk at all?

The reason is not the cryptographic algorithm but the upstream call chain. PDF streams are usually read lazily: a page content stream may be only a few hundred bytes while a scanned image may be tens of megabytes; if readBlock drained the entire underlying stream on its first call, then parsing the first operator at the top of a page would have to wait for the whole stream to arrive from the network or the disk. Chunking lets DecodeStream.getBytes drive on demand: the caller asks for 64 bytes, and decryption stops as soon as 64 bytes are available.

Chunking also works well with ChunkedStream's missing-data exception. When the underlying data has not finished downloading, getBytes throws MissingDataException; because reading advances in 512-byte steps, the exception appears only when the gap is actually hit, and PDFManager uses it to request exactly the missing range instead of blindly widening the prefetch window. Decryption itself is sequential, so the prefix already decrypted does not have to be redone.

Where the final Flag Goes

readBlock only passes !hasMoreData down; the cipher object is what actually interprets the boolean. crypto.js wires the callback into a closure when constructing the stream:

return new DecryptStream(stream, length, function cipherTransformDecryptStream(data, finalize) {
  return cipher.decryptBlock(data, finalize);
});

The callback's second parameter keeps the naming of decryptBlock, finalize. It does one thing: tell the cipher object that this is the last ciphertext segment and it may wrap up.

AES Padding Handling

AES is a block cipher working in CBC mode. PDF uses PKCS#7 padding: the plaintext length is padded to a multiple of 16, writing as many bytes as the shortfall, for example five 0x05 for a 5-byte shortfall; if the plaintext is already a multiple of 16, a whole extra block of sixteen 0x10 is added. Decryption must strip this part, otherwise the output ends with meaningless padding bytes.

AESBaseCipher._decryptBlock2 runs this logic when finalize is true:

if (finalize) {
  // undo a padding that is described in RFC 2898
  const lastBlock = result.at(-1);
  let psLen = lastBlock[15];
  if (psLen <= 16) {
    for (let i = 15, ii = 16 - psLen; i >= ii; --i) {
      if (lastBlock[i] !== psLen) {
        // Invalid padding, assume that the block has no padding.
        psLen = 0;
        break;
      }
    }
    outputLength -= psLen;
    result[result.length - 1] = lastBlock.subarray(0, 16 - psLen);
  }
}

It takes the last byte of the last block as the padding length, then validates it: the length must not exceed 16, and the last psLen bytes must all equal psLen. If either check fails, psLen is treated as 0 and the whole block is kept. This is a typical tolerant reading — better to keep a few extra bytes than to fail the whole document's decryption over a generator's non-standard padding.

When finalize is false the function skips this logic entirely. The reason is straightforward: only the last block has padding, and stripping the middle blocks would delete real data at the end of each block, instantly shifting the output.

RC4 and the Null Cipher

RC4 is an entirely different case:

  decryptBlock(data) {
    return this.encryptBlock(data);
  }

It takes only one parameter. The second argument that DecryptStream passes in is simply ignored in JavaScript. RC4 is a stream cipher: the keystream is XORed with the plaintext byte by byte, lengths correspond one to one, there is no notion of blocks and no padding to strip. Whether finalize is true or false, the output length equals the input length.

NullCipher is even simpler: decryptBlock returns the data unchanged. It is used for cases declared encrypted but actually not encrypted, or for the Identity crypto filter, and likewise ignores finalize.

Looking at the three ciphers together, the meaning of final becomes clear: it does not mean the last block needs special decryption, it means the cipher may wrap up. Block ciphers use it to strip padding; stream ciphers have nothing to do.

The IV of the First Chunk

An AES ciphertext stream also begins with a 16-byte initialisation vector. On its first call, if no IV was explicitly provided from outside, decryptBlock first accumulates those 16 bytes into an internal buffer; once they are complete it sets them as the iv and starts decrypting from byte 17. So the first chunk's actual output is the 512 bytes read minus the 16-byte IV, later chunks are a full 512 bytes, and the last chunk additionally drops the padding.

This also explains why the maybeLength the constructor receives can only serve as a capacity hint: the real plaintext length is the ciphertext length minus the IV, minus the padding — and the padding length is only known at the very last moment of decryption.

Stream Identity: getOriginalStream

The getOriginalStream Method

  getOriginalStream() {
    return this;
  }

The body is one line, yet it is the most easily overlooked design in this file.

BaseStream's default implementation passes through to the underlying stream:

  getOriginalStream() {
    return this.stream?.getOriginalStream() || this;
  }

A decoder wrapped around another stream forwards the original-stream question all the way down to the bottom, ultimately returning the stream that holds that run of bytes in the file. That is reasonable for pure decoders like Flate and Brotli: the underlying stream they wrap is their own data source.

DecryptStream deliberately breaks this chain. It does not forward the question to the underlying ciphertext stream but returns itself. The reason is that the definition of identity changes at the decryption layer: for the upper layer, the truly trustworthy original bytes are the decrypted result, not the ciphertext in the file. Continuing to pass through downwards would hand the caller wrong bytes and a wrong identity.

The plaintext here is relative to the crypto layer: it may still be Flate- or Brotli-compressed bytes, and may even still need predictor restoration, but it is no longer ciphertext. For writing back and cloning, this is exactly the layer needed — keep the compression and filter information, strip only the crypto layer.

Two concrete call sites show the consequences.

writer.writeStream unwraps before taking bytes when serialising a stream:

async function writeStream(stream, buffer, transform) {
  stream = stream.getOriginalStream();
  stream.reset();
  let bytes = stream.getBytes();

It then decides, based on /Filter in the dictionary, whether to recompress, and finally re-encrypts with transform.encryptString when writing back. If getOriginalStream returned the underlying ciphertext stream, the write-back flow would encrypt the ciphertext a second time, producing double-encrypted data, and the /Length in the dictionary would not match the actual content either. Returning the decryption layer is exactly the opposite: what is obtained here is plaintext (still compressed bytes as far as the inner compression is concerned), and one re-encryption puts it back into the correct state.

The editor relies on the same rule when copying objects. pdf_editor calls obj.getOriginalStream().clone() when cloning a stream; DecodeStream.clone() first reads the stream to eof and then wraps the output buffer into an ordinary Stream. For DecryptStream that process produces a complete plaintext byte sequence with the dictionary cloned alongside; if identity passed through to the ciphertext layer, the copied object would carry ciphertext bytes with a plaintext dictionary, and every later read would be inconsistent.

So getOriginalStream returning itself is not laziness but an explicit statement: the decryption layer is the best identity this stream has for the outside world; anything below is the crypto implementation's private business.

The DecodeStream Contract: Append, Not Replace

DecryptStream.readBlock uses the append style: first grow with ensureBuffer to the new length, then set the decrypt result after bufferLength. This contrasts with BrotliStream's wholesale replacement:

Appending assumes that each chunk can be handed to the caller as soon as it is decrypted. Decryption satisfies this: apart from the IV of the first chunk and the padding of the last, there is no dependency between chunks, and the part already written to the buffer will not change because of later decryption. The upper layer can therefore start parsing a content stream after only the first few hundred bytes are decrypted, without waiting for the whole stream.

Replacement assumes that the result must take shape in one piece. The Brotli decoder does not accept segmented input, and the output length is known only after decoding finishes, so appending would only bring repeated growth and surplus copies. Neither style is superior; each simply fits the shape of its decoder.

Appending has a memory advantage too: ensureBuffer grows in powers of two, so capacity grows in proportion to the amount already decrypted and there is never a peak that allocates the entire plaintext length up front. For large image streams that matters more than saving a few copies.

The price is moving old data when the buffer grows. With power-of-two growth the cumulative copy volume is on the same order as the total output length, not quadratic in the number of chunks; for a few-hundred-KB font or content stream the actual moves are only a handful. By contrast, trying to predict the plaintext length and allocate once would once again depend on the untrustworthy /Length and be more error-prone.

Edge Cases and Fault Tolerance

A few edge cases deserve their own explanation.

• Empty stream. The first readBlock gets a chunk of length 0, sets eof and returns immediately; the decrypt callback is never called at all. Empty plaintext and empty ciphertext produce the same result, and skipping the call has no side effect.
• Final chunk shorter than 16 bytes. AES's _decryptBlock2 leaves a tail shorter than one block in its internal buffer, and even with finalize true those few dozen bytes cannot be decrypted. The final output drops that incomplete tail instead of throwing. This is the choice to read out as much as can be read.
• Invalid padding. As described in the previous section, when validation fails it is treated as no padding and the whole block is kept. For documents from non-conforming generators this has more practical value than raising an error.
• Ciphertext too short. If the whole stream is under 16 bytes, AES cannot even accumulate a full IV, the first call returns an empty array, the stream then ends, and the upper layer gets empty plaintext. The caller handles it as empty data and does not hang.
• The /Crypt filter. When the /Filter array explicitly contains /Crypt, Parser.readStream does not decrypt by default; instead makeFilter constructs a DecryptStream in the chain and selects the crypto filter according to /Name in /DecodeParms. Both routes use the same class; only the construction timing and the chosen cipher differ.
• Unreliable length. /Length is only used as the buffer starting point; the real plaintext length is determined by the IV, block boundaries and padding together. Any code that tries to compute output positions directly from /Length will go wrong on AES documents.

These cases share one property: the errors are in the data itself, not in the reading framework. DecryptStream is not responsible for deciding whether a document is valid; it only guarantees that the cipher transform is applied to the given bytes in order and that the decodable prefix is handed out. Judgement and policy are left to the upper layer, which lets the same class serve both strict standard documents and malformed files from which as much as possible should be extracted.

Design note: This layer fully decouples cipher blocks from read blocks. The cipher object handles its own sub-block residue and the padding of the final block, while the stream layer only moves bytes at a fixed granularity and uses a single lookahead to identify the final chunk. With that division clear, the output is consistent whether the caller asks for 1 byte at a time or the whole stream.

Conclusion

DecryptStream is PDF.js's decryption adapter layer: it restores the ciphertext stream produced by the standard security handler into plaintext and hands it upward to the filter chain and the parser.

Where It Sits in the Overall Chain

In one sentence: CipherTransform decides which key and algorithm to use, DecryptStream decides at what pace to turn ciphertext into plaintext, and the filter chain turns the plaintext back into usable data.

Five Design Principles

1. Cipher block boundaries are decoupled from read boundaries. The stream layer moves a fixed 512 bytes at a time, and sub-block residue is cached by the cipher object itself. The caller gets exactly as much as it asks for and never has to care about AES's 16-byte blocks or RC4's byte-by-byte nature.
2. One lookahead, and the last-chunk flag appears exactly once. Read the next chunk before decrypting the current one, and use whether the next chunk is empty to decide whether the current one is final. The extra bytes stay in #nextChunk, and the underlying stream is still consumed sequentially only once.
3. Decrypt and append, rather than replace the whole buffer. Chunks do not depend on each other and plaintext already written is never rewritten by later chunks, so the upper layer can start parsing early; the buffer doubles on demand, avoiding a memory peak that reserves the whole plaintext.
4. Identity is cut off at the decryption layer. getOriginalStream returns itself, so write-back, cloning and editing all stop at the plaintext layer. Below that is ciphertext, which is not the original bytes the upper layer should see.
5. Fault tolerance before error reporting. When padding validation fails the whole block is kept, when the final chunk is incomplete the already-decrypted prefix is kept, and /Length is treated only as a capacity hint. Corrupt or non-conforming documents yield as much usable data as possible instead of failing the whole file.

With this, the road from file bytes to plaintext compressed bytes is complete. In encrypted documents what is really encrypted is usually not semantic data but the compressed stream; that is why decryption must happen before filters, and why DecryptStream and FlateStream stack in that order. Once you understand this order, the comment in Parser.readStream about skipping default decryption when /Crypt is explicit makes sense as avoiding decrypting the same data twice.

StreamPDF design principle: anything that can be handled ahead of time is absolutely not deferred to parse time; the server solves all performance problems once. The browser only fetches and renders the data it must.