PDF.js Source Walkthrough: jpeg_stream

Introduction

This is the first article in the "Image Decoding" layer, and its subject is JpegStream in the worker-side src/core/jpeg_stream.js (192 lines).

The earlier articles covered flate_stream, lzw_stream, ascii_85_stream, ascii_hex_stream and run_length_stream. All of them are essentially byte-to-byte transforms: they take a run of compressed bytes and emit bytes that can still be parsed, so they can be chained head-to-tail into filters of arbitrary length.

DCTDecode is different. Its output is not "shorter bytes" but pixels. PDF does not define its own DCT coding; it embeds a standard JPEG codestream and leaves this decoding step to a complete image decoder. JpegStream therefore stands at the boundary between two semantics: externally it is still a DecodeStream, internally it is a one-shot image decoder.

The specification names the JPEG filter /DCTDecode (abbreviation /DCT), where DCT stands for Discrete Cosine Transform. The codestream may be baseline or progressive, may carry a JFIF header, and may carry an Adobe APP14 marker. The image dictionary's /Width, /Height and /ColorSpace describe the decoded result, while the JPEG SOF segment also records a set of dimensions and component counts. The two usually agree; in damaged files they may not, and that difference directly affects the native decoding path described below.

The entry point is a single line in Parser.makeFilter:

        case "DCT":
        case "DCTDecode":
          return new JpegStream(stream, maybeLength, params);

In one sentence, its job is: fetch the whole upstream JPEG codestream, strip junk bytes before SOI, hand it to JpegImage for pixels, and when the browser supports it, try the native ImageDecoder to produce a VideoFrame directly.

Module Overview

jpeg_stream.js is only 192 lines. Leaving out the copyright header and imports, the body is a single class. By responsibility it splits into eight blocks:

Its position in the DecodeStream hierarchy is unusual. The base class assumes data can be decoded one small block at a time, but a JPEG decoder has no such incremental interface: JpegImage.parse needs the complete codestream, and getData returns the whole image at once. JpegStream does not reshape the base class; it overrides only the two hooks readBlock and ensureBuffer, mapping "pull incrementally" onto "decode once", so callers still receive an ordinary DecodeStream.

Inheritance and Three Overrides

class JpegStream extends DecodeStream {
  static #isImageDecoderSupported = FeatureTest.isImageDecoderSupported;
 
  constructor(stream, maybeLength, params) {
    super(maybeLength);
 
    this.stream = stream;
    this.dict = stream.dict;
    this.maybeLength = maybeLength;
    this.params = params;
  }

The constructor Method

The constructor has only four assignments, but each field's origin is worth spelling out:

• stream is the upstream stream. It may be a file-backed Stream, a network-chunked ChunkedStream, or another DecodeStream. Keeping it is what supports both the synchronous getBytes and the asynchronous asyncGetBytes byte paths.
• dict points straight at stream.dict, the image stream's own dictionary. Width, Height and ColorSpace are all read from it, and Jbig2Stream does the same.
• maybeLength comes from the stream dictionary's /Length. It is only an approximate length; it may be 0, and it may be null because Parser sets it to null when the filter sits second in the chain. In the latter case upstream getBytes reads all the way to EOF.
• params is /DecodeParms, which may be null or a Dict. JPEG's ColorTransform hides here.

super(maybeLength) computes minBufferLength as a power of two, but that value barely matters in this class: ensureBuffer is overridden to a no-op, so the base class's "grow gradually, preserve old bytes" logic never runs.

The Lazy bytes Getter

The entry point for compressed bytes is a getter:

  get bytes() {
    // If `this.maybeLength` is null, we'll get the entire stream.
    return shadow(this, "bytes", this.stream.getBytes(this.maybeLength));
  }

shadow replaces the prototype getter with an own data property of the same name. The first access runs getBytes; every later access returns the same Uint8Array and never touches upstream again. shadow also asserts in test builds that the property really exists on the prototype, so a typo cannot turn into a silent undefined.

One side effect is easy to overlook: bytes is computed only on first access. When upstream is a chunked network stream and the bytes have not finished downloading, this synchronous getBytes throws MissingDataException, so the asynchronous path must bypass the getter and call this.stream.asyncGetBytes() directly. With maybeLength set to null, getBytes reads to upstream EOF, which is safe for JPEG: the codestream ends with EOI and the decoder stops when it reaches that marker.

ensureBuffer and readBlock

  ensureBuffer(requested) {
    // No-op, since `this.readBlock` will always parse the entire image and
    // directly insert all of its data into `this.buffer`.
  }
 
  readBlock() {
    this.decodeImage();
  }

The base class's getBytes flow is:

1. If the caller supplied a length, call ensureBuffer(pos + length) first;
2. While not eof and bufferLength is still insufficient, call readBlock again and again;
3. Finally return buffer.subarray(pos, end).

In this class step 1 does nothing. The first readBlock in step 2 calls decodeImage, which assigns the whole pixel array to this.buffer, sets bufferLength to the length of that array, and sets eof to true. The loop therefore runs exactly once.

readBlock accepts neither length nor decoderOptions. That is not an oversight: JPEG decoding cannot "decode half", so any length argument could only be ignored. decodeImage guards its own idempotency with this.eof and returns the existing buffer on a repeated call.

Design note: ensureBuffer and readBlock are two sides of the same coin. The first tells the base class "do not prepare a buffer for incremental writes"; the second tells it "one pass has already read everything". Together the two hooks embed a non-incremental image decoder into the incremental stream interface without loss, and callers never need to know whether they received a plain FlateStream or a JpegStream.

jpegOptions and ColorTransform

The jpegOptions Method

  get jpegOptions() {
    const jpegOptions = { colorTransform: undefined };
 
    // Fetching the 'ColorTransform' entry, if it exists.
    if (this.params instanceof Dict) {
      const colorTransform = this.params.get("ColorTransform");
      if (Number.isInteger(colorTransform)) {
        jpegOptions.colorTransform = colorTransform;
      }
    }
    return shadow(this, "jpegOptions", jpegOptions);
  }

Three details:

1. Integers only. ColorTransform has only two legal values, 0 and 1. Non-integer values, such as a name or an array in a damaged file, are ignored and fall back to the default behaviour.
2. The default is undefined, not 0. Only then can JpegImage decide for itself from the JFIF and Adobe APP14 markers. Forcing 0 here would make every YCbCr JPEG decode as RGB and come out wrong.
3. Cached and reused. shadow makes the options object construct only once. The options are immutable, so sharing them across decoding calls is harmless.

The semantics of ColorTransform are simple: 0 means the three components are already RGB and no YCbCr transform is applied; 1 means apply the YCbCr transform. Four-component JPEG (CMYK or YCCK) is more involved: JpegImage.getData combines it with Adobe markers to decide whether to invert and whether to convert YCCK to CMYK.

ColorTransform is a filter parameter, not an image property. The image dictionary describes "what the result is"; DecodeParms describes "how to decode it". Mixing the two would make the same JPEG be interpreted wrongly under a different ColorSpace, so the source keeps the boundary sharp.

Junk Bytes Before SOI

The #skipUselessBytes Method

A JPEG codestream must start with SOI (Start Of Image), marker value 0xff 0xd8. Real-world PDFs are not always that clean:

  #skipUselessBytes(data) {
    // Some images may contain 'junk' before the SOI (start-of-image) marker.
    // Note: this seems to mainly affect inline images.
    for (let i = 0, ii = data.length - 1; i < ii; i++) {
      if (data[i] === 0xff && data[i + 1] === 0xd8) {
        if (i > 0) {
          data = data.subarray(i);
        }
        break;
      }
    }
    return data;
  }

• The loop bound is data.length - 1, which keeps data[i + 1] in range. Arrays shorter than 2 bytes are returned unchanged.
• Once SOI is found, if it is not at the start, subarray(i) cuts away the preceding bytes. subarray returns a view and copies nothing, so this step is nearly free.
• If SOI is already at index 0, nothing is sliced, avoiding a pointless object.
• If no SOI is found, the data is returned unchanged and JpegImage.parse throws SOI not found. The error is raised at the layer that owns it; this function does not overstep.

The source comment says this mainly affects inline images (BI ... ID ... EI). Inline image bytes sit inside the content stream, and when Parser scans for EI it may capture preceding whitespace or residual bytes. A standalone image stream should not have such a prefix in theory, but one scan of a dozen comparisons is far cheaper than failing to decode an entire page.

Note that only bytes before SOI are handled. Trailing data after EOI is left alone, because the JPEG decoder's marker loop exits at EOI and the tail never participates in decoding.

decodeImage: the JpegImage Path

The decodeImage Method

All the logic of the synchronous path lives in decodeImage:

  decodeImage(bytes) {
    if (this.eof) {
      return this.buffer;
    }
    bytes = this.#skipUselessBytes(bytes || this.bytes);
 
    // TODO: if an image has a mask we need to combine the data.
    // So ideally get a VideoFrame from getTransferableImage and then use
    // copyTo.
 
    const jpegImage = new JpegImage(this.jpegOptions);
    jpegImage.parse(bytes);
    const data = jpegImage.getData({
      width: this.drawWidth,
      height: this.drawHeight,
      forceRGBA: this.forceRGBA,
      forceRGB: this.forceRGB,
    });
    this.buffer = data;
    this.bufferLength = data.length;
    this.eof = true;
 
    return this.buffer;
  }

Each step corresponds to a concrete constraint:

1. The eof guard. decodeImage may be called repeatedly by readBlock, or directly by the asynchronous path; once decoding is done it returns the existing pixels. reset() only resets the read position, not buffer or eof, so redrawing the same page does not decode twice.
2. Bytes come from one of two sources. bytes || this.bytes lets the caller pass bytes already fetched asynchronously; otherwise the lazy getter runs. Both paths share the same decoding logic.
3. Strip junk bytes. Wherever the bytes came from, they pass through #skipUselessBytes first.
4. Masks are left to the layer above. The source TODO is blunt: an image with a mask needs its pixels combined with the mask, and that currently happens in PDFImage higher up; JpegStream only restores the main image. A VideoFrame plus copyTo on the native path is a possible future optimisation.
5. Parsing and data extraction are separate. parse only walks markers and builds quantisation and Huffman tables; getData performs dequantisation, IDCT, upsampling and colour conversion. This class creates a fresh instance each time, so one codestream is never shared between two output sizes.
6. The buffer is replaced wholesale. this.buffer = data appends nothing; bufferLength is the pixel array length; eof = true means there will be no next block.

getData Dimensions and Channels

None of the four arguments to getData comes from the JPEG file; they are the output the caller wants:

• drawWidth / drawHeight are written onto the image stream by PDFImage.getImageBytes before the call. They default to the dictionary's Width and Height, but the render size may be smaller.
• forceRGBA is true when alpha, a soft mask or the OffscreenCanvas path is involved, requiring four-channel output.
• forceRGB is true when only opaque three-channel output is needed, avoiding an extra alpha byte.

This point matters: JpegImage.getData does not "decode the full image at SOF size and then scale". It passes the target size into the block decoder and restores directly at the needed sampling rate. For a scan that is 6000 pixels wide displayed at 800 pixels, this step saves the memory and time of a full-size pixel buffer.

Output is always a Uint8ClampedArray, and the channel count depends on the JPEG component count and the caller's forcing options: greyscale defaults to one channel; three components become RGB or RGBA when needed; four components follow CMYK or YCCK rules; forceRGB and forceRGBA request three and four channels respectively. bufferLength may therefore be one, three or four times width * height, and PDFImage above interprets those bytes according to ColorSpace.

Design note: Decoding at the "draw size" rather than the "file size" is the single biggest win in the image chain. It drops the memory peak from "original pixels" to "display pixels" without an extra scaling step. The price is decoding twice when the same image appears at two sizes; that cost is borne by the upper-layer cache strategy, not by storing multiple results in the stream layer.

Asynchronous Byte Fetching

The canAsyncDecodeImageFromBuffer Method

The getter name is long, but it does not describe "asynchronous decoding"; it describes "can the compressed bytes be fetched asynchronously first":

  get canAsyncDecodeImageFromBuffer() {
    return this.stream.isAsync;
  }

DecodeStream.getImageData branches on it:

• true: first await this.stream.asyncGetBytes(), then call decodeImage(data, length, decoderOptions) with the complete compressed bytes;
• false: go through getBytes(length, decoderOptions), i.e. synchronous readBlock.

Why a dedicated path? Because JpegImage.parse is not resumable. It reads marker after marker, and halfway through, missing bytes can only throw MissingDataException; on retry the upper layer must decode from the start again. Rather than let the decoder fail mid-way, fetch all bytes first and hand them over in one go. A large image on a chunked network stream therefore avoids a great deal of repeated work.

Note that this is unrelated to the native ImageDecoder: this layer only answers "are the bytes all here"; the next layer answers "who decodes the pixels".

The Native Browser Decoding Path

Modern browsers expose WebCodecs ImageDecoder. It can decode a JPEG straight into a VideoFrame, skipping the JavaScript Huffman decoding and IDCT. JpegStream.getTransferableImage is that optional path, and PDFImage consumes its return value in #getImage:

const bitmap = await this.image.getTransferableImage(width, height);
if (!bitmap) {
  return null;
}
// ImageDecoder may ignore the requested dimensions.
return {
  data: null,
  width: bitmap.displayWidth ?? width,
  height: bitmap.displayHeight ?? height,
  bitmap,
  interpolate: this.interpolate,
};

Returning null means "the native path is unavailable", and the caller falls back to canvas plus pixels.

The getTransferableImage Method

The full implementation:

  async getTransferableImage(width, height) {
    if (!(await JpegStream.canUseImageDecoder)) {
      return null;
    }
    const jpegOptions = this.jpegOptions;
    let decoder;
    try {
      // TODO: If the stream is Flate & DCT we could try to just pipe the
      // the DecompressionStream into the ImageDecoder: it'll avoid the
      // intermediate ArrayBuffer.
      const bytes =
        (this.canAsyncDecodeImageFromBuffer &&
          (await this.stream.asyncGetBytes())) ||
        this.bytes;
      if (!bytes) {
        return null;
      }
      let data = this.#skipUselessBytes(bytes);
      const useImageDecoder = JpegImage.canUseImageDecoder(
        data,
        jpegOptions.colorTransform
      );
      if (!useImageDecoder) {
        return null;
      }
      const { width: frameWidth, height: frameHeight } = useImageDecoder;
      const reducePower = ImageResizer.getReducePower(frameWidth, frameHeight);
      if (
        (frameWidth !== width || frameHeight !== height) &&
        (reducePower || !frameHeight)
      ) {
        // Only downscale when the SOF and image dictionary dimensions match.
        return null;
      }
      if (useImageDecoder.exifStart) {
        // Replace the entire EXIF-block with dummy data, to ensure that a
        // non-default EXIF orientation won't cause the image to be rotated
        // when using `ImageDecoder` (fixes bug1942064.pdf).
        //
        // Copy the data first, to avoid modifying the original PDF document.
        data = data.slice();
        data.fill(0x00, useImageDecoder.exifStart, useImageDecoder.exifEnd);
      }
      const init = {
        data,
        type: "image/jpeg",
        preferAnimation: false,
      };
      // Request reduced dimensions; ImageDecoder treats them as best-effort.
      if (reducePower) {
        const factor = 2 ** reducePower;
        init.desiredWidth = Math.ceil(frameWidth / factor);
        init.desiredHeight = Math.ceil(frameHeight / factor);
      }
      decoder = new ImageDecoder(init);
 
      return (await decoder.decode()).image;
    } catch (reason) {
      warn(`getTransferableImage - failed: "${reason}".`);
      return null;
    } finally {
      decoder?.close();
    }
  }

The whole method is a pipeline that gives up layer by layer: capability detection fails, bytes are unavailable, the codestream is unsuitable for native decoding, the dimension relationship does not hold — any failure returns null; construction of ImageDecoder or a throwing decode only logs a warning in catch. The cost is always just a fallback to pure-JS decoding, never a failed page render. This is the classic "an optimisation path must be allowed to fail" design.

Static Detection and canUseImageDecoder

  static get canUseImageDecoder() {
    return shadow(
      this,
      "canUseImageDecoder",
      this.#isImageDecoderSupported
        ? ImageDecoder.isTypeSupported("image/jpeg")
        : Promise.resolve(false)
    );
  }
 
  static setOptions({ isImageDecoderSupported = false }) {
    this.#isImageDecoderSupported = isImageDecoderSupported;
  }

FeatureTest.isImageDecoderSupported does exactly one thing: decide whether the global ImageDecoder object exists. ImageDecoder.isTypeSupported("image/jpeg") returns a Promise, and shadow caches that Promise so all later JPEGs share one query and never ask the browser again.

setOptions is called by pdf_manager during document initialisation. The default value in evaluator.js is false, and pdf_manager ANDs it with the FeatureTest result. In other words, native decoding is an explicitly enabled capability, not "use it whenever the browser has it"; callers can turn it off through configuration for testing or to work around platform bugs.

JpegImage.canUseImageDecoder is a second layer of detection, looking at the codestream itself:

• scan JPEG markers lightly, read SOF0, SOF1 or SOF2, and take the dimensions and component count;
• on an APP1 whose payload starts with Exif, record the start and end of the EXIF block for the later zeroing;
• four components (CMYK/YCCK) return null immediately, because the native decoder handles colour for such JPEGs differently from what PDF expects;
• three components with ColorTransform 0 also return null, because that means the components are already RGB while ImageDecoder would treat them as YCbCr;
• the SOF height may be 0, meaning the real height is defined by a later DNL marker; that case passes 0 through unchanged and lets the dimension guard deal with it.

This step only scans markers and performs no entropy decoding, so it costs far less than a real decode. The overhead of "ask first whether native decoding is usable" is therefore negligible.

The Dimension Guard and Downscaling

ImageResizer.getReducePower(frameWidth, frameHeight) returns "how many powers of two to drop":

• 0 when the image is within the MAX_DIM and MAX_AREA limits;
• otherwise compute the smallest scale factor from width, height and area, then take ceil(log2(factor));
• 0 when an argument is not a positive integer, so invalid dimensions cannot skew the calculation.

The value has two uses: deciding whether downsampling should be requested, and setting desiredWidth / desiredHeight. ImageDecoder treats those two fields as hints, not commands, and the final size still comes from the returned VideoFrame; PDFImage falls back to displayWidth / displayHeight precisely because of that uncertainty.

The guard condition deserves to be read literally:

(frameWidth !== width || frameHeight !== height) && (reducePower || !frameHeight);

It blocks the native path only in the cases the table lists:

The third row is the DNL (Define Number of Lines) compatibility branch. A few JPEGs write height 0 and provide the real line count only after the scan data; JpegImage's full parse handles that, but the lightweight scan only sees 0 and cannot use it for size decisions.

Zeroing EXIF and Releasing Resources

The JPEG APP1 segment may carry EXIF, whose Orientation tag tells a viewer how far to rotate the image. PDF does not need that hint: the image matrix in the page content stream already determines placement. If ImageDecoder rotates once more according to EXIF, the image ends up rotated by an extra 90 or 180 degrees. bug1942064.pdf, mentioned in the source comment, is exactly that kind of file.

The handling is to overwrite the whole EXIF payload with zeros:

data = data.slice();
data.fill(0x00, useImageDecoder.exifStart, useImageDecoder.exifEnd);

slice() first is mandatory. bytes may be a subarray view of an upstream stream buffer, and a direct fill would modify the PDF document's own bytes, so a second pass of the same image through the pure-JS path would see corrupted EXIF. After copying, anything can be modified without affecting the original document.

After zeroing, the Orientation in the TIFF header becomes the default value, ImageDecoder no longer rotates, and placement is still owned by the PDF transformation matrix.

decoder.close() lives in finally, so it runs on success, on error and on early return. ImageDecoder holds a platform decoder and frame buffers internally, and not closing it leaks native resources; decoder?. also covers the case of a throw before construction, when the variable is still undefined. catch only logs a warning and returns null, handing the decision back to the upper layer.

Design note: Every check on the native path answers the same question: "will this shortcut change the result?" Capability unsupported, colour model mismatched, dimension relationship invalid, EXIF interference — any hit means giving up. An optimisation must be result-equivalent to the pure-JS path, or it should not run. That conservatism turns ImageDecoder into a switchable accelerator rather than a second implementation that needs separate maintenance.

isImageStream

The last getter is three lines, yet it carries the type marker:

  get isImageStream() {
    return true;
  }

The base class BaseStream returns false by default. Only two places in the whole repository consume it:

• StreamsSequenceStream filters out streams whose isImageStream is true when it is constructed. Concatenating content streams must not treat image pixels as page operators.
• PDFPage.getContentStream excludes image streams when deciding whether content is a parseable content stream. In a damaged file where /Contents points at a JPEG, without this marker the pixels would be fed to the lexer and produce a flood of meaningless warnings.

Comparing the Two Decoding Paths

The two paths are not mutually exclusive: native is tried first, returns null on failure, and PDFImage then takes the canvas path; the pure-JS implementation is always kept, both as a fallback and as the correctness reference.

Conclusion

JpegStream is the adapter of the image decoding layer: upward it preserves the DecodeStream read interface, downward it hands the whole JPEG codestream to JpegImage, and when the browser supports it, it forwards the same bytes to the native ImageDecoder.

Where It Sits in the Overall Chain

In one sentence: Parser wraps /DCTDecode in a JpegStream, JpegStream turns the whole JPEG into pixels or a VideoFrame, and PDFImage turns that into a drawable bitmap.

Five Design Principles

1. Override only the necessary hooks and keep the base class shape. readBlock and ensureBuffer are enough to embed a non-incremental decoder into the incremental interface; getByte, getBytes, reset and makeSubStream are all reused, and callers need no branches.
2. Decode the whole image once, use eof for idempotency. It does not pretend to support blocks and keeps no intermediate state in readBlock; once eof is set, repeated reads hit the buffer directly. Less state means fewer failure paths.
3. Encoded bytes and pixels stay strictly separate. bytes caches compressed data; buffer holds pixels; reset only moves the read position and destroys neither. Mixing them would let redraws or mask composition read the wrong content.
4. Detect native capability statically and degrade on any failure. Capability queries become a cached Promise, codestream suitability checks live in a lightweight marker scan, and any failure returns null. The native path may accelerate, never change, the result.
5. Never modify the caller's bytes. SOI trimming uses a subarray view, and EXIF zeroing copies with slice first. PDF stream buffers may be shared in several places, and any in-place write becomes hard-to-find dirty data on another decoding path.

This completes the "how JPEG becomes pixels" link. The next article handles jbig2_stream: also whole-image decoding, also a lazy bytes and a no-op ensureBuffer, but the decoder becomes an asynchronous WASM singleton and it additionally has to deal with the JBIG2 file header, global segments and dictionary dimensions.

StreamPDF design principle: anything that can be handled ahead of time is absolutely not deferred to parse time; the server solves all performance problems once. The browser only fetches and renders the data it must.