PDF.js Source Walkthrough: obj_bin_transform_core

Introduction

src/core/obj_bin_transform_core.js is only 364 lines long, yet it is the hardest protocol layer between the worker and the main thread: it compiles five kinds of objects into an ArrayBuffer, so that postMessage carries bytes instead of objects.

The problem it solves is very concrete. Once a font has been parsed on the worker side, the main thread needs its name, metrics, embedded data and CSS fallback information; once a pattern has been evaluated on the worker side, the main thread needs its coordinates, colors and color stops; glyph outlines of Type3 fonts also have to travel from the worker to the main thread. Sending these as class instances hits three problems at once: structured cloning drops the prototype, large payloads are copied, and repeated strings are encoded one by one.

So PDF.js flattens the output of these five functions into a flat buffer: the worker writes it, the main thread reads it, and both sides share the same field order and offset definitions.

This article covers the write side only. The read side mirrors the field order byte for byte, but is out of scope here.

Module Overview

The file exports five functions that cover four kinds of data:

All five are pure functions: no class, no state, no cache, only reading fields, computing lengths and writing bytes. That shape makes them easy to cover with unit tests, and avoids leaving an extra lifecycle behind in the worker.

The call chain compresses into one table:

Why Class Instances Cannot Be Sent Directly

MessageHandler.send goes through structured cloning. Structured cloning copies only the enumerable data properties of the object itself; it does not copy the prototype chain, private fields or accessors. A cloned Dict is no longer a Dict, and a cloned Font no longer has methods such as exportData and fallback. The main thread receives a plain object that looks like the original but has no behaviour.

Two more costs are easy to overlook:

1. Repeated strings. Every font carries name, loadedName, fallbackName and mimetype; when objects are sent directly these are four independent JS strings, each stored once on both sides of the channel.
2. Large binary payloads. Embedded fonts are routinely hundreds of KB to several MB; structured cloning copies them in full on the worker side before handing them over, so peak memory doubles immediately.

The compile approach fixes both at once: strings are encoded contiguously into a single buffer, the font data is appended at the tail, and the whole buffer is sent as a transferable, which detaches the original buffer in the worker so it no longer occupies memory.

Note that this does not turn the whole message protocol into binary. In the same batch of messages, charProcOperatorList and extra remain plain objects, and non-Type3 pattern IR still travels in a plain obj message. Binary is used only on the hot path that runs once per font, once per glyph and once per Type3 pattern.

Design note: The first rule of cross-thread transfer is to send data, not behaviour. The value of a class instance lies in its methods, caches and private state, none of which belongs across a thread boundary; only the fields really need to travel. Once the fields are flattened, the protocol becomes a byte layout that can be described by constants, and those constants can live in the shared directory and be referenced by both sides at once.

Shared Schema: Fields and Offsets Written Once

shared/obj_bin_transform_utils.js contains no encode or decode logic at all, only four groups of constants plus a lazy TextEncoder / TextDecoder holder. They define which field is written at which byte.

FONT_INFO offsets are computed rather than hand-written:

These five values are used by both the writer and the reader. The writer runs an assert at the end of every block to check that the current offset equals the next constant, while the reader addresses fields with the constants directly. If someone adds a field to FONT_INFO.bools and forgets to update the layout, the assertion fails at once instead of letting the main thread silently read misaligned data.

Byte order is another detail. The numeric font block uses DataView's default big-endian, yet fontMatrix explicitly passes true for little-endian; the counts, coordinates and color stops in patterns are also explicitly little-endian. Mixing two byte orders in one file looks odd, but as long as both sides agree field by field there is no problem. The real danger is assuming a single byte order and reading everything with it, which yields completely wrong numbers for some fields.

Design note: The hardest part of maintaining a binary protocol is not the encoding but trust in one shared layout on both sides. PDF.js turns that trust into a checkable constraint with shared constants and runtime assertions; without that layer, any addition or removal of a field becomes a misalignment that is only discovered in production.

Compiling Font Info

The font export object comes from Font.exportData, and its field set is decided by EXPORT_DATA_PROPERTIES. That set contains only the properties rendering must have: metrics, bbox, fontMatrix, names, embedded data, CSS and system font fallback information. The genuinely optional debug information lives in EXPORT_DATA_EXTRA_PROPERTIES and is sent as a plain object with the message only when fontExtraProperties is enabled.

compileCssFontInfo

CSS font info serves exactly one purpose: the main thread needs fontFamily, fontWeight and the italic angle when it creates a FontFace.

function compileCssFontInfo(info) {
  const { encoder } = InfoUtils;
  const encodedStrings = {};
  let stringsLength = 0;
  for (const prop of CSS_FONT_INFO.strings) {
    const encoded = encoder.encode(info[prop]);
    encodedStrings[prop] = encoded;
    stringsLength += 4 + encoded.length;
  }
 
  const buffer = new ArrayBuffer(stringsLength);
  const data = new Uint8Array(buffer);
  const view = new DataView(buffer);
  let offset = 0;
 
  for (const prop of CSS_FONT_INFO.strings) {
    const encoded = encodedStrings[prop];
    const length = encoded.length;
    view.setUint32(offset, length);
    data.set(encoded, offset + 4);
    offset += 4 + length;
  }
  assert(offset === buffer.byteLength, 'compileCssFontInfo: Buffer overflow');
  return buffer;
}

Field order comes from CSS_FONT_INFO.strings: fontFamily, fontWeight, italicAngle.

The layout is a length prefix plus UTF-8 bytes, repeated three times, with no total length, no field count and no type tags. The reader loops by index: every string it skips advances the offset by 4 + length, and the last one it reads is the target. With only three fields the linear scan costs nothing.

stringsLength is accumulated before allocation, so the buffer size is exact and the final assert checks strict equality rather than "less than or equal". There is no branch for a missing field: encoder.encode(undefined) yields the bytes of the string "undefined", so callers must guarantee that all three fields are strings. On the font fallback path these three values are produced by the font substitution table and are never missing.

compileSystemFontInfo

System font info is used when a PDF does not embed a font but the operating system may still hold one of the same or a similar name. It has one extra boolean flag and one optional style block compared with CSS info.

function compileSystemFontInfo(info) {
  const { encoder } = InfoUtils;
  const encodedStrings = {};
  let stringsLength = 0;
  for (const prop of SYSTEM_FONT_INFO.strings) {
    const encoded = encoder.encode(info[prop]);
    encodedStrings[prop] = encoded;
    stringsLength += 4 + encoded.length;
  }
  stringsLength += 4;
  let encodedStyleStyle,
    encodedStyleWeight,
    lengthEstimate = 1 + stringsLength;
  if (info.style) {
    encodedStyleStyle = encoder.encode(info.style.style);
    encodedStyleWeight = encoder.encode(info.style.weight);
    lengthEstimate += 4 + encodedStyleStyle.length + 4 + encodedStyleWeight.length;
  }
 
  const buffer = new ArrayBuffer(lengthEstimate);
  const data = new Uint8Array(buffer);
  const view = new DataView(buffer);
  let offset = 0;
 
  view.setUint8(offset++, info.guessFallback ? 1 : 0);
  view.setUint32(offset, 0);
  offset += 4;
  stringsLength = 0;
  for (const prop of SYSTEM_FONT_INFO.strings) {
    const encoded = encodedStrings[prop];
    const length = encoded.length;
    stringsLength += 4 + length;
    view.setUint32(offset, length);
    data.set(encoded, offset + 4);
    offset += 4 + length;
  }
  view.setUint32(offset - stringsLength - 4, stringsLength);
 
  if (info.style) {
    view.setUint32(offset, encodedStyleStyle.length);
    data.set(encodedStyleStyle, offset + 4);
    offset += 4 + encodedStyleStyle.length;
    view.setUint32(offset, encodedStyleWeight.length);
    data.set(encodedStyleWeight, offset + 4);
    offset += 4 + encodedStyleWeight.length;
  }
  assert(offset <= buffer.byteLength, 'compileSystemFontInfo: Buffer overflow');
  return buffer.transferToFixedLength(offset);
}

The layout splits into four blocks:

Field order comes from SYSTEM_FONT_INFO.strings: css, loadedName, baseFontName, src.

The total string length is a jump index: the reader does not have to scan the four strings one by one, since reading the total length takes it straight to the style block. The backfill expression offset - stringsLength - 4 looks convoluted, but it simply subtracts back to byte 1: at that point offset equals 5 plus the total string length, and subtracting both that total and the four bytes of the field itself lands exactly on the placeholder.

The position of the style block is decided solely by whether info.style exists; there is no separate presence flag. The reader assumes the style block is present when it receives systemFontInfo, and the font substitution logic guarantees that: the object which generates system font info always carries style, taken from the built-in NORMAL, BOLD, ITALIC and BOLDITALIC descriptors.

guessFallback may already have been modified before compilation. In fonts.js, when a font is a guessed fallback, guessFallback is first set to false and the font's fallbackName is appended to css. Compilation happens inside exportData, so the buffer stores the corrected final values and the main thread does not have to check again.

transferToFixedLength returns a new buffer of the specified length. lengthEstimate above is already exact, but the call is kept to produce an independent buffer of precise length, so that no slack from the estimation is carried across.

compileFontInfo

This is the only one of the five with a complete layout: fixed header, string block, two nested buffers and the font data. It first compiles the two sub-structures recursively:

function compileFontInfo(font) {
  const systemFontInfoBuffer = font.systemFontInfo
    ? compileSystemFontInfo(font.systemFontInfo)
    : null;
  const cssFontInfoBuffer = font.cssFontInfo
    ? compileCssFontInfo(font.cssFontInfo)
    : null;
 
  const { encoder } = InfoUtils;
  const encodedStrings = {};
  let stringsLength = 0;
  for (const prop of FONT_INFO.strings) {
    encodedStrings[prop] = encoder.encode(font[prop]);
    stringsLength += 4 + encodedStrings[prop].length;
  }
 
  const lengthEstimate =
    FONT_INFO.OFFSET_STRINGS +
    4 +
    stringsLength +
    4 +
    (systemFontInfoBuffer?.byteLength ?? 0) +
    4 +
    (cssFontInfoBuffer?.byteLength ?? 0) +
    4 +
    (font.data?.length ?? 0);

The sub-buffers are not transferable; they are plain byte segments copied into the parent buffer. Sending therefore transfers the parent buffer once.

Fixed Header: Booleans, Numbers and Three Optional Arrays

The first 92 bytes of the header follow the constant layout, with an assertion at the end of every block:

const buffer = new ArrayBuffer(lengthEstimate);
const data = new Uint8Array(buffer);
const view = new DataView(buffer);
let offset = 0;
 
const numBools = FONT_INFO.bools.length;
let boolByte = 0,
  boolBit = 0;
for (let i = 0; i < numBools; i++) {
  const value = font[FONT_INFO.bools[i]];
  // eslint-disable-next-line no-nested-ternary
  const bits = value === undefined ? 0x00 : value ? 0x02 : 0x01;
  boolByte |= bits << boolBit;
  boolBit += 2;
  if (boolBit === 8 || i === numBools - 1) {
    view.setUint8(offset++, boolByte);
    boolByte = 0;
    boolBit = 0;
  }
}
assert(offset === FONT_INFO.OFFSET_NUMBERS, 'compileFontInfo: Boolean properties offset mismatch');
 
for (const prop of FONT_INFO.numbers) {
  view.setFloat64(offset, font[prop]);
  offset += 8;
}
assert(offset === FONT_INFO.OFFSET_BBOX, 'compileFontInfo: Number properties offset mismatch');

There are ten boolean fields, in the order black, bold, disableFontFace, fontExtraProperties, isInvalidPDFjsFont, isType3Font, italic, missingFile, remeasure, vertical. Each boolean takes 2 bits, deliberately encoded as three states:

Four 2-bit groups pack into one byte, and ten booleans fit exactly into three bytes. The reader extracts the same bit offsets and turns 0x00 back into undefined.

Preserving undefined matters. Much of the font logic tests whether a property was explicitly set, not whether it is truthy. For instance black, remeasure and isInvalidPDFjsFont may stay undefined through the font repair flow so that later code takes the default branch. If compilation collapsed undefined into false, the main thread would behave differently from the worker.

The numeric block holds only three fields: ascent, defaultWidth and descent, each an 8-byte Float64. Font metrics feed directly into line height and text positioning, and single precision is not enough here. None of the three has a missing marker, because the font object has its defaults filled in before export.

All three optional arrays use the same encoding: one length byte followed by a fixed number of elements:

if (font.bbox) {
  view.setUint8(offset++, 4);
  for (const coord of font.bbox) {
    view.setInt16(offset, coord, true);
    offset += 2;
  }
} else {
  view.setUint8(offset++, 0);
  offset += 2 * 4; // TODO: optimize this padding away
}

Offsets still advance when the data is missing, so that every later block keeps a fixed offset. An ArrayBuffer starts zero-filled, so skipped regions are already zeros and no explicit write is needed. The TODO: optimize this padding away comment in the source shows that the author knows this is wasted space, but the decoding simplicity of fixed offsets is worth more.

The reader treats the length marker as a presence flag: a 0 returns undefined, while a non-zero value is asserted to equal the expected element count. That assertion is the second safety net for the layout: if the writer and the reader disagree about a field length, it fails immediately.

Variable Tail: Strings, Nested Buffers and Font Data

The tail uniformly uses a Uint32 length followed by data:

view.setUint32(FONT_INFO.OFFSET_STRINGS, 0);
offset += 4;
for (const prop of FONT_INFO.strings) {
  const encoded = encodedStrings[prop];
  const length = encoded.length;
  view.setUint32(offset, length);
  data.set(encoded, offset + 4);
  offset += 4 + length;
}
view.setUint32(FONT_INFO.OFFSET_STRINGS, offset - FONT_INFO.OFFSET_STRINGS - 4);
 
if (!systemFontInfoBuffer) {
  view.setUint32(offset, 0);
  offset += 4;
} else {
  const length = systemFontInfoBuffer.byteLength;
  view.setUint32(offset, length);
  assert(
    offset + 4 + length <= buffer.byteLength,
    'compileFontInfo: Buffer overflow at systemFontInfo',
  );
  data.set(new Uint8Array(systemFontInfoBuffer), offset + 4);
  offset += 4 + length;
}

The tail order is fixed:

The four strings are ordered fallbackName, loadedName, mimetype, name.

The backfill expression for the total string length is offset - OFFSET_STRINGS - 4, which yields the sum of the four length prefixes and their contents, excluding the four bytes of the length field itself. The reader uses it to skip the whole string block at once, then reads the two nested buffers and the font data in order.

The font data comes last, and the position is no accident:

  if (font.data === undefined) {
    view.setUint32(offset, 0);
    offset += 4;
  } else {
    view.setUint32(offset, font.data.length);
    data.set(font.data, offset + 4);
    offset += 4 + font.data.length;
  }
 
  assert(offset <= buffer.byteLength, "compileFontInfo: Buffer overflow");
  return buffer.transferToFixedLength(offset);
}

After binding the font to the DOM, the main thread calls clearData() immediately when fontExtraProperties is false, releasing the font file. clearData() does something simple: it sets the length field of data to 0, truncates the buffer to the prefix just after that length field, and copies it. Because the font data sits in the final block, truncation never damages any other field.

For data, the reader returns a Uint8Array view onto the buffer rather than a copy. The view lives as long as the buffer; after clearData() the length field is 0, and reading data yields undefined.

The nested buffers impose an implicit constraint on compileFontInfo: the sub-functions must finish before the parent. This is not only to obtain the lengths but also the bytes. Compiling systemFontInfo and cssFontInfo first and copying them into the parent buffer is the only workable order.

Design note: compileFontInfo combines a fixed header with a length-prefixed tail, satisfying three needs in one buffer: header fields are addressed directly by constants, tail fields are skipped in order by length, and the last block can be truncated in place to free memory. Putting the large font data last is a layout decision made purely for memory reclamation.

Compiling Patterns and Paths

Beyond fonts, two more kinds of objects have to cross the thread boundary: patterns inside Type3 fonts, and glyph outlines of Type3 fonts. They share the same traits: a large payload, a simple structure, and use only for drawing.

compilePatternInfo

The input is the pattern IR array, the product of Pattern.getIR(). The three pattern kinds map to two shapes:

function compilePatternInfo(ir) {
  let kind,
    bbox = null,
    coords = [],
    colors = [],
    colorStops = [],
    shadingType = null, // only needed for mesh patterns
    background = null; // background for mesh patterns
 
  switch (ir[0]) {
    case "RadialAxial":
      kind = ir[1] === "axial" ? 1 : 2;
      bbox = ir[2];
      colorStops = ir[3];
      if (kind === 1) {
        coords.push(...ir[4], ...ir[5]);
      } else {
        coords.push(ir[4][0], ir[4][1], ir[6], ir[5][0], ir[5][1], ir[7]);
      }
      break;
    case "Mesh":
      kind = 3;
      shadingType = ir[1];
      coords = ir[2];
      colors = ir[3];
      bbox = ir[6];
      background = ir[7];
      break;
    default:
      throw new Error(`Unsupported pattern type: ${ir[0]}`);
  }

Radial patterns pack the two centers and the two radii into six numbers, in the order start x, start y, start radius, end x, end y, end radius. Axial patterns take only the two endpoints, four numbers. The reader splits them back into the original array shapes according to kind.

Mesh patterns drop ir[4] and ir[5]: the first is the vertex count, recoverable as half the length of the coordinate array; the second is the bounds the worker already computed, which the main thread recomputes from the coordinates. The declared bounding box ir[6] and the background ir[7] are still carried.

const nCoord = Math.floor(coords.length / 2);
const nColor = Math.floor(colors.length / 4);
const nStop = colorStops.length;
 
const byteLen = 20 + nCoord * 8 + nColor * 4 + nStop * 8 + (bbox ? 16 : 0) + (background ? 3 : 0);
const buffer = new ArrayBuffer(byteLen);
const dataView = new DataView(buffer);
const u8data = new Uint8Array(buffer);
 
dataView.setUint8(PATTERN_INFO.KIND, kind);
dataView.setUint8(PATTERN_INFO.HAS_BBOX, bbox ? 1 : 0);
dataView.setUint8(PATTERN_INFO.HAS_BACKGROUND, background ? 1 : 0);
dataView.setUint8(PATTERN_INFO.SHADING_TYPE, shadingType); // Only for mesh pattern, null otherwise
dataView.setUint32(PATTERN_INFO.N_COORD, nCoord, true);
dataView.setUint32(PATTERN_INFO.N_COLOR, nColor, true);
dataView.setUint32(PATTERN_INFO.N_STOP, nStop, true);
dataView.setUint32(PATTERN_INFO.N_FIGURES, 0, true);

The 20-byte header splits into four single bytes and four 32-bit counts:

Choosing 20 for the header length also has a technical reason: Float32Array byte offsets must be 4-aligned, and 20 satisfies that, so the coordinate block can be viewed in place without another copy.

  let offset = 20;
  const coordsView = new Float32Array(buffer, offset, nCoord * 2);
  coordsView.set(coords);
  offset += nCoord * 8;
 
  u8data.set(colors, offset);
  offset += nColor * 4;
 
  for (const [pos, hex] of colorStops) {
    dataView.setFloat32(offset, pos, true);
    offset += 4;
    dataView.setUint32(offset, parseInt(hex.slice(1), 16), true);
    offset += 4;
  }
  if (bbox) {
    for (const v of bbox) {
      dataView.setFloat32(offset, v, true);
      offset += 4;
    }
  }
 
  if (background) {
    u8data.set(background, offset);
  }
  return buffer;
}

The body follows the order coordinates, colors, color stops, declared bounding box, background:

Color stops are parsed with parseInt(hex.slice(1), 16), assuming a six-digit hex input such as #a1b2c3. The reader rebuilds the same string form with toString(16).padStart(6, "0"). The round trip loses the letter case, but the color value itself is unchanged.

N_COORD, N_COLOR and N_STOP are all derived from array lengths, so a mismatch between declaration and reality is impossible. N_FIGURES is a reserved field that neither side reads in this version.

The return value of compilePatternInfo is not truncated, because byteLen is already exact. PartialEvaluator.parseShading sends it as a transferable, and it is used only while parsing a Type3 font; non-Type3 patterns still go through the plain obj message and send the IR array directly. The main thread's PatternInfo calls getIR() at once to rebuild the IR array. For mesh patterns coords and colors are views onto the buffer with no copy; for axial and radial patterns the coordinates are copied into a small array. The difference reflects a trade-off: zero-copy where the payload is large, convenience first where it is small.

compileFontPathInfo

A glyph outline is a flat typed array: each glyph consists of several commands, and the first number of every command is a drawing opcode followed by coordinates. The worker has already transformed the coordinates with the font matrix, and the main thread only needs to replay the same opcodes.

function compileFontPathInfo(path) {
  if (typeof PDFJSDev === 'undefined' || PDFJSDev.test('TESTING')) {
    assert(
      FeatureTest.isFloat16ArraySupported
        ? path instanceof Float16Array
        : path instanceof Float32Array,
      'compileFontPathInfo: Unexpected path format.',
    );
  }
  return path.slice().buffer;
}

The function does exactly one thing: make a copy and hand over the underlying buffer. slice() cannot be skipped, for two reasons:

1. The cache still uses the original array. CompiledFont stores each glyph's compiled result in #compiledGlyphs, and later lookups, checks and debugging may read it again. Transferring the original buffer would leave those cached arrays detached.
2. The view may be larger than the data. A typed array can be a view onto a larger buffer; slice() guarantees the output starts at 0 and has the exact length.

The buffer carries no type tag. The reader picks the constructor from the same build condition as the writer: the MOZCENTRAL build, or a browser that supports Float16Array, reads half precision, everything else reads single precision. Both sides live in the same build artefact, so the decision matches and no extra tag is needed. The cost: if that assumption breaks, the reader interprets the whole block with the wrong element width and produces a pile of meaningless coordinates.

Design note: compileFontPathInfo is the most protocol-like function in the file: it defines no fields and only agrees that these bytes are the binary form of the original array. The shorter the agreement, the more it depends on context; the type tag is omitted, which saves one byte per glyph and one branch at decode time. The premise is that both sides share the same runtime feature, and that premise holds in the worker architecture.

Conclusion

obj_bin_transform_core.js is PDF.js's cross-thread serialization layer: upward it receives font, pattern and glyph objects from the worker, downward it emits transferable ArrayBuffers, and in between the field constants in the shared directory keep both sides aligned.

Where It Sits in the Overall Chain

In one sentence: the worker flattens objects into bytes, the main thread rebuilds them as its own objects, and no class instance ever crosses the thread boundary.

Five Design Principles

1. Protocol separated from implementation. Field order, offsets and types all live in shared/obj_bin_transform_utils.js; the writer and the reader reference the constants instead of each maintaining its own copy of the agreement.
2. Only necessary fields travel. The three boolean states, three metrics, four names and the optional bbox and fontMatrix are what the render path actually reads; the wide debugging table and the ToUnicode map stay in extra.
3. Fixed header plus length-prefixed tail. The header is addressed by constants, the tail is skipped in order by length, and optional fields are expressed by a zero length or a zero marker, with no field bitmap needed.
4. Transfer ownership instead of copying. Compile functions return an exclusive buffer that goes into the transferables list; data that must be reused is slice()d first so that cached arrays are not detached.
5. Assertions lock the layout. Every block checks its end offset, every nested buffer checks its bounds, every array checks its length; layout drift shows up at compile time or in tests instead of surfacing as misplaced glyphs during main-thread rendering.

The next article returns to the worker's object loading path, to see how ObjectLoader reads an object graph ahead of time over a chunked network stream so that later synchronous parsing never runs out of bytes.

StreamPDF design principle: anything that can be handled ahead of time is absolutely not deferred to parse time; the server solves all performance problems once. The browser only fetches and renders the data it must.