PDF.js Source Walkthrough: cmap

Introduction

The previous thirty-two articles have already laid out the main line of "how a document is opened and how a page is evaluated": xref handles object addressing, parser handles the lexing and parsing from bytes to objects, document handles the page model, evaluator handles the content stream to operatorList, stream and the various decoders handle byte streams, primitives handles the basic objects, and catalog handles document-level semantics. At the content-evaluation level one unavoidable question remains: which glyphs do the strings in a content stream actually refer to?

For simple fonts, the character code can be looked up directly in /Encoding to get a glyph name; for composite fonts, the character code first goes through a CMap to become a CID, and the CID is then used to fetch glyph widths and outlines. For text extraction, the character code must also go through a ToUnicode CMap to become Unicode. src/core/cmap.js is the shared foundation of this layer: it defines the CMap data structure, the parser for text CMaps, the loading path for built-in CMaps and the unified entry point CMapFactory.

In one sentence: cmap.js translates "character codes" into "CIDs or Unicode" and shapes that mapping into a form the rendering layer and the text layer can index directly.

Module Overview

cmap.js is 724 lines and its body splits into eight blocks:

Its place in the overall chain can be drawn like this:

// Font dictionary /Encoding, /ToUnicode
//   → CMapFactory.create
//   → createBuiltInCMap(name)          built-in name → .bcmap → BinaryCMapReader
//   → parseCMap(lexer)                 embedded stream → Lexer → text operators
//   → CMap.codespaceRanges / _map
//   → evaluator: composite fonts take CIDs, ToUnicode takes Unicode
//   → font / text layer

Version note: This article follows exactly this version of cmap.js in the current repository. In this version CMap has no adjustMapping, reverseMap or getCharCode methods, and codespace ranges are not looked up with binary search; it provides lookup, contains, charCodeOf, readCharCode and getCharCodeLength, and _map is a plain array rather than a Map. CMapFactory.create accepts a destructured object { encoding, fetchBuiltInCMap, useCMap } rather than positional arguments. Do not apply APIs from other versions you remember when reading the source.

Two Kinds of CMap: CID Mapping and ToUnicode Mapping

A CMap in the PDF specification is a mapping table from character codes to target values. The target value has two forms, which correspond exactly to two completely different chains of use:

• CID CMap: the target value is an integer CID. A composite font's /Encoding points to it, and the font layer uses the CID to fetch the glyph from /CIDToGIDMap, the width table and the outlines.
• ToUnicode CMap: the target value is a string, usually one or more UTF-16BE code units. Text extraction and copy-paste depend on it to turn character codes back into Unicode.

The two are written differently in a file, but their representation inside CMap is the same set: the index into _map is the source character code and the value is either an integer CID or a string with one byte per character. The constructor states this in a comment:

class CMap {
  constructor(builtInCMap = false) {
    // Codespace ranges are stored as follows:
    // [[1BytePairs], [2BytePairs], [3BytePairs], [4BytePairs]]
    // where nBytePairs are ranges e.g. [low1, high1, low2, high2, ...]
    this.codespaceRanges = [[], [], [], []];
    this.numCodespaceRanges = 0;
    // Map entries have one of two forms.
    // - cid chars are 16-bit unsigned integers, stored as integers.
    // - bf chars are variable-length byte sequences, stored as strings, with
    //   one byte per character.
    this._map = [];
    this.name = "";
    this.vertical = false;
    this.useCMap = null;
    this.builtInCMap = builtInCMap;
  }

The meaning of a few fields must be pinned down first:

Design note: Sharing one class between CID CMaps and ToUnicode CMaps is not laziness, but because their indexing scheme is exactly the same: both are sparse tables from source character code to target value, and both need the same codespace check to decide how many bytes a character code occupies. The only differences are the type of the target value and which downstream consumer uses it. Keeping the differences at the call site lets CMap itself serve both the font layer and the text layer.

The differences can be collected into a single table:

Data Structure: codespace and Sparse Mapping

addCodespaceRange: Buckets by Code Length

  addCodespaceRange(n, low, high) {
    this.codespaceRanges[n - 1].push(low, high);
    this.numCodespaceRanges++;
  }

The first argument n is the code length, not the bucket index. The source uses n - 1 to find the matching bucket and pushes low and high in consecutively. Therefore:

A bucket stores a flat array rather than an array of objects. Matching then only needs one loop with a stride of 2, and no temporary object has to be created for each range; for the hundreds or thousands of ranges in one built-in CMap this difference is real.

Why _map Is a Sparse Array

The index into _map is the source character code itself. The double-byte codes common in PDF reach at most 0xFFFF, so the array length usually does not exceed 65536; but the specification allows four-byte codes, and a damaged file may even contain code values close to 0xFFFFFFFF. The source chooses a plain array rather than a Map for two reasons:

1. The vast majority of queries are direct indexes. lookup(code) is just this._map[code], and V8 is very fast at array access with an integer index.
2. The cost of a sparse array is bounded. A genuinely huge code value only leaves holes and does not allocate memory for every position; iteration then uses for..in to step around those holes.

The price is that reverse lookup and iteration must handle sparseness themselves, which is exactly why charCodeOf and forEach exist.

codespace: How Many Bytes One Character Code Takes

Strings inside a PDF content stream carry no length information. When Tj receives a run of bytes it must know whether the first character code is 1 byte or 2 bytes, otherwise the whole string is misaligned. codespace answers this question.

  readCharCode(str, offset, out) {
    let c = 0;
    const codespaceRanges = this.codespaceRanges;
    // 9.7.6.2 CMap Mapping
    // The code length is at most 4.
    for (let n = 0, nn = codespaceRanges.length; n < nn; n++) {
      c = ((c << 8) | str.charCodeAt(offset + n)) >>> 0;
      // Check each codespace range to see if it falls within.
      const codespaceRange = codespaceRanges[n];
      for (let k = 0, kk = codespaceRange.length; k < kk;) {
        const low = codespaceRange[k++];
        const high = codespaceRange[k++];
        if (c >= low && c <= high) {
          out.charcode = c;
          out.length = n + 1;
          return;
        }
      }
    }
    out.charcode = 0;
    out.length = 1;
  }

This loop has three key points:

1. Bytes accumulate one step at a time. The first round reads 1 byte, the second combines the first two bytes into 16 bits, the third into 24 bits and the fourth into 32 bits. >>> 0 guarantees an unsigned integer and keeps the highest bit from being interpreted as a negative number.
2. Each round only checks the bucket for the matching code length. A one-byte candidate is compared only with single-byte ranges, and a two-byte candidate only with double-byte ranges, so a two-byte code can never falsely match a single-byte range.
3. The bucket is scanned linearly and the first hit wins. The source does not sort and does not binary-search; each bucket is a flat array compared pair by pair as low, high from start to end. For typical CMaps the number of ranges in a bucket is limited, and a linear scan is simpler than maintaining a sorted structure.

getCharCodeLength is the "length only" version of the same logic:

  getCharCodeLength(charCode) {
    const codespaceRanges = this.codespaceRanges;
    for (let n = 0, nn = codespaceRanges.length; n < nn; n++) {
      // Check each codespace range to see if it falls within.
      const codespaceRange = codespaceRanges[n];
      for (let k = 0, kk = codespaceRange.length; k < kk;) {
        const low = codespaceRange[k++];
        const high = codespaceRange[k++];
        if (charCode >= low && charCode <= high) {
          return n + 1;
        }
      }
    }
    return 1;
  }

It receives an already assembled character code rather than a string and an offset. Both return which bucket matched, that is the code length. When nothing is found they fall back to 1 byte uniformly, which is a fault-tolerance choice: when a broken CMap lacks codespaces, parsing can at least continue byte by byte instead of throwing and failing the whole page.

Design note: The codespace check lives on CMap rather than in the font or the content-stream parser because it describes the byte structure of the encoding itself. The same character code may be 1 byte under one CMap and 2 bytes under another; the font layer only needs to ask CMap and does not need to maintain its own branch for whether this encoding is single-byte or double-byte. This is also why PDF.js can handle Identity-H, GBK-EUC-H and UniJIS-UCS2-H uniformly.

Writing: How Single Points and Ranges Expand

mapOne

  // This is used for both bf and cid chars.
  mapOne(src, dst) {
    this._map[src] = dst;
  }

The smallest write interface. src is the character code and dst may be a CID integer or a UTF-16BE string. A single-point write performs no range expansion and does not check the target type.

mapCidRange

  mapCidRange(low, high, dstLow) {
    if (high - low > MAX_MAP_RANGE) {
      throw new Error("mapCidRange - ignoring data above MAX_MAP_RANGE.");
    }
    while (low <= high) {
      this._map[low++] = dstLow++;
    }
  }

A CID range is an arithmetic mapping: the source runs from low to high, and the CID starts at dstLow and increases by one per item. The source writes each item directly into the array with while instead of keeping a triple of "low, high, starting CID" to compute at query time. Queries are therefore O(1), at the cost of expanding the range into concrete entries during parsing.

MAX_MAP_RANGE is 2 to the 24th power minus 1:

// Heuristic to avoid hanging the worker-thread for CMap data with ridiculously
// large ranges, such as e.g. 0xFFFFFFFF (fixes issue11922_reduced.pdf).
const MAX_MAP_RANGE = 2 ** 24 - 1; // = 0xFFFFFF

A broken file may declare a CID range from 0 to 0xFFFFFFFF. Expanded as written, the worker thread would hang inside that single loop or even exhaust memory. The source checks the span before expanding and throws when it exceeds the limit; parseCMap catches that error and warns, then continues with the following records. This is the typical way of keeping one bad record from taking down the whole CMap.

mapBfRange

  mapBfRange(low, high, dstLow) {
    if (high - low > MAX_MAP_RANGE) {
      throw new Error("mapBfRange - ignoring data above MAX_MAP_RANGE.");
    }
    const lastByte = dstLow.length - 1;
    while (low <= high) {
      this._map[low++] = dstLow;
      // Only the last byte has to be incremented (in the normal case).
      const nextCharCode = dstLow.charCodeAt(lastByte) + 1;
      if (nextCharCode > 0xff) {
        dstLow =
          dstLow.substring(0, lastByte - 1) +
          String.fromCharCode(dstLow.charCodeAt(lastByte - 1) + 1) +
          "\x00";
        continue;
      }
      dstLow =
        dstLow.substring(0, lastByte) + String.fromCharCode(nextCharCode);
    }
  }

The target of a bf range is a string, so the increment rule has one more layer than CID: only the last byte is incremented. For example, when the target string is "\x00A", expanding the range yields "\x00A", "\x00B" and "\x00C" in turn. If the last byte overflows from 0xFF, the second-to-last byte is incremented and the last byte resets to zero. The source performs this carry with string slices and String.fromCharCode rather than converting the string to a number and back.

This matches the bfrange rule in the PDF specification, where the target string is incremented in big-endian byte order. Note that it only handles a carry out of the last byte and does not implement general big-integer addition of arbitrary length; this is a simplification for the common form in the specification, and longer target strings are usually expressed through beginbfchar or the array form.

mapBfRangeToArray

  mapBfRangeToArray(low, high, array) {
    if (high - low > MAX_MAP_RANGE) {
      throw new Error("mapBfRangeToArray - ignoring data above MAX_MAP_RANGE.");
    }
    const ii = array.length;
    let i = 0;
    while (low <= high && i < ii) {
      this._map[low] = array[i++];
      ++low;
    }
  }

This is the array branch of beginbfrange: the target is not a single starting string but a whole array, and source codes map one-to-one onto array elements. The loop is bounded by both high and the array length, so a short array does not overrun and a long array does not keep writing. The specification requires the array length to equal the range length; here the choice is to take whichever is shorter, which is tolerance rather than strict validation.

The three range writers can be compared side by side:

Design note: Ranges are expanded into item-by-item arrays at parse time rather than kept as an interval tree. This makes lookup a single array access, and every character in a content stream no longer needs an interval search. The price is that memory grows with the span of the range, which is why the source puts a hard cap on expansion through MAX_MAP_RANGE. This is the classic trade of one-time parse cost for per-query cost: one CMap is reused by every character on the same page, so expanding once is worthwhile.

Lookup, contains and Reverse Lookup

lookup and contains

  lookup(code) {
    return this._map[code];
  }
 
  contains(code) {
    return this._map[code] !== undefined;
  }

lookup performs no type check and no range check; it returns the array element directly. When nothing is found it returns undefined, and callers usually test with if (cid) or !== undefined. contains merely wraps "is it undefined" into a query with clearer semantics, and extendCMap uses it when merging the parent mapping.

forEach

  forEach(callback) {
    // Most maps have fewer than 65536 entries, and for those we use normal
    // array iteration. But really sparse tables are possible -- e.g. with
    // indices in the *billions*. For such tables we use for..in, which isn't
    // ideal because it stringifies the indices for all present elements, but
    // it does avoid iterating over every undefined entry.
    const map = this._map;
    const length = map.length;
    if (length <= 0x10000) {
      for (let i = 0; i < length; i++) {
        if (map[i] !== undefined) {
          callback(i, map[i]);
        }
      }
    } else {
      for (const i in map) {
        callback(i, map[i]);
      }
    }
  }

There is a clear dividing line here: when the length does not exceed 0x10000 a normal for loop is used, and above that for..in. 65536 is the full space of double-byte codes and the vast majority of CMaps stay inside it; exceeding it usually means three-byte or four-byte codes are present and the array may be very sparse. A normal for would visit every hole, while for..in only visits assigned indexes at the cost of stringifying them. The source picks the latter because "iterating billions of holes" is worse than "stringifying a handful of real entries".

Note that the index passed to the callback in the for..in branch is a string rather than a number. readToUnicode therefore tests typeof token === "number" specifically to handle CMaps that contain only CIDs.

charCodeOf

  charCodeOf(value) {
    // `Array.prototype.indexOf` is *extremely* inefficient for arrays which
    // are both very sparse and very large (see issue8372.pdf).
    const map = this._map;
    if (map.length <= 0x10000) {
      return map.indexOf(value);
    }
    for (const charCode in map) {
      if (map[charCode] === value) {
        return charCode | 0;
      }
    }
    return -1;
  }

This is the reverse lookup: given a CID or a Unicode value, ask which character code maps to it. A small array uses indexOf directly; a large sparse array uses a linear scan with for..in. The return value is converted back to a number with charCode | 0, because the keys of for..in are strings. When nothing is found it returns -1 rather than undefined, which makes numeric tests easy for callers.

charCodeOf is linear in complexity. It does not maintain a reverse index because reverse lookups are far less frequent than forward lookup, so building another table for it is not worthwhile. This is a very typical trade: the forward path must be fast, the reverse path only has to be correct.

length and isIdentityCMap

  get length() {
    return this._map.length;
  }
 
  get isIdentityCMap() {
    if (!(this.name === "Identity-H" || this.name === "Identity-V")) {
      return false;
    }
    if (this._map.length !== 0x10000) {
      return false;
    }
    for (let i = 0; i < 0x10000; i++) {
      if (this._map[i] !== i) {
        return false;
      }
    }
    return true;
  }

length returns the array length and is not the number of valid mapping entries; a sparse array may have a large length with only a few entries. isIdentityCMap continues checking only when the name is Identity-H or Identity-V, and it requires the array length to be exactly 65536 with every entry equal to its own index. CMapFactory.create uses it to decide that an embedded stream actually parsed into an identity mapping, and replaces it with the cheaper IdentityCMap.

IdentityCMap: Implicit Identity, No Materialised Array

Identity-H and Identity-V are the most common encodings in PDF: the character code is the CID and no table lookup is needed. Expanded like an ordinary CMap, _map would hold 65536 entries each equal to its own index, which is pure waste. IdentityCMap overrides the query methods in a subclass and turns that relation into something implicit:

// A special case of CMap, where the _map array implicitly has a length of
// 65536 and each element is equal to its index.
class IdentityCMap extends CMap {
  constructor(vertical, n) {
    super();
 
    this.vertical = vertical;
    this.addCodespaceRange(n, 0, 0xffff);
  }

The constructor registers a single codespace range: code length n (2 for Identity-H), from 0 to 0xFFFF. Every method that would write to _map is changed to unreachable:

  mapCidRange(low, high, dstLow) {
    unreachable("should not call mapCidRange");
  }
 
  mapBfRange(low, high, dstLow) {
    unreachable("should not call mapBfRange");
  }
 
  mapBfRangeToArray(low, high, array) {
    unreachable("should not call mapBfRangeToArray");
  }
 
  mapOne(src, dst) {
    unreachable("should not call mapCidOne");
  }

The query side simply returns the source code itself:

  lookup(code) {
    return Number.isInteger(code) && code <= 0xffff ? code : undefined;
  }
 
  contains(code) {
    return Number.isInteger(code) && code <= 0xffff;
  }
 
  charCodeOf(value) {
    return Number.isInteger(value) && value <= 0xffff ? value : -1;
  }

forEach calls back for every value from 0 to 0xFFFF, and length always returns 0x10000. getMap materialises a concrete array on demand:

  getMap() {
    // Sometimes identity maps must be instantiated, but it's rare.
    const map = new Array(0x10000);
    for (let i = 0; i <= 0xffff; i++) {
      map[i] = i;
    }
    return map;
  }

Only when a caller explicitly needs a plain array is one generated, which the comment calls rare. The isIdentityCMap getter is unreachable in the subclass because the type itself already states the identity and nothing needs to be scanned again.

Design note: IdentityCMap turns the mapping relation from data into a type. A normal CMap needs 65536 entries to prove that it is an identity mapping, while IdentityCMap needs only a class name and one instanceof check. After an embedded stream has been parsed, CMapFactory runs a single recognition with isIdentityCMap and can replace an ordinary object that merely happens to look like an identity mapping with this lightweight type; every later query then follows the implicit path and never touches an array again.

Parsing a Text CMap

A CMap in the form of an embedded stream is plain text whose syntax resembles PDF objects but which is driven by a dedicated set of operators. Parsing has two layers: the outer parseCMap scans operators, and the inner parseXxx functions read the corresponding block until they reach an end... command.

Hex Strings and Type Checks

Character codes inside a CMap are written as hex strings, for example <8140>. strToInt assembles them into an integer in big-endian order:

function strToInt(str) {
  let a = 0;
  for (let i = 0; i < str.length; i++) {
    a = (a << 8) | str.charCodeAt(i);
  }
  return a >>> 0;
}

It relies on Lexer having already decoded the hex string into a string with one byte per character. a << 8 shifts left by 8 bits on each round and supports up to 4 bytes; >>> 0 keeps a four-byte code from becoming negative.

Two validation functions concentrate the failure points of a bad CMap:

function expectString(obj) {
  if (typeof obj !== 'string') {
    throw new FormatError('Malformed CMap: expected string.');
  }
}
 
function expectInt(obj) {
  if (!Number.isInteger(obj)) {
    throw new FormatError('Malformed CMap: expected int.');
  }
}

They do not return a boolean but throw FormatError directly. The outer catch in parseCMap catches it and warns, then continues with the next record. A single bad token therefore does not terminate the whole CMap.

bf Blocks: beginbfchar and beginbfrange

parseBfChar handles single-point mappings from source to target string:

function parseBfChar(cMap, lexer) {
  while (true) {
    let obj = lexer.getObj();
    if (obj === EOF) {
      break;
    }
    if (isCmd(obj, 'endbfchar')) {
      return;
    }
    expectString(obj);
    const src = strToInt(obj);
    obj = lexer.getObj();
    // TODO are /dstName used?
    expectString(obj);
    const dst = obj;
    cMap.mapOne(src, dst);
  }
}

Each round reads two strings: the first is the source and the second is the target string. A leftover comment in the source still wonders whether /dstName is used, which shows that the optional target name field in the specification is ignored here. It returns normally on endbfchar; it also returns on EOF without throwing, which is tolerance for truncated files.

parseBfRange adds an array branch:

function parseBfRange(cMap, lexer) {
  while (true) {
    let obj = lexer.getObj();
    if (obj === EOF) {
      break;
    }
    if (isCmd(obj, 'endbfrange')) {
      return;
    }
    expectString(obj);
    const low = strToInt(obj);
    obj = lexer.getObj();
    expectString(obj);
    const high = strToInt(obj);
    obj = lexer.getObj();
    if (Number.isInteger(obj) || typeof obj === 'string') {
      const dstLow = Number.isInteger(obj) ? String.fromCharCode(obj) : obj;
      cMap.mapBfRange(low, high, dstLow);
    } else if (isCmd(obj, '[')) {
      obj = lexer.getObj();
      const array = [];
      while (!isCmd(obj, ']') && obj !== EOF) {
        array.push(obj);
        obj = lexer.getObj();
      }
      cMap.mapBfRangeToArray(low, high, array);
    } else {
      break;
    }
  }
  throw new FormatError('Invalid bf range.');
}

The target value may take three forms: an integer, a string or an array. An integer is converted by String.fromCharCode into a single-character string and passed to mapBfRange; an array is collected item by item and passed to mapBfRangeToArray. If the third item is neither an integer or string nor a left square bracket, the loop breaks and a uniform Invalid bf range is thrown.

cid Blocks: begincidchar and begincidrange

CID blocks are parsed in the same shape as bf blocks, except that the target value must be an integer CID:

function parseCidChar(cMap, lexer) {
  while (true) {
    let obj = lexer.getObj();
    if (obj === EOF) {
      break;
    }
    if (isCmd(obj, 'endcidchar')) {
      return;
    }
    expectString(obj);
    const src = strToInt(obj);
    obj = lexer.getObj();
    expectInt(obj);
    const dst = obj;
    cMap.mapOne(src, dst);
  }
}

parseCidRange instead reads three values, namely low, high and the starting CID, and then calls mapCidRange:

function parseCidRange(cMap, lexer) {
  while (true) {
    let obj = lexer.getObj();
    if (obj === EOF) {
      break;
    }
    if (isCmd(obj, 'endcidrange')) {
      return;
    }
    expectString(obj);
    const low = strToInt(obj);
    obj = lexer.getObj();
    expectString(obj);
    const high = strToInt(obj);
    obj = lexer.getObj();
    expectInt(obj);
    const dstLow = obj;
    cMap.mapCidRange(low, high, dstLow);
  }
}

CID blocks have no array branch because a CID target can only be an integer. When the target is not an integer expectInt throws directly, and the outer parseCMap catches it and continues.

codespace Blocks

function parseCodespaceRange(cMap, lexer) {
  while (true) {
    let obj = lexer.getObj();
    if (obj === EOF) {
      break;
    }
    if (isCmd(obj, 'endcodespacerange')) {
      return;
    }
    if (typeof obj !== 'string') {
      break;
    }
    const low = strToInt(obj);
    obj = lexer.getObj();
    if (typeof obj !== 'string') {
      break;
    }
    const high = strToInt(obj);
    cMap.addCodespaceRange(obj.length, low, high);
  }
  throw new FormatError('Invalid codespace range.');
}

The code length comes from the string length of the high value, not the low value. Normally both have the same length; if a producer omits leading zeros in the low value but writes the high value in full, taking the high value gives a more reliable length. obj.length counts bytes because Lexer decoded the hex string into a string with one character per byte.

WMode and CMapName

function parseWMode(cMap, lexer) {
  const obj = lexer.getObj();
  if (Number.isInteger(obj)) {
    cMap.vertical = !!obj;
  }
}
 
function parseCMapName(cMap, lexer) {
  const obj = lexer.getObj();
  if (obj instanceof Name) {
    cMap.name = obj.name;
  }
}

Both parsers read exactly one value. WMode is silently ignored when it is not an integer, and so is CMapName when it is not a Name. They do not throw because these two fields only affect layout direction and later recognition, not the code mapping itself.

The Outer Loop: parseCMap

async function parseCMap(cMap, lexer, fetchBuiltInCMap, useCMap) {
  let previous, embeddedUseCMap;
  objLoop: while (true) {
    try {
      const obj = lexer.getObj();
      if (obj === EOF) {
        break;
      } else if (obj instanceof Name) {
        if (obj.name === 'WMode') {
          parseWMode(cMap, lexer);
        } else if (obj.name === 'CMapName') {
          parseCMapName(cMap, lexer);
        }
        previous = obj;
      } else if (obj instanceof Cmd) {
        switch (obj.cmd) {
          case 'endcmap':
            break objLoop;
          case 'usecmap':
            if (previous instanceof Name) {
              embeddedUseCMap = previous.name;
            }
            break;
          case 'begincodespacerange':
            parseCodespaceRange(cMap, lexer);
            break;
          case 'beginbfchar':
            parseBfChar(cMap, lexer);
            break;
          case 'begincidchar':
            parseCidChar(cMap, lexer);
            break;
          case 'beginbfrange':
            parseBfRange(cMap, lexer);
            break;
          case 'begincidrange':
            parseCidRange(cMap, lexer);
            break;
        }
      }
    } catch (ex) {
      if (ex instanceof MissingDataException) {
        throw ex;
      }
      warn('Invalid cMap data: ' + ex);
      continue;
    }
  }
 
  if (!useCMap && embeddedUseCMap) {
    // Load the useCMap definition from the file only if there wasn't one
    // specified.
    useCMap = embeddedUseCMap;
  }
  if (useCMap) {
    return extendCMap(cMap, fetchBuiltInCMap, useCMap);
  }
  return cMap;
}

This loop is the dispatch centre of the whole file, and a few details deserve separate explanation:

1. previous records the last Name. A usecmap inside a CMap is written as "/Identity-H usecmap", where usecmap is the command and the name comes before it. So when the parser reaches usecmap the source takes previous rather than reading another token.
2. WMode and CMapName are prefix markers. They are Names themselves and are followed by a value; the source calls the corresponding parser in place when it sees the Name and consumes the value, which keeps the main loop from mistaking the value for a new command.
3. endcmap leaves through a label. break objLoop ends the outer loop directly rather than relying on EOF. Many CMaps have whitespace or comments after endcmap, and reaching EOF would also end the loop, but the label is clearer.
4. Exceptions are classified. MissingDataException must be rethrown unchanged because it means the bytes have not arrived yet, not that the data is broken; every other exception only warns and continues so the next record can still be parsed.

The mapping from operators to parsers can be collected into a table:

useCMap: How the Parent Map Is Merged

PDF allows a CMap to write only the delta and inherit the rest from another CMap:

if (!useCMap && embeddedUseCMap) {
  // Load the useCMap definition from the file only if there wasn't one
  // specified.
  useCMap = embeddedUseCMap;
}
if (useCMap) {
  return extendCMap(cMap, fetchBuiltInCMap, useCMap);
}

The priority is that an explicit useCMap passed by the caller wins over the embeddedUseCMap written in the stream. This matches the CMapFactory.create scenario: when the caller already knows the parent mapping, the declaration in the stream does not need to be parsed.

async function extendCMap(cMap, fetchBuiltInCMap, useCMap) {
  cMap.useCMap = await createBuiltInCMap(useCMap, fetchBuiltInCMap);
  // If there aren't any code space ranges defined clone all the parent ones
  // into this cMap.
  if (cMap.numCodespaceRanges === 0) {
    const useCodespaceRanges = cMap.useCMap.codespaceRanges;
    for (let i = 0; i < useCodespaceRanges.length; i++) {
      cMap.codespaceRanges[i] = useCodespaceRanges[i].slice();
    }
    cMap.numCodespaceRanges = cMap.useCMap.numCodespaceRanges;
  }
  // Merge the map into the current one, making sure not to override
  // any previously defined entries.
  cMap.useCMap.forEach(function (key, value) {
    if (!cMap.contains(key)) {
      cMap.mapOne(key, value);
    }
  });
 
  return cMap;
}

There are three merge rules:

1. The parent mapping is loaded by name only. createBuiltInCMap accepts built-in names only; whatever usecmap points to must be a built-in CMap and cannot be another embedded stream. This is the common usage in the specification and it also bounds parent loading to the built-in set, preventing arbitrary chained references between embedded streams.
2. codespaces are inherited only when the child map has none at all. Parent buckets are cloned only when numCodespaceRanges === 0; as soon as the child defines even one range it uses only its own. This avoids parent and child ranges piling up and making the code-length decision ambiguous.
3. The child wins. Every parent entry is written only when the child has no such key. contains tests _map[code] !== undefined, so a key the child explicitly wrote is never overwritten.

createBuiltInCMap is the end of this chain:

async function createBuiltInCMap(name, fetchBuiltInCMap) {
  if (name === 'Identity-H') {
    return new IdentityCMap(false, 2);
  } else if (name === 'Identity-V') {
    return new IdentityCMap(true, 2);
  }
  if (!BUILT_IN_CMAPS.includes(name)) {
    throw new Error('Unknown CMap name: ' + name);
  }
  if (!fetchBuiltInCMap) {
    throw new Error('Built-in CMap parameters are not provided.');
  }
 
  const { cMapData, isCompressed } = await fetchBuiltInCMap(name);
  const cMap = new CMap(true);
 
  if (isCompressed) {
    return new BinaryCMapReader().process(cMapData, cMap, useCMap =>
      extendCMap(cMap, fetchBuiltInCMap, useCMap),
    );
  }
  const lexer = new Lexer(new Stream(cMapData));
  return parseCMap(cMap, lexer, fetchBuiltInCMap, null);
}

The three branches correspond to three built-in forms:

BUILT_IN_CMAPS is a static array of strings containing Adobe's four UCS2 mappings, the various EUC/RKSJ/GBK/B5 encodings and a large number of names such as UniGB-UTF16-H and UniJIS-UCS2-H. It is only a whitelist of names and carries no mapping data. The real data is fetched by fetchBuiltInCMap from .bcmap or .cmap resources, with the path decided by the configuration of the layer above.

CMapFactory: The Unified Entry Point

class CMapFactory {
  static async create({ encoding, fetchBuiltInCMap, useCMap }) {
    if (encoding instanceof Name) {
      return createBuiltInCMap(encoding.name, fetchBuiltInCMap);
    } else if (encoding instanceof BaseStream) {
      if (encoding.isAsync) {
        const bytes = await encoding.asyncGetBytes();
        if (bytes) {
          encoding = new Stream(bytes, 0, bytes.length, encoding.dict);
        }
      }
      const parsedCMap = await parseCMap(
        /* cMap = */ new CMap(),
        /* lexer = */ new Lexer(encoding),
        fetchBuiltInCMap,
        useCMap,
      );
 
      if (parsedCMap.isIdentityCMap) {
        return createBuiltInCMap(parsedCMap.name, fetchBuiltInCMap);
      }
      return parsedCMap;
    }
    throw new Error('Encoding required.');
  }
}

It recognises exactly two kinds of input:

The async stream branch is worth noting: parseCMap needs random access, and a stream with isAsync (for example a network chunk stream) cannot guarantee that synchronous getByte always hits, so the source first reads all the bytes and replaces the stream with an in-memory Stream before parsing. This loads the CMap stream into memory at once, but CMaps are usually only tens to hundreds of KB, and the stability is worth more.

The parsedCMap.isIdentityCMap test after parsing is the last optimisation: if the embedded stream contains a complete Identity-H mapping, the factory discards the freshly parsed 65536-entry array and goes back to the lightweight IdentityCMap from createBuiltInCMap. When the name is not Identity-H or Identity-V, isIdentityCMap returns false immediately without a 65536-step scan.

Callers: Font Layer and Text Layer

CMapFactory has two main call sites in evaluator.js. When a composite font is loaded, /Encoding is passed in:

if (composite) {
  const cidEncoding = baseDict.get('Encoding');
  if (cidEncoding instanceof Name) {
    properties.cidEncoding = cidEncoding.name;
  }
  const cMap = await CMapFactory.create({
    encoding: cidEncoding,
    fetchBuiltInCMap: this._fetchBuiltInCMapBound,
    useCMap: null,
  });
  properties.cMap = cMap;
  properties.vertical = properties.cMap.vertical;
}

Here encoding may be a Name or a stream; the factory already handles both forms. properties.cMap is later used by the font layer to turn character codes in the content stream into CIDs, while properties.vertical directly affects the layout direction.

Text extraction goes through readToUnicode:

  async readToUnicode(cmapObj) {
    if (!cmapObj) {
      return null;
    }
    if (cmapObj instanceof Name) {
      const cmap = await CMapFactory.create({
        encoding: cmapObj,
        fetchBuiltInCMap: this._fetchBuiltInCMapBound,
        useCMap: null,
      });
 
      if (cmap instanceof IdentityCMap) {
        return new IdentityToUnicodeMap(0, 0xffff);
      }
      return new ToUnicodeMap(cmap.getMap());
    }

A Name form ToUnicode is converted directly into a ToUnicodeMap; if the factory returns an IdentityCMap, IdentityToUnicodeMap is used instead and no 65536 entries have to be copied. The stream form is in the second half of the same method: it converts each bf target string from UTF-16BE into Unicode code points and handles surrogate pairs. That conversion does not happen inside CMap but stays in evaluator, because CMap only needs to store the "one character per byte" string faithfully.

Fetching the built-in data is the job of PartialEvaluator.fetchBuiltInCMap:

  async fetchBuiltInCMap(name) {
    const cachedData = this.builtInCMapCache.get(name);
    if (cachedData) {
      return cachedData;
    }
    let data;
 
    if (this.options.useWorkerFetch) {
      // Only compressed CMaps are (currently) supported here.
      data = {
        cMapData: await fetchBinaryData(`${this.options.cMapUrl}${name}.bcmap`),
        isCompressed: true,
      };
    } else {
      if (typeof PDFJSDev !== "undefined" && PDFJSDev.test("MOZCENTRAL")) {
        throw new Error("Only worker-thread fetching supported.");
      }
      // Get the data on the main-thread instead.
      data = {
        cMapData: await this.handler.sendWithPromise("FetchBinaryData", {
          kind: "cMapUrl",
          filename: `${name}${this.options.cMapPacked ? ".bcmap" : ""}`,
        }),
        isCompressed: this.options.cMapPacked,
      };
    }
    // Cache the CMap data, to avoid fetching it repeatedly.
    this.builtInCMapCache.set(name, data);
 
    return data;
  }

The key of builtInCMapCache is the CMap name and the value is an object holding the bytes plus a compressed flag, not an already parsed CMap. This means that when several pages reuse the same built-in data, the bytes are fetched only once, but each CMap is still parsed separately. This matches the cleanup policy of Catalog.builtInCMapCache: cleanup empties it and releases the raw bytes without keeping parse results.

Fault Tolerance, Boundaries and Performance

The fault-tolerance policy of cmap.js can be summarised in one table:

Three main lines run through performance:

1. Expand at parse time, index at query time. Both mapCidRange and mapBfRange turn ranges into item-by-item arrays, so lookup and contains are O(1). The larger the range, the higher the parse cost, but the same CMap is reused by a great many characters, so the overall trade still pays off.
2. Linear scan inside codespace buckets. The number of ranges in each bucket is usually small and the constant of a linear scan is very low; the source keeps no sorted array or binary-search structure for it, which reduces construction and update costs.
3. Zero arrays for the identity mapping. IdentityCMap turns a 65536-entry concrete array into a type check, saving both memory and query time; the factory also performs one recognition after parsing an embedded stream, so an ordinary object that merely happens to be an identity mapping does not stay in the cache.

Design note: The core trade in cmap.js is materialising the mapping relation. It does not preserve the range syntax of the PDF file; instead it expands ranges during parsing into an array indexed by character code. The content-stream parser then needs only one array access for every character it meets. The price is memory and parse time, which is why MAX_MAP_RANGE caps the worst case and IdentityCMap removes the most common waste. This trade matches the overall PDF.js strategy of "parse once, render many times" on the worker side.

Conclusion

cmap.js is PDF.js's character-encoding translation layer: downward it parses text CMaps and loads built-in .bcmap files, upward it provides CIDs to the font layer and Unicode to the text layer. It decides neither glyph outlines nor text positions; it only answers which target value a given character code corresponds to.

Where It Sits in the Overall Chain

In one sentence: the font dictionary decides which CMap to use, cmap.js translates character codes into CIDs or Unicode, and the font layer and the text layer each consume those target values.

Five Design Principles

1. One container serves two mappings: CID CMaps and ToUnicode CMaps share CMap because their index structure is the same, and the differences lie only in the target type and the downstream consumer. Leaving the differences to the call site keeps the container itself simple.
2. Expand at parse time, index at query time: ranges become item-by-item arrays during writing, so lookup is a single array access. A one-off parse cost is exchanged for a per-character query cost, which suits the case where one CMap is reused by a great many characters.
3. Express the identity mapping as a type: IdentityCMap builds no 65536-entry array but overrides the query methods; the factory then replaces an identity mapping from an embedded stream with this lightweight type through isIdentityCMap. The most common encoding does not pay the most common memory cost.
4. Inherit only gaps, never overwrite: extendCMap writes a parent entry only when the child has no such key, and codespaces are inherited only when the child defines none at all. The rule is simple, the priority is explicit, and parent and child mappings never pollute each other.
5. Degrade on bad records, rethrow on missing bytes: an oversized range, an invalid bf target or a malformed codespace only affects the current record; MissingDataException is rethrown unchanged so the layer above can fetch the data. Each kind of error is handled only at the level that can handle it.

At this point the CMap layer is complete: it translates "character codes" into "CIDs or Unicode". The next article moves into binary_cmap.js to see how a built-in .bcmap uses binary records, delta encoding and varints to make the same mapping smaller and faster to read.

StreamPDF design principle: anything that can be handled ahead of time is absolutely not deferred to parse time; the server solves all performance problems once. The browser only fetches and renders the data it must.