StreamPDFStreamPDF
返回博客

PDF.js 深入浅出 4: MessageHandler

2026年10月22日 · StreamPDF Team

PDF.js 深入浅出 4: MessageHandler

前言

在本系列的前三篇文章中, 我们已经了解了 PDF.js 的渲染架构, 以及消息通讯的基本原理. 那么在本篇文章之中, 我们将深入探索消息通讯的细节, 并理解 PDF.js 如何在主线程与 Worker 中进行消息通讯.

本系列分析基于 PDF.js v6.3.289 源码.

MessageHandler

无论是 main, 还是 Worker, 都通过类 MessageHandler 来进行消息通讯.

线程级通讯

// Source: display/api.js PDFWorker.#initialize
const worker = new Worker(workerSrc, { type: 'module' });
const messageHandler = new MessageHandler('main', 'worker', worker);
// Source: core/worker.js WorkMessageHandler.initializeFromPort
static initializeFromPort(port) {
  const handler = new MessageHandler("worker", "main", port);
  this.setup(handler, port);
  handler.send("ready", null);
}

文档级通讯

// Source: display/api.js getDocument
const messageHandler = new MessageHandler(docId, workerId, worker.port);
const transport = new WorkerTransport(
  messageHandler,
  task,
  networkStream,
  transportParams,
  transportFactory,
  pagesMapper,
);
// Source: core/worker.js WorkMessageHandler.createDocumentHandler
const workerHandlerName = docId + '_worker';
let handler = new MessageHandler(workerHandlerName, docId, port);

MessageHandler 提供了如下方法:

  • constructor
  • #onMessage
  • on
  • send
  • sendWithPromise
  • sendWithStream
  • #createStreamSink
  • #processStreamMessage
  • #deleteStreamController

让我们一一详解.

constructor

// Source: shared/message_handler.js MessageHandler
constructor(sourceName, targetName, comObj) {
  this.sourceName = sourceName;
  this.targetName = targetName;
  this.comObj = comObj;
  this.callbackId = 1;
  this.streamId = 1;
  this.streamSinks = Object.create(null);
  this.streamControllers = Object.create(null);
  this.callbackCapabilities = Object.create(null);
  this.actionHandler = Object.create(null);
 
  comObj.addEventListener("message", this.#onMessage.bind(this), {
    signal: this.#messageAC.signal,
  });
}

接受 sourceName, targetName, comObj 作为参数.

  • sourceName 表示发送消息的名称.
  • targetName 表示接收消息的名称.
  • comObj 表示通讯对象. 这里一般是一个 Worker, 通过 message 和 postMessage 来进行消息通讯.

通过 addEventListener 监听 message 事件, 并在处理函数中委托给 #onMessage 方法.

这里还传入了第三个参数 signal, 在 MessageHandler 实例被销毁的时候, 取消监听 message 事件.

  • actionHandler

用于处理基础事件的回调.

  • callbackId
  • callbackCapabilities

用于 sendWithPromise 方法.

  • streamId
  • streamSinks
  • streamControllers

用于 sendWithStream 方法, 处理流式数据, 也是该类中最复杂的部分.

#onMessage

#onMessage({ data }) {
  if (data.targetName !== this.sourceName) {
    return;
  }
  if (data.stream) {
    this.#processStreamMessage(data);
    return;
  }
  if (data.callback) {
    const callbackId = data.callbackId;
    const capability = this.callbackCapabilities[callbackId];
    if (!capability) {
      throw new Error(`Cannot resolve callback ${callbackId}`);
    }
    delete this.callbackCapabilities[callbackId];
 
    if (data.callback === CallbackKind.DATA) {
      capability.resolve(data.data);
    } else if (data.callback === CallbackKind.ERROR) {
      capability.reject(wrapReason(data.reason));
    } else {
      throw new Error("Unexpected callback case");
    }
    return;
  }
  const action = this.actionHandler[data.action];
  if (!action) {
    throw new Error(`Unknown action from worker: ${data.action}`);
  }
  if (data.callbackId) {
    const sourceName = this.sourceName,
      targetName = data.sourceName,
      comObj = this.comObj;
 
    Promise.try(action, data.data).then(
      function (result) {
        comObj.postMessage({
          sourceName,
          targetName,
          callback: CallbackKind.DATA,
          callbackId: data.callbackId,
          data: result,
        });
      },
      function (reason) {
        comObj.postMessage({
          sourceName,
          targetName,
          callback: CallbackKind.ERROR,
          callbackId: data.callbackId,
          reason: wrapReason(reason),
        });
      }
    );
    return;
  }
  if (data.streamId) {
    this.#createStreamSink(data);
    return;
  }
  action(data.data);
}

首先做防御检查, 如果消息不是针对当前 MessageHandler 实例的 则直接返回.

否则根据消息的类型, 进行不同的处理.

如果 data.stream 为 true, 则调用 #processStreamMessage 方法. 这里对应着下方的 sendWithStream 中的 pull 和 cancel 方法.

如果 data.callback 为 true, 则为 sendWithPromise 的异步结束回调.

如果 data.callbackId 为 true, 则为 sendWithPromise 的异步启动函数.

如果 data.streamId 为 true, 则为 sendWithStream 的启动函数, 调用 #createStreamSink 方法创建一个流接收器.

on

on(actionName, handler) {
  if (typeof PDFJSDev === "undefined" || PDFJSDev.test("TESTING")) {
    assert(
      typeof handler === "function",
      'MessageHandler.on: Expected "handler" to be a function.'
    );
  }
  const ah = this.actionHandler;
  if (ah[actionName]) {
    throw new Error(`There is already an actionName called "${actionName}"`);
  }
  ah[actionName] = handler;
}

接受 actionName, handler 作为参数.

  • actionName 表示要监听的事件名称.
  • handler 表示事件处理函数.

send

send(actionName, data, transfers) {
  this.comObj.postMessage(
    {
      sourceName: this.sourceName,
      targetName: this.targetName,
      action: actionName,
      data,
    },
    transfers
  );
}

接受 actionName, data, transfers 作为参数.

  • actionName 表示要发送的事件名称.
  • data 表示要发送的数据.
  • transfers 表示要转移的 ArrayBuffer 或 ArrayBufferView 对象.

Q: transfers 是什么?

A: 它就是 postMessage 的第二个参数, 一组 Transferable Objects 的列表 (ArrayBuffer, TypedArray, MessagePort, ImageBitmap, OffscreenCanvas 等). 它的含义不是「传哪些数据」, 而是「这些对象的所有权移交给对端」.

不传 transfers 时, postMessage 走结构化克隆 (Structured Clone), 接收端会拿到一份深拷贝, 开销与数据量成正比; 传了 transfers 时, 数据本身不拷贝, 只是把所有权交出去, 发送端对应的 ArrayBuffer 会立即被 detach (byteLength 变成 0, 再访问就抛错). 所以它只适用于「发出去之后发送方不再需要」的数据.

PDF.js 里恰好有大量这样的数据: Worker 解析出的图片像素 (imgData.data.buffer / imgData.bitmap), 路径数据与包围盒 (data.buffer, minMax.buffer), 字体导出数据 (fontData.buffer), 以及 operatorList 分块 flush 时批量收集的 _transfers. 例如在 core/evaluator.js 中:

const transfers = imgData ? [imgData.bitmap || imgData.data.buffer] : null;

this.handler.send("commonobj", [objId, "Image", imgData], transfers);

注意 data 中引用的仍然是同一个 ArrayBuffer, 接收端拿到后直接解引用即可 —— 零拷贝正是这样实现的.

此外, 传入 transfers 还兼作环境探测: 若运行环境不支持转移, postMessage 会抛 DataCloneError. PDF.js 在 PDFWorker.#initialize 中就用一条空消息提前试探 —— messageHandler.send("test", testObj, [testObj.buffer]), 一旦抛错就回落到 fake worker.

send 方法, 没有 stream, callback, callbackId, streamId. 走向 onMessage 基础的路径.

// foo
const messageHandlerFoo = new MessageHandler('foo', 'bar', comObj);
 
// bar
const messageHandlerBar = new MessageHandler('bar', 'foo', comObj);
messageHandlerBar.on('test', data => {
  console.log(data);
});
 
messageHandlerFoo.send('test', { a: 1 });
 
// result: log { a: 1 }

基础的事件监听+回调模式. 就好像 dom 的 addEventListener 一样.

send 基础消息路径图

sendWithPromise

sendWithPromise(actionName, data, transfers) {
  const callbackId = this.callbackId++;
  const capability = Promise.withResolvers();
  this.callbackCapabilities[callbackId] = capability;
  try {
    this.comObj.postMessage(
      {
        sourceName: this.sourceName,
        targetName: this.targetName,
        action: actionName,
        callbackId,
        data,
      },
      transfers
    );
  } catch (ex) {
    capability.reject(ex);
  }
  return capability.promise;
}

增强型 send 方法, 支持通过 Promise 的方式进行调用.

调用 sendWithPromise 方法, 携带 callbackId, 用于异步处理.

在 onMessage 方法中, 携带 callbackId, 将通过 Promise.try 调用 action, 然后将异步结果返回, 同时携带上 callback, callbackId.

这时候, 调用方的 onMessage 方法收到之后, 走向 data.callback 分支, resolve/reject Promise. 最终拿到数据.

sendWithPromise 一问一答时序图

sendWithStream

sendWithStream(actionName, data, queueingStrategy, transfers) {
  const streamId = this.streamId++,
    sourceName = this.sourceName,
    targetName = this.targetName,
    comObj = this.comObj;
 
  return new ReadableStream(
    {
      start: controller => {
        const startCapability = Promise.withResolvers();
        this.streamControllers[streamId] = {
          controller,
          startCall: startCapability,
          pullCall: null,
          cancelCall: null,
          isClosed: false,
        };
        comObj.postMessage(
          {
            sourceName,
            targetName,
            action: actionName,
            streamId,
            data,
            desiredSize: controller.desiredSize,
          },
          transfers
        );
        // Return Promise for Async process, to signal success/failure.
        return startCapability.promise;
      },
 
      pull: controller => {
        const pullCapability = Promise.withResolvers();
        this.streamControllers[streamId].pullCall = pullCapability;
        comObj.postMessage({
          sourceName,
          targetName,
          stream: StreamKind.PULL,
          streamId,
          desiredSize: controller.desiredSize,
        });
        // Returning Promise will not call "pull"
        // again until current pull is resolved.
        return pullCapability.promise;
      },
 
      cancel: reason => {
        assert(reason instanceof Error, "cancel must have a valid reason");
        const cancelCapability = Promise.withResolvers();
        this.streamControllers[streamId].cancelCall = cancelCapability;
        this.streamControllers[streamId].isClosed = true;
        comObj.postMessage({
          sourceName,
          targetName,
          stream: StreamKind.CANCEL,
          streamId,
          reason: wrapReason(reason),
        });
        // Return Promise to signal success or failure.
        return cancelCapability.promise;
      },
    },
    queueingStrategy
  );
}

在探索 sendWithStream 方法之前, 我们先了解一下 ReadableStream 这个对象.

ReadableStream

流会将网络资源分割成小分块,按位处理 —— 这正是浏览器渐进渲染视频/图片的方式。 From MDN

传统的一次性下载有两个缺点:

  • 内存占用高 — 全量读入 G 级二进制,容易 OOM
  • 响应慢 — 100S 视频要等下载完才开始播放

备注:绝大多数非线性化 PDF(xref 在文件末尾)Chrome PDFium 也全量下载;仅线性化 PDF 支持渐进渲染。

全部下载 vs 流式渲染对比图

ReadableStream 接受一个构建器对象,三个回调:

回调触发时机职责
start(controller)new 时立即初始化资源,不生产数据
pull(controller)消费者 read() 且队列为空时按需生产 chunk 或 close
cancel(reason)流被取消时释放资源

三种使用模式:

模式生产者角色生产时机何时调用 pull适用场景
start-only主动 pushsetInterval / onmessage 硬塞永不(队列一直有货)WebSocket / SSE 推送
pull-only被动 pull消费者驱动每次 read() 队列为空即时可计算数据、TransformStream
start + pull预热 + 按需start 初始化,pull 生产每次 read() 队列为空Fetch、DB 分页、pipe 流水线

关键点:enqueue() 永不阻塞,队列长度(desiredSize 的相反数)只是给生产者一个建议信号。pull 只在队列空 + 消费者 read 时触发,是天然的背压。默认 highWaterMark = 1,desiredSize = highWaterMark - queue.length。

start + pull 最小示例(PDF.js sendWithStream 用的就是这一档)
// start 发 START 建通道,pull 按需拉 —— 双模式配合
let pullCount = 0;
const stream = new ReadableStream({
  start(controller) {
    this.resource = { ready: true }; // 建"远程连接"
    log('start', '通道就绪');
  },
  pull(controller) {
    pullCount++;
    controller.enqueue(new Uint8Array([pullCount]));
    if (pullCount >= 5) controller.close();
  },
  cancel() {
    pullCount = 0;
  },
});
 
// start 先跑完(打印"通道就绪"),然后每次 read 队列为空才触发 pull
await stream.getReader().read();

💡 在浏览器中依次体验三种模式: start-only demo · pull-only demo · start + pull demo

sendWithStream 在 Worker 侧的配套实现

start 方法将流的 controller 存下来, 并且通过 postMessage 通知生产者初始化流. 然后 return Promise, 等待生产者初始化完成.

当生产者收到消息之后, 通过 onMessage 的分支走向 this.#createStreamSink(data); 方法.

#createStreamSink(data) {
  const streamId = data.streamId,
    sourceName = this.sourceName,
    targetName = data.sourceName,
    comObj = this.comObj;
  const self = this,
    action = this.actionHandler[data.action];
 
  const streamSink = {
    enqueue(chunk, size = 1, transfers) {
      if (this.isCancelled) {
        return;
      }
      const lastDesiredSize = this.desiredSize;
      this.desiredSize -= size;
      // Enqueue decreases the desiredSize property of sink,
      // so when it changes from positive to negative,
      // set ready as unresolved promise.
      if (lastDesiredSize > 0 && this.desiredSize <= 0) {
        this.sinkCapability = Promise.withResolvers();
        this.ready = this.sinkCapability.promise;
      }
      comObj.postMessage(
        {
          sourceName,
          targetName,
          stream: StreamKind.ENQUEUE,
          streamId,
          chunk,
        },
        transfers
      );
    },
 
    close() {
      if (this.isCancelled) {
        return;
      }
      this.isCancelled = true;
      comObj.postMessage({
        sourceName,
        targetName,
        stream: StreamKind.CLOSE,
        streamId,
      });
      delete self.streamSinks[streamId];
    },
 
    error(reason) {
      assert(reason instanceof Error, "error must have a valid reason");
      if (this.isCancelled) {
        return;
      }
      this.isCancelled = true;
      comObj.postMessage({
        sourceName,
        targetName,
        stream: StreamKind.ERROR,
        streamId,
        reason: wrapReason(reason),
      });
    },
 
    sinkCapability: Promise.withResolvers(),
    onPull: null,
    onCancel: null,
    isCancelled: false,
    desiredSize: data.desiredSize,
    ready: null,
  };
 
  streamSink.sinkCapability.resolve();
  streamSink.ready = streamSink.sinkCapability.promise;
  this.streamSinks[streamId] = streamSink;
 
  Promise.try(action, data.data, streamSink).then(
    function () {
      comObj.postMessage({
        sourceName,
        targetName,
        stream: StreamKind.START_COMPLETE,
        streamId,
        success: true,
      });
    },
    function (reason) {
      comObj.postMessage({
        sourceName,
        targetName,
        stream: StreamKind.START_COMPLETE,
        streamId,
        reason: wrapReason(reason),
      });
    }
  );
}

调用 actionHandler 中的方法, 并传递 data.data 和 streamSink. 在方法执行完毕之后, 发送 postMessage 通知消费者, 生产者已初始化完成.

以 GetOperationList 为例, 说明这个过程.

进入 Main: PDFPageProxy._pumpOperatorList

const readableStream = this._transport.messageHandler.sendWithStream(
  'GetOperatorList',
  {
    pageId: this.#pagesMapper.getPageId(this._pageIndex + 1) - 1,
    pageIndex: this._pageIndex,
    intent: renderingIntent,
    cacheKey,
    annotationStorage: map,
    modifiedIds,
  },
  /* queueingStrategy = */ undefined,
  transfer,
);
const reader = readableStream.getReader();
 
const intentState = this._intentStates.get(cacheKey);
intentState.streamReader = reader;
 
const pump = () => {
  reader.read().then(
    ({ value, done }) => {
      if (done) {
        intentState.streamReader = null;
        return;
      }
      if (this._transport.destroyed) {
        return; // Ignore any pending requests if the worker was terminated.
      }
      this._renderPageChunk(value, intentState);
      pump();
    },
    reason => {
      intentState.streamReader = null;
 
      if (this._transport.destroyed) {
        return; // Ignore any pending requests if the worker was terminated.
      }
      if (intentState.operatorList) {
        // Mark operator list as complete.
        intentState.operatorList.lastChunk = true;
 
        for (const internalRenderTask of intentState.renderTasks) {
          internalRenderTask.operatorListChanged();
        }
        this.#tryCleanup();
      }
 
      if (intentState.displayReadyCapability) {
        intentState.displayReadyCapability.reject(reason);
      } else if (intentState.opListReadCapability) {
        intentState.opListReadCapability.reject(reason);
      } else {
        throw reason;
      }
    },
  );
};
pump();

sendWithStream('GetOperatorList') 调用通过 messageHandler 走向 Worker 中

进入 Worker: WorkMessageHandler.createDocumentHandler

handler.on(
  'GetOperatorList',
  function ({ pageId, pageIndex, intent, cacheKey, annotationStorage, modifiedIds }, sink) {
    pdfManager.getPage(pageId).then(function (page) {
      const task = new WorkerTask(`GetOperatorList: page ${pageIndex}`);
      startWorkerTask(task);
 
      // NOTE: Keep this condition in sync with the `info` helper function.
      const start = verbosity >= VerbosityLevel.INFOS ? Date.now() : 0;
 
      // Pre compile the pdf page and fetch the fonts/images.
      page
        .getOperatorList({
          handler,
          sink,
          task,
          intent,
          cacheKey,
          annotationStorage,
          modifiedIds,
          pageIndex,
        })
        .then(
          opListInfo => {
            if (start) {
              info(`${task.name}; time=${Date.now() - start}ms, len=${opListInfo.length}`);
            }
            sink.close();
          },
          reason => {
            if (task.terminated) {
              return; // ignoring errors from the terminated thread
            }
            sink.error(reason);
          },
        )
        .finally(() => {
          finishWorkerTask(task);
        });
    });
  },
);

注意这里的回调函数, 没有返回任何值. 因此在 #createStreamSink 方法中, 直接走向了 .then 分支.

Promise.try(action, data.data, streamSink).then(
  function () {
    comObj.postMessage({
      sourceName,
      targetName,
      stream: StreamKind.START_COMPLETE,
      streamId,
      success: true,
    });
  },
  function (reason) {
    comObj.postMessage({
      sourceName,
      targetName,
      stream: StreamKind.START_COMPLETE,
      streamId,
      reason: wrapReason(reason),
    });
  },
);

这里回应 stream: StreamKind.START_COMPLETE, 因此消费者可以知道生产者已初始化完成.

回到 Main: messageHandler.#processStreamMessage

#processStreamMessage(data) {
  const streamId = data.streamId,
    sourceName = this.sourceName,
    targetName = data.sourceName,
    comObj = this.comObj;
  const streamController = this.streamControllers[streamId],
    streamSink = this.streamSinks[streamId];
 
  switch (data.stream) {
    case StreamKind.START_COMPLETE:
      if (data.success) {
        streamController.startCall.resolve();
      } else {
        streamController.startCall.reject(wrapReason(data.reason));
      }
      break;
 
    case StreamKind.PULL_COMPLETE:
      if (data.success) {
        streamController.pullCall.resolve();
      } else {
        streamController.pullCall.reject(wrapReason(data.reason));
      }
      break;
 
    case StreamKind.PULL:
      // 主线程 pull 请求 — 交给 Worker 的 onPull 处理
      if (!streamSink) {
        comObj.postMessage({ stream: StreamKind.PULL_COMPLETE, success: true });
        break;
      }
      // desiredSize 从负变正 → resolve sink.ready,解除 Worker 侧背压
      if (streamSink.desiredSize <= 0 && data.desiredSize > 0) {
        streamSink.sinkCapability.resolve();
      }
      streamSink.desiredSize = data.desiredSize;
      Promise.try(streamSink.onPull).then(
        () => comObj.postMessage({ stream: StreamKind.PULL_COMPLETE, success: true }),
        reason => comObj.postMessage({ stream: StreamKind.PULL_COMPLETE, reason })
      );
      break;
 
    case StreamKind.ENQUEUE:
      // Worker 生产了数据 — 直接塞进 ReadableStream 内部队列
      if (streamController.isClosed) break;
      streamController.controller.enqueue(data.chunk);
      break;
 
    case StreamKind.CLOSE:
      streamController.isClosed = true;
      streamController.controller.close();
      this.#deleteStreamController(streamController, streamId);
      break;
 
    case StreamKind.ERROR:
      streamController.controller.error(wrapReason(data.reason));
      this.#deleteStreamController(streamController, streamId);
      break;
 
    default:
      throw new Error("Unexpected stream case");
  }
}

在这里, START_COMPLETE 让 start() 返回的 Promise resolve.

之后 pump() 的 reader.read() 被调用, 队列为空时触发 pull 回调, 主线程发 PULL 消息 → Worker 处理 PULL 分支 → 生产好数据通过 ENQUEUE 回传 → Main 塞进 ReadableStream 队列 → reader.read() resolve, _renderPageChunk 消费, 然后递归调 pump().

整条链路是:

reader.read() → pull() → postMessage(PULL)
    ↓
Worker #processStreamMessage PULL → sink.onPull
    ↓
Worker 解析 PDF → sink.enqueue(chunk) → postMessage(ENQUEUE)
    ↓
Main #processStreamMessage ENQUEUE → controller.enqueue
    ↓
reader.read() resolve → _renderPageChunk → pump()
    ↓ 循环直到 Worker 发 CLOSE

同时 Worker 端 sink.enqueue 时会检查 desiredSize: 如果从正变非正, 创建一个未 resolve 的 sink.ready = Promise. Worker 的解析循环里 await sink.ready 就会暂停产生新数据 —— 这就是背压在 Worker 端的实现. 主线程消费快了, 队列空了, 下次 PULL 带回更大的 desiredSize, Worker 收到后 resolve ready, 继续解析. 两边节奏完全对齐, 内存占用恒定.

sendWithStream 完整时序图

总结

PDF.js 的 MessageHandler 提供了三级通讯抽象,对应三种需求层级:

方法模式适用场景消息特征
sendFire-and-forget通知类消息(ready、close、commonobj)单向、无回调
sendWithPromiseRequest-response一次性异步查询(getPage、getMetadata)带 callbackId,一问一答
sendWithStreamStreaming + 背压连续大量数据传输(operatorList、图片、字体)带 streamId,多轮 PULL/ENQUEUE

这三级的选择很有意思:

  • sendWithPromise 是 send 的增强版 — 多了 callbackId,把 Promise resolve/reject 和消息配对
  • sendWithStream 是 send + pull 的组合 — Main 侧返回一个真实的 ReadableStream,Worker 侧拿到一个 streamSink 对象。两边的接口差异很大,但通过 postMessage 上的 stream 字段(PULL / ENQUEUE / CLOSE / ERROR / PULL_COMPLETE / START_COMPLETE / CANCEL / CANCEL_COMPLETE)统一协调

sendWithStream 是本文的主角,也是 PDF.js 渲染流水线能做到「边解析边渲染、内存占用恒定」的根本原因。它的实现是一个教科书式的 ReadableStream 用例:

  • start 里发 START,Worker 侧 #createStreamSink 立即回 START_COMPLETE —— 让主线程的 start Promise resolve
  • pull 里发 PULL,Worker 侧 streamSink.onPull 生产数据,通过 ENQUEUE 回传 —— 这是天然的按需生产,没有主线程 read 就没有数据
  • Worker 侧自己实现背压:sink.enqueue 时扣减 desiredSize,水位从正变负时挂起 sink.ready,Worker 解析循环 await ready 暂停;下次 PULL 带回更大 desiredSize 时 resolve ready,恢复解析

这套背压的精妙之处在于:Worker 完全不阻塞 JS 线程。它不是忙等 while (desiredSize <= 0),而是用一个 Promise 挂起 async 函数。主线程和 Worker 两边都同时处理多个消息,背压只是协作式地让 Worker "歇歇"而已。

而这一切的核心契约,不过是 desiredSize 这个数字在两条 postMessage 里来回传。一个 Promise 的 resolve/reject,就把"消费者想要多少数据"这个信号,从主线程一路传导回 Worker 的解析循环。

相关文章