Repository navigation
Worker threads and C++ backed objects #2754
Description
- Node.js Version: 12.16.3
- OS: Linux
- Scope: runtime
- Module (and version): worker_threads
Hi, I've been recently playing with worker threads as in version 12.16.3.
The doc explicitly says that you can't pass a C++ backed object via postMessage: https://nodejs.org/dist/latest-v12.x/docs/api/worker_threads.html#worker_threads_port_postmessage_value_transferlist
And this is true, you can't :) The standard makes it pretty obvious.
Any chance someone may shed some light on why is it so? Or even better, maybe someone has already found a workaround?
Activity
addaleax commentedon Jun 4, 2020
postMessage()needs to be able to serialize the value that it is being passed, but if it contains a C++-backed object, there is no generic way to know how to serialize those.We do have a special handler for
MessagePorts in place there, and are expanding that functionality in nodejs/node#33360. After that, it would not be unreasonable to also provide a public C++ API that addon authors could use to make their own objects transferable/cloneable.Flackus commentedon Jun 4, 2020
@addaleax thanks for a quick response. I have a follow-up question if it's ok.
At the moment is it reasonable to play around custom v8 serialiser for C++ objects or it wouldn't help with transfer issues?
Even if we can make an object transferable (now or in future), it still would be a pack/unpack routine, right? So we wouldn't be able to pass an object without copying?
addaleax commentedon Jun 4, 2020
@Flackus Not sure what a custom V8 serializer would look like – of course you can write your own serializer/deserializer functions (e.g. based on https://www.npmjs.com/package/serdes) and serialize/deserialize the C++ objects yourself.
For objects that Node.js transfers internally, like the
MessagePortobjects, it does transfer them without extra serialization and deserialization, i.e. the C++ object identity remains the same.Flackus commentedon Jun 4, 2020
@addaleax thanks, things are getting clearer.
Me, too, actually :) I'm just considering every option here.
Also, you said:
Does that imply that there would be a way to transfer custom objects without extra serialization and deserialization? It would be a powerful tool IMO.
addaleax commentedon Jun 4, 2020
Yes, exactly. For example, when
MessagePorts are currently being transferred, no serialization or deserialization is performed on their internals.Flackus commentedon Jun 4, 2020
@addaleax thank you, now it's clear. Looking forward to it to happen :)
I think I'll just close the issue now as I don't have further questions. Thanks again for you help.
libingyu96 commentedon Jan 2, 2024
Hi @addaleax
Does Node.js now provide such a public C++ API ?