Transfering array of C structs to main JS thread with zero-copy via SharedArrayBuffer

[badgermole]

Hi all,

Executive Summary:

I am trying to accomplish a zero-copy transfer of an array of structured data from C++ up to the main JS thread.
Please see here for a diagram showing the basic runtime environment.

Main JS Thread          Web Worker containing WASM module instantiation        WASM/C++(pthreads/SharedArrayBuffer)
Use DataView    <<----- SharedArrayBuffer, Memory offset, Sizeof array  <<----- Array of nested structs w/ std::string
to simulate the        (Module.HEAPU8.buffer, JSRet.$$.ptr, JSRet.size())       Return std::vector<MyStruct>&
original structs

I would like to know if the method I'm attempting is plausible, or if there is a better way.
I know that Emscripten gives me a JS object representing my array of structs, and that it can correctly find the data for each element, via its JSRet.get(i) method. I'm hoping that I can do similarly using a DataView up in the main JS thread.

'Why':

1. I am using pthreads in the C++
2. In order to avoid blocking on the main thread, or at least assure I have the most control, I am instantiating the WASM instance in a Web Worker.
3. I want to take advantage of SharedArrayBuffer for transferring large amounts of data (I used to use flattbuffers, but that entails more copying).
4. The JS object returned from the exposed C++ API does not transfer successfully via postMessage from the Web Worker to the JS main thread

I'm using Emscripten version 3.1.58.

Details:

Given the following C++ source:

//-- BasicTests.h ----------------------------------------------------
//--------------------------------------------------------------------
typedef uint32_t MyUInt32;

struct Tag {
    MyUInt32 iT1, iT2;
};

struct MyStruct {
    MyUInt32 iFoo;
    Tag tag;
    std::string sName;
};

using MyStructVector = std::vector<MyStruct>;

//-----
class BasicTests
{
public:
    BasicTests();

    const MyStructVector &ReturnStructArrayByRef() const;

private:
    MyStructVector m_myStructVector;
};
//-- End BasicTests.h ------------------------------------------------

//-- BasicTests.cpp --------------------------------------------------
//--------------------------------------------------------------------
EMSCRIPTEN_BINDINGS(BasicTests)
{
	emscripten::class_<BasicTests>("BasicTests")
		.constructor<>()
		.function("ReturnStructArrayByRef", &BasicTests::ReturnStructArrayByRef)
		;
}

EMSCRIPTEN_BINDINGS(MyStruct)
{
	emscripten::value_object<Tag>("Tag")
		.field("iT1", &Tag::iT1)
		.field("iT2", &Tag::iT2);

	emscripten::value_object<MyStruct>("MyStruct")
		.field("iFoo", &MyStruct::iFoo)
		.field("tag", &MyStruct::tag)
		.field("sName", &MyStruct::sName);

	emscripten::register_vector<MyStruct>("MyStructVector");
}

//-----
BasicTests::BasicTests()
{
	printf("BasicTests::BasicTests() constructor\n");

	// Initialize the m_myStructVector with some data.
	for (int i = 0; i < 20; ++i)
	{
		MyStruct myStruct;
		myStruct.iFoo = i;
		myStruct.tag.iT1 = i * 10;
		myStruct.tag.iT2 = i * 20;
		myStruct.sName = "Instance " + std::to_string(i);
		m_myStructVector.push_back(myStruct);
	}
}

//-----
const MyStructVector &
BasicTests::ReturnStructArrayByRef() const
{
	return m_myStructVector;
}
//-- End BasicTests.cpp ----------------------------------------------

I have a WebWorker, BasicTests.js that instantiates the WASM and calls its APIs. The worker will subsequently postMessage(results) to pass the results of the WASM API to the main thread(main.mjs).

//-- BasicTests.mjs --------------------------------------------------
//--------------------------------------------------------------------
        . . .  // BasicTests class was instantiated above

    run: () => {
        const myStructVec = BasicTests.ReturnStructArrayByRef();

        . . .

Now, I can iterate via myStructVec.get(i) and see the elements and their values nicely.
My problem comes when I try
self.postMessage(myStructVec);
It seems the structured clone algorithm can't work with that object, and main.mjs receives an empty object, {}.
I know, however, that the WebAssembly.memory is a SharedArrayBuffer (SAB) and that I can tranfer that across without the cost of a copy. And I can see that the offset of myStructVec in the module instance HEAPU8 is given by:
const vecAddr = myStructVec.$$.ptr;
So my thought is to post vecAddr and the number of elements up to main.mjs, and there in the main thread, take advantage of the JS DataView to give me a proper MyStruct view on the SAB.

        self.postMessage ({heapu8:       testWasmInst.HEAPU8.buffer,
                           msvStartAddr: vecAddr,
                           size:         myStructVec.size() });
    }

I have in fact had some success doing just that. However, I don't understand the layout of m_myStructVector in WASM memory. As a result, I get elements 0-9 okay, but then the pattern in memory gets wonky as far as the strings are concerned:

{iFoo: 0, tag: {iT1: 0, iT2: 0}, sName: 'Instance 0'}
{iFoo: 1, tag: {iT1: 10, iT2: 20}, sName: 'Instance 1'}
{iFoo: 2, tag: {iT1: 20, iT2: 40}, sName: 'Instance 2'}
{iFoo: 3, tag: {iT1: 30, iT2: 60}, sName: 'Instance 3'}
{iFoo: 4, tag: {iT1: 40, iT2: 80}, sName: 'Instance 4'}
{iFoo: 5, tag: {iT1: 50, iT2: 100}, sName: 'Instance 5'}
{iFoo: 6, tag: {iT1: 60, iT2: 120}, sName: 'Instance 6'}
{iFoo: 7, tag: {iT1: 70, iT2: 140}, sName: 'Instance 7'}
{iFoo: 8, tag: {iT1: 80, iT2: 160}, sName: 'Instance 8'}
{iFoo: 9, tag: {iT1: 90, iT2: 180}, sName: 'Instance 9'}
{iFoo: 10, tag: {iT1: 100, iT2: 200}, sName: 'H¼\x01'}
{iFoo: 268435456, tag: {iT1: 192937984, iT2: 1845493760}, sName: ''}
{iFoo: 56320, tag: {…}, sName: ''}
{iFoo: 209715200, tag: {…}, sName: ''}

etc.

As far as I'm concerned this is a partial win as I've been able to show that I can transfer the SAB to main and look at it using a DataView.

Does this sound reasonable, and am I going about this correctly, just with misunderstanding the memory layout for structures such as this (especially ones with std::string)? Or is there a completely different 0-copy way I can go about this?
I figure the memory layout must be understandable since Emscripten implements a JS object.get(i) method that gives the correct values. Can I more-or-less copy that (where is that?)?

Thanks for comments or questions!

[sbc100]

I think you the right track here in terms of using SAB to trasfrer your data.

However, I would avoid using postMessage at all and instead start you application on the main thread. You will then already have access to the SAD on the main thread and you create a new pthread to do all your long running work.

When the long-running work is done you can signal the main thread and it can retrieve the data from memory via your embind binding. This way you can use embind rather than trying to hard code the data layout of your objects.

[badgermole]

Okay, so now I believe I understand that PROXY_TO_PTHREAD only runs the C++ main() in its own em-pthread worker, and it has no effect on the actual process of creating new em-pthreads when C++ pthread_create() calls are made.

And I think you're saying that the process of creating em-pthreads is not time intensive enough to worry about so that PTHREAD_POOL_SIZE is really not so important either.

[badgermole]

So the main concern is that calls made to the exposed WASM API don't block the main browser thread. For example, wasmModule.ourAPI.processData(data) - since the wasmModule lives on the main browser thread instead of in a worker - must not block, but should do any heavy processing via a pthread it creates.

[sbc100]

Yes, you would want to have processData do that actual work on a secondary thread, and perhaps return promise to JS on the main thread which resolves when the work is done.

[badgermole]

Thanks for answering my questions @sbc100 👍.

One other somewhat related question: this WASM module will have an intermittent lifespan. When I was hosting it in a worker, when appropriate, I would just terminate the worker, which would also clean up all em-pthreads. Now however, since I'm hosting it on the main browser thread, I can't just set wasmModule = null. Doing that allows the garbage collector to reclaim resources, but it doesn't clean up the em-pthreads. I think that the C++ should be responsible for cleaning up the threads it creates. Only, if I'm using PTHREAD_POOL_SIZE, those threads in the pool I think need to be cleaned up via some call into wasmModule? Or otherwise, how can I clean up pthreads if there is some catastrophic exception?

[sbc100]

When the module will clean up all its threads on exit. You should build with -sEXIT_RUNTIME and then just call the C exit function.

See

emscripten/src/preamble.js

Line 270 in b0ec4e1

PThread.terminateAllThreads();