WebAssembly (Wasm) audits in Lighthouse

[tomayac]

There are a number of things which apps that include WebAssembly (Wasm) code can get wrong, and that Lighthouse could warn about:

• When Lighthouse detects a targeted-at-all-environments Wasm file is executed in the browser, print a hint that points at the -s ENVIRONMENT option as an optimization opportunity. JavaScript glue code size savings for a "hello world" example: ~5% smaller (-s ENVIRONMENT=web vs. regular [diff]).

• When Lighthouse detects an unoptimized Wasm file is executed in the browser, print a hint that points at the optimization options. Wasm code size savings for a "hello world" example: ~83% smaller (-O3 vs. -O0).

• When Lighthouse detects a Wasm file is delivered to production with the -g debug option enabled, show a warning message. Wasm code size savings for a "hello world" example: ~95% smaller (regular vs. -g).

Now these savings of course need to be taken with a huge grain of salt and will vary in actually useful code. Find the source code in the accompanying testing repo. It's napkin math that doesn't fully demonstrate the savings potential yet, but that could help start a discussion.

[connorjclark]

Did we ever find a reasonable approach for attaching the build options used (optimization level, env, etc), or is the idea here to infer the build options by analyzing the end result?

[tomayac]

The idea would be to find this out as part of the discussion… Some (internal to Google, sorry) commenting happened in this doc.

[connorjclark]

The most relevant comment thread: here

[tomayac]

FYI: @dschuff, @kripken.

[tomayac]

For detecting unoptimized Wasm, @connorjclark and @kripken suggest to run wasm-opt over the Wasm file and to compare the results with what was shipped. This was also the approach taken by the HTTP Almanac, though with a little unclear outcomes.

[tomayac]

For detecting Wasm with debug information, you can run wasm-opt --strip-debug and compare to the original file size.

[tomayac]

For detecting Wasm that has not been optimized to run exclusively on the Web, you can check for the non-existence of the following string (it's there if the library was optimized to run exclusively on the Web):

throw new Error('not compiled for this environment (did you build to HTML and try to run it not on the web, or set ENVIRONMENT to something - like node - and run it someplace else - like on the web?)')

Not sure how robust this would be, though…

[kripken]

Another option here might be to look for any Node.js support code, which is not emitted when -sENVIRONMENT=web,worker etc. Some examples in the code, that appear even in minified fully with closure:

require("fs")
require("path")
.readFileSync
.readFile
process.argv
process.on
process.exitCode

[connorjclark]

Looking for code like this is too much of a moving target. It's good for existing stuff to know how to detect, but for things moving forward we need a better indicator. Can the JS output from emscripten include some global meta object with any relative build details? What can we include by default that 1) does not have privacy or security implications and 2) is not unreasonably large?

[kripken]

Yeah, I get that a moving target may not be worth handling, definitely a fair point.

We have discussed in the past including build info in Emscripten, but the decision has been negative (at least by default), for reasons including the following:

• Code size, since even a small amount of extra code is a negative for many of our users. Adding more code size by default forces them to set a flag and be aware they need to set it; they really dislike us adding such flags over time.
• Awkward incentives. We as toolchain makers would love to know how much our toolchain is used, but making our users ship larger builds for our benefit feels wrong. (In the context here, the user would benefit, so this is not entirely applicable.)
• Lack of precedent in the JS space. (Has this changed?)
• Poor precedent in the native space. (There actually is a precedent, but at least my investigation could not actually find a good historical reason for it, and plenty of online threads of people surprised and annoyed by it.)

[connorjclark]

For the very specific "was this built with NODE support", can we just look for ENVIRONMENT_IS_NODE variable being present?

More generally ...

1. Code size, since even a small amount of extra code is a negative for many of our users. Adding more code size by default forces them to set a flag and be aware they need to set it; they really dislike us adding such flags over time.

This is fair. I do think we can get a lot from a few bytes of JSON in the JS payload (<100?).

But maybe I can "purchase" some bytes by pointing out a possible size reduction :))

I opened up an emscripten js (this one, it's using a few months old toolchain) and what jumped out instantly was many invoke_vii* of various lengths

Maybe this compresses well, I don't know. But perhaps some of this could be generated at runtime like:

// from
"nc": invoke_viiiiiiiiiiiiiiiiiiiiiiiiiiiiii,
// to
"nc": makeInvoke(30, 0) // or whatever

function makeInvoke(i, j) {
const dyncall = globalThis[`dynCall_${somehow generate those letters representing call signature idk}`]
return () => { ... }
}

Just considering the invoke_* methods, that's something like 16KB in this bundle.

2. Awkward incentives. We as toolchain makers would love to know how much our toolchain is used, but making our users ship larger builds for our benefit feels wrong. (In the context here, the user would benefit, so this is not entirely applicable.)

3. Lack of precedent in the JS space. (Has this changed?)

No - but that's largely because extended information is in an external resource (.js.map). Maybe we can put emscripten's meta information into a json file next to the source but with a .json extension? Then developer tools could know to look there if a emscripten WASM is detected on the page.

[kripken]

or the very specific "was this built with NODE support", can we just look for ENVIRONMENT_IS_NODE variable being present?

Unfortunately, no, as closure compiler will minify that name.

About those invokes, yeah, that is wasteful! This is actually due to either exceptions or longjmp, and both can be avoided in modern browsers using the new wasm exceptions spec, so optimally that codebase would be rebuilt with it. (Wasm exceptions are fairly recent, so not everyone has heard of them or is ok with depending on it.) That's one reason we haven't heavily optimized that JS, but you are definitely right that we could in principle.

No - but that's largely because extended information is in an external resource (.js.map). Maybe we can put emscripten's meta information into a json file next to the source but with a .json extension? Then developer tools could know to look there if a emscripten WASM is detected on the page.

I think an option to emit that file could be ok. But we wouldn't want it by default for the usual reasons. Perhaps it could be something like "build a release build + this flag locally, and then run devtools to get suggestions" - something like that?

[tomayac]

• Lack of precedent in the JS space. (Has this changed?)

Actually a lot of JS libraries leave traces that tools like Wappalyzer use to detect the presence of said libraries, including version numbers and sometimes more. For example, this is the tool's output for https://emscripten.org/:

[kripken]

Interesting, thanks. Might be different for Emscripten, then, as we haven't needed it for that type of bug report situation.

Overall the difference might be that Emscripten is a compiler that happens to ship some library code (but mostly ships compiled user code) as opposed to a library itself. In the past when we looked for precedent in the Web space we compared Emscripten to other compilers and minifiers (like closure compiler, Uglify, Terser, etc.).

Overall I'm open-minded here, but making a change in Emscripten to increase code size for this reason would not be a simple process. If we think it's really worth trying I'm not opposed to that discussion, though.

[tomayac]

Given that Wappalyzer already detects Wasm, I think it’s not out of this world to mark the Emscripten glue code with meta information like the used Emscripten version, since there’s clearly interest outside of the Lighthouse issue we’re discussing here. There may be a clever way to encode important other information in a bit vector or so.

[kripken]

Sorry, I'm not familiar with Wappalyzer and I'm not sure what I should infer from that link (I just see a little JSON that for some reason mentions cats: 27 🐱 😄 ).

I think if you can find precedent in the minifier/compiler space that would be much more compelling for me personally. But, regardless, it's not going to be up to myself specifically, we'd need to see what the community thinks. We can either continue to discuss here, or we could open an Emscripten issue for that.

[tomayac]

FWIW, the fields are documented. The idea of Wappalyzer is essentially what we're discussing here: analyzing the source code or network traces for telling patterns that let the tool find out if a certain technology is used (here: the header is "Content-Type": "application/wasm" and it has wasmBinary or wasmBinaryFile in its JavaScript). So I think before we start changing the way Emscripten works, we should continue exploring which of the "telling traces" approaches could work and look to remain stable in the future.

[kripken]

I see, thanks. Checking the content-type looks like the only robust thing there, as wasmBinary[File] will not exist in a fully minified JS file, and wasm_exec.js is a very specific thing to the main Go wasm toolchain. But the content-type should work for the great majority of cases (except for pages that JIT wasm at runtime, or download and decompress it in JS for some reason).