Support GHC JavaScript backend #738

hamishmack · 2024-04-17T23:55:01Z

js-sources from the miso.cabal file are not linked into all.js.

One work around for this is to cp jsbits/* to the .jsexe directory and edit the index.html file and add:

  <script language="javascript" src="delegate.js" defer></script>
  <script language="javascript" src="diff.js" defer></script>
  <script language="javascript" src="isomorphic.js" defer></script>
  <script language="javascript" src="util.js" defer></script>

To build one of the examples with the included haskell.nix flake:

nix develop .#
javascript-unknown-ghcjs-cabal build todo-mvc
cp jsbits/* dist-newstyle/build/javascript-ghcjs/ghc-9.8.2/miso-examples-1.8.3.0/x/todo-mvc/build/todo-mvc/todo-mvc.jsexe/
vim dist-newstyle/build/javascript-ghcjs/ghc-9.8.2/miso-examples-1.8.3.0/x/todo-mvc/build/todo-mvc/todo-mvc.jsexe/index.html

js-sources from the `miso.cabal` file are not linked into all.js. One work around for this is to `cp jsbits/*` to the `.jsexe` directory and edit the `index.html` file and add: ``` <script language="javascript" src="delegate.js" defer></script> <script language="javascript" src="diff.js" defer></script> <script language="javascript" src="isomorphic.js" defer></script> <script language="javascript" src="util.js" defer></script> ``` To build one of the examples with the included haskell.nix flake: ``` nix develop .# javascript-unknown-ghcjs-cabal build todo-mvc cp jsbits/* dist-newstyle/build/javascript-ghcjs/ghc-9.8.2/miso-examples-1.8.3.0/x/todo-mvc/build/todo-mvc/todo-mvc.jsexe/ vim dist-newstyle/build/javascript-ghcjs/ghc-9.8.2/miso-examples-1.8.3.0/x/todo-mvc/build/todo-mvc/todo-mvc.jsexe/index.html ```

ysangkok · 2024-04-18T14:21:20Z

Considering that the linking would be done in Cabal 3.12, wouldn't it make most sense to just gate the buildability on Cabal 3.12 in the cabal files?

hasufell · 2024-04-18T14:33:31Z

Considering that the linking would be done in Cabal 3.12, wouldn't it make most sense to just gate the buildability on Cabal 3.12 in the cabal files?

I can make a cabal pre-release in GHCup, so we can test this. (I have no interest in the nix portion)

miso.cabal

hasufell · 2024-04-19T08:56:23Z

I added cabal prerelease 3.11.0.0.2024.4.19 haskell/ghcup-metadata@eaf801c

But I can't build the todo-mvc. There are too many errors with dependency resolution. It has not been kept up to date.

hamishmack · 2024-04-19T13:40:11Z

I forgot to include cabal in the nix develop shell and had 3.9 in my PATH. I've updated that and now the .js files are linked:

Tested with

rm -rf dist-newstyle
nix develop .#
javascript-unknown-ghcjs-cabal build todo-mvc
open dist-newstyle/build/javascript-ghcjs/ghc-9.8.2/miso-examples-1.8.3.0/x/todo-mvc/build/todo-mvc/todo-mvc.jsexe/index.html

hamishmack · 2024-04-19T13:45:18Z

But I can't build the todo-mvc. There are too many errors with dependency resolution. It has not been kept up to date.

What are the errors?

dmjio · 2024-04-19T14:01:09Z

@hamishmack I noticed this uses haskell.nix. Are you aware of any efforts to upstream the GHC derivations (both js backend and wasm backend) into nixpkgs proper (similar to what ghcjs did earlier), utilizing @Ericson2314's existing cross infrastructure?

@hsyl20 Since ghcjs-8.10 there seems to have been regressions in the optimization backend (leading to larger payloads), this is why miso stays pinned on ghcjs-8.6. If this PR were to be merged, would the new js backend be at feature parity to ghcjs-8.6 ? Or, is this merge at risk of causing regressions? Current status page seems to indicate the optimizations are still not ported yet from 8.6. I wouldn't mind helping with this endeavor unless its already in flight.

@hasufell thanks for building todo-mvc w/ the new js backend, can we capture the build failure log in a new issue

hsyl20 · 2024-04-22T08:33:17Z

@hsyl20 Since ghcjs-8.10 there seems to have been regressions in the optimization backend (leading to larger payloads), this is why miso stays pinned on ghcjs-8.6. If this PR were to be merged, would the new js backend be at feature parity to ghcjs-8.6 ? Or, is this merge at risk of causing regressions? Current status page seems to indicate the optimizations are still not ported yet from 8.6. I wouldn't mind helping with this endeavor unless its already in flight.

We haven't ported the Compactor from ghcjs so code size is probably larger with the JS backend than with ghcjs-8.6. So expect regressions regarding code size for now. Performance should be better though (especially TH). We don't have anything in flight about this (JS backend is no longer a priority at work).

Nevertheless a new contributor has been working on code size: https://gitlab.haskell.org/ghc/ghc/-/issues/24584 / https://gitlab.haskell.org/ghc/ghc/-/merge_requests/12330
It's promising: Google Closure Compiler support should be enabled again soon thanks to him. Any help appreciated :)

hasufell · 2024-04-22T09:54:42Z

But I can't build the todo-mvc. There are too many errors with dependency resolution. It has not been kept up to date.

What are the errors?

$ cabal build --with-ghc=javascript-unknown-ghcjs-ghc --with-ghc-pkg=javascript-unknown-ghcjs-ghc-pkg --with-hsc2hs=javascript-unknown-ghcjs-hsc2hs
  todo-mvc
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
[__0] trying: app-0.1.0.0 (user goal)
[__1] trying: base-4.20.0.0/installed-inplace (dependency of app)
[__2] trying: haskell-miso-0.1.0.1 (user goal)
[__3] next goal: servant (dependency of haskell-miso)
[__3] rejecting: servant-0.20.1 (conflict: base==4.20.0.0/installed-inplace, servant => base>=4.9 && <4.20)
[__3] skipping: servant; 0.20, 0.19.1, 0.19, 0.18.3, 0.18.2, 0.18.1, 0.18, 0.17, 0.16.2, 0.16.1, 0.16.0.1, 0.16, 0.15, 0.14.1, 0.14, 0.13.0.1, 0.13, 0.12.1, 0.12, 0.11, 0.10, 0.9.1.1, 0.9.1, 0.9.0.1, 0.9, 0.8.1, 0.8, 0.7.1, 0.7, 0.6.1, 0.6, 0.5, 0.4.4.7, 0.4.4.6, 0.4.4.5, 0.4.4.4, 0.4.4.3, 0.4.4.2, 0.4.4, 0.4.3.1, 0.4.3, 0.4.2, 0.4.1, 0.4.0 (has the same characteristics that caused the previous version to fail: excludes 'base' version 4.20.0.0)
[__3] trying: servant-0.2.2
[__4] trying: text-2.1.1/installed-inplace (dependency of servant)
[__5] trying: binary-0.8.9.2/installed-inplace (dependency of text)
[__6] next goal: aeson (dependency of haskell-miso)
[__6] rejecting: aeson-2.2.1.0 (conflict: binary => containers==0.7/installed-inplace, aeson => containers>=0.5.10.2 && <0.7)
[__6] rejecting: aeson-2.2.0.0 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson => bytestring>=0.10.8.2 && <0.12)
[__6] rejecting: aeson; 2.1.2.1, 2.1.2.0, 2.1.1.0, 2.1.0.0, 2.0.3.0 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson => bytestring>=0.10.8.1 && <0.12)
[__6] rejecting: aeson; 2.0.2.0, 2.0.1.0 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson => bytestring>=0.10.8.1 && <0.11.2)
[__6] rejecting: aeson-2.0.0.0 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson => bytestring>=0.10.8.1 && <0.11.2.0)
[__6] rejecting: aeson; 1.5.6.0, 1.5.5.1 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson => bytestring>=0.10.8.1 && <0.11.2)
[__6] rejecting: aeson-1.5.5.0 (conflict: base==4.20.0.0/installed-inplace, aeson => base(>=4.7.0.0 && <5) && <0)
[__6] rejecting: aeson-1.5.4.1 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson => bytestring>=0.10.8.1 && <0.11.2)
[__6] rejecting: aeson; 1.5.4.0, 1.5.3.0, 1.5.2.0, 1.5.1.0, 1.5.0.0, 1.4.7.1, 1.4.7.0 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson => bytestring>=0.10.8.1 && <0.11)
[__6] rejecting: aeson; 1.4.6.0, 1.4.5.0, 1.4.4.0 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson +/-bytestring-builder => bytestring>=0.10.8.1 && <0.11)
[__6] rejecting: aeson; 1.4.3.0, 1.4.2.0, 1.4.1.0, 1.4.0.0, 1.3.1.1, 1.3.1.0, 1.3.0.0, 1.2.4.0, 1.2.3.0, 1.2.2.0 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson +/-bytestring-builder => bytestring>=0.9.2 && <0.11)
[__6] rejecting: aeson-1.2.1.0 (conflict: base==4.20.0.0/installed-inplace, aeson => base>=4.5 && <4.13)
[__6] skipping: aeson; 1.2.0.0, 1.1.2.0, 1.1.1.0, 1.1.0.0 (has the same characteristics that caused the previous version to fail: excludes 'base' version 4.20.0.0)
[__6] rejecting: aeson; 1.0.2.1, 1.0.2.0, 1.0.1.0, 1.0.0.0 (conflict: text => bytestring==0.12.1.0/installed-inplace, aeson +/-bytestring-builder => bytestring>=0.9 && <0.11)
[__6] rejecting: aeson-0.11.3.0 (conflict: base==4.20.0.0/installed-inplace, aeson => base>=4.5 && <4.13)
[__6] skipping: aeson; 0.11.2.1, 0.11.2.0, 0.11.1.4, 0.11.1.3, 0.11.1.2, 0.11.1.1, 0.11.1.0, 0.11.0.0, 0.9.0.1, 0.9.0.0, 0.8.1.1, 0.8.1.0, 0.8.0.2 (has the same characteristics that caused the previous version to fail: excludes 'base' version 4.20.0.0)
[__6] trying: aeson-0.7.0.6
[__7] next goal: unordered-containers (dependency of aeson)
[__7] rejecting: unordered-containers; 0.2.20, 0.2.19.1 (conflict: text => template-haskell==2.22.0.0/installed-inplace, unordered-containers => template-haskell<2.22)
[__7] rejecting: unordered-containers; 0.2.18.0, 0.2.17.0 (conflict: text => template-haskell==2.22.0.0/installed-inplace, unordered-containers => template-haskell<2.19)
[__7] rejecting: unordered-containers-0.2.16.0 (conflict: base==4.20.0.0/installed-inplace, unordered-containers => base>=4.9 && <4.20)
[__7] skipping: unordered-containers-0.2.15.0 (has the same characteristics that caused the previous version to fail: excludes 'base' version 4.20.0.0)
[__7] trying: unordered-containers-0.2.14.0
[__8] next goal: time (dependency of aeson)
[__8] rejecting: time-1.12.2/installed-inplace (conflict: aeson => time>=1.1.3 && <1.5)
[__8] skipping: time; 1.14, 1.12.2, 1.12.1, 1.12, 1.11.1.2, 1.11.1.1, 1.11.1, 1.11, 1.10, 1.9.3, 1.9.2, 1.9.1, 1.9, 1.8.0.4, 1.8.0.3, 1.8.0.2, 1.8.0.1, 1.8, 1.7.0.1, 1.7, 1.6.0.1, 1.6, 1.5.0.1, 1.5 (has the same characteristics that caused the previous version to fail: excluded by constraint '>=1.1.3 && <1.5' from 'aeson')
[__8] rejecting: time; 1.4.2, 1.4.1 (conflict: text => deepseq==1.5.0.0/installed-inplace, time => deepseq>=1.1 && <1.4)
[__8] rejecting: time-1.4.0.2 (conflict: base==4.20.0.0/installed-inplace, time => base>=4.4 && <4.7)
[__8] skipping: time; 1.4.0.1, 1.4, 1.3, 1.2.0.5, 1.2.0.4, 1.2.0.3, 1.2.0.2, 1.2.0.1, 1.2, 1.1.4, 1.1.3, 1.1.2.4, 1.1.2.3, 1.1.2.2, 1.1.2.1, 1.1.2.0, 1.0, 1.13 (has the same characteristics that caused the previous version to fail: excludes 'base' version 4.20.0.0)
[__8] fail (backjumping, conflict set: aeson, base, text, time)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: hashable, base, aeson, unordered-containers, time, text, os-string, haskell-miso, binary, servant, witherable, indexed-traversable-instances, app
Try running with --minimize-conflict-set to improve the error message.

Then adding --allow-newer=base --allow-newer=lens --allow-newer=template-haskell it leads to:

Failed to build text-short-0.1.5.
Build log (
/home/hasufell/.cabal/logs/ghc-9.10.0.20240413/text-short-0.1.5-22117b601864ae04326999c24331c77959069c958772ffb1dcf27566c8436c41.log
):
Configuring library for text-short-0.1.5...
Preprocessing library for text-short-0.1.5...
Building library for text-short-0.1.5...
[1 of 5] Compiling PrimOps          ( src-ghc804/PrimOps.hs, dist/build/PrimOps.o )
[2 of 5] Compiling Data.Text.Short.Internal ( src/Data/Text/Short/Internal.hs, dist/build/Data/Text/Short/Internal.o )

src/Data/Text/Short/Internal.hs:69:7: error: [GHC-87543]
    Ambiguous occurrence ‘foldl'’.
    It could refer to
       either ‘Prelude.foldl'’,
              imported from ‘Prelude’ at src/Data/Text/Short/Internal.hs:(139,1)-(144,80)
              (and originally defined in ‘ghc-internal-9.1001.0:GHC.Internal.Data.Foldable’),
           or ‘Data.Text.Short.Internal.foldl'’,
              defined at src/Data/Text/Short/Internal.hs:586:1.
   |
69 |     , foldl'
   |       ^^^^^^
Error: [Cabal-7125]
Failed to build text-short-0.1.5 (which is required by exe:todo-mvc from miso-examples-1.8.3.0). See the build log above for details.

Ericson2314 · 2024-04-22T12:09:18Z

utilizing @Ericson2314's existing cross infrastructure?

FWIW, Obsidian has also been using Haskell.nix as of late. @sternenseemann has done much more work with GHC packaging in Nixpkgs than me as of late, I would ask him about this.

ysangkok · 2024-04-22T17:13:30Z

@hasufell Are you using head.hackage? They should have a patch for text-short: https://gitlab.haskell.org/ghc/head.hackage/-/blob/master/patches/text-short-0.1.5.patch

tysonzero · 2024-04-22T19:47:24Z

@hsyl20 IIRC the js payload sizes I've seen for ghcjs without the compactor are a full order of magnitude or two too large for production use when compiling larger codebases. I'm assuming this PR should not be merged into master prior to the addition of such a compactor or equivalent. Although it's possible I'm mixing it up with other code size related ghcjs regressions I've seen, will try to test soon to verify.

noinia · 2024-04-22T20:46:28Z

@hsyl20 IIRC the js payload sizes I've seen for ghcjs without the compactor are a full order of magnitude or two too large for production use when compiling larger codebases. I'm assuming this PR should not be merged into master prior to the addition of such a compactor or equivalent. Although it's possible I'm mixing it up with other code size related ghcjs regressions I've seen, will try to test soon to verify.

TBH I don't think that argument of not merging this until there is a compactor makes sense. As far as I understand it, merging this patch does not prevent you from sticking to an ancient ghcjs, so whatever you are using remains usable. On the other hand, not merging it does make sure that newer people cannot use the new, up to date, ghc js backend with miso, even for small projects where code size does not matter too much [1]. Once we/people can actually build/use the js backend, that may also stimulate contributions to actually improving these aspects of the current js backend. I think that simply waiting even longer will (even further) hamper adaption of the js backend.

[1] yes, I know you can again check out this specific PR branch etc. But all of this makes life much more annoying than needed.

sternenseemann · 2024-04-23T09:49:47Z

Are you aware of any efforts to upstream the GHC derivations (both js backend and wasm backend) into nixpkgs proper (similar to what ghcjs did earlier)

It has been possible to use pkgsCross.ghcjs for the JS backend for a while. Recently there have been some regressions which all have been addressed in the last weeks. Once NixOS/nixpkgs#304640 has made it into released channels, the JS backend in nixpkgs should be in very good shape. We have 9.6, 9.8 and a pretty recent version of HEAD packaged.

As for the wasm backend, I concluded last year that it the packaging situation is way to brittle at the moment: NixOS/nixpkgs#225000 (comment). I haven't checked if anything's changed lately. From my perspective the LLVM patches the wasm backend needs would need to be upstreamed and probably the ones for wasi-libc as well.

hasufell · 2024-04-23T11:05:16Z

@hasufell Are you using head.hackage? They should have a patch for text-short: https://gitlab.haskell.org/ghc/head.hackage/-/blob/master/patches/text-short-0.1.5.patch

Yes that works. But I believe that should be part of this PR then?

tysonzero · 2024-04-24T15:48:04Z

@noinia ah I had assumed it would replace the existing setup, but it looks like it takes advantage of the flake.nix vs default.nix distinction to have both exist. I'm no nix expert but I assume this means that nix-build and nix build will build wildly different versions instead of doing the same thing? Having nix-build use old production ready versions and nix build use new maybe-not-production-ready versions and maintaining them in parallel seems a bit weird, and at bare minimum should update the README.

Getting ghc javascript backend support for end users seems like it doesn't actually require miso to use the new stuff internally though. As long as miso gets the flag changes and gets pushed to hackage/stackage then haskell.nix users are just going to pull it from there anyway and not touch any nix/similar files in the miso repo itself. I'd personally say it makes the most sense to PR just those changes and leave miso's nix config alone until there is high certainty that it is an upgrade for real world ghcjs users to full-switch to the new stuff.

dmjio · 2024-04-25T19:12:35Z

@sternenseemann thank you for your work on upstreaming the js and wasm backends into nixpkgs proper.

These derivations would fit nicely with the existing nix infrastructure in miso

@sternenseemann for this PR ideally we could utilize the cabal file and CPP changes that @hamishmack has provided, along with your nixpkgs work (both js and wasm -- linked above) as well.

Regarding the wasm backend, maybe we could all (@dmjio @sternenseemann @TerrorJack) sync over matrix what's required to get your PR updated and into nixpkgs in a way that doesn't feel brittle (since some corporate users are enthusiastic about this work).

Per talks with @TerrorJack, it might no longer be necessary to apply additional patches to GHC but it seems we still need to maintain a custom patchset on top of llvm.

I'd be willing to help maintain these derivations if they were merged into nixpkgs and perform any necessary changes as they arise from @TerrorJack's development.

tysonzero · 2024-04-28T20:19:34Z

Oof. >14MB binary size for Miso's TodoMVC. This is before closure and any other build size reducing flags (are there any? I've used dedupe and GHCJS_BROWSER in ghcjs-8.6 but I don't think either work with new backend), but still a pretty insane difference vs ghcjs-8.6's 1MB. I'm guessing our currently-10MB much larger work codebase would exceed 100MB, although the effort required to get it building is too high for it to be worth it right now without higher odds of a viable end result.

Given the above I stand by the above course of action, namely removing the flake.nix, hix.nix and cabal.project portions of this PR and merging the rest of it. That way miso avoids tacitly endorsing the new backend for production use before it's ready, as well as avoiding maintaining two different nix configurations simultaneously. @hamishmack

$ du -hs result/bin/todo-mvc
14M	result/bin/todo-mvc

hsyl20 · 2024-04-29T07:58:32Z

Thanks for checking this! Could you please open a ticket on GHC's gitlab about this JS code size issue:

with the attached 1MB file
giving the instructions to build the same project with the JS backend

I believe that having the old small version as a reference could be motivating for someone to try to do at least as well and to look for differences in the generated code.

I also agree with the suggested plan for this repo.

dmjio · 2024-04-29T14:11:26Z

@hsyl20 It seems this issue is documented here ghcjs/ghcjs#821 (unless this is no longer relevant? and code size is now due to a different issue). Is @luite no longer going to be prioritizing work on the JS backend? I think he might be the best chance to solve this to get us back to 8.6 standards.

@Ericson2314 you mentioned Obsidian is using this new backend w/ haskell.nix, how is the increase in payload size being addressed?

@stepcut, curious to know if you've tried the new backend as well.

Ericson2314 · 2024-04-29T14:21:19Z

@dmjio We've been using haskell.nix but have not yet tried out the new backend yet. We switched to haskell.nix for using the version solver.

hsyl20 · 2024-04-29T15:00:23Z

@dmjio Most of the optimizer mentioned in this ticket has been reimplemented by Luite in the JS backend for 9.10 (I don't remember if it has been backported into 9.8). Iirc some parts of it were unsound in the earlier implementation and that's why it had been disabled in the past and reimplemented from scratch.

I would recommend migrating ghcjs issues that are still relevant to the JS backend to GHC's issue tracker: https://gitlab.haskell.org/ghc/ghc/-/issues We've been unsure about what to do with GHCJS issues now that GHCJS is basically discontinued.

Is @luite no longer going to be prioritizing work on the JS backend? I think he might be the best chance to solve this to get us back to 8.6 standards.

Yes Luite is a member of my team and we had to deprioritize the JS backend to work on other things, at least for now. Some of our objectives with the JS backend were to reduce the maintenance cost (bumping GHC versions...) and to allow for new devs to join the effort so that it doesn't fully rest on Luite's shoulders. It's starting to pay off, e.g. Serge has been doing some good work lately and we're assisting him and others as much as we can.

I'd recommend opening a ticket about this issue on GHC's gitlab, so that GHC devs (including Luite) can discuss what to do about it and hopefully find someone to execute. If there is a pressing need, maybe we (IOG) can provide commercial support and prioritize the JS backend again, but you should get in touch with @angerman to discuss this.

tysonzero · 2024-04-29T18:25:54Z

Thanks @hamishmack for the PR update

@hsyl20 I can add this stuff to gitlab soon yes. Should I make a new issue or add it into https://gitlab.haskell.org/ghc/ghc/-/issues/24584?

Obviously I'm not paying for the js backend work so anything you guys do is greatly appreciated, but I will say I'm a tad surprised at the timing for the deprioritization. I realize the sunk cost fallacy is a fallacy for a reason, but spending over a year working hard to get this into ghc only for it to be deemed low prio seemingly one remaining issue/epic away from production readiness is kinda wild. Hopefully as you say other devs will take up the mantle, but i'm assuming every ghcjs-8.6 project/organization out there is going to be sticking with 8.6 for now, as I know sadly we will have to.

dmjio · 2024-04-29T19:41:25Z

@hsyl20 I tend to agree with @tysonzero here. Despite the fact that most social media / news websites all range in 1MB-10MB (sometimes more) of total js delivered, js payload size is the number one impediment to adoption of this library, and has been for ~6 years (as stated by other developers). We've done as much as we can to reduce the problem by putting most of the critical logic in js, using pre-rendering w/ async loading of js, browser caching, closure compiling, but it is still an impasse in the minds of developers (for some reason, due to prior experience etc.), so any regressions in this direction are difficult to justify.

In summary, I agree what @tysonzero has said, go with a merge, but avoid tacit approval until feature parity w/ 8.6 is reached.

As an aside, I think a solution could lie in the direction of moving GHC towards a whole program optimization approach w/ a dead code elimination pass, similar to what @csabahruska has done / is doing with the GRIN project..

Ericson2314 · 2024-04-29T22:15:03Z

I am happy that the "hamishverse" of libraries is now ported over. Being able to try out the new backend in real-world projects can help foster a sense of "the end being in sight", and also triage what exactly is needed to reach feature/perf/etc. parity with GHCJS again for real-world apps. This is a good way to help avoid things getting deprioritized again/further.

I also agree it is very important we squash out these code size regressions --- even having an unmerged branch of GHC (the last GHCJS) with the old unsound algorithms restored might be a useful stepping stone.

hsyl20 · 2024-04-30T07:42:26Z

@hsyl20 I can add this stuff to gitlab soon yes. Should I make a new issue or add it into https://gitlab.haskell.org/ghc/ghc/-/issues/24584?

I would create a new issue with the reproducer for TodoMVC. There may be more things to do than for the linked issue (e.g. browser specific optimizations). And for this new one we'll have a concrete size target: the code generated by ghcjs-8.6.

Obviously I'm not paying for the js backend work so anything you guys do is greatly appreciated, but I will say I'm a tad surprised at the timing for the deprioritization. I realize the sunk cost fallacy is a fallacy for a reason, but spending over a year working hard to get this into ghc only for it to be deemed low prio seemingly one remaining issue/epic away from production readiness is kinda wild.

Deprioritization was a bit of a surprise for us too. The roadmap of other products changed and the need for the JS backend declined.

As an aside, I think a solution could lie in the direction of moving GHC towards a whole program optimization approach w/ a dead code elimination pass, similar to what @csabahruska has done / is doing with the GRIN project..

The JS backend already does some aggressive dead code elimination (a bit similar to native with -split-sections), but yes that could be an interesting approach to remove even more code and to specialize it even more.

I also have some ideas to change the representation of closures to share more JS code (same entry code for closures). I'll open a ticket about it.

alexfmpe · 2024-05-06T22:23:46Z

It has been possible to use pkgsCross.ghcjs for the JS backend for a while. Recently there have been some regressions which all have been addressed in the last weeks. Once NixOS/nixpkgs#304640 has made it into released channels, the JS backend in nixpkgs should be in very good shape. We have 9.6, 9.8 and a pretty recent version of HEAD packaged.

Adding to this, I've been able to get pkgsCross.ghcjs.haskell.packages.ghc98.ghcjs-dom to build (NixOS/nixpkgs#309650), and there's only a few more overrides to upstream until miso also does.

@dmjio Seeing as the nixpkgs snapshot is obtained from hackage/stackage, would it be possible to make a hackage release of miso with the js backend support?

I've got it to build locally with some invasive sed surgery, but it's still based on the 1.8.3 release, not the parent commit of this PR.

miso = (addBuildDepend self.ghcjs-base (disableLibraryProfiling super.miso)).overrideAttrs (_: {
  postPatch = ''
    sed -i 's@impl(ghcjs)@arch(javascript)@' miso.cabal
    sed -i 's@__GHCJS__@ghcjs_HOST_OS@' src/Miso.hs
    sed -i 's@__GHCJS__@ghcjs_HOST_OS@' jsstring-src/Miso/String.hs
  '';
});

dmjio · 2024-05-07T11:12:43Z

@alexfmpe

alexfmpe · 2024-06-20T13:26:03Z

@dmjio nix-build -A pkgsCross.ghcjs.haskell.packages.ghc98.miso now works on nixpkgs' master.

tysonzero · 2024-06-30T22:15:50Z

@alexfmpe out of curiosity what are the binary sizes involved when doing that? Can any of the examples be built more or less the same way without too much fanfare, it'd be nice to be able to easily monitor those.

My current smaller 4k LOC / 55 module application clocks in at ~30MB js size without any gzipping, minification or closure compilation with a recent js backend haskell.nix commit.

alexfmpe · 2024-07-03T19:36:48Z

@alexfmpe out of curiosity what are the binary sizes involved when doing that?

Dunno, never used miso. It was sort of a drive-by for me when fixing other stuff in pkgsCross.ghcjs.
I suspect pkgsCross.ghcjs.haskell.packages.ghc98 still has the same bloating you observed, and pkgsCross.ghcjs.haskell.packages.ghc910 isn't quite there yet to see the effect of optimizer changes.
I wouldn't be surprised if we need to wait for 9.12 when https://gitlab.haskell.org/ghc/ghc/-/issues/24584 lands to finally be able to ditch old 8.6/8.10 ghcjs.

Can any of the examples be built more or less the same way without too much fanfare, it'd be nice to be able to easily monitor those.

Not sure. The executables in miso-examples are all behind a if ghcjs which makes nix-build -A pkgsCross.ghcjs.haskell.packages.ghc98.miso-examples fail due to not finding buildable components due to a current limitation of cabal2nix not picking up things behind conditionals, so I can't check this via nixpkgs without patching something. The other option would be updating the nix here to use pkgsCross.ghcjs but there's many places where that seems necessary to change before it can be checked and I don't know my way around this repo.

dmjio · 2024-07-03T19:54:10Z

@alexfmpe I pushed 1.8.4.0 to https://hackage.haskell.org/package/miso-examples. So nix-build -A pkgsCross.ghcjs.haskell.packages.ghc98.miso-examples will work next hackage-to-nixpkgs update

ysangkok mentioned this pull request Apr 18, 2024

GHC WebAssembly/JS backend support #724

Open

hsyl20 reviewed Apr 19, 2024

View reviewed changes

miso.cabal Show resolved Hide resolved

Include cabal in nix develop shell

321ca67

Remove cabal.project and nix files

f2743b3

dmjio merged commit 2797f75 into dmjio:master Apr 29, 2024
3 checks passed

tysonzero mentioned this pull request May 8, 2024

Update for new GHC JS backend haskell-servant/servant-jsaddle#7

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support GHC JavaScript backend #738

Support GHC JavaScript backend #738

hamishmack commented Apr 17, 2024

ysangkok commented Apr 18, 2024

hasufell commented Apr 18, 2024 •

edited

Loading

hasufell commented Apr 19, 2024

hamishmack commented Apr 19, 2024

hamishmack commented Apr 19, 2024

dmjio commented Apr 19, 2024

hsyl20 commented Apr 22, 2024

hasufell commented Apr 22, 2024 •

edited by dmjio

Loading

Ericson2314 commented Apr 22, 2024

ysangkok commented Apr 22, 2024

tysonzero commented Apr 22, 2024 •

edited

Loading

noinia commented Apr 22, 2024 •

edited

Loading

sternenseemann commented Apr 23, 2024

hasufell commented Apr 23, 2024

tysonzero commented Apr 24, 2024

dmjio commented Apr 25, 2024

tysonzero commented Apr 28, 2024 •

edited

Loading

hsyl20 commented Apr 29, 2024

dmjio commented Apr 29, 2024 •

edited

Loading

Ericson2314 commented Apr 29, 2024

hsyl20 commented Apr 29, 2024

tysonzero commented Apr 29, 2024

dmjio commented Apr 29, 2024 •

edited

Loading

Ericson2314 commented Apr 29, 2024

hsyl20 commented Apr 30, 2024

alexfmpe commented May 6, 2024 •

edited

Loading

dmjio commented May 7, 2024

alexfmpe commented Jun 20, 2024 •

edited

Loading

tysonzero commented Jun 30, 2024

alexfmpe commented Jul 3, 2024

dmjio commented Jul 3, 2024

Support GHC JavaScript backend #738

Support GHC JavaScript backend #738

Conversation

hamishmack commented Apr 17, 2024

ysangkok commented Apr 18, 2024

hasufell commented Apr 18, 2024 • edited Loading

hasufell commented Apr 19, 2024

hamishmack commented Apr 19, 2024

hamishmack commented Apr 19, 2024

dmjio commented Apr 19, 2024

hsyl20 commented Apr 22, 2024

hasufell commented Apr 22, 2024 • edited by dmjio Loading

Ericson2314 commented Apr 22, 2024

ysangkok commented Apr 22, 2024

tysonzero commented Apr 22, 2024 • edited Loading

noinia commented Apr 22, 2024 • edited Loading

sternenseemann commented Apr 23, 2024

hasufell commented Apr 23, 2024

tysonzero commented Apr 24, 2024

dmjio commented Apr 25, 2024

tysonzero commented Apr 28, 2024 • edited Loading

hsyl20 commented Apr 29, 2024

dmjio commented Apr 29, 2024 • edited Loading

Ericson2314 commented Apr 29, 2024

hsyl20 commented Apr 29, 2024

tysonzero commented Apr 29, 2024

dmjio commented Apr 29, 2024 • edited Loading

Ericson2314 commented Apr 29, 2024

hsyl20 commented Apr 30, 2024

alexfmpe commented May 6, 2024 • edited Loading

dmjio commented May 7, 2024

alexfmpe commented Jun 20, 2024 • edited Loading

tysonzero commented Jun 30, 2024

alexfmpe commented Jul 3, 2024

dmjio commented Jul 3, 2024

hasufell commented Apr 18, 2024 •

edited

Loading

hasufell commented Apr 22, 2024 •

edited by dmjio

Loading

tysonzero commented Apr 22, 2024 •

edited

Loading

noinia commented Apr 22, 2024 •

edited

Loading

tysonzero commented Apr 28, 2024 •

edited

Loading

dmjio commented Apr 29, 2024 •

edited

Loading

dmjio commented Apr 29, 2024 •

edited

Loading

alexfmpe commented May 6, 2024 •

edited

Loading

alexfmpe commented Jun 20, 2024 •

edited

Loading