Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: use esbuild for building main renderer code #722

Draft
wants to merge 46 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
9076ec2
chore: upgraded electron and node
ErikSin Jan 30, 2023
aa774f2
bring back map server using latest
achou11 Feb 1, 2023
3184ed1
replace electron.remote with @electron/remote
achou11 Feb 1, 2023
6ec4819
fix usage of electron-is-dev module
achou11 Feb 1, 2023
ba81279
add patch for ecstatic
achou11 Feb 1, 2023
5ce26c8
remove duplicate code in app.js
achou11 Feb 2, 2023
5248e38
fix import of electron remote in renderer
achou11 Feb 2, 2023
3d06334
remove electron remote from externals
achou11 Feb 2, 2023
b1c0c25
fix issue with creating map server db before directory exists
achou11 Feb 2, 2023
5cc4c5b
fix mapServer scoping error when quitting app
achou11 Feb 2, 2023
364241b
upgrade react-intl to remove usage of broken peer dep
achou11 Feb 8, 2023
d92a821
remove unused livereload functionality
achou11 Feb 8, 2023
536c002
remove commented-out webpack plugin
achou11 Feb 8, 2023
c548a1a
update comment in new api file
achou11 Feb 8, 2023
9ea06e0
adjustments for window.mode in background windows
achou11 Feb 8, 2023
d78ae70
upgrade electron to 22.2.0
achou11 Feb 8, 2023
4278c40
ci changes
achou11 Feb 8, 2023
1d62322
Merge branch 'master' into es/chore-electron-upgrade
achou11 Feb 8, 2023
39fabc8
maybe fix electron build issue with fsevents
achou11 Feb 8, 2023
d230364
maybe fix rimraf issue on windows ci
achou11 Feb 8, 2023
666ba8f
partially undo previous
achou11 Feb 8, 2023
9e9e37d
try node script for ci artifact removal
achou11 Feb 9, 2023
461bf68
upgrade map server to latest
achou11 Feb 9, 2023
3f98e7d
upgrade electron to latest 22
achou11 Feb 9, 2023
41f5a65
update babel config
achou11 Feb 14, 2023
c730f47
Merge branch 'master' into es/chore-electron-upgrade
achou11 Feb 28, 2023
709fa4d
update electron to latest v22
achou11 Feb 28, 2023
fa7bd87
upgrade electron to v22.3.2
achou11 Mar 8, 2023
1029845
update defineMessages usage to avoid using react-intl babel transform
achou11 Feb 20, 2023
68c6cf7
add react-pdf step to script
achou11 Feb 21, 2023
5526975
move preload scripts
achou11 Feb 21, 2023
e2e4672
project config stuff
achou11 Feb 21, 2023
26c0090
fix es module issue with react-mapbox-gl
achou11 Feb 21, 2023
b6c34c2
changes to renderer code
achou11 Feb 21, 2023
e85b801
update main.html
achou11 Feb 21, 2023
30c888f
make backend work okay with dev server
achou11 Feb 22, 2023
f1740dc
clean up static dir
achou11 Feb 22, 2023
a4a0255
update files based on static directory changes
achou11 Feb 22, 2023
a55a572
update build script
achou11 Feb 22, 2023
a6089eb
update readme
achou11 Feb 22, 2023
0359290
more changes to build config
achou11 Feb 28, 2023
c94707b
random package.json changes
achou11 Feb 28, 2023
04dee39
update build script to use parseArgs
achou11 Feb 28, 2023
163a0ab
WIP: worker stuff
achou11 Mar 8, 2023
e7c7697
Merge branch 'master' into esbuild
achou11 Mar 8, 2023
4106593
Revert "WIP: worker stuff"
achou11 Mar 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,11 @@ vendor
dist
examples/.dat
static/*.bundle.js*
static/*.bundle.css*
static/*.worker.js*
static/*.ttf
static/*.chunk.js*

test/lib/test-data
bin/test-data
translations
Expand Down
7 changes: 3 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,19 @@ environments. It uses [mapeo-core](https://github.com/digidem/mapeo-core) for
offline peer-to-peer synchronization of an OpenStreetMap database, without any
server. The map editor is based on [iDEditor](https://github.com/openstreetmap/iD/),
a simple and easy to use editor for OpenStreetMap. The app is built with
[Electron](http://electron.atom.io).
[Electron](http://electronjs.org).

This project is considered stable and used by over 150 communities.

![screenshot](static/screenshot.png)
![screenshot](docs/screenshot.png)

For a mobile application that is compatible with Mapeo Desktop, see [Mapeo Mobile](https://github.com/digidem/mapeo-mobile).

## Guide

Read the [online user guide](https://digital-democracy.gitbook.io/mapeo/) for
information on how to install aerial imagery and tiles, custom configurations,
and more.
and more.

![architecture](docs/desktop-architecture.png)

Expand Down Expand Up @@ -57,7 +57,6 @@ tail -f USERDATA/Mapeo/logs/$DATE.debug.log

See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for more details.


## Community

Connect with the Mapeo community for support & to contribute!
Expand Down
3 changes: 0 additions & 3 deletions builder.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,6 @@ const files = [
// Include everything in src/ apart from src/renderer
'src/**/*',
'!src/renderer/**/*',
// but also include src/renderer/index-preload.js since this is not included
// in the renderer bundle
'src/renderer/index-preload.js',
// Ignore a bunch of noise that is in node_modules that is not needed in the
// packaged app (identified these by scanning app.asar for the largest files)
'!**/node_modules/osm-p2p-db/benchmark${/*}',
Expand Down
File renamed without changes
201 changes: 201 additions & 0 deletions esbuild.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
#!/usr/bin/env node

// @ts-check
import fs from 'node:fs'
import path from 'node:path'
import { promisify, parseArgs } from 'node:util'
import rimraf from 'rimraf'
import * as esbuild from 'esbuild'

const VALID_COMMANDS = ['build', 'watch', 'clean']
const VALID_MODES = ['dev', 'prod']
const OUTPUT_DIR = 'static'

// Changes based on Electron version
// See https://releases.electronjs.org/releases/stable
const CHROME_VERSION = 108

/**
* @type {import('esbuild').Plugin}
*/
const copyNodeModulesAssetsPlugin = {
name: 'copy-node-modules-assets',
setup (build) {
const { outdir } = build.initialOptions

if (!outdir) {
console.warn(
'copy-node-modules-assets failed to run. no `outdir` defined'
)
return
}

// TODO: Consider moving iD + Mapbox CSS assets to static/css
// Would need to update builder.config.js

// fs.copyFileSync(
// path.resolve('node_modules/id-mapeo/dist/iD.css'),
// path.join(outdir, 'css/id-mapeo.css')
// )
// fs.copyFileSync(
// path.resolve('node_modules/mapbox-gl/dist/mapbox-gl.css'),
// path.join(outdir, 'css/mapbox-gl.css')
// )

// https://github.com/wojtekmaj/react-pdf#standard-browserify-esbuild-and-others
fs.copyFileSync(
path.resolve('node_modules/pdfjs-dist/build/pdf.worker.js'),
path.join(outdir, 'pdf.worker.js')
)

console.log('copy-node-modules-assets ran successfully!')
}
}

await runCommand(extractArgs())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

top level await...the future is now


/* ------------------------------------------------------- */

/**
* @returns {{command: 'build'|'watch'|'clean', mode: 'dev'|'mode'}}
*/
function extractArgs () {
const { values, positionals } = parseArgs({
options: {
mode: {
type: 'string',
short: 'm'
}
},
allowPositionals: true
})

const command = positionals[0]
const mode = values.mode || 'dev'

if (!VALID_COMMANDS.includes(command)) {
const message =
(command
? `Invalid command '${command}' specified.`
: `No command specified`) +
`\nPlease use one of the following: ${VALID_COMMANDS.join(', ')}`

console.error(message)

process.exit(1)
}

if (!VALID_MODES.includes(mode)) {
const message =
`Invalid mode '${mode}' specified.` +
`\nPlease use of of the following: ${VALID_MODES.join(', ')}`

console.error(message)

process.exit(1)
}

return {
command,
mode
}
}

/**
* @param {Object} opts
* @param {'build'|'watch'|'clean'} opts.command
* @param {'dev' | 'prod'} opts.mode
*/
async function runCommand (opts) {
const buildOptions = createBuildOptions(opts.mode)

switch (opts.command) {
case 'clean': {
await cleanBuildAssets()

console.log(`Successfully cleaned ${buildOptions.outdir}`)

break
}
case 'build': {
await cleanBuildAssets()

await esbuild.build(buildOptions)

console.log('Successfully built!')

break
}
case 'watch': {
const ctx = await esbuild.context(buildOptions)

await cleanBuildAssets()

await ctx.watch()

const { host, port } = await ctx.serve({
host: 'localhost',
servedir: '.'
})

console.log(`Listening at http://${host}:${port}`)

break
}
}
}

/**
* @param {'dev'|'prod'} mode
* @returns {import('esbuild').BuildOptions}
*/
function createBuildOptions (mode) {
return {
entryPoints: [
{ in: 'src/renderer/app.js', out: 'app' },
{ in: 'src/renderer/components/MapFilter/index.js', out: 'map-filter' },
{ in: 'src/renderer/components/MapEditor/index.js', out: 'map-editor' },
Comment on lines +155 to +157
Copy link
Contributor

@ErikSin ErikSin Jun 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

app.js loads home.js, which loads map-filter, and map-editor. Does esbuild just know that, and keeps map-filter and map-editor out of the app bundle? Or do we need to indicate that in the home.js file like we do with webpack?

{
in: 'src/renderer/components/MapFilter/ReportView/renderReport.worker.js',
out: 'reportWorker'
}
],
entryNames: '[name].bundle',
chunkNames: '[name]-[hash].chunk',
outdir: OUTPUT_DIR,
// Think Electron disables ESM but this should be okay, I think
// https://github.com/electron/electron/issues/21457
format: 'esm',
splitting: true,
minify: mode === 'prod',
bundle: true,
target: `chrome${CHROME_VERSION}`,
plugins: [copyNodeModulesAssetsPlugin],
loader: {
'.js': 'jsx',
'.svg': 'file',
'.ttf': 'file'
},
// Anything specified here has to use `require` to import in renderer app code
external: [
'electron',
'process',
'path',
'https',
'fs',
'child_process',
'os',
'crypto',
'stream',
'constants',
'http',
'domain',
'zlib',
'timers'
]
}
}

async function cleanBuildAssets () {
return promisify(rimraf)(`./${OUTPUT_DIR}/*.{bundle,chunk}.{js,css}`)
}
Loading