🐛 BUG: create cloudflare fails on better-sqlite3 (with python 3.12) #3531

punkeel · 2023-06-27T22:23:50Z

Which Cloudflare product(s) does this pertain to?

C3

What version of `Wrangler` are you using?

latest, 2.0.10

What operating system are you using?

Mac

Describe the Bug

Creating a new project fails, see logs:

~/code/tmp ❯ npx create-cloudflare@latest

using create-cloudflare version 2.0.10

╭ Create an application with Cloudflare Step 1 of 3
│
╰ Where do you want to create your application? also used as application name
--quiet-snow-fc42
Directory `` already exists and is not empty. Please choose a new name.
~/code/tmp ❯ npx create-cloudflare@latest

using create-cloudflare version 2.0.10

╭ Create an application with Cloudflare Step 1 of 3
│
├ Where do you want to create your application?
│ dir aged-mountain-38c0
│
├ What type of application do you want to create?
│ type "Hello World" Worker
│
├ Do you want to use TypeScript?
│ typescript no
│
├ Copying files from "hello-world" template
│
├ Do you want to use git for version control?
│ git no
│
╰ Application created

╭ Installing dependencies Step 2 of 3
│
npm ERR! gyp info spawn args   '/Users/maxime/Library/Caches/node-gyp/20.0.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/Users/maxime/Library/Caches/node-gyp/20.0.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/Users/maxime/Library/Caches/node-gyp/20.0.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/Users/maxime/Code/tmp/aged-mountain-38c0/node_modules/better-sqlite3',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! Traceback (most recent call last):
npm ERR!   File "/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py", line 42, in <module>
npm ERR!     import gyp  # noqa: E402
npm ERR!     ^^^^^^^^^^
npm ERR!   File "/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 9, in <module>
npm ERR!     import gyp.input
npm ERR!   File "/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 19, in <module>
npm ERR!     from distutils.version import StrictVersion
npm ERR! ModuleNotFoundError: No module named 'distutils'
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:325:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:511:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:293:12)
npm ERR! gyp ERR! System Darwin 22.5.0
npm ERR! gyp ERR! command "/Users/maxime/.nvm/versions/node/v20.0.0/bin/node" "/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.j
s" "rebuild" "--release"
npm ERR! gyp ERR! cwd /Users/maxime/Code/tmp/aged-mountain-38c0/node_modules/better-sqlite3
npm ERR! gyp ERR! node -v v20.0.0
npm ERR! gyp ERR! node-gyp -v v9.3.1
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /Users/maxime/.npm/_logs/2023-06-27T22_11_05_665Z-debug-0.log

│
Error: npm WARN deprecated [email protected]: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-inject.
npm WARN deprecated [email protected]: Please use @jridgewell/sourcemap-codec instead
npm ERR! code 1
npm ERR! path /Users/maxime/Code/tmp/aged-mountain-38c0/node_modules/better-sqlite3
npm ERR! command failed
npm ERR! command sh -c prebuild-install || node-gyp rebuild --release
npm ERR! prebuild-install warn install No prebuilt binaries found (target=20.0.0 runtime=node arch=arm64 libc= platform=darwin)
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | darwin | arm64
npm ERR! gyp info find Python using Python version 3.12.0 found at "/Users/maxime/.nix-profile/bin/python3"
npm ERR! gyp info spawn /Users/maxime/.nix-profile/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/maxime/Code/tmp/aged-mountain-38c0/node_modules/better-sqlite3/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/maxime/Library/Caches/node-gyp/20.0.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/Users/maxime/Library/Caches/node-gyp/20.0.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/Users/maxime/Library/Caches/node-gyp/20.0.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/Users/maxime/Code/tmp/aged-mountain-38c0/node_modules/better-sqlite3',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! Traceback (most recent call last):
npm ERR!   File "/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py", line 42, in <module>
npm ERR!     import gyp  # noqa: E402
npm ERR!     ^^^^^^^^^^
npm ERR!   File "/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 9, in <module>
npm ERR!     import gyp.input
npm ERR!   File "/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 19, in <module>
npm ERR!     from distutils.version import StrictVersion
npm ERR! ModuleNotFoundError: No module named 'distutils'
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:325:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:511:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:293:12)
npm ERR! gyp ERR! System Darwin 22.5.0
npm ERR! gyp ERR! command "/Users/maxime/.nvm/versions/node/v20.0.0/bin/node" "/Users/maxime/.nvm/versions/node/v20.0.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--release"
npm ERR! gyp ERR! cwd /Users/maxime/Code/tmp/aged-mountain-38c0/node_modules/better-sqlite3
npm ERR! gyp ERR! node -v v20.0.0
npm ERR! gyp ERR! node-gyp -v v9.3.1
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /Users/maxime/.npm/_logs/2023-06-27T22_11_05_665Z-debug-0.log

This seems to be related to compiling better-sqlite3 (why do we compile sqlite to create a project 😬 )

The text was updated successfully, but these errors were encountered:

punkeel · 2023-06-27T22:30:34Z

I suspect this is related: nodejs/node-gyp#2869

chrizy · 2023-07-12T20:41:16Z

Hi getting what I think is same error when creating a new project. npm install., wrangler v3.2.0

npm ERR! path /Users/chris/Library/Mobile Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3
npm ERR! command failed
npm ERR! command sh -c -- prebuild-install || node-gyp rebuild --release
npm ERR! TOUCH ba23eeee118cd63e16015df367567cb043fed872.intermediate
npm ERR!   ACTION deps_sqlite3_gyp_locate_sqlite3_target_copy_builtin_sqlite3 ba23eeee118cd63e16015df367567cb043fed872.intermediate
npm ERR! rm ba23eeee118cd63e16015df367567cb043fed872.intermediate
npm ERR! prebuild-install warn install No prebuilt binaries found (target=18.12.1 runtime=node arch=arm64 libc= platform=darwin)
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | darwin | arm64
npm ERR! gyp info find Python using Python version 3.9.9 found at "/opt/homebrew/opt/[email protected]/bin/python3.9"
npm ERR! gyp info spawn /opt/homebrew/opt/[email protected]/bin/python3.9
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/chris/Library/Mobile Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/chris/Library/Caches/node-gyp/18.12.1/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/Users/chris/Library/Caches/node-gyp/18.12.1',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/Users/chris/Library/Caches/node-gyp/18.12.1/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/Users/chris/Library/Mobile Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info spawn make
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm ERR! /bin/sh: Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3/build/Release/lib.host:/Users/chris/Library/Mobile: No such file or directory
npm ERR! node:internal/fs/utils:348
npm ERR!     throw err;
npm ERR!     ^
npm ERR! 
npm ERR! Error: ENOENT: no such file or directory, copyfile '/Users/chris/Library/Mobile Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3/deps/sqlite3/sqlite3.c' -> '/Users/chris/Library/Mobile Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3/build/Release/obj/gen/sqlite3/sqlite3.c'
npm ERR!     at Object.copyFileSync (node:fs:2866:3)
npm ERR!     at Object.<anonymous> (/Users/chris/Library/Mobile Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3/deps/copy.js:25:5)
npm ERR!     at Module._compile (node:internal/modules/cjs/loader:1159:14)
npm ERR!     at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
npm ERR!     at Module.load (node:internal/modules/cjs/loader:1037:32)
npm ERR!     at Module._load (node:internal/modules/cjs/loader:878:12)
npm ERR!     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
npm ERR!     at node:internal/main/run_main_module:23:47 {
npm ERR!   errno: -2,
npm ERR!   syscall: 'copyfile',
npm ERR!   code: 'ENOENT',
npm ERR!   path: '/Users/chris/Library/Mobile Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3/deps/sqlite3/sqlite3.c',
npm ERR!   dest: '/Users/chris/Library/Mobile Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3/build/Release/obj/gen/sqlite3/sqlite3.c'
npm ERR! }
npm ERR! 
npm ERR! Node.js v18.12.1
npm ERR! make: *** [ba23eeee118cd63e16015df367567cb043fed872.intermediate] Error 1
npm ERR! gyp ERR! build error 
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:201:23)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Darwin 22.5.0
npm ERR! gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--release"
npm ERR! gyp ERR! cwd /Users/chris/Library/Mobile Documents/com~apple~CloudDocs/devunus/test/apis/node_modules/better-sqlite3
npm ERR! gyp ERR! node -v v18.12.1
npm ERR! gyp ERR! node-gyp -v v9.1.0
npm ERR! gyp ERR! not ok

admah · 2023-07-13T16:58:56Z

I can't seem to replicate this with create-cloudflare's latest version (2.0.12) and node v18.12.1.

@punkeel are you still running into this issue?

punkeel · 2023-07-13T17:18:44Z

The issue comes from Python 1.12 (beta 1), at least for me. Downgrading to Python 1.11 solved it 👍

I'm not sure @chrizy's bug is the same

chrizy · 2023-07-13T18:46:06Z

Hi, I don't have python installed. is it a requirement to install it in order to use Wranger v3.x? v2 installs without issues.

mrbbot · 2023-07-31T15:20:24Z

Hey! 👋 Yep, better-sqlite3 is now a required dependency. If better-sqlite3 is unable to find prebuilt binaries, it will need to compile them from source, which will require Python to be installed. We're working on porting Miniflare's simulators (which use SQLite databases) to run as Durable Objects in workerd, which already includes SQLite. This would allow us to remove the better-sqlite3 dependency, and therefore the Python requirement. 👍

lrapoport-cf · 2023-07-31T15:25:04Z

hi @chrizy ! we'll close this issue for now but please feel free to re-open if this is still an issue for you after installing python (see @mrbbot 's comment above).

punkeel · 2023-07-31T15:30:45Z

Hi @lrapoport-cf
Could we keep this open until Python 3.12 is supported (which requires updating gyp), or add pre built packages for better-sqlite3? That’s what this issue is originally about

mrbm · 2023-08-06T20:18:19Z

Hi @lrapoport-cf Could we keep this open until Python 3.12 is supported (which requires updating gyp), or add pre built packages for better-sqlite3? That’s what this issue is originally about

Agreed, either this needs to stay open or documentation needs to change as environmental dependancies have changed.

lrapoport-cf · 2023-08-07T15:18:11Z

hi @punkeel and @mrbm , thanks for following up :) we are re-opening the ticket 👍

the work that @mrbbot noted above i.e. to port Miniflare's simulators to run as Durable Objects in workerd, which would allow us to remove the better-sqlite3 dependency and thus Python as a requirement, is being tracked internally with ticket DEVX-779.

documentation updates are in progress in cloudflare/cloudflare-docs#10241.

...and move dependencies only used in Workers to `devDependencies`. They're bundled in with the Worker code, so don't need to be installed again. Importantly, this change removes `better-sqlite3`, meaning Miniflare no longer has any native dependencies other than `workerd`. Fixes #599 Fixes cloudflare/workers-sdk#3423 Fixes cloudflare/workers-sdk#3449 Fixes cloudflare/workers-sdk#3531 Fixes cloudflare/workers-sdk#3534 Fixes cloudflare/workers-sdk#3708 Fixes cloudflare/workers-sdk#3746 Closes cloudflare/cloudflare-docs#10241

lrapoport-cf · 2023-09-25T15:29:51Z

the dependency on better-sqlite3 has been removes as part of cloudflare/miniflare#656, so this should now be resolved 👍

punkeel · 2023-09-25T15:34:27Z

Yay, thanks @mrbbot and @lrapoport-cf! c3 feels faster now, too! 🥳

...and move dependencies only used in Workers to `devDependencies`. They're bundled in with the Worker code, so don't need to be installed again. Importantly, this change removes `better-sqlite3`, meaning Miniflare no longer has any native dependencies other than `workerd`. Fixes cloudflare/miniflare#599 Fixes #3423 Fixes #3449 Fixes #3531 Fixes #3534 Fixes #3708 Fixes #3746 Closes cloudflare/cloudflare-docs#10241

punkeel added the bug Something that isn't working label Jun 27, 2023

punkeel changed the title ~~🐛 BUG: create cloudflare fails on better-sqlite3~~ 🐛 BUG: create cloudflare fails on better-sqlite3 (with python 3.12) Jun 27, 2023

admah added the c3 Relating to C3 (create-cloudflare) package label Jul 5, 2023

admah added the needs reproduction Needs reproduction from OP label Jul 13, 2023

chrizy mentioned this issue Jul 19, 2023

🐛 BUG: npm install wrangler --save-dev fails #3625

Closed

lrapoport-cf added awaiting Cloudflare response Awaiting response from workers-sdk maintainer team and removed needs reproduction Needs reproduction from OP labels Jul 31, 2023

lrapoport-cf added awaiting reporter response Needs clarification or followup from OP and removed awaiting Cloudflare response Awaiting response from workers-sdk maintainer team c3 Relating to C3 (create-cloudflare) package labels Jul 31, 2023

lrapoport-cf closed this as completed Jul 31, 2023

lrapoport-cf removed the awaiting reporter response Needs clarification or followup from OP label Jul 31, 2023

mrbm mentioned this issue Aug 6, 2023

Instructions for updating wrangler from v2.20.0 to 3.1 cloudflare/cloudflare-docs#9463

Closed

mrbm mentioned this issue Aug 7, 2023

Wrangler install and upgrade documentation missing noting new dependency that is blocking cloudflare/cloudflare-docs#10230

Closed

admah mentioned this issue Aug 7, 2023

[workers] add wrangler v3 dependency info for updating cloudflare/cloudflare-docs#10241

Closed

lrapoport-cf reopened this Aug 7, 2023

jculvey added the c3 Relating to C3 (create-cloudflare) package label Aug 23, 2023

lrapoport-cf closed this as completed Sep 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 BUG: create cloudflare fails on better-sqlite3 (with python 3.12) #3531

🐛 BUG: create cloudflare fails on better-sqlite3 (with python 3.12) #3531

punkeel commented Jun 27, 2023

punkeel commented Jun 27, 2023

chrizy commented Jul 12, 2023

admah commented Jul 13, 2023

punkeel commented Jul 13, 2023 •

edited

Loading

chrizy commented Jul 13, 2023

mrbbot commented Jul 31, 2023

lrapoport-cf commented Jul 31, 2023

punkeel commented Jul 31, 2023 •

edited

Loading

mrbm commented Aug 6, 2023

lrapoport-cf commented Aug 7, 2023

lrapoport-cf commented Sep 25, 2023

punkeel commented Sep 25, 2023

🐛 BUG: create cloudflare fails on better-sqlite3 (with python 3.12) #3531

🐛 BUG: create cloudflare fails on better-sqlite3 (with python 3.12) #3531

Comments

punkeel commented Jun 27, 2023

Which Cloudflare product(s) does this pertain to?

What version of Wrangler are you using?

What operating system are you using?

Describe the Bug

punkeel commented Jun 27, 2023

chrizy commented Jul 12, 2023

admah commented Jul 13, 2023

punkeel commented Jul 13, 2023 • edited Loading

chrizy commented Jul 13, 2023

mrbbot commented Jul 31, 2023

lrapoport-cf commented Jul 31, 2023

punkeel commented Jul 31, 2023 • edited Loading

mrbm commented Aug 6, 2023

lrapoport-cf commented Aug 7, 2023

lrapoport-cf commented Sep 25, 2023

punkeel commented Sep 25, 2023

What version of `Wrangler` are you using?

punkeel commented Jul 13, 2023 •

edited

Loading

punkeel commented Jul 31, 2023 •

edited

Loading