Update monorepo to node 18 (#10274)

* Node 18

* update truffle-ganache-ethereumjs dependencies

* fix build errors

* split migration into JS & TS dir

* build with latest node 18.14

* -- ganache.server promise wrapper
∆ ganache.server option format

* ignore migrations_ts js files

* sync with master

* added resolutions

* remove lib

* removed leftover old dependencies

* fixed revert assersion error message

* WIP notes

* WIP todos

* updated web3 & ethereumjs in utils & protocol

* ++ ethereum-cryptography dependency

* Merge branch 'soloseng/fixing-gas-price'

* timeout fix

* -- comments

* Fixed overflow test

* Merge branch 'soloseng/ast-version_fix'

* using istanbul hardfork

* ++ yarn.lock

* update ganache & ethereumjs version

* - removed resolutions
- using fixed node-fetch types version

* built using latest web3

* Updated ganache config

* Specify gas and await tx receipt

* updated dependencies to fix cross-fecth bind error

* updated yarn.lock

* updated node version in circleci docker file

* updated expected error message

* ensure leading 0x instead of trimming

* increased jest version

* Node 18.16.0

* - update node-fetch
- manage migrations folder during build and clean

* yarn installed using node 18.16.0

* Appease linter

* fixed import issue

* use devchain chain copy

* using fixed jest version

* disabled flake-traker

* generated CLI docs

* buypass jest_setup
in phone-number-privacy/combiner

* ignored ts error due to properties mismatch

* ++ @types/node-fetch

* yarn.lock

* using same @types/node-fetch where applicable

* ts-ignored for abort-controller

* Fix broken kit headers test

* temp force exit on contractkit test

* no detectOpenHandles

* Fix cross-fetch jest mocking issues

* lint fix

* disabled verbose logs during testing

* disabled rpc log in protocol test

* Disabled:
- pre-protocol-test-release
- protocol-test-release

* removed disabled jobs from workflow

* supress test logs

* disabled protocol test

* disabling flakey summary

* disabled odis test

* increased resource class to xlarge

* enaled protocol and odis CI test

* re-enabled
pre-protocol-test-release
protocol-test-release

* enable phone-number-privacy test logs

* Fixing mocha issues in celotool tests

* Change node to 18.14.0

* Node 18.14.2

* changed uniswap init code hash

* Revert "changed uniswap init code hash"

This reverts commit e8a0499.

* fix unicode char in datetime format

* Update e2e test docker image

* verbose output

* Pre-review cleanup

* appease linter

* removed dead code
comment cleanup

* deprecate defaultRpcCaller for httpRpcCaller
This allows compatibility with latest web3

* updated ganache-dev command

* appease linter

* updated revert assertion with message

* downgraded truffle

* updated truffle version and config

* add web3*@1.8.2 to resolutions

* Fixed stability test

* removed resolutions

* updated yarn lock

* updated governance/network test revert messages

* updated protocol common test revert assertion msg

* appease linter

* using assertRevertWithReason
in governance/validators

* using assertRevertWithReason in governance/voting

* angry linter

* using assertRevertWithReason in identity/

* fix linter issue

* using `@ethereumjs/rlp` for RLP encoding

* linter fix

* updated web3* dependencies

* specify type of transaction

* removed verbose output

* added `assertRevertWithoutReason()`

* linting fix

* added revert without msg for identity test

* updated deprecated functions

* distinguish between
transaction reverts and view call reverts

* fixed typing ignore

* specified gas, as gas estimations fails

* updated revert message in test

* updated yarn lock

* updated yarn lock again

* change docker image path

* updated revert msg to match contract

* updated docs to match current node version

* tracking issue in comment

* specified chain id

* revert to default verbose output

* removed debud logging

* GH Workflows run in node18 runners

* Force new node cache

* Disable flake-tracker

* prettify

* Updated test to revert with message

* quiet test

* fixed build errors

* updated scripts to better handle ts & js files

* moved `artifactSingleton` file to lib/

* added transaction type when using mento artifacts

* fix Migration test

* allow submodules in e2e matrix

* python2 -> python (3)

* using nvm to switch to node12 for pre-release test

* ∆ to trigger GH action

* ∆ to trigger full test in GH actions

* Using nvm for circleci image

* Do not use nvm with circleci

* use node12 to generate old release devchain

* reset and reinstall before cleaning

* set python2 in release branch

* installed python2

* added log

* removed python2 install script

* set new release tag

* trigger all test

* WIP comments

* trigger all test

* CircleCi Build pre-protocol with node12

* Circle node12 fix

* Adding python 2 and 3 to circleci image

* run devchain from tar in detached mode

* ++ comments

* updated release tag in circleCI config

* Replace pull_request by pull_request_target

* bump MTWdeployer patch version

* testing bypass dryrun

* loggin and force CI run

* removed false param

* revert change and added try catch

* moved await

* added type to tx detail

* included tx type in contract deployment

* increased logging level in ganache and truffle

* specified gas to deployment tx

* removed truffle debug logging

* keep chaincopy

* removed debug logs

* clean up redundant install
++ comment

* revert GH action trigger on current branch push

* remove comments

* remove gas specfication

* updated node version in docs

* unskip migration test

* PR feedback

* revert back to skip

* ∆ forked packages to celo namespace

* reinstalled

* cleanup TODO and comments

* python -> python3

* updated node base images to 18

* python -> python3

* Updated netcat dependency

* Fix build

* Fix build docker

* force GH action test

* removed flake-tracker

---------

Co-authored-by: Daniel Kesler <[email protected]>
Co-authored-by: Javier Cortejoso <[email protected]>
Co-authored-by: Javier Cortejoso <[email protected]>

.circleci/config.yml

@@ -4,6 +4,9 @@
 version: 2.1
 setup: true
 
+orbs:
+  node: circleci/[email protected]
+
 parameters:
   run-workflow-general:
     type: boolean
@@ -37,20 +40,22 @@ reference:
 defaults: &defaults
   working_directory: ~/app
   docker:
-    - image: us.gcr.io/celo-testnet/circleci-node12:1.1.0
+    - image: us.gcr.io/celo-testnet/circleci-node18:1.0.0
   environment:
     # To avoid ENOMEM problem when running node
     NODE_OPTIONS: '--max-old-space-size=4096'
 
 contract-defaults: &contract-defaults
   <<: *defaults
   environment:
-    RELEASE_TAG: core-contracts.v9
+    # RELEASE_TAG: core-contracts.v9
+    # TODO(soloseng): switch back to core-contract.v* once the latest contract release is done on node18.
+    RELEASE_TAG: ganache-v7-core-contracts.v9
 
 e2e-defaults: &e2e-defaults
   <<: *defaults
   docker:
-    - image: us-west1-docker.pkg.dev/devopsre/celo-monorepo/circleci-geth:latest
+    - image: us-west1-docker.pkg.dev/devopsre/clabs-public-images/circleci:node18.14-go1.16
   environment:
     CELO_BLOCKCHAIN_BRANCH_TO_TEST: release/1.7.x
 general:
@@ -87,6 +92,9 @@ commands:
       - run:
           name: Install dependencies
           command: |
+            # node-gyp needs to know where python is at when installed
+            yarn config set python $(which python3)
+
             if [ ! -e ~/.tmp/yarn_deps_have_changed ]; then
               # Though `yarn install` is instantaneous with up-to-date node_modules which is the case when we've restored cached node_modules,
               # there's currently a bug with the generated node_modules/.yarn-integrity file which is apparently not entirely stable
@@ -100,7 +108,6 @@ commands:
               yarn run postinstall
               exit 0
             fi
-
             # Deals with yarn install flakiness which can come due to yarnpkg.com being
             # unreliable. For example, https://circleci.com/gh/celo-org/celo-monorepo/82685
             yarn install || yarn install || yarn install
@@ -137,7 +144,6 @@ jobs:
     # Source: https://circleci.com/docs/2.0/configuration-reference/#resource_class
     resource_class: large
     steps:
-
       - restore_cache:
           keys:
             - source-v1-{{ .Branch }}-{{ .Revision }}
@@ -163,7 +169,7 @@ jobs:
             # Verify that following commands work, they are later called in the incremental testing script
             # There output does not matter here, the fact that they finish successfully does.
             git rev-parse --abbrev-ref HEAD
-            git fetch --all --tags
+            git fetch --all --tags -f
 
       - attach_workspace:
           at: ~/app
@@ -206,7 +212,8 @@ jobs:
 
   general-test:
     <<: *defaults
-    resource_class: large
+    # XXX (soloseng): Increased size because this test was hitting the ram limit and failing
+    resource_class: xlarge
     steps:
       - attach_workspace:
           at: ~/app
@@ -243,6 +250,7 @@ jobs:
   # crashes on the second re-install.
   # This pre script, generates the build and devchain, and adds it to the workspace to be used for the
   # other script
+
   pre-protocol-test-release:
     <<: *contract-defaults
     resource_class: large
@@ -256,6 +264,10 @@ jobs:
       - run:
           name: Opcode tests
           command: yarn --cwd packages/protocol check-opcodes
+      - node/install:
+      # TODO(soloseng): remove this node 12 install once the latest contract release is done on node18.
+          install-yarn: true
+          node-version: '12.22.11'
       - run:
           name: Generate devchain of previous release
           command: |
@@ -270,7 +282,6 @@ jobs:
           root: .
           paths:
             - 'packages/protocol/.tmp/released_chain/*'
-
   protocol-test-release:
     <<: *contract-defaults
     resource_class: large
@@ -739,7 +750,7 @@ jobs:
   test-typescript-npm-package-install:
     working_directory: ~/app
     docker:
-      - image: us.gcr.io/celo-testnet/circleci-node12:1.1.0
+      - image: us.gcr.io/celo-testnet/circleci-node18:1.0.0
     steps:
       - run:
           name: Check if the test should run
@@ -752,7 +763,7 @@ jobs:
   test-utils-npm-package-install:
     working_directory: ~/app
     docker:
-      - image: us.gcr.io/celo-testnet/circleci-node12:1.1.0
+      - image: us.gcr.io/celo-testnet/circleci-node18:1.0.0
     steps:
       - run:
           name: Check if the test should run
@@ -765,7 +776,7 @@ jobs:
   test-contractkit-npm-package-install:
     working_directory: ~/app
     docker:
-      - image: us.gcr.io/celo-testnet/circleci-node12:1.1.0
+      - image: us.gcr.io/celo-testnet/circleci-node18:1.0.0
     steps:
       - run:
           name: Check if the test should run
@@ -781,7 +792,7 @@ jobs:
   test-celocli-npm-package-install:
     working_directory: ~/app
     docker:
-      - image: us.gcr.io/celo-testnet/circleci-node12:1.1.0
+      - image: us.gcr.io/celo-testnet/circleci-node18:1.0.0
     steps:
       - run:
           name: Check if the test should run
@@ -886,17 +897,6 @@ jobs:
               cd packages/protocol
               ./specs/scripts/reserve.sh
             fi
-
-  flakey-test-summary:
-    <<: *defaults
-    steps:
-      - attach_workspace:
-          at: ~/app
-      - run:
-          name: Add summary of flakey tests to GitHub Checks
-          command: |
-            node ./packages/flake-tracker/scripts/summary.js
-  
   # Dummy job for not showing CircleCI error: "Error: All Workflows have been filtered from this Pipeline"
   # Check https://github.com/CircleCI-Public/circleci-cli/issues/577 for more context
   noop:
@@ -1013,24 +1013,6 @@ workflows:
       - odis-test:
           requires:
             - lint-checks
-      - flakey-test-summary:
-          requires:
-            - protocol-test-common
-            - protocol-test-compatibility
-            - protocol-test-governance-network
-            - protocol-test-governance-validators
-            - protocol-test-governance-voting
-            - protocol-test-identity
-            - protocol-test-stability
-            - end-to-end-geth-transfer-test
-            - end-to-end-geth-blockchain-parameters-test
-            - end-to-end-geth-slashing-test
-            - end-to-end-geth-governance-test
-            - end-to-end-geth-replica-test
-            - end-to-end-geth-sync-test
-            - end-to-end-geth-validator-order-test
-            - end-to-end-cip35-eth-compatibility-test
-            - odis-test
   npm-install-testing-cron-workflow:
     # Contitionally triggered
     when: 

.github/CODEOWNERS

@@ -13,7 +13,6 @@
 /packages/docs/ @celo-org/applications @celo-org/devrel
 /packages/env-tests/ @celo-org/applications
 /packages/faucet/ @celo-org/applications
-/packages/flake-tracker/ @celo-org/flake-trackers
 /packages/helm-charts/ @celo-org/devopsre
 /packages/helm-charts/blockscout/ @celo-org/data-services @celo-org/devopsre
 /packages/helm-charts/oracle/ @celo-org/mento  @celo-org/devopsre