v0.1.47

What's Changed

‼️ V1 Consensus Upgrade ‼️

Iron Fish's first consensus upgrade will activate at block 204,000 estimated to occur on September 28th, 2022. You must upgrade your node by block 204,000 or you will be on a hard fork. The fix was merged in PR #2201.

Why are you doing this?

A bug was discovered and reported by our community that allowed double spends to occur on the chain. The team has done a full analysis on the bug and it has been resolved in PR #2201. We're planning on keeping the current chain at 190k blocks and using this opportunity to test our first ever consensus upgrade.

The double spends that have occurred on the network have not impacted the test net points or leaderboard points.

Other changes

• Converted our RollingBloomFilter implementation to rust for a 10x speed improvement (@mat-if in #2189)
• Fixed critical RPC bug that would crash node if connection timed out while streaming a response (@danield9tqh in #2183)

Full Changelog: v0.1.46...v0.1.47

v0.1.46

What's Changed

Status command additions

When running the status command with --all, you can now see a "Mining" section that displays connected miners, time to fetch transactions from the mempool, and time to construct a block (Added by @NullSoldier in #2145). @danield9tqh also added a display of the time since the expected HEAD block in #2176.

Performance improvements

We're focusing on improving the node's performance in this and upcoming versions. @mat-if in #2160 identified and fixed some memory usage issues, while @dguenther removed some unused verification of new transactions that was wasting CPU time in #2143 and #2161.

Other changes

• Add key ranges to all range based DB ops (@NullSoldier in #2140)
• Normalization cli description (@hairtail in #2130)
• List commands to root (@wd021 in #2098)
• Change balance error to typed error (@NullSoldier in #2101)
• Add a withSegment fn to improve ux of bench utils (@mat-if in #2152)
• Make segment memory non nullable (@NullSoldier in #2156)
• Hide inspect port in use error (@NullSoldier in #2149)
• Make each TcpAdapter test use an OS assigned port (@NullSoldier in #2155)
  Remove duplicate type (@hairtail in #2157)
  Store all test data in the repo (@NullSoldier in #2139)
• Ignore /target dir in build script, minor dockerfile improvements (@mat-if in #2158)
• Move migrations:status and config:show to root (@mirayadav in #2171)
• Show status response after disconnect (@ygao76 in #2172)
• Upgrade @oclif/core to latest (@wd021 and @mirayadav in #2174)
• Add telemetry for RPC response times and success rate (@dguenther in #2177)
• Version bump fix (@NullSoldier in #2179)

Full Changelog: v0.1.45...v0.1.46

v0.1.45

What's Changed

Block broadcasting V2

@dguenther and @danield9tqh in #2045 rewrote our block broadcasting by adding support for broadcasting compact blocks and blocks by hash. Compact blocks send out the transaction hashes along with the block header instead of the full transactions, so receiving nodes can use transactions they've already received to construct the block locally. We expect this to reduce bandwidth used for block propagation, and consequently reduce block latency, as well as allow for updates to how frequently we fetch peers from other nodes.

Ignore transactions if notes have already been spent on chain

@dguenther in #2037 updated our transaction verification to check whether the transaction is spending a note that has already been spent on chain before accepting the transaction into the mempool and rebroadcasting it. We found that over half of transaction broadcasts contained already-spent notes, so we expect this to reduce network traffic and mempool size.

Dynamic fees

@wd021 contributed an initial implementation for dynamic fees based on past blocks, first in the form of a fees command in #2009 , then by updating the pay, deposit, and deposit-all commands to use them in #2051.

Other changes

• Show mempool memory usage in status command (@danield9tqh in #2036)
• Improve public address validation (@mat-if in #2040)
• Add more statuses for transactions in transaction list (@hairtail in #1882)
• Add some additional logging for pool payouts and block template (@mat-if in #2050)
• Cache unsigned hash on Transaction, same way we cache the signed hash (@mat-if in #2054)
• Require pay and deposit commands to connect to a running node (@mat-if in #2058)
• Add accounts status to status command (@hairtail in #1927)
• Enable source maps and stack traces (@NullSoldier in #2084 and #2085)
• Display error when deposit-all crashes (@NullSoldier in #2099)
• Retry 'rm' on chain database to address EBUSY errors in chain:download (@hughy in #2070)
• deposit-all should not crash if balance changes while creating a transaction (@NullSoldier in #2103)
• Add repl command to interact with the node (@NullSoldier in #2110)
• Fix not closing indexer DB (@NullSoldier in #2109)

Full Changelog: v0.1.44...v0.1.45

v0.1.44

What's Changed

Transaction Broadcasting V2

@danield9tqh in #1803 rewrote our transaction broadcasting by adding support for broadcasting transactions by hash in addition to broadcasting full transactions. Transactions are relatively large, so we expect this to generally reduce bandwidth usage. After the same work is done for block broadcasting, it'll allow us to no longer continuously fetch the list of peers from all connected nodes.

chain:prune command

@NullSoldier in #1948 added a chain:prune command to delete forks from the chain and compact the database. It doesn't yet reduce the size of the database as much as we'd like, so we're planning to make further improvements here.

Rust box/unbox message

@mat-if in #1872 moved message encryption from the JavaScript worker pool to Rust. We expect this to reduce the frequency of WebRTC signaling timeouts when a node's worker pool is under load, and will monitor to make sure it doesn't significantly increase load on the main thread.

Other Changes

• Fixed referencing wrong @ironfish/rust-nodejs version in ironfish by @mat-if in #1966
• Fixed a potential missing event unsubscription when downloading snapshots by @NullSoldier in #1974
• Add telemetry for transaction creation by @danield9tqh in #1981
• Attempt to fix createWriteStream error by @NullSoldier in #1980
• Allow for only uploading the accounts DB as a backup by @NullSoldier in #2002
• accounts:transactions: show expiration by default by @hamza-siddiq in #2005
• Remove short-hand -d for --dry in chain:prune and migrations:start by @mat-if in #2007
• Use same logic for verifying single transactions as batches of transactions by @mat-if in #1875
• Add list of routes with pending rpc requests to rpc:status by @hughy in #2006
• Allow submitting telemetry without fields by @danield9tqh in #2001

Full Changelog: v0.1.43...v0.1.44

v0.1.43

What changed

This release includes fixes to the chain:database command. An issue prevented snapshot downloads from resolving and proceeding to snapshot import.

Full changelog: v0.1.42...v0.1.43

v0.1.42

What changed

Snapshots

Service command to create and upload chain snapshots

The service:snapshot command zips the chain database and uploads it to an S3
bucket. A manifest file indicates the block sequence of the snapshot and the
creation time of the snapshot.

CLI command to download a chain snapshot

The chain:download command will download a chain database snapshot and replace
a node's database. This allows nodes that are far behind the head of the chain
to sync more quickly by downloading a pre-synced chain.

> ironfish chain:download
Download 42.57 GB snapshot to update chain head from block 1 to 124125?
Are you sure? (Y)es / (N)o: Y
Downloading snapshot from https://d1kj1bottktsu0.cloudfront.net/ironfish_snapshot_1659661487895.tar.gz
Downloading snapshot: [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] 1% | 428.41 MB / 42.57 GB | 38.06 MB/s | ETA: 17m 31

Migration dry run support

Developers writing database migrations can use the dry run feature to test
migrations without affecting the database.

Disallow importing the same account twice.

Importing an account multiple times and affects node performance by requiring
each note to be decrypted multiple times.

Full changelog: v0.1.41...v0.1.42

v0.1.41

What's Changed

A big change in this release is the database migration system which will be used for many future features. Currently database migrations must either be run manually with ironfish migrations:start or by running ironfish start --upgrade or by setting the databaseMigrate flag in config. This release does not have any long running migrations but in the future migrations may be long running. You WILL need to run ironfish migrations:start or ironfish start --upgrade or set databaseMigrate after upgrading or else your NODE WILL NOT START.

There's also some changes to our transaction gossip. We are using block and transaction hashes to determine when to send to a peer instead of gossip message nonce. This is a step in the direction towards gossiping blocks and transactions by hash which will reduce bandwidth.

We now have the ability to batch verify transactions in Rust which will improve accounts and gossip performance. There's also some code cleanup and helpful CLI improvements. All changes can be seen below

CLI Improvements

• Add nodeName and blockGraffiti to debug command by @drunkensiberia in #1862
• Add nodeName and blockGraffiti to status command by @drunkensiberia in #1863
• Add new renderSpan() to render time to human text by @NullSoldier in #1878
• Display options extend from CliUx.ux.table by @hairtail in #1855
• Enable fullUnicode rendering for blessed by @drunkensiberia in #1876

Batch Verify Transactions

• Upgrade rust dependencies by @mat-if in #1864
• Batch verify transactions by @mat-if in #1867

Gossip

• Replace gossip nonce filter with seen transaction filter by @danield9tqh in #1865
• Create GetCompactBlock request and response messages by @dguenther in #1856
• Add handler for GetCompactBlockRequest by @dguenther in #1859
• Remove filter for gossip nonces by @dguenther in #1890

Database Migrations

• Add Database Migration System by @NullSoldier in #1793

Developer Tools

• Add IRONFISH_EXPOSE_GC to expose garbage collector by @NullSoldier in #1880
• Add tools to bench mark segments of code by @NullSoldier in #1881

Code Cleanup

• Fix file name consistency of RoundRobinQueue by @NullSoldier in #1843
• Remove most of Strategy by @mat-if in #1841
• Remove unused privateIdentity param in IronfishSdk by @mat-if in #1885

Full Changelog: v0.1.40...v0.1.41

v0.1.40

What's Changed

The previous version v0.1.39 used a lot of additional memory because adding known transaction filters to each peer. This release reverts that change and adds a few more changes since the last release.

Full Changes

• Revert 'Replace gossip nonces with peer seen transaction filter' by @danield9tqh in #1851
• Remove TransactionSerde by @mat-if in #1825
• Remove some unnecessary vec allocations by @mat-if in #1830
• Show max heap sizes in debug command by @NullSoldier in #1839
• Change status memory to be relative to max heap by @NullSoldier in #1840

Full Changelog: v0.1.39...v0.1.40

v0.1.39

What's Changed

This is a smaller release. There are some improvements to the pool code, code cleanup (we've deleted rust-wasm). We are also making a change to how transactions are gossiped to see if that improves transaction propagation time and network usage for nodes.

Pool

• Require stratum client version by @NullSoldier in #1784
• FEAT: more miner status info by @hairtail in #1729

Bug Fixes / Code Improvements

• change metrics meter to exponential weighted moving average by @wd021 in #1646
• FEAT: Add expiration to accounts:transaction command by @hairtail in #1801
• Remove wasm code by @mat-if in #1814
• Remove unnecessary type aliases by @mat-if in #1815
• fix(ironfish): Remove submittedSequence check in accounts:transactions by @rohanjadvani in #1813
• Add default trait to memo for better memo ergonomics by @mat-if in #1820

Block and Transaction Gossip

• Create GetBlockTransactions request and response messages by @dguenther in #1788
• Add handler for GetBlockTransactionsRequest by @dguenther in #1795
• Add NewPooledTransactionHashes message by @danield9tqh in #1804
• Refactor Transaction gossip in PeerNetwork by @danield9tqh in #1817
• Fix message type on pooled transaction response by @dguenther in #1822
• Replace gossip nonces with peer seen transaction filter by @danield9tqh in #1829

Full Changelog: v0.1.38...v0.1.39

v0.1.38

What's Changed

We've made some improvements in the mining pool, transaction and block propagation as well as many bug fixes and CLI improvements! Read on for more info.

Mining Pool

We've made some improvements to the mining pool by adding a command to get the status from the mining pool with the command ironfish miners:pools:status -p pool.ironfish.network. This shows things like subscribed count and number of clients connected to the pool

• adds cli command to get pool status by @hughy in #1704
• fixes sql error from missing space by @hughy in #1715
• Materialize subscribed count by @NullSoldier in #1717
• Add clients to status by @NullSoldier in #1718

Transaction + Block Propagation

We've merged a lot of code related to transaction and block propagation improvements. Those improvements won't be seen in this release since the feature is not ready, but look forward to that code being used soon.

• Add new Transaction.hash() method to transactions by @danield9tqh in #1694
• Telemetry for network bandwidth usage per message type by @danield9tqh in #1691
• Upload peer lists to telemetry by @danield9tqh in #1720
• Track known block hashes on Peers by @dguenther in #1683
• Add NewBlockHashes message by @dguenther in #1737
• Replace transaction hashing method in mempool by @danield9tqh in #1740
• Add NewBlockV2 message by @dguenther in #1745
• Create network message PooledTransactionsRequest for querying for transactions based on hash by @danield9tqh in #1747
• Log transaction propagation to telemetry by @danield9tqh in #1734
• Create network message PooledTransactionsResponse for responding to transaction hash query by @danield9tqh in #1752
• Handle PooledTransactionRequest by @danield9tqh in #1756

CLI Improvements

We've made some usability improvements to CLI commands notably adding progress bar to the long running accounts:rescan command and fixing the progress bar for chain:export.

• Always show syncer progress if not synced by @NullSoldier in #1719
• Fix chain:export progress bar by @vchs1v in #1712
• Enable telemetry as part of testnet command by @NullSoldier in #1760
• Minor rescan improvements by @NullSoldier in #1766
• Abort rescan if aborted after waiting for head by @NullSoldier in #1772
• Fix rescan to scan to chain head if null processor by @NullSoldier in #1774
• Improve accounts:rescan CLI command UX by @NullSoldier in #1776

Code Cleanup / Bug Fixes

Some users were seeing Node already in use messages even when a node was not running. That has been changed to surface underlying DB errors that may now show up instead. Also adding a potential fix for noteToNullifiers map error that some users encounter.

• Delete old open api files by @NullSoldier in #1698
• feat(ironfish): Move note decryption when syncing transactions to worker pool by @rohanjadvani in #1708
• Remove .entires() call on iterating a map by @NullSoldier in #1733
• Update README with information about preferred installation methods by @mat-if in #1753
• Update accountsToRemove map in minedBlocksIndexer by @mat-if in #1761
• fix(ironfish): Add mutex for sending transactions by @rohanjadvani in #1762
• Properly catch and surface leveldb corruption errors by @NullSoldier in #1763
• Add Rpc prefix to all Rpc types by @NullSoldier in #1765
• Spelling and Grammar fixes of Markdown files by @hamza-siddiq in #1759
• Clamp telemetry floats to 0 by @NullSoldier in #1769
• Tests for TimeUtils by @vchs1v in #1742
• Use built-in no-unused-expressions lint by @mat-if in #1773
• Make blockchain iteration return buffer hashes by @NullSoldier in #1771

Full Changelog: v0.1.37...v0.1.38