All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog.
- cleanup of print and log statements
- removed ingest to-csv, replacement is export to delta lake, which is more efficient
- renamed delta lake-commands ingest dump-rawdata -> ingest delta-lake ingest; ingest optimize deltalake -> ingest delta-lake optimize
- removed fs-cache helper for trx and eth delta-update, now uses delta lake directly
- removed typechecked dependency, removed disk-cache dependency
- tron delta-dump: fix missing transferto_address in some tron traces
- tron delta-dump freezes on grpc asyncio requests
- safer handling of ctrl-c on delta-dumps
- increase timeout limit for s3 requests from the default 30s to 300s
- Allow optimizing single delta table
- Add timeout for grpc calls to fix freezing of trx ingest
- limit compaction parallelism delta lake
- evaluating tables to fix in optimize deltalake step, remove direct boto3 dep.
- passing s3 credentials to boto3
- Write raw data to delta tables on s3 or local using graphsense-cli dump-rawdata
- graphsense-cli optimize-deltalake to optimize tables of a currency (vacuum and/or compact)
- Fixing release tag issue
- numpy 2.0.0 problem (numpy.dtype size changed error)
- cryptocompare exchange rates to have a free version again (graphsense-cli exchange-rates cryptocompare)
- graphsense-cli trace event to print prettyfied event logs for tron and eth
- fixed coingecko z-cash currency key to fetch exchange rates
- coinmarketcap allow configuration of api key for pro api (free is not available anymore)
- graphsense-cli exchange-rates coingecko to allow fetching exchange rates via coingecko pro api
- csv export with new version of ethereum etl 2.4
- performance problem (timeouts) on fetching transactions per block for utxo currencies.
- tron delta update: missing tx_hash for traces in deployment txs.
- Changed number of backoff blocks used in ingestion to avoid spurious data (mostly lowered)
- gracefully handle inconsistencies in address relations
- Warning instead of exception on ingest filelock timeout
- full delta updates for tron and ethereum
- block tx table to long format instead of Cassandra lists (breaking)
- off by one error in utxo delta updates
- change consistency level Cassandra, consistency_level=LOCAL_QUORUM, serial_consistency_level=LOCAL_SERIAL
- address.first_tx_id and last_tx_id should be long type
- ingest now works stores additional details/tables in raw keyspace (tx_type, fees)
- new field for address table, zero value tx stats (eth and trx)
- graphsense-cli config get --path function to access config values for scripting
- more robust retry handling on ingest
- minor bug with system.exit handling and slack notifications
- timestamp micro instead of milliseconds bug trx transactions
- new pk for summary stats to avoid duplicate entries. Breaking: needs recreation of table
- ingest default config to raw keyspace on create to avoid problems.
- handle error missing quotes field on coinmarketcap exchange rates ingest
- handle zcash shielded inputs in import
- added flag forward-fill-rates to allow transform even if no current rates are available (last rate avail is used)
- fix performance degradation on because of inefficient config lookups
- (critical) delta update only inserts coinbase txs
- setup automatic pypi publish with github actions
- ingest/delta update test script to setup a fully functional Cassandra instance for development (script/dev-ingest.sh)
- fixed bug on empty output list on coinbase txs.
- delta updater support for pseudo coinbase address
- inconsistent db state after write timeout -> added retry logic for delta updater on write timeouts
- ingest for utxo now creates new tables for transaction references
- added cli ingest command (ingest from-node) for ethereum-like currencies #6
- added cli ingest command to export node data to csv
- added cli ingest commands (ingest from-node) for btc-like currencies #4
- add ingest to parquet files as ingest output option, additional to cassandra #2
- alpha support for transaction-monitoring #4
- compatibility with tron data in raw keyspaces #3
- delta updater bug with zero value and zero fee txs in btc
- added cli command graphsense-cli db logs get-decodeable-logs to decoded logs in a given block range.
- added all event definitions to decode all USDT event logs
- added keyspace name to monitoring output
- slack notifications and cli notify endpoint
- exception notification via slack
- bash completion file generation
- enable specifying a config file (allowing mulitple configs)
- initial support for decoding eth logs
- functions to efficiently find the closest block to a given date and vice versa
- delta updater fixed skipped blocks
- error when data is up to date
- getting highest block with exchange rates
- Delta updater v2 for utxo currencies
- Config flag to disable delta updater
- Simple monitoring of database state
- Colorized output
- More readable logger format
- Changed schema files to reflect the current version of the graphsense db
- Initial release