v1.10.2

Tiny fix over v1.10.1:

See https://github.com/streamingfast/substreams/releases/tag/v1.10.1 for the much bigger changelog !

• substreams gui: fix panic in some conditions when streaming from block 0

v1.10.1

Server

Note Since a bug that affected substreams with "skipping blocks" was corrected in this release, any previously produced substreams cache should be considered as possibly corrupted and be eventually replaced

• Fix handling of modules that receive both filtered AND unfiltered data as their inputs -> some "repeated entries" could appear where no data should have showed up
• Fix stalling on substreams with both map and store with different initialBlocks on the same stage
• Fix: prevent execution of modules that should be skipped when running live or dev mode (different outputs than when running in batch mode on tier2)

Client

• substreams gui fixed a panic occuring if the given package path doesn't exist
• substreams init must now be called from within your project folder (it no longer downloads file in a subdirectory)
• (since v1.10.0) substreams gui no longer accepts "output_module" as a single argument. It either receives nothing, the package, or the package followed by the output_module

v1.10.0

Server

• Add sf.substreams.rpc.v2.EndpointInfo/Info endpoint (if the infoserver is given as a module, i.e. from firehose-core)
• Add an execution timeout of 3 minutes per block by default (can be overriden in tier1/tier2 Configs) -- this is useful when an external (eth_call) is stuck on a forked block hash.
• Revert 'initialBlocks' changes from v1.9.1 because a 'changing module hash' causes more trouble.
• Wazero: bump v1.8.0 and activate caching of precompiled wasm modules in /tmp/wazero to decrease compilation time
• Metering update: more detailed metering with addition of new metrics (live_uncompressed_read_bytes, live_uncompressed_read_forked_bytes, file_uncompressed_read_bytes, file_uncompressed_read_forked_bytes, file_compressed_read_forked_bytes, file_compressed_read_bytes, file_uncompressed_write_bytes, file_compressed_write_bytes). DEPRECATION WARNING: bytes_read and bytes_written metrics will be removed in the future, please use the new metrics for metering instead.
• Manifest reader: increase timeout of remote spkg fetch to 5 minutes, up from 30 seconds

Client

• Add substreams auth command, to authenticate via thegraph.market and to get a dev API Key.
• Rename --discovery-endpoint into codegen-endpoint in substreams init command.
• Add substreams codegen subgraph command that takes a substreams module and an spkg and that generates a simple subgraph from the module output.
• On substreams init command, if flag --state-file is provided, the state file is used by default for project generation.
• In substreams init command, the state file is named using a Date format and not using Unix anymore.
• Tools->prometheus: added the possibility to override the start-block on an endpoint

v1.9.3

• Fixed error handling issue in 'backprocessing' causing high CPU usage in tier1 servers
• Fixed handling of packages referenced by ipfs:// URL (now simply using /api/v0/cat?arg=...)
• Added --used-modules-only flag to substreams info to only show modules that are in execution tree for the given output_module

v1.9.2

Added

• Added support for directly reading spkg file that is compressed with zstd (from http, gs, s3, azure or local)

Fixed

• Prevent Noop handler from sending outputs with 'Stalled' step in cursor (which breaks substreams-sink-kv)

v1.9.1

Fixed

Fixed substreams hanging in production-mode on chains with a 'first-streamable-block' higher than 0:

• all initialBlocks will be 'bumped' to the first-streamable-block if it is higher
• this will affect the module hashes: use substreams info --first-streamable-block=<block_num> to see how a value will affect your modules
• modules with initialBlocks higher than the first-streamable-block of a chain will be unaffected.

v1.9.0

Important BUG FIX

• Fix a bug introduced in v1.6.0 that could result in corrupted store "state" file if all
  the "outputs" were already cached for a module in a given segment (rare occurence)
• We recommend clearing your substreams cache after this upgrade and re-processing or
  validating your data if you use stores.

Fixed

• substreams 'tools decode state' now correctly prints the kvops when pointing to store output files

Added

• Expose a new intrinsic to modules: skip_empty_output, which causes the module output to be skipped if it has zero bytes. (Watch out, a protobuf object with all its default values will have zero bytes)
• Improve schedule order (faster time to first block) for substreams with multiple stages when starting mid-chain

v1.8.2

• substreams init (code generation): fix displaying of saved path in filenames

v1.8.1

• Add NoopMode enabling to avoid live processing, while flag noop-mode is set.

v1.8.0

Remote Code Generation

The substreams init command now fetches a list of available 'code generators' to "https://codegen.substreams.dev".
Upon selection of a code generator, it launches an interactive session to gather the information necessary to build your substreams.
This allows flexibility and getting anything from "skeleton" of a substreams for a given chain up to a fully built .spkg file with subgraph bindings.

Added

• Add 'compressed' boolean field to the 'incoming request' log
• Add a substreams live back filler, so a request running close to HEAD in production-mode on tier1 will trigger jobs on tier2 when boundaries are passed by final blocks, backfilling the cache. These jobs will be "unmetered".

Fixed

• Fixed Substreams tier1 active worker request metrics that was not decrementing correctly.
• Truncate error messages log lines to 18k characters to prevent them from disappearing through some load balancers.

Removed

• Removed local ethereum code generation from init command.