Skip to content

Commit

Permalink
[receiver/filelog] Add documentation about contents of the offset tra…
Browse files Browse the repository at this point in the history
…cking file (open-telemetry#31886)

**Description:** 

This PR adds documentation explaining how to interpret the contents of
the `filelog` receiver's offset tracking file, configured using the
`storage` extension.

**Link to tracking Issue:** open-telemetry#30970

---------

Co-authored-by: Tiffany Hrabusa <[email protected]>
Co-authored-by: Chris Mark <[email protected]>
Co-authored-by: Daniel Jaglowski <[email protected]>
  • Loading branch information
4 people committed Apr 4, 2024
1 parent 30423cc commit 1e51071
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion receiver/filelogreceiver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,17 @@ Exception in thread 2 "main" java.lang.NullPointerException
## Offset tracking
`storage` setting allows to define the proper storage extension to be used for storing file offsets.
The `storage` setting allows you to define the proper storage extension for storing file offsets.
While the storage parameter can ensure that log files are consumed accurately, it is possible that
logs are dropped while moving downstream through other components in the collector.
For additional resiliency, see [Fault tolerant log collection example](../../examples/fault-tolerant-logs-collection/README.md)
Here is some of the information the `filelog` receiver stores:
- The number of files it is currently tracking (`knownFiles`).
- For each file being tracked:
- The [fingerprint](../../pkg/stanza/fileconsumer/design.md#fingerprints) of the file (`Fingerprint.first_bytes`).
- The byte offset from the start of the file, indicating the position in the file from where the
`filelog` receiver continues reading the file (`Offset`).
- An arbitrary set of file attributes, such as the name of the file (`FileAttributes`).
Exactly how this information is serialized depends on the type of storage being used.

0 comments on commit 1e51071

Please sign in to comment.