Home
E2SAR (EJFAT Event Segmentation And Reassembly) is a library for streaming event data from science instruments through FPGA-based EJFAT hardware-assisted load-balancers (LBs). It is intended for integration with streaming science applications where the instrument (source of data) is separated from the consumers of event data, data rates are presumed to be high (10s to 100s of Gbps) and no single event consumer is able to consume the entire stream, thus requiring intelligent splitting and load balancing of the event stream across multiple consumers.
Architectural invariants:
- A single event is assumed to always be processed by a single consumer
- Event numbers are unique within a given time period of reasonable length (i.e. don't roll over often)
- LB address(es) is known ahead of time. Switching between LBs in case of failure (LB or network in-between) is left to the application
- ...
E2SAR defines two types of headers - LB and event metadata. The LB header serves to provide data to the LB, while event metadata serves to reassemble events at the consumer.
E2SAR's basic job is to
- Fragment event data (presumed to be large) into UDP packets with LB and event metadata headers prepended
- Efficiently stream fragments to the LB (send side)
- Efficiently receive portions of the stream intended for the specific consumer, reassemble events and forward them to processing or storage (receive side)
- Communicate with the Control Plane (CP) of the LB to indicate consumer queue occupancy back-pressure information to inform LB balancing decisions
- Allow introspection into its performance to help debug issues
The figure below shows two possible envisioned ways to integrate E2SAR into an application:
- Via direct inclusion of the library into the application code on both sides
- Bump-in-Wire (BIW) where the application send and receive ends communicate with a proxy built around E2SAR
- Hybrid deployments are also possible (where a sender is the hardware DAQ that already produces packets with correct LB and metadata headers)
- In this case some performance options may not be available, like e.g. selective retransmit
- A set of well-maintaned software artifacts resuable in different contexts
- Flexibility of development and deployment as well as interfacing to existing systems
- Well-designed abstractions for sending/receiving/performance introspection
- Integration of multiple optimization options (sender and receiver)
- Integration of multiple resilience options (Reed-Solomon, selective retransmit)
- Hide the details of CP interactions and other complexities from the application developer
- APIs consumable in other programming languages (Python)