-
Notifications
You must be signed in to change notification settings - Fork 0
Headers
Ilya Baldin edited this page Jul 30, 2024
·
5 revisions
EJ-FAT uses gRPC messages and UDP packets for its control plane and UDP only for its data plane. E2SAR has 3 pre-defined headers that go immediately after the UDP header: Sync header for Sync messages sent from the sender to the control plane, LoadBalancer (LB) and Reassembly (RE) headers. The latter can appear together (LB+RE) when sent by the data sender and when the load balancer strips off the LB header needed for it, only the RE header and data proceed. As the name suggests, LB header is needed by the Load Balancer and the RE header is needed by the receiver to reassemble the event data.
IP
IP Source = IP address of the Data Source
IP Destination = Sync IP from ReserveLoadBalancerReply
IP Protocol/NextHeader = UDP
UDP
UDP Destination Port = Specified in EJFAT URI
UDP Source Port = Any
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| L | C | Version | Rsvd |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| EventSrcId |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ EventNumber +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| AvgEventRateHz |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ UnixTimeNano +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
32b Control Word
16b Magic = 0x4C43 ("LC")
8b Version = 1
8b Rsvd, MBZ
32b EventSrcId
Set this to a unique ID for the Event Source that produced this message
64b EventNumber
Set this to the EventNumber that would be used if an Event Data message were to be sent right now
32b AvgEventRateHz
Set this to the best available estimate of how much the EventNumber increases per second
If no estimate is available, set this field to zero
64b UnixTimeNano
Set this to the current wall clock time of this Event Source. Time is in nanoseconds since 1970-01-01T00:00:00Z (ie. midnight UTC)
If no wall clock timestamp is available, set this field to zero
The C/C++ definition looks as follows:
using EventNum_t = u_int64_t;
using UnixTimeNano_t = u_int64_t;
using EventRate_t = u_int32_t;
struct SyncHdr
{
char preamble[2] {'L', 'C'};
u_int8_t version{synchdrVersion};
u_int8_t rsvd{0};
u_int32_t eventSrcId{0};
EventNum_t eventNumber{0LL};
EventRate_t avgEventRateHz{0};
UnixTimeNano_t unixTimeNano{0LL};
} __attribute__((__packed__));
IP
IP Source = Any
IP Destination = LB Hardware Data Plane IPv4 or IPv6
IP Protocol/NextHeader = UDP
UDP
UDP Destination Port = 19522 (0x4C42) (“LB”)
UDP Source Port = Any (typically set to 16 least significant bits of Event Number). It is important to have entropy in this so that packets are spread across LAGs.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| L | B | Version | NextProtocol |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Rsvd | Entropy |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ EventNumber +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
64b LB Control Word
16b Magic = 0x4C42 (“LB”)
8b Version = 2
8b NextProtocol ID
16b Reserved, MBZ
16b Entropy
64b LB Event Number - this can just be a timestamp
Used to partition the event space into Calendar Epochs
Repeats across multiple packets related to the same event
MUST NOT wrap
The C/C++ definition looks as follows:
struct LBHdr
{
char preamble[2] {'L', 'B'};
u_int8_t version{lbhdrVersion};
u_int8_t nextProto{rehdrVersion};
u_int16_t rsvd{0};
u_int16_t entropy{0};
EventNum_t eventNum{0L};
} __attribute__((__packed__));
IP
IP Source = Any
IP Destination = LB Hardware Data Plane IPv4 or IPv6
IP Protocol/NextHeader = UDP
UDP
UDP Destination Port = 19522 (0x4C42) (“LB”)
UDP Source Port = Any (typically set to 16 least significant bits of Event Number). It is important to have entropy in this so that packets are spread across LAGs.
LB Header
Stripped off by load balancer and not seen by the receiver
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| Rsvd | Data-ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Buffer Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Buffer Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ EventNumber +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
4b Version = 1
12b Reserved
16b Data-ID = source identifier
32b Buffer Offset = The byte index of where the data in this packet should be put in the buffer
32b Buffer Length = The size of buffer needed to store this event
64b Event Number = Same as in the event data header but for your application
The C/C++ definition looks like this:
struct REHdr
{
u_int8_t preamble[2] {rehdrVersionNibble, 0}; // 4 bit version + reserved
u_int16_t dataId{0}; // source identifier
u_int32_t bufferOffset{0};
u_int32_t bufferLength{0};
EventNum_t eventNum{0};
} __attribute__((__packed__));