[CI] Unify pipeline files into one, change harbor url #210

Signed-off-by: Alexander Grissik <[email protected]>

At most a single element of this vector is always used. Once rfs constructor is complete there must be exactly one attach_flow_data element in case of ring_simple. For ring_tap this element remains null. Signed-off-by: Alexander Grissik <[email protected]>

Signed-off-by: Alexander Grissik <[email protected]>

Signed-off-by: Alex Briskin <[email protected]>

Set ETIMEDOUT errno and return -1 from recv in case a socket was timed out, instead of 0 return value and 0 errno. For instance, in case of TCP keep alive timeout. Signed-off-by: Alexander Grissik <[email protected]>

The idea is to scan all rpm/deb packages for personal emails we should not be releasing packages with such emails the scan is done on both the metadat info and the changelog of a specific package Issue: HPCINFRA-919 Signed-off-by: Daniel Pressler <[email protected]>

- Enable LTO by default - Add support for PGO - Switch orig_os_api with direct syscalls Signed-off-by: Alex Briskin <[email protected]>

Signed-off-by: Alex Briskin <[email protected]>

PRM allows minimum of 512 RX strides per WQE. Adjust minimum value for NGINX master process from 32 to 512. Using also minimum stride size allowed - 64 instead of 512. Signed-off-by: Iftah Levi <[email protected]>

If all unacked queue is acked - update last_unacked to null. Signed-off-by: Iftah Levi <[email protected]>

In case of pcb->is_last_seg_dropped - we might end up with wrong pcb->last_unsent. Signed-off-by: Iftah Levi <[email protected]>

Update last_unacked to null after RTO. In this case we move all unacked queue to unsent. Signed-off-by: Iftah Levi <[email protected]>

In tcp_rexmit we take the first segment from unacked and put it in unsent. In case its the only segment in unacked, we need to update last_unacked. Signed-off-by: Iftah Levi <[email protected]>

Remove iterating over unsent / unacked to find last in lists. Signed-off-by: Iftah Levi <[email protected]>

Checking for last_unsent and unsent for NULL is the same. Signed-off-by: Iftah Levi <[email protected]>

This is leftover from the Multi Packet RQ removal. In current code, this option doesn't have effect and only wastes 96 bytes per socket. Signed-off-by: Dmytro Podgornyi <[email protected]>

Remove multiple fields from the ring_allocation_logic class: - m_active is redundant. - m_type / m_owner are needed only for debug logs. 16 bytes per object is a high cost taking into account that ring migration is disabled by default. Also compact the structure. This patch saves 64 bytes per socket (sockinfo and dst_entry). Signed-off-by: Dmytro Podgornyi <[email protected]>

Ring migration is disabled in most scenarios, so improve is_logic_support_migration() - check whether migration is disabled first. Signed-off-by: Dmytro Podgornyi <[email protected]>

Add to_str() which returns pointer to "this". Also, print the type in constructor, so each further debug message can be identified by the pointer. Signed-off-by: Dmytro Podgornyi <[email protected]>

Fields m_n_pbufs_rcvd and m_n_pbufs_freed are unused. Signed-off-by: Dmytro Podgornyi <[email protected]>

Jira task: HPCINFRA-1430 Signed-off-by: Viacheslav Login <[email protected]>

Signed-off-by: Alex Briskin <[email protected]>

Add tcp_tx_express to tcp_sockinfo_ops_tls::tx. I was unable to perform any refactoring, since: - tx_arg is the method argument - tls_arg is initialized at the beginning of the method and used in the nested loops. - tls_record class needs constant reassurance whether it is zerocopy or not. - tls_record class needs constant reassurance whether it is TLS1.2 or TLS1.3 version. - The blocking, non-blocking, zerocopy, non-zerocopy, TLS1.2., and TLS1.2 parts are tightly-coupled and no clean separation is possible without rewriting the code. - Need to allow blocking on small sndbuf in TLS Signed-off-by: Alex Briskin <[email protected]>

Signed-off-by: Alex Briskin <[email protected]>

- Refactor cannot_do_requested_partial_write - Extract static function is_invalid_iovec - Extract method is_connected_and_ready_to_send - Extract method tcp_tx_slow_path - Extract method tcp_tx_handle_errno_and_unlock - Extract method tcp_tx_handle_done_and_unlock - Extract method tcp_tx_handle_partial_send_and_unlock - Check ready-to-send socket in express path Signed-off-by: Alex Briskin <[email protected]>

- Remove oversize from pbuf_prealloc_express - Extract tcp_tx_handle_sndbuf_unavailable method - Promote pbuf.len checks to u32_t - Combine express_mkey and mkey into mkey - Remove pbuf_clen Signed-off-by: Alex Briskin <[email protected]>

Signed-off-by: Alex Briskin <[email protected]>

Signed-off-by: Ben Walker <[email protected]> Signed-off-by: Alex Briskin <[email protected]>

Signed-off-by: Alex Briskin <[email protected]>

The deleted define returned always true because it the address of orig_os_api._func instead of checking it's value. Signed-off-by: Alex Briskin <[email protected]>

Signed-off-by: Gal Noam <[email protected]>

Enum with underlying type is C++ or C23 feature. Move the enum definition to an internal header, because currently, it's unused in public API. Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Gal Noam <[email protected]>

The false positive warning is generated during linking phase. Cast array size to 32bit type to suppress the warning. The warning appears in --enable-lto configuration likely due to inlined new[] operator and generated to the underlying malloc() call. Signed-off-by: Dmytro Podgornyi <[email protected]>

Accessing completion event channel for dummy RQ, that was created automatically by ibv when we create QP. Signed-off-by: Iftah Levi <[email protected]>

XLIO receives any incoming TCP payload in pbuf chains less than 64k. There is no point to do the splits on each receive. Sockinfo_tcp has no 64k limitation and this avoid extra operations. To clean the code and achieve possible minor performance improvement, remove pbuf_split_64k() and accept entire pbuf chain as is. Signed-off-by: Dmytro Podgornyi <[email protected]>

refused_data was used to keep TCP payload which couldn't be received by sockinfo_tcp object and retry receive operation later. Only a single receive operation could be postponed in such a way and all the subsequent failures led to packet drops. sockinfo_tcp::rx_lwip_cb() doesn't produce an error by itself. All the possible errors are propagated by L3_tcp_input(). And all the errors lead to connection reset. So, there is no point to keep refused_data in current implementation. Remove refused_data to clean the code and save extra cache line read in fast TCP timer. Signed-off-by: Dmytro Podgornyi <[email protected]>

Split poll_sn to poll_sn_rx and poll_sn_tx. Required after splitting cq to cq_rx and cq_tx. Signed-off-by: Iftah Levi <[email protected]>

Panic level of logs terminate the process and don't return. Call std::terminate() to terminate the process unconditionally. Also, this fixes cppcheck warning that an exception is thrown without an object. Signed-off-by: Dmytro Podgornyi <[email protected]>

IF_VERBS_FAILURE if supposed to be used with ibverbs API as a workaround for inconsistent API. However, using it for an internal method make the code dirty. Replace 'throw' with 'throw_xlio_exception' to suppress a cppcheck warning. Signed-off-by: Dmytro Podgornyi <[email protected]>

cppcheck generates a warning about uninitialized fields even if they're not used in fact. Signed-off-by: Dmytro Podgornyi <[email protected]>

This field is used as a local variable in 3 places. Allocate a variable on stack in the respective places instead of keeping it per socket. This fixes cppcheck warning about lifecycles of local and non-local variables. Signed-off-by: Dmytro Podgornyi <[email protected]>

cppcheck complains about 'int' type overflow in 0x3 << 30. Fix this with explicit unsigned type. Signed-off-by: Dmytro Podgornyi <[email protected]>

Debug code is disabled by default in ip_frag and has become broken. It assumes that mem_buf_desc_t::n_ref_count has a non atomic type. Signed-off-by: Dmytro Podgornyi <[email protected]>

Use <> for the system headers. This suppresses "information" level warning of cppcheck. Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Alex Briskin <[email protected]>

It's not thread_local specific anymore. This event handler will also be used within poll_group to provide TCP timers for the respective sockets. Signed-off-by: Dmytro Podgornyi <[email protected]>

After the 1st call to xlio_get_socket_rings_fds(), XLIO remembers the resulting array within the socket object. However, subsequent calls return the array immediately without setting number of elements in the array. As result, the function returns 0 rings by mistake. Always set the number of elements to resolve the issue. Signed-off-by: Dmytro Podgornyi <[email protected]>

xlio_get_socket_rings_fds() and xlio_get_socket_rings_num() return the TX ring now. This can be useful in case of TX ring != RX ring when extra API is used. Also don't remember resulting array and construct it on the fly. Usually, this API is called only once for a socket, so remembering the array is a waste of memory. Remove the check_rings() and hardcoded number of rings for SockeXtreme flow. Return actual rings to make more precise result. Signed-off-by: Dmytro Podgornyi <[email protected]>

XLIO_TCP_ABORT_ON_CLOSE option forces XLIO to send RST instead of FIN handshake. In this case, XLIO reset each connection twice. Fix this and check only once whether connection needs to send RST in prepare_to_close(). Signed-off-by: Dmytro Podgornyi <[email protected]>

Once TCP_NODELAY option is set, XLIO disables Nagle's algorithm and sets TCP send buffer to a hardcoded value. This applies to setsockopt() and XLIO_TCP_NODELAY configuration. Avoid changing the send buffer and continues using either SO_SNDBUF or XLIO_TCP_SEND_BUFFER_SIZE value. Also, remove duplicated send buffer size from sockinfo_tcp object which was used only by getsockopt(). Signed-off-by: Dmytro Podgornyi <[email protected]>

The tests fail to receive notifications via the error queue and disabled in CI as result. However, running the tests manually is inconvenient, because of the known failing test cases. Disable the failing tests in the code instead of CI script. Signed-off-by: Dmytro Podgornyi <[email protected]>

XLIO_ZC_TX_SIZE limited zerocopy segment size to the specified 16bit value. By default, this was 32KB. However, pbuf len is not limited to 16bit anymore and TCP segment has TSO limitation instead. Remove the option and use TSO capability to limit zerocopy TCP segments. Signed-off-by: Dmytro Podgornyi <[email protected]>

tcp_pcb::max_send_sge is a copy of tcp_pcb::tso.max_send_sge, but reduced by 1. It was used in zerocopy POC implementation, because zerocopy segments were TSO segments with the TCP header in the 1st sge. Current implementation uses tcp_seg::l2_l3_tcphdr_zc[] to hold the header and doesn't require the reduced limit. Signed-off-by: Dmytro Podgornyi <[email protected]>

tcp_write_express() is a zerocopy version of tcp_write() and not limited by the sndbuf or queue length. Therefore, it's unlikely to fail. However, it still can fail to allocate pbuf or TCP segment in theory. Move the loop over iovec from higher level methods to tcp_write_express(), so we keep consistent TCP state if the memory allocation error happens in the middle of a complex send operation. Additionally, improve tcp_write_express() robustness and don't allow to append data to last segment if it's a retransmit. Signed-off-by: Dmytro Podgornyi <[email protected]>

Condition is_connected_and_ready_to_send() does RX polling in case of TCP_CONN_CONNECTING state. The idea is to speedup the connected state, however, this has few drawbacks: - This is unlikely path, because there are mechanisms to async connect() to finish; - RX polling requires to be unlocked, so there is a race window between TX thread and concurrent RX which may change the state; - The RX polling doesn't guarantee that there is the last handshake packet in the RQ. Remove the polling, so checking for connected state is more lightweight and allowed under the socket lock. Signed-off-by: Dmytro Podgornyi <[email protected]>

Replace u16_t with u32_t to avoid type overflow. Signed-off-by: Dmytro Podgornyi <[email protected]>

pcb::snd_buf can become negative in some flows. Signed-off-by: Dmytro Podgornyi <[email protected]>

Higher level logic checks for sndbuf, so no need to check it again in tcp_write(). Also account SYN/FIN segments in snd_buf. Previously, this code was disabled, because of the corner case when snd_buf is 0. But now it can become negative. Signed-off-by: Dmytro Podgornyi <[email protected]>

This field was used to hold mapping_t pointer, but the mapping is a mem_desc object now. Remove the leftover. Signed-off-by: Dmytro Podgornyi <[email protected]>

This is performance oriented and event based API for TCP sockets. It introduces a polling group (xlio_poll_group_t) which is a collection of sockets (xlio_socket_t) and a serialization context. Signed-off-by: Dmytro Podgornyi <[email protected]>

Remove stub initialization functions to clean the code. Signed-off-by: Dmytro Podgornyi <[email protected]>

Structure pbuf_custom doesn't serve its original purpose anymore and contains a single field pbuf. Remove the structure and use pbuf directly. Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Gal Noam <[email protected]>

Signed-off-by: Viacheslav Login <[email protected]>

There is no path to create pipeinfo objects. Signed-off-by: Alexander Grissik <[email protected]>

To simplify inheritance and make socket objects cache friendly, we avoid inheriting cleanable_obj which inserts only one boolean but trashes the sapce after vptr. Signed-off-by: Alexander Grissik <[email protected]>

The class pkt_sndr_source is unused empty interface. This adds undesired vptr to inheriting classes and trashes the cache. Signed-off-by: Alexander Grissik <[email protected]>

The only consumer of pkt_rcvr_source is sockinfo hierarchy objects. Removing this abstraction allows cache friendly layout of sockinfo. Signed-off-by: Alexander Grissik <[email protected]>

Removing inheritance of timer_handler from sockinfo_tcp allows o order fields in cache friendly manner. timer_hander add vptr ad related fields in an undesired location in the object layout. sockinfo_tcp is always handled by tcp_timer_collection which itslef is timer_handler. Thus, there is no real need in registering the socket itself into the internal thread. This removes complex code with handling groups inside timers registration which are used only for tcp_collection. For XLIO_TCP_CTL_THREAD=with_wakeup/no_wakeup we put the listen socket events to be handled as part of tcp_timer_collections instead of being registered directly on internal thread. Signed-off-by: Alexander Grissik <[email protected]>

Having sockinfo inheriting from wakeup_pipe inserts wakeup_pipe members in an undesired location in term of cache utilization for data path. The same functionallity can be achieved by having the wakeup_pipe as a member. Signed-off-by: Alexander Grissik <[email protected]>

Before merging socket_fd_api with sockinfo, replace all access through socket_fd_api with sockinfo. Signed-off-by: Alexander Grissik <[email protected]>

Removing redundant base from sockinfo allows to put data path members near the vptr for better cache utilization. Signed-off-by: Alexander Grissik <[email protected]>

Having inline methods with long implementation outside the class makes the class definition more readable. Signed-off-by: Alexander Grissik <[email protected]>

Reordering sockinfo methods for more readable definition. Signed-off-by: Alexander Grissik <[email protected]>

Moving socket stats to be a global resource instead of inside of each socket. This reduces socket size by +500B and allows to disable stats for sockets or make few sockets to use stats. Socket stats are generally unneeded and limited in monitoring to 1024 anyway. In cases of 1K+ sockets these stats are overhead. This also allows to disable stats for sockets and improve cache misses. Signed-off-by: Alexander Grissik <[email protected]>

Reordering the sockinfo fields from random locations to group data path frequently accessed fields together. Placing frequently fields at the beggining of the object allows these members to be fetched with the cache line that fetches the vptr. Signed-off-by: Alexander Grissik <[email protected]>

The check for m_flow_tag_enabled is redundant as we already got flow-tag packet and know the socket. In worst case the flow_tag_id can be checked. Avoid unneeded memory access for critical data path. Signed-off-by: Alexander Grissik <[email protected]>

This support was added long ago. However, there are no more users for this option. Signed-off-by: Alexander Grissik <[email protected]>

Checking if timestamps are needed per each packet consumes more cpu cycles than checking once and then running another loop is better for most cases when timestamps are not required. wq Signed-off-by: Alexander Grissik <[email protected]>

Precaching constant sysvar inside socket enlarges the socket and requires more cache lines. Signed-off-by: Alexander Grissik <[email protected]>

Socketxtrme does not need to wakeup sockets by using m_sock_wakeup_pipe. This mechanism is used when a seperate thread sleeps on a blocking socket. Signed-off-by: Alexander Grissik <[email protected]>

This mechanism is used for poll/select only. Socketxtrme has its own polling mechanism. Accesing and checking this member/method is cycles/cache overhead. Signed-off-by: Alexander Grissik <[email protected]>

Cache optimization for data path. Signed-off-by: Alexander Grissik <[email protected]>

It is enough to hold a thread_local dummy lock instead of allocating a seperate object for each socket. Dummy lock contains only empty methods. Signed-off-by: Alexander Grissik <[email protected]>

TCP 5T flow-tag data path always contains src/dst addresses in m_connected/m_bound memebers. Signed-off-by: Alexander Grissik <[email protected]>

There is no bug, this fix is aimed at suppressing the optimizer errors. Signed-off-by: Alex Briskin <[email protected]>

A TCP time-wait used socket can call register_timer although it is already registered. This made the socket to be added more than once to the timers list. Adding a check to verify that the socket is not already in the list. Signed-off-by: Alexander Grissik <[email protected]>

A TCP reused socket calls socket_stats_init. However, the check inside socket_stats_init was for m_p_socket_stats which may point to sock_stats::t_dummy_stats and thus pass the condition and init the sockets as one that has stats. Further a pointer to dummy is returned to stats vector as regular stats object. Checking for m_has_stats instead of for m_p_socket_stats solves the issue. Since, m_has_stats initialized to true only if a real stats object is returned by sock_stats. This stats object will be held by the socket untill its destruction. A reused timewait socket that has stats will just reset its stats as part of socket_stats_init. Signed-off-by: Alexander Grissik <[email protected]>

XLIO Socket API introduced memory callback which provides information about internal memory allocation for RX/TX buffers. The callback also notifies about hugepage size and guarantees hugepage boundary alignment if hugepages are used. This makes XLIO hugepage configuration is more critical to user if their logic depends on hugepage properties. XLIO_HUGEPAGE_LOG2 is not user friendly, so replace it with XLIO_HUGEPAGE_SIZE configuration which accepts user readable format of the hugepage size. For example, XLIO_HUGEPAGE_SIZE=2MB. XLIO checks whether the requested hugepage size is supported by the system. But this check is moved to hugepage_mgr constructor, because g_hugepage_mgr is not initialized during reading XLIO configuration. Signed-off-by: Dmytro Podgornyi <[email protected]>

XLIO socket API doesn't support crypto offload on this stage. Signed-off-by: Dmytro Podgornyi <[email protected]>

xlio.h and xlio_extra.h provides public API for different scenarios: explicit linkage and LD_PRELOAD respectively. Therefore, usually, it makes sense to use a single header. However, the headers have common types definitions. Move them to a separate xlio_types.h instead of including xlio_extra.h from xlio.h. Signed-off-by: Dmytro Podgornyi <[email protected]>

Allow user to take responsibility for XLIO buffers allocation. This can be useful if RX buffers needs to be placed to a special memory for further handling. Signed-off-by: Dmytro Podgornyi <[email protected]>

This allows to use XLIO Socket API with LD_PRELOAD approach via pointers. Signed-off-by: Dmytro Podgornyi <[email protected]>

While lock_spin requires 16 bytes, the lock_spin_simple requires 4 bytes. This allows squashing data path members into less cache lines. Signed-off-by: Alexander Grissik <[email protected]>

Adding a class to act as a pool of different objects. Other objects that use this pool are introduced next. Signed-off-by: Alexander Grissik <[email protected]>

Introduce ring pool of ring_ec objects which are used for Socketxtreme completion. Using such a pool decreases significantly the number of ring_ec obejcts needed. This, in turn, improves cache utilization. Signed-off-by: Alexander Grissik <[email protected]>

Compiler defines __BYTE_ORDER__ and others, but XLIO uses wrong names in the macros. Preprocessor replaces undefined names with 0, so XLIO always builds little endian branch. Fix the byte order names and fail a build if __BYTE_ORDER__ is not defined. Otherwise, the build would be broken on a big endian system. Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Gal Noam <[email protected]>

Keep a global collection to destroy remaining groups at exit. Usually, polling groups are pre-initialized in a small number (up to the number of CPU cores) and destroyed at exit. Therefore, the collection doesn't have to be efficient. Moreover, a group destruction is a heavy operation itself, therefore, optimizing the collection doesn't make sense. Signed-off-by: Dmytro Podgornyi <[email protected]>

Keep a sockets list per polling group to close the remaining sockets on a group destruction. Signed-off-by: Dmytro Podgornyi <[email protected]>

To avoid a race and extra destruction operations, take additional reference to each ring within a group. The race can happen if a socket destruction leads to its ring destruction, but user returns RX buffers after that. User may return buffers to a group regardless of socket lifecycle. In a scenario when user closes all the sockets in runtime, the respective ring would be destroyed and created with new socket again. This is expensive operation and the reference avoids the destruction. Current implementation keeps all the rings until net_device_val destructor. In the future, poll_group should release the reference and destroy the ring. Signed-off-by: Dmytro Podgornyi <[email protected]>

Rings are created in context of a net_device_val. Therefore, the net_device_val object must be present for an existing ring. However, API allows to return NULL by the netdev lookup and coverity generates a warning because of that. Throw an exception to suppress the coverity warning. Signed-off-by: Dmytro Podgornyi <[email protected]>

poll_group takes additional reference to each its ring. But it doesn't release it once the group is destroyed. This leads to two issues: 1. Extra resources are utilized if user destroys a polling group before the application terminates. 2. Polling is not possible for a destroyed group. Therefore, if there are not completed WQEs in the SQ, respective sockets won't report TX completions and cannot be fully terminated. The ring needs to be destroyed to flush all the completions. Release all the native rings explicitly in the poll_group destructor to resolve the above issues. Signed-off-by: Dmytro Podgornyi <[email protected]>

When an RX packet event happens, XLIO passes the ownership to user. Further, user releases the buffer explicitly. However, XLIO frees the buffer unconditionally just after emitting the event. Fix this and free buffers only if user doesn't provides the RX event callback. Signed-off-by: Dmytro Podgornyi <[email protected]>

reclaim_recv_single_buffer() accumulates buffers in a list. In the performance oriented API we want to reuse hot buffers immediately, so reclaim_recv_buffers() implementation is more suitable. Signed-off-by: Dmytro Podgornyi <[email protected]>

The memory callback provides hugepage size of the underlying pages. Replace hardcoded 0 with real hugepage size. Keep the page size in xlio_allocator object. This field a relevant only the hugepage allocation method and 0 in all other cases. Signed-off-by: Dmytro Podgornyi <[email protected]>

XLIO Socket API must guarantee that the XLIO_SOCKET_EVENT_TERMINATED is not followed by any other events. Therefore, all the TX completion events must be completed by that moment. Do a polling iteration before calling socket destructor to increase the chance that all the relevant WQEs are completed. This mechanism needs to be improved in the future. Signed-off-by: Dmytro Podgornyi <[email protected]>

xlio_init_ex() changes some default parameters. However, a global object can trigger safe_mce_sys() constructor at the start. Therefore, we need to re-read the environment variables again to guarantee that the changed parameters take place. Signed-off-by: Dmytro Podgornyi <[email protected]>

Avoid using connect() with sock fd interface, because fd_collection doesn't keep xlio_socket_t objects. Signed-off-by: Dmytro Podgornyi <[email protected]>

xlio_socket_t objects aren't connected to the fd_collection anymore. Therefore, all the methods must be called from the sockinfo_tcp objects directly. Also, xlio_socket_fd() is not relevant anymore and can be removed. Signed-off-by: Dmytro Podgornyi <[email protected]>

Iterate over std::list of TCP sockets while erasing socket during iteration. Overcomed by increasing iterator before erase. Signed-off-by: Iftah Levi <[email protected]>

rdma-core limits number of UARs per context to 16 by default. After creating 16 QPs, XLIO receives duplicates of blueflame registers for each subsequent QP. As results, blueflame doorbell method can write WQEs concurrently without serialization and this leads to a data corruption. BlueFlame can make impact on throughput, since copy to the blueflame register is expensive. It can improve latency in some low latency scenarios, however, XLIO targets high traffic/PPS rates. Removing blueflame method also slightly improves performance in some scenarios. BlueFlame can be returned back in the future to improve low-latency scenarios, however, it will need some rework to avoid the data corruption. Signed-off-by: Dmytro Podgornyi <[email protected]>

The inline WQE branch is not likely in most throughput scenarios. Signed-off-by: Dmytro Podgornyi <[email protected]>

Avoid calling register_socket_timer_event when a socket is already registered (TIME-WAIT). Although there is no functionality issue with that, it produces too high rate of posting events for internal-thread. This leads to lock contantion inside internal-thread and degraded performance of HTTP CPS. Signed-off-by: Alexander Grissik <[email protected]>

Signed-off-by: Gal Noam <[email protected]>

UTLS uses tcp_tx_express() for non blocking sockets. However, this TX method doesn't support XLIO_RX_POLL_ON_TX_TCP. Additional RX polling improves scenarios such as WEB servers. Insert RX polling into UTLS TX path to resolve performance degradation. Signed-off-by: Dmytro Podgornyi <[email protected]>

In heavy CPS scenarios a socket may go to TIME-WAIT state and be reused before first TCP timer registration is performed by internal-thread. 1. Setting timer_registered=true while posting the event prevents the second attemp to try and post the event again. 2. Adding sanity check in add_new_timer that verifies that the socket is not already in the timer map. Signed-off-by: Alexander Grissik <[email protected]>

Added new env parameter - XLIO_MAX_TSO_SIZE. It allows the user to control maximum size of TSO, instead of taking the maximum cap by HW. The default size is 256KB (maximum by current HW). Values higher than HW capabilities won't be taken into account. Signed-off-by: Iftah Levi <[email protected]>

Signed-off-by: Gal Noam <[email protected]>

When sock_stats was static its destructor was called before xlio_exit that destroys the internal-thread which destroys sockets. We should avoid having global objects with untrivial constructors/destructors, since there is no control of their execution order. Signed-off-by: Alexander Grissik <[email protected]>

When TCP socket is destroyed it frees the preallocated buffers after dst_entry is deleted. This returns the buffers to the global pool directly and breaks m_tx_num_bufs,m_zc_num_bufs ring counters. 1. Move the preallocated buffers cleanup before dst_entry destruction. 2. Add ring stats for m_tx_num_bufs and m_zc_num_bufs. Signed-off-by: Alexander Grissik <[email protected]>

1. Removing hardcoded check that switches AIM to latency mode. In case of low packet rate the calculation will result in 0 count anyway. In case packet rate is higher than the desired interrupt rate we do want to utilize the AIM correctly. 2. Changing default AIM values to more reasonable. 3. Removing default values for Nginx and use AIM by default. This improves CPU utilization in low congested cases significantly.: Signed-off-by: Alexander Grissik <[email protected]>

These parameters are deprecated and will be removed in the future. Use XLIO_MEMORY_LIMIT instead. Signed-off-by: Dmytro Podgornyi <[email protected]>

MCE_MAX_CQ_POLL_BATCH usage requires it to be small enough. However, this is a logical upper limit and we want be able to raise it if necessary. Remove unused cq_mgr_tx::clean_cq() which uses MCE_MAX_CQ_POLL_BATCH for an array on stack. Adjust condition for RX buffers compensation to remove MCE_MAX_CQ_POLL_BATCH. However, this changes the logic and now, we forcibly compensate only the last RX buffer in RQ. Signed-off-by: Dmytro Podgornyi <[email protected]>

MCE_MAX_CQ_POLL_BATCH is a logical upper limit for CQ polling batch size. There is no hard limitation for it, so raise it to maximum CQ size. This value can even exceed CQ size, because HW continue receiving packets during polling. Be default, this change doesn't have effect unless a higher value for XLIO_CQ_POLL_BATCH_MAX is set explicitly. This can be helpful in a scenario when a high traffic rate stops for a long time and number of packets in an RQ exceeds the batch size. Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Gal Noam <[email protected]>

Valgrind becomes too slow in case checked process uses huge number of memory. So reduce memory usage to make valgrind working. Signed-off-by: Igor Ivanov <[email protected]>

Issue: HPCINFRA-2179 Signed-off-by: Viacheslav Login <[email protected]>

Signed-off-by: Dmytro Podgornyi <[email protected]>

There is a mix of "Enable"/"Enabled" in the default value descriptions. Replace it with "Enabled" everywhere. Signed-off-by: Dmytro Podgornyi <[email protected]>

Replace #error directive with #warning if __BYTE_ORDER__ or __ORDER_..._ENDIAN__ is misconfigured. This resolves an issue with cppcheck when it cannot determine a working configuration and skips most of the files from scanning. The warning still fails the compilation process because of -Werror, however, cppcheck can scan sources now. Note, this is not an XLIO issue. Signed-off-by: Dmytro Podgornyi <[email protected]>

This is unused macro which is supposed to allow developers to change err_t type with the definition. However, err_t definition can be changed directly in the err.h file if needed. Signed-off-by: Dmytro Podgornyi <[email protected]>

This field is used to avoid partial write of logical data blocks like PDUs. However, XLIO supports send buffer "overcommit" now and allows to push more data to the queue than the send buffer allows. Use this property of the send buffer to allow a single NVME PDU to exceed the limitation. This will save extra field per socket and simplify logic. Signed-off-by: Dmytro Podgornyi <[email protected]>

Printing a buffer is an unlikely branch, so no need to inline it. Signed-off-by: Dmytro Podgornyi <[email protected]>

While a socket is locked, it's state cannot be changed and the only scenario to change the state is to call polling or TCP timers. In the TX path we don't call timers and polling is called when the send buffer is full. Therefore, it's enough to check the connection state under the lock once for non-blocking sockets. For blocking sockets the check can be moved after the polling, because the connection state cannot be changed before. Check for g_b_exit is relevant only for blocking operations, so check it just before polling. Signed-off-by: Dmytro Podgornyi <[email protected]>

list_node is always inlined to a class and its offset is known. Therefore, we can find the object pointer with a single arithmetic operation and save 8 bytes per list_node. Remove the list iterator. It's mostly unused and requires special treatment of the head and tail. Replace it with next() and prev(). Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Dmytro Podgornyi <[email protected]>

The ESTABLISHED event was called in a moment where the socket is not in a consistent state. Call the event after socket state transits to the CONNECTED state (not only TCP state). Signed-off-by: Dmytro Podgornyi <[email protected]>

Remove private access specifier to meet requirement of the standard layout type. Signed-off-by: Dmytro Podgornyi <[email protected]>

Converting mem_buf_desc_t to xlio_buf can overwrite critical field pbuf::next, because user may change xlio_buf::userdata. Instead, provide mem_buf_desc_t::ee as xlio_buf, so we reuse an insignificant field in RX path. Signed-off-by: Dmytro Podgornyi <[email protected]>

Add XLIO sockets to the fd_collection array, but bypassing all the internal logic. This allows ring_slave to find socket object for the flow tag feature. Signed-off-by: Dmytro Podgornyi <[email protected]>

rx_lwip_cb_xlio_socket() missed RX statistics and xlio_stats showed zeros as result. Save the stats. Signed-off-by: Dmytro Podgornyi <[email protected]>

The recent xlio_list change introduced a bug into epoll ready list iteration: a socket can be removed from the list before getting the next item. This leads to iteration over a removed item causing endless loop. Remember 'si_next' before removing a socket from the list. Signed-off-by: Dmytro Podgornyi <[email protected]>

List node class has been reduced by 8 bytes and this made the mem_buf_desc_t structure unaligned. Add padding at the end. Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Gal Noam <[email protected]>

Adjust XLIO behavior to generate warnings instead of error messages when it is unable to access FS files. This may occur in containerized environments. Since default values are provided, the severity of the issue is less critical. Change the default log severity in read_file_to_int() from VLOG_ERROR to VLOG_WARNING,and rephrase print message for better clarity. Signed-off-by: Bashar Abdelgafer <[email protected]>

Change proj_jjb.yaml and job_matrix.yaml to fit the new nbu jenkins master, to allow it to run on the new jenkins instance Remove tap archives from job matrix, as it is not in use by anyone and the plugin is not installed on the new Blossom master Removed xlio-perf and libxlio pipelines as they arent being used anymore, only libxlio-opensource is. Issue: HPCINFRA-2278 Signed-off-by: NirWolfer <[email protected]>

Changes made in this PR: Mellanox/ci-demo#90 requires the following changes in this repo Add credentialsId to login into the new gerrit url, skipDockerDaemonCheck configuration provided by the gerrit team. Issue: HPCINFRA-2289 Signed-off-by: NirWolfer <[email protected]>

Modified the default ring allocation logic from ‘ring per interface’ to ‘ring per thread’. Adjusted the MCE_SPEC_NGINX_DPU and MCE_SPEC_NGINX by setting ring_allocation_logic_tx/rx to ‘ring per interface’. Fixed a minor side-effect in the ring_allocation_logic constructor. Signed-off-by: Bashar Abdelgafer <[email protected]>

Upgrade coverity to version 2023.12 Signed-off-by: Bashar Abdelgafer <[email protected]>

Removed redundant if check after ensuring buffer stack initialization Suppressed Coverity null dereference warning (False Positive) Initialized uninitialized class members stats_file and m_ioctl in mce_sys_var constructor Checked return value of gettime() in timer constructor and added error logging Signed-off-by: Bashar Abdelgafer <[email protected]>

- Use move constructors and assignment operators for m_source, ring_allocation_logic, m_ip_addrs, and source_t to replace copy constructors and assignment - Simplify event_data_t initialization with concise constructor calls in event_handler_manager.cpp, causing assigned objects to be temporary rvalue references which invoke move semantics - Add specialized constructors for EV_IBVERBS, EV_RDMA_CM and EV_COMMAND types in event_data_t - Remove use of std::forward in ip_address constructor because addr is used afterward and would be in an unknown state if moved Signed-off-by: Bashar Abdelgafer <[email protected]>

Guard m_error_queue in handle_recv_errqueue with lock when checking if it's empty since it's a shared resource. handle_recv_errqueue is only called in a few cases when the flag MSG_ERRQUEUE is set, so this change will cause no degradation Change iftype_value from char to int in get_iftype_from_ifname since iftype_value, which is returned from the function, is derived from atoi, which returns an int Change proto variable 16-bit positive number type from short to uint16_t, and update all occurrences in all functions Signed-off-by: Bashar Abdelgafer <[email protected]>

Suppress irrelevant and false positive Coveritywarnings in various files using comments Turn off Coverity checks for sum of null dereference overflow, missing lock, assigned pointer, copy assignment call and other identified issues Signed-off-by: Bashar Abdelgafer <[email protected]>

Fixes latency degradation caused by the reduced size of xlio_list. It created a gap in the ring class which is not cache line aligned. Added padding bytes to align it to a full cache line. Signed-off-by: Iftah Levi <[email protected]>

TX polling can trigger a zerocopy completion event. If this happens in context of xlio_poll_group_flush(), a send operation can break consistency between socket dirty status and their group's dirty list. To avoid the issue, don't poll TX outside the xlio_poll_group_poll() context. This will make the zerocopy completion events possible only from the polling context. Signed-off-by: Dmytro Podgornyi <[email protected]>

If a dirty socket is closed without a flush, it must be removed from the group's dirty list. Otherwise, this can lead to a use-after-free issue. Signed-off-by: Dmytro Podgornyi <[email protected]>

TSO statistics are very important for tracking TX performance. Two variables are added, TSO packets number and TSO bytes number. With these two and with conjunction of total TX packets and bytes, the needed information can be derrived. Signed-off-by: Alexander Grissik <[email protected]>

Adding Nginx Constant Reply Module. This module allows to send simple constant replies from Nginx without accessing files or system resources. See module README for more information. Signed-off-by: Alexander Grissik <[email protected]>

This fixes gcc-14 build. Signed-off-by: Dmytro Podgornyi <[email protected]>

Need to add the length of VLAN header to L2 header length in case of VLAN tagged packets, the issue was only in send to neigh flow when neigh is not resolved yet Signed-off-by: Bashar Abdelgafer <[email protected]>

Early logs can be printed before vlogger subsystem initialization. If they are printed to stdout, this can interfere with tools that parse output, such as grep, awk, etc. Gtest calls a subshell command to find a default gw and early logs breaks this functionality. Example of such scenario is a container with prohibited read access to sysfs files which XLIO reads. Signed-off-by: Dmytro Podgornyi <[email protected]>

LibXLIO CI definition under job_matrix.yaml has the credentials of rivermax which do not have permissions to pull docker images needed to build LibXLIO docker images Change harbor credentialsID under matrix_job.yaml to one with permissions over libxlio docker images Signed-off-by: NirWolfer <[email protected]>

Add support for UDP reuse port for listen socket for both IPv4 and IPv6 at the same conf file. Signed-off-by: Iftah Levi <[email protected]>

Base class event_handler_manager closes m_epfd unconditionally in the destructor. But in case of local copy, the constructor exits early and leaves m_epfd initialized as 0. This leads to closing 0 file descriptor during the local copy destruction what causes runtime issues for objects related to the fd 0. Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Alexander Grissik <[email protected]>

Non-zerocopy (XLIO_SOCKET_SEND_FLAG_INLINE) operations reuse common tcp_write() which doesn't allow the queue to grow beyond specific number of pbufs. The limit is derived from the TCP send buffer and usually equals to hundreds or thousands of pbufs. XLIO Socket API encourages users to limit the inflight data and allows send operations to fails with the unrecoverable ENOMEM error. However, on the other hand, the API describes that send operations aren't limited by send buffer logic. To follow current XLIO Socket API description, allow the TCP queue to grow. To do this, increase the queue limit to 16M pbufs for XLIO sockets. Signed-off-by: Dmytro Podgornyi <[email protected]>

- we're switching here to ubuntu22.04-based container providing the support to clang-15 and clang-format-15 Signed-off-by: Viacheslav Login <[email protected]>

Signed-off-by: Dmytro Podgornyi <[email protected]>

Blackduck module report output shows the wrong version, this is due to projectVersion under args being set to a static value Change projectVersion to the branch entered instead of static value Signed-off-by: NirWolfer <[email protected]>

sys_str2addr() converts a string address representation to the sockaddr structure. However, an invalid address sets either AF_INET or AF_INET6 address family and returns corrupted IP portion of the sockaddr object. Return AF_UNSPEC address family in case of an invalid IP address. So, default gateway detection can handle address parsing. Invalid port portion is still silently ignored. In the future, the function should return the conversion status and this needs to be handled explicitly by the function users. Signed-off-by: Dmytro Podgornyi <[email protected]>

sys_gateway() converts string address to a temporary sockaddr object on stack before changing the result object. However, a different uninitialized object is checked for the conversion status. Check the correct object. Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Dmytro Podgornyi <[email protected]>

Involving internal thread in checking OS events introduces huge lock contentions and serious performance degradation. The mechanism relied on an assumption that internal thread will wake up for OS events since each epfd is added to internal thread epfd thus saving OS polls in main thread. This assumption is incorrect since in reality most of the time CQ events wake up internal thread as well. To resolve that epoll os polling mechanism we make epoll Os polling to be performed the same as for poll and select via OS poll ration parameters. Thus we remove involvment of internal thread, remove lock contention and improve performance. Signed-off-by: Alexander Grissik <[email protected]>

Removing cached sys_vars improves CPU cache usage and makes the structs smaller. Global sys_vars do not change during runtime and are hot in CPU cache. Signed-off-by: Alexander Grissik <[email protected]>

Unti now, release process is done manually by the dev team. This process involves creating a tag on a commit that needs to be released, building that tag and copying the files to NFS location The idea is to create a jenkins pipeline that will do the entire process with given inputs, This commit will commence phase 1 - doing the build and copying of packages created to NFS. THE CREATION OF A RELEASE TAG ON A COMMIT IS NOT PART OF THIS PR Add release_jjb and the corosponding matrix file for a new pipeline definition, the release is run on rhel8.6 Add ARG HARBOR_URL to dockerfile, to support DRP in the future since the release process needs to run there as well Change do_release.sh script to match the new parameters provided by the jenkins job and pass ShellCheck Change create_pkg.sh to pass ShellCheck Signed-off-by: NirWolfer <[email protected]>

The DEFINED_FORCE_TX_POLLING is always used and should not be turned off. This is an obsolete leftover which marked TX poll section in epoll and global-ring-poll. Signed-off-by: Alexander Grissik <[email protected]>

Using trylock for request_notification introduces more empty pollings. It is better to wait for lock to be available. Using trylock for wait_for_notification prevents cleanup of ibv events which will accumulate with each failed trylock. Signed-off-by: Alexander Grissik <[email protected]>

The return value of wait_for_notification methods is unused by upper layers. We never expect errors in wait_for_notification so errors are printed inside wait_for_notification itself. Signed-off-by: Alexander Grissik <[email protected]>

This condition is only for Internal Thread. However this may prevent even internal thread from polling when CQ is not empty. Packets may arrive while m_b_was_drained set to true. Signed-off-by: Alexander Grissik <[email protected]>

Removing m_n_wce_counter which prevents from Internal Thread to perform full polling cycle. This memeber also introduces unnecessary operations and cache access for fast path. Signed-off-by: Alexander Grissik <[email protected]>

Cleanup of deprecated env params XLIO_RX_POLL_OS_RATIO and XLIO_RX_SKIP_OS. Signed-off-by: Alexander Grissik <[email protected]>

Rx Polling pools the CQ only up to X iterations. However, iomux call ring poll and if the polled CQEs do not generate an event, iomux may falsly conclude that the CQ is empty and go to sleep on non empty CQ which may lead to infinite sleep althouth there are unpolled CQEs in the CQ. To fix that, we should drain the CQ before going to sleep. In case a CQE arrives after we drained the CQ but before we requested a notification, we have another poll attempt after requesting the notification. Signed-off-by: Alexander Grissik <[email protected]>

BlueFlame support has been removed. Besides, XLIO_BF was unused and an rdma-core specific env parameter was checked instead. Remove the parameter as a leftover. Signed-off-by: Dmytro Podgornyi <[email protected]>

TCP_OVERSIZE is enabled by default and there is no reason to disable it. Remove TCP_OVERSIZE macro to reduce #ifdef creep. If needed, this feature can be disabled manually in a single place in the code for debugging purposes. Also remove TCP_OVERSIZE_DBGCHECK code blocks as unused and not covered. Signed-off-by: Dmytro Podgornyi <[email protected]>

tcp_rexmit_segment() is unused. Remove it to avoid code rotting. Signed-off-by: Dmytro Podgornyi <[email protected]>

This function is overcomplicated and keeps some legacy logic. Remove the detection of buffer size to allocate, because all the TX buffers have constant size. This also removes unnecessary flags from the prototype. Signed-off-by: Dmytro Podgornyi <[email protected]>

This is a duplicate information. Signed-off-by: Dmytro Podgornyi <[email protected]>

Signed-off-by: Dmytro Podgornyi <[email protected]>

TCP_SNDQUEUELEN_OVERFLOW is treated as hardcoded maximum allowed queuelen and checked in runtime additionally to snd_queuelen_max. Instead, set upper limit to snd_queuelen_max in control flow and remove the extra checks. This also allows to configure "endless" queue unambiguously by setting it to exact TCP_SNDQUEUELEN_OVERFLOW value. Signed-off-by: Dmytro Podgornyi <[email protected]>

Physical buffers have constant size. So, no point in providing max_length information during pbuf allocation and no need to keep unsent_overize. A PBUF_RAM always can keep at least mss_local bytes and we always can copy so many bytes to the last_unsent segment. This also makes impossible to reach the case when an extra PBUF_RAM is added to the last_unsent segment. Simplify the code around oversize feature and pbuf allocation. Signed-off-by: Dmytro Podgornyi <[email protected]>

Issue: HPCINFRA-2185 Signed-off-by: Viacheslav Login <[email protected]>

Signed-off-by: Alexander Grissik <[email protected]>

Jenkins job builder (JJB) allows for multiple job definitions in one file, this will make it easier to share common variables such as folder or git url, as well as make it easier to maintain in the future. In the PDC site the harbor url is different and needs to be changed Merge all jjb files into one under proj_jjb.yaml Change registry url in release_matrix to match the PDC harbor url Remove the latest_release link from do_release script, requrest by the dev team Signed-off-by: NirWolfer <[email protected]>

[CI] Unify pipeline files into one, change harbor url #210

Are you sure you want to change the base?

[CI] Unify pipeline files into one, change harbor url #210

Commits on Jan 14, 2024

Commits on Jan 17, 2024

Commits on Jan 18, 2024

Commits on Jan 22, 2024

Commits on Jan 25, 2024

Commits on Jan 30, 2024

Commits on Feb 1, 2024

Commits on Feb 6, 2024

Commits on Feb 7, 2024

Commits on Feb 12, 2024

Commits on Feb 22, 2024

Commits on Feb 25, 2024

Commits on Feb 26, 2024

Commits on Mar 10, 2024

Commits on Mar 11, 2024

Commits on Mar 12, 2024

Commits on Mar 18, 2024

Commits on Mar 19, 2024

Commits on Mar 20, 2024

Commits on Mar 31, 2024

Commits on Apr 1, 2024

Commits on Apr 3, 2024

Commits on Apr 4, 2024

Commits on Apr 8, 2024

Commits on Apr 9, 2024

Commits on Apr 25, 2024

Commits on Apr 28, 2024

Commits on May 1, 2024

Commits on May 2, 2024

Commits on May 19, 2024

Commits on May 29, 2024

Commits on Jun 13, 2024

Commits on Jun 16, 2024

Commits on Jun 17, 2024

Commits on Jun 18, 2024

Commits on Jun 27, 2024

Commits on Jun 30, 2024

Commits on Jul 4, 2024

Commits on Jul 8, 2024

Commits on Jul 10, 2024

Commits on Jul 11, 2024

Commits on Jul 14, 2024

Commits on Jul 15, 2024

Commits on Jul 24, 2024

Commits on Jul 30, 2024

Commits on Jul 31, 2024

Commits on Aug 4, 2024

Commits on Aug 7, 2024

Commits on Aug 8, 2024

Commits on Aug 11, 2024