[CI] Unify pipeline files into one, change harbor url #210
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
At most a single element of this vector is always used. Once rfs constructor is complete there must be exactly one attach_flow_data element in case of ring_simple. For ring_tap this element remains null. Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Signed-off-by: Alex Briskin <[email protected]>
Set ETIMEDOUT errno and return -1 from recv in case a socket was timed out, instead of 0 return value and 0 errno. For instance, in case of TCP keep alive timeout. Signed-off-by: Alexander Grissik <[email protected]>
The idea is to scan all rpm/deb packages for personal emails we should not be releasing packages with such emails the scan is done on both the metadat info and the changelog of a specific package Issue: HPCINFRA-919 Signed-off-by: Daniel Pressler <[email protected]>
Non-zerocopy (XLIO_SOCKET_SEND_FLAG_INLINE) operations reuse common tcp_write() which doesn't allow the queue to grow beyond specific number of pbufs. The limit is derived from the TCP send buffer and usually equals to hundreds or thousands of pbufs. XLIO Socket API encourages users to limit the inflight data and allows send operations to fails with the unrecoverable ENOMEM error. However, on the other hand, the API describes that send operations aren't limited by send buffer logic. To follow current XLIO Socket API description, allow the TCP queue to grow. To do this, increase the queue limit to 16M pbufs for XLIO sockets. Signed-off-by: Dmytro Podgornyi <[email protected]>
- we're switching here to ubuntu22.04-based container providing the support to clang-15 and clang-format-15 Signed-off-by: Viacheslav Login <[email protected]>
Signed-off-by: Dmytro Podgornyi <[email protected]>
Blackduck module report output shows the wrong version, this is due to projectVersion under args being set to a static value Change projectVersion to the branch entered instead of static value Signed-off-by: NirWolfer <[email protected]>
sys_str2addr() converts a string address representation to the sockaddr structure. However, an invalid address sets either AF_INET or AF_INET6 address family and returns corrupted IP portion of the sockaddr object. Return AF_UNSPEC address family in case of an invalid IP address. So, default gateway detection can handle address parsing. Invalid port portion is still silently ignored. In the future, the function should return the conversion status and this needs to be handled explicitly by the function users. Signed-off-by: Dmytro Podgornyi <[email protected]>
sys_gateway() converts string address to a temporary sockaddr object on stack before changing the result object. However, a different uninitialized object is checked for the conversion status. Check the correct object. Signed-off-by: Dmytro Podgornyi <[email protected]>
Signed-off-by: Dmytro Podgornyi <[email protected]>
Signed-off-by: Dmytro Podgornyi <[email protected]>
Involving internal thread in checking OS events introduces huge lock contentions and serious performance degradation. The mechanism relied on an assumption that internal thread will wake up for OS events since each epfd is added to internal thread epfd thus saving OS polls in main thread. This assumption is incorrect since in reality most of the time CQ events wake up internal thread as well. To resolve that epoll os polling mechanism we make epoll Os polling to be performed the same as for poll and select via OS poll ration parameters. Thus we remove involvment of internal thread, remove lock contention and improve performance. Signed-off-by: Alexander Grissik <[email protected]>
Removing cached sys_vars improves CPU cache usage and makes the structs smaller. Global sys_vars do not change during runtime and are hot in CPU cache. Signed-off-by: Alexander Grissik <[email protected]>
Unti now, release process is done manually by the dev team. This process involves creating a tag on a commit that needs to be released, building that tag and copying the files to NFS location The idea is to create a jenkins pipeline that will do the entire process with given inputs, This commit will commence phase 1 - doing the build and copying of packages created to NFS. THE CREATION OF A RELEASE TAG ON A COMMIT IS NOT PART OF THIS PR Add release_jjb and the corosponding matrix file for a new pipeline definition, the release is run on rhel8.6 Add ARG HARBOR_URL to dockerfile, to support DRP in the future since the release process needs to run there as well Change do_release.sh script to match the new parameters provided by the jenkins job and pass ShellCheck Change create_pkg.sh to pass ShellCheck Signed-off-by: NirWolfer <[email protected]>
The DEFINED_FORCE_TX_POLLING is always used and should not be turned off. This is an obsolete leftover which marked TX poll section in epoll and global-ring-poll. Signed-off-by: Alexander Grissik <[email protected]>
Using trylock for request_notification introduces more empty pollings. It is better to wait for lock to be available. Using trylock for wait_for_notification prevents cleanup of ibv events which will accumulate with each failed trylock. Signed-off-by: Alexander Grissik <[email protected]>
The return value of wait_for_notification methods is unused by upper layers. We never expect errors in wait_for_notification so errors are printed inside wait_for_notification itself. Signed-off-by: Alexander Grissik <[email protected]>
This condition is only for Internal Thread. However this may prevent even internal thread from polling when CQ is not empty. Packets may arrive while m_b_was_drained set to true. Signed-off-by: Alexander Grissik <[email protected]>
Removing m_n_wce_counter which prevents from Internal Thread to perform full polling cycle. This memeber also introduces unnecessary operations and cache access for fast path. Signed-off-by: Alexander Grissik <[email protected]>
Cleanup of deprecated env params XLIO_RX_POLL_OS_RATIO and XLIO_RX_SKIP_OS. Signed-off-by: Alexander Grissik <[email protected]>
Rx Polling pools the CQ only up to X iterations. However, iomux call ring poll and if the polled CQEs do not generate an event, iomux may falsly conclude that the CQ is empty and go to sleep on non empty CQ which may lead to infinite sleep althouth there are unpolled CQEs in the CQ. To fix that, we should drain the CQ before going to sleep. In case a CQE arrives after we drained the CQ but before we requested a notification, we have another poll attempt after requesting the notification. Signed-off-by: Alexander Grissik <[email protected]>
BlueFlame support has been removed. Besides, XLIO_BF was unused and an rdma-core specific env parameter was checked instead. Remove the parameter as a leftover. Signed-off-by: Dmytro Podgornyi <[email protected]>
TCP_OVERSIZE is enabled by default and there is no reason to disable it. Remove TCP_OVERSIZE macro to reduce #ifdef creep. If needed, this feature can be disabled manually in a single place in the code for debugging purposes. Also remove TCP_OVERSIZE_DBGCHECK code blocks as unused and not covered. Signed-off-by: Dmytro Podgornyi <[email protected]>
tcp_rexmit_segment() is unused. Remove it to avoid code rotting. Signed-off-by: Dmytro Podgornyi <[email protected]>
This function is overcomplicated and keeps some legacy logic. Remove the detection of buffer size to allocate, because all the TX buffers have constant size. This also removes unnecessary flags from the prototype. Signed-off-by: Dmytro Podgornyi <[email protected]>
This is a duplicate information. Signed-off-by: Dmytro Podgornyi <[email protected]>
Signed-off-by: Dmytro Podgornyi <[email protected]>
TCP_SNDQUEUELEN_OVERFLOW is treated as hardcoded maximum allowed queuelen and checked in runtime additionally to snd_queuelen_max. Instead, set upper limit to snd_queuelen_max in control flow and remove the extra checks. This also allows to configure "endless" queue unambiguously by setting it to exact TCP_SNDQUEUELEN_OVERFLOW value. Signed-off-by: Dmytro Podgornyi <[email protected]>
Physical buffers have constant size. So, no point in providing max_length information during pbuf allocation and no need to keep unsent_overize. A PBUF_RAM always can keep at least mss_local bytes and we always can copy so many bytes to the last_unsent segment. This also makes impossible to reach the case when an extra PBUF_RAM is added to the last_unsent segment. Simplify the code around oversize feature and pbuf allocation. Signed-off-by: Dmytro Podgornyi <[email protected]>
Issue: HPCINFRA-2185 Signed-off-by: Viacheslav Login <[email protected]>
Signed-off-by: Alexander Grissik <[email protected]>
Jenkins job builder (JJB) allows for multiple job definitions in one file, this will make it easier to share common variables such as folder or git url, as well as make it easier to maintain in the future. In the PDC site the harbor url is different and needs to be changed Merge all jjb files into one under proj_jjb.yaml Change registry url in release_matrix to match the PDC harbor url Remove the latest_release link from do_release script, requrest by the dev team Signed-off-by: NirWolfer <[email protected]>
AlexanderGrissik
force-pushed
the
vNext
branch
from
cfce2e8
to
51c2340
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Jenkins job builder (JJB) allows for multiple job definitions in one file, this will make it easier to share common variables such as folder or git url, as well as make it easier to maintain in the future.
In the PDC site the harbor url is different and needs to be changed
What
Merge all jjb files into one under proj_jjb.yaml
Change registry url in release_matrix to match the PDC harbor url Remove the latest_release link from do_release script, requrest by the dev team
Change type
What kind of change does this PR introduce?
Check list