Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch Timeout on test connection #4191

Closed
wants to merge 3,148 commits into from

Conversation

Glutexo
Copy link
Collaborator

@Glutexo Glutexo commented Aug 14, 2024

All Pull Requests:

Check all that apply:

  • Have you followed the guidelines in our Contributing document, including the instructions about commit messages?
  • Is this PR to correct an issue?
  • Is this PR an enhancement?

Complete Description of Additions/Changes:

The --test-connection command only used to catch ConnectionError, including ConnectionTimeout, but not other timeouts such as ReadTimeoutError. Re-used the existing REQUEST_FAILED_EXCEPTIONS list to process all timeouts in the same way.

There is an integration test pull request RedHatInsights/insights-client#244 for this fix.

Card IDs:

ahitacat and others added 30 commits August 28, 2023 22:54
Change the tar command to create the archive tarfile with the correct
structure.

- insights-localhost-XxXxX.tar
|_ - insights-localhost-XxXxX
  |_ - /data
  |_ - /meta_data

Signed-off-by: ahitacat <[email protected]>
- and mare Lssap parser as deprecated (remove from 3.3.0)

Signed-off-by: Xiangce Liu <[email protected]>
- ref: INSGHTCORE-229
- and remove non-exist symbolic_name

Signed-off-by: Xiangce Liu <[email protected]>
Requests and urllib3 now support proxy authentication natively so we
don't need this proxy hack stuff anymore. Instead we can just directly
read in the proxy authentication.

The auth split from the url is left in so that debu messages can still
work without printing the authentication to the console.

Signed-off-by: Stephen Adams <[email protected]>
…ights#3902)

* feat: add spec and parser for udev 66-md-auto-re-add.rules

Signed-off-by: Xiaoxue Wang <[email protected]>

* Replace the Non-ASCII character in test file

Signed-off-by: Xiaoxue Wang <[email protected]>
…3792)

* Add new spec for lvm_fullreport
* Add parser LvmFullReport
* Add tests and documentation

Signed-off-by: Bob Fahr <[email protected]>

* Fix py27 error
* Fix docs error
* Fixes to parser and tests
* Fixed some column names that are diff between RHEL 6 and newer RHEL
* Add nolocking to command

Signed-off-by: Bob Fahr <[email protected]>

* Add nolocking to archive spec

Signed-off-by: Bob Fahr <[email protected]>
* Faat: Add new parser postfix_conf

Signed-off-by: Xinting Li <[email protected]>

* update default.py
* Update doc string
* UPdate postfix_conf

Signed-off-by: Xinting Li <[email protected]>
insights-client was using /var/tmp to store the egg release file.

This commit changes that temporary directory to /var/cache/insights-client.

Signed-off-by: ahitacat <[email protected]>
…ghts#3896)

- uploader.json is not used by core collection anymore
- map the rm_conf from the components in memory directly
- add tests for the new updated apply_blacklist
- for details, see INSGHTCORE-229 and RHELPLAN-119003

Signed-off-by: Xiangce Liu <[email protected]>
…dHatInsights#3909)

* RHINENG-1764: send every errata for available package in RHEL8

Signed-off-by: yungbender <[email protected]>

* RHINENG-1764: send every errata for available package in RHEL7

Signed-off-by: yungbender <[email protected]>

* RHINENG-1764: start processing every errata for available package

Signed-off-by: yungbender <[email protected]>
…hts#3755)

- also fix  RedHatInsights#3739
- new built-in metadata files is generated in data dir for core collection
- mark the MetadataProvider as deprecated and remove it from 3.5.0
-  the old built-in files are kept as they were for backward compatibility
- the blacklist_report is not implemented in datasource yet
- see INSGHTCORE-203

Signed-off-by: Xiangce Liu <[email protected]>
)

- It's possible that multiple intersystem instances are installed.

Signed-off-by: jiazhang <[email protected]>
- add a new rpm_v_package using foreach_execute
   in order to distinguish the output for the package.
- add new RpmVPackage Parser correspondingly
- deprecate the RpmVPackages

Signed-off-by: Xinting Li <[email protected]>
…ts#3924)

- show internal DeprecationWarnings only when pytest
- it's helpful for execution performance
- See TYCHE-1016
- and skip the test for py26

Signed-off-by: Xiangce Liu <[email protected]>
- deprecate parsers that are using pyparsing
- deprecate RabbitMQReport related parsers
- Deprecate MultipathConf related parsers
- Fix parser_error of MultipathConfTree parse_doc
- Enhance MultipathConfTree with more special test cases

Signed-off-by: Xiaoxue Wang <[email protected]>
…ghts#3928) (RedHatInsights#3929)

The Broker.store_skips property needs to be set before the dependency
graph execution in run_input_data().
- And use a constant for the rules download directory

Signed-off-by: Mark Huth <[email protected]>
…ghts#3927)

* feat: New spec and parser for xfs_db -r -c freesp command

Signed-off-by: Ping Qin <[email protected]>

* fix: make the code robust when there are other lines in output

Signed-off-by: Ping Qin <[email protected]>
@Glutexo Glutexo marked this pull request as ready for review August 19, 2024 14:41
ptoscano and others added 2 commits August 20, 2024 14:19
…s#4190)

Add a simple GitHub workflow to build the egg for new PRs and pushes to
the maintained branches. The resulting egg is kept as artifact, so it
can be downloaded and used later on.

Signed-off-by: Pino Toscano <[email protected]>
@Glutexo Glutexo force-pushed the read-timeout-error branch 2 times, most recently from 7276f65 to aaa32a4 Compare August 20, 2024 11:54
@Glutexo
Copy link
Collaborator Author

Glutexo commented Aug 20, 2024

I improved the tests by raising actual exceptions: SSLError (a non-Timeout ConnectionError), ConnectionTimeout (both a Timeout and a ConnectionError) and ReadTimeout (a non-ConnectionError Timeout) rather than the abstract ancestors ConnectionError and Timeout that are caught in the code.

Rebased on current master.

I see the python2x tests are failing. I believed we dropped suppport for Python 2. I will fix the unsupported syntax.

xiangce and others added 4 commits August 22, 2024 14:06
…dHatInsights#4189)

- For the first cmd to run in subprocess.call, specified
  "stdin=subprocess.DEVNULL" to avoid it from eating
  the stdin behind it
- also fix: RedHatInsights#4187 in the base call

Signed-off-by: Xiangce Liu <[email protected]>

* undo the wrapped lines which is unnecessary

Signed-off-by: Xiangce Liu <[email protected]>
…ights#4197)

- to remove the following unexpected tracebacks in client log:
  DEBUG insights.core.spec_factory:709 Traceback (most recent call last):
    ...
    raise NoFilterException("Skipping %s due to no filters." % dr.get_name(self.ds))

Signed-off-by: Xiangce Liu <[email protected]>
…#4199)

- py26 should be supported until one-egg/rpm-per-rhel is implemented

Signed-off-by: Xiangce Liu <[email protected]>
@Glutexo Glutexo force-pushed the read-timeout-error branch 2 times, most recently from 37ab317 to 316a612 Compare August 24, 2024 18:03
@Glutexo
Copy link
Collaborator Author

Glutexo commented Aug 24, 2024

Fixed the unsupported Python 2 syntax – an f-string. Rebased on the current master.

@Glutexo
Copy link
Collaborator Author

Glutexo commented Aug 24, 2024

Re-requesting a review. @m-horky might be the right guy for it

Copy link
Contributor

@m-horky m-horky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally fine.

When I set the timeout to even lower value (0.01), the output is different -- it is more verbose and the exception is printed twice. Would a fix of that be in scope of this change?

$ insights-client --test-connection
Running Connection Tests...
=== Begin Upload URL Connection Test ===
Testing https://cert.cloud.redhat.com/api/ingress/v1/upload
POST https://cert.cloud.redhat.com/api/ingress/v1/upload
Could not successfully connect to: https://cert.cloud.redhat.com/api/ingress/v1/upload
HTTPSConnectionPool(host='cert.cloud.redhat.com', port=443): Max retries exceeded with url: /api/ingress/v1/upload (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f4fd4550ad0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
HTTPSConnectionPool(host='cert.cloud.redhat.com', port=443): Max retries exceeded with url: /api/ingress/v1/upload (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f4fd4550ad0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
Connectivity test failed! Please check your network configuration
Additional information may be in /var/log/insights-client/insights-client.log

insights/tests/client/connection/test_test_connection.py Outdated Show resolved Hide resolved
insights/tests/client/connection/test_test_connection.py Outdated Show resolved Hide resolved
insights/tests/client/connection/test_test_connection.py Outdated Show resolved Hide resolved
The --test-connection command only used to catch ConnectionError, including ConnectionTimeout, but not other timeouts such as ReadTimeoutError. Re-used the existing REQUEST_FAILED_EXCEPTIONS list to process all timeouts in the same way.

* Card ID: CCT-506

Signed-off-by: Štěpán Tomsa <[email protected]>
@Glutexo
Copy link
Collaborator Author

Glutexo commented Aug 27, 2024

I found the culprit for the exception being printed twice:

There is another try-except block in the _test_urls function, that only catches ConnectionError (not Timeout). There, the exception is printed and then re-raised. This is completely redundant, as _test_urls is only used in test_connection, which has its own error handling.

I think this can go into this pull request. Will remove.

@xiangce
Copy link
Contributor

xiangce commented Sep 5, 2024

Hi @Glutexo - Sorry for bringing this trouble to you, as we just resolved an urgent bug within the repo. Please rebase the master branch of your fork of insights-core and then rebase this branch again. Please reach out to me for any problems when doing the rebase. Apologize again.

@Glutexo Glutexo closed this Sep 10, 2024
@Glutexo Glutexo deleted the read-timeout-error branch September 10, 2024 08:54
@Glutexo
Copy link
Collaborator Author

Glutexo commented Sep 10, 2024

@xiangce Thank you for reminding me! I fixed everything. However, I accidentally deleted my original remote branch before rebasing. As a result, this pull request got closed and it’s not possible to re-open it even after a new push. I‘ll open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client These issues represent work to be done by the "client" team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.