Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix tests (for real?) #35608

Conversation

stephane-airbyte
Copy link
Contributor

@stephane-airbyte stephane-airbyte commented Feb 24, 2024

We're disabling some tests that are failing or hanging both on CI and on localhost: SshPasswordMssqlSourceAcceptanceTest, SshKeyMssqlSourceAcceptanceTest and part of AbstractMssqlSourceDatatypeTest. Hopefully we'll fix and reenable them in a subsequent change

Some tests were failing because of a timing issue. For those, we do a few things:

  1. the enablement of CDC could fail, so now we call a function that will try for up to 5 minutes to enable CDC for a given table.
  2. we remove the test-only wait from production code in MssqlCdcTargetPosition
  3. instead of waiting for a set time, we have a busy loop that will wait for a certain number of records to be present in the CDC tables (waitForCdcRecords)

In the process of doing that, there's an edge case that was found in MsssqlCdcTargetPosition where the MinLsn could be null if there hasn't been any changes to the source DB since CDC was enabled. It's a rare case, but not one that should cause a sync failure.

Finally, even with the changes above, some tests are still failing on CI. We're disabling those as well, with the goal of really fixing them soon:

Copy link

vercel bot commented Feb 24, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Feb 27, 2024 0:30am

Copy link
Contributor Author

stephane-airbyte commented Feb 24, 2024

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @stephane-airbyte and the rest of your teammates on Graphite Graphite

@stephane-airbyte stephane-airbyte changed the base branch from stephane/02-23-disable_source-mssql_tests_that_are_failing_on_localhost to stephane/02-21-improve_logging_in_MsSQLTestDatabase February 24, 2024 09:02
@stephane-airbyte stephane-airbyte force-pushed the stephane/02-21-improve_logging_in_MsSQLTestDatabase branch from 5dd97b2 to d21b594 Compare February 24, 2024 09:18
@stephane-airbyte stephane-airbyte changed the base branch from stephane/02-21-improve_logging_in_MsSQLTestDatabase to stephane/02-09-try_removing_concurrency_for_source-mssql_tests February 24, 2024 17:26
@stephane-airbyte stephane-airbyte force-pushed the stephane/02-24-fix_tests_for_real_ branch 8 times, most recently from ed55ae1 to 05c3750 Compare February 26, 2024 21:58
@stephane-airbyte stephane-airbyte force-pushed the stephane/02-21-move_some_very_verbose_log_messages_to_info branch from f390b9e to 942a0f2 Compare February 26, 2024 22:58
@stephane-airbyte stephane-airbyte force-pushed the stephane/02-24-fix_tests_for_real_ branch 2 times, most recently from 64068bd to 80a5729 Compare February 26, 2024 23:19
@stephane-airbyte stephane-airbyte changed the base branch from stephane/02-21-move_some_very_verbose_log_messages_to_info to stephane/02-24-move_source-mssql_to_latest_CDK February 26, 2024 23:19
@stephane-airbyte stephane-airbyte force-pushed the stephane/02-24-move_source-mssql_to_latest_CDK branch 3 times, most recently from e45df64 to 029a4d3 Compare February 26, 2024 23:42
@stephane-airbyte stephane-airbyte force-pushed the stephane/02-24-move_source-mssql_to_latest_CDK branch 2 times, most recently from bbe3176 to 0f65fb8 Compare February 27, 2024 00:06
@stephane-airbyte stephane-airbyte force-pushed the stephane/02-24-move_source-mssql_to_latest_CDK branch from 0f65fb8 to dd7b58e Compare February 27, 2024 00:09
@stephane-airbyte stephane-airbyte force-pushed the stephane/02-24-move_source-mssql_to_latest_CDK branch from dd7b58e to 90275fb Compare February 27, 2024 00:30
@stephane-airbyte stephane-airbyte merged commit ce3d6be into stephane/02-24-move_source-mssql_to_latest_CDK Feb 27, 2024
24 of 38 checks passed
@stephane-airbyte stephane-airbyte deleted the stephane/02-24-fix_tests_for_real_ branch February 27, 2024 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues CDK Connector Development Kit connectors/source/mssql
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants