Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix load profiles TSAN issue [17162] #3281

Merged
merged 4 commits into from
Jun 22, 2023
Merged

Conversation

jparisu
Copy link
Contributor

@jparisu jparisu commented Feb 8, 2023

Signed-off-by: jparisu [email protected]

load_profiles() method is called in a singleton without guards, and this could lead to multiple threads loading files at the same time.
TSAN fails in most tests for this (every time a DomainParticipant is created).

Description

Contributor Checklist

  • Commit messages follow the project guidelines.
  • The code follows the style guidelines of this project.
  • N/A Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally
  • Any new/modified methods have been properly documented using Doxygen.
  • ¿? Changes are ABI compatible.
  • Changes are API compatible.
  • Documentation builds and tests pass locally.
  • New feature has been added to the versions.md file (if applicable).
  • N/A New feature has been documented/Current behavior is correctly described in the documentation.
  • Applicable backports have been included in the description.

Reviewer Checklist

  • Check contributor checklist is correct.
  • Check CI results: changes do not issue any warning.
  • Check CI results: failing tests are unrelated with the changes.

@jparisu jparisu changed the title Fix load profiles TSAN issue Fix load profiles TSAN issue [17162] Feb 10, 2023
@JLBuenoLopez JLBuenoLopez added this to the v2.10.1 milestone Mar 16, 2023
@EduPonz EduPonz modified the milestones: v2.10.1, v2.10.2 Mar 31, 2023
Copy link
Member

@MiguelCompany MiguelCompany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a regression test.

    PubSubWriter<HelloWorldPubSubType> participant_1(TEST_TOPIC_NAME);
    participant_1.init();

    auto second_participant_creation = []()
            {
                PubSubReader<HelloWorldPubSubType> participant_2(TEST_TOPIC_NAME);
                participant_2.init();
                participant_2.wait_discovery();
            };

    // Start thread creating second participant
    std::thread thr(second_participant_creation);

    participant_1.wait_discovery();
    thr.join();

@MiguelCompany
Copy link
Member

@richiprosima Please test this

@MiguelCompany MiguelCompany added the ci-pending PR which CI is running label Apr 28, 2023
@EduPonz
Copy link

EduPonz commented May 2, 2023

@jparisu the new test is failing in both windows pipelines, please take a look whenever you have time

@jparisu
Copy link
Contributor Author

jparisu commented May 3, 2023

Some issues have been encountered adding this test:

  1. ASAN fails randomly because of Dynamic Types
  2. Windows fails randomly with this error Assertion failed: open_or_created && winapi::get_last_error() != winapi::error_already_exists, file C:\Users\eProsima\paris_release_debug\src\fastdds\thirdparty\boost\include\boost/interprocess/sync/windows/mutex.hpp, line 65

These errors are not introduced (at least it does not look like) but this PR, but they arise because there weren't previous tests with parallel creation of Domain Participants

@jparisu
Copy link
Contributor Author

jparisu commented May 3, 2023

@richiprosima please test windows

1 similar comment
@JLBuenoLopez
Copy link
Contributor

@richiprosima please test windows

@JLBuenoLopez
Copy link
Contributor

@jparisu uncrustify job is failing in this PR

@jparisu
Copy link
Contributor Author

jparisu commented May 10, 2023

@jparisu uncrustify job is failing in this PR

@JLBuenoLopez-eProsima test introduce in this PR produces several ASAN errors due to not related issues. Do you still want to merge it?

@JLBuenoLopez
Copy link
Contributor

@jparisu, as far as I understand the test introduced in this PR is flaky because the strategy followed in the test has not been used previously. So the test is surfacing some other issue that will have to be deal with at some point. As the CI runs several times the failing tests to check if they are flaky, I am not really concerned about the test failing randomly. @EduPonz, what do you think?

@EduPonz
Copy link

EduPonz commented May 11, 2023

@jparisu, as far as I understand the test introduced in this PR is flaky because the strategy followed in the test has not been used previously. So the test is surfacing some other issue that will have to be deal with at some point. As the CI runs several times the failing tests to check if they are flaky, I am not really concerned about the test failing randomly. @EduPonz, what do you think?

We may merge this if we exclude this new test from the TSAN analysis until we fix it

@jparisu
Copy link
Contributor Author

jparisu commented May 11, 2023

We may merge this if we exclude this new test from the TSAN analysis until we fix it

It produces ASAN errors, not TSAN errors (that are also produced, but not new ones).

@JLBuenoLopez JLBuenoLopez modified the milestones: v2.10.2, v2.11.0 Jun 5, 2023
@JLBuenoLopez
Copy link
Contributor

@jparisu, it has been decided that this is going to be merged and afterwards the ASAN errors will be fixed. Can you please fix the uncrustify job so we can proceed? Thanks!

@Mario-DL
Copy link
Member

Friendly pinging @jparisu

@Mario-DL
Copy link
Member

@richiprosima please test this

@MiguelCompany
Copy link
Member

@richiprosima Please test windows

Copy link
Member

@MiguelCompany MiguelCompany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with green CI

@MiguelCompany
Copy link
Member

@Mergifyio rebase

Signed-off-by: jparisu <[email protected]>
@mergify
Copy link
Contributor

mergify bot commented Jun 22, 2023

rebase

✅ Branch has been successfully rebased

@MiguelCompany MiguelCompany merged commit f658ee2 into master Jun 22, 2023
@MiguelCompany MiguelCompany deleted the fix/tsan-load-profiles branch June 22, 2023 13:55
@SunnyNetX
Copy link

SunnyNetX commented Feb 27, 2024

@jparisu Have you solved this problem?how?

Some issues have been encountered adding this test:

  1. ASAN fails randomly because of Dynamic Types
  2. Windows fails randomly with this error Assertion failed: open_or_created && winapi::get_last_error() != winapi::error_already_exists, file C:\Users\eProsima\paris_release_debug\src\fastdds\thirdparty\boost\include\boost/interprocess/sync/windows/mutex.hpp, line 65

These errors are not introduced (at least it does not look like) but this PR, but they arise because there weren't previous tests with parallel creation of Domain Participants

roscan-tech pushed a commit to roscan-tech/Fast-DDS that referenced this pull request Sep 8, 2024
* Add regression test

Signed-off-by: jparisu <[email protected]>

* Fix load profiles TSAN issue

Signed-off-by: jparisu <[email protected]>

* make test to wait to all discovery

Signed-off-by: jparisu <[email protected]>

* uncrustify

Signed-off-by: jparisu <[email protected]>

---------

Signed-off-by: jparisu <[email protected]>
Signed-off-by: roscan-tech <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-pending PR which CI is running
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants