wait_for_all_checks timeout on merge with rulesets

[ofer-classiq]

Describe the bug

merge wait_for_all_checks gets timed out waiting for all required checks to complete, resulting in a "skipped" status.
There are no failures on the action and in the checks.

To Reproduce

We're using the safe_changes example with rebase_on_merge.

run:
  - action: add-label@v1
    args:
      label: 'safe-change'
  - action: approve@v1
  - action: add-comment@v1
    args:
      comment: |
        This PR is considered a safe change and has been automatically approved.
  - action: merge@v1
    args:
      wait_for_all_checks: true
      rebase_on_merge: true

Expected behavior

Wait for all checks to complete or the specific action should be re-triggered on check results.

Screenshots

Additional context

[BenLloydPearson]

Thanks for reporting this @ofer-classiq, we already spoke with you about this, but I want to provide an update here for everyone else. We determined that this is an issue with GitHub Rulesets, which are currently unsupported. We've added this to our roadmap backlog and will prioritize it when we have the opportunity.

We'll provide updates on this improvement in this thread.

[floskel]

Is there a work around for this? Is it possible to ensure that this workflow is executed after the fx CircleCI tests?

[PavelLinearB]

Hi @floskel, The problem reported in this issue is an issue with the merge action on repositories that use rulesets. Is this the same issue you are experiencing?

[floskel]

Hm, it might not be. Im seeing the "merge" stream is just skipped fairly fast. The workflow is configure like this:

merge_translations:
    if:
      - {{ branch.name | includes(term='crowdin') }}
    run:
      - action: merge@v1
        args:
          rebase_on_merge: true
          wait_for_all_checks: true

The workflow is executed with a org wide repository.

The code_quality_assurance action takes ~2 minutes to complete and the shared timeout for gitStream is 5 min.

[PavelLinearB]

Can you please share the method code-quality-assurance updates its check?
We have recently found that check resolutions via API calls fails to trigger the merge action. This is currently under investigation and will be fixed soon

[floskel]

It is setup with CircleCIs GitHub app as described here https://circleci.com/docs/enable-checks/#enable-github-checks

[floskel]

And on main branch we have required checks from PRs