Gitlab regression in 2.5.0 #863

M0NsTeRRR · 2024-09-17T10:09:43Z

Is there an existing issue for this?

I have searched the existing issues

Description of the bug

On a self-hosted GitLab instance, I encountered a 401 error with git-cliff 2.5.0 using the action below. It was working fine with version 2.4.0.

Steps To Reproduce

Using this action

test_changelog:
  image:
    name: ghcr.io/orhun/git-cliff/git-cliff:2.5.0
    entrypoint: [""]
  variables:
    GIT_STRATEGY: clone # clone entire repo instead of reusing workspace
    GIT_DEPTH: 0 # avoid shallow clone to give cliff all the info it needs
    GITLAB_API_URL: "${CI_API_V4_URL}"
    GITLAB_TOKEN: "${CI_JOB_TOKEN}"
    GITLAB_REPO: "${CI_PROJECT_PATH}"
    RUST_BACKTRACE: full
  stage: test
  script:
    - git-cliff --config cliff/cliff.toml -v -x

stages:
  - test

Expected behavior

Changelog should be properly generated

Screenshots / Logs

2.5.0

$ git-cliff --config cliff/cliff.toml -v -x
 DEBUG git_cliff > Failed to get remote from GitHub repository: RepoError("no remotes configured")
 DEBUG git_cliff_core::changelog > Adding remote data...
 WARN  git_cliff_core::changelog > You are using an experimental feature! Please report bugs at <https://git-cliff.org/issues>
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.xxxx.xxxx/api/v4/projects/xxx%2Fxxx
 DEBUG reqwest::connect          > starting new connection: https://gitlab.xxxx.xxxx/
 ERROR git_cliff_core::remote    > Request error: {"message":"401 Unauthorized"}
 ERROR git_cliff_core::changelog > Failed to lookup project! Cannot de/serialize JSON: `missing field `id` at line 1 column 30`
thread 'main' panicked at git-cliff-core/src/changelog.rs:452:18:
Could not get gitlab metadata: JsonError(Error("missing field `id`", line: 1, column: 30))
stack backtrace:
   0:     0x56100d4a8b7c - <unknown>
   1:     0x56100d0edfbb - <unknown>
   2:     0x56100d470ba2 - <unknown>
   3:     0x56100d4aa538 - <unknown>
   4:     0x56100d4ab452 - <unknown>
   5:     0x56100d4aaf45 - <unknown>
   6:     0x56100d4aaea9 - <unknown>
   7:     0x56100d4aae94 - <unknown>
   8:     0x56100d0ecd22 - <unknown>
   9:     0x56100d0f2b45 - <unknown>
  10:     0x56100d17bdf8 - <unknown>
  11:     0x56100d111cb0 - <unknown>
  12:     0x56100d02272c - <unknown>
  13:     0x56100d01b983 - <unknown>
  14:     0x56100d02311d - <unknown>
  15:     0x7f0a9d6d909b - __libc_start_main
  16:     0x56100d01896a - <unknown>
  17:                0x0 - <unknown>
Cleaning up project directory and file based variables

2.4.0

$ git-cliff --config cliff/cliff.toml -v -x
 DEBUG git_cliff > Failed to get remote from GitHub repository: RepoError("no remotes configured")
 DEBUG git_cliff_core::changelog > Processing the commits...
 DEBUG git_cliff_core::changelog > Processing the releases...
 DEBUG git_cliff_core::changelog > Adding remote data...
[{"version":"1.5.0" ...

Software information

Project version: 2.5.0

Additional context

No response

The text was updated successfully, but these errors were encountered:

welcome · 2024-09-17T10:09:46Z

Thanks for opening your first issue at git-cliff! Be sure to follow the issue template! ⛰️

orhun · 2024-09-18T13:26:26Z

Hello, thanks for the issue!

How do you set your GitLab remote? We now URL-encode the remote requests for GitLab which might be related. See: #742

orhun · 2024-09-18T13:28:58Z

Btw, was the GitLab integration working on 2.4.0? I don't see anything related in the logs.

M0NsTeRRR · 2024-09-18T13:43:28Z

Hello, thanks for the issue!

How do you set your GitLab remote? We now URL-encode the remote requests for GitLab which might be related. See: #742

I've provided the snippet above but I just set theses two variables :

GITLAB_API_URL is set to CI_API_V4_URL (predefined gitlab variable) it looks like https://gitlab.xxxx.xxxx/api/v4
GITLAB_REPO is set to CI_PROJECT_PATH (predefined gitlab variable) it looks like my-namespace/my-project

I confirm in 2.5.0 output I can see the encoded URL https://gitlab.xxxx.xxxx/api/v4/projects/my-namespace%2Fmy-project

Btw, was the GitLab integration working on 2.4.0? I don't see anything related in the logs.

Yes it was, I just truncated it [{"version":"1.5.0" ... in the log output.

orhun · 2024-09-21T12:53:44Z

It seems correct, can you go to https://gitlab.xxxx.xxxx/api/v4/projects/my-namespace%2Fmy-project and see if it returns anything?

I tried to reproduce with the following command and it worked for me:

GITLAB_API_URL=https://gitlab.archlinux.org/api/v4 git cliff -vv --gitlab-repo archlinux/packaging/packages/arch-repro-status -c github.toml

It first sends a request to:

 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/archlinux%2Fpackaging%2Fpackages%2Farch-repro-status

After fetching the project id, the rest is like:

 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=1
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=2
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=3
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=4
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=5
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=6
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=7
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=8
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=9
 DEBUG git_cliff_core::remote    > Sending request to: https://gitlab.archlinux.org/api/v4/projects/32412/repository/commits?per_page=100&page=10

dessalines · 2024-09-22T13:14:31Z

This is also affecting all github projects too:

ERROR git_cliff_core::changelog > Failed to lookup project! Cannot de/serialize JSON: `missing field `id` at line 1 column 35`
thread 'main' panicked at git-cliff-core/src/changelog.rs:458:18:
Could not get gitlab metadata: JsonError(Error("missing field `id`", line: 1, column: 35))

M0NsTeRRR · 2024-09-22T21:21:20Z

It seems correct, can you go to https://gitlab.xxxx.xxxx/api/v4/projects/my-namespace%2Fmy-project and see if it returns anything?

Yes, I received the JSON response from the GitLab API with the repository information.

Interestingly, when testing in the CLI with a personal token that has full access, it worked, but in CI with the CI_JOB_TOKEN, it doesn’t work (I just retested). @orhun, are you experiencing the same issue in CI with the code provided above? Does git-cliff make new API calls from version 2.5.0 that require additional permissions ?

I can't test with 2.6.0 for now due to #879

orhun · 2024-09-23T19:11:07Z

This is also affecting all github projects too:

That looks odd, normally you wouldn't need to send project look up requests for GitHub. Are you sure you have your remote set up correctly?

are you experiencing the same issue in CI with the code provided above?

I tested locally and it all seems fine to me. Can you point me to the GitLab repository where you are testing this, if possible?

Does git-cliff make new API calls from version 2.5.0 that require additional permissions ?

No, not at all. There is only one significant change which is #742 and I guess that's not related.

dessalines · 2024-09-23T21:57:17Z

That looks odd, normally you wouldn't need to send project look up requests for GitHub. Are you sure you have your remote set up correctly?

Yep, you can test this on any repo with a [remote.github] block, or a fresh git cliff --init github.

There should probably be some tests added for this to prevent breaking all github projects in the future.

M0NsTeRRR · 2024-09-24T12:35:09Z

I tested locally and it all seems fine to me. Can you point me to the GitLab repository where you are testing this, if possible?

It's on a private gitlab instance but I got the same result on a gitlab public instance

2.5.0 fail (main branch)
https://gitlab.com/M0NsTeRRR/git-cliff-debug/-/jobs/7905153575
2.4.0 success (2.4.0 branch)
https://gitlab.com/M0NsTeRRR/git-cliff-debug/-/jobs/7906252606

orhun · 2024-09-25T07:59:18Z

@dessalines that bug is now fixed in #885

@M0NsTeRRR it turns out the root of this issue is #782

In your example, there isn't any GitLab-related variables in cliff.toml therefore the GitLab data is not fetched for 2.4.0. However, this requirement was removed in 2.5.0 which means the data will be fetched if you set the remote in any way. (e.g. using GITLAB_REPO)

To reproduce the error for 2.4.0:

Now, when it comes to the actual permission issue, my guess is CI_JOB_TOKEN does not have access to the API endpoint that is being used. See:

So the solution would be generating a personal access token and using that instead :) Can you give it a try?

M0NsTeRRR · 2024-09-25T11:46:45Z

@dessalines that bug is now fixed in #885

@M0NsTeRRR it turns out the root of this issue is #782

In your example, there isn't any GitLab-related variables in cliff.toml therefore the GitLab data is not fetched for 2.4.0. However, this requirement was removed in 2.5.0 which means the data will be fetched if you set the remote in any way. (e.g. using GITLAB_REPO)

To reproduce the error for 2.4.0:
* Commit: https://gitlab.com/orhunp/git-cliff-debug/-/commit/842f2bb7cbcfb1b80b7674231ca9fa2cdf14c9bb

* Result: https://gitlab.com/orhunp/git-cliff-debug/-/jobs/7913151665
Now, when it comes to the actual permission issue, my guess is CI_JOB_TOKEN does not have access to the API endpoint that is being used. See:
* https://stackoverflow.com/questions/77697649/using-ci-job-token-for-api-call-inside-ci-pipeline-returns-404

* https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html
So the solution would be generating a personal access token and using that instead :) Can you give it a try?

I confirm it works with personal tokens: https://gitlab.com/M0NsTeRRR/git-cliff-debug/-/jobs/7915295382

It would be helpful to know which permission is missing on the CI_JOB_TOKEN to apply the correct rights with the least privilege necessary.

For now, I only need to set the GitLab URL to display a compare link for the full changelog, like :

{% if previous %}\
    {% if previous.version %}
      **Full Changelog**: {{ get_env(name="CI_PROJECT_URL") }}/compare/{{ previous.version }}...{{ version }}
    {% endif %}
{% endif %}

This rewrite avoid to use a personnal token :)
https://gitlab.com/M0NsTeRRR/git-cliff-debug/-/jobs/7915525687

In the documentation : https://git-cliff.org/docs/integration/gitlab#remote

https://gitlab.com/{{ remote.gitlab.owner }}/{{ remote.gitlab.repo }}/-/tags/{{ version }}

This could be written as {{ get_env(name="CI_PROJECT_URL") }}/-/tags/{{ version }} to avoid needing a personal token.

orhun · 2024-09-26T08:24:18Z

Very nice, thanks for debugging :)

It would be helpful to know which permission is missing on the CI_JOB_TOKEN to apply the correct rights with the least privilege necessary.

Not sure if we can tweak the permissions on that token but yeah. It needs to be documented somewhere.

This could be written as

Awesome, added to tips & tricks :)
dfe4459

M0NsTeRRR added the bug Something isn't working label Sep 17, 2024

M0NsTeRRR assigned orhun Sep 17, 2024

orhun mentioned this issue Sep 25, 2024

fix(remote): avoid setting multiple remotes #885

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gitlab regression in 2.5.0 #863

Gitlab regression in 2.5.0 #863

M0NsTeRRR commented Sep 17, 2024

welcome bot commented Sep 17, 2024

orhun commented Sep 18, 2024

orhun commented Sep 18, 2024

M0NsTeRRR commented Sep 18, 2024 •

edited

Loading

orhun commented Sep 21, 2024

dessalines commented Sep 22, 2024

M0NsTeRRR commented Sep 22, 2024 •

edited

Loading

orhun commented Sep 23, 2024

dessalines commented Sep 23, 2024 •

edited

Loading

M0NsTeRRR commented Sep 24, 2024 •

edited

Loading

orhun commented Sep 25, 2024

M0NsTeRRR commented Sep 25, 2024 •

edited

Loading

orhun commented Sep 26, 2024

Gitlab regression in 2.5.0 #863

Gitlab regression in 2.5.0 #863

Comments

M0NsTeRRR commented Sep 17, 2024

Is there an existing issue for this?

Description of the bug

Steps To Reproduce

Expected behavior

Screenshots / Logs

Software information

Additional context

welcome bot commented Sep 17, 2024

orhun commented Sep 18, 2024

orhun commented Sep 18, 2024

M0NsTeRRR commented Sep 18, 2024 • edited Loading

orhun commented Sep 21, 2024

dessalines commented Sep 22, 2024

M0NsTeRRR commented Sep 22, 2024 • edited Loading

orhun commented Sep 23, 2024

dessalines commented Sep 23, 2024 • edited Loading

M0NsTeRRR commented Sep 24, 2024 • edited Loading

orhun commented Sep 25, 2024

M0NsTeRRR commented Sep 25, 2024 • edited Loading

orhun commented Sep 26, 2024

M0NsTeRRR commented Sep 18, 2024 •

edited

Loading

M0NsTeRRR commented Sep 22, 2024 •

edited

Loading

dessalines commented Sep 23, 2024 •

edited

Loading

M0NsTeRRR commented Sep 24, 2024 •

edited

Loading

M0NsTeRRR commented Sep 25, 2024 •

edited

Loading