Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

createMDTChangelogUsersMetrics error after each metrics query #36

Open
chadgent opened this issue Aug 1, 2024 · 3 comments
Open

createMDTChangelogUsersMetrics error after each metrics query #36

chadgent opened this issue Aug 1, 2024 · 3 comments

Comments

@chadgent
Copy link

chadgent commented Aug 1, 2024

Is anyone else having this issue using this exporter? After compiling every time I query /metrics it generates the below error.

source "lctl" failed after x seconds - github.com/GSI-HPC/lustre_exporter/sources.(*lustreLctlSource).createMDTChangelogUsersMetrics-fm - exit status2

We query metrics every 15 seconds so this tends to fill up our log files.

Thanks

@gabrieleiannetti
Copy link

Hi @chadgent,

indeed, the exporter lacks on documentation about which metrics are exported when...

As the metric MDTChangelogUsersMetrics implies, it is about the Metadata Target - Changelog Users.

This means, the processing of that metric has been activated for that proper collector collector.mdt (more technically it is --collector.mdt=extended).

If you are running the exporter on a non Metadata Server you should deactivate that collector with --collector.mdt=disable. I would suggest just to activate the proper collector for that Lustre machine type. The exporter does not recognize the machine type by itself and also will not know how detailed a collector should export metrics.

Please refer to the Running Flags of the exporter.

Indeed, the default for all running flags for collectors is set to extended.
That will trigger the error on non Lustre MDS machines.

I will create an separate issue for improving that situation here and link to this one.

Best
Gabriele

@chadgent
Copy link
Author

chadgent commented Aug 2, 2024

Sounds good and thanks for the follow up.

@severgun
Copy link

I have same error on MDS node.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants