What's the difference between mimir and thanos ?

[kwenzh]

Is your feature request related to a problem? Please describe.

Our team has recently been looking for a highly available solution for prometheus， A lot of them are thanos， it also contains: query，query-frontend，store，compactor and object bucket。thanos is very similar to mimir。
so i want to know the diffent between thanos and mimir。 and mimir how to reslove Problems that tsdb cannot be scaled？

thanos： https://thanos.io/tip/thanos/getting-started.md/

[kwenzh]

the store-gateway of mimir， can it be Multiple instances？

[kwenzh]

Store-gateway both support sharding and replication. The default configuration runs store-gateway with a replication factor of 3, which means each TSDB block is replicated across to 3 instances. If you run more than 3 instances, then TSDB blocks are sharded across the replicas, so each instance will keep a subset of the data, not all the data (otherwise wouldn't scale).

i notice these params:
-blocks-storage.tsdb.retention-period if i want query a long time metric data, such as : 3 months or half of years, set it -blocks-storage.tsdb.retention-period=90d, 180d ? Isn't the data in the bucket kept permanently by default?

[kwenzh]

Store-gateway both support sharding and replication. The default configuration runs store-gateway with a replication factor of 3, which means each TSDB block is replicated across to 3 instances. If you run more than 3 instances, then TSDB blocks are sharded across the replicas, so each instance will keep a subset of the data, not all the data (otherwise wouldn't scale).

i notice these params: -blocks-storage.tsdb.retention-period if i want query a long time metric data, such as : 3 months or half of years, set it -blocks-storage.tsdb.retention-period=90d, 180d ? Isn't the data in the bucket kept permanently by default?

I think I misunderstood this parameter, it is control the ingest save tsdb period, right?

[pracucci]

i notice these params:
-blocks-storage.tsdb.retention-period if i want query a long time metric data, such as : 3 months or half of years, set it -blocks-storage.tsdb.retention-period=90d, 180d ? Isn't the data in the bucket kept permanently by default?

No. The -blocks-storage.tsdb.retention-period is just the period we keep TSDB blocks in the ingesters. Don't change it. Blocks in the long-term storage will be kept forever, unless you configure the retention explicitly using:

# Delete blocks containing samples older than the specified retention period.
# Also used by query-frontend to avoid querying beyond the retention period. 0
# to disable.
# CLI flag: -compactor.blocks-retention-period
[compactor_blocks_retention_period: <duration> | default = 0s]

[kwenzh]

i notice these params:
-blocks-storage.tsdb.retention-period if i want query a long time metric data, such as : 3 months or half of years, set it -blocks-storage.tsdb.retention-period=90d, 180d ? Isn't the data in the bucket kept permanently by default?

No. The -blocks-storage.tsdb.retention-period is just the period we keep TSDB blocks in the ingesters. Don't change it. Blocks in the long-term storage will be kept forever, unless you configure the retention explicitly using:

# Delete blocks containing samples older than the specified retention period.
# Also used by query-frontend to avoid querying beyond the retention period. 0
# to disable.
# CLI flag: -compactor.blocks-retention-period
[compactor_blocks_retention_period: <duration> | default = 0s]

ok! get it ,

it is period during which data is saved ingest locally ?

[pracucci]

it is period during which data is saved ingest locally ?

The -blocks-storage.tsdb.retention-period is how long to keep the most recent data in the ingesters.

[pracucci]

so i want to know the diffent between thanos and mimir

Thanos (when deployed with the receiver and not the sidecar) has an architecture similar to Mimir (see Mimir architecture doc). Both architectures are microservices-based. Mimir supports also a deployment mode called monolithic, where all microservices run in the same process (see deployment modes doc).

Mimir started as a fork of Cortex. Cortex and Thanos share some code (shipper, store-gateway, compactor), so from an implementation perspective some components like the store-gateway are quite similar between Cortex and Thanos. Over the time Mimir decoupled from Cortex and Thanos (for example, we're not using Thanos code anymore, except for the object storage clients), and introduced several changes with regards to scalability and performance improvements.

At the same time, Cortex and Thanos evolved too (several improvements there as well), so it's quite difficult to compare the two just "on paper", and any comment I may make on this topic could be biased (I know Mimir quite well, I don't follow much what happens in Thanos).

That being said, I would recommend you to join #mimir Slack channel and ask the same question there. I'm aware of several people from the community which has moved from Thanos or Cortex to Mimir, so you can ask them any unbiased feedback.

and mimir how to reslove Problems that tsdb cannot be scaled？

Mimir uses a different compactor algorithm called split-and-merge compactor which allow us to overcome the TSDB index limitations (e.g. max 64GB size for the index, or 4GB for each index section). Learn more here:

• https://grafana.com/docs/mimir/latest/operators-guide/architecture/components/compactor/#compaction-algorithm
• https://grafana.com/blog/2022/04/19/how-grafana-mimirs-split-and-merge-compactor-enables-scaling-metrics-to-1-billion-active-series/

Mimir has been successfully tested up to 1 billion active series for a single tenant (before replication, so 3 billions after replication). Learn more here:

• https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-prometheus-tsdb-grafana-mimir-to-1-billion-active-series/

[kwenzh]

thanks you very much, Sure enough, similar to thanos and improved and optimized,

our prometheus indicators mainly consisted of not having many tags for the same series,

such as pulsar cluster monitoring, which had multiple topic tags for one series, resulting in many series, there are One hundred thousand topic partition, millions of them collected series at a time，
eg:

pulsar_rate_in{topic="xxxx"}

if you query with a long time range in grafana, it will be versy slow maybe timeout

if store-gateway is support Distributed extension， These problems should improve significantly

[kwenzh]

@pracucci how to join https://slack.grafana.com ? it seems need to be invited

[pracucci]

Just insert your email and click the "Get invite" button. You will get an invite directly to your email. It's no automatic.

[kwenzh]

Just insert your email and click the "Get invite" button. You will get an invite directly to your email. It's no automatic.

it need a mail address with @grafana.com ending