Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storage): GCS backend using thanos.io/objstore #11132

Merged
merged 54 commits into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
326d994
GCP implementation using thanos.io/objstore
JoaoBraveCoding Nov 3, 2023
5960741
Implemented all ObjectClient methods
JoaoBraveCoding Nov 8, 2023
9878c97
Cleaned up code
JoaoBraveCoding Nov 9, 2023
f6ea86d
Change Iter options to be more standard
JoaoBraveCoding Nov 9, 2023
0c88216
Allow support for ruler to use thanos/obj storage
JoaoBraveCoding Nov 9, 2023
184ea74
Add thanos/objstore to common configuration
JoaoBraveCoding Nov 16, 2023
e816a3c
Fix metric registration by solving the "component" label conflict
JoaoBraveCoding Nov 21, 2023
1af852e
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Nov 24, 2023
19f610c
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Mar 14, 2024
80333a4
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Mar 18, 2024
32a9e4c
feat(storage): implements hedging for thanos gcs client
JoaoBraveCoding Mar 18, 2024
1bf07cd
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Mar 21, 2024
1a0da8b
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Apr 19, 2024
6728e2f
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Jul 1, 2024
3a2fe9e
Revert func signature of NewObjectClient and introduce NewObjectClientV2
JoaoBraveCoding Jul 5, 2024
17ee4ec
Merge branch 'main' into log-4550-gcp
kavirajk Jul 12, 2024
4451f23
fix the putObject api arguments
kavirajk Jul 12, 2024
919d04f
Merge branch 'main' into log-4550-gcp
kavirajk Jul 22, 2024
f051abf
vendor thanos with chunk-size patch
kavirajk Jul 23, 2024
ea17fc5
support chunk-buffer-size
kavirajk Jul 23, 2024
3e5e772
Merge branch 'main' into log-4550-gcp
kavirajk Jul 30, 2024
e0d5a93
implement `GetRange` to satisfy the interface
kavirajk Jul 30, 2024
2074894
`make doc` add new `chunk-buffer-size` to thanos GCS
kavirajk Jul 30, 2024
b86c4d7
Merge branch 'main' into log-4550-gcp
kavirajk Jul 30, 2024
1060422
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Sep 16, 2024
e0dc5aa
Start using objstore.WrapWith to avoid duplicate metric registration
JoaoBraveCoding Sep 20, 2024
a3b367c
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Sep 20, 2024
3fef0bf
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Sep 30, 2024
6e98e22
chore: add ObjectExistsWithSize to respect the interface
JoaoBraveCoding Sep 30, 2024
e4fd02d
Merge branch 'main' into log-4550-gcp
JoaoBraveCoding Oct 4, 2024
e908566
fix: update interface to GetAttributes
JoaoBraveCoding Oct 7, 2024
ca835ad
use global thanos metrics and other changes (#1)
ashwanthgoli Oct 9, 2024
36de56f
use existing NewObjectClient method
ashwanthgoli Oct 9, 2024
b057471
fixup! use existing NewObjectClient method
ashwanthgoli Oct 9, 2024
f120df2
fix tests
ashwanthgoli Oct 9, 2024
225866f
wire http conf + transport for gcs correctly
ashwanthgoli Oct 9, 2024
7ba4c3b
remove backend field, loki supports multiple backends (per period)
ashwanthgoli Oct 10, 2024
9977d5b
fixup! remove backend field, loki supports multiple backends (per per…
ashwanthgoli Oct 10, 2024
0baea5a
update docs + config cleanup
ashwanthgoli Oct 15, 2024
df50926
move storage backend to root block
ashwanthgoli Oct 15, 2024
0aa6c3c
fixup! move storage backend to root block
ashwanthgoli Oct 15, 2024
5a1d2e5
metric reg cleanup
ashwanthgoli Oct 15, 2024
a0478d0
remove sse limits, add back in s3 pr
ashwanthgoli Oct 15, 2024
529f1d6
use objstore.TryToGetSize
ashwanthgoli Oct 15, 2024
5788eec
nits
ashwanthgoli Oct 15, 2024
f473928
fix compile + update docs
ashwanthgoli Oct 15, 2024
3da88e5
Merge branch 'main' into log-4550-gcp
ashwanthgoli Oct 15, 2024
1e6c079
go mod tidy & vendor
ashwanthgoli Oct 15, 2024
66dc091
fixup! go mod tidy & vendor
ashwanthgoli Oct 15, 2024
db57fca
fix lint
ashwanthgoli Oct 16, 2024
6050e0b
add tests
ashwanthgoli Oct 17, 2024
45d33c3
update docs and comments
ashwanthgoli Oct 18, 2024
d7d0243
Merge branch 'main' into log-4550-gcp
ashwanthgoli Oct 18, 2024
ef26a40
add back after fixing doc-gen
ashwanthgoli Oct 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/sources/shared/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1779,6 +1779,17 @@ storage:
# CLI flag: -common.storage.congestion-control.hedge.strategy
[strategy: <string> | default = ""]

object_store:
# The gcs_storage_backend block configures the connection to Google Cloud
# Storage object storage backend.
# The CLI flags prefix for this block configuration is: common.storage
[gcs: <gcs_storage_backend>]

# Prefix for all objects stored in the backend storage. For simplicity, it
# may only contain digits and English alphabet letters.
# CLI flag: -common.storage.object-store.storage-prefix
[storage_prefix: <string> | default = ""]

[persist_tokens: <boolean>]

[replication_factor: <int>]
Expand Down Expand Up @@ -2544,6 +2555,35 @@ The `frontend_worker` configures the worker - running within the Loki querier -
[query_scheduler_grpc_client: <grpc_client>]
```

### gcs_storage_backend

The `gcs_storage_backend` block configures the connection to Google Cloud Storage object storage backend.

```yaml
# GCS bucket name
# CLI flag: -<prefix>.object-store.gcs.bucket-name
[bucket_name: <string> | default = ""]

# JSON either from a Google Developers Console client_credentials.json file, or
# a Google Developers service account key. Needs to be valid JSON, not a
# filesystem path. If empty, fallback to Google default logic:
# 1. A JSON file whose path is specified by the GOOGLE_APPLICATION_CREDENTIALS
# environment variable. For workload identity federation, refer to
# https://cloud.google.com/iam/docs/how-to#using-workload-identity-federation on
# how to generate the JSON configuration file for on-prem/non-Google cloud
# platforms.
# 2. A JSON file in a location known to the gcloud command-line tool:
# $HOME/.config/gcloud/application_default_credentials.json.
# 3. On Google Compute Engine it fetches credentials from the metadata server.
# CLI flag: -<prefix>.object-store.gcs.service-account
[service_account: <string> | default = ""]

# The maximum size of the buffer that GCS client for a single PUT request. 0 to
# disable buffering.
# CLI flag: -<prefix>.object-store.gcs.chunk-buffer-size
[chunk_buffer_size: <int> | default = 0]
```

### gcs_storage_config

The `gcs_storage_config` block configures the connection to Google Cloud Storage object storage backend. The supported CLI flags `<prefix>` used to reference this configuration block are:
Expand Down Expand Up @@ -3685,6 +3725,20 @@ ruler_remote_write_sigv4_config:
# CLI flag: -limits.per-user-override-period
[per_tenant_override_period: <duration> | default = 10s]

# S3 server-side encryption type. Required to enable server-side encryption
# overrides for a specific tenant. If not set, the default S3 client settings
# are used.
[s3_sse_type: <string> | default = ""]

# S3 server-side encryption KMS Key ID. Ignored if the SSE type override is not
# set.
[s3_sse_kms_key_id: <string> | default = ""]

# S3 server-side encryption KMS encryption context. If unset and the key ID
# override is set, the encryption context will not be provided to S3. Ignored if
# the SSE type override is not set.
[s3_sse_kms_encryption_context: <string> | default = ""]

ashwanthgoli marked this conversation as resolved.
Show resolved Hide resolved
# Deprecated: Use deletion_mode per tenant configuration instead.
[allow_deletes: <boolean>]

Expand Down Expand Up @@ -5622,6 +5676,20 @@ congestion_control:
# CLI flag: -store.max-parallel-get-chunk
[max_parallel_get_chunk: <int> | default = 150]

# Enable the thanos.io/objstore to be the backend for object storage
# CLI flag: -use-thanos-objstore
[use_thanos_objstore: <boolean> | default = false]

object_store:
# The gcs_storage_backend block configures the connection to Google Cloud
# Storage object storage backend.
[gcs: <gcs_storage_backend>]

# Prefix for all objects stored in the backend storage. For simplicity, it may
# only contain digits and English alphabet letters.
# CLI flag: -object-store.storage-prefix
[storage_prefix: <string> | default = ""]

# The maximum number of chunks to fetch per batch.
# CLI flag: -store.max-chunk-batch-size
[max_chunk_batch_size: <int> | default = 50]
Expand Down
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ require (
golang.org/x/sys v0.25.0
golang.org/x/time v0.6.0
google.golang.org/api v0.193.0
google.golang.org/grpc v1.65.0
google.golang.org/grpc v1.66.0
gopkg.in/alecthomas/kingpin.v2 v2.2.6
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.1
Expand Down Expand Up @@ -139,7 +139,7 @@ require (
github.com/richardartoul/molecule v1.0.0
github.com/schollz/progressbar/v3 v3.14.6
github.com/shirou/gopsutil/v4 v4.24.9
github.com/thanos-io/objstore v0.0.0-20240818203309-0363dadfdfb1
github.com/thanos-io/objstore v0.0.0-20241015070247-5f04b8b0b52a
github.com/twmb/franz-go v1.17.1
github.com/twmb/franz-go/pkg/kadm v1.13.0
github.com/twmb/franz-go/pkg/kfake v0.0.0-20240821035758-b77dd13e2bfa
Expand Down
8 changes: 4 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1793,8 +1793,8 @@ github.com/tedsuo/ifrit v0.0.0-20191009134036-9a97d0632f00/go.mod h1:eyZnKCc955u
github.com/tencentcloud/tencentcloud-sdk-go v1.0.162/go.mod h1:asUz5BPXxgoPGaRgZaVm1iGcUAuHyYUo1nXqKa83cvI=
github.com/tencentyun/cos-go-sdk-v5 v0.7.40 h1:W6vDGKCHe4wBACI1d2UgE6+50sJFhRWU4O8IB2ozzxM=
github.com/tencentyun/cos-go-sdk-v5 v0.7.40/go.mod h1:4dCEtLHGh8QPxHEkgq+nFaky7yZxQuYwgSJM87icDaw=
github.com/thanos-io/objstore v0.0.0-20240818203309-0363dadfdfb1 h1:z0v9BB/p7s4J6R//+0a5M3wCld8KzNjrGRLIwXfrAZk=
github.com/thanos-io/objstore v0.0.0-20240818203309-0363dadfdfb1/go.mod h1:3ukSkG4rIRUGkKM4oIz+BSuUx2e3RlQVVv3Cc3W+Tv4=
github.com/thanos-io/objstore v0.0.0-20241015070247-5f04b8b0b52a h1:0etzAoXPjVVUnscliA+xy8vWdE88jbvhcVMr1rVHc60=
github.com/thanos-io/objstore v0.0.0-20241015070247-5f04b8b0b52a/go.mod h1:/ZMUxFcp/nT6oYV5WslH9k07NU/+86+aibgZRmMMr/4=
github.com/tidwall/gjson v1.6.0/go.mod h1:P256ACg0Mn+j1RXIDXoss50DeIABTYK1PULOJHhxOls=
github.com/tidwall/match v1.0.1/go.mod h1:LujAq0jyVjBy028G1WhWfIzbpQfMO8bBZ6Tyb0+pL9E=
github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
Expand Down Expand Up @@ -2657,8 +2657,8 @@ google.golang.org/grpc v1.46.2/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACu
google.golang.org/grpc v1.47.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk=
google.golang.org/grpc v1.48.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk=
google.golang.org/grpc v1.49.0/go.mod h1:ZgQEeidpAuNRZ8iRrlBKXZQP1ghovWIVhdJRyCDK+GI=
google.golang.org/grpc v1.65.0 h1:bs/cUb4lp1G5iImFFd3u5ixQzweKizoZJAwBNLR42lc=
google.golang.org/grpc v1.65.0/go.mod h1:WgYC2ypjlB0EiQi6wdKixMqukr6lBc0Vo+oOgjrM5ZQ=
google.golang.org/grpc v1.66.0 h1:DibZuoBznOxbDQxRINckZcUvnCEvrW9pcWIE2yF9r1c=
google.golang.org/grpc v1.66.0/go.mod h1:s3/l6xSSCURdVfAnL+TqCNMyTDAGN6+lZeVxnZR128Y=
google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
Expand Down
2 changes: 1 addition & 1 deletion pkg/bloombuild/builder/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ func New(
builderID := uuid.NewString()
logger = log.With(logger, "builder_id", builderID)

tsdbStore, err := common.NewTSDBStores(schemaCfg, storeCfg, storageMetrics, logger)
tsdbStore, err := common.NewTSDBStores("bloom-builder", schemaCfg, storeCfg, storageMetrics, logger)
if err != nil {
return nil, fmt.Errorf("error creating TSDB store: %w", err)
}
Expand Down
4 changes: 2 additions & 2 deletions pkg/bloombuild/common/tsdb.go
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ type TSDBStores struct {
}

func NewTSDBStores(
component string,
schemaCfg config.SchemaConfig,
storeCfg baseStore.Config,
clientMetrics baseStore.ClientMetrics,
Expand All @@ -185,8 +186,7 @@ func NewTSDBStores(

for i, cfg := range schemaCfg.Configs {
if cfg.IndexType == types.TSDBType {

c, err := baseStore.NewObjectClient(cfg.ObjectType, storeCfg, clientMetrics)
c, err := baseStore.NewObjectClient(cfg.ObjectType, component, storeCfg, clientMetrics)
if err != nil {
return nil, errors.Wrap(err, "failed to create object client")
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/bloombuild/planner/planner.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ func New(
) (*Planner, error) {
utillog.WarnExperimentalUse("Bloom Planner", logger)

tsdbStore, err := common.NewTSDBStores(schemaCfg, storeCfg, storageMetrics, logger)
tsdbStore, err := common.NewTSDBStores("bloom-planner", schemaCfg, storeCfg, storageMetrics, logger)
if err != nil {
return nil, fmt.Errorf("error creating TSDB store: %w", err)
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/ingester-rf1/objstore/storage.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ func New(
return periodicConfigs[i].From.Time.Before(periodicConfigs[j].From.Time)
})
for _, periodicConfig := range periodicConfigs {
objectClient, err := storage.NewObjectClient(periodicConfig.ObjectType, storageConfig, clientMetrics)
objectClient, err := storage.NewObjectClient(periodicConfig.ObjectType, "storage-rf1", storageConfig, clientMetrics)
if err != nil {
return nil, fmt.Errorf("creating object client for period %s: %w ", periodicConfig.From, err)
}
Expand Down
10 changes: 1 addition & 9 deletions pkg/logcli/query/query.go
Original file line number Diff line number Diff line change
Expand Up @@ -538,15 +538,7 @@ func (q *Query) DoLocalQuery(out output.LogOutput, statistics bool, orgID string
}

func GetObjectClient(store string, conf loki.Config, cm storage.ClientMetrics) (chunk.ObjectClient, error) {
oc, err := storage.NewObjectClient(
store,
conf.StorageConfig,
cm,
)
if err != nil {
return nil, err
}
return oc, nil
return storage.NewObjectClient(store, "logcli-query", conf.StorageConfig, cm)
}

var errNotExists = stdErrors.New("doesn't exist")
Expand Down
4 changes: 4 additions & 0 deletions pkg/loki/common/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"github.com/grafana/dskit/flagext"
"github.com/grafana/dskit/netutil"

"github.com/grafana/loki/v3/pkg/storage/bucket"
"github.com/grafana/loki/v3/pkg/storage/chunk/client/alibaba"
"github.com/grafana/loki/v3/pkg/storage/chunk/client/aws"
"github.com/grafana/loki/v3/pkg/storage/chunk/client/azure"
Expand Down Expand Up @@ -78,6 +79,7 @@ type Storage struct {
Hedging hedging.Config `yaml:"hedging"`
COS ibmcloud.COSConfig `yaml:"cos"`
CongestionControl congestion.Config `yaml:"congestion_control,omitempty"`
ObjectStore bucket.Config `yaml:"object_store"`
}

func (s *Storage) RegisterFlagsWithPrefix(prefix string, f *flag.FlagSet) {
Expand All @@ -91,6 +93,8 @@ func (s *Storage) RegisterFlagsWithPrefix(prefix string, f *flag.FlagSet) {
s.Hedging.RegisterFlagsWithPrefix(prefix, f)
s.COS.RegisterFlagsWithPrefix(prefix, f)
s.CongestionControl.RegisterFlagsWithPrefix(prefix, f)

s.ObjectStore.RegisterFlagsWithPrefix(prefix+"object-store.", f)
}

type FilesystemConfig struct {
Expand Down
8 changes: 8 additions & 0 deletions pkg/loki/config_wrapper.go
Original file line number Diff line number Diff line change
Expand Up @@ -566,6 +566,14 @@ func applyStorageConfig(cfg, defaults *ConfigWrapper) error {
}
}

if !reflect.DeepEqual(cfg.Common.Storage.ObjectStore, defaults.StorageConfig.ObjectStore) {
configsFound++

applyConfig = func(r *ConfigWrapper) {
r.StorageConfig.ObjectStore = r.Common.Storage.ObjectStore
}
}

if configsFound > 1 {
return ErrTooManyStorageConfigs
}
Expand Down
12 changes: 6 additions & 6 deletions pkg/loki/modules.go
Original file line number Diff line number Diff line change
Expand Up @@ -724,8 +724,7 @@ func (t *Loki) initTableManager() (services.Service, error) {
}

reg := prometheus.WrapRegistererWith(prometheus.Labels{"component": "table-manager-store"}, prometheus.DefaultRegisterer)

tableClient, err := storage.NewTableClient(lastConfig.IndexType, *lastConfig, t.Cfg.StorageConfig, t.ClientMetrics, reg, util_log.Logger)
tableClient, err := storage.NewTableClient(lastConfig.IndexType, "table-manager", *lastConfig, t.Cfg.StorageConfig, t.ClientMetrics, reg, util_log.Logger)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -1414,7 +1413,7 @@ func (t *Loki) initCompactor() (services.Service, error) {
continue
}

objectClient, err := storage.NewObjectClient(periodConfig.ObjectType, t.Cfg.StorageConfig, t.ClientMetrics)
objectClient, err := storage.NewObjectClient(periodConfig.ObjectType, "compactor", t.Cfg.StorageConfig, t.ClientMetrics)
if err != nil {
return nil, fmt.Errorf("failed to create object client: %w", err)
}
Expand All @@ -1425,7 +1424,8 @@ func (t *Loki) initCompactor() (services.Service, error) {
var deleteRequestStoreClient client.ObjectClient
if t.Cfg.CompactorConfig.RetentionEnabled {
if deleteStore := t.Cfg.CompactorConfig.DeleteRequestStore; deleteStore != "" {
if deleteRequestStoreClient, err = storage.NewObjectClient(deleteStore, t.Cfg.StorageConfig, t.ClientMetrics); err != nil {
deleteRequestStoreClient, err = storage.NewObjectClient(deleteStore, "delete-store", t.Cfg.StorageConfig, t.ClientMetrics)
if err != nil {
return nil, fmt.Errorf("failed to create delete request store object client: %w", err)
}
} else {
Expand Down Expand Up @@ -1492,7 +1492,7 @@ func (t *Loki) initIndexGateway() (services.Service, error) {
}
tableRange := period.GetIndexTableNumberRange(periodEndTime)

indexClient, err := storage.NewIndexClient(period, tableRange, t.Cfg.StorageConfig, t.Cfg.SchemaConfig, t.Overrides, t.ClientMetrics, shardingStrategy,
indexClient, err := storage.NewIndexClient("index-store", period, tableRange, t.Cfg.StorageConfig, t.Cfg.SchemaConfig, t.Overrides, t.ClientMetrics, shardingStrategy,
prometheus.DefaultRegisterer, log.With(util_log.Logger, "index-store", fmt.Sprintf("%s-%s", period.IndexType, period.From.String())), t.Cfg.MetricsNamespace,
)
if err != nil {
Expand Down Expand Up @@ -1738,7 +1738,7 @@ func (t *Loki) initAnalytics() (services.Service, error) {
return nil, err
}

objectClient, err := storage.NewObjectClient(period.ObjectType, t.Cfg.StorageConfig, t.ClientMetrics)
objectClient, err := storage.NewObjectClient(period.ObjectType, "analytics", t.Cfg.StorageConfig, t.ClientMetrics)
if err != nil {
level.Info(util_log.Logger).Log("msg", "failed to initialize usage report", "err", err)
return nil, nil
Expand Down
3 changes: 1 addition & 2 deletions pkg/ruler/base/storage.go
Original file line number Diff line number Diff line change
Expand Up @@ -136,8 +136,7 @@ func NewRuleStore(ctx context.Context, cfg rulestore.Config, cfgProvider bucket.
if cfg.Backend == local.Name {
return local.NewLocalRulesClient(cfg.Local, loader)
}

bucketClient, err := bucket.NewClient(ctx, cfg.Config, "ruler-storage", logger, reg)
bucketClient, err := bucket.NewClient(cfg.Backend, ctx, cfg.Config, "ruler-storage", logger)
if err != nil {
return nil, err
}
Expand Down
2 changes: 2 additions & 0 deletions pkg/ruler/rulestore/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (
// Config configures a rule store.
type Config struct {
bucket.Config `yaml:",inline"`
Backend string `yaml:"backend"`
ConfigDB client.Config `yaml:"configdb"`
Local local.Config `yaml:"local"`
}
Expand All @@ -26,6 +27,7 @@ func (cfg *Config) RegisterFlags(f *flag.FlagSet) {
cfg.ExtraBackends = []string{configdb.Name, local.Name}
cfg.ConfigDB.RegisterFlagsWithPrefix(prefix, f)
cfg.Local.RegisterFlagsWithPrefix(prefix, f)
f.StringVar(&cfg.Backend, prefix+"backend", "filesystem", "Backend storage to use. Supported backends are: s3, gcs, azure, swift, filesystem.")
cfg.RegisterFlagsWithPrefix(prefix, f)
}

Expand Down
Loading