This docker compose is to enable Clickhouse on-prem deployment with Zookeeper and Multi-Replica Clickhouse with storage as Wasabi/S3 buckets
Wasabi is extremely low-cost compared to AWS S3 and does not charge any egress fee!
- Zookeeper syncs data between multiple clickhouse replicas
- Clickhouse by defaults uses local disk to store the data and every 10s sync to Wasabi/S3.
Tip
You can also update the storage config tiered for s3 as "cold" and use a TTL table in clickhouse to sync every few seconds.
docker compose up -d
- Change your Bucket endpoint inside the
clickhouse-storage.xml
<endpoint>https://s3.wasabisys.com/WASABI_BUCKET_NAME/data/</endpoint>
<access_key_id>WASABI_ACCESS_KEY_ID</access_key_id>
<secret_access_key>WASABI_SECRET_ACCESS_KEY</secret_access_key>
- Set the Username and Password for Clickhouse in
docker-compose.yml
environment:
- CLICKHOUSE_USER=<USERNAME>
- CLICKHOUSE_PASSWORD=<PASSWORD>
- Add multi-replicas/shards in
cluster-config.xml
<shard>
<replica>
<host>clickhouse-2</host>
<port>9000</port>
</replica>
</shard>