Skip to content

High-performance data ingestion for GreptimeDB using QUIC by YoMo

Notifications You must be signed in to change notification settings

yomorun/GreptimeDB-YoMo

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GreptimeDB x Yomo

Boost your GreptimeDB performance with YoMo: Leverage QUIC for rapid and reliable data ingestion.

This repository showcases how to utilize GreptimeDB for data transportation via YoMo. We will use Yomo's fast and reliable QUIC transport to send local IoT or Observability metrics data to a remote function that ingests the data into time-series database, GreptimeDB.

flowchart LR
    greptimedb[(GreptimeDB)]

    subgraph yomo[YoMo]
      sfn
    end

    subgraph Edge
      source
    end

    subgraph Central
      yomo
      greptimedb
    end

    source --> |fa:fa-cloud Line Protocol over YoMo QUIC Transport| sfn
    sfn --> |HTTP| greptimedb

Loading

Run this demo

  1. Clone this reposiotry
git clone https://github.com/yomorun/GreptimeDB-YoMo.git
cd GreptimeDB-YoMo
  1. Install GreptimeDB (This example uses Docker. For additional installation methods, refer to the GreptimeDB documentation).
docker run -p 127.0.0.1:4000-4003:4000-4003 \
-v "$(pwd)/greptimedb:/tmp/greptimedb" \
--name greptime --rm \
greptime/greptimedb:v0.8.2 standalone start \
--http-addr 0.0.0.0:4000 \
--rpc-addr 0.0.0.0:4001 \
--mysql-addr 0.0.0.0:4002 \
--postgres-addr 0.0.0.0:4003
  1. Install YoMo
curl -fsSL https://get.yomo.run | sh
  1. Start yomo zipper service
yomo serve -c config.yaml
  1. Run yomo sfn, sfn bridges GreptimeDB and YoMo
cd sfn && GREPTIMEDB_HTTP_ADDR=localhost:4000 yomo run app.go
  1. Start the log generator, this will generate ingest data in Line Protocol format and send it to the YoMo zipper service over QUIC.
# This sample shows how to monitor the cpu metrics
go run cpu-agent/main.go
# This sample shows how to monitor the gpu metrics
go run gpu-agent/main.go
  1. Verify that the data has been successfully written to GreptimeDB.
curl -X POST \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'sql=select * from monitor' \
http://localhost:4000/v1/sql?db=public
  1. Configure charts for GPU metrics in Greptime Cloud Workbench
image

About

High-performance data ingestion for GreptimeDB using QUIC by YoMo

Topics

Resources

Stars

Watchers

Forks

Contributors 4

  •  
  •  
  •  
  •  

Languages