Skip to content
/ MemoryKV Public

A KV in memory database built in Rust for learning purposes

Notifications You must be signed in to change notification settings

auyer/MemoryKV

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

71 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MemoryKV

I like to learn thing by doing them. This is a in memory Key Value store in Rust. It also has a Write Ahead Log served with Websockets.

About this project

What I wanted to create:

  • In memory database with a hashmap structure
  • Using concurrent patterns
  • In the Rust programming language
  • With a live feed of changes (it felt like a cool demo for my personal site)

I started implementing an http web server using Axum. I did this because I am used to working with http and rest interfaces. I might implement my own connection protocol, but this will be fine for now. Since I was using Axum I also added a /metrics endpoint for Prometheus.

The "database" part I created with a simple HashMap, and protected it behind a RWMutex.

The "WAL/live feed" part was implemented internally using a Broadcast channel. The http part exposes this as a websocket.

All of this is shared between threads (or tokio tasks) with Arc (Atomic Atomically Reference Counted pointers).

A sample of the WAL received by a WebSocket client:

23:42 23.17 Heartbeat from 127.0.0.1:36460
23:42 14.02 Read { key: pre:fix:for_key_value:perfect}
23:42 03.70 List
23:42 03.17 Heartbeat from 127.0.0.1:53374
23:41 29.96 Read { key: pre:demo:key}
23:41 23.17 Heartbeat from 127.0.0.1:36460
23:41 17.34 ListPrefix { prefix: pre}
23:41 05.19 Insert { key: pre:demo:key, value: {"source": "CloudFlare Worker", "type": "WASM"} }
23:41 03.19 Heartbeat from 127.0.0.1:53374
23:41 03.18 Connected to ws://localhost:8080/wal
23:41 03.17 Connecting to ws://localhost:8080/wal

The live demo is available in my personal website: https://rcpassos.me/projects/kv And the source for that is here: https://github.com/auyer/auyer.github.io/blob/main/src/lib/services.js

SDKs

I am also creating SDKs for this project using different languages. Since the server uses REST with HTTP for the database interface, this is the main part that needs to be implemented in the SDKs. The Live Feed is implemented using WebSockets, and is a less important feature.

Feature/SDK C Rust
Rest Endpoints x
Live Feed (WS)

About

A KV in memory database built in Rust for learning purposes

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published