DBJ SQLite Hash Table Research

© 2017 - 2021 by [email protected] & dbj.systems uk ltd.

Licence -- https://dbj.org/license_dbj

---

Why are we here?

The idea is to use the SQLite in memory database for Hash Tables implementations. If interested you are very welcome.

This folder is ISO C code only.

Operational Environment

• Windows 10 Pro x64
• Clone the MACHINE_WIDE to your D: drive.
  • Hint: Use subst to "make" D: drive if you do not have one
  • SQLite 3 is part of that too
• For SQLite databases management we recommend SQLite Studio
• Please install Visual Studio 2019 with Clang
• There are three VStudio projects in this folder
  • each is best experienced by following it through the Visual Studio debugger

What is what

File	Description
benchmarking.c	This is from where the actual original work has started. Study that one first.
dbhash.c	Sampler. Hashing the whole database. Taken from SQLite dev tree.
sha1.c	SQLite "extension" sampler. sha1 hash function. Taken from SQLite dev tree.

On The Road (not again)

aka "the Roadmap"

For full functionality one needs an "handle" (just a "number") to interface with hash tables feasibly. Unlike pointers, handles can be checked if they are valid. Handle is implemented as a result of the hash function

// general unique values HT storing algorithm
auto handle = hash_function(value_to_store) ;
if ( ! already_stored(handle) )
      store(value_to_store) ;
//to the caller
return handle;
// there are other big issues around that 
// we will talk later

User/caller is given that handle, not some pointer. Repeat: Handlers not pointers.

General aim is to develop SQLite extension that will aid creating hash tables, which basically means several hashing functions:

• Paul Hsieh
• Jenkins
• there are others although it is a moot point why would "the others" be used

So far

So good. SQLite HT (in memory) seems rather fast (hash functions are not used, yet). Keep in mind performance is not the only sign of quality. There is also a question of space aka size.

Even without testing one can be reasonably sure SQLite DB file (not :memory:) supported HT implementation can store gigabytes of strings or some general data. Persistent and in a very safe manner too. Put a "micro service" in front of that and "you can fly".

This is far from finished and documented. Stay tuned.

---

Contact : [email protected] | Skype