Skip to content

Commit

Permalink
Use generics and go1.23 iterators (#26)
Browse files Browse the repository at this point in the history
Use generics and go1.23 iterators

- Store specific types of values, using generics to specify type.
- Replace iterators (Walk, WalkPath, Iterator) with go1.23 iterators. This allows ranging over key-value pairs.
- Update README.md
  • Loading branch information
gammazero authored Sep 11, 2024
1 parent 037a743 commit 213b8a1
Show file tree
Hide file tree
Showing 10 changed files with 317 additions and 507 deletions.
32 changes: 16 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,21 @@
[![codecov](https://codecov.io/gh/gammazero/radixtree/branch/master/graph/badge.svg)](https://codecov.io/gh/gammazero/radixtree)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)

Package `radixtree` implements an Adaptive [Radix Tree](https://en.wikipedia.org/wiki/Radix_tree), aka compressed [trie](https://en.wikipedia.org/wiki/Trie) or compact prefix tree. This data structure is useful to quickly lookup data by key, find values whose keys have a common prefix, or find values whose keys are a prefix (i.e. found along the way) of a search key.
Package `radixtree` implements an Adaptive [Radix Tree](https://en.wikipedia.org/wiki/Radix_tree), aka compressed [trie](https://en.wikipedia.org/wiki/Trie) or compact prefix tree. This data structure is useful to quickly lookup data by key, find values whose keys have a common prefix, or find values whose keys are a prefix (i.e. found along the way) of a search key.

It is adaptive in the sense that nodes are not constant size, having only as many children, up to the maximum, as needed to branch to all subtrees. This package implements a radix-256 tree where each key symbol (radix) is a byte, allowing up to 256 possible branches to traverse to the next node.

The implementation is optimized for Get performance and allocate 0 bytes of heap memory for any read operation (Get, Walk, WalkPath, etc.); therefore no garbage to collect. Once a radix tree is built, it can be repeatedly searched quickly. Concurrent searches are safe since these do not modify the data structure. Access is not synchronized (not concurrent safe with writes), allowing the caller to synchronize, if needed, in whatever manner works best for the application.
The implementation is optimized for read performance and does not allocate heap memory for read operation (Get, Iter, IterPath, etc.). Once a radix tree is built, it can be repeatedly searched quickly. Concurrent searches are safe since these do not modify the data structure. Access is not synchronized (not concurrent safe with writes), allowing the caller to synchronize, if needed, in whatever manner works best for the application.

This radix tree offers the following features:

- Efficient: Operations are O(k). Zero memory allocation for all read operations.
- Ordered iteration: Walking and iterating the tree is done in lexical order, making the output deterministic.
- Efficient: Operations are O(key-len). Zero memory allocation for reading items or iteration.
- Ordered iteration: Iterating the tree is done in lexical order, making the output deterministic.
- Store `nil` values: Read operations differentiate between missing and `nil` values.
- Compact: When values are stored using keys that have a common prefix, the common part of the key is only stored once. Consider this when keys are similar to a timestamp, OID, filepath, geohash, network address, etc. Only the minimum number of nodes are kept to branch at the points where keys differ.
- Iterators: An `Iterator` type walks every key-value pair stored in the tree. A `Stepper` type of iterator traverses the tree one specified byte at a time, and is useful for incremental lookup. A Stepper can be copied in order to branch a search and iterate the copies concurrently.
- Iterators: Go 1.23 iterators allow ranging over key-value pairs stored in the tree. Iterators can traverse all key-value pairs, pairs with a key having specified prefix, or pairs along a key-path from root to a specified key.
- A `Stepper` type of iterator traverses the tree one specified byte at a time, and is useful for incremental lookup. A Stepper can be copied in order to branch a search and iterate the copies concurrently.
- Generics: The tree stores the specified type of value without needing to do interface type assertion.

## Install

Expand Down Expand Up @@ -49,21 +51,19 @@ func main() {
}
// Output: Found TOM

// Find all items whose keys start with "tom"
rt.Walk("tom", func(key string, value any) bool {
fmt.Println(value)
return false
})
// Find all items whose keys start with "tom".
for key, value := range rt.IterAt("tom") {
fmt.Println(key, "=", value)
}
// Output:
// TOM
// TOMATO
// TOMMY
// tom = TOM
// tomato = TOMATO
// tommy = TOMMY

// Find all items whose keys are a prefix of "tomato"
rt.WalkPath("tomato", func(key string, value any) bool {
for _, value := range rt.IterPath("tomato") {
fmt.Println(value)
return false
})
}
// Output:
// TOM
// TOMATO
Expand Down
38 changes: 18 additions & 20 deletions bench_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,23 +44,23 @@ func BenchmarkPut(b *testing.B) {
})
}

func BenchmarkWalk(b *testing.B) {
func BenchmarkIter(b *testing.B) {
b.Run("Words", func(b *testing.B) {
benchmarkWalk(b, web2Path)
benchmarkIter(b, web2Path)
})

b.Run("Web2a", func(b *testing.B) {
benchmarkWalk(b, web2aPath)
benchmarkIter(b, web2aPath)
})
}

func BenchmarkWalkPath(b *testing.B) {
func BenchmarkIterPath(b *testing.B) {
b.Run("Words", func(b *testing.B) {
benchmarkWalkPath(b, web2Path)
benchmarkIterPath(b, web2Path)
})

b.Run("Web2a", func(b *testing.B) {
benchmarkWalkPath(b, web2aPath)
benchmarkIterPath(b, web2aPath)
})
}

Expand All @@ -69,7 +69,7 @@ func benchmarkGet(b *testing.B, filePath string) {
if err != nil {
b.Skip(err.Error())
}
tree := new(Tree)
tree := new(Tree[string])
for _, w := range words {
tree.Put(w, w)
}
Expand All @@ -92,19 +92,19 @@ func benchmarkPut(b *testing.B, filePath string) {
b.ResetTimer()
b.ReportAllocs()
for n := 0; n < b.N; n++ {
tree := new(Tree)
tree := new(Tree[string])
for _, w := range words {
tree.Put(w, w)
}
}
}

func benchmarkWalk(b *testing.B, filePath string) {
func benchmarkIter(b *testing.B, filePath string) {
words, err := loadWords(filePath)
if err != nil {
b.Skip(err.Error())
}
tree := new(Tree)
tree := new(Tree[string])
for _, w := range words {
tree.Put(w, w)
}
Expand All @@ -113,22 +113,21 @@ func benchmarkWalk(b *testing.B, filePath string) {
var count int
for n := 0; n < b.N; n++ {
count = 0
tree.Walk("", func(k string, value any) bool {
for range tree.Iter() {
count++
return false
})
}
}
if count != len(words) {
b.Fatalf("Walk wrong count, expected %d got %d", len(words), count)
b.Fatalf("iter wrong count, expected %d got %d", len(words), count)
}
}

func benchmarkWalkPath(b *testing.B, filePath string) {
func benchmarkIterPath(b *testing.B, filePath string) {
words, err := loadWords(filePath)
if err != nil {
b.Skip(err.Error())
}
tree := new(Tree)
tree := new(Tree[string])
for _, w := range words {
tree.Put(w, w)
}
Expand All @@ -137,13 +136,12 @@ func benchmarkWalkPath(b *testing.B, filePath string) {
for n := 0; n < b.N; n++ {
found := false
for _, w := range words {
tree.WalkPath(w, func(key string, value any) bool {
for range tree.IterPath(w) {
found = true
return false
})
}
}
if !found {
b.Fatal("Walk did not find word")
b.Fatal("IterPath did not find word")
}
}
}
Expand Down
34 changes: 16 additions & 18 deletions doc.go
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
/*
Package radixtree implements an Adaptive Radix Tree, aka compressed trie or
compact prefix tree. It is adaptive in the sense that nodes are not constant
size, having as few or many children as needed to branch to all subtrees.
This package implements a radix-256 tree where each key symbol (radix) is a
byte, allowing up to 256 possible branches to traverse to the next node.
The implementation is optimized for Get performance and allocates 0 bytes of
heap memory per Get; therefore no garbage to collect. Once the radix tree is
built, it can be repeatedly searched quickly. Concurrent searches are safe
since these do not modify the radixtree. Access is not synchronized (not
concurrent safe with writes), allowing the caller to synchronize, if needed, in
whatever manner works best for the application.
The API uses string keys, since strings are immutable and therefore it is not
necessary make a copy of the key provided to the radix tree.
*/
// Package radixtree implements an Adaptive Radix Tree, aka compressed trie or
// compact prefix tree. It is adaptive in the sense that nodes are not constant
// size, having as few or many children as needed to branch to all subtrees.
//
// This package implements a radix-256 tree where each key symbol (radix) is a
// byte, allowing up to 256 possible branches to traverse to the next node.
//
// The implementation is optimized for read performance and allocates zero
// bytes of heap memory for read oprations. Once the radix tree is built, it
// can be repeatedly searched quickly. Concurrent searches are safe since these
// do not modify the radixtree. Access is not synchronized (not concurrent safe
// with writes), allowing the caller to synchronize, if needed, in whatever
// manner works best for the application.
//
// The API uses string keys, since strings are immutable and therefore it is
// not necessary make a copy of the key provided to the radix tree.
package radixtree
63 changes: 30 additions & 33 deletions doc_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,74 +6,71 @@ import (
"github.com/gammazero/radixtree"
)

func ExampleTree_Walk() {
rt := radixtree.New()
rt.Put("tomato", "TOMATO")
rt.Put("tom", "TOM")
rt.Put("tommy", "TOMMY")
rt.Put("tornado", "TORNADO")
func ExampleTree_Iter() {
rt := radixtree.New[int]()
rt.Put("mercury", 1)
rt.Put("venus", 2)
rt.Put("earth", 3)
rt.Put("mars", 4)

// Find all items whose keys start with "tom"
rt.Walk("tom", func(key string, value any) bool {
fmt.Println(value)
return false
})
// Find all items that that have a key that is a prefix of "tomato".
for key, value := range rt.Iter() {
fmt.Println(key, "=", value)
}
// Output:
// earth = 3
// mars = 4
// mercury = 1
// venus = 2
}

func ExampleTree_WalkPath() {
rt := radixtree.New()
func ExampleTree_IterAt() {
rt := radixtree.New[string]()
rt.Put("tomato", "TOMATO")
rt.Put("tom", "TOM")
rt.Put("tommy", "TOMMY")
rt.Put("tornado", "TORNADO")

// Find all items that are a prefix of "tomato"
rt.WalkPath("tomato", func(key string, value any) bool {
// Find all items whose keys start with "tom".
for _, value := range rt.IterAt("tom") {
fmt.Println(value)
return false
})
}
// Output:
// TOM
// TOMATO
// TOMMY
}

func ExampleTree_NewIterator() {
rt := radixtree.New()
func ExampleTree_IterPath() {
rt := radixtree.New[string]()
rt.Put("tomato", "TOMATO")
rt.Put("tom", "TOM")
rt.Put("tommy", "TOMMY")
rt.Put("tornado", "TORNADO")

iter := rt.NewIterator()
for {
key, val, done := iter.Next()
if done {
break
}
fmt.Println(key, "=", val)
// Find all items that that have a key that is a prefix of "tomato".
for key, value := range rt.IterPath("tomato") {
fmt.Println(key, "=", value)
}

// Output:
// tom = TOM
// tomato = TOMATO
// tommy = TOMMY
// tornado = TORNADO
}

func ExampleTree_NewStepper() {
rt := radixtree.New()
rt := radixtree.New[string]()
rt.Put("tomato", "TOMATO")
rt.Put("tom", "TOM")
rt.Put("tommy", "TOMMY")
rt.Put("tornado", "TORNADO")

iter := rt.NewStepper()
stepper := rt.NewStepper()
word := "tomato"
for i := range word {
if !iter.Next(word[i]) {
if !stepper.Next(word[i]) {
break
}
if val, ok := iter.Value(); ok {
if val, ok := stepper.Value(); ok {
fmt.Println(val)
}
}
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module github.com/gammazero/radixtree

go 1.21
go 1.23
47 changes: 0 additions & 47 deletions iterator.go

This file was deleted.

Loading

0 comments on commit 213b8a1

Please sign in to comment.