Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MB-57871: Recover panic by GetCardinality #259

Closed
wants to merge 1 commit into from
Closed

Conversation

moshaad7
Copy link
Contributor

@moshaad7 moshaad7 commented Sep 3, 2024

invalid memory address or nil pointer dereference
 
goroutine 266940 [running]:
github.com/blevesearch/bleve/v2/index/scorch.(*IndexSnapshotTermFieldReader).Count.func1()
	/home/couchbase/jenkins/workspace/toy-unix-simple/godeps/src/github.com/blevesearch/bleve/index/scorch/snapshot_index_tfr.go:181 +0x85
panic({0xf47c00, 0x1e2e650})
	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.18.7/go/src/runtime/panic.go:838 +0x207
github.com/RoaringBitmap/roaring.(*Bitmap).GetCardinality(...)
	/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/!roaring!bitmap/[email protected]/roaring.go:667
github.com/blevesearch/zapx/v15.(*PostingsList).Count(0xc00f171680?)
	/home/couchbase/jenkins/workspace/toy-unix-simple/godeps/src/github.com/blevesearch/zapx/posting.go:240 +0xd4
github.com/blevesearch/bleve/v2/index/scorch.(*IndexSnapshotTermFieldReader).Count(0xc01c9a0d60?)
	/home/couchbase/jenkins/workspace/toy-unix-simple/godeps/src/github.com/blevesearch/bleve/index/scorch/snapshot_index_tfr.go:191 +0xe3
// GetCardinality returns the number of integers contained in the bitmap
func (rb *Bitmap) GetCardinality() uint64 {
	size := uint64(0)
	for _, c := range rb.highlowcontainer.containers {
		size += uint64(c.getCardinality())
	}
	return size
}
c.getCardinality panicked because c was nil

We couldn't determine the data and operations that lead our bitmap to this situation where once of the container was nil.
Adding panic recover method to log bitmap data and stats in order to position us better in case this rare bug strikes again.

posting.go Outdated
log.Printf("zapx: PostingsList.Count() bitmap stats: %+v",
p.postings.Stats())
log.Printf("zapx: PostingsList.Count() bitmap content: %+v",
p.postings.String())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@moshaad7 would you consolidate this all into a single log line? Also would you share a sample of the Stats() and String() output in the commit message.

+ roaring.Bitmap's GetCardinality method is causing panic.
+ Panic logs are indicating towards a roaring container being nil.
+ Which shouldn't be the case and we aren't able to reproduce this to
  know what operations on bitap lead us to this state.

+ This change intends to add panic recovery and logging bitmap related
  stats in case of panic to better equip us in reasoning about the panic.
@moshaad7
Copy link
Contributor Author

Closing it as bitmap Stat collection will also fail in case of a presence of a nil container in it.
My apologies for missing this detail

Adding relevant code sections here

// GetCardinality returns the number of integers contained in the bitmap
func (rb *Bitmap) GetCardinality() uint64 {
	size := uint64(0)
	for _, c := range rb.highlowcontainer.containers {
		size += uint64(c.getCardinality())
	}
	return size
}
// Stats returns details on container type usage in a Statistics struct.
func (rb *Bitmap) Stats() Statistics {
	stats := Statistics{}
	stats.Containers = uint64(len(rb.highlowcontainer.containers))
	for _, c := range rb.highlowcontainer.containers {
		stats.Cardinality += uint64(c.getCardinality())
		...
	}
	return stats
}

@moshaad7 moshaad7 closed this Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants