-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speed up source filtering in neo4j #201
Comments
I noticed this is slow when the full CSKG is loaded, it takes on the order of seconds to get "all nodes in ConceptNet" (an example queries), which suggests it's doing a full scan of the database. |
@gordom6 Did some digging and I think the culprit is the For the "all nodes in ConceptNet" query
the cypher query run by
I also profiled the query which confirms your suspicion that "it's doing a full scan of the database" I think that MATCH then WHERE as opposed to just using MATCH (as shown below) is the source of the slowdown
|
Sounds reasonable. Can you fix it or would you like me to? |
I will give it a shot |
This is still taking ~15 seconds to load the example "nodes from WordNet" query. |
Sources are currently (20200813) modeled as separate nodes in neo4j, with KgNode's connected via :SOURCE. Neo4j doesn't allow you to index relationships like that:
https://community.neo4j.com/t/how-can-i-use-index-in-relationship/1627
Apparently Lucene indices are the way to go:
https://neo4j.com/docs/cypher-manual/current/administration/indexes-for-full-text-search/
Relationship indexes?
I'm also open to remodeling the way we handle sources. We moved to the current model because nodes can't have multi-valued properties.
The text was updated successfully, but these errors were encountered: