Is it possible to directly use Cypher based on existing node and edge tables without creating nodes or edges by using "CREATE"?

[LeSaRDe]

Dear Experts,

I would like to ask a naive question. Here is the scenario. Suppose that I have created a node table (with some node properties) and an edge table (with src, trg and some edge properties) for a graph in Postgres without using AGE, and also I have loaded in the actual data for this graph which is very large (with millions of nodes and billions of edges). Now, I would like to use Cypher as my query language to compose queries on this graph. Apparently, I did NOT create the nodes or edges using "CREATE" provided by Cypher, and I prefer to NOT using "CREATE" in any sense. Though, somehow I want to directly use Cypher over the two tables I created for nodes and edges. My question is whether or not my idea is feasible? And if it is feasible, how should I do it?

Thank you!

[pdpotter]

I'm not sure if it is possible to use Cypher to query data stored in regular tables.

There has very recently been a commit (6279c10) enabling GIN indeces on properties, making it possible to create edges a lot faster (~10 000 inserts per second from a python application on a local VM and probably even faster when directly using data from a database table using a mixture of SQL and Cypher). Maybe it is worth reconsidering the "CREATE" option.

When searching for a quick way to insert edges between existing nodes, I found out that it is possible to only use CREATE for the first edge (after which Apache age sets up everything that it needs to afterwards query)

SELECT * FROM cypher('graph_name', $$
MATCH (d:LabelA {id: 1}), (r:LabelB {id: 2})
CREATE (d)-[:RelationA {prop: 'value'}]->(r)
$$) as (a agtype);

and add all other edges using a regular insert.

INSERT INTO graph_name.RelationA (start_id, end_id, properties)
VALUES ($1, $2, $3)

Where $1 are the domain_ids, $2 are the range_ids and $3 are the properties (as json dump).

I think it might be possible to do the same for nodes. Would that be a feasible alternative?

[jrgemignani]

The tables that are used by AGE are maintained by AGE and follow a particular form for vertices and edges. In order for AGE to use your graph tables instead of its own, you would need to set your tables up to look exactly like AGE's tables for said graph.

But, that would be fixing your graph to a specific set of versions of AGE. If we were to decide to adjust the format of those tables and how they work, it would break your ability to use them.

[jrgemignani]

If we were to add functionality to allow this to happen, this would require some standardized format for the input vertex and edge tables. The problem with this approach is that there are many ways to represent such data as tables. Additionally, suppose that we added the ability to point to those tables, what about queries that need to write data? Where should that data go?

So, while I understand your desire to make AGE more flexible with external data, this unfortunately poses a lot of technical issues that would need to be addressed first.

I hope this is helpful.

john

[JoshInnis]

Heres an example, https://github.com/JoshInnis/AGE-Machine-Learning/blob/main/datasets/reddit-hyperlinks/import.sql

use the create_vlabel and elabel functions instead of line 7 and 8.