Unable to create an edge to existing vertex

[lokune]

I am able to create the below graph in neo4j using this library - I can create a Person vertex, Email vertex and Domain vertex and the relationships.

Now, let's say the Person vertex is removed, I also remove the Email vertex but leave the Domain vertex. This is how my graph now looks.

Now another Person vertex need to be created. Their email address has same Domain as the existing Domain. I am able to create the new Person vertex and Email vertex. I am also able to retrieve the existing Domain vertex with the following code.

val existingDomainVertex: Vertex = graph.V.has(label = "Domain", key = Key[String]("domain"), predicate = P.eq[String]("www.gmail.com")).toList.head

Just to make sure, existingDomainVertex.toList.size is 1. I can also convert this vertex to a my case class and it has all properties.

The problem is I cannot create an edge to this vertex from the newly created Email vertex.
newEmailVertex --- ("DOMAIN") --> existingDomainVertex does not work.

What might be the issue? This is blocking me on my project because this is going to be a common use case and I have no idea why it works for new vertices by not to existing vertices.

[mpollmeier]

Could you please share a small project that demonstrates this?

…

-----Original Message----- From: Laban Okune <[email protected]> To: mpollmeier/gremlin-scala <[email protected]> Cc: Subscribed <[email protected]> Sent: Sun, 25 Feb 2018 5:55 Subject: [mpollmeier/gremlin-scala] Unable to create an edge to existing vertex (#233) I am able to create the below graph in neo4j using this library - I can create a `Person` vertex, `Email` vertex and `Domain` vertex and the relationships.  Now, let's say the `Person` vertex is removed, I also remove the `Email` vertex but leave the `Domain` vertex. This is how my graph now looks.  Now another `Person` vertex need to be created. Their email address has same `Domain` as the existing `Domain`. I am able to create the new `Person` vertex and `Email` vertex. I am also able to retrieve the existing `Domain` vertex with the following code. `val existingDomainVertex: Vertex = graph.V.has(label = "Domain", key = Key[String]("domain"), predicate = P.eq[String]("www.gmail.com")).toList.head` Just to make sure, `existingDomainVertex.toList.size` is 1. I can also convert this vertex to a my case class and it has all properties. The problem is I cannot create an edge to this vertex from the newly created `Email` vertex. `newEmailVertex --- ("DOMAIN") --> existingDomainVertex` does not work.  What might be the issue? This is blocking me on my project because this is going to be a common use case and I have no idea why it works for new vertices by not to existing vertices.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #233

[lokune]

@mpollmeier thanks a lot for your response. The code snippets shared below. I hope this gives you a sense of what I am talking about. I have removed some extra code that is not related to the question. Kindly look at the comments to get context.

sealed trait RequestEntity

object RequestEntity {
  case class Person(uuid: Option[String],
                    full_name: String,
                    email: Option[String]) extends RequestEntity
}

sealed trait DatabaseEntity {
  def uuid: Option[String]
}

object DatabaseEntity {

  @label(Graph.Label.Person.toString)
  case class Person(@id uuid: Option[String],
                    full_name: String) extends DatabaseEntity

 @label(Graph.Label.Email.toString)
  case class Email(@id uuid: Option[String], email_address: String) extends DatabaseEntity

 @label(Graph.Label.Domain.toString)
  case class Domain(@id uuid: Option[String], domain: String) extends DatabaseEntity
}

sealed trait Dao {
  import Gremlin._

  /**
    * Find a vertex based on a unique property
    *
    * @param label    the label for the vertex
    * @param key      the property key
    * @param optValue the optional value to match. If `None`, we get `None` back
    * @tparam A the type of the key value
    * @return a `Vertex`
    */
  protected def findVertex[A](label: String,
                              key: Key[A],
                              optValue: Option[A]): Option[Vertex] =
    for {
      value <- optValue
      vertices = graph.V.has(label = label, key = key, predicate = P.eq[A](value)).toList
      if vertices.nonEmpty
    } yield vertices.head
}

object Dao {

  object Gremlin {
    /**
      * The `Configuration` required to connect to Neo4j using the bolt protocol.
      *
      * @see <a href="https://boltprotocol.org">http://google.com</a>
      */
    val configuration: Configuration = Neo4JGraphConfigurationBuilder.connect(
      AppConfig.neo4j.getString("host"), AppConfig.neo4j.getString("user"), 
      AppConfig.neo4j.getString("password"))
      .withName(s"Profiles")
      .withElementIdProvider(classOf[ElementIdProvider])
      .build()
    lazy val graph: ScalaGraph = Neo4JGraphFactory.open(configuration).asScala
  }

  object Person extends Dao {

    import Gremlin._

    /**
      * Create a person vertex
      *
      * @param p      the request body from the caller unmarshalled into `RequestEntity.Person`.
      * @param userId the id of the user doing the operation. This is decoded from the JWT token
      * @return the database vertex deserialized into `ResponseEntity.Person`
      */
    def create(p: RequestEntity.Person)(implicit userId: Long): ResponseEntity.Person = {
      // This simply creates one case class from another
      val databasePersonEntity = from(p).toOptional[DatabaseEntity.Person].onCreate()

      /* Find email vertex in database */
      val existingEmailVertex: Option[Vertex] =
        findVertex(Graph.Label.Email.toString, Graph.Property.EmailAddress, p.email)

      /* If email vertex exists, throw a `ValidationException` */
      if (existingEmailVertex.nonEmpty) throw ValidationException("A profile with the email address 
      provided exists!")

      val newPersonVertex: Vertex = graph + databasePersonEntity
      val newOptEmailVertex = p.email.map(email => graph + DatabaseEntity.Email(uuid = None, 
      email_address = email))

      newOptEmailVertex.foreach { emailVertex =>
        // Create a domain vertex
        val emailAddress = emailVertex.toCC[DatabaseEntity.Email].email_address
        val domain = s"www.${emailAddress.split("@")(1)}"
        val domainEntity = DatabaseEntity.Domain(None, domain)

        val existingDomainVertex: Option[Vertex] = findVertex(Graph.Label.Domain.toString,
          Graph.Property.Domain, Some(domain))

        val domainVertex = existingDomainVertex.getOrElse(graph + domainEntity)

        // Create edge: emailVertex --> Domain
        // TODO: Find out why it's not able to create an edge to an existing vertex yet it works for new.
        emailVertex --- Graph.Relationship.DOMAIN.toString --> domainVertex

        //Create edge: personVertex --> emailVertex
        newPersonVertex --- Graph.Relationship.PERSONAL_EMAIL.toString --> emailVertex
      }

      graph.tx.commit()

      /* Construct the response entity */
      val savedPerson = newPersonVertex.toCC[DatabaseEntity.Person]
      from(savedPerson).toOptional[ResponseEntity.Person]
    }
}

[lokune]

Other issues I have discovered:
graph.V(id).toSet // This is nonEmpty which is expected. However,
graph.V(id).outE.toSet // Following outgoing edges returns empty even thought there are edges.
graph.V(id).outV.toSet // This is also empty even when we have vertices connected

graph.V(id).toList.foreach(v => v.remove) // This does not remove vertex from db. I had to use Cypher directly

[mpollmeier]

The code snippets give me an idea, but a 'small project' including e.g. a build.sbt (with your dependencies) and the complete list of imports for the snippets above take out the guesswork. Can you share that as well please?

[lokune]

@mpollmeier here we go. I have put together this project https://github.com/lokune/neo4j-scala-example

Let me know if you have any question.

[mpollmeier]

The project builds and all tests are green - nothing for me to do I guess :)
If you want me to look at something specific, please provide a concrete example (e.g. create a test inside the project) that actually fails.
Sorry for being blunt - I'm doing this in my limited spare time.

[lokune]

@mpollmeier my bad! I left some workarounds in there and assumed you'll just go through the code :) I just removed them, so tests are failing. Kindly pull the latest changes. Pardon me if there might be some minor mistakes, I rushed through it.

[mpollmeier]

Sorry, you'll have to make my life easier. There's far too many layers that I would have to get familiar with and unwrap in my head. The tests are failing with http responses etc.

Just provide one test that only uses gremlin and neo4j, without calls to some DAO or calling HTTP routes, and assert what it's supposed to do.

[lokune]

@mpollmeier this is just an Akka Http REST service that does CRUD on neo4j... I am traveling for two weeks, I will try to simplify when I get back. In the mean time, I will use Cypher with Neo4j driver directly. Thank you a lot for your time so far.

[bdine]

@lokune Have you find a solution for this ? I can not manage to add edge between two retrieving vertices too
Simple code like :

val node1 = graph.V.has(code, "50").head()
val node2 = graph.V.has(code, "51").head()
node1 --- "LINK" --> node2

gives me at runtime:

Error:(126, 26) could not find implicit value for parameter graph: gremlin.scala.ScalaGraph
        node1 --- "LINK" --> node2
Error:(126, 26) not enough arguments for method -->: (implicit graph: gremlin.scala.ScalaGraph)gremlin.scala.Edge.
Unspecified value parameter graph.
        node1 --- "LINK" --> node2

I am working with a remote gremlin-server graph initialize this way :

val serializer = new GraphSONMessageSerializerV3d0()
    val cluster = Cluster.build.addContactPoint("localhost").port(8182).serializer(serializer).create
    implicit val graph = EmptyGraph.instance.asScala.configure(_.withRemote(DriverRemoteConnection.using(cluster)))

I have try to by pass implicit function using :

 val a = StepLabel[Vertex]()
        val b = StepLabel[Vertex]()
        // Find vertices by id and then connect them by 'knows' edge
        graph.V(node1.id()).as(a).V(node2.id()).as(b).addE("LINK").from(a).to(b).iterate()

node1 and node2 are well retrieved
and I got this exception

java.util.concurrent.CompletionException: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: null:none([])
	at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
	at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
	at org.apache.tinkerpop.gremlin.driver.ResultSet.one(ResultSet.java:107)
	at org.apache.tinkerpop.gremlin.driver.ResultSet$1.hasNext(ResultSet.java:159)
	at org.apache.tinkerpop.gremlin.driver.ResultSet$1.next(ResultSet.java:166)
	at org.apache.tinkerpop.gremlin.driver.ResultSet$1.next(ResultSet.java:153)
	at org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteTraversal$TraverserIterator.next(DriverRemoteTraversal.java:142)
	at org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteTraversal$TraverserIterator.next(DriverRemoteTraversal.java:127)
	at org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteTraversal.nextTraverser(DriverRemoteTraversal.java:108)
	at org.apache.tinkerpop.gremlin.process.remote.traversal.step.map.RemoteStep.processNextStart(RemoteStep.java:80)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38)
	at org.apache.tinkerpop.gremlin.process.traversal.Traversal.iterate(Traversal.java:203)
	at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal.iterate(GraphTraversal.java:2664)
	at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal$Admin.iterate(GraphTraversal.java:177)
	at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal.iterate(DefaultGraphTraversal.java:48)
	at gremlin.scala.GremlinScala.iterate(GremlinScala.scala:82)
	at com.enedis.spark.Load$$anonfun$main$1$$anonfun$apply$1.apply(Load.scala:129)
	at com.enedis.spark.Load$$anonfun$main$1$$anonfun$apply$1.apply(Load.scala:121)
	at scala.collection.Iterator$class.foreach(Iterator.scala:750)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1202)
	at com.enedis.spark.Load$$anonfun$main$1.apply(Load.scala:121)
	at com.enedis.spark.Load$$anonfun$main$1.apply(Load.scala:118)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:109)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: null:none([])
	at org.apache.tinkerpop.gremlin.driver.Handler$GremlinResponseHandler.channelRead0(Handler.java:246)
	at org.apache.tinkerpop.gremlin.driver.Handler$GremlinResponseHandler.channelRead0(Handler.java:197)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.apache.tinkerpop.gremlin.driver.Handler$GremlinSaslAuthenticationHandler.channelRead0(Handler.java:123)
	at org.apache.tinkerpop.gremlin.driver.Handler$GremlinSaslAuthenticationHandler.channelRead0(Handler.java:67)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.apache.tinkerpop.gremlin.driver.handler.WebSocketClientHandler.channelRead0(WebSocketClientHandler.java:91)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)

[mpollmeier]

without a full project setup to reproduce it's hard to help, but here's a working example with gremlin-server:

https://github.com/mpollmeier/gremlin-scala-examples/blob/master/gremlin-server/src/test/scala/SimpleSpec.scala

I'm pretty sure the implicit not found doesn't happen at runtime, but at compile time. Make sure you have an implicit graph: ScalaGraph in scope.

[bdine]

@mpollmeier I have posted a response, but failed to submit it..
Yeah this is for sure a problem at compile time, I was missing the implicit val graph, sorry I'm bad :/
This is working pretty well by now ! (with a Gremlin Server)
I'll try it with JanusGraph soon
Thank you for your time !

[mpollmeier]

Sweet.
Note that I wasn't able to get Janusgraph working with their current (very old) 0.2.0 release, see #223 (comment)

[lokune]

@bdine I am glad you were able to get assistance from @mpollmeier . I have been traveling a lot. For my case (I use Neo4j), I decided to use Cypher directly because it's the only reliable approach for now. I combined this with Scala macros for case class to/from graph mapping and that works perfect!

[voroninp]

@lokune Have you written macros yourself or used some library for mapping?

[lokune]

@voroninp I got a lot of help from this link http://blog.echo.sh/2013/11/04/exploring-scala-macros-map-to-case-class-conversion.html