Upgrade to Tinkerpop 3.2.3

[dylanht]

This PR gets Titan to build against TinkerPop 3.2.0-incubating and passes most module test suites. TinkerPop 3.2.0-incubating has some significant advantages over 3.0.2-incubating and even 3.1.x-incubating, particularly with respect to OLAP capabilities/flexibility. Missing from this PR are many practical things such as a CassandraOutputFormat, CassandraInput/OutputRDD, GraphFilter support for CassandraInputFormat, and updates to examples / docs that would really allow people to take advantage of the new features and understand how they can work with Titan. Also missing from this PR is compiling g.V(x) and g.V().hasId(x) to graph.vertices(x) in TitanGraphStepStrategy for Titan as I believe @okram has previously recommended. I attempted to modify the HasStepFolder.foldInHasContainers() method which seemed to match the code example in the TinkerPop upgrade docs about the new GraphStep.processHasContainerIds() helper method but couldn't get it to go.

I commented out occurrences of TitanGraphTest.verifyMetrics() because I was getting a null pointer from trying to access the TraversalMetrics stored via ...profile("metrics") and accessed via t.getSideEffects().get("metrics") after changing the test traversals to match the new syntax implemented the profiling overhaul @rjbriody did. I am hopeful I just missed what was really happening under the hood and changes to TP3ProfileWrapper will be sufficient down the line - I failed to grasp how QueryProfiler works to let Titan interface with profile() step, and surprisingly in the console on an inmemory TitanGraph the same traversals seemed to work and I could access the metrics and nested metrics from getSideEffects().get("the-metrics-key-or-~metrics-whichever") as the tests attempt to do. I tried other ways of getting the metrics out of the traversal in the tests and couldn't get it done.

Modules with failing tests are titan-hbase, titan-solr, and of less concern titan-hadoop-1. The HBase tests get hung up indefinitely on "connection refused" accompanied by this warning when I leave it running overnight:

WARN zookeeper.clientcnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

Hopefully @graben1437 has a tip here or I will try something mentioned elsewhere (e.g. http://stackoverflow.com/questions/7755525/why-this-error-is-coming-in-zookeeper). Near as I can tell the hadoop 1 failures (10/10 test cases) are all class incompatibility or no class def found errors, and the following Solr test fails with the xml element in the surefire report reading <error message="Not enough nodes to handle the request" ...>...<error/> which sounds kind of OK to me:

SolrIndexTest>IndexProviderTest.testUpdateAddition:772->IndexProviderTest.runConflictingTx:617->IndexProviderTest.checkResult:630 » Solr

I seem to be having a bit of trouble getting my titan C*/ES/Spark/Hadoop cluster back into an operational state after this "upgrade", but it seems to be some kind of Spark or ES mis-config. I am also quite sure that what I did to all the poms in order to address this curator-recipes:2.7.2 artifact that kept popping up after I built Hadoop 2.7.2 from source is so very, very bad - I just don't quite know where I went wrong in the first place and what the right correction was. Perhaps @spmallette you could shed some light on that if you have time?

I'm going to ping @dalaro, @mbroecheler, and @dkuppitz as well as my goal with this PR is to highlight some hotspots for TinkerPop 3.2.0-incubating compatibility and take a shot at engaging the wiser and more experienced in a less time consuming process than this may otherwise have been for them.

[titan-cla]

Hi @dylanht, thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes.

[titan-cla]

You did it @dylanht!

Thank you for signing the Contribution License Agreement.

[dylanht]

Getting an exception I don't know quite what to do with when I test FulgoraGraphComputer manually on my titan-cassandra-es cluster. Just doing g.V().next() gives me the trace in the attached file, though it works for a standard traversal. Need to see what's going on here.

FulgoraException.txt

[spmallette]

Thanks for taking a swipe at this @dylanht - tbh the pom files are a bit beyond me in titan. dan will have to take a look at that when he is free to do so.

[analytically]

I'd love to see this merged.

[pluradj]

@analytically have you tested it out yet in your environment?

[dylanht]

@pluradj did you manage to Jason? @analytically as well if you've tried it out I would love to get some feedback about the good/bad/ugly in this PR so I can try and get it merge ready on a second pass. I have been using this for quite a while myself and pulling in the newest changes from tinkerpop master as they come out, but I do think it needs some work before being merged.

[analytically]

It builds fine (without Hadoop) but I'm using Titan 1.0.0 for now.

[pluradj]

@dylanht I've pulled it down and built it with -DskipTests -- I'll try to do some more banging on it, mostly on the OLTP side.

[sjudeng]

The curator-recipes dependency is defined in the Titan parent pom using hadoop2.version, which is why updating this to 2.7.2 caused issues. If you remove this definition, you can remove all your other associated pom updates and then version 2.7.1 will be brought in transitively as desired (see this commit).

Tests in all modules except titan-hadoop-1 are passing. In particular I couldn't reproduce any test failures or errors in titan-hbase or titan-solr. I tested on a CentOS 7 x64 instance with 7.5G and 2 vCPU.

There were also no issues running simple test OLAP queries using the hadoop2 distribution created from this branch. Verified both composite and mixed (Elasticsearch) index queries against HBase using SparkComputer in yarn-client mode on a Cloudera quickstart CDH VM. I did have to apply the HBaseBinaryInputFormat fix from #1268, but this wouldn't be necessary for other indexing backends (e.g. Cassandra, etc.).

[pluradj]

Thanks @sjudeng +1 on the curator-recipes. TinkerPop 3.2.1 is in code freeze and about to be released. I'm going to start testing this PR against it. I'll keep this thread posted on the results.

[pluradj]

Running into a problem with this test case after switching from 3.2.0-incubating to 3.2.1-SNAPSHOT:

https://github.com/apache/tinkerpop/blob/master/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/filter/TailTest.java#L120

Returns 3 results, but expecting 7.

[okram]

Huh, thats crazy. DSEGraph JUST ran into that problem too!!! I don’t know what the solution was… if any. cc/ @dalaro

On Jul 15, 2016, at 12:20 PM, Jason Plurad [email protected] wrote:

Running into a problem with this test case after switching from 3.2.0-incubating to 3.2.1-SNAPSHOT:

https://github.com/apache/tinkerpop/blob/master/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/filter/TailTest.java#L120 https://github.com/apache/tinkerpop/blob/master/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/filter/TailTest.java#L120
Returns 3 results, but expecting 7.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub #1312 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AAJFvR4IrhlfH010ZK1Uyt7NGXM_Lc8yks5qV898gaJpZM4IVkGB.

[pluradj]

Yeah, this seems crazy. Spits out 3 results, then count is -5!

gremlin> g.V().repeat(both()).times(3).tail(7)
==>v[8312]
==>v[4160]
==>v[4120]
gremlin> g.V().repeat(both()).times(3).tail(7).count()
==>-5
gremlin> g.V().repeat(both()).times(3).tail(7).toString()
==>[GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7)]
gremlin> g.V().repeat(both()).times(3).tail(7).count().toString()
==>[GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]

[okram]

The problem is most likely in the compilation. your toString() shows raw steps, not Titan’s strategies. Do your traversal with .explain() at the end.

One of those strategies is bad.

Marko.

On Jul 15, 2016, at 12:43 PM, Jason Plurad [email protected] wrote:

Yeah, this seems crazy. Spits out 3 results, then count is -5!

gremlin> g.V().repeat(both()).times(3).tail(7)
==>v[8312]
==>v[4160]
==>v[4120]
gremlin> g.V().repeat(both()).times(3).tail(7).count()
==>-5
gremlin> g.V().repeat(both()).times(3).tail(7).toString()
==>[GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7)]
gremlin> g.V().repeat(both()).times(3).tail(7).count().toString()
==>[GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub #1312 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AAJFvfYAqZaEG_Dd-VYjkI6wqAjUFxa_ks5qV9TCgaJpZM4IVkGB.

[pluradj]

Thanks for the tip @okram

Update: I disabled all of the Titan-specific strategies, then rewrote the query not to use the repeat -- g.V().both().both().both().tail(7).count(). This appears to work consistently. Here is the explain():

Original Traversal                 [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]

ConnectiveStrategy           [D]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
IdentityRemovalStrategy      [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
IncidentToAdjacentStrategy   [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
AdjacentToIncidentStrategy   [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
FilterRankingStrategy        [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
MatchPredicateStrategy       [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
RepeatUnrollStrategy         [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
RangeByIsCountStrategy       [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
PathRetractionStrategy       [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
ProfileStrategy              [F]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
StandardVerificationStrategy [V]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]

Final Traversal                    [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]

Reverting back to the original query -- g.V().repeat(both()).times(3).tail(7).count() -- this works some times, but it fails other times. Here is the explain():

Original Traversal                 [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]

ConnectiveStrategy           [D]   [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
IdentityRemovalStrategy      [O]   [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
IncidentToAdjacentStrategy   [O]   [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
AdjacentToIncidentStrategy   [O]   [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
FilterRankingStrategy        [O]   [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
MatchPredicateStrategy       [O]   [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
RepeatUnrollStrategy         [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
RangeByIsCountStrategy       [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
PathRetractionStrategy       [O]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
ProfileStrategy              [F]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
StandardVerificationStrategy [V]   [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]

Final Traversal                    [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]

The NoOpBarrierStep introduced by the RepeatUnrollStrategy is not working with TitanGraph in some cases. I saw the same behavior with the NoOpBarrierStep before the changes with the 5000 were put in.

Both queries work fine with TinkerGraph.

[okram]

Probably means Titan has bad equals()/hashCode() definitions for their Elements. You will probably want to look there and see where things take you.

Marko.

On Jul 18, 2016, at 12:00 PM, Jason Plurad [email protected] wrote:

Thanks for the tip @Orkam

Update: I disabled all of the Titan-specific strategies, then rewrote the query not to use the repeat -- g.V().both().both().both().tail(7).count(). This appears to work consistently. Here is the explain():

Original Traversal [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]

ConnectiveStrategy [D] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
IdentityRemovalStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
IncidentToAdjacentStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
AdjacentToIncidentStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
FilterRankingStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
MatchPredicateStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
RepeatUnrollStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
RangeByIsCountStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
PathRetractionStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
ProfileStrategy [F] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
StandardVerificationStrategy [V] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]

Final Traversal [GraphStep(vertex,[]), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
Reverting back to the original query -- g.V().repeat(both()).times(3).tail(7).count() -- this works some times, but it fails other times. Here is the explain():

Original Traversal [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]

ConnectiveStrategy [D] [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
IdentityRemovalStrategy [O] [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
IncidentToAdjacentStrategy [O] [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
AdjacentToIncidentStrategy [O] [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
FilterRankingStrategy [O] [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
MatchPredicateStrategy [O] [GraphStep(vertex,[]), RepeatStep([VertexStep(BOTH,vertex), RepeatEndStep],until(loops(3)),emit(false)), TailGlobalStep(7), CountGlobalStep]
RepeatUnrollStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
RangeByIsCountStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
PathRetractionStrategy [O] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
ProfileStrategy [F] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
StandardVerificationStrategy [V] [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]

Final Traversal [GraphStep(vertex,[]), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), NoOpBarrierStep(5000), VertexStep(BOTH,vertex), TailGlobalStep(7), CountGlobalStep]
The NoOpBarrierStep introduced by the RepeatUnrollStrategy is not working with TitanGraph in some cases. I saw the same behavior with the NoOpBarrierStep before the changes with the 5000 were put in.

Both queries work fine with TinkerGraph.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub #1312 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AAJFvR7hToogGfZ06N_tr2KmtD2E4touks5qW79HgaJpZM4IVkGB.

[pluradj]

I opened up https://issues.apache.org/jira/browse/TINKERPOP-1379

[bhalaji]

Any tentative date when this will be merged to titan11 branch ?

[frankiebagodonuts]

Hello,

We are currently using Titan 1.0 in production with Tinkerpop 3.1.1-incubating to execute OLAP queries via SparkGraphComputer. I'd like to take advantage of the GraphFilter functionality introduced in TP 3.2.0 and so I started port of our program by using this PR.

While testing the results I found that any OLAP query that ending in a select() step would never save out halted traversers. In order to alias the query results with the TP 3.1.1 version, we used a rather verbose query pattern:

g.V().has('type').as('a')
.both().has('createTime').as('b')
.select('a').values('type').as('firstType')
.select('b').values('name').as('secondName')
.select('firstType', 'secondName')

This worked and I was able to take the correct results from the halted traversers. With the TP 3.2-incubating port, no halted traversers are ever existing after the query completes. If I remove the select steps, chained ReferenceVertex objects are saved to the halted traversers:

g.V().has('type').as('a')
.both().has('createTime').as('b')

Additionally, I noticed the introduction of a keepDistributedHaltedTraversers flag in TraversalVertexProgram for TP 3.2.0. This flag seemed to dictate whether the halted traversers should remain after the program execution and was always initialized to false for our queries. To get past this in a quick and dirty way, I just manually set it to true in order to continue my testing. Unfortunately, this did not fix the issue below.

        this.keepDistributedHaltedTraversers = true;
//                !(this.traversal.get().getParent().asStep().getNextStep() instanceof ComputerResultStep || // if its just going to stream it out, don't distribute
//                        this.traversal.get().getParent().asStep().getNextStep() instanceof EmptyStep ||  // same as above, but if using TraversalVertexProgramStep directly
//                        (this.traversal.get().getParent().asStep().getNextStep() instanceof ProfileStep && // same as above, but needed for profiling
//                                this.traversal.get().getParent().asStep().getNextStep().getNextStep() instanceof ComputerResultStep));

I know TP 3.2 is not the latest version, however with using Titan in production this seemed like a step forward. Looking for any pointers or patches to get past this.

Cheers,
Frank

[pluradj]

@frankiebagodonuts check out #1269

[frankiebagodonuts]

@pluradj I already applied this to our TP 3.1.1 version to get it working with Hadoop 2. I believe this issue is with the traversal logic and the fact that the traversers do not "halt" when using a select() step and thus breaks prior to any OutputFormat execution.

[sjudeng]

I put in a pull request to your branch which updates to TinkerPop 3.2.3. There were a large number of test errors/failures, mostly involving OLAP traversals, that the PR resolves. It's also worth noting that one of the updates (adding support for TraverserSet serialization) may be relevant to the above issue with storing halted traversals.

[graben1437]

Thank you very much for the pull request. I will review it in the next few
days and hopefully will be able to integrate it.

On Wed, Nov 9, 2016 at 6:42 AM, sjudeng [email protected] wrote:

I put in a pull request dylanht#1 to your
branch which updates to TinkerPop 3.2.3. There were a large number of test
errors/failures, mostly involving OLAP traversals, that the PR resolves.
It's also worth noting that one of the updates (adding support for
TraverserSet serialization) may be relevant to the above issue with storing
halted traversals.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1312 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA6sk1gTtzahs4Kq_HahjpD9WiDKnmHiks5q8bG6gaJpZM4IVkGB
.

[dylanht]

Updated the name of the PR to reflect the version of Tinkerpop now being targeted as part of this work at the suggestion of @sjudeng - to my knowledge I can't rename the branch being merged without closing/re-opening a PR and I don't see the point so leaving that as is.