Chris Splinter

<p>If you want more performant graph traversals, upgrade to <a href="https://docs.datastax.com/en/dse/5.1/dse-dev/datastax_enterprise/releaseNotes/RNdse.html#RNdse5110">DSE 5.1.10</a> or <a href="https://docs.datastax.com/en/dse/6.0/dse-admin/datastax_enterprise/releaseNotes/RNdse.html#RNdse601">DSE 6.0.1</a>. In this post, we take a deeper look at the improvement.</p>

<h2>A Tale of Traversal Construction</h2>

<p>DataStax Enterprise Graph leverages <a href="https://tinkerpop.apache.org/gremlin.html">Gremlin</a>, the property graph query language of <a href="http://tinkerpop.apache.org/">Apache TinkerPop™</a>, as its core API. DataStax is not only committed to providing the best choice for distributed graph databases, but since the DSE 5.1 release, DataStax has also invested heavily in advancing the Apache TinkerPop™ framework, leading to a lot of great enhancements for graph querying. In this blog post, we will show an example of one of these enhancements that manifests as a 1.5x - 2x+ throughput performance improvement for the <a href="https://docs.datastax.com/en/developer/java-driver-dse/latest/manual/graph/querying/fluent_api/">Java DSE Graph Fluent API</a> ( analogous to the Bytecode API for those familiar with TinkerPop conventions ).</p>

<h2>The Fix</h2>

<p>We just made a significant performance improvement to Gremlin traversal construction, that targets <a href="http://tinkerpop.apache.org/docs/current/reference/#traversalstrategy">TraversalStrategy</a> initialization in the traversals themselves. For some context, all traversals are initialized to include a body of TraversalStrategy instances that will be applied at the time the traversal is iterated. It is through TraversalStrategy application that a Gremlin traversal can be optimized and potentially rewritten to achieve some greater efficiency.</p>

<p>The TraversalStrategy GlobalCache, which holds the common body of TraversalStrategy instances to apply to traversals, is accessed quite frequently during this process and more so for traversals that have a large number of child traversals ( such as g.V().local(outV()) where outV() is the child ). The specific fix ( <a href="https://issues.apache.org/jira/browse/TINKERPOP-1950">TINKERPOP-1950</a> ) introduces a map to this GlobalCache to keep track of the Graph or Graph Computer classes that have already been loaded, removing the somewhat expensive need to reload these classes each time <a href="https://github.com/apache/tinkerpop/blob/TINKERPOP-1950/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java#L251">TraversalStrategy.getStrategies</a> is called.</p>

<p>This traversal construction optimization technique benefits both the server and the client. On the DSE server side, it is applied when the traversal is reconstructed and executed. On clients using the Java DSE Graph Fluent API, the fix is applied when the traversal is initially constructed. As stated above, this improvement will be included in DSE versions 5.1.10 and 6.0.1, and in <a href="https://docs.datastax.com/en/developer/java-driver-dse/1.6/">DSE Java Driver 1.6.7</a>.</p>

<h2>The Results</h2>

<p><img alt="DSP" data-align="right" data-entity-type="file" data-entity-uuid="3afe0af3-a92c-4adf-9ede-c013a5e2daac" src="https://www.datastax.com/sites/default/files/inline-images/DSP-15686-dse6-all.png" /></p>

<p>Okay, sounds good, show me the data.</p>

<p>Yes, you are reading that correctly. For this run against DSE 6.0.x, the Java DSE Graph Fluent API throughput increased by more than 2x and the String API also showed a boost. It is expected that both of these would show improvement as we mentioned that this fix benefits both the client and the server.</p>

<p><img alt="Legend" data-entity-type="file" data-entity-uuid="fd488a62-51c1-42c9-9726-45c6d846216a" src="https://www.datastax.com/sites/default/files/inline-images/Screen%20Shot%202018-06-06%20at%2010.57.57%20AM-1.png" /></p>

<p>Now let’s compare how this same test looked against DSE 5.1.x.</p>

<p><img alt="DSE . 5.1 " data-entity-type="file" data-entity-uuid="49a23233-c7c6-4672-ae03-af0d3d0f1308" src="https://www.datastax.com/sites/default/files/inline-images/DSE-15686-dse51-all.png" /></p>

<p><img alt="Legend" data-entity-type="file" data-entity-uuid="d8fc19cc-bad4-4f93-afdd-3662ad4be15c" src="https://www.datastax.com/sites/default/files/inline-images/Screen%20Shot%202018-06-06%20at%2011.04.00%20AM.png" /></p>

<p>Again, we see a significant increase in throughput for the Java DSE Graph Fluent API, right around 1.7x. The String API throughput also exhibits an increase as shown in the DSE 6.0.x test. Looking at the numbers side by side between the 5.1.x and the 6.0.x runs, we see better throughput overall in the DSE 6.0.x run.</p>

<p>DSE 5.1.10 vs DSE 6.0.1</p>

<p><img alt="DSE5.1.10 vs. DSE6.0.1" data-entity-type="file" data-entity-uuid="8c27b4c5-daed-4b0b-bd2a-2546f0f6ddc5" src="https://www.datastax.com/sites/default/files/inline-images/Screen%20Shot%202018-06-06%20at%2011.05.23%20AM.png" /></p>

<h2>Test Strategy</h2>

<p>As always, we urge you to do your own testing to gauge the impact to your workload as results will vary depending on the traversal and environment.</p>

<p>This test used a 5 node DSE cluster and writes a total of 20 million edges and 40 million vertices using 100 concurrent operators. The DSE nodes were running on machines with 2 CPU with 12 cores each. Hyperthreading is enabled, resulting in 48 HT cores. The instances have 128 GB RAM. They can store up to 2 TB of data, using 2x1 TB SSD.</p>

<h2>Schema</h2>

<p><code>schema.propertyKey('personId').Text().create(); schema.propertyKey('name').Text().create(); schema.propertyKey('age').Bigint().create(); schema.vertexLabel('person').partitionKey('personId').properties('name', 'age').create(); schema.edgeLabel('knows').connection('person', 'person').create(); </code></p>

<h2>Traversal</h2>

<p>Java DSE Graph Fluent API ( analogous to Tinkerpop Bytecode API )</p>

<p><code>DseGraph.statementFromTraversal(DseGraph.traversal() &nbsp;&nbsp;&nbsp;<br />
&nbsp; .addV("person") &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />
&nbsp; &nbsp; &nbsp;.property("personId", session("edgeCount").asOption[String].get + "from") &nbsp;&nbsp;&nbsp;&nbsp;<br />
&nbsp; &nbsp; &nbsp;.property("name", "marko" + session("edgeCount").asOption[String].get + "from") &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />
&nbsp; &nbsp; &nbsp;.property("age", session("edgeCount").asOption[Long].get + 1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.as("a") &nbsp;&nbsp;&nbsp;.addV("person") &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />
&nbsp; &nbsp; &nbsp;.property("personId", session("edgeCount").asOption[String].get + "to") &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />
&nbsp; &nbsp; &nbsp;.property("name", "marko" + session("edgeCount").asOption[String].get + "to") &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />
&nbsp; &nbsp; &nbsp;.property("age", session("edgeCount").asOption[Long].get + 1)&nbsp;&nbsp;&nbsp;<br />
&nbsp;&nbsp;.addE("knows").from("a") ) </code></p>

<p>String API ( analogous to Tinkerpop Script API )</p>

<p><code>g.addV("person") &nbsp;&nbsp;&nbsp;<br />
&nbsp; .property("personId", edgeCount + "from") &nbsp;&nbsp;&nbsp;<br />
&nbsp; .property("name", "marko" + edgeCount + "from") &nbsp;&nbsp;&nbsp;<br />
&nbsp; .property("age", edgeCount + 1) &nbsp;&nbsp;&nbsp;<br />
&nbsp; .as("a")<br />
.addV("person") &nbsp;&nbsp;&nbsp;<br />
&nbsp;&nbsp;.property("personId", edgeCount + "to") &nbsp;&nbsp;&nbsp;<br />
&nbsp;&nbsp;.property("name", "marko" + edgeCount + "to") &nbsp;&nbsp;&nbsp;<br />
&nbsp; .property("age", edgeCount + 1)<br />
.addE("knows").from("a") </code></p>

<h2>Conclusion</h2>

<p>To wrap things up, we hope that this inspires you to upgrade to the latest versions of DataStax Enterprise and the DSE Java Driver to reap the benefits of this improvement for your DSE Graph traversals. Our road does not stop here and we will continue to pour enhancements into both Apache TinkerPop™ and DSE Graph to make your applications that leverage DSE Graph truly the best in class.</p>

<p>For more on DataStax Enterprise Graph, please visit the following links.</p>

<ul>
	<li><a href="/products/datastax-graph">DataStax Enterprise Graph Product Page</a></li>
	<li><a href="https://docs.datastax.com/en/dse/6.0/dse-dev/datastax_enterprise/graph/dseGraphAbout.html">DSE Graph Documentation</a></li>
	<li><a href="https://www.datastax.com/dev/blog/datastax-drivers-fluent-apis-for-dse-graph-are-out">DSE Graph Fluent API</a></li>
	<li><a href="https://academy.datastax.com/resources/getting-started-tinkerpop-and-gremlin">Getting Started with Tinkerpop and Gremlin</a></li>
</ul>


DSE Graph Just Got Faster

Chris SplinterProduct Management

Discover more

Share

Share

A Tale of Traversal Construction

The Fix

The Results

Test Strategy

Schema

Traversal

Conclusion

More Company

DataStax Acquires Langflow to Accelerate Generative AI Development

The Top 5 DataStax Stories from 2023

2023 Recap: Data = AI

DataStax Astra DB Nabs Three Prestigious 2023 TrustRadius “Best of” Awards, Dominates the Vector Databases Category

One-stop Data API for Production GenAI