NebulaGraph Database v3.4.0 Benchmark Report

Disclaimer: We used the LDBC SNB benchmark as a starting point. However, the test results aren’t audited, so we want to be clear that this is not an LDBC Benchmark test run, and these numbers are not LDBC Benchmark Results.

This is the performance report for NebulaGraph Community Edition v3.4.0.

In this version, NebulaGraph has added Memory Tracker and has enabled concurrent queries.

Highlight Improvements：

MatchTest4, MatchTest5, MATCH two-hop count QPS improved by 20%, 65%, 20% respectively after merging some performance optimization PR；
With concurrent queries on and Memory Tracker added, the relevant cases have different degrees of improvement or decrease.

Test Environment

Both the server and the load testing machine are physical machines:：

Test Data

The test data uses the LDBC-SNB SF100 dataset, with a size of 100G, which contains 282,386,021 vertices and 1,775,513,185 edges. The graph space used for testing is partitioned into 24 partitions with 3 replications.

About LDBC-SNB

The Linked Data Benchmark Council (LDBC) is the official platform for international benchmarking of graph and RDF data. The Social Network Benchmark (SNB) is one of the benchmarks developed by the LDBC. For more information on the LDBC-SNB dataset, please refer to the following documents：

LDBC-SNB Specification：https://ldbcouncil.org/ldbc_snb_docs/ldbc-snb-specification.pdf
LDBC-SNB Docs：https://github.com/ldbc/ldbc_snb_docs
LDBC-SNB test datasets production tool：https://github.com/ldbc/ldbc_snb_datagen_spark

NebulaGraph Commit

nebula-graphd version db3c1b3
nebula-storaged version db3c1b3
nebula-metad version db3c1b3

Test Description

The load testing tool used is K6 based on the Go language. For more information, please refer to K6 website；The client uses nebula-go
The "vu" in the horizontal axis of the chart, such as "50_vu" and "100_vu," represents the concept of "virtual user" used by k6, which is the concurrency in performance testing. "50_vu" represents 50 concurrent users, and "100_vu" represents 100 concurrent users, and so on.
The performance baseline uses the officially released version 3.3.0.
ResponseTime = Latency (server processing time) + network transmission time + client deserialization time.

Baseline Test

注：下图涉及的词语解释

QPS 即吞吐率
Latency 即服务端耗时
ResponseTime 即客户端耗时
RowSize 即请求返回行数

Test Cases and Results

Queries that return properties of edges

GO {} STEP FROM {} OVER KNOWS yield KNOWS.creationDate

Go 1 Step With Edge · QPS

Go 1 Step With Edge · Latency(ms)

Go 1 Step With Edge · Response Time(ms)

Go 1 Step With Edge · Row Size

Go 2 Step With Edge · QPS

Go 2 Step With Edge · Latency(ms)

Go 2 Step With Edge · Response Time(ms)

Go 2 Step With Edge · Row Size

Go 3 Step With Edge · QPS

Go 3 Step With Edge · Latency(ms)

Go 3 Step With Edge · Response Time(ms)

Go 3 Step With Edge · Row Size

Queries that return properties of destination vertices

GO {} STEP FROM {} OVER KNOWS yield $$.Person.firstName

Go 1 Step With Dst Edge · QPS

Go 1 Step With Dst Edge · Latency(ms)

Go 1 Step With Dst Edge · Response Time(ms)

Go 1 Step With Dst Edge · Row Size

Go 2 Step With Dst Edge · QPS

Go 2 Step With Dst Edge · Latency(ms)

Go 2 Step With Dst Edge · Response Time(ms)

Go 2 Step With Dst Edge · Row Size

Go 3 Step With Dst Edge · QPS

Go 3 Step With Dst Edge · Latency(ms)

Go 3 Step With Dst Edge · Response Time(ms)

Go 3 Step With Dst Edge · Row Size

Queries that return properties of edges and destination vertices

GO {} STEP FROM {} OVER KNOWS yield DISTINCT KNOWS.creationDate as t, $$.Person.firstName, $$.Person.lastName, $$.Person.birthday as birth | order by $-.t, $-.birth | limit 10

Go 1 Step All · QPS

Go 1 Step All · Latency(ms)

Go 1 Step All · Response Time(ms)）

Go 1 Step All · Row Size

Go 2 Step All · QPS

Go 2 Step All · Latency(ms)

Go 2 Step All · Response Time(ms)

Go 2 Step All · Row Size

Go 3 Step All · QPS

Go 3 Step All · Latency(ms)

Go 3 Step All · Response Time(ms)

Go 3 Step All · Row Size

LOOKUP

LOOKUP ON Person WHERE Person.firstName == '{}' YIELD Person.firstName, Person.lastName, Person.gender, Person.birthday, Person.creationDate, Person.locationIP, Person.browserUsed

LOOKUP·QPS

LOOKUP·Latency(ms)

LOOKUP·Response Time(ms)

LOOKUP·Row Size

FETCH Tag

FETCH PROP ON Person {} YIELD Person.firstName, Person.lastName, Person.gender, Person.birthday, Person.creationDate, Person.locationIP, Person.browserUsed

FETCH Tag·QPS

FETCH Tag·Latency(ms)

FETCH Tag·Response Time(ms)

FETCH Tag·Row Size

FETCH Edge

FETCH PROP ON KNOWS {} -> {} YIELD KNOWS.creationDate

FETCH Edge·QPS

FETCH Edge·Latency(ms)

FETCH Edge·Response Time(ms)

FETCH Edge·Row Size

MATCH Index

MATCH (v:Person) WHERE v.Person.firstName == '{}' RETURN v

MATCH·QPS

MATCH·Latency(ms)

MATCH·Response Time(ms)

MATCH·Row Size

MATCH

MATCH (v1:Person)-[e:KNOWS]->(v2:Person) WHERE id(v1) == {} RETURN v2

MATCH with 1 Hop·QPS

MATCH with 1 Hop·Latency Time（ms）

MATCH with 1 Hop·Response Time（ms）

MATCH with 1 Hop·Row Size

MATCH with Two-Hop

MATCH (v1:Person)-[e:KNOWS*2]->(v2:Person) WHERE id(v1) == {} RETURN v2

MATCH with 2 Hop·QPS

MATCH with 2 Hop·Latency Time（ms）

MATCH with 2 Hop·Response Time（ms）

MATCH with 2 Hop·Row Size

MatchTest1

match (v:Person) where id(v) == {} return count(v.Person.firstName)

QPS

Latency Time（ms）

Response Time（ms）

Row Size

MatchTest2

match (v:Person)-[e:KNOWS]-(v2) where id(v) == {} and v2.Person.locationIP != 'yyy' return length(v.Person.browserUsed) + length(v2.Person.gender)

QPS

Latency Time（ms）

Response Time（ms）

Row Size

MatchTest3

match (v:Person)-[e:KNOWS]-(v2) where id(v) == {} and v2.Person.locationIP != 'yyy' with v, v2 as v3 return length(v.Person.browserUsed) + (v3.Person.gender)

QPS

Latency Time（ms）

Response Time（ms）

Row Size

MatchTest4

MATCH (m)-[:KNOWS]-(n) WHERE id(m)=={} OPTIONAL MATCH (n)<-[:KNOWS]-(l) RETURN length(m.Person.lastName) AS n1, length(n.Person.lastName) AS n2, l.Person.creationDate AS n3 ORDER BY n1, n2, n3 LIMIT 10

QPS

Latency Time (ms)

Response Time（ms）

Row Size

MatchTest5

MATCH (m)-[:KNOWS]-(n) WHERE id(m)=={} MATCH (n)-[:KNOWS]-(l) WITH m AS x, n AS y, l RETURN x.Person.firstName AS n1, y.Person.firstName AS n2, CASE WHEN l.Person.firstName is not null THEN l.Person.firstName WHEN l.Person.gender is not null THEN l.Person.birthday ELSE 'null' END AS n3 ORDER BY n1, n2, n3 LIMIT 10

QPS

Latency Time（ms）

Response Time（ms）

Row Size

Go 1 Step with Edge count

GO {} STEP FROM {} OVER KNOWS yield KNOWS.creationDate | return count(*) ;

Go 1 Step With Edge_count · QPS

Go 1 Step With Edge _count· Latency(ms)

Go 1 Step With Edge_count · Response Time(ms)

Go 1 Step With Edge_count · Row Size

Go 2 Step With Edge_count · QPS

Go 2 Step With Edge _count· Latency(ms)

Go 2 Step With Edge_count · Response Time(ms)

Go 2 Step With Edge_count · Row Size

Go 3 Step With Edge_count · QPS

Go 3 Step With Edge _count· Latency(ms)

Go 3 Step With Edge_count · Response Time(ms)

Go 3 Step With Edge_count · Row Size

Queries that return properties of destination vertices count

GO 1 STEP FROM {} OVER KNOWS yield $$.Person.firstName | return count(*)

Go 1 Step With Dist_count · QPS

Go 1 Step With Dist _count· Latency(ms)

Go 1 Step With Dist_count · Response Time(ms)

Go 1 Step With Dist_count · Row Size

Go 2 Step With Dist_count · QPS

Go 2 Step With Dist _count· Latency(ms)

Go 2 Step With Dist_count · Response Time(ms)

Go 2 Step With Dist_count · Row Size

Go 3 Step With Dist_count · QPS

Go 3 Step With Dist _count· Latency(ms)

Go 3 Step With Dist_count · Response Time(ms)

Go 3 Step With Dist_count · Row Size

LOOKUP count

LOOKUP ON Person WHERE Person.firstName == '{}' YIELD Person.firstName | return count(*)

QPS

Latency Time（ms）

Response Time（ms）

Row Size

MATCH count

MATCH (v:Person) WHERE v.Person.firstName == '{}' RETURN count(*)

QPS

Latency Time（ms）

Response Time（ms）

Row Size

MATCH 1-Hop count

MATCH (v1:Person)-[e:KNOWS]->(v2:Person) WHERE id(v1) == {} RETURN count(*)

QPS

Latency Time（ms）

Response Time（ms）

Row Size

MATCH Two-Hop count

MATCH (v1:Person)-[e:KNOWS*2]->(v2:Person) WHERE id(v1) == {} RETURN count(*)

QPS

Latency Time（ms）

Response Time（ms）

Row Size

Insert Vertex

INSERT VERTEXComment(creationDate, locationIP, browserUsed, content, length) VALUES {}:('{}', '{}', '{}', '{}', {})

QPS

Latency Time（ms）

Response Time（ms）

Row Size

Insert Edge

INSERT EDGE LIKES (creationDate) VALUES {}→{}:('{}')

QPS

Latency Time（ms）

Response Time（ms）

Row Size

Insert Vertex with Property Index

INSERT VERTEXPerson(firstName,lastName,gender,birthday,creationDate,locationIP,browserUsed) VALUES {0}:(\"{1}\", \"{2}\",\"{3}\",\"{4}\",datetime(\"{5}\"), \"{6}\",\"{7}\")

QPS