Introduction

In our work on NebulaGraph, one early design question had consequences across the system: should a columnar batch become the common unit of work?

The answer has shaped the memory format, the expression framework, graph-specific operators, intermediate-result representation, the execution kernel used for graph computation in NebulaGraph Analytics, and the data transport layer.

This article explains how we arrived at that design, what we borrowed from analytical database systems, where graph workloads required different choices, and where the limits remain. It is about the engineering decisions behind the system, not a benchmark comparison. We focus on the interfaces and trade-offs that made later optimizations possible.

The rest of the article follows the dependencies in that design: establish the vector model, build database execution around it, then carry it across process boundaries and into Analytics. We close with the engineering trade-offs that remained visible throughout the work.

Part I: Establishing the Vector Model

From row-at-a-time evaluation to a vectorized baseline

The expression path in NebulaGraph Database 3.x was organized around a generic Value. An expression exposed a virtual eval(ExpressionContext&) method. An arithmetic expression recursively walked its children, obtained two Value objects, switched on the expression kind, and then used the dynamic Value operations. A function call repeated the same pattern for every row: evaluate each argument, assemble their references, and invoke a dynamically selected function.

That design had useful properties. It was direct, easy to extend, and a natural fit for an interpreter. Its main drawback was the repeated per-row overhead: work that could have been resolved earlier, along with framework costs that could have been amortized across a batch, remained in the hot loop:

• expression traversal happened for each row;

• virtual calls and operator dispatch happened for each row;

• values carried dynamic type information even when validation had already established the types;

• temporary objects and generic access paths obscured contiguous data from the compiler;

• one row rarely provided enough work to amortize framework overhead.

The simplified contrast looks like this:

Fig.1 Comparison of row-at-a-time and vectorized expression evaluation

The expression semantics are the same. What changes is the unit of work: a generic value going through dispatch for each row, or a typed batch with dispatch performed at batch granularity.

Moving from the first model to the second does more than change the loop syntax. It moves decisions out of the row loop. Parsing and validation establish expression and value types. Initialization resolves a concrete function implementation. At execution time, dispatch selects a path based on the batch's vector encodings. The innermost loop can then operate on typed views or raw buffers instead of rediscovering the same information for every row.

When we began designing NebulaGraph Database 5.x, both vectorization and query compilation were plausible directions. We did not treat them as opposing ideologies. Kersten et al. reached a similar conclusion in their comparison of compiled and vectorized query engines: both models can be efficient, but they have different strengths. Their experiments found vectorization effective at hiding cache-miss latency, while data-centric compilation reduced instruction count on cache-resident workloads (Kersten et al., 2018). For us, the important point was that this choice would shape how the execution engine was structured, not merely how individual queries were executed.

We chose vectorization as the baseline for three practical reasons.

• First, it could be introduced incrementally. We could define a vector representation, move operator boundaries to batches, and specialize expression paths without building a complete query-compilation infrastructure at the outset.

• Second, the execution loops remain ordinary C++ in the source tree and work with familiar debuggers, sanitizers, profilers, and compiler diagnostics.

• Third, it provides mechanisms that span the execution stack: batch-level dispatch, columnar access, explicit row selections, and encoding-aware paths.

Together, they reduce repeated dispatch, improve locality, avoid unnecessary copying and materialization, and provide a common basis for later specialization.

JIT compilation remains part of the design space. We may use it where information available only when a query runs is valuable enough to justify compilation—for example, to compile a fixed expression tree into a specialized evaluation path or specialize code for a query-specific schema. But a future JIT would build on the vector model rather than replace it. Today the expression framework is compiled ahead of time as ordinary C++, with dense loops optimized by the C++ compiler rather than a runtime JIT.

The vector model comes first

The vector model discussed here has three parts: typed batches, physical encodings, and explicit row selections. Together, they define how data is organized into batches, how a logical column is represented, and which rows participate in the current operation.

Apache Arrow was the natural starting point for the memory model. It established a practical standard around contiguous typed buffers, validity bitmaps, and nested columnar data (Apache Arrow overview). For flat vectors of primitive types, NebulaGraph's value and validity layouts are Arrow-compatible: values are contiguous, with nullability stored separately.

The overall vector system separates a logical type from its physical encoding. Flat and Constant encodings cover the basic cases. Slice, Concat, Dictionary, and related encodings let a logical column represent ranges, repeated or indirect values, and concatenated segments without first copying everything into a new flat buffer.

Our basic unit between operators is a vector batch: a set of vectors with a common logical row count. A selection vector describes which rows in that batch participate in the current operation. This separation is more important than any individual encoding. Values remain in their columns, while filters and control flow update the active row set to determine which rows later expressions process without copying the underlying data.

Primitive layout compatibility was only a starting point. We also needed the execution representation to cover graph-native types, GQL schema information, factorized intermediates, and buffer-lifetime and transport requirements. We therefore built NebulaGraph's own vector layer so these concerns could be handled within one execution model.

Making vectors graph-native

In a relational engine, a vector usually represents a column with one logical type. GQL includes familiar values such as scalars, lists, maps, and records, but property graphs add several complications.

A node has an identity, belongs to a graph, and may carry multiple labels and properties. An edge has identity, direction, endpoints, labels or types, and properties. A path contains a heterogeneous sequence of nodes and edges. GQL also allows schema configurations in which different rows of the same result may carry different node or edge types.

Storing every graph value in a generic Value object would preserve generality, but it would make the internal graph structure opaque to the vector layer, hiding exactly the information that vectorization is meant to expose. We therefore added graph-specialized vector types.

Rows in a single NodeVector may belong to different node types. It stores compact node headers and one child vector for each property in the union of their schemas. A map from graph and node-type identifiers to property-column indexes records which columns belong to each type. EdgeVector follows the same principle for edge types and properties. Properties remain columns rather than a per-row map of generic values.

PathVector uses a decomposed representation: typed node and edge vectors plus adjacency metadata that connects each element to the next one in a path. This lets heterogeneous paths share storage for elements while retaining the order needed to reconstruct a path value. A reader presents a graph-level path view; the physical representation remains decomposed underneath it.

These types were a concrete reason for building our own vector layer. They encode GQL and property-graph semantics directly. We could have stored those generic Value objects in ordinary columns, but the execution engine would still need graph-aware readers, writers, schema resolution, and serialization. Implementing the vector format ourselves made those requirements explicit instead of hiding them behind a generic value boundary.

Part II: Building Execution around Vectors

Graph-native execution: Expand and factorization

Expand as a graph-specialized join

Logically, traversal does not abolish joins. Expanding from a node to adjacent edges and destination nodes relates records through keys and produces the same cardinality effects that make joins difficult: fan-out, duplication, filtering, and potentially large intermediates.

Physically, however, Expand can use the adjacency structure in graph storage directly and organize access around the source identifier, direction, edge type, and required properties. Filters may be applied while the frontier is still compact.

NebulaGraph therefore has specialized Expand variants rather than expressing every hop as a scan followed by a general join. Logically, Expand can still be understood as a join over adjacency. The specialized operator aligns this frequently repeated join shape with the organization of graph storage.

Factorizing multi-hop results

The need becomes clearer with multiple hops. Consider:

MATCH (v)-[e1]->(m)-[e2]->(t)
RETURN v, e1, m, e2, t

If v has many outgoing edges, a flat table repeats v for every e1. If each m has many outgoing edges, the second hop repeats v, e1, and m again for every e2. This is a valid relational result, but eagerly materializing those repeated columns is a poor intermediate representation.

The idea behind NebulaGraph's FTree comes from work on factorized databases, especially Olteanu and Závodný's Size Bounds for Factorised Representations of Query Results. A flattened result may repeat the same path prefix many times even though that prefix only needs to be stored once. A factorized representation makes those dependencies explicit instead of materializing the fully flattened result immediately.

In NebulaGraph, each level of an FTree keeps the values introduced at that expansion step and records which child range belongs to each parent, so a repeated path prefix is stored once. When downstream execution requires the standard vector-batch interface, the factorized result is exposed through encodings that express the required slicing and repetition without copying every value row by row.

Fig.2 Comparison of flattened rows and a factorized multi-hop result

The same paths in two forms: flattening repeats prefixes, while the factorized form keeps a parent once and records its child ranges.

Factorization and vectorization address different dimensions of the same problem. Vectors make work over a column regular; factorization prevents redundant rows from being created too early. We apply factorization where the query shape and downstream consumers can benefit from it. An FTree does not imply that every MATCH result remains factorized through the entire plan.

A pipeline executor around vectors

These representations define what graph operators can exchange. The next question is how to drive those operators without leaking scheduling policy into every implementation.

A NebulaGraph physical plan is divided into linear pipelines. Operators inside a pipeline exchange vector batches. An operator implements the local state machine for its own logic: through consume(), it accepts a batch when it needs input; through produce(), it emits a batch when output is available. Source, transform, and sink operators use the same interaction model, with bridges connecting pipelines where joins, exchanges, or other multi-input boundaries require coordination.

The pipeline owns the driving loop. It asks whether an operator needs data, pulls output from its upstream neighbor, passes the batch downstream, and handles end-of-input. Blocking, yielding, cancellation, and resumption are coordinated outside the operator-specific logic. Concurrency comes from scheduling pipeline instances rather than asking an individual operator to manage threads.

The combination of consume and produce is deliberately more expressive than a one-way callback chain. A projection can turn an input batch into output immediately. An aggregation may consume many batches before producing anything. A join can wait for state built by another pipeline, and a limit can finish before its source is exhausted. The operator reports these states through the same small contract; the pipeline decides when to pull, push, stop, or reschedule. Backpressure is therefore coordinated by the pipeline using the state reported by each operator, rather than through ad hoc logic embedded in the operators.

Keeping scheduling policy outside the operators also leaves room to change scheduling without rewriting their algorithms. A pipeline may run on a different scheduling backend, use a different concurrency level, or suspend on an asynchronous storage request. This separation is especially useful for graph-specific work: an Expand operator can concentrate on adjacency access, filtering, and result construction, while the pipeline pauses and resumes it around storage access. Here, the vector batch is both a data layout and the interface that keeps physical operators composable.

DuckDB's vectorized execution engine and Velox's organization of operators and pipelines helped us reason about this design space (DuckDB: an Embeddable Analytical Database, Velox tasks and drivers). We borrowed those design ideas and built the executor around NebulaGraph's own graph operators, asynchronous boundaries, and execution lifecycle.

A vectorized expression engine

Once vectors became the data contract between physical operators, we had to return to the problem that started this work: expression evaluation. Keeping expression evaluation row-oriented would reintroduce per-row interpretation inside an otherwise vectorized pipeline.

For each function, the framework begins with a small typed scalar implementation. Template traits extract its signature and the call forms it provides, including nullable, null-free, and ASCII-specialized forms. The same metadata records default null behavior, whether the result may be null, determinism, side effects, and whether ASCII input implies ASCII output. The framework turns those facts into an execution adapter, so function authors do not have to repeat vector decoding and batch-loop setup in every function.

The framework therefore combines two kinds of information. Traits establish which paths are legal; the input encodings, validity, ASCII state, and current selection determine which legal path fits the current batch.

For a default-null function, a null argument produces a null result without entering the scalar implementation. If the selected inputs are null-free, that check can be removed from the inner loop. On a nullable dense batch, validity bitmaps are combined and processed in machine-word-sized groups: a fully valid group reuses the null-free dense loop, while a mixed group evaluates only valid rows and preserves null semantics. A function that explicitly accepts nullable arguments stays on its nullable path instead of receiving this default behavior.

String functions add another axis. When the function exposes an ASCII-specialized implementation and every selected string argument is known to be ASCII, the adapter selects that implementation once for the batch. Operations such as length, substring, trimming, and pattern matching can then avoid general Unicode work. If the precondition is absent, the same expression falls back to the general string path. The framework selects this fast path from a declared function trait and a vector-level data property rather than guessing for each row.

Encoding affects the loop shape as well. Flat and Constant primitive inputs can be decoded into specialized readers before iteration. For dense batches, the framework can expose contiguous buffers and straightforward loops that GCC or Clang can auto-vectorize. For each function's batch evaluation, the framework selects the function and encoding paths before entering the inner loop, avoiding the same dispatch on every row.

Other paths remain vectorized without necessarily becoming dense primitive loops. Sparse selections visit only the selected row indexes, while graph and container values, such as nodes, edges, and maps, are processed in batches through typed vector accessors. Expressions with stricter semantic requirements retain the necessary checks and ordering while preserving batch-oriented execution.

There is no single vectorized loop. The framework chooses from a small family: dense or sparse, null-free or nullable, ASCII or general Unicode, primitive or complex, flat or indirect. Individual functions remain small, while the compiler still sees a regular hot path.

Velox's simple-function framework is a useful reference for this style of compile-time trait extraction and execution-time selection of vector paths (Velox scalar functions). We use the same broad idea with NebulaGraph's type system, graph values, null semantics, and expression lifecycle.

Optimization layers built on the vector model

The vector model creates several opportunities for optimization at different levels:

These mechanisms reinforce one another. Batching, columnar layout, selections, and encoding-aware fast paths make execution more regular; on dense primitive loops, that regularity also gives the compiler a better chance to emit SIMD. Whether those instructions improve end-to-end performance still depends on conversion costs, code size, and where the workload spends its time.

Our normal sequence is therefore to make the data and loop regular, let the compiler optimize it, inspect the resulting machine code, and measure the complete workload. We consider hand-written SIMD only when a stable, important kernel still exposes a measurable compiler gap and end-to-end evidence on the target hardware justifies the maintenance and portability costs.

Part III: Extending the Vector Model across the System

Vectorization across the data path

Turning a batch back into rows at every process boundary adds format conversion and data copying that can offset part of the benefit of columnar execution. We therefore kept vector tables, which collect one or more vector batches, as the communication unit on both sides of graph execution. The internal and client-facing paths share the same columnar model, but they have different compatibility and evolution requirements.

Between storaged and graphd, NebulaGraph uses its own RPC framework, NRPC. Its vector transport can take ownership of columnar buffers produced by execution, providing a zero-copy handoff from execution buffers into the RPC path at that boundary.

From graphd to clients, the public result contract is a vector-oriented columnar transport protocol. Schema information and column buffers travel as a table rather than as an array of generic row values, and the SDKs deserialize that columnar representation directly. The motivation is similar to Arrow ADBC: keep database results columnar across the client boundary and avoid an unnecessary column-to-row-to-column conversion.

Fig.3 NebulaGraph's internal and client-facing vector transport boundaries

One columnar model spans two boundaries: execution buffers are handed to NRPC without a full-payload copy at that boundary, while the SDK protocol preserves columnar results.

Both boundaries preserve the columnar structure of the result. The difference is in their evolution constraints: the internal path can adopt encodings that suit joins, expansion, or factorization, while the public client protocol must retain a stable set of representations and graph schema metadata. End-to-end vectorization means preserving the columnar unit of work, not sharing one allocation or one wire format across every layer.

Vectorized control flow in NebulaGraph Analytics

The same execution model also proved useful outside the database query engine. In NebulaGraph Analytics, the kernel that scans and computes over graph data uses selection vectors to vectorize not only expressions, but procedural control flow: IF/ELSEIF/ELSE branches, WHILE loops, and row-selective BREAK and CONTINUE.

A scalar interpreter asks which instruction to execute next for one item. Our vectorized engine asks which rows in the current batch should participate in the next instruction. A selection vector carries that answer.

An IF/ELSEIF/ELSE chain partitions the active selection into branch-specific selections, with unmatched rows flowing to the next condition or the ELSE branch. A WHILE repeatedly updates the selection of rows that remain in the loop. CONTINUE carries selected rows to the next iteration, while BREAK moves them to the post-loop selection. The expressions inside each branch still receive vectors and evaluate a batch.

Fig.4 Selection Vector-driven control flow

The diagram shows the core idea: one batch moves through changing row selections rather than interpreting the same control flow separately for every row.

Procedural control flow and expression evaluation therefore share the same row-selection mechanism. Control statements update the set of active rows, while the expression framework evaluates those rows in batches. The execution kernel still preserves scope and language semantics without interpreting the entire control flow independently for every row.

How the vector model connects the system

Seen together, these are not isolated uses of vectorization. They rely on the same model: typed batches, explicit row selections, and encodings that preserve useful structure until a consumer needs another representation. That model links graph-native values, physical operators, expression evaluation, transport, and the Analytics kernel.

Fig.5 Overview of the vector model across NebulaGraph

The shared vector model connects graph data representation, database execution, expression evaluation, columnar transport, and the Analytics kernel, while allowing each layer to specialize around the same foundation.

This progression—from the vector model to execution, transport, and graph computation—leads to several broader engineering lessons.

Part IV: Engineering Takeaways and the Remaining Design Space

1. Establish the unit of work before optimizing the work

   A shared typed-batch model makes it easier for optimizations to compose across operators, expressions, and transport. Row-oriented boundaries introduce conversion work and interrupt that flow.

2. Representation is part of query execution

   Representation determines how much work later operators must perform. Graph vectors preserve typed structure, while factorization avoids creating redundant rows too early.

3. Specialize physical execution without changing logical cardinality

   Expand and FTree exploit adjacency and shared path prefixes in physical execution, but they do not merge logically distinct matches. Downstream results must preserve the duplicates, null behavior, path rules, errors, and cardinality required by the GQL plan.

4. Transport boundaries are part of execution design

   Internal RPC and public client protocols have different compatibility constraints, but both preserve columnar data. The internal handoff can reuse owned buffers; the public protocol needs a stable, cross-language representation.

5. Vector metadata is not free

   We encountered a different in-memory cost profile in production: point lookups over very wide tables returned few rows but many columns, allowing vector metadata to approach the size of the data payload. We reduced this overhead by sharing immutable type descriptors and avoiding repeated schemas across vectors and batches.

6. Keep the baseline portable and optimizations incremental

   Portable, inspectable compiler-optimized C++ is our baseline. Broader use of factorized results, targeted JIT compilation, and selective use of intrinsics remain incremental extensions to the vector model, adopted where workloads justify their additional complexity.

Conclusion

In NebulaGraph, we treated vectorization as a system-level design choice rather than a local optimization for expression evaluation. Building on the vector model, we redesigned how graph data and type information are represented in memory, how operators are driven, how graph expansion carries repeated data, how procedural control flow operates on a batch, and how results cross process and client boundaries.

The central lesson is that a shared vector model gives data flow and control flow a regular shape across subsystem boundaries, allowing the CPU, compiler, and surrounding system to optimize within a common structure.

References

1. Timo Kersten et al., Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask, PVLDB 11(13), 2018.

2. Apache Arrow, Arrow Columnar Format Overview.

3. Apache Arrow, Arrow Database Connectivity (ADBC).

4. Dan Olteanu and Jakub Závodný, Size Bounds for Factorised Representations of Query Results, ACM TODS 40(1), 2015.

5. Pedro Pedreira et al., Velox: Meta's Unified Execution Engine, PVLDB 15(12), 2022.

6. Velox documentation, Scalar Functions and What's in the Task?.

7. Mark Raasveldt and Hannes Mühleisen, DuckDB: an Embeddable Analytical Database, SIGMOD 2019.