open FFS ffs.db

Pre-v0 database engine

FFS is a database engine for data movement at 100TB scale.

Start with serious single-node installs. Design toward datasets measured in tens to hundreds of terabytes: column scans, vector search, graph traversal, write-back, and recovery in one Rust core.

See measurements Current status

3.01x vs pgvector HNSW query

3.03x vs LanceDB indexed query

3.94x vs sled range scan

9.28x vs petgraph 1-hop traversal

Harness: cargo run -p ffs-evals --release

Shape: database + warehouse
Surfaces: serve · flow · studio
Status: pre-v0, tested

Measure first Every claim needs a workload, comparator, and reproducible path

Own tradeoffs Read wins, write costs, and server overhead stay visible

Stay pre-v0 honest Local harness today; 100GB, 10TB, and 100TB evals are the path

Evals

The product promise starts in the harness.

Every platform claim has to turn into a measured path: lookup, range scan, traversal, vector query, write-back, reopen, and later 100GB / 10TB / 100TB workloads.

Comparator	Path	Current local result
Postgres + pgvector	HNSW query, same M and ef_construction	3.01x faster
LanceDB	HNSW query vs indexed IVF-PQ path	3.03x faster
LanceDB	HNSW query vs brute-force path	4.78x faster
sled	KV range scan over 100-key windows	3.94x faster
sled	random u64 insert path	2.89x faster
instant-distance	HNSW query, dim 128, k=10	2.64x faster
petgraph	1-hop graph traversal	9.28x faster
FFS integrated	filter, traverse, write-back, reopen	tested

Local harness results, pre-v0. The loud graph-server numbers stay in the README with caveats; this page leads with cleaner measured paths and reproducible direction.

Why

The platform vocabulary matters, but the engine still has to earn it.

Modern data platforms make their surfaces obvious: database, warehouse, storage, pipelines, governance, AI, apps, and sharing. FFS should speak that language while staying honest about what the engine can prove today.

The stance is smaller and sharper: one Rust engine first, then product surfaces around the same storage, WAL, catalog, graph, vector, and execution path. Start single-node and serious; grow into the platform from there.

Product surface

A database platform surface around one core.

The monorepo now names the surfaces users expect from a serious data platform, while keeping implementation close enough to move fast: serve, store, warehouse, flow, catalog, studio, vector, features, apps, and agents.

database

FFS Serve

Operational endpoints for apps, agents, workflows, and write-heavy state.

storage

FFS Store

Single-file pages, typed columns, graph relationships, indexes, WAL, recovery.

warehouse

FFS Warehouse

Columnar scans, query planning, batch operators, and high-concurrency reads.

engineering

ffs-flow

Ingest, transform, orchestrate, re-index, and write derived facts back.

governance

FFS Catalog

Schema today; index metadata, lineage, models, permissions, and audit next.

workspace

ffs-studio

Query, catalog exploration, lineage views, dashboards, and query history.

AI + vector

FFS Vector

HNSW retrieval beside the facts and relationships it uses.

apps + agents

FFS Agents

Agent state, feature lookups, model output, provenance, and write-back loops.

Platform architecture

Design for 100GB single-node installs before distributed scale.

FFS should be able to run as a local engine, a desktop database, or a Postgres-style server process. The same file format and execution core should carry each deployment mode.

storeone file, paged columns, WAL

serveembedded calls or ffsd

warehousescans, joins, planner, metrics

catalogschema, lineage, models, policy

flowpipelines, vector, agents

Platform menu

Operational database

Serve app and agent state through ffsd, with embedded mode for direct-call latency.

Lakehouse storage

Keep typed columns, graph edges, vector data, catalog state, and WAL together.

SQL warehouse

Make scans, filters, joins, and dashboards a first-class target for the engine.

Data engineering

Ingest, transform, validate, materialize, and monitor data movement.

Governance

Unify schema, lineage, audit, ownership, and policy at the catalog boundary.

Vector search

Build retrieval indexes next to the data, metadata, and graph they reference.

Feature serving

Materialize features and serve low-latency lookups without a separate feature DB.

AI workspace

Give humans query, catalog, lineage, history, dashboards, and natural-language hooks.

Application data

Power secure data apps that read operational state and write results back.

Sharing

Plan for portable snapshots, exports, federation, and controlled cross-system sync.

Rust first

Open a graph. Traverse it. Keep the data path native.

use ffs::cypher::{compile_query, parse};
use ffs::exec::collect_column;
use ffs::planner::{compile, CompileCtx};
use ffs::storage::RelTable;

let mut knows = RelTable::new(0, "KNOWS", 4, 4);
knows.add_edge(0, 0, 1, 1);
knows.add_edge(0, 0, 2, 2);

let q = parse("MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN b")?;
let plan = compile_query(&q, |_| vec![0, 1, 2, 3]);
let ctx = CompileCtx::empty().with_rel_table("KNOWS", &knows);
let mut op = compile(plan, &ctx)?;
let neighbours = collect_column(&mut *op, 0);

Status

Pre-v0, with the platform vocabulary now mapped to code.

ffs is early. The core engine, ffsd server scaffold, control plane surface, studio surface, flow surface, evals, and docs are all in the monorepo now.

Now harden storage, ffsd ownership, and product-surface contracts
Next query protocol, catalog/index metadata, warehouse scans, Studio query/catalog UI
Later 100GB, 10TB, and 100TB evals; backups; branch/restore; flow and feature workloads