Persistence Configuration

Fleans supports two persistence providers: SQLite (default, for local development) and PostgreSQL (for production and load testing). The provider is selected at startup via configuration — no code changes are required to switch.

Provider Selection

Set Persistence__Provider (case-insensitive: Sqlite or Postgres) on every fleans-* workload via your deployment tool. For PostgreSQL, also provide a connection string via ConnectionStrings__fleans. The runtime key is documented in Configuration / Persistence.

Persistence:Provider picks SQLite (default) or PostgreSQL; writes go through the command context, reads through the query context, which can target an optional replica. Open interactive diagram ↗

Compose bundle
Helm

The release pipeline publishes the compose bundle with PostgreSQL as the provider (fleans-postgres ships in the bundle). To switch to SQLite (single-host debug), or to point at an external Postgres, drop a docker-compose.override.yaml next to the bundle’s docker-compose.yaml:

services:
  fleans-core:
    environment:
      Persistence__Provider: Postgres
      ConnectionStrings__fleans: "Host=my-external-pg;Database=fleans;Username=fleans;Password=secret"
  fleans-management:
    environment:
      Persistence__Provider: Postgres
      ConnectionStrings__fleans: "Host=my-external-pg;Database=fleans;Username=fleans;Password=secret"
  fleans-mcp:
    environment:
      Persistence__Provider: Postgres
      ConnectionStrings__fleans: "Host=my-external-pg;Database=fleans;Username=fleans;Password=secret"
  fleans-worker:
    environment:
      Persistence__Provider: Postgres
      ConnectionStrings__fleans: "Host=my-external-pg;Database=fleans;Username=fleans;Password=secret"

docker compose up -d from the bundle directory merges the override automatically. The override file is not regenerated by the post-processor, so it persists across aspire publish re-runs.

# SQLite (chart default — single-host debug)
helm install fleans nightbaker/fleans

# Bundled PostgreSQL (chart provisions a containerised Postgres + secret)
helm install fleans nightbaker/fleans \
  --set persistence.provider=Postgres

The chart emits Persistence__Provider unconditionally. When persistence.provider=Postgres AND postgres.enabled=true (the default), ConnectionStrings__fleans is injected from the chart-managed secret. When postgres.enabled=false, the chart skips that injection so you can supply the connection string yourself via extraEnv — see below.

With the bundled PostgreSQL the chart also caps each workload’s connection pool and sizes the server for them. See Connection budget. helm template fails when the replicas can open more connections than postgres.maxConnections allows:

postgres:
  maxConnections: 250        # the server's max_connections
persistence:
  postgres:
    maxPoolSize:             # Npgsql Maximum Pool Size per pod (0 = Npgsql's 100)
      core: 20
      worker: 10
      api: 5
      web: 5
      mcp: 5
      plugins: 10

External PostgreSQL (managed RDS, Cloud SQL, …):

postgres:
  enabled: false
persistence:
  provider: Postgres
extraEnv:
  - name: ConnectionStrings__fleans
    valueFrom:
      secretKeyRef:
        name: my-external-pg-secret
        key: connection-string

kubectl create secret generic my-external-pg-secret \
  --from-literal=connection-string='Host=my-external-pg.example.com;Port=5432;Database=fleans;Username=fleans;Password=...'
helm install fleans nightbaker/fleans -f values-external-pg.yaml

extraEnv accepts either value: "..." (literal — convenient for --set one-liners) or valueFrom.secretKeyRef (GitOps-friendly — recommended for production credentials). Both shapes propagate to every fleans-* workload via the chart’s commonEnv helper.

Manual / non-Compose / non-Helm hosts

If you’re running silos outside Compose or Helm (bare-VM systemd units, ECS task definitions, hand-crafted Kubernetes manifests), set the same runtime keys directly. Either via appsettings.json:

{
  "Persistence": { "Provider": "Postgres" },
  "ConnectionStrings": {
    "fleans": "Host=postgres;Database=fleans;Username=fleans;Password=secret"
  }
}

Or via environment variables (.NET configuration precedence puts env above appsettings.json):

Persistence__Provider=Postgres
ConnectionStrings__fleans="Host=postgres;Database=fleans;Username=fleans;Password=secret"

Accepted values for Persistence__Provider (case-insensitive): Sqlite, Postgres. Default: Sqlite.

SQLite (default)

SQLite requires no extra configuration for local development. The default connection string points to a temp-directory database.

Configuration Key	Description	Default
`ConnectionStrings:fleans`	SQLite connection string	`DataSource=fleans-dev.db`
`ConnectionStrings:fleans-query`	Optional read-side connection string	Same as primary

Startup behaviour: SQLite uses EnsureCreated() — schema is created from the EF Core model on first run. There is no migration runner for SQLite.

PostgreSQL

PostgreSQL requires an accessible Postgres instance and a connection string.

Configuration Key	Description	Required
`ConnectionStrings:fleans`	Primary PostgreSQL connection string	Yes
`ConnectionStrings:fleans-query`	Read-replica connection string	No

Example appsettings.json:

{
  "Persistence": {
    "Provider": "Postgres"
  },
  "ConnectionStrings": {
    "fleans": "Host=localhost;Database=fleans;Username=fleans;Password=secret"
  }
}

Startup behaviour: PostgreSQL uses MigrateAsync() — EF Core migrations are applied automatically on Fleans.Api startup. Fleans.Web and Fleans.Mcp share the same database but do not run migrations.

Connection budget

Every Fleans process (Core, Worker and Api silos, Web, MCP, workers that open a database) opens one Npgsql connection pool per connection string, and each pool grows to 100 connections by default. A few pods are enough to overrun a server’s max_connections. Postgres then answers 53300: sorry, too many clients already, and throughput falls as load rises. With a capped pool, requests wait in the pool instead. Keep the worst case under the server limit:

processes × pools × Maximum Pool Size + other clients ≤ max_connections − 3 (reserved for superusers)

Pools per process: one for fleans, one more for each of fleans-query / fleans-journal when set, and one more when Postgres clustering or reminders share the fleans database (they share it with each other). fleans-cluster / fleans-reminders get pools of their own on their servers. Plugin hosts add their plugins’ databases.
Pool size: add Maximum Pool Size=<n> to every connection string. Something close to the server’s cores × 2–4 per process is enough. In the reference load test, 3 Core silos at 30 and 2 Worker silos at 10 peaked at 94 server connections and ran at full throughput, while the default pools failed requests.
Helm with the bundled PostgreSQL: set by persistence.postgres.maxPoolSize.* and postgres.maxConnections (see the Helm tab above). The render fails when the budget does not fit. Autoscaled workloads count at maxReplicas.
External servers (Helm postgres.enabled=false, Compose, bare hosts): the chart cannot see your server’s limit. Put the cap in your own connection strings.

Behind PgBouncer: Fleans works through PgBouncer in transaction pooling mode — the startup migrations serialize with a transaction-scoped advisory lock, which a transaction pooler keeps on one server connection. With a pooler, the connection budget applies to the pooler’s max_client_conn. Two connection-string settings are worth adding with a pooler or without one:

Maximum Pool Size=<n> — see Connection budget.
No Reset On Close=true — Npgsql otherwise sends DISCARD ALL whenever a pooled connection is reused; under load that was ~7 extra server transactions per workflow instance on the read model. Fleans keeps no session state on pooled connections, so the reset buys nothing.

Read replica

Point ConnectionStrings:fleans-query at a read replica of the fleans database (for example a PostgreSQL streaming-replication standby) and every read-model query moves there: the admin UI’s lists and finished-instance views, the MCP tools, and the REST listings and state endpoint. Writes — grain state, the projector’s read-model writes, migrations and health checks — stay on ConnectionStrings:fleans, so every process still needs it. Set the same value on every silo and on Fleans.Web and Fleans.Mcp. Unset, reads use the primary.

What to expect:

Lists lag a little more. The read model already trails each instance by a moment (the projector builds it from the event log); replication lag adds to that. Each list row carries the event-log version it reflects (stateVersion).
Instance state never goes back in time. A running instance is answered by its grain. A finished one is answered from the replica only once its row there says finished; until then the grain answers.
The chart and the compose bundle provision only the primary; provision the replica yourself and pass the connection string (Helm: extraEnv).

Read-model batching

The projector writes the read model of many instances at once. Each silo that runs it collects projections into batches of up to 64 and writes each batch with one statement for the running instances and one transaction for the finished ones. A typical instance is projected three times (created, started, finished). Written one at a time, that is 3 write transactions per instance on the primary. In batches of 64, it is about 0.05. This matters most on PostgreSQL, where commits on the primary are the first limit at high throughput. SQLite uses the same statement.

A projection is acknowledged only after its batch is written, so nothing is lost when a batch fails: every instance in it is retried. Rows never move back to an older version. The batch size, the delay a batch waits to fill (20 ms) and the number of batches in flight are set under Fleans:Projector — see Configuration.

Separate journal database

Every workflow instance is event-sourced: its journal (the WorkflowEventBatches table) is the source of truth, and its snapshots (WorkflowSnapshots) are a cache over it. Under load the journal takes most of the writes. It can run on a database of its own — for example a sharded one — while the read model, the admin UI’s queries and the other grain state stay on fleans.

Configuration Key	Description	Default
`Persistence:Journal:Provider`	`Postgres`, `Citus` or `Sqlite`	`Postgres`; on a SQLite main database without `fleans-journal`, `Sqlite` (local dev)
`Persistence:Journal:Citus:ShardCount`	Shards per journal table on Citus (1–64000)	`128`
`ConnectionStrings:fleans-journal`	The journal’s database	The `fleans` database

With neither set, nothing changes: the journal tables live in the main database. The journal has its own EF Core migrations (history table __EFJournalMigrationsHistory), applied at startup to whichever database it uses, so a journal database holds only the two journal tables.

# The journal on its own PostgreSQL server
Persistence__Provider=Postgres
ConnectionStrings__fleans="Host=postgres;Database=fleans;Username=fleans;Password=secret"
ConnectionStrings__fleans-journal="Host=journal-db;Database=fleans_journal;Username=fleans;Password=secret"

Both tables are read and written by workflow instance id only — no joins, no queries across instances — and instance ids are random (Guid v4). That is what lets a distributed database shard them.

Citus

With Persistence:Journal:Provider=Citus, startup applies the journal migrations and then distributes both tables by InstanceId into Persistence:Journal:Citus:ShardCount shards (default 128), snapshots colocated with their journal (create_distributed_table). Every journal command then goes to a single shard with plain PostgreSQL semantics. Tables that are already distributed are left alone, so every silo can run it — and a changed shard count does not reshard them: choose it with headroom before the first start.

Persistence__Provider=Postgres
Persistence__Journal__Provider=Citus
ConnectionStrings__fleans="Host=postgres;Database=fleans;Username=fleans;Password=secret"
ConnectionStrings__fleans-journal="Host=citus-coordinator;Database=fleans_journal;Username=fleans;Password=secret"

Install the extension in the journal database first (CREATE EXTENSION citus;) and add your worker nodes; startup fails with a clear message otherwise.
Start from a fresh journal database: the tables are distributed right after they are created, while still empty.
Without ConnectionStrings:fleans-journal, Citus distributes the journal tables inside the fleans database, which then has to be a Citus coordinator; the other tables stay local to it.
Citus is AGPL-3.0. Running it unmodified is fine; review the licence before you modify or redistribute it.

YugabyteDB (YSQL speaks the PostgreSQL protocol) is the alternative when built-in Raft failover matters more than the smallest change. It is not a supported provider yet: it needs retries on snapshot-isolation conflicts and squashed migrations.

Retention: purging finished instances

Fleans keeps every finished instance until you delete it: its event journal, its snapshot, and its rows in the read model. A typical 10-step instance stores about 15 KB, so a million finished instances is about 15 GB, and list queries scan ever-larger tables.

To delete old instances, call POST /Instances/purge with a cutoff. It deletes instances that finished (completed, cancelled or failed) before CompletedBefore. Running instances are never deleted. A call-activity child is kept while its parent is still running.

Fleans does not run the purge on a schedule. Run it from a cron job, a Kubernetes CronJob, or by hand. Repeat each run until the response says "hasMore": false:

The same call also removes message-idempotency records once their 24-hour window has expired, even if no finished instances match. This expiry is independent of CompletedBefore. Each call examines at most MaxInstances message keys in batches of BatchSize, in addition to its instance budget. Records refreshed by a concurrent send are retained. The response reports instance deletions in purgedCount and message-record deletions in purgedMessageDeduplicationsCount; hasMore is true when either category still has work. A purged key can be used for a new message.

CUTOFF=$(date -u -d '90 days ago' +%Y-%m-%dT%H:%M:%SZ)
while :; do
  RESP=$(curl -sf -X POST "$FLEANS_API/Instances/purge" \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $TOKEN" \
    -d "{\"CompletedBefore\":\"$CUTOFF\"}") || exit 1
  echo "$RESP"
  echo "$RESP" | grep -q '"hasMore":true' || break
done

Things to know:

It cannot be undone. Back up the database first if you need the history, for example for audits.
The cutoff must be at least 1 hour old. A recently finished instance may still be loaded in a silo’s memory. Purging it would let that copy write again later.
A purged id reads as “not found”. GET /Instances/{id}/state returns 404.
PostgreSQL drops whole months. The bulk of an instance’s read-model rows, its activity entries, variable scopes and condition results, sits in one partition per completion month. A batch deletes an instance’s journal and its instance row. A month’s details are dropped whole once the month ends before CompletedBefore and every instance that completed in it is purged. A drop takes a fraction of a second and frees the disk at once, where deleting the same rows one by one takes seconds, writes about a gigabyte of WAL per 100,000 long instances and leaves bloat behind. Until its month is dropped, a purged instance’s details still use disk space, but nothing can read them. Partitions are created for the current month and the next two, at startup and on every purge, so run the purge at least monthly.
Disk space. On PostgreSQL, dropped months free their space at once. The rest (instance rows, user tasks) is reused after VACUUM, which autovacuum runs for you. An SQLite file does not shrink until you run VACUUM.
The first purge of a large database is best run off-peak, with a smaller BatchSize such as 200.
Logs: EventIds 14100 (instance purge started), 14101 (instance batch purged), 14102 (instance purge finished) and 14103 (expired message records purged). On PostgreSQL: 14104 (a month’s details dropped), 14106 (a month’s partitions created), 14107 (warning: a month’s rows were already in the default partition, so it gets no partition and is deleted row by row) and 14108 (warning: tables busy, the month is retried on the next purge).
Separate journal database: the purge deletes the journal and snapshots by instance id in the journal’s database, and the read-model rows in fleans.