Performance Tuning
This page documents the operator-tunable knobs that affect Fleans throughput. For provider selection and full configuration reference, see Streaming and Persistence.
Stream provider sizing
Section titled “Stream provider sizing”Three knobs on KafkaStreamingOptions:
| Knob | Default | What it controls |
|---|---|---|
Fleans:Streaming:Kafka:QueueCount | 8 | Orleans-side pulling-agent count — one consumer per queue, each subscribed to one topic. Matches Redis:TotalQueueCount semantics. |
Fleans:Streaming:Kafka:NumPartitions | 1 | Kafka-side per-topic partition count at topic creation. Independent of QueueCount — does not multiply Orleans consumer parallelism. Forward-only via kafka-topics --alter --partitions N. |
Fleans:Streaming:Kafka:ReplicationFactor | 1 | Per-topic broker replication. Set to 3 in multi-broker clusters for durability. |
Production deployments: see #474 for security (SASL/TLS), durability (idempotent producer), and operational (lag metrics) gaps not yet shipped. Poison records can be routed to a dead-letter topic — see Streaming.
One knob:
| Knob | Default | What it controls |
|---|---|---|
Fleans:Streaming:Redis:TotalQueueCount | 8 | Orleans-side queue count for the Redis stream provider. Each silo’s pulling agents spread across this many queues. FIFO is guaranteed per stream; cross-instance traffic distributes across TotalQueueCount Redis queues. |
Redis streaming is supported via a third-party stream-provider package (not in-repo). Helm chart support for Redis streaming knobs is tracked in #610.
AzureQueue
Section titled “AzureQueue”QueueNames list length (default 8 entries). Tune by populating Fleans__Streaming__AzureQueue__QueueNames__0..N environment variables.
Persistence query pagination
Section titled “Persistence query pagination”User-task list query
Section titled “User-task list query”The user-task list query is paginated via Sieve and applies the assignee/candidate-group filter at the SQL level on PostgreSQL (via FromSqlInterpolated + JSON-text LIKE). The PostgreSQL path (PostgresUserTaskFilterStrategy) avoids materializing the full table. SQLite users with similar-scale workloads should switch to PostgreSQL via FLEANS_PERSISTENCE_PROVIDER=Postgres.
Event store reads
Section titled “Event store reads”EfCoreEventStore.ReadEventsAsync honors a configurable row limit. The default is MaxEventsPerLoad = 1000 (from FleansPersistenceOptions).
| Knob | Default | What it controls |
|---|---|---|
Persistence:MaxEventsPerLoad | 1000 | Maximum number of events loaded per grain-activation replay. If a workflow exceeds this limit, the silo logs: “Increase Persistence:MaxEventsPerLoad if this is intentional.” |
Persistence:SnapshotInterval | 100 | A workflow instance snapshot (its full state) is written every this many events. Activation loads the snapshot and replays at most about this many events after it. Lower it for faster reactivation, raise it to write less. |
For typical workflow volumes, the defaults are appropriate. MaxEventsPerLoad must be greater than SnapshotInterval; the silo refuses to start otherwise. If snapshot writes keep failing (log EventId 1131), the event history after the last snapshot keeps growing. Fix the cause before it passes MaxEventsPerLoad, or raise the limit.
Related
Section titled “Related”- Streaming — provider selection and config matrix.
- Persistence — SQLite vs Postgres trade-offs.
docs/conventions/streaming.md— contributor-side tuning guide.