Prod Server Sizing and Indexing for Flowable 7.2 standalone deployment with a spring API layer

Flowable Engine
1

We are deploying Flowable 7.2 in production with SQL Server as our backend database. Our architecture includes Flowable Engine (port 8081) communicating via HTTP tasks to a Spring Boot API layer (port 8080), which calls external client APIs. We need official guidance on the following:

1. Database Indexing: What are the recommended indexing strategies (clustered/non-clustered) for Flowable tables? Which columns should be indexed for optimal performance? What is the recommended index maintenance strategy (rebuild/reorganize frequency)?

2. Database Maintenance: Best practices for archiving/purging historical data? Recommended SQL Server connection pooling settings? Any query optimization techniques?

3. Server Sizing & Capacity Planning: Official recommendations for production environment: RAM, CPU cores, JVM heap allocation for Flowable Engine and Spring Boot API layers? Any performance benchmarks for Flowable 7.2?

4. Clustering: Resource requirements if we implement clustered Flowable deployments on SQL Server?

5. Performance Metrics: What key metrics should we monitor?

2

Hi VaibhavTripathi,

Great questions. The honest headline first: Flowable does not publish official hardware-sizing numbers or throughput benchmarks for the open-source engine. Because the engine is a generic platform, the “right” sizing depends almost entirely on your models and load, so any fixed number would be misleading. What Flowable does give you is solid architectural guarantees and tuning levers, plus a clear recommendation to size from measurement. Here is the practical, defensible guidance for a 7.2 standalone (Spring Boot) deployment on SQL Server.

  1. Database indexing

You generally should not hand-craft indexes on the Flowable tables. The engine ships its own schema, and that schema already includes a comprehensive set of secondary indexes on the columns the engine actually queries — across the runtime tables (executions, tasks, jobs, timer jobs, variables, identity links) and the history tables (process/case instances, activities, variables, details, task instances). On SQL Server these are non-clustered indexes; the primary keys are the clustered indexes.

Recommendations:

  • Start with the shipped indexes and measure. Use SQL Server’s tooling (Query Store, execution plans, missing-index DMVs) against your real workload before adding anything.
  • Only add custom indexes for your own access patterns — e.g. if you run reporting/audit queries directly against the history tables on columns the engine itself never filters on. Validate each one with a plan; extra indexes add write cost.
  • Index maintenance follows standard SQL Server practice, not anything Flowable-specific. A typical policy is reorganize at moderate fragmentation and rebuild at high fragmentation, with statistics kept current; schedule it in your normal DBA maintenance window. There is no Flowable-mandated rebuild frequency.
  1. Database maintenance

Archiving / purging: the engine keeps runtime tables small by design — rows for an instance are removed as soon as it finishes; the history tables are what grow. Two levers control that growth:

  • History level. Set it to the lowest level that meets your needs. The levels, lowest to highest data volume, are: none, instance, task, activity, audit, full (the default is audit). For high-volume, fire-and-forget orchestrations, a lower level can dramatically cut write volume and table size. You can also keep a lower global level and selectively retain only specific variables/activities you care about.
  • Built-in history cleanup. The engine has a scheduled history-cleanup job that periodically deletes completed instances older than a retention period, in batches. It is off by default; you enable it and configure the retention period, schedule (cron), and batch size via the standard Flowable Spring Boot properties (flowable.enable-history-cleaning, plus the cleaning cycle / retention / batch-size properties). Tune the batch size so each run stays within a comfortable transaction footprint for SQL Server.

Connection pooling: the starter uses HikariCP. The key settings are the minimum idle and maximum pool size (spring.datasource.hikari.minimumIdle, spring.datasource.hikari.maximumPoolSize). The critical rule: the pool must be large enough to cover your async-executor threads plus your peak concurrent request/API load — every background job thread needs its own connection while it runs. If you raise the async-executor thread count (below), raise the pool accordingly, and watch for pool exhaustion in monitoring.

SQL Server query optimization — two settings matter a lot and are documented:

  • Use the READ_COMMITTED_SNAPSHOT isolation level. Flowable is built around optimistic locking and transaction-scoped caching, which fits snapshot isolation well, and it markedly reduces lock contention / deadlocks under load (Azure SQL already defaults to it — verify).
  • Add sendStringParametersAsUnicode=false to the JDBC URL so the driver doesn’t send every string as Unicode; this lets the optimizer use the indexes correctly and avoids runtime type conversions that burn CPU under load. Only do this if your text fields don’t need Unicode-only characters.

Both are described here: https://documentation.flowable.com/howto/howto/howto-basic-performance-tuning

  1. Server sizing & capacity planning

There are no official benchmarks, so size from principles and then measure:

  • Database first. The relational database is the single most important performance component — all runtime and historical state lives there under ACID transactions. Give the DB generous CPU, RAM and fast storage; it is usually the first bottleneck, not the engine.
  • Engine JVM. Standard JVM tuning applies — set -Xms/-Xmx (or -XX:MaxRAMPercentage). The engine itself is lightweight; heap pressure typically comes from your workload (large variables/payloads, history volume), so profile under realistic load and size the heap to your steady-state plus headroom.
  • CPU. More cores help when you run many parallel async jobs; the async-executor thread pool is what consumes them.
  • Your Spring Boot API layer (port 8080) is sized independently as a normal Spring Boot service, driven by its own request concurrency and what it does when it calls the external client APIs. Keep it separate from the engine’s sizing.

Because no two Flowable deployments behave alike, the official guidance is explicitly to monitor production and tune from observed resource usage rather than fix numbers up front.

  1. Clustering

Flowable’s clustering model is deliberately simple, which keeps resource requirements low:

  • Every node is stateless and connects to the same shared database. There is no cluster manager, no node-to-node communication, no membership/gossip protocol, and no distributed cache to operate. You scale horizontally by running more identical engine nodes behind your load balancer against the one database.
  • Background jobs are coordinated through the database: a node locks a job before executing it, so the same job is never run twice across the cluster, and if a node dies its locked jobs are automatically reclaimed by the others after the lock expires. This is built in — you don’t configure a coordination layer.
  • Resource impact: the main consequence of adding nodes is more load on the shared database (more connections, more job-acquisition queries). So when you scale out: size the DB for the aggregate, and make sure the DB’s max connections comfortably exceed the sum of all nodes’ Hikari pools. The nodes themselves need no extra inter-node infrastructure.
  1. Performance metrics to monitor
  • Database: CPU, memory, disk I/O and latency, lock/deadlock counts, and slow-query/plan stats — this is your primary signal.
  • Connection pool: active vs. maximum connections, and time spent waiting for a connection. Sustained saturation means the pool (and likely the DB) is the bottleneck.
  • Async (job) executor: queue depth / rejected-job rate and average job execution time. Brief spikes are fine; sustained job rejection means the executor or the DB needs more capacity.
  • History table growth over time, to confirm your history level + cleanup policy is keeping the database in check.
  • JVM: heap usage and GC pause times on the engine and on your API layer.
  • API layer: request throughput, latency, and web-container thread-pool saturation.

Async executor tuning (relevant to #3, #4 and #5)

The async executor runs your asynchronous tasks/timers in the background and is the main throughput knob. By default the engine starts it with a modest thread pool (8 threads) and an internal job queue. For higher background throughput you typically raise the thread-pool size and queue capacity — the public tuning guide above walks through these settings (and shows example values such as a 32-thread pool and a 256-entry queue). Two cautions: (1) each executor thread needs a DB connection, so enlarge the Hikari pool to match, and (2) the exact property keys differ slightly between a plain open-source Spring Boot starter and the packaged Flowable products, so confirm the property names for your 7.2 build.

Useful public references

In short: trust the shipped indexes and tune from measurement; control history volume with the history level plus built-in cleanup; apply the two SQL Server settings; size the database first; and scale out with stateless nodes against one DB while keeping the connection pool ahead of your thread counts.

Best regards

Note: This Answer is AI generated and manually reviewed

3

Thank you so much for your comprehensive and thoughtful response, Kevin. Your detailed guidance is invaluable and much appreciated.

I wanted to clarify our specific deployment architecture, as it differs from the Spring Boot embedded scenario and I believe this context may help in getting more insights:

Our Current Architecture:

• Flowable Engine: Deployed as a completely independent standalone service on Apache Tomcat 10, operating on Port 80, managing all BPMN workflows and process orchestration
• Spring API Facade Layer: Deployed as a separate standalone service on Port 81, serving as an API facade layer between Exisiting Client application and Flowable Engine
• Inter-Component Communication: All communication between Flowable (Port 80) and Spring (Port 81) and Client Application occurs exclusively via REST APIs
• External Integration: Client and external applications interact solely through RESTful API endpoints exposed by the Spring layer on Port 81
• Database Connectivity: Container-managed JDBC connection pooling accessed via JNDI, shared across both services for centralized database access management
• Technology Stack: Java 21, Flowable 7.2, Apache Tomcat 10
• All Interactions: Every interaction across all three layers (Flowable → Spring → Client/External Apps) utilizes REST APIs

This microservices-oriented, loosely-coupled architecture means the tuning considerations for your embedded Spring Boot deployment may differ from our topology. We would greatly appreciate any insights on optimizing performance, connection pooling, and database access patterns specifically for a distributed REST API-based architecture with standalone services.

Thank you again for your invaluable guidance!

4

Hi VaibhavTripathi,

Thanks for the detailed architecture — that context is genuinely helpful. The good news up front: moving from embedded Spring Boot to standalone WARs on Tomcat 10 changes nothing about how the Flowable engine behaves or what its tuning levers are. The async job executor, the shipped schema and indexes, the history configuration, and the database-coordinated job locking all work identically regardless of packaging. What changes is where you configure them — Tomcat container config and your JNDI DataSource instead of Spring Boot properties — not the underlying principles. So everything in my first reply still applies; below is how it maps onto your topology, plus the few things your distributed REST setup makes more important.

Container-managed JNDI connection pool (the part to get right)

Flowable fully supports a container-managed / JNDI DataSource — that’s a standard deployment mode, so no concern there. The thing to watch is that you described one shared pool serving both the Flowable engine webapp and the Spring facade. Those two have very different connection profiles competing for the same connections:

  • The engine holds a connection for the duration of each running background job — every active async-executor thread occupies one connection while it works.
  • The facade holds connections for its inbound REST request load.

So size the shared pool as: engine async-executor max threads + peak concurrent facade DB usage + headroom. If it’s under-sized, the symptom is requests and jobs stalling while they wait for a free connection (connection-wait / starvation), which looks like a slow system even though CPU is idle. Monitor active-vs-max connections and connection-wait time.

If the two services run as separate Tomcat instances, a clean and often safer option is to give each its own pool (each still pointing at the same database) so the facade’s request spikes can’t starve the engine’s job threads, and each can be sized for its own profile. A single shared pool is fine too — it just has to be sized for the sum of both workloads.

Also: the two SQL Server settings from my first reply still apply, now configured on the container-managed DataSource rather than a Spring Boot property:

  • READ_COMMITTED_SNAPSHOT isolation level.
  • sendStringParametersAsUnicode=false on the JDBC URL.

Async executor on standalone Tomcat

The async executor still runs inside the engine webapp on Tomcat — it is active and you tune the same thread-pool size and queue capacity as before, just configured for this packaging. Remember the coupling: each executor thread needs one connection from the JNDI pool while it runs a job. So any time you raise the executor thread count, raise the pool to match (this is the most common cause of pool starvation in setups like yours).

The REST hop and outbound calls (most relevant to your topology)

The network hop between Flowable (port 80) and the facade (port 81) adds latency, but it does not change database tuning — the database remains the primary bottleneck.

What does deserve attention is outbound calls from the engine to the facade and onward to external client APIs. A synchronous outbound REST call made from a service/HTTP step holds the thread that is executing that step for the entire duration of the remote call — and if that step is running on the async executor, the thread also holds its database connection the whole time. Slow or unbounded external calls therefore throttle engine throughput and can exhaust the connection pool, even though the engine itself is doing nothing but waiting. Recommendations:

  • Set aggressive connect and read timeouts on every outbound call so a slow remote endpoint can’t pin a thread indefinitely.
  • Keep engine transactions short. Don’t do long external work inside the same transaction that touches the engine tables.
  • Make outbound integration steps asynchronous (async continuations) so the engine commits and frees the thread, and the remote call happens on a controlled background thread rather than blocking the request that advanced the instance. Flowable’s HTTP-task and async-execution patterns are designed for exactly this.
  • Bound your outbound HTTP client’s own connection pool so a burst of integration calls is back-pressured rather than spawning unbounded concurrency.

Tomcat connector thread pools

Size the HTTP connector thread pool (maxThreads) on both the engine connector (port 80) and the facade connector (port 81) for your expected concurrency, and keep them consistent with the JDBC pool — there’s no point accepting far more concurrent requests than you have database connections to service. Apply your standard JVM settings (-Xms/-Xmx or -XX:MaxRAMPercentage) to the Tomcat process.

Java 21 + Tomcat 10 is a perfectly good stack for Flowable 7.2. I’d suggest treating any virtual-threads experimentation as exactly that — validate it under load for your version before relying on it, rather than assuming it’s a free win for the engine’s job-executor and JDBC paths.

Clustering / scale-out is unchanged

Your standalone Tomcat nodes are stateless and coordinate only through the shared database, exactly as in my first reply — so you scale horizontally by adding more identical engine nodes against the one database, with the engine ensuring each background job runs on exactly one node. The same caveats hold: more nodes means more aggregate load and more connections on the shared database, so size the database for the total and make sure its max connections comfortably exceed the sum of all nodes’ pools.

Same public references

As before, Flowable doesn’t publish official sizing numbers for this engine, so the right approach is to apply these principles and then size from what you measure under realistic load. In your topology specifically, the two things most likely to bite are (1) the shared JNDI pool being under-sized for engine threads plus facade load, and (2) slow synchronous outbound calls pinning engine threads and connections — get those two right and the rest is standard tuning.

Best regards

Note: This Answer is AI generated and manually reviewed