Flowable 7.2.0: Node Affinity Hoards All Jobs – All Async Jobs Locked by Node 1 while only 8 are being processed

Environment:

  • Flowable Version: 7.2.0

  • Cluster: 2 Nodes (8 CPUs each) sharing a single database.

  • Scenario: Parent Process A uses script tasks with Thread.sleep to stagger calls to Child Process B. Process B contains several script tasks that also Thread.sleep for 1 minute.

  • The Issue: Lack of load balancing. Node 1 locks every job created under its own LOCK_OWNER_ ID.

Technical Observations:

  1. Job Acquisition: Even though Node 1 only has 8 CPUs and can only execute 8 script tasks concurrently, it sets itself as the LOCK_OWNER_ for every single job (40+ job) in the ACT_RU_JOB table immediately upon creation and for future jobs created from the same process.

  2. Queue Capacity Ignored: I have set flowable.process.async-executor-thread-pool-queue-capacity=10 and flowable.process.async-executor.thread-pool-queue-size=10. I expected Node 1 to stop acquiring/locking once the 10 queue slots were full, but it continues to lock the entire burst of jobs.

  3. Node 2 Idle: Because Node 1 has “kidnapped” the jobs by locking them, Node 2’s acquisition thread sees 0 available jobs and remains idle.

  4. Task Configuration: All tasks are async=true and exclusive=false.

How can I force Flowable to distribute these “excess” locked jobs to Node 2 so the cluster is actually utilized?

Process A

This text will be hid<?xml version="1.0" encoding="UTF-8"?>

>> DEBUG: Thread '${threadName}' is freezing for 10 seconds..."

// This freezes the executor thread
Thread.sleep(10000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<![CDATA[String threadName = Thread.currentThread().getName()
println “>>> DEBUG: Thread ‘${threadName}’ is freezing for 20 seconds…”

// This freezes the executor thread
Thread.sleep(20000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<![CDATA[String threadName = Thread.currentThread().getName()
println “>>> DEBUG: Thread ‘${threadName}’ is freezing for 10 seconds…”

// This freezes the executor thread
Thread.sleep(10000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<![CDATA[String threadName = Thread.currentThread().getName()
println “>>> DEBUG: Thread ‘${threadName}’ is freezing for 20 seconds…”

// This freezes the executor thread
Thread.sleep(20000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<bpmndi:BPMNDiagram id=“BPMNDiagram_parentprocessCopy”>
<bpmndi:BPMNPlane bpmnElement=“parentprocessCopy” id=“BPMNPlane_parentprocessCopy”>
<bpmndi:BPMNShape bpmnElement=“startEvent1” id=“BPMNShape_startEvent1”>
<omgdc:Bounds height=“30.0” width=“30.0” x=“100.0” y=“163.0”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-08EB496C-2449-4CBB-8ECB-A2C5B4DDEB2E” id=“BPMNShape_sid-08EB496C-2449-4CBB-8ECB-A2C5B4DDEB2E”>
<omgdc:Bounds height=“79.99999999999999” width=“100.0” x=“389.99999418854725” y=“59.99999910593034”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-1E351B2A-4031-4470-8F76-0BA2F4818468” id=“BPMNShape_sid-1E351B2A-4031-4470-8F76-0BA2F4818468”>
<omgdc:Bounds height=“40.0” width=“40.0” x=“164.99999754130843” y=“149.99999776482585”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-59F36139-D31A-4237-A41E-B7E2D30B3E97” id=“BPMNShape_sid-59F36139-D31A-4237-A41E-B7E2D30B3E97”>
<omgdc:Bounds height=“40.0” width=“40.0” x=“644.9999812245372” y=“164.99999508261692”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-D228A6D8-790D-454A-BA0C-14F524FBA53A” id=“BPMNShape_sid-D228A6D8-790D-454A-BA0C-14F524FBA53A”>
<omgdc:Bounds height=“28.0” width=“28.0” x=“704.5999648779635” y=“170.99999754130843”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“testnode02” id=“BPMNShape_testnode02”>
<omgdc:Bounds height=“79.99999999999997” width=“100.0” x=“389.99998837709455” y=“149.9999955296517”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-B0A3FB4A-953A-4D35-A790-0154E2FE9159” id=“BPMNShape_sid-B0A3FB4A-953A-4D35-A790-0154E2FE9159”>
<omgdc:Bounds height=“79.99999999999994” width=“99.99999999999994” x=“389.999982565642” y=“254.99999620020395”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-D69DEFA4-C91D-4BFE-823E-BD46E9212CAE” id=“BPMNShape_sid-D69DEFA4-C91D-4BFE-823E-BD46E9212CAE”>
<omgdc:Bounds height=“80.0” width=“100.0” x=“629.9999593198324” y=“1064.9999877065422”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-3BE09084-B9E0-4900-9917-65C3009CEE7E” id=“BPMNShape_sid-3BE09084-B9E0-4900-9917-65C3009CEE7E”>
<omgdc:Bounds height=“80.0” width=“100.0” x=“614.9999816715721” y=“824.9999754130846”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-1E758DB1-5BE3-42A7-A394-F29D53CD7F0C” id=“BPMNShape_sid-1E758DB1-5BE3-42A7-A394-F29D53CD7F0C”>
<omgdc:Bounds height=“55.000000372529” width=“64.00000055134296” x=“299.9999955296517” y=“150.49999757111075”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-3BFA51A2-7FF3-4F72-A8EA-C1224DD98C9D” id=“BPMNShape_sid-3BFA51A2-7FF3-4F72-A8EA-C1224DD98C9D”>
<omgdc:Bounds height=“50.00000043213362” width=“51.0000007301569” x=“299.9999955296517” y=“209.9999968707562”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-26E82376-A659-4C88-8701-22AE9C8D76D2” id=“BPMNShape_sid-26E82376-A659-4C88-8701-22AE9C8D76D2”>
<omgdc:Bounds height=“43.000000551342964” width=“50.0” x=“554.9999917298557” y=“539.9999919533731”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-AEE75275-0AC8-42BE-843F-24EF4F858742” id=“BPMNShape_sid-AEE75275-0AC8-42BE-843F-24EF4F858742”>
<omgdc:Bounds height=“50.00000043213356” width=“51.0000007301569” x=“554.4999831020835” y=“749.9999776557092”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-FCDC240D-DD77-4A3F-8BEF-BA25621CA7D2” id=“BPMNShape_sid-FCDC240D-DD77-4A3F-8BEF-BA25621CA7D2”>
<omgdc:Bounds height=“80.0” width=“100.0” x=“614.999990835786” y=“479.99997854232856”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-0BE53FDE-D653-4E2F-B1EC-19AA851C1005” id=“BPMNShape_sid-0BE53FDE-D653-4E2F-B1EC-19AA851C1005”>
<omgdc:Bounds height=“80.0” width=“100.0” x=“614.9999816715721” y=“599.9999910593034”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-23CF6F13-7D40-4B37-903C-29D1C3002634” id=“BPMNShape_sid-23CF6F13-7D40-4B37-903C-29D1C3002634”>
<omgdc:Bounds height=“80.0” width=“100.0” x=“614.9999816715721” y=“704.9999894946815”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNEdge bpmnElement=“sid-340755BA-22ED-4D68-8EB9-854A09719704” id=“BPMNEdge_sid-340755BA-22ED-4D68-8EB9-854A09719704” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“25.50000036507845” flowable:targetDockerY=“25.00000021606681”>
<omgdi:waypoint x=“198.63799607658757” y=“176.31021674848748”></omgdi:waypoint>
<omgdi:waypoint x=“299.99999552965164” y=“223.20284387781257”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-B40DD071-A1DB-4F95-B4F1-D4226E72120B” id=“BPMNEdge_sid-B40DD071-A1DB-4F95-B4F1-D4226E72120B” flowable:sourceDockerX=“50.0” flowable:sourceDockerY=“39.99999999999999” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“489.9499941885473” y=“118.86999930000309”></omgdi:waypoint>
<omgdi:waypoint x=“650.4627352160829” y=“179.51612407393947”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-20FA64AA-7671-4550-92F0-3C6D7912560A” id=“BPMNEdge_sid-20FA64AA-7671-4550-92F0-3C6D7912560A” flowable:sourceDockerX=“32.00000027567148” flowable:sourceDockerY=“27.5000001862645” flowable:targetDockerX=“50.0” flowable:targetDockerY=“39.999999999999986”>
<omgdi:waypoint x=“363.9499960809944” y=“181.5499973731391”></omgdi:waypoint>
<omgdi:waypoint x=“389.9999883770945” y=“184.44444062333915”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-5C96B7D1-A024-4852-B6FA-EBF512123036” id=“BPMNEdge_sid-5C96B7D1-A024-4852-B6FA-EBF512123036” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“50.0” flowable:targetDockerY=“39.99999999999999”>
<omgdi:waypoint x=“200.65307448464947” y=“165.69230547744849”></omgdi:waypoint>
<omgdi:waypoint x=“389.9999941885472” y=“113.7117637293974”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-6A3A91C1-104E-4C18-BADA-8552CC9962CC” id=“BPMNEdge_sid-6A3A91C1-104E-4C18-BADA-8552CC9962CC” flowable:sourceDockerX=“50.0” flowable:sourceDockerY=“39.999999999999986” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“489.9499883770942” y=“188.8888842838783”></omgdi:waypoint>
<omgdi:waypoint x=“645.4347638847813” y=“185.43369078621043”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-5DD49702-3F62-40C6-9D85-EC902555843B” id=“BPMNEdge_sid-5DD49702-3F62-40C6-9D85-EC902555843B” flowable:sourceDockerX=“50.0” flowable:sourceDockerY=“40.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“664.9999816322724” y=“599.9999910593034”></omgdi:waypoint>
<omgdi:waypoint x=“664.999981244138” y=“204.90217686132473”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-FFBEB59C-1CC8-49EC-8E0B-06B5C920FDDB” id=“BPMNEdge_sid-FFBEB59C-1CC8-49EC-8E0B-06B5C920FDDB” flowable:sourceDockerX=“50.0” flowable:sourceDockerY=“40.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“664.9999816452759” y=“824.9999754130846”></omgdi:waypoint>
<omgdi:waypoint x=“664.9999812376525” y=“204.90145484310295”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-6020CE75-DD29-4BF1-89AC-4E71903680BD” id=“BPMNEdge_sid-6020CE75-DD29-4BF1-89AC-4E71903680BD” flowable:sourceDockerX=“50.0” flowable:sourceDockerY=“40.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“664.999981639641” y=“704.9999894946815”></omgdi:waypoint>
<omgdi:waypoint x=“664.9999812404628” y=“204.90176770475574”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-15E5786E-CE59-4B6F-8F1A-C562B55255A8” id=“BPMNEdge_sid-15E5786E-CE59-4B6F-8F1A-C562B55255A8” flowable:sourceDockerX=“25.50000036507845” flowable:sourceDockerY=“25.00000021606678” flowable:targetDockerX=“50.0” flowable:targetDockerY=“40.0”>
<omgdi:waypoint x=“603.5638727060735” y=“799.9499780878427”></omgdi:waypoint>
<omgdi:waypoint x=“627.2222036597909” y=“824.9999754130846”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-2D8269F6-A43D-478E-B812-CC8861011B0A” id=“BPMNEdge_sid-2D8269F6-A43D-478E-B812-CC8861011B0A” flowable:sourceDockerX=“15.0” flowable:sourceDockerY=“15.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“129.85466958315442” y=“176.2966450925603”></omgdi:waypoint>
<omgdi:waypoint x=“167.0512801715902” y=“172.04615218853195”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-D624AFB2-0D41-4B73-BCDE-9D86EA4B1B19” id=“BPMNEdge_sid-D624AFB2-0D41-4B73-BCDE-9D86EA4B1B19” flowable:sourceDockerX=“49.99999999999997” flowable:sourceDockerY=“39.99999999999997” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“489.94998256564185” y=“270.5555513617066”></omgdi:waypoint>
<omgdi:waypoint x=“651.567145474745” y=“191.55074142219925”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-F3B152F4-3046-4D37-A37F-B4574072A3DF” id=“BPMNEdge_sid-F3B152F4-3046-4D37-A37F-B4574072A3DF” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“50.0” flowable:targetDockerY=“40.0”>
<omgdi:waypoint x=“196.53715892591333” y=“178.412648168”></omgdi:waypoint>
<omgdi:waypoint x=“614.9999908357859” y=“483.5416467020217”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-BB288F49-8986-4D03-B52E-288C3D9E85CD” id=“BPMNEdge_sid-BB288F49-8986-4D03-B52E-288C3D9E85CD” flowable:sourceDockerX=“25.50000036507845” flowable:sourceDockerY=“25.00000021606681” flowable:targetDockerX=“49.99999999999997” flowable:targetDockerY=“39.99999999999997”>
<omgdi:waypoint x=“350.9499962598086” y=“248.33624317503484”></omgdi:waypoint>
<omgdi:waypoint x=“389.999982565642” y=“268.7991201748517”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-BBCBD707-8CBB-4088-9E02-A307A75F810D” id=“BPMNEdge_sid-BBCBD707-8CBB-4088-9E02-A307A75F810D” flowable:sourceDockerX=“50.0” flowable:sourceDockerY=“40.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“664.9999896881741” y=“479.99997854232856”></omgdi:waypoint>
<omgdi:waypoint x=“664.9999817969086” y=“204.90295808422115”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-296EB00B-2935-4B6B-AFBB-257E5E548EED” id=“BPMNEdge_sid-296EB00B-2935-4B6B-AFBB-257E5E548EED” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“50.0” flowable:targetDockerY=“40.0”>
<omgdi:waypoint x=“194.07677470008667” y=“180.87313547450918”></omgdi:waypoint>
<omgdi:waypoint x=“631.6086779474654” y=“704.9999894946815”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-26ED17CA-F02D-4453-B2A2-66879B34DEAA” id=“BPMNEdge_sid-26ED17CA-F02D-4453-B2A2-66879B34DEAA” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“32.00000027567148” flowable:targetDockerY=“27.5000001862645”>
<omgdi:waypoint x=“203.91455265635298” y=“171.0296751948035”></omgdi:waypoint>
<omgdi:waypoint x=“299.99999552965164” y=“176.25850112478912”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-C4F40405-CFC6-45C4-9966-4C03F06035E6” id=“BPMNEdge_sid-C4F40405-CFC6-45C4-9966-4C03F06035E6” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“25.50000036507845” flowable:targetDockerY=“25.00000021606678”>
<omgdi:waypoint x=“192.88024752819922” y=“182.06953832387205”></omgdi:waypoint>
<omgdi:waypoint x=“563.677669321388” y=“749.9999776557092”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-C5A2D04A-9929-4471-A6FF-CCFB9DEA91F4” id=“BPMNEdge_sid-C5A2D04A-9929-4471-A6FF-CCFB9DEA91F4” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“14.0” flowable:targetDockerY=“14.0”>
<omgdi:waypoint x=“684.9314048872809” y=“184.99999599774577”></omgdi:waypoint>
<omgdi:waypoint x=“704.5999648779635” y=“184.99999689911272”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-FD3B8E4D-779E-42FD-BD0C-8BC5BE389A5E” id=“BPMNEdge_sid-FD3B8E4D-779E-42FD-BD0C-8BC5BE389A5E” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“25.0” flowable:targetDockerY=“21.500000275671482”>
<omgdi:waypoint x=“195.0193815964239” y=“179.9306080664738”></omgdi:waypoint>
<omgdi:waypoint x=“558.3073596163845” y=“539.9999919533731”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-4A9CD1E6-3D3D-4B49-92B2-AB1C49707D6F” id=“BPMNEdge_sid-4A9CD1E6-3D3D-4B49-92B2-AB1C49707D6F” flowable:sourceDockerX=“25.0” flowable:sourceDockerY=“21.500000275671482” flowable:targetDockerX=“50.0” flowable:targetDockerY=“40.0”>
<omgdi:waypoint x=“603.2249737141062” y=“582.949992504716”></omgdi:waypoint>
<omgdi:waypoint x=“621.6858550275167” y=“599.9999910593034”></omgdi:waypoint>
</bpmndi:BPMNEdge>
</bpmndi:BPMNPlane>
</bpmndi:BPMNDiagram>
den

adding process B here.

Process B <?xml version="1.0" encoding="UTF-8"?> >> DEBUG: Thread '${threadName}' is freezing for 60 seconds..."

// This freezes the executor thread
Thread.sleep(60000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<![CDATA[String threadName = Thread.currentThread().getName()
println “>>> DEBUG: Thread ‘${threadName}’ is freezing for 60 seconds…”

// This freezes the executor thread
Thread.sleep(60000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<![CDATA[String threadName = Thread.currentThread().getName()
println “>>> DEBUG: Thread ‘${threadName}’ is freezing for 60 seconds…”

// This freezes the executor thread
Thread.sleep(60000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<![CDATA[String threadName = Thread.currentThread().getName()
println “>>> DEBUG: Thread ‘${threadName}’ is freezing for 60 seconds…”

// This freezes the executor thread
Thread.sleep(60000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<![CDATA[String threadName = Thread.currentThread().getName()
println “>>> DEBUG: Thread ‘${threadName}’ is freezing for 60 seconds…”

// This freezes the executor thread
Thread.sleep(60000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<![CDATA[String threadName = Thread.currentThread().getName()
println “>>> DEBUG: Thread ‘${threadName}’ is freezing for 60 seconds…”

// This freezes the executor thread
Thread.sleep(60000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<![CDATA[String threadName = Thread.currentThread().getName()
println “>>> DEBUG: Thread ‘${threadName}’ is freezing for 60 seconds…”

// This freezes the executor thread
Thread.sleep(60000)

println “>>> DEBUG: Thread ‘${threadName}’ is now awake!”]]>

<bpmndi:BPMNDiagram id=“BPMNDiagram_testnode02”>
<bpmndi:BPMNPlane bpmnElement=“testnode02” id=“BPMNPlane_testnode02”>
<bpmndi:BPMNShape bpmnElement=“startEvent1” id=“BPMNShape_startEvent1”>
<omgdc:Bounds height=“30.0” width=“30.0” x=“89.9999986588955” y=“149.99999776482585”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-03012528-DD97-4222-869F-AFA2BC6EAF6A” id=“BPMNShape_sid-03012528-DD97-4222-869F-AFA2BC6EAF6A”>
<omgdc:Bounds height=“28.0” width=“28.0” x=“794.9999881535771” y=“149.99999776482585”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-DC1D6D39-9CB2-4D07-A5AF-4E9308DC1ADB” id=“BPMNShape_sid-DC1D6D39-9CB2-4D07-A5AF-4E9308DC1ADB”>
<omgdc:Bounds height=“80.0” width=“99.99999999999989” x=“419.99998748302494” y=“704.9999789893632”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-16A7E0C7-0537-4E08-9027-5D65FC155A96” id=“BPMNShape_sid-16A7E0C7-0537-4E08-9027-5D65FC155A96”>
<omgdc:Bounds height=“80.0” width=“99.99999999999994” x=“419.9999812245374” y=“29.999999105930343”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-5A53ADC1-1163-41CE-B7E1-CF39B95D5611” id=“BPMNShape_sid-5A53ADC1-1163-41CE-B7E1-CF39B95D5611”>
<omgdc:Bounds height=“80.0” width=“99.99999999999994” x=“419.9999874830249” y=“599.9999910593034”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-C4D6A57E-86C9-448C-B7D3-5A62131BB4E3” id=“BPMNShape_sid-C4D6A57E-86C9-448C-B7D3-5A62131BB4E3”>
<omgdc:Bounds height=“80.00000000000003” width=“100.00000000000006” x=“419.99997496605005” y=“134.99999396502992”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-89EA964D-CBD0-4859-8325-56589FC4A4A2” id=“BPMNShape_sid-89EA964D-CBD0-4859-8325-56589FC4A4A2”>
<omgdc:Bounds height=“80.00000000000003” width=“100.0” x=“419.9999937415124” y=“244.99998860061206”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-90BBCE40-657D-4245-B654-B17B892ACC6E” id=“BPMNShape_sid-90BBCE40-657D-4245-B654-B17B892ACC6E”>
<omgdc:Bounds height=“80.0” width=“99.99999999999994” x=“419.9999874830249” y=“344.99999485909944”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-BF82A2EC-7B89-461B-AF9E-25B200DA8A5B” id=“BPMNShape_sid-BF82A2EC-7B89-461B-AF9E-25B200DA8A5B”>
<omgdc:Bounds height=“80.0” width=“100.0” x=“419.99998748302494” y=“464.99999307096016”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-2A7EC477-4177-4EE0-BC57-D1BBA5F8E370” id=“BPMNShape_sid-2A7EC477-4177-4EE0-BC57-D1BBA5F8E370”>
<omgdc:Bounds height=“40.0” width=“40.0” x=“165.49999798834324” y=“145.4999966472388”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNShape bpmnElement=“sid-AF9D8DCE-5764-43CE-A045-65F83977050A” id=“BPMNShape_sid-AF9D8DCE-5764-43CE-A045-65F83977050A”>
<omgdc:Bounds height=“40.0” width=“40.0” x=“663.4999847337606” y=“272.4999934732915”></omgdc:Bounds>
</bpmndi:BPMNShape>
<bpmndi:BPMNEdge bpmnElement=“sid-65CA8AB5-04D2-4C9E-9153-FE054114E7E5” id=“BPMNEdge_sid-65CA8AB5-04D2-4C9E-9153-FE054114E7E5” flowable:sourceDockerX=“15.0” flowable:sourceDockerY=“15.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“119.9496574721223” y=“165.09285297361748”></omgdi:waypoint>
<omgdi:waypoint x=“165.58631090795168” y=“165.37654013033733”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-64BEDC3D-61DC-4283-8A3C-DD0B5CA61483” id=“BPMNEdge_sid-64BEDC3D-61DC-4283-8A3C-DD0B5CA61483” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“50.00000000000003” flowable:targetDockerY=“40.000000000000014”>
<omgdi:waypoint x=“204.80218274474683” y=“166.14463937873595”></omgdi:waypoint>
<omgdi:waypoint x=“419.99997496605005” y=“173.3303985192392”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-04F0CEDA-ECE7-4F9F-BCC6-BD0CB21477C3” id=“BPMNEdge_sid-04F0CEDA-ECE7-4F9F-BCC6-BD0CB21477C3” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“49.99999999999997” flowable:targetDockerY=“40.0”>
<omgdi:waypoint x=“192.97796229764978” y=“177.9717033910552”></omgdi:waypoint>
<omgdi:waypoint x=“446.016847938655” y=“599.9999910593034”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-8BADB97E-6135-4769-BE67-16D7BC33E2DB” id=“BPMNEdge_sid-8BADB97E-6135-4769-BE67-16D7BC33E2DB” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“49.99999999999997” flowable:targetDockerY=“40.0”>
<omgdi:waypoint x=“200.43624786380138” y=“160.4736807329111”></omgdi:waypoint>
<omgdi:waypoint x=“419.9999812245374” y=“86.76704711390184”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-00B7A315-4E1E-4641-8573-16B52386D00D” id=“BPMNEdge_sid-00B7A315-4E1E-4641-8573-16B52386D00D” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“49.99999999999994” flowable:targetDockerY=“40.0”>
<omgdi:waypoint x=“192.06918198768133” y=“178.8804185018399”></omgdi:waypoint>
<omgdi:waypoint x=“450.36236897302337” y=“704.9999789893632”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-05334B06-02C8-4CAC-B632-AF9FD2D0B884” id=“BPMNEdge_sid-05334B06-02C8-4CAC-B632-AF9FD2D0B884” flowable:sourceDockerX=“49.99999999999997” flowable:sourceDockerY=“40.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“519.9499874830249” y=“363.33723078955813”></omgdi:waypoint>
<omgdi:waypoint x=“669.5457364852614” y=“298.53063084541367”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-A943BF8E-3514-460D-8008-CC9A4C3EF4F4” id=“BPMNEdge_sid-A943BF8E-3514-460D-8008-CC9A4C3EF4F4” flowable:sourceDockerX=“50.0” flowable:sourceDockerY=“40.000000000000014” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“519.9499937415118” y=“286.75467365539475”></omgdi:waypoint>
<omgdi:waypoint x=“664.1363488259316” y=“291.8212599879513”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-4F113895-2457-44A8-8D19-FF17678BAC2F” id=“BPMNEdge_sid-4F113895-2457-44A8-8D19-FF17678BAC2F” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“50.0” flowable:targetDockerY=“40.0”>
<omgdi:waypoint x=“194.59579112696855” y=“176.35406263265108”></omgdi:waypoint>
<omgdi:waypoint x=“436.48010618799196” y=“464.99999307096016”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-F83BAE96-64E2-4B12-9AF1-34A3DACBDBB7” id=“BPMNEdge_sid-F83BAE96-64E2-4B12-9AF1-34A3DACBDBB7” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“50.0” flowable:targetDockerY=“40.000000000000014”>
<omgdi:waypoint x=“199.5479380724152” y=“171.40104840965003”></omgdi:waypoint>
<omgdi:waypoint x=“419.9999937415124” y=“263.99823223203714”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-17B31EBB-364E-4674-B970-14DB6AA4B017” id=“BPMNEdge_sid-17B31EBB-364E-4674-B970-14DB6AA4B017” flowable:sourceDockerX=“49.99999999999997” flowable:sourceDockerY=“40.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“508.3340277688758” y=“109.94999910593035”></omgdi:waypoint>
<omgdi:waypoint x=“673.7064065434388” y=“282.2925871248286”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-D0A94340-37F0-498E-9EAA-826FD6E98EEB” id=“BPMNEdge_sid-D0A94340-37F0-498E-9EAA-826FD6E98EEB” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“49.99999999999997” flowable:targetDockerY=“40.0”>
<omgdi:waypoint x=“196.76120098151526” y=“174.18853845505393”></omgdi:waypoint>
<omgdi:waypoint x=“419.9999874830248” y=“346.42354383677633”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-E12BD1A2-1783-440F-84FD-81501C4C2B80” id=“BPMNEdge_sid-E12BD1A2-1783-440F-84FD-81501C4C2B80” flowable:sourceDockerX=“49.99999999999994” flowable:sourceDockerY=“40.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“488.8493248602168” y=“704.9999789893632”></omgdi:waypoint>
<omgdi:waypoint x=“677.0885732390101” y=“306.05461050727797”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-3D0C5A8C-BE14-4B43-8270-F1F25712724D” id=“BPMNEdge_sid-3D0C5A8C-BE14-4B43-8270-F1F25712724D” flowable:sourceDockerX=“50.00000000000003” flowable:sourceDockerY=“40.000000000000014” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“519.9499749660501” y=“202.4900394307108”></omgdi:waypoint>
<omgdi:waypoint x=“670.5860612179707” y=“285.40029581677”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-C55DF709-BCD1-4370-BEDE-6514E736E03C” id=“BPMNEdge_sid-C55DF709-BCD1-4370-BEDE-6514E736E03C” flowable:sourceDockerX=“20.0” flowable:sourceDockerY=“20.0” flowable:targetDockerX=“14.0” flowable:targetDockerY=“14.0”>
<omgdi:waypoint x=“693.3571700755638” y=“282.38188354026215”></omgdi:waypoint>
<omgdi:waypoint x=“799.2180940506113” y=“173.97995095688705”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-C548F1F9-F54D-4731-ADD8-47625B3F593D” id=“BPMNEdge_sid-C548F1F9-F54D-4731-ADD8-47625B3F593D” flowable:sourceDockerX=“49.99999999999997” flowable:sourceDockerY=“40.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“494.5448074813495” y=“599.9999910593034”></omgdi:waypoint>
<omgdi:waypoint x=“675.8885765620772” y=“304.85761382203737”></omgdi:waypoint>
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge bpmnElement=“sid-A972D8BE-CA90-43F2-BE46-0F79444BD6E5” id=“BPMNEdge_sid-A972D8BE-CA90-43F2-BE46-0F79444BD6E5” flowable:sourceDockerX=“50.0” flowable:sourceDockerY=“40.0” flowable:targetDockerX=“20.0” flowable:targetDockerY=“20.0”>
<omgdi:waypoint x=“510.1379870421574” y=“464.99999307096016”></omgdi:waypoint>
<omgdi:waypoint x=“673.4765106102755” y=“302.4515780351152”></omgdi:waypoint>
</bpmndi:BPMNEdge>
</bpmndi:BPMNPlane>
</bpmndi:BPMNDiagram>

The lockOwner indeed gets set optimistically (which in a good tuned system is faster), however if the queue is full, the DefaultAsyncJobExecutor#unacquireJobAfterRejection kicks in, which unlocks the job again.

This happens here:

protected boolean executeAsyncJob(final JobInfo job, Runnable runnable) {
    try {
        taskExecutor.execute(runnable);
        return true;

    } catch (RejectedExecutionException e) {
        sendRejectedEvent(job);
        unacquireJobAfterRejection(job);

        // Job queue full, returning false so (if wanted) the acquiring can be throttled
        return false;
    }
}

Did you have any customizations in this area that could influence this?

Sidenote : not that CPU’s don’t influence the number of threads available to the job executor, this is a different setting and does not need to related to CPU cores, as all realistic job logic has IO that would interrupt the current work on OS level.

Hi @joram ,

i dont have any extra customization from my end.
I have confirmed that my properties file is being loaded because flowable.process.async.executor.lock-owner=flowable-node-01 is working correctly (I see the name in the LOCK_OWNER_ column).

However, despite setting:
flowable.process.async.executor.core-pool-size=5 flowable.process.async.executor.max-pool-size=10 flowable.process.async.executor.thread-pool-queue-size=10

My logs still show: Creating executor service with corePoolSize 8, maxPoolSize 8 and queue of size 100

Am i missing any parameter or using a wrong naming convention?

Are you using Spring Boot Starters? Can you share the logs where you are getting this?

HI @filiphr ,

Yes, this is a fresh installation of the Flowable Rest application on a VM. Based on the startup logs, it is indeed using Spring Boot (v3.5.4) and the Flowable Spring Boot Starters.

Here are the relevant logs from catalina.out showing the initialization:

:: Flowable :: (v7.2.0) :: Spring Boot :: (v3.5.4)

2026-02-23T16:23:04.427+01:00 INFO 22554 — [flowable-rest-node1] [ main] o.flowable.rest.app.ServletInitializer : Starting ServletInitializer v7.2.0 using Java 21.0.10 with PID 22554 (/data/tomcat-11/webapps/flowable-rest/WEB-INF/classes started by root in /)
2026-02-23T16:23:04.429+01:00 INFO 22554 — [flowable-rest-node1] [ main] o.flowable.rest.app.ServletInitializer : No active profile set, falling back to 1 default profile: “default”
2026-02-23T16:23:05.823+01:00 INFO 22554 — [flowable-rest-node1] [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1100 ms
2026-02-23T16:23:06.147+01:00 INFO 22554 — [flowable-rest-node1] [ main] o.f.s.b.ProcessEngineAutoConfiguration : No deployment resources were found for autodeployment
2026-02-23T16:23:06.168+01:00 INFO 22554 — [flowable-rest-node1] [ main] o.f.c.e.i.a.DefaultAsyncTaskExecutor : Creating thread pool queue of size 100
2026-02-23T16:23:06.169+01:00 INFO 22554 — [flowable-rest-node1] [ main] o.f.c.e.i.a.DefaultAsyncTaskExecutor : Creating thread factory with naming pattern flowable-async-task-invoker-%d
2026-02-23T16:23:06.169+01:00 INFO 22554 — [flowable-rest-node1] [ main] o.f.c.e.i.a.DefaultAsyncTaskExecutor : Creating executor service with corePoolSize 8, maxPoolSize 8 and keepAliveTime 5000
2026-02-23T16:23:06.304+01:00 INFO 22554 — [flowable-rest-node1] [ main] o.f.s.b.c.CmmnEngineAutoConfiguration : No deployment resources were found for autodeployment

Hey @ROY,

Thanks for sharing the logs. What you are seeing is actually coming from the flowable.task-invoker which is in

This is what is used for something else, the FutureJavaDelegate.

The properties you are mentioning are defined in

and this is what would be used to determine the thread pool for the async job execution.

Cheers,
Filip

HI @filiphr ,

Thank you for the detailed breakdown of the task-invoker. I realize now I was looking at the wrong logs; I will ignore the task-invoker for now as my primary goal is resolving the load-sharing issue between Node 1 and Node 2.

I have tried your suggestion regarding the Async History Executor, and I have also double-checked my Async Job Executor (for ACT_RU_JOB) properties. Despite setting the following, Node 1 still ‘hoards’ all the jobs and Node 2 remains idle:

Async Job Executor Properties Tried:

  • flowable.process.async.executor.thread-pool-queue-size=10

  • flowable.process.async.executor.core-pool-size=5

  • flowable.process.async.executor.max-pool-size=10

  • flowable.process.async.executor.max-async-jobs-due-per-acquisition=1

Async History Executor Properties Tried (as per your suggestion):

  • flowable.process.async-history.executor.thread-pool-queue-size=9

  • flowable.process.async-history.executor.core-pool-size=9

  • flowable.process.async-history.executor.max-pool-size=9

  • flowable.process.async-history.executor.max-async-jobs-due-per-acquisition=1

The Result: Node 1 still locks every single job under its own LOCK_OWNER_ immediately. It processes only 8 concurrently, It doesnt use my properties file.

Hey @ROY,

I have not tried the processes you’ve shared. However, looking at your Async Job executor properties:

flowable.process.async.executor.thread-pool-queue-size=10
flowable.process.async.executor.core-pool-size=5
flowable.process.async.executor.max-pool-size=10
flowable.process.async.executor.max-async-jobs-due-per-acquisition=1

If we assume that you have started 30 jobs on Node 1 and each job blocks for 1 minutes. The following should happen:

  • 5 jobs would start immediately
  • 15 jobs would go in the queue - this would cause the pool size to increase and expand to 10 (the max-pool-size)
  • The remaining 10 jobs should be without a lock owner and should be picked up by the Node 2 async executor

Having written this, I might have understood what is causing this. In the out-of-the-box configuration from Open Source we are actually going to use the Spring Boot auto configured TaskExecutor. So the properties you are looking for should be:

spring.task.execution.pool.queue-capacity=10
spring.task.execution.pool.core-size=5
spring.task.execution.pool.max-size=10

With Spring Boot

flowable.process.async.executor.thread-pool-queue-size=10
flowable.process.async.executor.core-pool-size=5
flowable.process.async.executor.max-pool-size=10

are actually getting ignored.

Sorry, I mixed up things a bit. In our enterprise Spring Boot properties are exposing a bit more configuration options to more granularly configure the task executors for the different async executors.

The Spring Boot defaults are:

spring.task.execution.pool.queue-capacity=Integer.MAX_VALUE
spring.task.execution.pool.core-size=8
spring.task.execution.pool.max-size=Integer.MAX_VALUE

which actually means that it will hoard all jobs and there will only be 8 cores running, due to the queue capacity being unbounded the thread pool will never start increasing (it starts increasing once the queue is full).

HI @filiphr @joram

That was it! Using the spring.task.execution.pool.* properties instead of the flowable.process.async.executor.* properties fixed the issue immediately.

Because the default Spring queue-capacity was Integer.MAX_VALUE, Node 1 was indeed hoarding every job. Once I limited the Spring queue capacity, the unacquireJobAfterRejection logic triggered perfectly, and Node 2 started picking up the excess load.

As a followup topic, it would be better to update documents like Advanced · Flowable Open Source Documentation about how spring parameters are used.

Thank you again for helping me to understand this better.

Regards,
Roy