Hello,
I have a BPMN with a multiple instances subprocess.
This subprocess will have 2 service tasks (quite fasts; making simple calls to remote rest API).
This subprocess may treat a list of 200K items with it’s multiInstanceLoop. Each item can be treated independantly so the loop can be parallelized (isSequential=“false”).
With a sigle instance of the engine the full treatment takes around 1 minute to treat 1000 items.
I tried to use multiple instances of the flowable engine and it looks like that the default behaviour is that the engine entering the subprocess will treat every loop iteration (load is not spread across the differents engines).
I tried many configuration changes (global acquire lock ; default-async-job-acquire-wait-time ; .max-async-jobs-due-per-acquisition; async-job-lock-time…) but when i succeed to have multiples engines working on the loop at the same time, it is really slower that with a single engine and it ends up getting stucked… probably due to optimistic lock.
1 - With my scenario (simple subprocess with only service tasks; fast parallelizable loop; 200K iterations), is it a good idea to try to spread the load over multiple instances of the flowable engine?
2 - If yes, could somebody suggest me a suitable configuration?
Thanks