Parallel multi-instance subprocess not working

Flowable Engine
#1

Hi guys,
when trying to use the power of Flowable I hit a problem and need your help.
My scenario is very simple - I have a big number of files in one directory and for performance reasons I want to upload them in parallel. Here is a simplified version of my workflow:

image
image3471×1730 428 KB

Problem is that when I run this process I get a lot of FlowableOptimisticLockingExceptions nevertheless non of the async “Upload File Tasks” finish at the same time. I even put a breakpoint in “UploadFile” and resumed each execution one after another and the same exception was thrown.
Can you please tell me what I am doing wrong? It should be possible to run service tasks in parallel.

best regards
Stoyan

#2

Hey @stojoman,

When using parallel multi instance sub process with a lot of async non exclusive service tasks there will be FlowableOptimisticLockingException(s) due to the fact that all of them are trying to update same variables on the process instance. We are working on some idea for how to optimise this.

In any case what might be interesting to you is the new true parallel service task execution capabilities of Flowable. With this you can do the parallel multi instance subprocess trully in parallel in one transaction. I would suggest reading the True Parallel Service Task Execution blog post. You will need to rewrite your uploadFile JavaDelegate to be FutureJavaDelegate and do not make the service task async. You can read more about the FutureJavaDelegate here in the documentation.

I think that for your use case, doing this would yield to a better performance than the async one (even without the Optimistic locking exceptions).

Cheers,
Filip

#3

Hi @filiphr ,
thank you very much for the help. I have followed your advice and implemented parallel processing using FutureJavaDelegate. What I noticed is that configuration that we have for async tasks
for retry and timeout is not applied to FutureJavaDelegate.
How can we configure this? What I noticed is that if async thread that is started from some “future task” takes a long time to finish task is retried infinitely.

thanks
Stoyan

#4

Hey @stojoman,

When you are using the FutureJavaDelegate then the execution is happening in the same transaction. It is exactly like using the standard JavaDelegate with the difference being that the actual execution of that service logic is done on a different thread and thus the engine can continue executing operations.

There are no retries or timeouts applied to it.

I am not quite following you here. What is retried infinitely. If there are no other tasks the main thread (the thread of executing the Commands) will wait. Have a look at the WaitForAnyFutureToFinishOperation.

Cheers,
Filip