Using CallActivity

Flowable Engine
21

Hi Martin,

Thanks. For this answer.
I have another question:
“But if I start a diagram which does not have CallActivity in it, the result is the same - I have a process which is not progressing at all.
Anyway, could you help me in order to resolve that? Why is this OptimisticLockingException thrown- what have caused it(or to ask, which two parallel transactions are executed against the same table at the same time)?”
Could you give me an answer to this one?

Also, I have written you a personal message? Would you be able to see it?

Thanks a lot and best regards,
Encho

22

Hi Encho.

When I run the test without CallActivity (without multiinstance, without async) execution works fine. If you want to resolve it create a jUnit test to know exactly where the problem is.

The optimistic locking exception which is thrown:

[flowable-async-job-executor-thread-2] ERROR org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler  - Job 50 failed
org.flowable.common.engine.api.FlowableOptimisticLockingException: HistoricVariableInstanceEntity[id=16, name=nrOfCompletedInstances, revision=0, type=integer, longValue=1, textValue=1] was updated by another transaction concurrently

The problem is that variable nrOfCompletedInstances is updated from each job. That’s why optimistic locking exception is thrown.
Solution proposal:

  1. Create async job in the loop. In that case you will avoid multiinstance behavior which is trying to update nrOfCompletedInstances
  2. change callActivity to synchronous and multiinstance to sequential. The first task in the subprocess is asynchronous. What I would expect is that jobs are generated and job executors will execute them in parallel.

Regards
Martin

23

Hello Martin,

Thanks for the answer. Before I try it I have another problem which I am trying to understand where is it coming from:
2018-07-02T17:47:54.14+0300 [APP/PROC/WEB/4] OUT { “written_at”:“2018-07-02T14:47:54.140Z”,“written_ts”:418509103545880,“component_type”:“application”,“component_id”:“77874fa5-5747-44b9-8b52-77f50e1dbc57”,“space_name”:“i321665”,“component_name”:“deploy-service”,“component_instance”:“4”,“organization_id”:"-",“correlation-id”:“177501”,“correlation_id”:"-",“organization_name”:"-",“space_id”:“3331f015-6cf6-457e-84b4-efd5af1b1684”,“container_id”:“10.0.137.106”,“type”:“log”,“logger”:“org.activiti.engine.impl.db.DbSqlSession”,“thread”:“pool-8-thread-7”,“level”:“DEBUG”,“categories”:[],“msg”:“updating: ScopeExecution[180016]” }
2018-07-02T17:47:54.14+0300 [APP/PROC/WEB/4] OUT { “written_at”:“2018-07-02T14:47:54.140Z”,“written_ts”:418509103812906,“component_type”:“application”,“component_id”:“77874fa5-5747-44b9-8b52-77f50e1dbc57”,“space_name”:“i321665”,“component_name”:“deploy-service”,“component_instance”:“4”,“organization_id”:"-",“correlation-id”:“177501”,“correlation_id”:"-",“organization_name”:"-",“space_id”:“3331f015-6cf6-457e-84b4-efd5af1b1684”,“container_id”:“10.0.137.106”,“type”:“log”,“logger”:“org.activiti.engine.impl.persistence.entity.ExecutionEntity.updateExecution”,“thread”:“pool-8-thread-7”,“level”:“DEBUG”,“categories”:[“MYBATIS”],“msg”:"==> Preparing: update ACT_RU_EXECUTION set REV_ = ?, BUSINESS_KEY_ = ?, PROC_DEF_ID_ = ?, ACT_ID_ = ?, IS_ACTIVE_ = ?, IS_CONCURRENT_ = ?, IS_SCOPE_ = ?, IS_EVENT_SCOPE_ = ?, PARENT_ID_ = ?, SUPER_EXEC_ = ?, SUSPENSION_STATE_ = ?, CACHED_ENT_STATE_ = ?, NAME_ = ? where ID_ = ? and REV_ = ? " }
2018-07-02T17:47:54.14+0300 [APP/PROC/WEB/4] OUT { “written_at”:“2018-07-02T14:47:54.142Z”,“written_ts”:418509106241716,“component_type”:“application”,“component_id”:“77874fa5-5747-44b9-8b52-77f50e1dbc57”,“space_name”:“i321665”,“component_name”:“deploy-service”,“component_instance”:“4”,“organization_id”:"-",“correlation-id”:“177501”,“correlation_id”:"-",“organization_name”:"-",“space_id”:“3331f015-6cf6-457e-84b4-efd5af1b1684”,“container_id”:“10.0.137.106”,“type”:“log”,“logger”:“org.activiti.engine.impl.persistence.entity.ExecutionEntity.updateExecution”,“thread”:“pool-8-thread-7”,“level”:“DEBUG”,“categories”:[“MYBATIS”],“msg”:"==> Parameters: 3(Integer), null, xs2-deploy:6:20018(String), deployAppSubProcess(String), false(Boolean), false(Boolean), true(Boolean), false(Boolean), 177501(String), null, 1(Integer), 6(Integer), null, 180016(String), 2(Integer)" }
2018-07-02T17:47:54.14+0300 [APP/PROC/WEB/4] OUT { “written_at”:“2018-07-02T14:47:54.145Z”,“written_ts”:418509108528218,“component_type”:“application”,“component_id”:“77874fa5-5747-44b9-8b52-77f50e1dbc57”,“space_name”:“i321665”,“component_name”:“deploy-service”,“component_instance”:“4”,“organization_id”:"-",“correlation-id”:“177501”,“correlation_id”:"-",“organization_name”:"-",“space_id”:“3331f015-6cf6-457e-84b4-efd5af1b1684”,“container_id”:“10.0.137.106”,“type”:“log”,“logger”:“org.activiti.engine.impl.persistence.entity.ExecutionEntity.updateExecution”,“thread”:“pool-8-thread-7”,“level”:“DEBUG”,“categories”:[“MYBATIS”],“msg”:"<== Updates: 0" }
2018-07-02T17:47:54.15+0300 [APP/PROC/WEB/4] OUT { “written_at”:“2018-07-02T14:47:54.157Z”,“written_ts”:418509120447383,“component_type”:“application”,“component_id”:“77874fa5-5747-44b9-8b52-77f50e1dbc57”,“space_name”:“i321665”,“component_name”:“deploy-service”,“component_instance”:“4”,“organization_id”:"-",“correlation-id”:“177501”,“correlation_id”:"-",“organization_name”:"-",“space_id”:“3331f015-6cf6-457e-84b4-efd5af1b1684”,“container_id”:“10.0.137.106”,“type”:“log”,“logger”:“org.activiti.engine.impl.interceptor.CommandContext”,“thread”:“pool-8-thread-7”,“level”:“DEBUG”,“categories”:[],“msg”:“Optimistic locking exception : org.activiti.engine.ActivitiOptimisticLockingException: ScopeExecution[180016] was updated by another transaction concurrently” }
As you can see, there is optimistic locking exception during the update of the ExecutionEntity. My question is, why this is happening? From where the ExecutionEntity is being updated. As far as I debugged, the Entity is the parent of all ConcurrentExecutions which are created.

Thanks,
Encho

24

Hi Encho,

Do you still use flowable 5 engine? I did not see this configuration in the jUnit test.

Martin

25

Hello Martin,

Please ignore my previous comment. I have spent some time debugging the whole code and see that in the ParallelMultiInstanceActivityBehaviour in the leave method there were those two lines of code which update the parent with the variables. So after some debugging, I decided that these variables NUMBER_OF_COMPLETED_INSTANCES and NUMBER_OF_ACTIVE_INSTANCES are used to determine whether the execution has finished or not. So I have changed the logic a bit. Now each ConcurrentExecution keeps the value of the variable nrOfCompletedInstances which will be either 0 or 1. 0 - if the execution has not reached the leave method, 1 - if the execution reached the leave method. With this, I have also made a change how the overall numberOfCompletedInstances is calculated. See the below code:

> private int determineNumberOfCompletedInstances(DelegateExecution miRootExecution) {
>         int result = 0;
>         for (DelegateExecution childExecution : miRootExecution.getExecutions()) {
>             int nrOfCompletedInstancesFromChild = getLoopVariable(childExecution, NUMBER_OF_COMPLETED_INSTANCES);
>             if (nrOfCompletedInstancesFromChild == 1) {
>                 result++;
>             }
>         }
>         return result;
>     }

The above method returns the number of completed instances. When the last concurrent execution pass through the leave method of the ParallelMultiInstanceActivityBehavior and the condition:

nrOfCompletedInstances >= nrOfInstances

is satisfied.

The problem now comes from the following exception:

05:56:37,809 [flowable-async-job-executor-thread-2] ERROR org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler - Job 88 failed
org.flowable.common.engine.api.FlowableOptimisticLockingException: ProcessInstance[8] was updated by another transaction concurrently
at org.flowable.common.engine.impl.db.DbSqlSession.flushUpdates(DbSqlSession.java:502)
at org.flowable.common.engine.impl.db.DbSqlSession.flush(DbSqlSession.java:289)
at org.flowable.common.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:191)
at org.flowable.common.engine.impl.interceptor.CommandContext.close(CommandContext.java:61)
at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:80)
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30)
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56)
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51)
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.executeJob(ExecuteAsyncRunnable.java:128)
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.run(ExecuteAsyncRunnable.java:116)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

As far as I understand, this exception is produced because some parallel executions try to tell the parent to continue working(update him) and thus this error is produced. Have you seen such errors before? Is there some way to gracefully ignore them? I would be very very happy for some guidance from your side in order to fix the issue :slight_smile:

Also, I disabled the retry mechanism of the JobExecutor because when the exception above rises, I see that my sub-process is re-executed which in my case is not needed because my requirement is to have processes which are executed(successfully or not) and could not be retried because of the operations they are executing inside them. - If the retry would fix the things and there is a way ti tell which to retry and which not, I would be fine with that aproach :slight_smile:

Thanks and best regards,
Encho

P.S. I am using the latest flowable-engine from Github.
The latest version of my test code is also pushed.

26

Hi Encho,

Yes sure. As you said, these errors are caused by parallel update.

Optimistic locking exception says in this case that process instance was already updated. I would not ignore it. he problem with optimistic locking is which value is correct.

I am not sure whether retry would fix the optimistic locking. You can change default retry mechanism to use your custom implementation

As I wrote Using CallActivity - #22 by martin.grofcik, from my point of view these 2 solutions are still valid.

Regards
Martin

27

Hello Martin,

Thanks for the answer.
I have already tried that and the result is a sequential completion of the sub-processes. I like this but the problem is that my requirement is to have a parallel execution of the sub-processes.
I am thinking of whether, the logic for the update of the parent process could be “synchronised” in some way. For example, when the logic reaches the point where the parent process instance needs to be updated the “synchronisation” mechanism to be triggered and to now which update of the parent process to be executed first, second and so on.

What do you say about this? Are you seeing it reasonable?

Please do not take the word synchronisation as a synch on a DB level - I mean some kind of sync between the child executions, again, not on DB level :slight_smile:

Thanks and best regards,
Encho

28

Hi Encho,

The synchronization which you propose is done in the solution proposal option2.
You can try to avoid parent execution update -> e.g. just create executions and jobs in the parallel gateway (fork) and do not join them. The last execution in the row could trigger parent to continue. That could work too.

Martin

29

Hello,

What do you mean by “create executions and jobs in the parallel gateway”? I do not get that.
Do you mean a parallel gateway in the diagram? And if yes, could you show me example how this could be achieved with CallActivity?
Something more, what about the async job in a loop which you mentioned in the solution proposal 1?
Do you mean some async service task or ?

Moreover, could you explain to me why the concurrent executions are joined at the end? What is the reason for the joining?

Thanks,
Encho

30

Hi,

It’s getting confusing to know how to help. You said you’re moving from Activiti to Flowable. Did you have a working solution under Activiti?

Regards
Paul.

31

Hello Paul,

No, we decided that if we move to flowable, we would have future support of the product.
That is the reason, I decided to experiment with the Flowable.

I do not have a solution for the issue for the moment.

Regards,
Encho

32

Makes sense. I know Martin has some heavy work on at the moment, so it may be a while before he can get back to you. Meanwhile, someone else in the community may be able to help.

Cheers
Paul.

33

Thanks Paul,

I am really grateful that Martin is able to answer my questions. :slight_smile:

34

Hello Paul and Martin,

I hope you are doing good.
Regarding my issue, I have a fix preposition. Here is an explanation about the fix:

Instead of joining the executions in the leave method why not creating a job which to join the executions and continue the process execution. The job will receive the execution which to monitor. Then the job will find all the child executions of the execution to monitor and in each will look for variable which will indicate whether the child execution has finished or not. After all child executions finish, the job will perform operation CommandContextUtil.getAgenda()
.planTakeOutgoingSequenceFlowsOperation(newExecution, true); which in my understanding will trigger the flow processing and the process instance will continue. The job will be triggered non-stop and always will be checking whether the child executions have finished.
I have developed a solution and have performed various of unit tests over it and it seems to work.
However, you are the experts in the Flowable area and I want to hear from you whether this is a good fix preposition or not? :slight_smile:
Something more, the above preposition fixes the concurrent update of the parent execution. Also, in my opinion performs a join of all child executions before continuing the process instance.

What do you say about it?

Thanks and best regards,
Encho

35

That could cause performance issues.

I am not sure about this proposal. May be in some special cases it is applicable.

Regards
Martin

36

Hello Martin,

Thanks, I will submit a PR soon and will involve you as a reviewer :slight_smile:

Regards,
Encho

37

Hello Martin,

I have a question, could you share with me how the tests in the flowable-engine are ran?
For instance, I expected that when I use the async executor to start jobs, the jobs will be executed. However, when I place a debuggers in the corresponding JobHandler, the debugger is never triggered, when I start a certain test.
Could you tell me why is that?

Thanks,
Encho

38

Hey Encho,

In order to simplify the test the async executor is not automatically started. It needs to be “manually” invoked. Have a look at the JobTestHelper and how it is used in our tests to have a look and feel how it works

39

Hello,

Thanks for the answer, I have tried it and it works - thanks a lot.
I have one more question, when I run a test, the code that the test executes sets some variables in the ExecutionEntity and after that, from another part of the code, I want to retrieve those variables. The problem is that when I require them, they are null. I have seen how the variables are persisted in the database and the session was flushed.
Is there another magic which could be done here as well?
:slight_smile:

Thanks,
Encho

40

Hello guys,

I have another proposition which could fix the issue with the CallActivity.
When the ParallelMultiInstaneBehavior is activated for some activity, a “special” listner to be created. This listener is “special” because it will be invoked by some condition. The condition in this case will be when all the concurrent executions finish. And now the workflow, when the listener is created it will be given to some global listener handler which would check whether the condition of the listener is met. If the condition is met the listener will activate the process execution which will leave the ParallelMultiInstaneBehavior.
This preposition is better than the one with the job, because it is not all the time polling given job which will not cause any performance issues. The problem is that, i do not know of such mechanism implemented in the flowable-engine at the moment.
Do you, guys as experts, think that this is a good approach?
Also, if there is a way to achieve the above explanation with the current state of the code, could you share it with me - I am willing to fix this bug at any price :slight_smile:

Thanks,
Encho