Flowable 8: Async Executor from one service picking jobs created by another service

Flowable Engine
1

Hi Flowable team,

I am using Flowable 8 with Spring Boot 4 in a microservices architecture.

Setup:

  • Multiple Spring Boot services are using Flowable

  • All services are connected to the same Flowable database

  • Each service deploys and owns its own BPMN processes

  • Each service has its own Spring ApplicationContext and process-specific beans

For example:

  • email-service deploys email-process.bpmn20.xml

  • The process contains expressions like:

${emailCaseProcessService.processAttachment(execution)}

The issue is that the Async Executor of another service is picking up and executing jobs created by the email-service process.

Flow:

  1. email-service deploys a BPMN process and creates an async job

  2. Another service’s Async Executor acquires that job from the shared Flowable database

  3. That service tries to execute the process in its own Spring context

  4. The required bean is not available, causing:

FlowableException:
Unknown property used in expression: emailCaseProcessService

Expected behavior:

  • A service should only execute jobs related to its own deployed processes

  • Jobs created by one service should not be executed by another service’s Async Executor

Questions:

  1. Is there a recommended way in Flowable 8 to isolate Async Executor job acquisition between different Spring Boot services sharing the same database?

  2. What is the recommended Flowable 8 architecture for running BPMN processes across multiple microservices?

Thanks.
@martin.grofcik @joram

2

Hey @Zeeshan1,

Flowable does the synchronisation through the database. If you want to deploy different services with different models then you can’t share the Flowable database.

When you deploy a process then all services that share the same database are going to have access to that process and they would be able to run it.

You could circumvent this by using job categories. However, the fact that all services have access to all deployed processes will still be valid.

Cheers,
Filip

3

@filiphr
Actually i have shfited from camunda 7 where we have this property
camunda.bpm.job-execution.deployment-aware=true , so job executor only picks up deployments that are registered with the process engine.so just wanna know that any similar functioanliy is available in flowable too? or job categories provide similar functioanlity

4

I believe the closest to that would be the job category I mentioned.

However, there is no property for that at the moment. You’ll need to write something on your own.

e.g.

@Bean
public EngineConfigurationConfigurer<SpringProcessEngineConfiguration> customProcessEngineConfigurer() {
    return configuration -> {
        configuration.addEnabledJobCategory("myService");
        configuration.setJobProcessors(List.of(jobProcessorContext -> {
            if (jobProcessorContext.getPhase() == JobProcessorContext.Phase.BEFORE_CREATE) 
                if (jobProcessorContext.getJobEntity() instanceof AbstractRuntimeJobEntity job && job.getCategory() == null) {
                    job.setCategory("myService");
                }
            }
        }));
    };
}

With configuration.addEnabledJobCategory("myService") the Async Executor will only pick up jobs that have that category.

With the other part (the job processors) it will set the category for every async and timer job to the that category.

Hope this helps,
Filip

5

Hey @filiphr thanks for solution , just one more followup question what if i have multiple instance of same application running and i use your solution ,how can isolate execution for this , so whichever instance creates the job that instance job executor pick that job .

6

Why would you want to do this? I would assume that if you deploy your application to multiple nodes it does not matter which node will run the job.

In any case, we have an optimization in our job executor, that would schedule the job on the same instance it got created immediately after the transaction has committed. Only if the task executor queue on the node is full it will not do this so called “kick” and the job will be available for another node to pick it up.

Cheers,
Filip

7

like if i am using any in memory data which is available in that instance in which job was created and that job is picked by other instance job exectuor .then i guess it will cause issue

Also this optimization you mentioned is available bydefault or we have to enable this via any property?

8

This means that if your application crashes the job can never complete since it will not have access to that data. I would not advise that.

This is not something you need to enable or disable. It’s just there, that’s how Flowable’s job executor works.

9

@filiphr thanks for detail discussion