Cannot configure Async Job Executor Thread Pool parameters

Flowable Engine
1

Hi all,
I’m using Flowable 6.2.1 with flowable-task installed as a web application and using PostgreSQL as the database. I have a very simple test process definition containing only one service task (which is defined as async). My custom java code (JavaDelegate implementation) is working properly. When I start 1000 process instances using the API, they all get process ınstance ids and async job executor starts processing them. However, it starts with only 2 threads (as expected by the default values) and does not increase the thread count although there are about 1000 async jobs waiting in ACT_RU_JOB table (some has locks and expire times and some or not). As I see in the documents, the default max thread pool size is 10 but I never see third thread starting, only two are working till all process instances are complete.

I tried changing pool parameters by adding these to the “flowable-ui-app.properties”:

asyncExecutorCorePoolSize=10
asyncExecutorMaxPoolSize=100
asyncExecutorThreadPoolQueueSize=500

But nothing has changed, only two threads are working.

Please tell me what I’m doing wrong and how to configure async job executor thread pool parameters for the flowable-task application properly.

2

Hi @eesengin

I know this topic is quite old, but I came upon the same issue with Spring Boot.
According to docs - properties you mentioned (asyncExecutorCorePoolSize, …) should be available to override via setters on ProcessEngineConfigurationImpl.

It looks like that there is an issue when you start Flowable with default Spring configuration:

  • SpringAsyncExecutor#initAsyncJobExecutionThreadPool method has an empty body, so it’s Spring task executor used
  • FlowableJobConfiguration provides TaskExecutor with fixed core pool size. Currently, this property is set to 2.

There is a pretty easy workaround. Just provide custom task executor bean in your app, i.e.:

@Bean
@ConditionalOnMissingBean
public TaskExecutor taskExecutor() {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(10);
    executor.setMaxPoolSize(20);
    executor.setQueueCapacity(100);
    executor.setThreadNamePrefix("my-task-Executor");
    executor.setAwaitTerminationSeconds(30);
    executor.setWaitForTasksToCompleteOnShutdown(true);
    executor.initialize();
    return executor;
}

or provide such properties:

flowable.process.async.executor.task-executor.core-pool-size
flowable.process.async.executor.task-executor.max-pool-size
flowable.process.async.executor.task-executor.queue-capacity

It looks quite tricky but works, and there is a small inconsistency between code and docs. Maybe there is missing info in docs that those properties work with standalone process engine and with Spring you should use those I’ve pasted above.

@tijs @filiphr Can you confirm or deny?

1 Like
3

Get @rgorzkowski,

The link you referenced is correct when you are not using Spring Boot (maybe we need to stress that out in that section of the documentation).

Looking more into the code and the auto configuration. Seems like the properties that you mentioned would have no effect as the SpringAsyncExecutor does not use them, but uses the TaskExecutor that is provided by the FlowableJobConfiguration.

The way to change the Spring Task Executor is to provide your own and the one from Flowable won’t kick in. The reason why we did it like this is that we would have needed to do a lot of customizations, and we knew that Spring Boot 2.1 will have an auto configuration for the task executor, so we did a temporary solution like it is now.

Did you try using the properties you mentioned? Did it work?

Cheers,
Filip

4

Hi @filiphr,

Thank you for your quick response.
Regarding those properties, I’ve used them and it works (I see in logs that there are more threads). I think it works because there is

@ConfigurationProperties(prefix = “flowable.process.async.executor”) on SpringAsyncExecutor bean.

5

Hi @rgorzkowski @filiphr -
I am working with flowable 6.4.0 and need to change the property asyncExecutorMaxPoolSize. The application is running in tomcat 8.5 container. I’ve added application.properties file to tomcat’s lib directory with the following entry:

flowable.process.async.executor.asyncExecutorMaxPoolSize=20
or
flowable.process.async.executor.max-pool-size=20

A few questions:

  1. Is application.properties file the right file ?
  2. Which one of the above property is correct ? If neither, what is the right key ?
  3. Is there a way to confirm that the property has been applied ?
  4. Is there any log entry which will confirm the above ?
  5. Is there any documentation for all all the spring property configurations keys ?

Thanks

6

You can set it via asyncExecutorCorePoolSize on the engine configuration. Not sure if there’s a direct Spring Boot setting.

In the current version (haven’t checked 6.4.0), the logs will show ‘creating executor service with corePoolSize x’

7

Hi @joram -

Thanks for your response.

However, the documentation (https://flowable.com/open-source/docs/bpmn/ch18-Advanced/) indicates that corePoolSize and maxPoolSize are different and I want to set the latter.

Secondly, I am using the war files built with spring boot so, I am trying to do it the spring way using application.properties file.

Please let me know if there is any pointer/advice.

Thanks

8

Hey @mans2singh,

The configuration of the thread pools when using the OSS starters needs to be done with the spring.task.execution properties.

So to increase the max pool size you will need to set spring.task.execution.pool.max-size.

I would also reading this before setting the properties. The crucial part is:

If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full.

So the max pool size will only be reached when the queue is full. And the default for Spring Boot 2.3 is unbounded, which means that the max-size property is ignored.

Cheers,
Filip

9

Thanks @filiphr - will try that.

10

I have a spring boot application using Flowable. Overriding taskExecutor works (I do not know why) in 6.4 but in 6.5 the app fails to start saying:

APPLICATION FAILED TO STARTDescription:The bean ‘taskExecutor’, defined in class path resource [org/flowable/spring/boot/FlowableJobConfiguration.class], could not be registered. A bean with that name has already been defined in class path resource [SOME PATH TO RESOURCE IN MY PROJECT] and overriding is disabled.Action:Consider renaming one of the beans or enabling overriding by setting spring.main.allow-bean-definition-overriding=true

After looking some forums I learnt that @ConditionalOnMissingBean annotation used in org.flowable.spring.boot.FlowableJobConfiguration is probably ignored by spring boot because it only works when the class is declared as auto-configuration in spring.factories which is not the case.
I finished by setting spring.main.allow-bean-definition-overriding to true to be able to upgrade and as of now have not yet come up with a better solution.