Nullpointer exception at DefaultInternalJobManager.handleJobDelete

Flowable Engine
1

Hi Team,

We are getting null pointer at handleJobDelete which is flowable internal method. We are using flowable from almost 2 years but we never faced this kind of issue.

at java.lang.Thread.run(Thread.java:748)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.run(ExecuteAsyncRunnable.java:113)

at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.lockJob(ExecuteAsyncRunnable.java:194)

at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.unacquireJob(ExecuteAsyncRunnable.java:207)

at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51)

at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56)

at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30)

at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:46)

at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)

at org.flowable.common.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:49)

at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:71)

at org.flowable.common.engine.impl.interceptor.TransactionContextInterceptor.execute(TransactionContextInterceptor.java:53)

at org.flowable.engine.impl.interceptor.BpmnOverrideContextInterceptor.execute(BpmnOverrideContextInterceptor.java:25)

at org.flowable.engine.impl.interceptor.CommandInvoker.execute(CommandInvoker.java:56)

at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperations(CommandInvoker.java:72)

at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperation(CommandInvoker.java:93)

at org.flowable.engine.impl.interceptor.CommandInvoker$1.run(CommandInvoker.java:51)

at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable$3.execute(ExecuteAsyncRunnable.java:207)

at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable$3.execute(ExecuteAsyncRunnable.java:210)

at org.flowable.job.service.impl.asyncexecutor.DefaultJobManager.unacquire(DefaultJobManager.java:267)

at org.flowable.job.service.impl.persistence.entity.AbstractEntityManager.delete(AbstractEntityManager.java:83)

at org.flowable.job.service.impl.persistence.entity.JobEntityManagerImpl.delete(JobEntityManagerImpl.java:32)

at org.flowable.job.service.impl.persistence.entity.JobEntityManagerImpl.delete(JobEntityManagerImpl.java:79)

at org.flowable.job.service.impl.persistence.entity.AbstractEntityManager.delete(AbstractEntityManager.java:88)

at org.flowable.job.service.impl.persistence.entity.JobEntityManagerImpl.delete(JobEntityManagerImpl.java:32)

at org.flowable.job.service.impl.persistence.entity.JobEntityManagerImpl.delete(JobEntityManagerImpl.java:93)

at org.flowable.engine.impl.cfg.DefaultInternalJobManager.handleJobDelete(DefaultInternalJobManager.java:119)

java.lang.NullPointerException: null

Could you please help us

Thanks & Regards,
Kalyan Kumar

2

Which version of Flowable are you using?

3

@joram Same exception observed:

Exception in thread "flowable-executor2" java.lang.NullPointerException
	at org.flowable.engine.impl.cfg.DefaultInternalJobManager.handleJobDeleteInternal(DefaultInternalJobManager.java:124)
	at org.flowable.job.service.ScopeAwareInternalJobManager.handleJobDelete(ScopeAwareInternalJobManager.java:67)
	at org.flowable.job.service.impl.persistence.entity.JobEntityManagerImpl.delete(JobEntityManagerImpl.java:98)
	at org.flowable.job.service.impl.persistence.entity.JobEntityManagerImpl.delete(JobEntityManagerImpl.java:82)
	at org.flowable.job.service.impl.persistence.entity.JobEntityManagerImpl.delete(JobEntityManagerImpl.java:31)
	at org.flowable.common.engine.impl.persistence.entity.AbstractEntityManager.delete(AbstractEntityManager.java:95)
	at org.flowable.job.service.impl.asyncexecutor.DefaultJobManager.unacquire(DefaultJobManager.java:380)
	at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable$2.execute(ExecuteAsyncRunnable.java:202)
	at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable$2.execute(ExecuteAsyncRunnable.java:199)
	at org.flowable.engine.impl.interceptor.CommandInvoker$1.run(CommandInvoker.java:67)
	at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperation(CommandInvoker.java:140)
	at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperations(CommandInvoker.java:114)
	at org.flowable.engine.impl.interceptor.CommandInvoker.execute(CommandInvoker.java:72)
	at org.flowable.engine.impl.interceptor.BpmnOverrideContextInterceptor.execute(BpmnOverrideContextInterceptor.java:26)
	at org.flowable.common.engine.impl.interceptor.TransactionContextInterceptor.execute(TransactionContextInterceptor.java:53)
	at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:105)
	at org.flowable.common.spring.SpringTransactionInterceptor.lambda$execute$0(SpringTransactionInterceptor.java:57)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
	at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:57)
	at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30)
	at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56)
	at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51)
	at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.unacquireJob(ExecuteAsyncRunnable.java:199)
	at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.lockJob(ExecuteAsyncRunnable.java:186)
	at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.run(ExecuteAsyncRunnable.java:112)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

Flowable engine version 6.7.2
Shallow job executor with queue of size 1
Database - Postgres 13

4

Looking at the source it seems the job being passed in is probably null.
What does your application architecture look like? How many engines, job schedule configuration etc.
Wondering if there was some sort of race condition where the same job was picked up by two schedulers.

5

@gdharley
BPMN engine only, configuration:

flowable:
  process-definition-location-prefix: classpath*:/flowable/**/
  enable-history-cleaning: true
  history-cleaning-after: 1d
  process:
    async:
      executor:
        async-job-lock-time: PT10M
        max-async-jobs-due-per-acquisition: 1

flowable-executor:
  pool:
    core-pool-size: 1
    max-pool-size: 1
    queue-capacity: 1

Multi-engine deployment across multiple worker instances (3 instances). Long-lived transactions (~2 hours) are also present in ServiceTask

6

Same here I could repeat this error on one machine with following config

cfg.setAsyncExecutorActivate(true);
cfg.setAsyncExecutorMaxPoolSize(5);
cfg.setAsyncExecutorCorePoolSize(5);

asyncExecutorConfiguration.setDefaultAsyncJobAcquireWaitTime(Duration.ofMillis(5000));
asyncExecutorConfiguration.setMaxTimerJobsPerAcquisition(20); 
asyncExecutorConfiguration.setMaxTimerJobsPerAcquisition(20)

If you give me some snippet(github) I could write test case

7

I assume the nullpointer happens when running with more than one async executor instance? And that due to the 5sec lock time, the other async executor assumes the first node has failed and tries to take over (and then gets the NPE)?