I have a process with few asynchronous service tasks in series in beginning. After getting an exception in a service task followed by retries , it rightly goes to dead letter job table and I am starting the job again using management service after fixing the service task java delegate code. This time the job unexpectedly runs twice and also the next service task does not complete and throws exception as below :
2019-11-26 12:58:46.402 INFO [flowable-server,] 16308 — [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8089 (http) with context path ‘/workflow’
2019-11-26 12:58:46.435 INFO [flowable-server,] 16308 — [ main] c.t.f.FlowableServerApplication : Started FlowableServerApplication in 25.718 seconds (JVM running for 26.892)
2019-11-26 12:58:51.774 INFO [flowable-server,] 16308 — [ main] c.t.f.b.delegate.TransactionDataCheck : TransactionDataCheck:
2019-11-26 12:58:51.773 INFO [flowable-server,e233f5f133f038bf,e233f5f133f038bf,false] 16308 — [ task-1] c.t.f.b.delegate.TransactionDataCheck : TransactionDataCheck:
Hibernate:
insert
into
task_history
(created_at, updated_at, assigned_to, loan_application_id, status, task_id, task_name, type)
values
(?, ?, ?, ?, ?, ?, ?, ?)
Hibernate:
insert
into
task_history
(created_at, updated_at, assigned_to, loan_application_id, status, task_id, task_name, type)
values
(?, ?, ?, ?, ?, ?, ?, ?)
2019-11-26 12:58:52.019 INFO [flowable-server,e233f5f133f038bf,e233f5f133f038bf,false] 16308 — [ task-1] c.t.f.b.network.HttpNetworkClient : http://127.0.0.1:9000/data/ts_summary?loan_application_id=ABL6794739901
2019-11-26 12:58:52.019 INFO [flowable-server,] 16308 — [ main] c.t.f.b.network.HttpNetworkClient : http://127.0.0.1:9000/data/ts_summary?loan_application_id=ABL6794739901
2019-11-26 12:58:52.250 INFO [flowable-server,e233f5f133f038bf,e233f5f133f038bf,false] 16308 — [ task-1] c.t.f.b.network.HttpNetworkClient : {
“number_of_months” : 14
}
2019-11-26 12:58:52.252 INFO [flowable-server,] 16308 — [ main] c.t.f.b.network.HttpNetworkClient : {
“number_of_months” : 14
}
2019-11-26 12:58:53.318 INFO [flowable-server,] 16308 — [ main] c.t.f.b.network.HttpNetworkClient : {
“is_rejected” : false,
“reject_reasons” : [ ]
}
2019-11-26 12:58:53.321 INFO [flowable-server,e233f5f133f038bf,e233f5f133f038bf,false] 16308 — [ task-1] c.t.f.b.network.HttpNetworkClient : {
“is_rejected” : false,
“reject_reasons” : [ ]
}
2019-11-26 12:58:53.384 INFO [flowable-server,] 16308 — [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with ‘debug’ enabled.
2019-11-26 12:58:53.391 INFO [flowable-server,e233f5f133f038bf,36f4ef69cc779db9,false] 16308 — [ task-2] c.t.f.business.delegate.NsdlCheck : NSDLCheck:
2019-11-26 12:58:53.398 INFO [flowable-server,] 16308 — [ main] o.f.j.s.i.a.AbstractAsyncExecutor : Shutting down the async job executor [org.flowable.spring.job.service.SpringAsyncExecutor].
2019-11-26 12:58:53.398 INFO [flowable-server,] 16308 — [uire-timer-jobs] o.f.j.s.i.a.AcquireTimerJobsRunnable : stopped async job due acquisition
2019-11-26 12:58:53.398 INFO [flowable-server,] 16308 — [uire-async-jobs] o.f.j.s.i.a.AcquireAsyncJobsDueRunnable : stopped async job due acquisition
2019-11-26 12:58:53.398 INFO [flowable-server,] 16308 — [et-expired-jobs] o.f.j.s.i.a.ResetExpiredJobsRunnable : stopped resetting expired jobs
2019-11-26 12:58:53.401 INFO [flowable-server,] 16308 — [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService ‘applicationTaskExecutor’
2019-11-26 12:58:53.404 INFO [flowable-server,] 16308 — [ main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit ‘default’
2019-11-26 12:58:53.407 INFO [flowable-server,] 16308 — [ main] com.zaxxer.hikari.HikariDataSource : SpringBootHikariCP - Shutdown initiated…
2019-11-26 12:58:53.424 INFO [flowable-server,] 16308 — [ main] com.zaxxer.hikari.HikariDataSource : SpringBootHikariCP - Shutdown completed.
Exception in thread “task-2” org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is java.lang.IllegalStateException: EntityManagerFactory is closed
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:446)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:378)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:137)
at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:46)
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30)
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56)
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51)
at org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler.handleException(DefaultAsyncRunnableExecutionExceptionHandler.java:25)
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.handleFailedJob(ExecuteAsyncRunnable.java:219)
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.executeJob(ExecuteAsyncRunnable.java:154)
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.run(ExecuteAsyncRunnable.java:116)
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:67)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: EntityManagerFactory is closed
at org.hibernate.internal.SessionFactoryImpl.validateNotClosed(SessionFactoryImpl.java:531)
at org.hibernate.internal.SessionFactoryImpl.createEntityManager(SessionFactoryImpl.java:613)
at org.hibernate.internal.SessionFactoryImpl.createEntityManager(SessionFactoryImpl.java:154)
at org.springframework.orm.jpa.JpaTransactionManager.createEntityManagerForTransaction(JpaTransactionManager.java:464)
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:390)
… 14 more
2019-11-26 12:58:53.443 ERROR [flowable-server,] 16308 — [ main] o.s.boot.SpringApplication : Application run failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:784) ~[spring-boot-2.1.8.RELEASE.jar:2.1.8.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:765) ~[spring-boot-2.1.8.RELEASE.jar:2.1.8.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:319) ~[spring-boot-2.1.8.RELEASE.jar:2.1.8.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) ~[spring-boot-2.1.8.RELEASE.jar:2.1.8.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1204) ~[spring-boot-2.1.8.RELEASE.jar:2.1.8.RELEASE]
at com.tribe3.flowableserver.FlowableServerApplication.main(FlowableServerApplication.java:14) ~[classes/:na]
Caused by: org.flowable.common.engine.api.FlowableOptimisticLockingException: Execution[ id ‘e8130d1d-101d-11ea-8611-02428cd20201’ ] - activity ‘sid-4A469F60-7B72-4656-9E98-46CBC7BFE53A’ - parent ‘e81297e8-101d-11ea-8611-02428cd20201’ was updated by another transaction concurrently
at org.flowable.common.engine.impl.db.DbSqlSession.flushUpdates(DbSqlSession.java:505) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.db.DbSqlSession.flush(DbSqlSession.java:292) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:191) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContext.close(CommandContext.java:61) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:80) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:49) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:46) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.ManagementServiceImpl.executeJob(ManagementServiceImpl.java:95) ~[flowable-engine-6.4.0.jar:6.4.0]
at com.tribe3.flowableserver.config.CommandLineConfig.executeJob(CommandLineConfig.java:88) ~[classes/:na]
at com.tribe3.flowableserver.config.CommandLineConfig.lambda$run$0(CommandLineConfig.java:70) ~[classes/:na]
at java.base/java.util.ArrayList.forEach(ArrayList.java:1540) ~[na:na]
at com.tribe3.flowableserver.config.CommandLineConfig.run(CommandLineConfig.java:68) ~[classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:781) ~[spring-boot-2.1.8.RELEASE.jar:2.1.8.RELEASE]
… 5 common frames omitted
Disconnected from the target VM, address: ‘127.0.0.1:43885’, transport: ‘socket’
Process finished with exit code 1
I am unable to resolve the exceptions . I tried marking the service tasks and exclusive gateways as exclusive but it does not helps.
I am attaching the workflow process diagram.
Please help me in resolving the exceptions. I am stuck here for a long time.
Thanks
Shashank