Hi
I am having a process instance with few sequential service tasks and in case of any technical exception in a service task , the job is getting inserted in act_ru_deadletter_job table as expected. I am picking the deadletter jobs and executing them again on application(spring boot) restart without solving the exception and facing several problems(exceptions) on restart.
Relevant code -
using Commandlinerunner to execute deadletter jobs -
@Component
public class
CommandLineConfig implements CommandLineRunner {
private final UserComponent userComponent;
private final ProcessEngineConfiguration processEngineConfiguration;
private final ManagementService managementService;
@Value("{config.internal.sentry.dsn:}")
private String sentryDsn;
@Value("{spring.profiles.active:dev}")
private String sentryEnv;
@Value("{config.internal.flowable-admin.username}")
private String flowableAdminUserName;
@Value("{config.internal.flowable-admin.password}")
private String flowableAdminPassword;
@Value("{config.internal.flowable.wait-time}")
private Integer flowableWaitTime;
@Value("{config.internal.flowable.max-retries}")
private Integer flowableMaxRetries;
@Autowired
public CommandLineConfig(final UserComponent userComponent, final ProcessEngineConfiguration processEngineConfiguration, final ManagementService managementService) {
this.userComponent = userComponent;
this.processEngineConfiguration = processEngineConfiguration;
this.managementService = managementService;
}
@Override
public void run(final String… args) {
SentryClient sentryClient = io.sentry.Sentry.init(sentryDsn);
sentryClient.setEnvironment(sentryEnv);
managementService.createDeadLetterJobQuery().list().stream().
map(JobInfo::getId).collect(Collectors.toList()).
forEach(input -> {
Job job = moveDeadLetterJobToExecutableJob(input);
managementService.executeJob(job.getId());});
}
@Transactional(propagation = Propagation.REQUIRES_NEW)
private Job moveDeadLetterJobToExecutableJob(String jobId) {
return managementService.moveDeadLetterJobToExecutableJob(jobId,1);
}
Service task throwing nullpointer exception -
@Component
@Slf4j
//@Transactional(rollbackFor = Exception.class, propagation = Propagation.REQUIRES_NEW)
public class TransactionDataCheck implements JavaDelegate, TaskHistoryDelegate {
private final DataAggregatorDataProvider dataAggregatorDataProvider;
private final PolicyDataProvider policyDataProvider;
@Autowired
public TransactionDataCheck(final DataAggregatorDataProvider dataAggregatorDataProvider, final PolicyDataProvider policyDataProvider) {
this.dataAggregatorDataProvider = dataAggregatorDataProvider;
this.policyDataProvider = policyDataProvider;
}
@Override
public void execute(final DelegateExecution execution) {
// try {
log.info(“TransactionDataCheck:”);
String s = null;
s.contains("");
execution.setVariable(WorkflowConfig.LOAN_APPLICATION_STATUS, String.valueOf(WorkflowStatusMaster.TRANSACTION_DATA_CHECK));
saveSystemTaskHistory(execution);
String loanApplicationId = (String)execution.getVariable(WorkflowConfig.BPMN_LOAN_VARIABLE);
TransactionDataResponse transactionDataResponse = dataAggregatorDataProvider.getTransactionData((String) execution.getVariable(WorkflowConfig.BPMN_LOAN_VARIABLE));
LoanRequestPolicyDataResponse loanRequestPolicyDataResponse = (LoanRequestPolicyDataResponse)execution.getVariable(WorkflowConfig.LOAN_REQUEST_POLICY_DATA_RESPONSE);
PolicyMasterDataRequest policyMasterDataRequest = new PolicyMasterDataRequest();
policyMasterDataRequest.setTransactionDataPolicyMasterData(loanRequestPolicyDataResponse, transactionDataResponse);
String formNumber = (String) execution.getVariable(WorkflowConfig.FORM_NUMBER);
policyMasterDataRequest.setFormNumber(formNumber);
PolicyRulesCheckResponse policyRulesCheckResponse = policyDataProvider.getPolicyRulesCheck(policyMasterDataRequest);
if (policyRulesCheckResponse.getIsRejected()) {
rejectLoanApplication(execution, loanApplicationId, policyRulesCheckResponse.getRejectReasons());
}
// } catch (Exception e) {
// execution.setVariable(WorkflowConfig.LOAN_APPLICATION_STATUS, String.valueOf(WorkflowStatusMaster.RETRY));
// Sentry.capture(e);
// }
}
}
In the modeler, I have marked all service tasks and exclusive gateways as asynchronous and exclusive.
Please find attached the image of workflow used from flowable modeler.
The exceptions I am getting are(on application restart) -
2019-11-21 18:08:22.701 INFO [flowable-server,] 15896 — [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8089 (http) with context path ‘/workflow’
2019-11-21 18:08:22.715 INFO [flowable-server,] 15896 — [ main] c.t.f.FlowableServerApplication : Started FlowableServerApplication in 23.68 seconds (JVM running for 24.674)
2019-11-21 18:08:27.262 INFO [flowable-server,] 15896 — [ main] c.t.f.b.delegate.TransactionDataCheck : TransactionDataCheck:
2019-11-21 18:08:29.577 INFO [flowable-server,1fe5c50059fae7b3,1fe5c50059fae7b3,false] 15896 — [ task-1] c.t.f.b.delegate.TransactionDataCheck : TransactionDataCheck:
2019-11-21 18:08:31.396 ERROR [flowable-server,1fe5c50059fae7b3,1fe5c50059fae7b3,false] 15896 — [ task-1] ltAsyncRunnableExecutionExceptionHandler : Job acfa8d48-0c5b-11ea-956e-024207d22949 failed
2019-11-21 18:08:31.480 INFO [flowable-server,] 15896 — [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with ‘debug’ enabled.
2019-11-21 18:08:31.492 INFO [flowable-server,] 15896 — [ main] o.f.j.s.i.a.AbstractAsyncExecutor : Shutting down the async job executor [org.flowable.spring.job.service.SpringAsyncExecutor].
2019-11-21 18:08:31.492 INFO [flowable-server,] 15896 — [uire-timer-jobs] o.f.j.s.i.a.AcquireTimerJobsRunnable : stopped async job due acquisition
2019-11-21 18:08:31.492 INFO [flowable-server,] 15896 — [et-expired-jobs] o.f.j.s.i.a.ResetExpiredJobsRunnable : stopped resetting expired jobs
2019-11-21 18:08:31.492 INFO [flowable-server,] 15896 — [uire-async-jobs] o.f.j.s.i.a.AcquireAsyncJobsDueRunnable : stopped async job due acquisition
2019-11-21 18:08:31.494 INFO [flowable-server,] 15896 — [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService ‘applicationTaskExecutor’
2019-11-21 18:08:31.495 INFO [flowable-server,] 15896 — [ main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit ‘default’
2019-11-21 18:08:31.496 INFO [flowable-server,] 15896 — [ main] com.zaxxer.hikari.HikariDataSource : SpringBootHikariCP - Shutdown initiated…
2019-11-21 18:08:31.516 INFO [flowable-server,] 15896 — [ main] com.zaxxer.hikari.HikariDataSource : SpringBootHikariCP - Shutdown completed.
2019-11-21 18:08:31.516 WARN [flowable-server,1fe5c50059fae7b3,1fe5c50059fae7b3,false] 15896 — [ task-1] com.zaxxer.hikari.pool.ProxyConnection : SpringBootHikariCP - Connection org.postgresql.jdbc.PgConnection@551a1ecc marked as broken because of SQLSTATE(08006), ErrorCode(0)
org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:336) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:149) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:138) ~[postgresql-42.2.6.jar:42.2.6]
at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44) ~[HikariCP-3.1.0.jar:na]
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java) ~[HikariCP-3.1.0.jar:na]
at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:46) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:198) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:185) ~[mybatis-3.4.6.jar:3.4.6]
at org.flowable.common.engine.impl.db.DbSqlSession.flushRegularInsert(DbSqlSession.java:442) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.db.DbSqlSession.flushInsertEntities(DbSqlSession.java:423) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.db.DbSqlSession.flushInserts(DbSqlSession.java:406) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.db.DbSqlSession.flush(DbSqlSession.java:291) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:191) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContext.close(CommandContext.java:61) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:80) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:49) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:46) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler$1.execute(DefaultAsyncRunnableExecutionExceptionHandler.java:48) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler$1.execute(DefaultAsyncRunnableExecutionExceptionHandler.java:25) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.CommandInvoker$1.run(CommandInvoker.java:51) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperation(CommandInvoker.java:93) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperations(CommandInvoker.java:72) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.CommandInvoker.execute(CommandInvoker.java:56) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.BpmnOverrideContextInterceptor.execute(BpmnOverrideContextInterceptor.java:25) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.TransactionContextInterceptor.execute(TransactionContextInterceptor.java:53) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:71) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:49) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:46) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler.handleException(DefaultAsyncRunnableExecutionExceptionHandler.java:25) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.handleFailedJob(ExecuteAsyncRunnable.java:219) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.executeJob(ExecuteAsyncRunnable.java:154) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.run(ExecuteAsyncRunnable.java:116) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:67) ~[spring-cloud-sleuth-core-2.1.3.RELEASE.jar:2.1.3.RELEASE]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: javax.net.ssl.SSLException: Socket closed
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127) ~[na:na]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321) ~[na:na]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264) ~[na:na]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:259) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1314) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:839) ~[na:na]
at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.PGStream.receiveChar(PGStream.java:321) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1978) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309) ~[postgresql-42.2.6.jar:42.2.6]
… 48 common frames omitted
Caused by: java.net.SocketException: Socket closed
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:183) ~[na:na]
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140) ~[na:na]
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:448) ~[na:na]
at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1104) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:823) ~[na:na]
… 54 common frames omitted
2019-11-21 18:08:31.518 ERROR [flowable-server,1fe5c50059fae7b3,1fe5c50059fae7b3,false] 15896 — [ task-1] o.f.c.e.impl.interceptor.CommandContext : Error while closing command context
org.apache.ibatis.exceptions.PersistenceException:
Error updating database. Cause: org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
The error may involve org.flowable.job.service.impl.persistence.entity.DeadLetterJobEntityImpl.insertDeadLetterJob-Inline
The error occurred while setting parameters
SQL: insert into ACT_RU_DEADLETTER_JOB ( ID_, REV_, TYPE_, EXCLUSIVE_, EXECUTION_ID_, PROCESS_INSTANCE_ID_, PROC_DEF_ID_, SCOPE_ID_, SUB_SCOPE_ID_, SCOPE_TYPE_, SCOPE_DEFINITION_ID_, EXCEPTION_STACK_ID_, EXCEPTION_MSG_, DUEDATE_, REPEAT_, HANDLER_TYPE_, HANDLER_CFG_, CUSTOM_VALUES_ID_, CREATE_TIME_, TENANT_ID_) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )
Caused by: javax.net.ssl.SSLException: Socket closed
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127) ~[na:na]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321) ~[na:na]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264) ~[na:na]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:259) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1314) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:839) ~[na:na]
at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.PGStream.receiveChar(PGStream.java:321) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1978) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309) ~[postgresql-42.2.6.jar:42.2.6]
… 48 common frames omitted
Caused by: java.net.SocketException: Socket closed
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:183) ~[na:na]
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140) ~[na:na]
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:448) ~[na:na]
at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1104) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:823) ~[na:na]
… 54 common frames omitted
2019-11-21 18:08:31.520 ERROR [flowable-server,1fe5c50059fae7b3,1fe5c50059fae7b3,false] 15896 — [ task-1] o.s.t.support.TransactionTemplate : Application exception overridden by rollback exception
2019-11-21 18:08:31.522 ERROR [flowable-server,1fe5c50059fae7b3,1fe5c50059fae7b3,false] 15896 — [ task-1] o.f.c.e.impl.interceptor.CommandContext : Error while closing command context
org.springframework.transaction.TransactionSystemException: Could not roll back JPA transaction; nested exception is org.hibernate.TransactionException: Unable to rollback against JDBC Connection
at org.springframework.orm.jpa.JpaTransactionManager.doRollback(JpaTransactionManager.java:563) ~[spring-orm-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:838) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:812) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.rollbackOnException(TransactionTemplate.java:168) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:144) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:46) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler$1.execute(DefaultAsyncRunnableExecutionExceptionHandler.java:48) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler$1.execute(DefaultAsyncRunnableExecutionExceptionHandler.java:25) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.CommandInvoker$1.run(CommandInvoker.java:51) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperation(CommandInvoker.java:93) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperations(CommandInvoker.java:72) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.CommandInvoker.execute(CommandInvoker.java:56) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.engine.impl.interceptor.BpmnOverrideContextInterceptor.execute(BpmnOverrideContextInterceptor.java:25) ~[flowable-engine-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.TransactionContextInterceptor.execute(TransactionContextInterceptor.java:53) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:71) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:49) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:46) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler.handleException(DefaultAsyncRunnableExecutionExceptionHandler.java:25) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.handleFailedJob(ExecuteAsyncRunnable.java:219) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.executeJob(ExecuteAsyncRunnable.java:154) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.run(ExecuteAsyncRunnable.java:116) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:67) ~[spring-cloud-sleuth-core-2.1.3.RELEASE.jar:2.1.3.RELEASE]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: org.hibernate.TransactionException: Unable to rollback against JDBC Connection
at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.rollback(AbstractLogicalConnectionImplementor.java:122) ~[hibernate-core-5.3.11.Final.jar:5.3.11.Final]
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.rollback(JdbcResourceLocalTransactionCoordinatorImpl.java:294) ~[hibernate-core-5.3.11.Final.jar:5.3.11.Final]
at org.hibernate.engine.transaction.internal.TransactionImpl.rollback(TransactionImpl.java:145) ~[hibernate-core-5.3.11.Final.jar:5.3.11.Final]
at org.springframework.orm.jpa.JpaTransactionManager.doRollback(JpaTransactionManager.java:559) ~[spring-orm-5.1.9.RELEASE.jar:5.1.9.RELEASE]
… 30 common frames omitted
Caused by: java.sql.SQLException: Connection is closed
at com.zaxxer.hikari.pool.ProxyConnection$ClosedConnection.lambda$getClosedConnection$0(ProxyConnection.java:490) ~[HikariCP-3.1.0.jar:na]
at com.sun.proxy.$Proxy118.rollback(Unknown Source) ~[na:na]
at com.zaxxer.hikari.pool.ProxyConnection.rollback(ProxyConnection.java:377) ~[HikariCP-3.1.0.jar:na]
at com.zaxxer.hikari.pool.HikariProxyConnection.rollback(HikariProxyConnection.java) ~[HikariCP-3.1.0.jar:na]
at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.rollback(AbstractLogicalConnectionImplementor.java:116) ~[hibernate-core-5.3.11.Final.jar:5.3.11.Final]
… 33 common frames omitted
2019-11-21 18:08:31.523 WARN [flowable-server,1fe5c50059fae7b3,1fe5c50059fae7b3,false] 15896 — [ task-1] com.zaxxer.hikari.pool.ProxyConnection : SpringBootHikariCP - Connection org.postgresql.jdbc.PgConnection@7db41493 marked as broken because of SQLSTATE(08003), ErrorCode(0)
org.postgresql.util.PSQLException: This connection has been closed.
at org.postgresql.jdbc.PgConnection.checkClosed(PgConnection.java:782) ~[postgresql-42.2.6.jar:42.2.6]
at org.postgresql.jdbc.PgConnection.rollback(PgConnection.java:789) ~[postgresql-42.2.6.jar:42.2.6]
at com.zaxxer.hikari.pool.ProxyConnection.rollback(ProxyConnection.java:377) ~[HikariCP-3.1.0.jar:na]
at com.zaxxer.hikari.pool.HikariProxyConnection.rollback(HikariProxyConnection.java) ~[HikariCP-3.1.0.jar:na]
at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.rollback(AbstractLogicalConnectionImplementor.java:116) ~[hibernate-core-5.3.11.Final.jar:5.3.11.Final]
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.rollback(JdbcResourceLocalTransactionCoordinatorImpl.java:294) ~[hibernate-core-5.3.11.Final.jar:5.3.11.Final]
at org.hibernate.engine.transaction.internal.TransactionImpl.rollback(TransactionImpl.java:145) ~[hibernate-core-5.3.11.Final.jar:5.3.11.Final]
at org.springframework.orm.jpa.JpaTransactionManager.doRollback(JpaTransactionManager.java:559) ~[spring-orm-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:838) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:812) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.rollbackOnException(TransactionTemplate.java:168) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:144) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:46) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.DefaultAsyncRunnableExecutionExceptionHandler.handleException(DefaultAsyncRunnableExecutionExceptionHandler.java:25) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.handleFailedJob(ExecuteAsyncRunnable.java:219) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.executeJob(ExecuteAsyncRunnable.java:154) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.run(ExecuteAsyncRunnable.java:116) ~[flowable-job-service-6.4.0.jar:6.4.0]
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:67) ~[spring-cloud-sleuth-core-2.1.3.RELEASE.jar:2.1.3.RELEASE]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Disconnected from the target VM, address: ‘127.0.0.1:42573’, transport: ‘socket’
Process finished with exit code 1
I am stuck in solving the exceptions I am getting again and again and need help in resolving them.
Solutions tried -
- used new transaction for deleting deadletter jobs and adding it to act_ru_job table
@Transactional(propagation = Propagation.REQUIRES_NEW)
private Job moveDeadLetterJobToExecutableJob(String jobId) {
return managementService.moveDeadLetterJobToExecutableJob(jobId,1);
}
-changed datasource config in application.yml to resolve ‘database connection has been closed’ error
datasource:
testOnBorrow: true
validationQuery: SELECT 1
-made all service tasks and exclusive gateways as asynchronous and exclusive to resolve optimistic locking exception
But none of above has worked. Please guide me through possible solutions that can be tried to resolve the exceptions mentioned.
Thanks & Regards
Shashank