Hi,
After a migration of running processes to a new version (with minor changes: added a path to an existing exclusive gateway), I started seeing deadletter jobs on a timer job that failed. The error is:
org.apache.ibatis.exceptions.PersistenceException:
### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (`flowable`.`ACT_RU_EVENT_SUBSCR`, CONSTRAINT `ACT_FK_EVENT_EXEC` FOREIGN KEY (`EXECUTION_ID_`) REFERENCES `ACT_RU_EXECUTION` (`ID_`))
### The error may involve org.flowable.engine.impl.persistence.entity.ExecutionEntityImpl.deleteExecution-Inline
### The error occurred while setting parameters
### SQL: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ?
### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (`flowable`.`ACT_RU_EVENT_SUBSCR`, CONSTRAINT `ACT_FK_EVENT_EXEC` FOREIGN KEY (`EXECUTION_ID_`) REFERENCES `ACT_RU_EXECUTION` (`ID_`))
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:200) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:213) ~[mybatis-3.4.6.jar:3.4.6]
at org.flowable.common.engine.impl.db.DbSqlSession.flushDeleteEntities(DbSqlSession.java:569) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.db.DbSqlSession.flushDeletes(DbSqlSession.java:526) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.db.DbSqlSession.flush(DbSqlSession.java:293) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:191) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContext.close(CommandContext.java:61) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:80) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:49) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.0.9.RELEASE.jar:5.0.9.RELEASE]
at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:46) ~[flowable-spring-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51) ~[flowable-engine-common-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.executeJob(ExecuteAsyncRunnable.java:128) [flowable-job-service-6.4.0.jar:6.4.0]
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.run(ExecuteAsyncRunnable.java:116) [flowable-job-service-6.4.0.jar:6.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_181]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_181]
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (`flowable`.`ACT_RU_EVENT_SUBSCR`, CONSTRAINT `ACT_FK_EVENT_EXEC` FOREIGN KEY (`EXECUTION_ID_`) REFERENCES `ACT_RU_EXECUTION` (`ID_`))
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_181]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) ~[?:1.8.0_181]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) ~[?:1.8.0_181]
at java.lang.reflect.Constructor.newInstance(Unknown Source) ~[?:1.8.0_181]
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.mysql.jdbc.Util.getInstance(Util.java:408) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:935) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3973) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3909) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2527) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2680) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2490) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1858) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1197) ~[mysql-connector-java-5.1.42-bin.jar:5.1.42]
at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44) ~[HikariCP-2.7.9.jar:?]
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java) ~[HikariCP-2.7.9.jar:?]
at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:46) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76) ~[mybatis-3.4.6.jar:3.4.6]
at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:198) ~[mybatis-3.4.6.jar:3.4.6]
... 18 more
Trying to delete the process instance gives a similar error message, as well as migrating it to the previous version or to a newer version.
2019-02-19 11:41:28.825 WARN 28871 --- [o-8009-exec-166] .m.m.a.ExceptionHandlerExceptionResolver : Resolved [org.flowable.ui.common.service.exception.BadRequestException:
### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (`flowable`.`ACT_RU_EVENT_SUBSCR`, CONSTRAINT `ACT_FK_EVENT_EXEC` FOREIGN KEY (`EXECUTION_ID_`) REFERENCES `ACT_RU_EXECUTION` (`ID_`))
### The error may involve org.flowable.engine.impl.persistence.entity.ExecutionEntityImpl.deleteExecution-Inline
### The error occurred while setting parameters
### SQL: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ?
### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (`flowable`.`ACT_RU_EVENT_SUBSCR`, CONSTRAINT `ACT_FK_EVENT_EXEC` FOREIGN KEY (`EXECUTION_ID_`) REFERENCES `ACT_RU_EXECUTION` (`ID_`))]~
HI @tijs, I migrated with some script like below that I ran via the debug mode. The processes migrated correctly (in that no error has been generated, and the version has been updated).
migrateToDef = "processdef:27:8c354f2f-3033-11e9-8020-005056904adb";
processDefName= "processdef"
processInstances = runtimeService.createProcessInstanceQuery().processDefinitionName(processDefName).list();
for (var i=0; i<processInstances.size(); i++)
{
print("checking process to migrate: "+processInstances[i].getId());
validationResult = runtimeService.createProcessInstanceMigrationBuilder()
.migrateToProcessDefinition(migrateToDef)
.validateMigration(processInstances[i].getId());
print(validationResult.isMigrationValid());
runtimeService.createProcessInstanceMigrationBuilder()
.migrateToProcessDefinition(migrateToDef)
.migrate(processInstances[i].getId());
}
Hi @tijs,
Is there a (safe) procedure to remove the failing processes in the database? I can recreate them afterwards, so that should solve (work-around) my problem.
Tnx,
Wim
Looks like it is not caused by the migration, but by something else. Still need to pinpoint what exactly.
I have started some new instances of the process, and they ended up having the same issue of deadletter jobs.
I managed to make a simple process that has the same behavior. It’s a reduced process that I have been using. Basically, it uses signals to trigger a sub-process that is being called on several occasions. This subprocess runs a http-task.
The problem can be simulated by moving the timer job 2 times. First time, it goes OK. 2nd time we move the timer job, it goes wrong. Same behavior if we just let the timer do its job: 1st round OK, 2nd round NOK.
The simple solution is to make the http-task asynchronous. But it should not lead to a situation where the process is “dead”: job can no longer be executed, process can’t be terminated, …
