Liquibase changeLogLock

My application using Flowable 6.4.1 crashed, when restarted it hung in a liquibase lock:

2019-06-05 11:29:37,266 INFO  [main] liquibase.logging.core.Slf4jLogger: SELECT COUNT(*) 
FROM public.act_dmn_databasechangeloglock
2019-06-05 11:29:37,268 INFO  [main] liquibase.logging.core.Slf4jLogger: SELECT COUNT(*) FROM public.act_dmn_databasechangeloglock
2019-06-05 11:29:37,269 INFO  [main] liquibase.logging.core.Slf4jLogger: SELECT LOCKED FROM public.act_dmn_databasechangeloglock WHERE ID=1
2019-06-05 11:29:37,270 INFO  [main] liquibase.logging.core.Slf4jLogger: Waiting for changelog lock....

I tried to set the locks to “false” in a spring boot data initializer but that didn’t seem to work. System ops don’t want to take the application in production with this lock issue. How do I set the locks to false at startup of my application? Or should I do that?

Hi Christine.
The locks can be released manually:

UPDATE <tableName> (e.g. DATABASECHANGELOGLOCK) SET LOCKED=FALSE, LOCKGRANTED=null, LOCKEDBY=null where ID=1;

(Clear the locks only if there is no upgrade running). The question is why Flowable crashed.

Regards
Martin

Hey Christine,

I would strongly advise not to do this. The locks are there in order to be able to start multiple instances of Flowable in the same time and to prevent both of them doing the same DB table updates.

Going into such a lock usually means that Flowable crashed during the startup of the application and failed to unlock the tables. This can happen if someone force killed the application during the startup.

Cheers,
Filip

In this case, it happened because the build process in Jenkins failed.

Hi Filip,

In my case, I started flowable-task application twice by mistake and killed the instance during startup, but what should be done now to release the lock?

Akash

Hey Akash,

You need to find out which changelog table is locked. And then do what @martin.grofcik said in Liquibase changeLogLock

The table was act_dmn_databasechangeloglock, name was in the exception and I did what @martin.grofcik suggested but I believe that’s not the only thing I need to do, because I am still getting error like:
The web application [flowable-task] appears to have started a thread named [flowable-ui-task housekeeper] but has failed to stop it. This is very likely to create a memory leak

There are other errors too, but I believe is it possible that due to that lock on changelog table there is a separate thread running, which is just not stopped by manually updating table.

Here is full stack trace:

2019-08-27 17:54:44.069  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-ui-task housekeeper] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
 java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-bpmn-acquire-async-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.AcquireAsyncJobsDueRunnable.sleep(AcquireAsyncJobsDueRunnable.java:138)
 org.flowable.job.service.impl.asyncexecutor.AcquireAsyncJobsDueRunnable.run(AcquireAsyncJobsDueRunnable.java:76)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-bpmn-acquire-timer-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.AcquireTimerJobsRunnable.run(AcquireTimerJobsRunnable.java:99)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-bpmn-reset-expired-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.ResetExpiredJobsRunnable.run(ResetExpiredJobsRunnable.java:70)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-cmmn-acquire-async-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.AcquireAsyncJobsDueRunnable.sleep(AcquireAsyncJobsDueRunnable.java:138)
 org.flowable.job.service.impl.asyncexecutor.AcquireAsyncJobsDueRunnable.run(AcquireAsyncJobsDueRunnable.java:76)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-cmmn-acquire-timer-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.AcquireTimerJobsRunnable.run(AcquireTimerJobsRunnable.java:99)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-cmmn-reset-expired-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)