Liquibase changeLogLock

Flowable Engine
1

My application using Flowable 6.4.1 crashed, when restarted it hung in a liquibase lock:

2019-06-05 11:29:37,266 INFO  [main] liquibase.logging.core.Slf4jLogger: SELECT COUNT(*) 
FROM public.act_dmn_databasechangeloglock
2019-06-05 11:29:37,268 INFO  [main] liquibase.logging.core.Slf4jLogger: SELECT COUNT(*) FROM public.act_dmn_databasechangeloglock
2019-06-05 11:29:37,269 INFO  [main] liquibase.logging.core.Slf4jLogger: SELECT LOCKED FROM public.act_dmn_databasechangeloglock WHERE ID=1
2019-06-05 11:29:37,270 INFO  [main] liquibase.logging.core.Slf4jLogger: Waiting for changelog lock....

I tried to set the locks to “false” in a spring boot data initializer but that didn’t seem to work. System ops don’t want to take the application in production with this lock issue. How do I set the locks to false at startup of my application? Or should I do that?

2

Hi Christine.
The locks can be released manually:

UPDATE <tableName> (e.g. DATABASECHANGELOGLOCK) SET LOCKED=FALSE, LOCKGRANTED=null, LOCKEDBY=null where ID=1;

(Clear the locks only if there is no upgrade running). The question is why Flowable crashed.

Regards
Martin

3 Likes
3

Hey Christine,

I would strongly advise not to do this. The locks are there in order to be able to start multiple instances of Flowable in the same time and to prevent both of them doing the same DB table updates.

Going into such a lock usually means that Flowable crashed during the startup of the application and failed to unlock the tables. This can happen if someone force killed the application during the startup.

Cheers,
Filip

4

In this case, it happened because the build process in Jenkins failed.

5

Hi Filip,

In my case, I started flowable-task application twice by mistake and killed the instance during startup, but what should be done now to release the lock?

Akash

6

Hey Akash,

You need to find out which changelog table is locked. And then do what @martin.grofcik said in Liquibase changeLogLock

7

The table was act_dmn_databasechangeloglock, name was in the exception and I did what @martin.grofcik suggested but I believe that’s not the only thing I need to do, because I am still getting error like:
The web application [flowable-task] appears to have started a thread named [flowable-ui-task housekeeper] but has failed to stop it. This is very likely to create a memory leak

There are other errors too, but I believe is it possible that due to that lock on changelog table there is a separate thread running, which is just not stopped by manually updating table.

Here is full stack trace:

2019-08-27 17:54:44.069  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-ui-task housekeeper] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
 java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-bpmn-acquire-async-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.AcquireAsyncJobsDueRunnable.sleep(AcquireAsyncJobsDueRunnable.java:138)
 org.flowable.job.service.impl.asyncexecutor.AcquireAsyncJobsDueRunnable.run(AcquireAsyncJobsDueRunnable.java:76)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-bpmn-acquire-timer-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.AcquireTimerJobsRunnable.run(AcquireTimerJobsRunnable.java:99)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-bpmn-reset-expired-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.ResetExpiredJobsRunnable.run(ResetExpiredJobsRunnable.java:70)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-cmmn-acquire-async-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.AcquireAsyncJobsDueRunnable.sleep(AcquireAsyncJobsDueRunnable.java:138)
 org.flowable.job.service.impl.asyncexecutor.AcquireAsyncJobsDueRunnable.run(AcquireAsyncJobsDueRunnable.java:76)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-cmmn-acquire-timer-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.flowable.job.service.impl.asyncexecutor.AcquireTimerJobsRunnable.run(AcquireTimerJobsRunnable.java:99)
 java.lang.Thread.run(Thread.java:745)
2019-08-27 17:54:44.073  WARN 2344 --- [  restartedMain] o.a.c.l.WebappClassLoaderBase            : The web application [flowable-task] appears to have started a thread named [flowable-cmmn-reset-expired-jobs] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
8

Hello, I am running into this issue and I don’t know where to apply the fix. I’ve read through the existing documentation as well as the replies that are already here. Where would I need to go to make these small change to the DB locally? This error is making it so that I cannot use the web modeler locally and work on any of my existing models.

9

You would need to connect to your database with a SQL tool (which depends on the database you’re using) and execute the SQL there.