Unknown cpu spike every 10 seconds

Hi all,

I have a pretty fresh install, and doing testing. I’m seeing flowable create a cpu spike to 80-100% every 10 seconds, lasting for 1-2 seconds. Every 4-5 hours, this crashes my server.

I have zero bpmn active instances, tasks, jobs or event subscriptions. There are some bpmn deployments. There is nothing in the form engine, dmn engine, or content engine area inside the admin UI.

I looked in the docs and didn’t see any cron activities. What could this be? How can I scale down the cpu usage or disable whatever is causing this?

Thanks!

Could be the Job Executor (no idea why the load is so high). Did you enable it? I think it runs every 10 seconds by default. If you do not need it (no async actvities, timer jobs etc) you can deactive it (see: http://www.flowable.org/docs/userguide/index.html#jobExecutorConfiguration).

Thanks, so I disabled the asyncexecutor and it did decrease the cpu, but only by about 10%. There is still something running every 10 seconds and it hits 75-80% cpu each time, causing the server overall to hit 90-100%. Are there any other cron jobs that could be responsible for this?

I’m not aware of other async jobs, but my knowledge of cmmn/dmn is limited. How many flowable-engine instances are running? Just the one embedded in the flowable-task app? Did you disable the asyn executor of ever instance?

If you mean how many instances of flowable on the server, there is one, it’s running on tomcat inside a docker container. There are zero process instances running, from what I see in the admin app.

For reference, I have disabled the asyncexecutor by changing the asyncExecutorActivate setting in the /usr/local/tomcat/lib/flowable.cfg.xml file to “false.”

Hi Rob,

Did you try to use profiler to find out what is CPU doing each 10sec? May be it could help to find out the cause.

Regards
Martin

Hi Martin,
I’ve been trying to setup a profiler that would work on command line, and I haven’t been able to get one working (I’ve tried yourkit, jvmtop and jvisualvm). Could you point me in the right directon of one that’s quick to setup and use? I’m using the tomcat docker container, which has an alpine linux base (so it has apk, not apt).

Ok so I was able to get jvmtop working, and this is the thread that is causing the spike:

ContainerBackgroundProcessor[S

Any idea what that thread is doing?

Hi,

ContainerBackgroundProcessor is not something from Flowable, so difficult to say.
What are you running exactly? Are you running all Flowable apps in this Docker instance? Or is it only the Flowable Task application?

Best regards,

Tijs

Hi Tijs,
I’m running all flowable apps in this docker instance. It’s using the tomcat 8.5-alpine image for the base, with the .war files copied over.

Not that I think this is a cause of your CPU spike, but be very careful of your memory settings when running any Java app inside Docker.
My go to blog post for tuning docker and JVm is here:
https://developers.redhat.com/blog/2017/03/14/java-inside-docker/
Like I said, I doubt this is the cause of the CPU spike, but let’s make sure you have enough memory setup in the container to remove the possibility.

Greg

Thanks for the comments everybody. I’ve resolved the cpu spike by moving to a jetty based docker image instead of a tomcat based image. I don’t know why this was the solution, but I’ll stick with it for now.