Classloading issues running flowable 6.4.0 on Karaf

donkon · November 29, 2018, 8:43am

Hi flowable team,

we use flowable 5 together with apache karaf 4.0.x and it’s working quite well. Now we would like to migrate to flowable 6.4.0, but we have classloading issues when bootstrapping the engine.

All required flowable bundles 325 │ Active │ 80 │ 6.4.0 326 │ Active │ 80 │ 6.4.0 327 │ Active │ 80 │ 6.4.0 328 │ Active │ 80 │ 6.4.0 329 │ Active │ 80 │ 6.4.0 330 │ Active │ 80 │ 6.4.0 331 │ Active │ 80 │ 6.4.0 332 │ Active │ 80 │ 6.4.0 333 │ Active │ 80 │ 6.4.0 334 │ Active │ 80 │ 6.4.0 335 │ Active │ 80 │ 6.4.0 336 │ Active │ 80 │ 6.4.0 337 │ Active │ 80 │ 6.4.0 338 │ Active │ 80 │ 6.4.0 339 │ Active │ 80 │ 6.4.0 340 │ Active │ 80 │ 6.4.0 341 │ Active │ 80 │ 6.4.0 342 │ Active │ 80 │ 6.4.0 343 │ Active │ 80 │ 6.4.0 344 │ Active │ 80 │ 6.4.0 345 │ Active │ 80 │ 6.4.0 346 │ Active │ 80 │ 6.4.0 347 │ Active │ 80 │ 6.4.0 348 │ Active │ 80 │ 6.4.0 349 │ Active │ 80 │ 6.4.0 are installed:
│ Flowable - BPMN Converter
│ Flowable - BPMN Model
│ Flowable - CMMN API
│ Flowable - CMMN Model
│ Flowable - Engine Common API
│ Flowable - Engine Common
│ Flowable - Content API
│ Flowable - DMN API
│ Flowable - DMN Model
│ Flowable - Engine
│ Flowable - Form API
│ Flowable - Form Model
│ Flowable - Identity Link Service API
│ Flowable - Identity Link Service
│ Flowable - IDM API
│ Flowable - IDM Engine
│ flowable-idm-engine-configurator
│ Flowable - Image Generator
│ Flowable - Job Service API
│ Flowable - Job Service
│ Flowable - Task Service API
│ Flowable - Task Service
│ Flowable - Process Validation
│ Flowable - Variable Service API
│ Flowable - Variable Service

The engine is configured as follows:

<bean id="processEngineFactory" class="org.flowable.osgi.blueprint.ProcessEngineFactoryWithELResolver" init-method="init" destroy-method="destroy">
		<property name="processEngineConfiguration" ref="processEngineConfiguration" />
		<property name="bundle" ref="blueprintBundle" />
		<property name="blueprintELResolver" ref="blueprintELResolver" />
		<property name="blueprintContextELResolver" ref="blueprintContainerListener" />
</bean>
<bean id="processEngineConfiguration" class="org.flowable.engine.impl.cfg.StandaloneProcessEngineConfiguration" ext:field-injection="true">
		<property name="dataSource" ref="dataSource" />
		<property name="databaseSchemaUpdate" value="true" />
		<property name="disableIdmEngine" value="true" />
		<property name="mailServerHost" value="${smtpHost}" />
		<property name="mailServerPort" value="${smtpPort}" />
		<property name="mailServerDefaultFrom" value="${fromEmail}" />
		<property name="asyncExecutorActivate" value="${jobExecutorActivate}" />
	</bean>

When starting our process-engine with blueprint.xml above, we get following exception:
Caused by: org.flowable.common.engine.api.FlowableException: Error while building ibatis SqlSessionFactory: inStream parameter is null
at org.flowable.common.engine.impl.AbstractEngineConfiguration.initSqlSessionFactory(AbstractEngineConfiguration.java:686)
at org.flowable.engine.impl.cfg.ProcessEngineConfigurationImpl.init(ProcessEngineConfigurationImpl.java:960)
at org.flowable.engine.impl.cfg.ProcessEngineConfigurationImpl.buildProcessEngine(ProcessEngineConfigurationImpl.java:887)
at org.flowable.osgi.blueprint.ProcessEngineFactory.init(ProcessEngineFactory.java:41)
at org.flowable.osgi.blueprint.ProcessEngineFactoryWithELResolver.init(ProcessEngineFactoryWithELResolver.java:42)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.aries.blueprint.utils.ReflectionUtils.invoke(ReflectionUtils.java:331)
at org.apache.aries.blueprint.container.BeanRecipe.invoke(BeanRecipe.java:838)
at org.apache.aries.blueprint.container.BeanRecipe.runBeanProcInit(BeanRecipe.java:591)
… 70 more
Caused by: java.lang.NullPointerException: inStream parameter is null
at java.base/java.util.Objects.requireNonNull(Objects.java:246)
at java.base/java.util.Properties.load(Properties.java:403)
at org.flowable.common.engine.impl.AbstractEngineConfiguration.initSqlSessionFactory(AbstractEngineConfiguration.java:679)
… 81 more

The org.flowable.common.engine.impl.AbstractEngineConfiguration.getResourceAsStream(String) Method does not work in OSGi, from OSGi classloading perspective, this bundle does not see the resources it wants to load (mybatis xml configurations).

Any help would be appreciated.

Regards, Björn

tijs · November 29, 2018, 9:23am

Hi Björn,

We have some basic tests in the flowable-osgi module to test the bootstrapping of the Flowable Engine and starting a simple process instance. Would it be possible to create a unit test in the Flowable OSGi module that shows this issue? Then it would be easier to think of a fix. Or maybe we should create a flowable-karaf module and do this testing from there?

Best regards,

Tijs

donkon · November 29, 2018, 10:55am

the best solution would be to provide a working karaf-example in https://github.com/flowable/flowable-examples. I could provide a sample Maven Module, which demonstrates, how to use Flowable with Karaf (and shows the problems I have with that).

If everything fits for you and it’s working, you provide the module in the flowable-examples repository.

What would be the best way to send the project to you?

donkon · November 29, 2018, 1:45pm

I added the project:

Can you have a look?

donkon · December 3, 2018, 5:46pm

Could you give us feedack, if you will take care of this issue? Otherwise, we have to re-think our migration plan …

tijs · December 6, 2018, 8:39pm

Sorry for the delay in answering. I will give your Karaf demo a try over the next couple of days and come back to you in this forum topic.

Best regards,

Tijs

tijs · December 9, 2018, 3:47pm

Hi,

I’ve tested your example project and could also reproduce the issue.
The strange thing is that it does work within the flowable-osgi unit test for some reason, maybe because of an extra restriction in Apache Karaf.
I’ve tried a couple of ways to get the issue resolved with exporting the IDM engine mapping files folder etc, but got the same error with every change. Also couldn’t find a solution yet while searching on loading a resource file from another bundle. A ways to make it work for now is to disable the IDM engine, with setting the disableIdmEngine property to true in the process engine configuration.

Best regards,

Tijs

donkon · December 11, 2018, 8:05am

I also noticed, that resources are loaded from other bundles (e.g. flowable-engine loads resources from flowable-engine-common:org.flowable.common.db.properties), so if using OSGi, those packages should also be added to the OSGi export/import declarations.

But imho, this should not work with the org.flowable.osgi.blueprint.BundleDelegatingClassLoader, which is configured for the ProcessEngineConfiguration. I dont understand, why a special classloader is needed here !?

The disableIdmEngine=true configuration changes the classloading behaviour but does not solve the problem, I still see Nullpointers caused by Resources, which could not be load.

tijs · December 11, 2018, 5:37pm

Yes I understand your point. But when I disabled the IDM engine the process engine was being created successfully in my tests. For you the process engine bundle still fails?

Best regards,

Tijs

donkon · December 12, 2018, 1:56pm

yes, it fails with

Caused by: java.lang.NullPointerException
	at java.io.Reader.<init>(Reader.java:78)
	at java.io.InputStreamReader.<init>(InputStreamReader.java:72)
	at org.flowable.common.engine.impl.AbstractEngineConfiguration.initSqlSessionFactory(AbstractEngineConfiguration.java:659)

I updated my sample project with disableIdmEngine=true, could you pull my changes and check, why we have different behaviour? (I also added a H2 Datasource, see changes in README.md)

Regards, Björn

Ceddaerrix · December 17, 2018, 12:01pm

Hello,
I am trying to use flowable-osgi v6.4.0 with Felix v6.0.1. From the bundle I want to configure and use the ProcessEngine from I also have the an input stream NULL:

Caused by: java.lang.IllegalArgumentException: InputStream cannot be null
        at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:117)
        at org.flowable.common.engine.impl.AbstractEngineConfigurator.registerCustomMybatisMappings(AbstractEngineConfigurator.java:106)
        at org.flowable.common.engine.impl.AbstractEngineConfigurator.beforeInit(AbstractEngineConfigurator.java:50)
        at org.flowable.common.engine.impl.AbstractEngineConfiguration.configuratorsBeforeInit(AbstractEngineConfiguration.java:852)
        at org.flowable.engine.impl.cfg.ProcessEngineConfigurationImpl.init(ProcessEngineConfigurationImpl.java:912)
        at org.flowable.engine.impl.cfg.ProcessEngineConfigurationImpl.buildProcessEngine(ProcessEngineConfigurationImpl.java:887)

From debugging, I found out that it is related to the IdmEngineConfigurator.getMybatisCfgPath() (i.e. ‘org/flowable/idm/db/mapping/mappings.xml’) which does not exist within that specific bundle, nor is the AbstractEngineConfiguration.getClassLoader() provided (i.e. NULL).
Thus I assume that it is in the same kind of ideas in regard to Classloading issues for Flowable 6.4.0 with OSGi.

donkon · December 17, 2018, 9:49pm

yes, when disableIdmEngine=false is set, I have the same behaviour. Looking forward to see this issue fixed

Ceddaerrix · December 19, 2018, 12:41pm

I have tried the process-engine module from the demo you posted in my case. The bundle start well but after some time I get the following error:

g! 13:57:52.283 [Blueprint Extender: 3] ERROR org.apache.aries.blueprint.container.BlueprintContainerImpl - Unable to start container for blueprint bundle com.ge.flowbase.flowbase-flowable-process-engine/1.0.0.SNAPSHOT due to unresolved dependencies [(objectClass=javax.sql.DataSource)]
java.util.concurrent.TimeoutException: null
        at org.apache.aries.blueprint.container.BlueprintContainerImpl$1.run(BlueprintContainerImpl.java:373) [org.apache.aries.blueprint.core-1.10.1.jar:1.10.1]
        at org.apache.aries.blueprint.utils.threading.impl.DiscardableRunnable.run(DiscardableRunnable.java:45) [org.apache.aries.blueprint.core-1.10.1.jar:1.10.1]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_121]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_121]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]

Maybe another symptom bound to the class loading issue

mimi · December 20, 2018, 10:41am

Hi all,

I am very interesting by this.
I would like to do that, so I took this sample and try it.
Unfortunatly, it’s does not work with the final command (i.e. diag process-engine)
Unable to initialize bean processEngineFactory
org.osgi.service.blueprint.container.ComponentDefinitionException: Unable to initialize bean processEngineFactory
at org.apache.aries.blueprint.container.BeanRecipe.runBeanProcInit(BeanRecipe.java:593)
at org.apache.aries.blueprint.container.BeanRecipe.internalCreate2(BeanRecipe.java:703)
at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:666)
at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:90)
at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:360)
at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:190)
at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:716)
at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:413)
at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:278)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.aries.blueprint.container.ExecutorServiceWrapper.run(ExecutorServiceWrapper.java:106)
at org.apache.aries.blueprint.utils.threading.impl.DiscardableRunnable.run(DiscardableRunnable.java:45)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.flowable.common.engine.api.FlowableException: Error while building ibatis SqlSessionFactory: null
at org.flowable.common.engine.impl.AbstractEngineConfiguration.initSqlSessionFactory(AbstractEngineConfiguration.java:686)
at org.flowable.engine.impl.cfg.ProcessEngineConfigurationImpl.init(ProcessEngineConfigurationImpl.java:960)
at org.flowable.engine.impl.cfg.ProcessEngineConfigurationImpl.buildProcessEngine(ProcessEngineConfigurationImpl.java:887)
at org.flowable.osgi.blueprint.ProcessEngineFactory.init(ProcessEngineFactory.java:41)
at org.flowable.osgi.blueprint.ProcessEngineFactoryWithELResolver.init(ProcessEngineFactoryWithELResolver.java:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.aries.blueprint.utils.ReflectionUtils.invoke(ReflectionUtils.java:331)
at org.apache.aries.blueprint.container.BeanRecipe.invoke(BeanRecipe.java:838)
at org.apache.aries.blueprint.container.BeanRecipe.runBeanProcInit(BeanRecipe.java:591)
… 21 more
Caused by: java.lang.NullPointerException
at java.io.Reader.(Reader.java:78)
at java.io.InputStreamReader.(InputStreamReader.java:72)
at org.flowable.common.engine.impl.AbstractEngineConfiguration.initSqlSessionFactory(AbstractEngineConfiguration.java:659)
… 32 more

tijs · December 23, 2018, 10:38am

Hi Björn,

Finally, I’ve been playing around with your example a bit more.
I’ve made some changes on the current Flowable master (6.4.1-SNAPSHOT):

With these changes the Flowable Engine boots ok now in your example. You can also include the IDM engine. Let me know if it also works for you.

Best regards,

Tijs

tijs · December 29, 2018, 10:43am

@donkon, did the changes I referred to in my last post solve the issue for you as well?

Best regards,

Tijs

donkon · December 29, 2018, 11:03am

@tijs yes, the changes solved the issue for setting up the engine in Karaf, the following was required in my flowable-karaf-demo:

I added more functionality to the flowable-karaf-demo to see a bpmn workflow running in Karaf, unfortunatilly I encountered ClassNotFoundExcpetions at runtime (executing a ServiceTask):

org.flowable.common.engine.api.FlowableException: Error while evaluating expression: ${stringOperations.getSize(input)}
	at org.flowable.common.engine.impl.el.JuelExpression.getValue(JuelExpression.java:56) ~[71:org.flowable.common.engine.impl:6.4.1.SNAPSHOT]
....
Caused by: java.lang.ClassNotFoundException: org.flowable.common.engine.impl.de.odysseus.el.ExpressionFactoryImpl
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:?]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:?]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:?]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:?]
	at org.flowable.common.engine.impl.javax.el.ExpressionFactory.newInstance(ExpressionFactory.java:204) ~[?:?]
	at org.flowable.common.engine.impl.javax.el.ExpressionFactory.newInstance(ExpressionFactory.java:183) ~[?:?]
	at org.flowable.common.engine.impl.javax.el.ExpressionFactory.newInstance(ExpressionFactory.java:89) ~[?:?]
	at org.flowable.common.engine.impl.javax.el.BeanELResolver.getExpressionFactory(BeanELResolver.java:564) ~[?:?]

I think, the problem is the

Thread.currentThread().getContextClassLoader()

part in ExpressionFactory, which is not OSGi capable.

You can pull from flowable-karaf-demo to reproduce this issue (also notice the updated README.md)

Ceddaerrix · January 7, 2019, 1:57pm

Event though my issue is on Flowable-engine v5.23.0 (as v6.4.0 currently issue), I have some class loading issues in regard to classes/beans used within a task delegate that are imported from another module. (see details in my last post under Flowable-engine in OSGi - Tutorials/examples?).

Has anyone met that situation?

donkon · January 9, 2019, 2:04pm

@tijs Any ideas regarding my last post?

@Ceddaerrix I answered in your post

tijs · January 9, 2019, 3:10pm

@donkon not yet but will look at it in the next days and come back to you.

Best regards,

Tijs

Topic		Replies	Views
Flowable + Apache Karaf Flowable Engine	7	1075	July 25, 2018
Flowable-Camel Compatibility	2	483	July 31, 2020
Flowable 6.2.0 released Announcements	0	1037	October 14, 2017
ServiceTask - FlowableClassLoadingException: Could not load class Flowable Application	2	1337	August 13, 2019
Error in Migration Documentation Flowable Engine	1	487	June 29, 2017

Classloading issues running flowable 6.4.0 on Karaf

Related topics