Any support for alternative persistence such as GraphDB?

Hi,

I recall some time ago there was an interesting post on using neo4j with activiti - http://www.jorambarrez.be/blog/2013/07/05/nosql-experimentations-with-activiti-a-very-simple-neo4j-prototype/

I was wondering has any work been done to allow this type of alternative persistence. I understand the main concern is transaction read_committed support and span of the transaction across the nodes of the graph.

Any pointers on where to start with this effort on the flowable code base e.g. is this still the best reference

Thanks, Conor.

Hi Conor,

We made the data persistence for Flowable 6 fully pluggable. So this allows you to implement a data persistency layer on top of a NoSQL database. Our first priority is to get the 6.0.0 release out, but we do have plans to work on a NoSQL data persistence for Flowable 6 in the near future. We are always open for discussion around this topic.

Best regards,

Tijs

It would be interesting to hear what non-relational DBs people think we should focus on, as there’s quite a range, each with their own “specialness”.

Thanks
Paul.

A triple store such as GraphDB would be very interesting. Could facilitate deeper understanding of related data such as Data Objects and enable richer semantics around the process model.

Hi Tijs,

I am glad to hear that you guys have plan to support NoSQL data persistent in near future, and we have similar requirements in our project. Is it possible for you to give more details on the timelines? It would help me to decide whether to wait for it or start developing it.

Thanks
Renu

Hi Renu,

We would like to develop such support together with the community. So if you want to work together on this and contribute to the Flowable open source project that would be great. Let me know how you would like to proceed and which NoSQL database you are thinking about.

Best regards,

Tijs

I am also very interested in helping with this - I am still thinking about GraphDB/neo4j or something similar.

One of the main points to tackle for the NoSQL support (after choosing the first NoSQL database for the implementation), is to come up with a data model design. Besides the executions, there are also the user tasks, event subscriptions, identity links and other information that need to be stored. Definitely needs some design discussions / sessions to get it right.

Best regards,

Tijs

Could you point to some classes / tests to help us start ?
We have an interest to support Apache Solr as persistence store.

Hi,

The DataManager and EntityManager classes are all pluggable via the Process engine configuration. You will need different DataManager implementations for each entity to store and retrieve the entity from the NoSQL database of your choice.

Best regards,

Tijs

Tijs, do you think we can take advantage of the new fully plugglable data persistace layer to implement CQRS Pattern. That way, we could produce events to a topic stream like kafka instead of directly to a database. Kafka consumers may store to SQL database like we do today and/or any other noSQL.

I think it’s should not be a matter of supporting a database or another, but both in a decoupled way. I’m pretty sure that a Graph Database like Titan is the best approach to some kind of query and document-oriented like elasticsearch or Solr to other - sometimes we need both. With CQRS flowable shouldn’t know anything but how to produce events. And consumers would “listen” only the events that matters to a specific context.

What do you think?

Any ideas about my suggestion?

Hi,

In general I would think that the approach should work. But it would be good to build a small proof of concept to see if you run into issues. The data layer is pluggable, so I don’t expect issues there, but there will definitely be some challenges when implementing full BPMN support.

Best regards,

Tijs