I was wondering has any work been done to allow this type of alternative persistence. I understand the main concern is transaction read_committed support and span of the transaction across the nodes of the graph.
Any pointers on where to start with this effort on the flowable code base e.g. is this still the best reference
We made the data persistence for Flowable 6 fully pluggable. So this allows you to implement a data persistency layer on top of a NoSQL database. Our first priority is to get the 6.0.0 release out, but we do have plans to work on a NoSQL data persistence for Flowable 6 in the near future. We are always open for discussion around this topic.
A triple store such as GraphDB would be very interesting. Could facilitate deeper understanding of related data such as Data Objects and enable richer semantics around the process model.
I am glad to hear that you guys have plan to support NoSQL data persistent in near future, and we have similar requirements in our project. Is it possible for you to give more details on the timelines? It would help me to decide whether to wait for it or start developing it.
We would like to develop such support together with the community. So if you want to work together on this and contribute to the Flowable open source project that would be great. Let me know how you would like to proceed and which NoSQL database you are thinking about.
One of the main points to tackle for the NoSQL support (after choosing the first NoSQL database for the implementation), is to come up with a data model design. Besides the executions, there are also the user tasks, event subscriptions, identity links and other information that need to be stored. Definitely needs some design discussions / sessions to get it right.
The DataManager and EntityManager classes are all pluggable via the Process engine configuration. You will need different DataManager implementations for each entity to store and retrieve the entity from the NoSQL database of your choice.
Tijs, do you think we can take advantage of the new fully plugglable data persistace layer to implement CQRS Pattern. That way, we could produce events to a topic stream like kafka instead of directly to a database. Kafka consumers may store to SQL database like we do today and/or any other noSQL.
I think it’s should not be a matter of supporting a database or another, but both in a decoupled way. I’m pretty sure that a Graph Database like Titan is the best approach to some kind of query and document-oriented like elasticsearch or Solr to other - sometimes we need both. With CQRS flowable shouldn’t know anything but how to produce events. And consumers would “listen” only the events that matters to a specific context.
In general I would think that the approach should work. But it would be good to build a small proof of concept to see if you run into issues. The data layer is pluggable, so I don’t expect issues there, but there will definitely be some challenges when implementing full BPMN support.