Multi-instance collection syntax proposal

#1

We are using the flowable extensions for multi-instance user assignment, but we would like to be able to support either a data object which contains a set of users or a string. For example, our design tool supports either a list of user/group ids associated with a role (e.g. potential owner) or a data object reference that contains a set of users/groups. We do not support a list type data object, so we are using JSON syntax:

            [
              {
                "principalType" : "User",
                "role" : "PotentialOwner",
                "principal" : "wfuser1",
                "version" : 1
              },
              {
                "principalType" : "User",
                "role" : "PotentialOwner",
                "principal" : "wfuser2",
                "version" : 1
              }
            ]

We are using a JSON participant task listener to parse the string. We were able to use a string in the collection attribute, but it was quite ugly because we had to escape it:

<multiInstanceLoopCharacteristics isSequential="true" flowable:collection="[ { &quot;principalType&quot; : &quot;User&quot;, &quot;role&quot; : &quot;PotentialOwner&quot;, &quot;principal&quot; : &quot;wfuser1&quot;, &quot;version&quot; : 1 }, { &quot;principalType&quot; : &quot;User&quot;, &quot;role&quot; : &quot;PotentialOwner&quot;, &quot;principal&quot; : &quot;wfuser2&quot;, &quot;version&quot; : 1 } ]" flowable:elementVariable="participant"/>

Ideally, if the collection were an element instead of an attribute we could use an expression for the data object and CDATA for the string value:

<userTask id="usertask1" name="Task A-${loopCounter}">
  <extensionElements>
    <flowable:taskListener event="create" flowable:delegateExpression="${jsonParticipantListener}">
      <flowable:field name="participants">
        <flowable:expression>${participant}</flowable:expression>
      </flowable:field>
    </flowable:taskListener>
  </extensionElements>
  <multiInstanceLoopCharacteristics isSequential="true" flowable:elementVariable="participant">
   <extensionElements>
    <flowable:collection>
     <![CDATA[
            [
              {
                "principalType" : "User",
                "role" : "PotentialOwner",
                "principal" : "wfuser1",
                "version" : 1
              },
              {
                "principalType" : "User",
                "role" : "PotentialOwner",
                "principal" : "wfuser2",
                "version" : 1
              }
            ]
            ]]>
    </flowable:collection>
   </extensionElements>
  </multiInstanceLoopCharacteristics>
</userTask>

I realize adding a new collection element might be confusing while continuing support for the collection attribute, but it could be more generically named.

Any thoughts? If you agree that this might be useful, I can extend the parser accordingly and submit a formal pull request.

Four-eyes principle for approval processes
#2

Imo, it makes sense to add it as an element, as it does makes this particular use case easier.

You could also use another ‘field’ for the task listener and add the collection there. Is there a particular reason you need to use the collection attribute here?

#3

Hi Joram,

The task listener doesn’t control how many instances of the task are created. That is controlled by the collection attribute of the multiInstanceLoopCharacteristics element. In order to re-use the existing multi-instance task behavior the value must be provided via the multiInstanceLoopCharacteristics element.

In our use case the task listener simply receives a single JSON object from the JSON Array, representing an identity, to be set on a particular instance of the task since we have overridden the Flowable SequentialMultiInstanceBehavior and ParallelMultiInstanceBehavior classes to accept an array of JSON objects.

In general, does Flowable support defining a multi-instance user task that is defined with a data object (which is not an implementation of java.util.Collection) that can be used in the collection attribute of a multiInstanceLoopCharacteristics element when setting the data object value via the REST API or in the BPMN file? I poked around the codebase some time ago but didn’t find any examples.

FWIW, our REST API does not allow variables of type java.util.Collection to be defined in Flowable. An alternative approach would be to add a collectionAdapter attribute on the multiInstanceLoopCharacteristics element which would convert the the value of the (collection) data object into a java.util.Collection instance. This would eliminate the need to define a task listener in this use case.

thanks,
Rob

#4

You’re right, I was confusing the task listener and multi-instance behaviour here :slight_smile:

The logic for resolving the collection can be found here: https://github.com/flowable/flowable-engine/blob/master/modules/flowable-engine/src/main/java/org/flowable/engine/impl/bpmn/behavior/MultiInstanceActivityBehavior.java#L211

as you can see, it does expect a java.util.Collection to be resolved eventually. I also think serializing collections is not a good approach … in fact I think it should be an instance of Iterable to be a bit more flexible anyway …

Adding in an adapter class and/or expression there would be useful imho. (ping @tijs : wdyt?)

#5

@robert.hafner : Another thought on this topic: why can’t you use an expression for the collection here? You could have a bean that transforms the json to a real collection in a bean you call in that expression?

#6

An expression function is not sufficient because we allow the IdentityLink type to be set (via JSON) on the Identities that are being associated with the Task instance. Ideally, I’d like to reach a point where we don’t have to override the default behavior of the *MultiInstanceBehavior classes provided by Flowable but use some provided flowable extensions.

#7

Opened Issue #flowable-engine 474

https://github.com/flowable/flowable-engine/issues/474

#8

Proposed another syntax revision in issue 512

#9

One more revision proposed 520 and merged 533

Refer to 523 for additional comments

Essentially refactored parsing function as a separate handler element rather than use JUEL expression syntax.

#10

Hello,

I am having a separate collection variable issue.

The script sets the array assigneeList and prints it to the log which I can confirm is working well. Once the process moves to the multi-instance task, the process just dies. There is nothing in the logs after the script task. Please advise.

Thank you,
Nathan

#11

Can you provide additional implementation details? Do you have a unit test case I could debug? Can you provide the final template. Also can you run your test with debug enabled so the loop details are logged when the multiinstance task is created and completes?

#13

Hi Lori,

Thank you for the response. Unfortunately, I do not have access to a debugger in Flowable. Do you mind telling me how to set this up? In regards to my original question, I have pasted the xml of the process flow below. I appreciate your help.

<process id="newmultiinstancetest" name="newmultiinstancetest" isExecutable="true">
    <startEvent id="startEvent1"></startEvent>
    <sequenceFlow id="sid-18F6D831-FF05-4E51-A135-BE532C3202BD" sourceRef="startEvent1" targetRef="sid-3DA07E73-4C05-44C2-A0C1-CBDEC491E7F6"></sequenceFlow>
    <scriptTask id="sid-3DA07E73-4C05-44C2-A0C1-CBDEC491E7F6" name="Create Collection" scriptFormat="javascript" flowable:autoStoreVariables="false">
      <script><![CDATA[var assigneeList = ["person1", "person2"];        
print("List: " + assigneeList);]]></script>
    </scriptTask>
    <sequenceFlow id="sid-A29BF2FA-669A-4578-B78A-99B44720AFB5" sourceRef="sid-3DA07E73-4C05-44C2-A0C1-CBDEC491E7F6" targetRef="miTasks"></sequenceFlow>
    <userTask id="miTasks" name="Multi Instance Test" flowable:assignee="${assignee}">
      <extensionElements>
        <modeler:initiator-can-complete xmlns:modeler="http://flowable.org/modeler"><![CDATA[false]]></modeler:initiator-can-complete>
      </extensionElements>
      <multiInstanceLoopCharacteristics isSequential="false" flowable:collection="assigneeList" flowable:elementVariable="assignee"></multiInstanceLoopCharacteristics>
    </userTask>
    <sequenceFlow id="sid-864EF8FF-7A21-4141-B93E-F1116C56F9F1" sourceRef="miTasks" targetRef="sid-3E534C8E-8EF5-4333-9E95-1F9B6BF4FA9E"></sequenceFlow>
    <endEvent id="sid-3E534C8E-8EF5-4333-9E95-1F9B6BF4FA9E"></endEvent>
  </process>

Best,
Nathan

#14

I believe the issue is in the script task and the “assigneeList” variable is not getting set properly. When I debugged the multiinstance behavior, I get a null value for “assigneeList”.

11:02:06,099 [main] ERROR org.flowable.engine.impl.test.AbstractTestCase - EXCEPTION: org.flowable.engine.common.api.FlowableIllegalArgumentException: Variable null is not found
org.flowable.engine.common.api.FlowableIllegalArgumentException: Variable null is not found
at org.flowable.engine.impl.bpmn.behavior.MultiInstanceActivityBehavior.resolveAndValidateCollection(MultiInstanceActivityBehavior.java:312)
at org.flowable.engine.impl.bpmn.behavior.MultiInstanceActivityBehavior.resolveNrOfInstances(MultiInstanceActivityBehavior.java:271)
at org.flowable.engine.impl.bpmn.behavior.ParallelMultiInstanceBehavior.createInstances(ParallelMultiInstanceBehavior.java:55)
at org.flowable.engine.impl.bpmn.behavior.MultiInstanceActivityBehavior.execute(MultiInstanceActivityBehavior.java:109)

I have not used the script task functionality, but I believe you need to explicitly set the variable:

http://www.flowable.org/docs/userguide/index.html#bpmnScriptTaskVariables

I tried updating your script task as follows:

	<scriptTask id="sid-3DA07E73-4C05-44C2-A0C1-CBDEC491E7F6"
		name="Create Collection" scriptFormat="javascript"
		flowable:autoStoreVariables="false">
		<script><![CDATA[
		var assigneeList = ["person1", "person2"]; 
		execution.setVariable("assigneeList", assigneeList);       
		print("List: " + assigneeList);
		]]></script>
	</scriptTask>

But I get the following error when it executes:

10:55:27,153 [main] ERROR org.flowable.engine.impl.test.AbstractTestCase - EXCEPTION: org.flowable.engine.common.api.FlowableException: problem evaluating script: sun.org.mozilla.javascript.internal.WrappedException: Wrapped org.flowable.engine.common.api.FlowableException: couldn’t find a variable type that is able to serialize sun.org.mozilla.javascript.internal.NativeArray@d6401d3 (#3) in at line number 3
org.flowable.engine.common.api.FlowableException: problem evaluating script: sun.org.mozilla.javascript.internal.WrappedException: Wrapped org.flowable.engine.common.api.FlowableException: couldn’t find a variable type that is able to serialize sun.org.mozilla.javascript.internal.NativeArray@d6401d3 (#3) in at line number 3
at org.flowable.engine.impl.scripting.ScriptingEngines.evaluate(ScriptingEngines.java:89)
at org.flowable.engine.impl.scripting.ScriptingEngines.evaluate(ScriptingEngines.java:73)
at org.flowable.engine.impl.bpmn.behavior.ScriptTaskActivityBehavior.execute(ScriptTaskActivityBehavior.java:78)

So I think it has to do with the fact that the variable is a collection type. I will have to defer to somneone who is more experienced with using the script task and javascript in general.

#15

I have posted some test files as a draft PR, but it will fail based on the previous post findings

I will open a new issue for the script task

#16

Opened issue https://github.com/flowable/flowable-engine/issues/713 for script task failure

#17

Hi Lori,

Thank you for posting the issue, I hope it can be resolved soon. While we are waiting, do you mind walking me through how I can enable the debugger? I saw some documentation on it but have not been able to figure out how to get it working.

Thank you.

#18

I just run the unit test in my IDE and set debug points either in the Flowable base code or the test itself. I do not use the debugger itself. Alternatively, you could enable debug using the log4j properties if running outside an IDE, but then you can only see explicit debug messages which may not be helpful (i.e. sometimes, there are not too many debug messages).

#19

Hi Lori,

Thank you for the help. I amended the script task with the below and it worked.

var ArrayList = Java.type(‘java.util.ArrayList’);
var assigneeList = new ArrayList();
assigneeList.add(‘kermit’);
assigneeList.add(‘fozzie’);
print("List: " + assigneeList);

execution.setVariable(“assigneeList”, assigneeList);

This may serve as a useful workaround for the time being.

Thank you.

#20

Hello all,

I’m also trying to use a script task to create a collection, but for now I seem to be facing the same serialization issue. I found this post that might help you resolve it by modifying the ProcessEngine (Array or JSON Object Process Variable), but unluckily I can’t do that myself right now. So, I tried nforrester’s amended script task, but it won’t work either, ie when I try to start the process in the Task part, nothing happens, the process doesn’t kick off. What might I be doing wrong that nforrester did right to bypass this serialization issue?

PS : The script is the very same as nforrester’s, and the process model is super standard (start event, script task, user task calling the collection, end task, all properly linked together), but again it just won’t start.

Thanks a lot for your help,
Cheers,
Antoine