How to get a single multi-instance task to execute in parallel?

For performance reasons, I’d like the execution of a multi-instance task to occur in parallel. If I set the “Multi-instance type” to “Parallel” with a cardinality of, say, 5, the execution is still sequential. So, #1 finishes before #2 starts, #2 finishes before #3 starts, etc. I would like for #1 to be started, and then for #2 to be started before #1 finishes, etc.

The only way I have found to get parallel execution is to have exclusive:false and async:true, but in that case, I see a FlowableOptimisticLockingException when the nrOfActiveInstances value is updated.

Other users have mentioned this issue. For example:


I have pasted below the BPML doc that I’m using to send email. Is there a way to send multiple emails in parallel?

Thanks!

Joel

<?xml version="1.0" encoding="UTF-8"?>
<definitions xmlns="http://www.omg.org/spec/BPMN/20100524/MODEL"     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:flowable="http://flowable.org/bpmn" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:omgdc="http://www.omg.org/spec/DD/20100524/DC" xmlns:omgdi="http://www.omg.org/spec/DD/20100524/DI" typeLanguage="http://www.w3.org/2001/XMLSchema" expressionLanguage="http://www.w3.org/1999/XPath" targetNamespace="http://www.flowable.org/processdef">
  <process id="multiSMTPHardcoded" name="Multiple SMTP Hardcoded" isExecutable="true">
    <startEvent id="startEvent1" flowable:formFieldValidation="true"></startEvent>
    <endEvent id="sid-75245952-2AAF-45E1-B435-20A66D156FF9"></endEvent>
    <sequenceFlow id="sid-C5E88507-EFD1-4BFE-8422-EE18BCA40C23" sourceRef="startEvent1" targetRef="sid-AAD20524-8797-4ABF-BEE6-8A4DF8CA7093"></sequenceFlow>
    <sequenceFlow id="sid-B9A098CF-AEDB-4D01-A055-CEDFB9874ABE" sourceRef="sid-AAD20524-8797-4ABF-BEE6-8A4DF8CA7093" targetRef="sid-75245952-2AAF-45E1-B435-20A66D156FF9"></sequenceFlow>
    <serviceTask id="sid-AAD20524-8797-4ABF-BEE6-8A4DF8CA7093" flowable:exclusive="false" flowable:async="true" flowable:type="mail">
      <extensionElements>
        <flowable:field name="to">
          <flowable:string><![CDATA[bob@example.com]]></flowable:string>
        </flowable:field>
        <flowable:field name="from">
          <flowable:string><![CDATA[bob@example.com]]></flowable:string>
        </flowable:field>
        <flowable:field name="subject">
          <flowable:string><![CDATA[Testing]]></flowable:string>
        </flowable:field>
        <flowable:field name="text">
          <flowable:string><![CDATA[from multi SMTP hardcoded]]></flowable:string>
        </flowable:field>
      </extensionElements>
      <multiInstanceLoopCharacteristics isSequential="false">
        <loopCardinality>5</loopCardinality>
      </multiInstanceLoopCharacteristics>
    </serviceTask>
  </process>
  <bpmndi:BPMNDiagram id="BPMNDiagram_multiSMTPHardcoded">
    <bpmndi:BPMNPlane bpmnElement="multiSMTPHardcoded"     id="BPMNPlane_multiSMTPHardcoded">
      <bpmndi:BPMNShape bpmnElement="startEvent1" id="BPMNShape_startEvent1">
        <omgdc:Bounds height="30.0" width="30.0" x="90.0" y="135.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-75245952-2AAF-45E1-B435-20A66D156FF9" id="BPMNShape_sid-75245952-2AAF-45E1-B435-20A66D156FF9">
        <omgdc:Bounds height="28.0" width="28.0" x="519.0" y="150.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-AAD20524-8797-4ABF-BEE6-8A4DF8CA7093" id="BPMNShape_sid-AAD20524-8797-4ABF-BEE6-8A4DF8CA7093">
        <omgdc:Bounds height="80.0" width="100.0" x="225.0" y="120.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNEdge bpmnElement="sid-B9A098CF-AEDB-4D01-A055-CEDFB9874ABE" id="BPMNEdge_sid-B9A098CF-AEDB-4D01-A055-CEDFB9874ABE">
        <omgdi:waypoint x="324.95000000000005" y="160.77441860465117"></omgdi:waypoint>
        <omgdi:waypoint x="519.0010168256147" y="163.78297128886987"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge bpmnElement="sid-C5E88507-EFD1-4BFE-8422-EE18BCA40C23" id="BPMNEdge_sid-C5E88507-EFD1-4BFE-8422-EE18BCA40C23">
        <omgdi:waypoint x="119.9499959921313" y="150.0"></omgdi:waypoint>
        <omgdi:waypoint x="172.5" y="150.0"></omgdi:waypoint>
        <omgdi:waypoint x="172.5" y="160.0"></omgdi:waypoint>
        <omgdi:waypoint x="225.0" y="160.0"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
    </bpmndi:BPMNPlane>
  </bpmndi:BPMNDiagram>
</definitions>

So what is happening is the following:

  • if you set async to true, the activity will be executed by the job executor, which has a threadpool with x threads by default.
  • if you set exclusive to true, it means only one element of your process instance is allowed to be executed at once. If false (like here), there’s no such guard.

The problem here is that sending the mails happens all very fast, and afterwards, the instances will all try to update the same process instance to indicate the instance has completed and the instance can (potentially) move ahead.

What you could do is

  • wrap the multi instance task in a subprocess and have a step after the sending of the email that does have exclusive to true, to force synchronization
  • Not use the multi instance for this, but have a custom service task where you boot up your own threads to do it, effectively making it a single step for the engine.

Thank you for the quick response. I think I’ll try the custom service task.

Joel

I tried creating a parallel, async, multi-instance subprocess. It contained nothing but the start and end tasks.

When I started a process instance, it executed forever. In my debug logs I kept seeing MULTI_INSTANCE_ACTIVITY_STARTED.

<?xml version="1.0" encoding="UTF-8"?>
<definitions xmlns="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:flowable="http://flowable.org/bpmn" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:omgdc="http://www.omg.org/spec/DD/20100524/DC" xmlns:omgdi="http://www.omg.org/spec/DD/20100524/DI" typeLanguage="http://www.w3.org/2001/XMLSchema" expressionLanguage="http://www.w3.org/1999/XPath" targetNamespace="http://www.flowable.org/processdef">
  <process id="subproc1" name="SubProc1" isExecutable="true">
    <startEvent id="startEvent1" flowable:formFieldValidation="true"></startEvent>
    <endEvent id="sid-053666F2-962F-4BD6-9FEE-C72AB7A76E84"></endEvent>
    <subProcess id="sid-D260ECA5-0D4D-4FCD-A7F4-CE2CEB28FBAE" name="subProcess" flowable:async="true" flowable:exclusive="false">
      <multiInstanceLoopCharacteristics isSequential="false" flowable:collection="delayList" flowable:elementVariable="delay">
        <loopCardinality>2</loopCardinality>
      </multiInstanceLoopCharacteristics>
      <startEvent id="sid-8C478945-392B-474F-BE82-391F309211C8" flowable:formFieldValidation="true"></startEvent>
      <endEvent id="sid-B4A7ECFD-0296-4376-8C63-C8D57CD204A2"></endEvent>
      <sequenceFlow id="sid-E7A840A0-FF66-4BC8-BF2D-26456EF1FF5A" sourceRef="sid-8C478945-392B-474F-BE82-391F309211C8" targetRef="sid-B4A7ECFD-0296-4376-8C63-C8D57CD204A2"></sequenceFlow>
    </subProcess>
    <sequenceFlow id="sid-5A5B7230-F8EB-47BA-B561-9C9E9D63BA48" sourceRef="sid-D260ECA5-0D4D-4FCD-A7F4-CE2CEB28FBAE" targetRef="sid-053666F2-962F-4BD6-9FEE-C72AB7A76E84"></sequenceFlow>
    <sequenceFlow id="sid-12B8ECAA-210E-4638-8EEC-A369C989B829" sourceRef="startEvent1" targetRef="sid-D260ECA5-0D4D-4FCD-A7F4-CE2CEB28FBAE"></sequenceFlow>
  </process>
  <bpmndi:BPMNDiagram id="BPMNDiagram_subproc1">
    <bpmndi:BPMNPlane bpmnElement="subproc1" id="BPMNPlane_subproc1">
      <bpmndi:BPMNShape bpmnElement="startEvent1" id="BPMNShape_startEvent1">
        <omgdc:Bounds height="30.0" width="30.0" x="90.0" y="150.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-053666F2-962F-4BD6-9FEE-C72AB7A76E84" id="BPMNShape_sid-053666F2-962F-4BD6-9FEE-C72AB7A76E84">
        <omgdc:Bounds height="28.0" width="28.0" x="990.0" y="165.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-D260ECA5-0D4D-4FCD-A7F4-CE2CEB28FBAE" id="BPMNShape_sid-D260ECA5-0D4D-4FCD-A7F4-CE2CEB28FBAE">
        <omgdc:Bounds height="158.0" width="267.0" x="615.0" y="150.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-8C478945-392B-474F-BE82-391F309211C8" id="BPMNShape_sid-8C478945-392B-474F-BE82-391F309211C8">
        <omgdc:Bounds height="30.0" width="30.0" x="630.0" y="210.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-B4A7ECFD-0296-4376-8C63-C8D57CD204A2" id="BPMNShape_sid-B4A7ECFD-0296-4376-8C63-C8D57CD204A2">
        <omgdc:Bounds height="28.0" width="28.0" x="825.0" y="195.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNEdge bpmnElement="sid-5A5B7230-F8EB-47BA-B561-9C9E9D63BA48" id="BPMNEdge_sid-5A5B7230-F8EB-47BA-B561-9C9E9D63BA48">
        <omgdi:waypoint x="881.9499999999999" y="202.8747553816047"></omgdi:waypoint>
        <omgdi:waypoint x="990.2580328151105" y="181.67962597475602"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge bpmnElement="sid-12B8ECAA-210E-4638-8EEC-A369C989B829" id="BPMNEdge_sid-12B8ECAA-210E-4638-8EEC-A369C989B829">
        <omgdi:waypoint x="119.83550679136727" y="166.8580868201084"></omgdi:waypoint>
        <omgdi:waypoint x="615.0" y="228.8747553816047"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge bpmnElement="sid-E7A840A0-FF66-4BC8-BF2D-26456EF1FF5A" id="BPMNEdge_sid-E7A840A0-FF66-4BC8-BF2D-26456EF1FF5A">
        <omgdi:waypoint x="659.9002175079687" y="223.76700610213987"></omgdi:waypoint>
        <omgdi:waypoint x="825.0466174018169" y="210.14668449180073"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
    </bpmndi:BPMNPlane>
  </bpmndi:BPMNDiagram>
</definitions>

What’s in this collection variable? Can you paste the logs (as it definitely shouldn’t loop forever)?