[Event Registry] Is it a good practise to use two rabbit queues(inbound and outbound), two events(inbound and outbound) and two channels(inbound and outbound) for lots of processes with many EventRegistry elements?

[aleksandrs]

I have quite a few processes, all of which contain EventRegistry blocks for both sending and receiving messages from RabbitMQ. Initially I wanted to make a new set(inbound and outbound events/channels, inbound queue) for each of the blocks, but as the numbers of these started to grow I decided that I’d want to utilized them as much as possible, ideally I’d want only two queues(inbound and outbound) and two sets of channels and event(inbound and outbound) used by all of these processes.

So far, making a new process that uses the same channels and events that are already in use in other processes really messes up how all of them receive messages. Sometimes an EventRegistry block just won’t read any messages, sometimes it will only read every second message. All of these blocks do have correlation parameters, so I don’t see what causes the problems.

What I’d like to know whether it is a good idea to try to utilize one set of channels, events and queues for multiple processes with many EventRegistry blocks, or should I make a new set for each one ? Is it designed to be used that way ?

[filiphr]

Hey @aleksandrs,

I don’t know why you are seeing what you are seeing. Can you perhaps share your models so we can have a look at them?

@aleksandrs

What I’d like to know whether it is a good idea to try to utilize one set of channels, events and queues for multiple processes with many EventRegistry blocks, or should I make a new set for each one ? Is it designed to be used that way ?

The event registry is designed in such a way that you can use a single inbound / outbound channel to receive messages for multiple processes or send out messages from multiple processes.

Cheers,
Filip

[aleksandrs]

Hey @filiphr

Here’s an example.

Process:

    <?xml version="1.0" encoding="UTF-8"?>
<definitions xmlns="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:flowable="http://flowable.org/bpmn" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:omgdc="http://www.omg.org/spec/DD/20100524/DC" xmlns:omgdi="http://www.omg.org/spec/DD/20100524/DI" typeLanguage="http://www.w3.org/2001/XMLSchema" expressionLanguage="http://www.w3.org/1999/XPath" targetNamespace="http://www.flowable.org/processdef">
  <process id="testReportTransfer" name="testReportTransfer" isExecutable="true">
    <sequenceFlow id="sid-99ABC489-0019-4B12-8E64-6B29C183FF04" sourceRef="sid-E99CBF2A-7632-43CF-B76E-0F66C1B7E37C" targetRef="sid-4329F77F-8E53-4CB5-A574-DA8707231985"></sequenceFlow>
    <serviceTask id="sid-7BB278FF-2648-4123-9630-C717C80316D7" name="missing report email" flowable:type="send-event">
      <extensionElements>
        <flowable:eventType><![CDATA[reportOutboundEvent]]></flowable:eventType>
        <flowable:eventInParameter source="sendMissingPreArbReportEmail" target="task" targetType="string"></flowable:eventInParameter>
        <flowable:eventInParameter sourceExpression="${pid}" target="pid" targetType="string"></flowable:eventInParameter>
        <flowable:eventInParameter source="test@test.com" target="email" targetType="string"></flowable:eventInParameter>
        <flowable:channelKey xmlns:flowable="http://flowable.org/bpmn"><![CDATA[outboundChannel]]></flowable:channelKey>
      </extensionElements>
    </serviceTask>
    <endEvent id="sid-500E50FA-4E1F-405C-84B4-F2AA776724A3"></endEvent>
    <sequenceFlow id="sid-1FE156C9-383E-4E9D-9947-BED9FFBEB88A" sourceRef="sid-7BB278FF-2648-4123-9630-C717C80316D7" targetRef="sid-500E50FA-4E1F-405C-84B4-F2AA776724A3"></sequenceFlow>
    <serviceTask id="sid-4329F77F-8E53-4CB5-A574-DA8707231985" name="retrieve processed report" flowable:type="send-event" flowable:triggerable="true">
      <extensionElements>
        <flowable:eventType><![CDATA[reportOutboundEvent]]></flowable:eventType>
        <flowable:triggerEventType><![CDATA[reportInboundEvent]]></flowable:triggerEventType>
        <flowable:eventInParameter source="sftpTransfer" target="task" targetType="string"></flowable:eventInParameter>
        <flowable:eventInParameter source="processedPreArbReportToLocalhost" target="ftTask" targetType="string"></flowable:eventInParameter>
        <flowable:eventInParameter sourceExpression="${pid}" target="pid" targetType="string"></flowable:eventInParameter>
        <flowable:eventInParameter sourceExpression="new_prearb_arb_${pid}.xlsx" target="fileName" targetType="string"></flowable:eventInParameter>
        <flowable:eventInParameter sourceExpression="${pid}_Pre-Arb/validation" target="fileDir" targetType="string"></flowable:eventInParameter>
        <flowable:eventOutParameter source="task" sourceType="string" target="task"></flowable:eventOutParameter>
        <flowable:eventOutParameter source="error" sourceType="string" target="error"></flowable:eventOutParameter>
        <flowable:eventOutParameter source="fileName" sourceType="string" target="fileName"></flowable:eventOutParameter>
        <flowable:eventOutParameter source="fileDir" sourceType="string" target="fileDir"></flowable:eventOutParameter>
        <flowable:channelKey xmlns:flowable="http://flowable.org/bpmn"><![CDATA[outboundChannel]]></flowable:channelKey>
        <flowable:triggerChannelKey xmlns:flowable="http://flowable.org/bpmn"><![CDATA[reportInboundChannel]]></flowable:triggerChannelKey>
        <flowable:triggerEventCorrelationParameter xmlns:flowable="http://flowable.org/bpmn" name="task" type="string" value="sftpTransfer"></flowable:triggerEventCorrelationParameter>
        <flowable:triggerEventCorrelationParameter xmlns:flowable="http://flowable.org/bpmn" name="fileName" type="string" value="new_prearb_arb_${pid}.xlsx"></flowable:triggerEventCorrelationParameter>
        <flowable:triggerEventCorrelationParameter xmlns:flowable="http://flowable.org/bpmn" name="fileDir" type="string" value="${pid}_Pre-Arb/validation"></flowable:triggerEventCorrelationParameter>
      </extensionElements>
    </serviceTask>
    <exclusiveGateway id="sid-E17557FC-462F-4C1E-BB04-F57B77752903"></exclusiveGateway>
    <sequenceFlow id="sid-9E153D3B-4BD1-456E-AA5B-FC0CC4EDEB23" sourceRef="sid-4329F77F-8E53-4CB5-A574-DA8707231985" targetRef="sid-E17557FC-462F-4C1E-BB04-F57B77752903"></sequenceFlow>
    <sequenceFlow id="sid-A352F906-B74B-4EDE-8443-E0DC2AD3B3EC" sourceRef="sid-E17557FC-462F-4C1E-BB04-F57B77752903" targetRef="sid-7BB278FF-2648-4123-9630-C717C80316D7">
      <conditionExpression xsi:type="tFormalExpression"><![CDATA[${error!=null}]]></conditionExpression>
    </sequenceFlow>
    <sequenceFlow id="sid-E9E2146D-91DF-4815-85F5-81ACA05C84B6" sourceRef="sid-E17557FC-462F-4C1E-BB04-F57B77752903" targetRef="sid-604A63B8-315A-429C-A195-6593CA5BB399">
      <conditionExpression xsi:type="tFormalExpression"><![CDATA[${error==null}]]></conditionExpression>
    </sequenceFlow>
    <startEvent id="sid-E99CBF2A-7632-43CF-B76E-0F66C1B7E37C" flowable:formFieldValidation="true"></startEvent>
    <userTask id="sid-604A63B8-315A-429C-A195-6593CA5BB399" flowable:formFieldValidation="true"></userTask>
    <endEvent id="sid-A9834B20-6EC8-4191-A4AB-E960D450029A"></endEvent>
    <sequenceFlow id="sid-2BFD642B-246F-47D7-848B-9CAFDC4F56C8" sourceRef="sid-604A63B8-315A-429C-A195-6593CA5BB399" targetRef="sid-A9834B20-6EC8-4191-A4AB-E960D450029A"></sequenceFlow>
  </process>
  <bpmndi:BPMNDiagram id="BPMNDiagram_testReportTransfer">
    <bpmndi:BPMNPlane bpmnElement="testReportTransfer" id="BPMNPlane_testReportTransfer">
      <bpmndi:BPMNShape bpmnElement="sid-7BB278FF-2648-4123-9630-C717C80316D7" id="BPMNShape_sid-7BB278FF-2648-4123-9630-C717C80316D7">
        <omgdc:Bounds height="80.0" width="100.0" x="495.0" y="255.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-500E50FA-4E1F-405C-84B4-F2AA776724A3" id="BPMNShape_sid-500E50FA-4E1F-405C-84B4-F2AA776724A3">
        <omgdc:Bounds height="28.0" width="28.0" x="531.0" y="375.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-4329F77F-8E53-4CB5-A574-DA8707231985" id="BPMNShape_sid-4329F77F-8E53-4CB5-A574-DA8707231985">
        <omgdc:Bounds height="80.0" width="100.0" x="360.0" y="135.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-E17557FC-462F-4C1E-BB04-F57B77752903" id="BPMNShape_sid-E17557FC-462F-4C1E-BB04-F57B77752903">
        <omgdc:Bounds height="40.0" width="40.0" x="525.0" y="155.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-E99CBF2A-7632-43CF-B76E-0F66C1B7E37C" id="BPMNShape_sid-E99CBF2A-7632-43CF-B76E-0F66C1B7E37C">
        <omgdc:Bounds height="30.0" width="30.0" x="255.0" y="160.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-604A63B8-315A-429C-A195-6593CA5BB399" id="BPMNShape_sid-604A63B8-315A-429C-A195-6593CA5BB399">
        <omgdc:Bounds height="80.0" width="100.0" x="615.0" y="135.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape bpmnElement="sid-A9834B20-6EC8-4191-A4AB-E960D450029A" id="BPMNShape_sid-A9834B20-6EC8-4191-A4AB-E960D450029A">
        <omgdc:Bounds height="28.0" width="28.0" x="780.0" y="161.0"></omgdc:Bounds>
      </bpmndi:BPMNShape>
      <bpmndi:BPMNEdge bpmnElement="sid-9E153D3B-4BD1-456E-AA5B-FC0CC4EDEB23" id="BPMNEdge_sid-9E153D3B-4BD1-456E-AA5B-FC0CC4EDEB23">
        <omgdi:waypoint x="459.95000000000005" y="175.0"></omgdi:waypoint>
        <omgdi:waypoint x="525.0" y="175.0"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge bpmnElement="sid-E9E2146D-91DF-4815-85F5-81ACA05C84B6" id="BPMNEdge_sid-E9E2146D-91DF-4815-85F5-81ACA05C84B6">
        <omgdi:waypoint x="564.9416944213081" y="175.0"></omgdi:waypoint>
        <omgdi:waypoint x="615.0" y="175.0"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge bpmnElement="sid-99ABC489-0019-4B12-8E64-6B29C183FF04" id="BPMNEdge_sid-99ABC489-0019-4B12-8E64-6B29C183FF04">
        <omgdi:waypoint x="284.9499990675947" y="175.0"></omgdi:waypoint>
        <omgdi:waypoint x="360.0" y="175.0"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge bpmnElement="sid-2BFD642B-246F-47D7-848B-9CAFDC4F56C8" id="BPMNEdge_sid-2BFD642B-246F-47D7-848B-9CAFDC4F56C8">
        <omgdi:waypoint x="714.9499999999999" y="175.0"></omgdi:waypoint>
        <omgdi:waypoint x="780.0" y="175.0"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge bpmnElement="sid-1FE156C9-383E-4E9D-9947-BED9FFBEB88A" id="BPMNEdge_sid-1FE156C9-383E-4E9D-9947-BED9FFBEB88A">
        <omgdi:waypoint x="545.0" y="334.95000000000005"></omgdi:waypoint>
        <omgdi:waypoint x="545.0" y="375.0"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge bpmnElement="sid-A352F906-B74B-4EDE-8443-E0DC2AD3B3EC" id="BPMNEdge_sid-A352F906-B74B-4EDE-8443-E0DC2AD3B3EC">
        <omgdi:waypoint x="545.0" y="194.94169442131556"></omgdi:waypoint>
        <omgdi:waypoint x="545.0" y="255.0"></omgdi:waypoint>
      </bpmndi:BPMNEdge>
    </bpmndi:BPMNPlane>
  </bpmndi:BPMNDiagram>
</definitions>

Outbound channel:

{
  "key": "outboundChannel",
  "category": "channel",
  "name": "outboundChannel",
  "description": "Outbound channel",
  "channelType": "outbound",
  "type": "rabbit",
  "serializerType": "json",
  "routingKey":"OutboundQueue"
}

Outbound event:

{
  "key": "reportOutboundEvent",
  "name": "reportOutboundEvent",
  "outboundChannelKeys": [
    "outboundChannel"
  ],
  "correlationParameters": [],
  "payload": [
    {
      "name": "task",
      "type": "string"
    },
    {
      "name": "email",
      "type": "string"
    },
    {
      "name": "pid",
      "type": "string"
    },
    {
       "name": "ftTask",
       "type": "string"
    },
    {
       "name": "fileDir",
       "type": "string"
    },
    {
       "name": "fileName",
       "type": "string"
    }
  ]
}

Inbound channel:

{
  "key": "reportInboundChannel",
  "category": "channel",
  "name": "reportInboundChannel",
  "description": "Inbound channel",
  "channelType": "inbound",
  "type": "rabbit",
  "deserializerType": "json",
  "channelEventKeyDetection": {
    "fixedValue": "reportInboundEvent"
  },
  "queues": [
    "InboundQueue"
  ]
}

Inbound event:

{
    "key": "reportInboundEvent",
    "name": "reportInboundEvent",
    "inboundChannelKeys": [
        "reportInboundChannel"
    ],
    "correlationParameters": [
        {
            "name": "task",
            "type": "string"
        },
        {
            "name": "fileDir",
            "type": "string"
        },
        {
            "name": "fileName",
            "type": "string"
        }
    ],
    "payload": [
        {
            "name": "task",
            "type": "string"
        },
        {
            "name": "error",
            "type": "string"
        },
        {
            "name": "fileDir",
            "type": "string"
        },
        {
            "name": "fileName",
            "type": "string"
        }
    ]
}

In this process example I have two possible scenarios:

1. the report is successfully retrieved, therefore the right pointing sequence flow is chosen.

2. the report cannot be retrieved, therefore the downwards pointing sequence flow is chosen, after that the process sends a task for sending an email.

Scenario number 1 always works as intended, the EventRegsitry element recieves something like this:
{"task":"sftpTransfer","fileDir":"20210112_Pre-Arb\/validation","fileName":"new_prearb_arb_20210112.xlsx"}
from the Rabbit queue, inserts these variable and the process continues.

However, when there is an error while trying to retrieve the report something like this will be returned to the queue:
{"task":"sftpTransfer","fileDir":"20210112_Pre-Arb\/validation","fileName":"new_prearb_arb_20210112.xlsx","error":"java.lang.Exception: Can't find the file specified"}., in which case the process stops at the first EventRegistry element. No variables inserted. The only difference is that for the second scenario there is an “error” parameter in the JSON. I can’t find the source of the problem, but I do have other processes which use the same events and channels. No inbound channels that listen to the same rabbit queue, though; removing those fixed lots of the problems explained in the OP post. Since all the configurations seem to be correct to me I’ve thought that maybe having multiple processes using the same channel and event could be what causes this problem.

Could you take a look and tell me if you can find what could cause the problem ?