Concurrent processing - aggregation

Hi,
I would like to implement a concurrent processing, then aggregate the result - like process images belonging to a pdf document.
I created a Kafka channel inbound. Images (or ref to an image) are retrieved from the topic.
I would like multiple workers (service tasks I guess), that process images from the topic in parallel.
Once all the images have been processed for a given pdf_id, I would like to aggregate the results of each processed image.
I thought of using parallel gateway but then it is a fixed number of workers, and let’s say a pdf contains only 2 images (and there are 3 workers), then join will not occur.
Is there a construct that better matches this need?

Thanks,
Michel

Any ideas how to implement such concurrent processing in Flowable ?
Thank you.

Hey @mykeul,

This feels like you need to use a Multi Instance Parallel execution. Every single task in Flowable can be marked as multi instance.

Cheers,
Filip

Thanks @filiphr
I will have a look at this.