Event registry kafka late events processing

[joram]

@danielmkraus

I think that we need to implement a custom logic either a custom InboundEventProcessingPipeline which keeps a state of a window of the data read from the topic and discards late events,

Yes, that would indeed the way to go.
A custom pipeline (reusing most of the default one). that applies this filtering for you.

The tricky thing is what to do when a server goes down, when the events are still in this ‘window’. These could potentially be lost.

@danielmkraus

other possibility that I though is to add a condition on intermediary catching event to process only if the incoming event has the timestamp greater than the previous one, is possible to ignore the incoming event via execution listener or other mechanism?

That would be possible (solving it in the model), but will get messy quickly.

@danielmkraus

do you have any other ideas to solve this issue?

Is it imperative this is solved on the receiving side? Isn’t there a way this could be solved before the events are passed to Flowable?