Event Registry - Kafka - Avro

Kind of late for this answer, but I’ll add my experience anyway, in case someone who comes across this post finds it useful.

I am a Flowable newbie and was struggling to have Flowable consume messages for the corporate Kafka, where all the messages are serialised by Avro. Funnily enough, our messages _before they are treated by Avro, are actually JSON’s. In other words, what we do is to serialise an object to JSON, then give it to Avro to do its magic, and finally send it to the Kafka topic.

The problem I was having with Flowable when following the solution in this thread is that even if I created a custom deserialiser by implementing InboundEventDeserializer, the raw messages received as parameter in the deserialize() method was an unreadable string. To be exact, the string that resulted when Avro serialised the original JSON message. I tried to deserialise that string myself with no luck.

It was then when I realised that the Avro deserialisation and the Flowable deserialisation are totally different things. The method InboundEventDeserializer.deserialise() should received the raw messages, once it has been deserialised by Avro! From there, you can implement your own magic. However, I didn’t know that and I was trying to reinvent the wheel by having my custom deserialiser work like the actual Avro deserialiser.

The final solution in my case was:

  1. Make sure that the Avro deserialiser is properly configured. This is easy as we are using Spring boot. A couple of configuration parameters do the trick (there is more than enough information about how to do this out there)

  2. Given that our original messages are JSON already, that’s it, I don’t need to add my custom deserialiser. It’s enough with using json deserialiser.

In summary, not sure if this was the problem of the person who created this thread originally, but it might be that people reading this post, even years later, are in the same situation that I am. If that’s the case, I hope this helps.

Cheers,
Pedro.

1 Like