Channel definition - Configurable Kafka topic

Hi,
I am using a channel definition to configure an inbound topic:
Is there a way to have a variable/expression for the topic name so that it is configurable per environment? Can it be a property, a bean method etc…?
I am using Flowable 6.6.0 and Spring Boot 2.3
Thanks!

{
    "key": "new_doc_channel",
    "name": "New Document channel",
    "description": "New Document channel",
    "channelType": "inbound",
    "type": "kafka",
    "deserializerType": "json",
    "channelEventKeyDetection": {
        "fixedValue": "new_doc_event"
    },
    "topics": ["hardcoded_topic"]
}

This is a great question and something we ran into when deploying our application into different environments where the topic names varied per environment. After much digging through the source code, we found we could control the topic names by subclassing the KafkaChannelDefinitionProcessor class (in the org.flowable.eventregistry.spring.kafka package) and introducing our implementation as a Spring bean named kafkaChannelDefinitionProcessor. Overriding the method:
protected Collection<String> resolveTopics(KafkaInboundChannelModel channelDefinition)
allows the topic names in the channel definition to be replaced with other values at runtime.

Hey @mykeul and @jwashington,

Most of the values in the Channel definitions can use SPeL (Spring Expression Language) to resolve the values. There is no need to override the KafkaChannelDefinitionProcessor.

If you look at

You’ll see that it invokes resolveExpression for every single value in the topics.

resolveExpression uses the Spring value resolver.

So finally if you want to use a property in your topic you can define it like ${application.test.kafka-topic}.

You can see some examples in this commit.

Cheers,
Filip

Thanks @jwashington and @filiphr for your answers.

I used the property directly in the channel definition file and it works perfectly.

"topics": ["${kafka.topic}"]

Thank you very much @filiphr for your detailed answer.

Michel

By the way what’s the pros and cons of using channel/event definition files vs programmatically?

The use of SPel, as @filiphr rightfully pointed out, would probably fit most needs of replacing topic names through property settings and/or referencing Spring beans with either no or minimal programming. If for some reason SPel couldn’t meet the requirements, then creating your own KafkaChannelDefinitionProcessor would provide the most amount of control.

1 Like