Using Apache Kafka with Flow Director

Integration with Ease.

Apache Kafka is an open-source stream-processing software platform developed by LinkedIn and donated to the Apache Software Foundation, written in Scala and Java. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. It can be easily integrated into Flow Director and with any other messaging system.

Get Flow Director

This example shows how to receive page view messages from a Kafka topic and create a history of it that can be shown in a Flow Director dashboard chart. A Kafka consumer provides a synchronous API that needs to be polled. It then returns all messages received within this poll interval. The above flow starts with a timer that calls a Kafka consumer on a PageView topic to receive messages. The messages contain a JSON in byte format so every message go through 2 steps: convert the bytes to a text message and then extract the JSON fields. The result is then passed to a history component which automatically creates histories on a minute, hour, day, month and year basis. These histories can then be directly connected to a group chart in Flow Director to see the page views in realtime. Flow Director can integrate Kafka consumer and producer in any combination. Once integrated it can be further connected to any other messaging system.

Share this page:
Data Privacy

Copyright ©2023 Edge Broker GmbH. All rights reserved.

Some icons by Icons8.