Adobe Experience Platform’s Streaming Sources and Destinations Overview and

4 years ago medium.com

Summary: This is a summary of an article originally published by the source. Read the full original article here →

Adobe recently announced two new Streaming Sources and Destinations as part of its real-time customer data platform offering: Azure Event Hub and AWS Kinesis. This blog will detail the use cases, how we started, what technical decisions we took, and the challenges we faced while executing this release. Adobe Experience Platform has a large Kafka infrastructure with multiple environments and data centers.

The options available from Kafka Connect were: “Failing fast to learn fast” is a great engineering principle and we can leverage some of Kafka Connect’s constructs to make sure that we fail fast if we are not able to instantiate any new connector.

You might need to spit some metrics out if you have done special handling for batching and error handling to track your latency and performance numbers as we did.

DevOps Articles