Vultr

Amazon MSK Connect Launched to Better Apache Kafka UX

Amazon follows up on its 2018 data streaming software, Amazon Managed Streaming for Apache Kafka, with a new offering optimized with managed connectors.

TL;DR

On 16 September 2021, AWS announced the launch of Amazon MSK Connect, an offering with extended compatibility to facilitate Kafka Connect efficiency.

It is automated to patch and manage the latent hardware, auto-scale connectors to match throughput changes.
It is automated to patch and manage the latent hardware, auto-scale connectors to match throughput changes.
Key Facts
  1. 1

    Amazon MSK Connect supports inter-platform migration with supported platforms.

  2. 2

    Amazon MSK Connect mitigates Kafka Connect setup processes with just a few clicks.

  3. 3

    It eliminates the burden of managing infrastructure, allowing users to focus on important events.

Details

Since its announcement at re: Invent 2018, the real-time data pipeline streaming platform has been optimized by many in the DevOps community.

Apache Kafka is a stream-processing framework that implements a software bus. It is a Scala and Java-based open source development platform developed by the Apache Software Foundation. It is oriented at providing a single, maximum-throughput, low-latency platform for real-time data flows. In 2018, Amazon launched an extension to facilitate real-time data streaming, Amazon Managed Streaming for Apache Kafka, which leverages Apache Kafka's capabilities to build and run applications and data pipelines with ease.

Kafka Connect is an Apache Kafka open source accessory that provides a framework for connecting to external systems such as databases, search indexes, file systems, and key-value stores. Manually running Kafka Connect clusters, on the other hand, necessitates users planning and provisioning the necessary infrastructure, managing cluster operations, and scaling the cluster in response to load changes.

Amazon's new offering is a game-changer in that instance. Amazon MSK Connect makes MSK Connect more soothing to operate. MSK Connect is an escape from the voluminous setup requirements associated with Kafka Connect. Users can now connect and deploy a connector with just a few clicks. It assumes the role of a virtual assistant, perennially monitoring the condition of the connectors to detect frailties that might cause any vacillation. It is automated to patch and manage the latent hardware, auto-scale connectors to match throughput changes. This way, users can implement this on large batches of data without the need for a heavy workforce or constant monitoring; therefore, they can focus on building applications.

MSK Connect and Kafka Connect are entirely compatible; customers may move their current connectors without changing any code. MSK Connect does not require an MSK cluster. It supports Amazon MSK, Apache Kafka, and Apache Kafka compatible clusters. As long as MSK Connect can privately connect to the clusters, these clusters can be self-managed or managed by AWS partners and third parties.


Get similar news in your inbox weekly, for free

Share this news:
Vultr

Latest stories


Why Your Business Should Connect Directly To Your Cloud

Today, companies make the most use of cloud technology regardless of their size and sector. …

7 Must-Watch DevSecOps Videos

Security is a crucial part of application development and DevSecOps makes it easy and continuous.The …

Top Ways To Optimize Your Cyber Security And Performance Monitoring Tools Using Load Balancing

In this post, you will learn how to optimize your cybersecurity and performance monitoring tools …

SRE: From Theory to Practice | What's difficult about on-call?

We launched the first episode of a webinar series to tackle one of the major …