Confluent Cloud Bringing Apache Kafka as-a-Service (AWS) Confluent Inc. will be bringing its open-source-based streaming data platform to Amazon Web Services Inc.’s (AWS). The company was founded by the original Kafka creators. Confluent Cloud, the new streaming technology, will be available to all users in the AWS cloud today. AWS is the first public cloud hosting provider for the managed service. Microsoft Azure and Google Cloud support are planned for later. Apache Kafka, a publish/subscribe system that streams records, is a hot topic within Big Data analytics. This makes it similar to an enterprise messaging system or message queue, but its streaming capabilities make it ideal for setting up data pipelines to ingest records for Big Data analytics, as well as applications that store, process, or work with the data. RedMonk, a developer-focused research firm, posted yesterday that Kafka usage continues to grow at a rapid pace across multiple industry sectors. Kafka is rapidly becoming a key component of data pipelines at large scale. Confluent, the main commercial backer of the technology wants to make it easier to developers to use the cloud without having to worry about setting up and maintaining operations. Today’s blog post by Neha Narkhede, a company executive, explains why Kafka was moved to the public cloud as managed service. “It is clear that developers love Kafka. We want Kafka as our central nervous system, a streaming platform for all our microservices and a commit log to service database changeslogs, a base for stream processing, but not to operate Kafka,” she stated. “Why? Because managing distributed systems in the cloud is extremely difficult. We expect distributed systems to be available only as managed services for developers so that we can concentrate on building applications and not managing infrastructure or data services. Confluent initially released Kafka as an open-source project and offers it as a free download with its enterprise offering. However, the company claims that the AWS-hosted fully managed service has more benefits than the base technology. It includes enhanced integration capabilities, additional tools for managing Kafka clusters, security, and additional tools. Confluent Cloud’s key features are listed by the company as: – Hoffman Vending News

Access to the Kafka ecosystem. Confluent Cloud does not offer proprietary streaming services. Instead, developers have access to the same open-source Apache Kafka APIs. This makes it easy to leverage connectors, clients, and tools that are supported by the Kafka or Confluent communities.
Cloud option: A recent survey of more than 350 Apache Kafka users found that 52 percent use Kafka in the cloud. Nearly a third (32%) of respondents who use Kafka on the cloud have at least six applications using Kafka. Confluent Cloud is vendor-neutral, so developers can “lift” and shift Kafka infrastructure from any place into — or out — any cloud.
Confluent Cloud helps developers unleash their developer velocity. Confluent Cloud is a streaming data service that hosts Kafka. It takes away the operational burden so developers can focus on building streaming apps. Teams can concentrate on what is most important — building the company.
Reduce the ops burden Kafka is not an exception. Failures are more common as cluster usage increases. Confluent Cloud makes it easy to get Kafka running.

The company stated that developers can program in Java and Python when using Confluent clients. It won’t be the first Kafka-based offering to hit general availability on AWS, which is expected later in the year.