Skip to Content
DocumentationEventsConfluent Cloud

Confluent Cloud

Confluent Cloud connections give you access to your company’s streaming event data in order to build analytics-ready tables.

Source Setup

You will need a valid API key and secret to set up a Confluent Cloud connection.

1. Authentication

To configure your Confluent Cloud connection, you will need information from your Confluent Cloud cluster. To find your cluster details, go to Environments → Select your environment → Select your cluster.

Bootstrap server

  1. With your cluster selected, on the left panel, click on Cluster Settings under Cluster Overview.
  2. In the General tab scroll down to Endpoints and copy the bootstrap server host and port.

API Key & API Secret

  1. Create a new API Key for your Etleap connection by going to API Keys under Cluster Overview on the left panel.
  2. Click on the + Add key button, choose Service account, create a new service account for Etleap.
  3. Add a consumer group ACL with the wildcard * for the consumer group ID, with pattern type as LITERAL and ALLOW for READ operations.
  4. Add a topic ACL with the topics you want to be able to ingest from (or use the wildcard * with pattern type as LITERAL for all topics) and ALLOW for READ operations.
  5. Copy the API Key and API Secret.

2. Schema Registry Setup

The Schema Registry is optional and should only be configured if you use the Confluent Schema Registry  to store your metadata. The following schema formats are supported: JSON, Protobuf, and Avro.

To obtain your Schema Registry information and credentials:

  1. Obtain the Schema Registry URL by going back to your environment in Environments → Your selected environment and copying the Endpoint URL located in Stream Governance API on the panel to the right.
  2. The Schema Registry API Key and Schema Registry API Secret can be created just below the Endpoint tab in the Credentials section.
Note

Prefer using an API? Go here  and select Confluent Cloud under the Body header to create a connection via API.

What Data is Available?

You can ingest data from any Confluent Cloud topic that you have defined on the server for this connection.

Note

Etleap uses Append mode for all pipelines.

Schema Formats

When using a Schema Registry, Etleap parses events according to the provided schema.

Given the following JSON schema :

{
  "type": "object",
  "properties": {
    "f1": {
      "type": "string"
    },
    "f2": {
      "type": "object",
      "properties": {
        "field": {
          "type": "integer"
        }
      }
    }
  }
}

Etleap produces the following relational structure:

topic VARCHAR,
key VARCHAR,
timestamp TIMESTAMP,
f1 VARCHAR,
f2_field: INTEGER
ZuoraEvent Stream