Connect Using kaf CLI

kaf is a modern CLI for Apache Kafka. You can connect to your Upstash Kafka cluster using kaf.

If you do not have a Kafka cluster and/or topic already, follow these steps to create one.

In the cluster details section of the Upstash Console copy bootstrap endpoint, username and password. Then replace following parameters in the code snippets below with the actual values you copied earlier.

• $BOOTSTRAP_ENDPOINT
• $UPSTASH_KAFKA_USERNAME
• $UPSTASH_KAFKA_PASSWORD
• $GROUP_ID
• $TOPIC_NAME

Initially we should add cluster configuration to kaf’s config file, which should be located in ~/.kaf/config. Open config file if it exists or create an empty one and insert following config:

clusters:
  - name: $CLUSTER_NAME
    brokers:
      - $BOOTSTRAP_ENDPOINT
    SASL:
      mechanism: SCRAM-SHA-512
      username: $UPSTASH_KAFKA_USERNAME
      password: $UPSTASH_KAFKA_PASSWORD
    security-protocol: SASL_SSL

$CLUSTER_NAME is a logical name, which is used to identify different Kafka cluster. You can use your Upstash cluster name.

To select the cluster configuration to use, run:

kaf config use-cluster $CLUSTER_NAME

At this point you should be able to connect to your Kafka cluster using kaf.

List Brokers and Topics:

kaf nodes

kaf topics

Produce a message:

echo "Hello Upstash!" | kaf produce $TOPIC_NAME

Fetch messages:

kaf consume $TOPIC_NAME

Consume messages using consumer groups:

kaf consume $TOPIC_NAME -g $GROUP_ID --offset oldest

For more information see kaf repository.