Producer API

Producer API is used to send one or more messages to the same or multiple Kafka topics.

There are three variants of the Producer API:

1. GET /produce/$TOPIC/$MESSAGE?key=$KEY:

   Sends a single message ($MESSAGE) to a topic ($TOPIC) using HTTP GET. Optionally message key can be appended with a query parameter?key=$KEY.

   Without message key:

   curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/greetings/hello_kafka -u myuser:mypass
   

   With a message key:

   curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/cities/Istanbul?key=city -u myuser:mypass
   

   Response:

   A single metadata response is returned as JSON. Its structure is:

   Metadata{
       topic: String,
       partition: Int,
       offset: Long,
       timestamp: Long
   }
   

2. [GET | POST] /produce/$TOPIC:

   Produces one or more messages to a single topic ($TOPIC). Messages are sent using request body as JSON. Structure of the message JSON is:

   Header {key: String, value: String}
   
   Message{
       partition?: Int,
       timestamp?: Long,
       key?: String,
       value: String,
       headers?: Array<Header>
   }
   

   Only value field is required. partition, timestamp, key and headers fields are optional. When left blank, partition and timestamp fields will be assigned by Kafka brokers. It’s valid to send a single message or array of messages as JSON.

   • Single message with only value:

   curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/greetings -u myuser:mypass \
       -d '{"value": "hello_kafka"}'
   

   • Single message with multiple attributes:

   curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/cities -u myuser:mypass \
       -d '{"partition": 1, "key": "city", "value": "Istanbul", "headers": [{"key": "expire", "value": "1637745834756"}] }'
   

   • Multiple messages with only values:

   curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/greetings -u myuser:mypass \
       -d '[
           {"value": "hello_world"},
           {"value": "hello_upstash"},
           {"value": "hello_kafka"}
       ]'
   

   • Multiple messages with attributes:

   curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/cities -u myuser:mypass \
       -d '[
           {"partition": 1, "timestamp": 1637745834744, "key": "city", "value": "Istanbul"},
           {"partition": 2, "timestamp": 1637745823147, "key": "city", "value": "London"},
           {"partition": 3, "timestamp": 1637455583829, "key": "city", "value": "Tokyo"},
       ]'
   

   Response:

   A metadata array response is returned as JSON. Its structure is same as above. Each metadata in the response array belongs the message with the same order in the request.

3. [GET | POST] /produce:

   Produces one or more messages to a single or multiple topics. Messages are sent using request body as JSON. Structure of the message JSON is the same as the above, there’s only an additional topic field:

   Message{
       topic: String,
       partition?: Int,
       timestamp?: Long,
       key?: String,
       value: String,
       headers?: Array<Header>
   }
   

   Only topic and value fields are required. partition, timestamp, key and headers fields are optional. When left blank, partition and timestamp fields will be assigned by Kafka brokers. It’s valid to send a single message or array of messages as JSON.

   • Single message:

   curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce -u myuser:mypass \
       -d '{"topic": "greetings", "value": "hello_kafka"}'
   

   • Multiple messages for a single topic:

   curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce -u myuser:mypass \
       -d '[
           {"topic": "greetings", "value": "hello_world"},
           {"topic": "greetings", "value": "hello_upstash"},
           {"topic": "greetings", "value": "hello_kafka"}
       ]'
   

   • Multiple messages to multiple topics:

   curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce -u myuser:mypass \
       -d '[
           {"topic": "greetings", "value": "hello_world"},
           {"topic": "greetings", "value": "hello_upstash"},
           {"topic": "cities", "partition": 1, "value": "Istanbul"},
           {"topic": "cities", "partition": 2, "value": "London"},
           {"topic": "cities", "partition": 3, "value": "Tokyo"},
       ]'
   

   Response:

   A metadata array response is returned as JSON. Its structure is same as above. Each metadata in the response array belongs the message with the same order in the request.

Producer API works with acks=all producer configuration, which means the topic partition’s leader will wait for the all in-sync replicas to acknowledge the produced messages.