kafka-pyrallel-consumer

Parallel Consumer for Kafka, based on Python's confluent_kafka lib.

It also has the capability to deduplicate messages within the topic partitions it is currently consuming from. It currently supports local (in-memory) or distributed (Redis) LRU caching mechanism, but you are free to create your own LRU cache backend (see dedupBackendClass below for details).

Requirements

• Docker
• Python 3.8+
• Install python requirements (python3 -m pip install -r requirements.txt)

Pending

• Create python package

What is it?

This a wrapper around the Python Consumer class (confluent_kafka Python lib), called PyrallelConsumer.

This wrapper class provides a way to process Kafka messages in parallel within the same consumer, improving efficiency and speed. It enables fine-grained control over parallelism beyond the default partition-level in Kafka, allowing key-level and message-level parallelism. This helps in handling fixed partition counts, integrating with slow databases or services (I/O bounded), and managing queue-like message processing more effectively. The class is designed to optimise performance without requiring extensive changes to your existing Kafka consumer.

To learn more about Parallel Consumers, check out the resources below (Java language):

• Introducing the Confluent Parallel Consumer
• How to use the Confluent Parallel Consumer
• Confluent's GitHub Repository for the Java Parallel Consumer

How the wrapper works?

This class functions similarly to the standard Consumer class but with additional optional parameters:

• max_concurrency (int): Specifies the number of concurrent threads for handling consumed messages. The default is 3.
• ordering (bool): When True (default), it hashes the message key, divides it, and assigns it to the corresponding queue/thread to maintain message order. If False, it randomly allocates the first message to a queue/thread and uses round-robin allocation for subsequent keys.
• record_handler (function): A function to process messages within each thread, taking a single parameter msg as returned from a consumer.poll call.
• max_queue_backlog (int): Max number of unprocessed items in the queue(s), if that number is reached the polling will be automatically paused and wait for the queue to be cleared. The default is 1024.
• dedupClass: Deduplicate messages. Instance of class to do the message deduplication. You can set any class instance here, however it must have at least one method called is_message_duplicate where its only argument is the Kafka polled message object. See example on the class DedupLRU (kafka_pyrallel_consumer/dedup.py) where it will use an in-memory LRU cache. This parameter is optional and if not set will not dedup any message. The arguments taken by DedupLRU, are:
  • dedup_by_key (bool): Deduplicate messages by the Key. The default is False. To deduplicate messages by Key and Value, set both dedup_by_key and dedup_by_value as True.
  • dedup_by_value (bool): Deduplicate messages by the Value. The default is False. To deduplicate messages by Key and Value, set both dedup_by_key and dedup_by_value as True.
  • dedup_max_lru (int): Max LRU cache size. The default is 32768.
  • dedup_algorithm (str): Deduplication hashing algorithm to use. Options available are: md5, sha1, sha224, sha256 (default), sha384, sha512, sha3_224, sha3_256, sha3_384, sha3_512. To reduce memory footprint, the cached dedup will be a hash of the Key/Value other than the actual data.
  • dedupBackendClass: Deduplication LRU cache backend class. Instance of class where the LRU cache backend is implemented. You can set any class instance here, however it must be an abstract class of LRUCacheBase. By default it will implement an in-memory local LRU cache (LocalLRUCache on kafka_pyrallel_consumer/lru_cache.py), please note that by having it in local memory it will not work properly in case of consumer rebalance as there will be no cached dedup shared between consumers within the consumer group. To have a more robust and shared LRU caching mechanism you can use RedisLRUCache (also on kafka_pyrallel_consumer/lru_cache.py), or feel free to create your own backend.

Example: test_parallel_consumer.py

Check the example on test_parallel_consumer.py, it imports the wrapper consumer library:

from kafka_pyrallel_consumer import PyrallelConsumer

Then when instantiating the consumer using the wrapper library it passes the consumer configuration (as in the standard Consumer class), but also the additional arguments:

consumer = PyrallelConsumer(
  consumer_config,
  ordering=True,
  max_concurrency=5,
  record_handler=record_handler.postmanEcho,
  max_queue_backlog=16,
  # dedupClass=DedupLRU(
  #     dedup_by_key=True,
  #     dedup_by_value=True,
  #     dedup_max_lru=1024,
  #     dedup_algorithm="sha256",
  #     dedupBackendClass=RedisLRUCache(
  #         host="localhost",
  #         port=6379,
  #         redis_key_name=f"pyrallel_consumer_lru_cache_{''.join(filter(str.isalnum, args.group_id))}",
  #     ),
  # ),
)

Where record_handler.postmanEcho is the function to handle the consumed messages in parallel. The wrapper consumer class will handle the processing in up to 5 separate threads (as, in this example, max_concurrency was set to 5).

Using the wrapper class, all the consumer needs to do is to poll Kafka. The queue allocation will be handled by the wrapper and the processing of messages, on its corresponding thread, will be hanlded by the record_handler function, as set by the user.

consumer.poll(timeout=0.25)

At last, upon calling the consumer method .close() as graceful_shutdown is set as True, it will wait all thread queue(s) to be processed. As additional queued items can be processed the argument commit_before_closing is set as True so a final commit can be issued before closing the consumer.

consumer.close(
  graceful_shutdown=True,
  commit_before_closing=True,
  commit_asynchronous=False,
)

Commit Strategy

The commit strategy is up to the user to have it defined as the wrapper consumer will not automatically handle that. However with the parallel consumer, upon calling the commit class method if the argument asynchronous is set as False, it will wait all queue(s) to be empty (forcing the poll to be paused), after than it will issue the commit and only then resume the poll. The commit method has also an argument called pause_poll (default False) where it will pause the poll and resumed only after the commit is issued, however that is only applicable if asynchronous is set as False.

If you want to implement your own commit strategy, make sure to set enable.auto.commit as False on your consumer.

The example test_parallel_consumer.py has implemented a synchronous commit strategy pausing the polling.

Output: test_parallel_consumer.py

Before running the examples below, make sure to have Docker up and running, then run docker-compose up -d so it can spin a minimal Confluent Platform Kafka cluster up.

The test_parallel_consumer.py python script will run a parallel consumer over the Kafka topic demo_parallel_consumer, here is an example of a message:

Key   = 09
Value = {
  "payload": "cad4f3486f2d41e895945afbaf47f9ac",
  "timestamp": 1719246245722
}

The consumer will then submit a POST request as follows:

POST https://postman-echo.com/post?key=09
Body as per message Value

To visualise the messages on that topic please navigate to Confluent Control Center at http://localhost:9021/clusters.

Running with one single thread, with ordering and processing 50 messages. Each message will be posted to Postman echo. It took around 18 seconds:

• First message: 16:33:26.414
• Last message: 16:33:44.329

% python3 avro_producer.py --iterations 50
2024-06-23 16:33:17.237 [INFO]: Progress: 10%
2024-06-23 16:33:17.237 [INFO]: Progress: 20%
2024-06-23 16:33:17.237 [INFO]: Progress: 30%
2024-06-23 16:33:17.237 [INFO]: Progress: 40%
2024-06-23 16:33:17.237 [INFO]: Progress: 50%
2024-06-23 16:33:17.238 [INFO]: Progress: 60%
2024-06-23 16:33:17.238 [INFO]: Progress: 70%
2024-06-23 16:33:17.238 [INFO]: Progress: 80%
2024-06-23 16:33:17.238 [INFO]: Progress: 90%
2024-06-23 16:33:17.238 [INFO]: Progress: 100%

% python3 test_parallel_consumer.py
2024-06-23 16:33:20.118 [INFO]: Starting parallel consumer thread: 0
2024-06-23 16:33:20.124 [INFO]: Started consumer avro-deserialiser-01 (avro-deserialiser) on topic 'demo_parallel_consumer'
2024-06-23 16:33:26.414 [INFO]: {'args': {}, 'data': {'payload': '96721478bdb640ebb321d311c7a49963', 'timestamp': 1719156804985}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156806.383', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784046-6b87239125e8ae3806265ba5', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '96721478bdb640ebb321d311c7a49963', 'timestamp': 1719156804985}, 'url': 'https://postman-echo.com/post?key=69'}
2024-06-23 16:33:26.768 [INFO]: {'args': {}, 'data': {'payload': '38050ed5748c416bba74da7972f853cd', 'timestamp': 1719156804998}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156806.741', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784046-4e8ab3da6b231cd122c10c6d', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '38050ed5748c416bba74da7972f853cd', 'timestamp': 1719156804998}, 'url': 'https://postman-echo.com/post?key=24'}
2024-06-23 16:33:27.120 [INFO]: {'args': {}, 'data': {'payload': '724d04925e11408b97d57cfe0fda7288', 'timestamp': 1719156804999}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156807.091', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784047-0b8d2ab673eae97e1b41c4bc', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '724d04925e11408b97d57cfe0fda7288', 'timestamp': 1719156804999}, 'url': 'https://postman-echo.com/post?key=70'}
2024-06-23 16:33:27.481 [INFO]: {'args': {}, 'data': {'payload': '39c1ec34fc5148d0844f36e68c1a68a4', 'timestamp': 1719156805001}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156807.450', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784047-40578e12336370567622e0ba', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '39c1ec34fc5148d0844f36e68c1a68a4', 'timestamp': 1719156805001}, 'url': 'https://postman-echo.com/post?key=02'}
2024-06-23 16:33:27.838 [INFO]: {'args': {}, 'data': {'payload': '334fb8982a7441f68218c84056f7689e', 'timestamp': 1719156805002}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156807.810', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784047-664dcab2540bce271bb3c325', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '334fb8982a7441f68218c84056f7689e', 'timestamp': 1719156805002}, 'url': 'https://postman-echo.com/post?key=14'}
2024-06-23 16:33:28.213 [INFO]: {'args': {}, 'data': {'payload': 'c319267d5e3d485fbf0eab59e6c5d062', 'timestamp': 1719156805003}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156808.185', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784048-4c16d69345d7af84289b218e', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'c319267d5e3d485fbf0eab59e6c5d062', 'timestamp': 1719156805003}, 'url': 'https://postman-echo.com/post?key=85'}
2024-06-23 16:33:28.572 [INFO]: {'args': {}, 'data': {'payload': '740e2eb3535947c79ac6081b9c611b68', 'timestamp': 1719156805005}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156808.544', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784048-2907a5d232b3fd8523973942', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '740e2eb3535947c79ac6081b9c611b68', 'timestamp': 1719156805005}, 'url': 'https://postman-echo.com/post?key=78'}
2024-06-23 16:33:28.923 [INFO]: {'args': {}, 'data': {'payload': '23d61fb39c404f398a1f1c3956a31aac', 'timestamp': 1719156805006}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156808.896', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784048-29baa320432e77ca7ba7e1d4', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '23d61fb39c404f398a1f1c3956a31aac', 'timestamp': 1719156805006}, 'url': 'https://postman-echo.com/post?key=52'}
2024-06-23 16:33:29.280 [INFO]: {'args': {}, 'data': {'payload': '5ec1716543754e6eae65150b65a5e3e4', 'timestamp': 1719156805007}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156809.252', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784049-640982411ee0e0045dad74e3', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '5ec1716543754e6eae65150b65a5e3e4', 'timestamp': 1719156805007}, 'url': 'https://postman-echo.com/post?key=16'}
2024-06-23 16:33:29.632 [INFO]: {'args': {}, 'data': {'payload': '0afe0bbab39b471cab22e02b1b2ca5ee', 'timestamp': 1719156805008}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156809.600', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784049-2b1dcd364df931b57c920b1c', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '0afe0bbab39b471cab22e02b1b2ca5ee', 'timestamp': 1719156805008}, 'url': 'https://postman-echo.com/post?key=12'}
2024-06-23 16:33:29.986 [INFO]: {'args': {}, 'data': {'payload': '0d723af3dae74cea9bbbd0a4887fa6d1', 'timestamp': 1719156805010}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156809.957', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784049-008d5e7b207d6d4b558fd9dd', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '0d723af3dae74cea9bbbd0a4887fa6d1', 'timestamp': 1719156805010}, 'url': 'https://postman-echo.com/post?key=81'}
2024-06-23 16:33:30.387 [INFO]: {'args': {}, 'data': {'payload': '6f67bffc83fe42ac879f655f07100abc', 'timestamp': 1719156805011}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156810.313', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404a-120c23ba26d8cba85be48b2d', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '6f67bffc83fe42ac879f655f07100abc', 'timestamp': 1719156805011}, 'url': 'https://postman-echo.com/post?key=26'}
2024-06-23 16:33:30.741 [INFO]: {'args': {}, 'data': {'payload': 'f36b3612b6b14f14bc812234cf24687f', 'timestamp': 1719156805012}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156810.712', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404a-21a700d7171cf7f014932156', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'f36b3612b6b14f14bc812234cf24687f', 'timestamp': 1719156805012}, 'url': 'https://postman-echo.com/post?key=50'}
2024-06-23 16:33:31.092 [INFO]: {'args': {}, 'data': {'payload': '03438ecde4e44750a1d76e831bc9ccd6', 'timestamp': 1719156805013}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156811.065', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404b-047596173463d53873c1c9a4', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '03438ecde4e44750a1d76e831bc9ccd6', 'timestamp': 1719156805013}, 'url': 'https://postman-echo.com/post?key=90'}
2024-06-23 16:33:31.443 [INFO]: {'args': {}, 'data': {'payload': 'ffc9419902f047a6b8d8d142fd50219a', 'timestamp': 1719156805015}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156811.415', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404b-04702f4562914d9c7341c657', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'ffc9419902f047a6b8d8d142fd50219a', 'timestamp': 1719156805015}, 'url': 'https://postman-echo.com/post?key=04'}
2024-06-23 16:33:31.801 [INFO]: {'args': {}, 'data': {'payload': '158ccdcf182d47659a0e2f1ddf97baa8', 'timestamp': 1719156805016}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156811.766', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404b-356026fd1646c50d1a6d0624', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '158ccdcf182d47659a0e2f1ddf97baa8', 'timestamp': 1719156805016}, 'url': 'https://postman-echo.com/post?key=13'}
2024-06-23 16:33:32.157 [INFO]: {'args': {}, 'data': {'payload': '1cd7cc2677a6407bb3089394645c2a60', 'timestamp': 1719156805017}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156812.127', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404c-3ae5f24b3495392a5dfdc3eb', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '1cd7cc2677a6407bb3089394645c2a60', 'timestamp': 1719156805017}, 'url': 'https://postman-echo.com/post?key=47'}
2024-06-23 16:33:32.509 [INFO]: {'args': {}, 'data': {'payload': 'ee3b805278ff4d95a3ab040e66735135', 'timestamp': 1719156805018}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156812.479', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404c-079cb8b24595995b6d6d8f5c', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'ee3b805278ff4d95a3ab040e66735135', 'timestamp': 1719156805018}, 'url': 'https://postman-echo.com/post?key=30'}
2024-06-23 16:33:32.865 [INFO]: {'args': {}, 'data': {'payload': '2c5ab722c1bc463680423a600e9e3928', 'timestamp': 1719156805020}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156812.832', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404c-254e080a4fe92a08779c232a', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '2c5ab722c1bc463680423a600e9e3928', 'timestamp': 1719156805020}, 'url': 'https://postman-echo.com/post?key=03'}
2024-06-23 16:33:33.221 [INFO]: {'args': {}, 'data': {'payload': '769638e670d148faa770b668d9f8c504', 'timestamp': 1719156805021}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156813.191', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404d-053ed5f6166bfe7160ce4996', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '769638e670d148faa770b668d9f8c504', 'timestamp': 1719156805021}, 'url': 'https://postman-echo.com/post?key=18'}
2024-06-23 16:33:33.570 [INFO]: {'args': {}, 'data': {'payload': '72219d5d710746c786e3a1c65ea49561', 'timestamp': 1719156805022}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156813.544', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404d-1dbee3220cdce63606463efa', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '72219d5d710746c786e3a1c65ea49561', 'timestamp': 1719156805022}, 'url': 'https://postman-echo.com/post?key=87'}
2024-06-23 16:33:33.927 [INFO]: {'args': {}, 'data': {'payload': '906d295afaeb477194b20ced8c26302d', 'timestamp': 1719156805024}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156813.897', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404d-33269fbf4c1fd63d56a53a6a', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '906d295afaeb477194b20ced8c26302d', 'timestamp': 1719156805024}, 'url': 'https://postman-echo.com/post?key=23'}
2024-06-23 16:33:34.282 [INFO]: {'args': {}, 'data': {'payload': '3797889e684f49d493f7c47ac5ab8e8b', 'timestamp': 1719156805025}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156814.255', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404e-4162c09a7cd27b76483931b0', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '3797889e684f49d493f7c47ac5ab8e8b', 'timestamp': 1719156805025}, 'url': 'https://postman-echo.com/post?key=62'}
2024-06-23 16:33:34.629 [INFO]: {'args': {}, 'data': {'payload': '3946ba86da494098999e4c7d6d5baf15', 'timestamp': 1719156805026}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156814.599', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404e-4a344af861e5ea55555fdb56', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '3946ba86da494098999e4c7d6d5baf15', 'timestamp': 1719156805026}, 'url': 'https://postman-echo.com/post?key=79'}
2024-06-23 16:33:34.980 [INFO]: {'args': {}, 'data': {'payload': '2a3a039dd07741a58ef9c1d33d6d2c49', 'timestamp': 1719156805027}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156814.950', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404e-29063f6d3476108a1adec090', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '2a3a039dd07741a58ef9c1d33d6d2c49', 'timestamp': 1719156805027}, 'url': 'https://postman-echo.com/post?key=74'}
2024-06-23 16:33:35.328 [INFO]: {'args': {}, 'data': {'payload': '37ab4b5110be446a9fa566d3d3de95ba', 'timestamp': 1719156805029}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156815.302', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404f-711bf8340606dcff3a026204', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '37ab4b5110be446a9fa566d3d3de95ba', 'timestamp': 1719156805029}, 'url': 'https://postman-echo.com/post?key=30'}
2024-06-23 16:33:35.676 [INFO]: {'args': {}, 'data': {'payload': 'db331c5e2c284f5783ecf67e6560e6bc', 'timestamp': 1719156805030}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156815.649', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404f-1b144372444a938339df9002', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'db331c5e2c284f5783ecf67e6560e6bc', 'timestamp': 1719156805030}, 'url': 'https://postman-echo.com/post?key=05'}
2024-06-23 16:33:36.029 [INFO]: {'args': {}, 'data': {'payload': 'ce9982592f06411f85c69e88831cc5c0', 'timestamp': 1719156805031}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156815.998', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678404f-5cbf17e03d179a04082aeb1e', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'ce9982592f06411f85c69e88831cc5c0', 'timestamp': 1719156805031}, 'url': 'https://postman-echo.com/post?key=83'}
2024-06-23 16:33:36.380 [INFO]: {'args': {}, 'data': {'payload': '352e8097fed0447fbc9d53540d6b1e0a', 'timestamp': 1719156805032}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156816.354', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784050-07fc33d7561b8ed30b0895ed', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '352e8097fed0447fbc9d53540d6b1e0a', 'timestamp': 1719156805032}, 'url': 'https://postman-echo.com/post?key=72'}
2024-06-23 16:33:36.735 [INFO]: {'args': {}, 'data': {'payload': 'da6900bd164c4269a3d777a389316575', 'timestamp': 1719156805034}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156816.707', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784050-5c05413937d207fc1b9686cd', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'da6900bd164c4269a3d777a389316575', 'timestamp': 1719156805034}, 'url': 'https://postman-echo.com/post?key=25'}
2024-06-23 16:33:37.200 [INFO]: {'args': {}, 'data': {'payload': 'bd3ba076b85b43e8a5aca58f9b566004', 'timestamp': 1719156805035}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156817.056', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784051-0f02095866a35b3460b20076', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'bd3ba076b85b43e8a5aca58f9b566004', 'timestamp': 1719156805035}, 'url': 'https://postman-echo.com/post?key=30'}
2024-06-23 16:33:37.735 [INFO]: {'args': {}, 'data': {'payload': 'bf49b66276344897826faee6991501e4', 'timestamp': 1719156805036}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156817.517', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784051-5bceed30647dca8538440729', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'bf49b66276344897826faee6991501e4', 'timestamp': 1719156805036}, 'url': 'https://postman-echo.com/post?key=39'}
2024-06-23 16:33:38.084 [INFO]: {'args': {}, 'data': {'payload': '503c2f199ea943c2b2247ed078a36a9c', 'timestamp': 1719156805037}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156818.056', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784052-5f2efc3e2cde40343affee30', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '503c2f199ea943c2b2247ed078a36a9c', 'timestamp': 1719156805037}, 'url': 'https://postman-echo.com/post?key=05'}
2024-06-23 16:33:38.448 [INFO]: {'args': {}, 'data': {'payload': '4380d52c438646988aa71d787bf5d1f9', 'timestamp': 1719156805039}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156818.411', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784052-12f5d58860f26870005b52da', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '4380d52c438646988aa71d787bf5d1f9', 'timestamp': 1719156805039}, 'url': 'https://postman-echo.com/post?key=96'}
2024-06-23 16:33:38.796 [INFO]: {'args': {}, 'data': {'payload': 'c7154e6f2e83498fa7363c9407fcd619', 'timestamp': 1719156805040}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156818.768', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784052-277206984900cf2b5904503f', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'c7154e6f2e83498fa7363c9407fcd619', 'timestamp': 1719156805040}, 'url': 'https://postman-echo.com/post?key=81'}
2024-06-23 16:33:39.145 [INFO]: {'args': {}, 'data': {'payload': '4cbea85c762b4c33ac30b3c77b1db40b', 'timestamp': 1719156805041}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156819.118', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784053-1564e5da6dafcecd798702f3', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '4cbea85c762b4c33ac30b3c77b1db40b', 'timestamp': 1719156805041}, 'url': 'https://postman-echo.com/post?key=76'}
2024-06-23 16:33:39.536 [INFO]: {'args': {}, 'data': {'payload': 'cae779a73e604fb5acc00541d445b5d7', 'timestamp': 1719156805043}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156819.467', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784053-7b03e6a8521fa5712fa979ed', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'cae779a73e604fb5acc00541d445b5d7', 'timestamp': 1719156805043}, 'url': 'https://postman-echo.com/post?key=73'}
2024-06-23 16:33:39.909 [INFO]: {'args': {}, 'data': {'payload': '78542ee5ff2e48898c1c976d534b0e35', 'timestamp': 1719156805044}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156819.860', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784053-047c148e0651cf42709cb1fd', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '78542ee5ff2e48898c1c976d534b0e35', 'timestamp': 1719156805044}, 'url': 'https://postman-echo.com/post?key=06'}
2024-06-23 16:33:40.262 [INFO]: {'args': {}, 'data': {'payload': 'da9aeb426ccb40888eb51589962dcee5', 'timestamp': 1719156805045}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156820.234', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784054-1135ac4a505625fb52c417ec', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'da9aeb426ccb40888eb51589962dcee5', 'timestamp': 1719156805045}, 'url': 'https://postman-echo.com/post?key=84'}
2024-06-23 16:33:40.615 [INFO]: {'args': {}, 'data': {'payload': '6aed7629c04a484e9ef4fb7ceb95071f', 'timestamp': 1719156805046}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156820.585', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784054-055929e11605c59b54eb8117', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '6aed7629c04a484e9ef4fb7ceb95071f', 'timestamp': 1719156805046}, 'url': 'https://postman-echo.com/post?key=99'}
2024-06-23 16:33:41.013 [INFO]: {'args': {}, 'data': {'payload': 'e255787dcf644310a20cb3b0ea1b8cc3', 'timestamp': 1719156805048}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156820.941', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784054-5fb2286a74ccff0e1dde0ada', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'e255787dcf644310a20cb3b0ea1b8cc3', 'timestamp': 1719156805048}, 'url': 'https://postman-echo.com/post?key=85'}
2024-06-23 16:33:41.369 [INFO]: {'args': {}, 'data': {'payload': '62037f62dd134902a4673322efe65981', 'timestamp': 1719156805049}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156821.342', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784055-378d4742468e4fbf79ba121a', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '62037f62dd134902a4673322efe65981', 'timestamp': 1719156805049}, 'url': 'https://postman-echo.com/post?key=20'}
2024-06-23 16:33:41.764 [INFO]: {'args': {}, 'data': {'payload': '3bc66b95a2d442b4bb9e129b204bb6e4', 'timestamp': 1719156805050}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156821.691', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784055-60778fa433eb1a531c83b74c', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '3bc66b95a2d442b4bb9e129b204bb6e4', 'timestamp': 1719156805050}, 'url': 'https://postman-echo.com/post?key=44'}
2024-06-23 16:33:42.122 [INFO]: {'args': {}, 'data': {'payload': 'd086383687a641a7945bef14009becab', 'timestamp': 1719156805051}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156822.086', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784056-78d455df4af7da7365a2f86e', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'd086383687a641a7945bef14009becab', 'timestamp': 1719156805051}, 'url': 'https://postman-echo.com/post?key=84'}
2024-06-23 16:33:42.488 [INFO]: {'args': {}, 'data': {'payload': 'af450e172ff246c1a3b3306584c24f20', 'timestamp': 1719156805053}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156822.453', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784056-0050a5581fadd36d1d89de0a', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'af450e172ff246c1a3b3306584c24f20', 'timestamp': 1719156805053}, 'url': 'https://postman-echo.com/post?key=98'}
2024-06-23 16:33:42.890 [INFO]: {'args': {}, 'data': {'payload': '2357091178b742b997f90ca5b970a725', 'timestamp': 1719156805054}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156822.816', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784056-458bcbcd6512569c0305cc70', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '2357091178b742b997f90ca5b970a725', 'timestamp': 1719156805054}, 'url': 'https://postman-echo.com/post?key=43'}
2024-06-23 16:33:43.265 [INFO]: {'args': {}, 'data': {'payload': '0d8f182971c54f25ad809c57b058ace7', 'timestamp': 1719156805055}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156823.237', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784057-17939e4c54f3ad5b190ec7f6', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '0d8f182971c54f25ad809c57b058ace7', 'timestamp': 1719156805055}, 'url': 'https://postman-echo.com/post?key=27'}
2024-06-23 16:33:43.620 [INFO]: {'args': {}, 'data': {'payload': '9ed89bbcede74f2caaba0de0ea73d2df', 'timestamp': 1719156805056}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156823.588', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784057-2399c33b5531354000b97347', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '9ed89bbcede74f2caaba0de0ea73d2df', 'timestamp': 1719156805056}, 'url': 'https://postman-echo.com/post?key=57'}
2024-06-23 16:33:43.977 [INFO]: {'args': {}, 'data': {'payload': '7f4e9d57074148ba906d2b77766681c8', 'timestamp': 1719156805058}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156823.950', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784057-2a9e4f4d260efa1e29be62b7', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '7f4e9d57074148ba906d2b77766681c8', 'timestamp': 1719156805058}, 'url': 'https://postman-echo.com/post?key=06'}
2024-06-23 16:33:44.329 [INFO]: {'args': {}, 'data': {'payload': '658fcc838aa34683a73ce4b17dcf2349', 'timestamp': 1719156805059}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719156824.301', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784058-40e0f96c4ad720093e5159d6', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '658fcc838aa34683a73ce4b17dcf2349', 'timestamp': 1719156805059}, 'url': 'https://postman-echo.com/post?key=72'}
^C2024-06-23 16:34:24.657 [INFO]: CTRL-C pressed by user!
2024-06-23 16:34:24.663 [INFO]: Closing consumer avro-deserialiser-01 (avro-deserialiser)
2024-06-23 16:34:24.670 [INFO]: Stopping parallel consumer threads
2024-06-23 16:34:24.676 [INFO]: Stopped consumer thread: 0
2024-06-23 16:34:24.676 [INFO]: All parallel consumer threads stopped

Running with five threads, with ordering and processing 50 messages. Each message will be posted to Postman echo. It took around 6 seconds:

• First message: 16:37:51.998
• Last message: 16:37:57.429

% python3 avro_producer.py --iterations 50
2024-06-23 16:37:41.977 [INFO]: Progress: 10%
2024-06-23 16:37:41.977 [INFO]: Progress: 20%
2024-06-23 16:37:41.977 [INFO]: Progress: 30%
2024-06-23 16:37:41.977 [INFO]: Progress: 40%
2024-06-23 16:37:41.977 [INFO]: Progress: 50%
2024-06-23 16:37:41.978 [INFO]: Progress: 60%
2024-06-23 16:37:41.978 [INFO]: Progress: 70%
2024-06-23 16:37:41.978 [INFO]: Progress: 80%
2024-06-23 16:37:41.978 [INFO]: Progress: 90%
2024-06-23 16:37:41.978 [INFO]: Progress: 100%

% python3 test_parallel_consumer.py
2024-06-23 16:37:47.457 [INFO]: Starting parallel consumer thread: 0
2024-06-23 16:37:47.463 [INFO]: Starting parallel consumer thread: 1
2024-06-23 16:37:47.494 [INFO]: Starting parallel consumer thread: 2
2024-06-23 16:37:47.500 [INFO]: Starting parallel consumer thread: 3
2024-06-23 16:37:47.507 [INFO]: Starting parallel consumer thread: 4
2024-06-23 16:37:47.574 [INFO]: Started consumer avro-deserialiser-01 (avro-deserialiser) on topic 'demo_parallel_consumer'
2024-06-23 16:37:51.998 [INFO]: {'args': {}, 'data': {'payload': '2cce0b30b5254e98b04f715ed0148a90', 'timestamp': 1719157070454}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157071.969', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678414f-27a3ce11168a79e675b5d3b3', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '2cce0b30b5254e98b04f715ed0148a90', 'timestamp': 1719157070454}, 'url': 'https://postman-echo.com/post?key=14'}
2024-06-23 16:37:52.000 [INFO]: {'args': {}, 'data': {'payload': 'd691e8f4f9cb40a7a6d26e3898caa7b9', 'timestamp': 1719157070453}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157071.971', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678414f-32a3e6941f5b3c14576866c6', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'd691e8f4f9cb40a7a6d26e3898caa7b9', 'timestamp': 1719157070453}, 'url': 'https://postman-echo.com/post?key=73'}
2024-06-23 16:37:52.001 [INFO]: {'args': {}, 'data': {'payload': '11855b8d7fe64f3ca05d5786012c5aeb', 'timestamp': 1719157070461}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157071.973', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678414f-506d61b350f8980118ac7bf9', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '11855b8d7fe64f3ca05d5786012c5aeb', 'timestamp': 1719157070461}, 'url': 'https://postman-echo.com/post?key=55'}
2024-06-23 16:37:52.002 [INFO]: {'args': {}, 'data': {'payload': '63c167886fd343858a99de5d1d240f57', 'timestamp': 1719157070442}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157071.974', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678414f-39c6f4a333f7f2d74427168e', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '63c167886fd343858a99de5d1d240f57', 'timestamp': 1719157070442}, 'url': 'https://postman-echo.com/post?key=37'}
2024-06-23 16:37:52.003 [INFO]: {'args': {}, 'data': {'payload': '79282b01f92a4859b96e3c1c155de8b0', 'timestamp': 1719157070458}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157071.972', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-6678414f-06ecc0d8107504b96c78030e', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '79282b01f92a4859b96e3c1c155de8b0', 'timestamp': 1719157070458}, 'url': 'https://postman-echo.com/post?key=11'}
2024-06-23 16:37:52.364 [INFO]: {'args': {}, 'data': {'payload': 'f46ee7ef20dd4dec847f4f2b26a8a6aa', 'timestamp': 1719157070463}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.333', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-1b1c0f6f1f49b2835b1d171a', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'f46ee7ef20dd4dec847f4f2b26a8a6aa', 'timestamp': 1719157070463}, 'url': 'https://postman-echo.com/post?key=65'}
2024-06-23 16:37:52.364 [INFO]: {'args': {}, 'data': {'payload': 'a6c654153c8446c6a43a2f9ddad8d991', 'timestamp': 1719157070457}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.333', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-4675b1907d21d7bc737d310f', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'a6c654153c8446c6a43a2f9ddad8d991', 'timestamp': 1719157070457}, 'url': 'https://postman-echo.com/post?key=71'}
2024-06-23 16:37:52.365 [INFO]: {'args': {}, 'data': {'payload': '096f7b531b464ad2bc29c857f0939581', 'timestamp': 1719157070462}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.335', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-380b69ba0b9e5e681d512ab8', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '096f7b531b464ad2bc29c857f0939581', 'timestamp': 1719157070462}, 'url': 'https://postman-echo.com/post?key=96'}
2024-06-23 16:37:52.404 [INFO]: {'args': {}, 'data': {'payload': '57b8a5a0e2154ccb9652106f6f468f79', 'timestamp': 1719157070456}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.334', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-21f96616641191e219ae8b49', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '57b8a5a0e2154ccb9652106f6f468f79', 'timestamp': 1719157070456}, 'url': 'https://postman-echo.com/post?key=02'}
2024-06-23 16:37:52.405 [INFO]: {'args': {}, 'data': {'payload': '330ef38b2b4a44f290856694194ee089', 'timestamp': 1719157070477}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.334', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-237ef15c309f72787478211f', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '330ef38b2b4a44f290856694194ee089', 'timestamp': 1719157070477}, 'url': 'https://postman-echo.com/post?key=62'}
2024-06-23 16:37:52.717 [INFO]: {'args': {}, 'data': {'payload': 'e688a007b9c74114abadce04abb439a1', 'timestamp': 1719157070467}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.689', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-307f50876b69a4ee50bf675b', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'e688a007b9c74114abadce04abb439a1', 'timestamp': 1719157070467}, 'url': 'https://postman-echo.com/post?key=08'}
2024-06-23 16:37:52.720 [INFO]: {'args': {}, 'data': {'payload': 'ac83afb159ae4913b0bfbe356396c73d', 'timestamp': 1719157070475}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.690', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-61abf953480db0a05ad76a04', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'ac83afb159ae4913b0bfbe356396c73d', 'timestamp': 1719157070475}, 'url': 'https://postman-echo.com/post?key=00'}
2024-06-23 16:37:52.759 [INFO]: {'args': {}, 'data': {'payload': 'e8374f4e8ca44b518aef9887bcd42f96', 'timestamp': 1719157070460}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.732', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-3a8261187d47fe3f22cd0940', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'e8374f4e8ca44b518aef9887bcd42f96', 'timestamp': 1719157070460}, 'url': 'https://postman-echo.com/post?key=88'}
2024-06-23 16:37:52.762 [INFO]: {'args': {}, 'data': {'payload': '2c8cc893f1c9414dacdd27740b9fd2d0', 'timestamp': 1719157070465}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.692', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-2011eb1e65c813b44867abdb', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '2c8cc893f1c9414dacdd27740b9fd2d0', 'timestamp': 1719157070465}, 'url': 'https://postman-echo.com/post?key=17'}
2024-06-23 16:37:52.806 [INFO]: {'args': {}, 'data': {'payload': '6bd65c338f6c44889148c254404e3976', 'timestamp': 1719157070480}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157072.732', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784150-37e037f14edd518173dc41c6', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '6bd65c338f6c44889148c254404e3976', 'timestamp': 1719157070480}, 'url': 'https://postman-echo.com/post?key=83'}
2024-06-23 16:37:53.078 [INFO]: {'args': {}, 'data': {'payload': 'c3f1df702f184b45a72fbc8061417482', 'timestamp': 1719157070476}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.050', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-26a471055c7ead4556adb511', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'c3f1df702f184b45a72fbc8061417482', 'timestamp': 1719157070476}, 'url': 'https://postman-echo.com/post?key=41'}
2024-06-23 16:37:53.079 [INFO]: {'args': {}, 'data': {'payload': 'c66445675c0b4dec9beaf5cbe35f5467', 'timestamp': 1719157070481}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.051', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-3ab659871254558638c74162', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'c66445675c0b4dec9beaf5cbe35f5467', 'timestamp': 1719157070481}, 'url': 'https://postman-echo.com/post?key=00'}
2024-06-23 16:37:53.119 [INFO]: {'args': {}, 'data': {'payload': 'fe27b0997e1b4509aa64e7f26d78a8c9', 'timestamp': 1719157070466}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.091', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-17f0d9a83a6ab56f6605608f', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'fe27b0997e1b4509aa64e7f26d78a8c9', 'timestamp': 1719157070466}, 'url': 'https://postman-echo.com/post?key=24'}
2024-06-23 16:37:53.128 [INFO]: {'args': {}, 'data': {'payload': 'c34c66c3b067480eb03c9fb568f190b5', 'timestamp': 1719157070470}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.091', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-39b7262061bc8ed268997c61', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'c34c66c3b067480eb03c9fb568f190b5', 'timestamp': 1719157070470}, 'url': 'https://postman-echo.com/post?key=02'}
2024-06-23 16:37:53.206 [INFO]: {'args': {}, 'data': {'payload': '8277fcc45aa74548b2ea0e23d55bd230', 'timestamp': 1719157070482}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.135', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-69a930487ca16f6c6df0e164', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '8277fcc45aa74548b2ea0e23d55bd230', 'timestamp': 1719157070482}, 'url': 'https://postman-echo.com/post?key=28'}
2024-06-23 16:37:53.436 [INFO]: {'args': {}, 'data': {'payload': 'becc00c4f8c547d79d8d90e209aff812', 'timestamp': 1719157070501}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.408', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-7957633040ab1f7252072a24', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'becc00c4f8c547d79d8d90e209aff812', 'timestamp': 1719157070501}, 'url': 'https://postman-echo.com/post?key=75'}
2024-06-23 16:37:53.486 [INFO]: {'args': {}, 'data': {'payload': 'cc6c42bd39484958a5ac1b72581fbcb7', 'timestamp': 1719157070474}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.454', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-736df85f50afacb50847ce97', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'cc6c42bd39484958a5ac1b72581fbcb7', 'timestamp': 1719157070474}, 'url': 'https://postman-echo.com/post?key=53'}
2024-06-23 16:37:53.750 [INFO]: {'args': {}, 'data': {'payload': '1adb45dedde14948b91e80d06c4efccc', 'timestamp': 1719157070479}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.408', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-7eb22a4f3139c7250e53b97b', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '1adb45dedde14948b91e80d06c4efccc', 'timestamp': 1719157070479}, 'url': 'https://postman-echo.com/post?key=86'}
2024-06-23 16:37:53.750 [INFO]: {'args': {}, 'data': {'payload': '9636d543605f4032957b608f7bda84e9', 'timestamp': 1719157070469}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.450', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-15680cbe5f6afb137ffe4d5d', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '9636d543605f4032957b608f7bda84e9', 'timestamp': 1719157070469}, 'url': 'https://postman-echo.com/post?key=17'}
2024-06-23 16:37:53.751 [INFO]: {'args': {}, 'data': {'payload': '9ee1e8f73e4d4e909310d0db1a32f3b0', 'timestamp': 1719157070490}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.554', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-7e7f467a7f6cc110494a599f', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '9ee1e8f73e4d4e909310d0db1a32f3b0', 'timestamp': 1719157070490}, 'url': 'https://postman-echo.com/post?key=18'}
2024-06-23 16:37:54.088 [INFO]: {'args': {}, 'data': {'payload': 'afd4e001459b4665aead354b5bea2641', 'timestamp': 1719157070485}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157073.890', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784151-7896c50425863e9c5f9fae50', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'afd4e001459b4665aead354b5bea2641', 'timestamp': 1719157070485}, 'url': 'https://postman-echo.com/post?key=82'}
2024-06-23 16:37:54.187 [INFO]: {'args': {}, 'data': {'payload': 'a0e8d3b334924270936cb3262025a248', 'timestamp': 1719157070484}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157074.155', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784152-12a37e2d2934ff7063984ffd', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'a0e8d3b334924270936cb3262025a248', 'timestamp': 1719157070484}, 'url': 'https://postman-echo.com/post?key=41'}
2024-06-23 16:37:54.226 [INFO]: {'args': {}, 'data': {'payload': '926ca5b482bb48589894d28ff2ff0a30', 'timestamp': 1719157070493}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157074.150', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784152-6d3759c723faf44b7cfd9f67', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '926ca5b482bb48589894d28ff2ff0a30', 'timestamp': 1719157070493}, 'url': 'https://postman-echo.com/post?key=57'}
2024-06-23 16:37:54.310 [INFO]: {'args': {}, 'data': {'payload': '51340a280bda4ffd86747ee64e607f6e', 'timestamp': 1719157070471}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157074.157', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784152-28a8fc5e2194a209478c4ae8', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '51340a280bda4ffd86747ee64e607f6e', 'timestamp': 1719157070471}, 'url': 'https://postman-echo.com/post?key=04'}
2024-06-23 16:37:54.545 [INFO]: {'args': {}, 'data': {'payload': '2c143f9715754073b7d716d91c1ada7a', 'timestamp': 1719157070486}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157074.472', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784152-5b06d273079307933c1e375b', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '2c143f9715754073b7d716d91c1ada7a', 'timestamp': 1719157070486}, 'url': 'https://postman-echo.com/post?key=29'}
2024-06-23 16:37:54.604 [INFO]: {'args': {}, 'data': {'payload': '24ed0229605e46bfb0f48e8208a685ce', 'timestamp': 1719157070488}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157074.576', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784152-283847ba3837ddb815c87bee', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '24ed0229605e46bfb0f48e8208a685ce', 'timestamp': 1719157070488}, 'url': 'https://postman-echo.com/post?key=76'}
2024-06-23 16:37:54.669 [INFO]: {'args': {}, 'data': {'payload': '74208c4d3b42429dbd8e216db0505fcb', 'timestamp': 1719157070494}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157074.637', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784152-2fa4a1297f9f1989560920a7', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '74208c4d3b42429dbd8e216db0505fcb', 'timestamp': 1719157070494}, 'url': 'https://postman-echo.com/post?key=59'}
2024-06-23 16:37:54.748 [INFO]: {'args': {}, 'data': {'payload': 'a5cd199f9dd9414688dc076d05d9110d', 'timestamp': 1719157070472}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157074.716', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784152-1351828f5db9d1103ace2b29', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'a5cd199f9dd9414688dc076d05d9110d', 'timestamp': 1719157070472}, 'url': 'https://postman-echo.com/post?key=24'}
2024-06-23 16:37:54.977 [INFO]: {'args': {}, 'data': {'payload': '99b9d788b1394c93ac3885c2dc7db1e7', 'timestamp': 1719157070491}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157074.945', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784152-5337729344bdd806609e765e', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '99b9d788b1394c93ac3885c2dc7db1e7', 'timestamp': 1719157070491}, 'url': 'https://postman-echo.com/post?key=88'}
2024-06-23 16:37:55.094 [INFO]: {'args': {}, 'data': {'payload': '58b1e82d1e204ee38923a26c09f5dc4e', 'timestamp': 1719157070498}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157075.062', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784153-43f1f6b515542a6b5d540888', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '58b1e82d1e204ee38923a26c09f5dc4e', 'timestamp': 1719157070498}, 'url': 'https://postman-echo.com/post?key=64'}
2024-06-23 16:37:55.242 [INFO]: {'args': {}, 'data': {'payload': '101e6ffe2b5c47cf902e2e1995430259', 'timestamp': 1719157070496}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157075.135', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784153-4631a62d562cd9f879910475', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '101e6ffe2b5c47cf902e2e1995430259', 'timestamp': 1719157070496}, 'url': 'https://postman-echo.com/post?key=14'}
2024-06-23 16:37:55.289 [INFO]: {'args': {}, 'data': {'payload': 'dbb6bf2696ae4316b3a7540828c6b8fc', 'timestamp': 1719157070489}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157074.990', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784152-0b281b4d344c90f659b86ec4', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'dbb6bf2696ae4316b3a7540828c6b8fc', 'timestamp': 1719157070489}, 'url': 'https://postman-echo.com/post?key=27'}
2024-06-23 16:37:55.463 [INFO]: {'args': {}, 'data': {'payload': 'b58dedc374cd4506a296ee16be2a4a34', 'timestamp': 1719157070495}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157075.388', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784153-7bb6e3d16ac971711a61e92c', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'b58dedc374cd4506a296ee16be2a4a34', 'timestamp': 1719157070495}, 'url': 'https://postman-echo.com/post?key=31'}
2024-06-23 16:37:55.501 [INFO]: {'args': {}, 'data': {'payload': '6e9cd13dcfc64a20ae5ceb49b2ec7a6d', 'timestamp': 1719157070503}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157075.473', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784153-39efae3c619825473bd3c4b3', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '6e9cd13dcfc64a20ae5ceb49b2ec7a6d', 'timestamp': 1719157070503}, 'url': 'https://postman-echo.com/post?key=32'}
2024-06-23 16:37:55.664 [INFO]: {'args': {}, 'data': {'payload': '6185a8568a1049058c1f816bd9b43817', 'timestamp': 1719157070499}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157075.626', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784153-21183d032c15355277ec50c2', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '6185a8568a1049058c1f816bd9b43817', 'timestamp': 1719157070499}, 'url': 'https://postman-echo.com/post?key=13'}
2024-06-23 16:37:55.734 [INFO]: {'args': {}, 'data': {'payload': '34743c44b21c4171aa1fb982d381dcbd', 'timestamp': 1719157070504}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157075.708', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784153-0856466f25b373fa603d4e5c', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '34743c44b21c4171aa1fb982d381dcbd', 'timestamp': 1719157070504}, 'url': 'https://postman-echo.com/post?key=58'}
2024-06-23 16:37:55.949 [INFO]: {'args': {}, 'data': {'payload': '2bf1cb42fb5b4f64b156b053fe6f6031', 'timestamp': 1719157070505}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157075.916', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784153-7126ac9749ac6ac001e3b7de', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '2bf1cb42fb5b4f64b156b053fe6f6031', 'timestamp': 1719157070505}, 'url': 'https://postman-echo.com/post?key=15'}
2024-06-23 16:37:56.145 [INFO]: {'args': {}, 'data': {'payload': '3e02f6d1add849f9a43fb3d57133220a', 'timestamp': 1719157070500}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157076.106', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784154-1aabe112485475bd094a2ae6', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '3e02f6d1add849f9a43fb3d57133220a', 'timestamp': 1719157070500}, 'url': 'https://postman-echo.com/post?key=01'}
2024-06-23 16:37:56.218 [INFO]: {'args': {}, 'data': {'payload': '4d0d1f21794d4a7e9580c0a64e27c9cb', 'timestamp': 1719157070510}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157076.188', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784154-0217d5da5891c07453d194ad', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '4d0d1f21794d4a7e9580c0a64e27c9cb', 'timestamp': 1719157070510}, 'url': 'https://postman-echo.com/post?key=72'}
2024-06-23 16:37:56.402 [INFO]: {'args': {}, 'data': {'payload': '3033fe007274457baf71cdb88da17964', 'timestamp': 1719157070508}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157076.375', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784154-7cbdb3bf1c359a507766fe00', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '3033fe007274457baf71cdb88da17964', 'timestamp': 1719157070508}, 'url': 'https://postman-echo.com/post?key=15'}
2024-06-23 16:37:56.628 [INFO]: {'args': {}, 'data': {'payload': '122a99a31f434397a0068038f418f396', 'timestamp': 1719157070506}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157076.591', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784154-0e66df037741277303917fbc', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '122a99a31f434397a0068038f418f396', 'timestamp': 1719157070506}, 'url': 'https://postman-echo.com/post?key=46'}
2024-06-23 16:37:56.685 [INFO]: {'args': {}, 'data': {'payload': '9f9e7b076bc54dffa96524467ecdedfc', 'timestamp': 1719157070514}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157076.658', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784154-1d0ff2582e53db2f7b428035', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '9f9e7b076bc54dffa96524467ecdedfc', 'timestamp': 1719157070514}, 'url': 'https://postman-echo.com/post?key=23'}
2024-06-23 16:37:56.885 [INFO]: {'args': {}, 'data': {'payload': '47955ae381ee4731a607c03d165d25ba', 'timestamp': 1719157070509}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157076.853', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784154-4c32d1f22f4cee3128f2026c', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '47955ae381ee4731a607c03d165d25ba', 'timestamp': 1719157070509}, 'url': 'https://postman-echo.com/post?key=62'}
2024-06-23 16:37:57.104 [INFO]: {'args': {}, 'data': {'payload': 'b4d05cd0e1c24a0d91a7251a71aa1648', 'timestamp': 1719157070511}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157077.077', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784155-0b31d0501e1184697f0cfbc9', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'b4d05cd0e1c24a0d91a7251a71aa1648', 'timestamp': 1719157070511}, 'url': 'https://postman-echo.com/post?key=74'}
2024-06-23 16:37:57.429 [INFO]: {'args': {}, 'data': {'payload': 'ad664b86b607409a9b6a5629b708e437', 'timestamp': 1719157070513}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157077.403', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784155-04cf28dc71ccaa916be22db6', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'ad664b86b607409a9b6a5629b708e437', 'timestamp': 1719157070513}, 'url': 'https://postman-echo.com/post?key=84'}
^C2024-06-23 16:38:00.102 [INFO]: CTRL-C pressed by user!
2024-06-23 16:38:00.121 [INFO]: Closing consumer avro-deserialiser-01 (avro-deserialiser)
2024-06-23 16:38:00.152 [INFO]: Stopping parallel consumer threads
2024-06-23 16:38:00.190 [INFO]: Stopped consumer thread: 1
2024-06-23 16:38:00.190 [INFO]: Stopped consumer thread: 3
2024-06-23 16:38:00.190 [INFO]: Stopped consumer thread: 0
2024-06-23 16:38:00.190 [INFO]: Stopped consumer thread: 4
2024-06-23 16:38:00.190 [INFO]: Stopped consumer thread: 2
2024-06-23 16:38:00.190 [INFO]: All parallel consumer threads stopped

Running with five threads, without ordering and processing 50 messages. Each message will be posted to Postman echo. It took around 4 seconds:

• First message: 16:42:11.800
• Last message: 16:42:15.601

% python3 avro_producer.py --iterations 50
2024-06-23 16:42:00.155 [INFO]: Progress: 10%
2024-06-23 16:42:00.155 [INFO]: Progress: 20%
2024-06-23 16:42:00.155 [INFO]: Progress: 30%
2024-06-23 16:42:00.155 [INFO]: Progress: 40%
2024-06-23 16:42:00.155 [INFO]: Progress: 50%
2024-06-23 16:42:00.156 [INFO]: Progress: 60%
2024-06-23 16:42:00.156 [INFO]: Progress: 70%
2024-06-23 16:42:00.156 [INFO]: Progress: 80%
2024-06-23 16:42:00.156 [INFO]: Progress: 90%
2024-06-23 16:42:00.156 [INFO]: Progress: 100%

% python3 test_parallel_consumer.py
2024-06-23 16:42:07.234 [INFO]: Starting parallel consumer thread: 0
2024-06-23 16:42:07.240 [INFO]: Starting parallel consumer thread: 1
2024-06-23 16:42:07.271 [INFO]: Starting parallel consumer thread: 2
2024-06-23 16:42:07.322 [INFO]: Starting parallel consumer thread: 3
2024-06-23 16:42:07.378 [INFO]: Starting parallel consumer thread: 4
2024-06-23 16:42:07.415 [INFO]: Started consumer avro-deserialiser-01 (avro-deserialiser) on topic 'demo_parallel_consumer'
2024-06-23 16:42:11.800 [INFO]: {'args': {}, 'data': {'payload': '93dfa4188c0643858813c4f7a7f9526b', 'timestamp': 1719157330318}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157331.773', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784253-4167caad5ae2405a1365ab27', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '93dfa4188c0643858813c4f7a7f9526b', 'timestamp': 1719157330318}, 'url': 'https://postman-echo.com/post?key=63'}
2024-06-23 16:42:11.802 [INFO]: {'args': {}, 'data': {'payload': '543157bd515c466a8e3f2eba703e48d9', 'timestamp': 1719157330316}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157331.773', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784253-4edff13a01c79ffd674d33fb', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '543157bd515c466a8e3f2eba703e48d9', 'timestamp': 1719157330316}, 'url': 'https://postman-echo.com/post?key=99'}
2024-06-23 16:42:11.803 [INFO]: {'args': {}, 'data': {'payload': '6cb93f2ed9b84d79ad3e359516319f83', 'timestamp': 1719157330320}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157331.777', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784253-3fda02da0a5aed2d726cde27', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '6cb93f2ed9b84d79ad3e359516319f83', 'timestamp': 1719157330320}, 'url': 'https://postman-echo.com/post?key=60'}
2024-06-23 16:42:11.804 [INFO]: {'args': {}, 'data': {'payload': 'b7166f97e78d40289d71f744b482c969', 'timestamp': 1719157330317}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157331.778', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784253-4b5389d94d3f3d79680b166a', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'b7166f97e78d40289d71f744b482c969', 'timestamp': 1719157330317}, 'url': 'https://postman-echo.com/post?key=55'}
2024-06-23 16:42:11.838 [INFO]: {'args': {}, 'data': {'payload': 'f324c3d0d14e465cab2f1dce1d348941', 'timestamp': 1719157330291}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157331.772', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784253-120dadd87d9b435c42b172e4', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'f324c3d0d14e465cab2f1dce1d348941', 'timestamp': 1719157330291}, 'url': 'https://postman-echo.com/post?key=65'}
2024-06-23 16:42:12.159 [INFO]: {'args': {}, 'data': {'payload': '6ffe2bb2d97a49909c89dc9babdaed7c', 'timestamp': 1719157330324}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.134', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-1764ed8d451d612129fc90f0', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '6ffe2bb2d97a49909c89dc9babdaed7c', 'timestamp': 1719157330324}, 'url': 'https://postman-echo.com/post?key=50'}
2024-06-23 16:42:12.162 [INFO]: {'args': {}, 'data': {'payload': '74b4de740ade4545bb49595a255a3259', 'timestamp': 1719157330322}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.133', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-0991fde33d4c095269e6c598', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '74b4de740ade4545bb49595a255a3259', 'timestamp': 1719157330322}, 'url': 'https://postman-echo.com/post?key=17'}
2024-06-23 16:42:12.186 [INFO]: {'args': {}, 'data': {'payload': '401d968d4e4a46708981f1a5bfa9ad36', 'timestamp': 1719157330325}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.134', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-6c6ba3304b0ec7552d79662d', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '401d968d4e4a46708981f1a5bfa9ad36', 'timestamp': 1719157330325}, 'url': 'https://postman-echo.com/post?key=07'}
2024-06-23 16:42:12.191 [INFO]: {'args': {}, 'data': {'payload': '1c7cedcddbd54355a78317fbfc961ebf', 'timestamp': 1719157330321}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.166', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-5b863d3a0c1b3f3c00563d60', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '1c7cedcddbd54355a78317fbfc961ebf', 'timestamp': 1719157330321}, 'url': 'https://postman-echo.com/post?key=73'}
2024-06-23 16:42:12.203 [INFO]: {'args': {}, 'data': {'payload': '38899ddfc63d4875bf180de38842c29d', 'timestamp': 1719157330326}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.133', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-0b746d2724b4f0886156a463', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '38899ddfc63d4875bf180de38842c29d', 'timestamp': 1719157330326}, 'url': 'https://postman-echo.com/post?key=31'}
2024-06-23 16:42:12.531 [INFO]: {'args': {}, 'data': {'payload': '515dc9beeeb04e0090f0c8c9c361038c', 'timestamp': 1719157330331}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.506', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-545714df18e1e01e29c14dc1', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '515dc9beeeb04e0090f0c8c9c361038c', 'timestamp': 1719157330331}, 'url': 'https://postman-echo.com/post?key=32'}
2024-06-23 16:42:12.581 [INFO]: {'args': {}, 'data': {'payload': 'b48a521415f8416081478a1556d9ce43', 'timestamp': 1719157330327}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.510', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-5b562da07e5f4b9e02c07351', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'b48a521415f8416081478a1556d9ce43', 'timestamp': 1719157330327}, 'url': 'https://postman-echo.com/post?key=14'}
2024-06-23 16:42:12.646 [INFO]: {'args': {}, 'data': {'payload': 'e926d27e260c4b9a89aea1e343877e03', 'timestamp': 1719157330329}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.481', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-75d49d7b5dcb3b3f2c14946c', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'e926d27e260c4b9a89aea1e343877e03', 'timestamp': 1719157330329}, 'url': 'https://postman-echo.com/post?key=08'}
2024-06-23 16:42:12.647 [INFO]: {'args': {}, 'data': {'payload': '301971065d9b49d9abe60fd0fcd288c7', 'timestamp': 1719157330330}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.482', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-77fe99a027a83f9d2f1d172b', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '301971065d9b49d9abe60fd0fcd288c7', 'timestamp': 1719157330330}, 'url': 'https://postman-echo.com/post?key=64'}
2024-06-23 16:42:12.742 [INFO]: {'args': {}, 'data': {'payload': 'f64161e8782a4f12b414f97eb32626db', 'timestamp': 1719157330333}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.528', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-4fff8f3a480cac0817d7aa03', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'f64161e8782a4f12b414f97eb32626db', 'timestamp': 1719157330333}, 'url': 'https://postman-echo.com/post?key=74'}
2024-06-23 16:42:12.888 [INFO]: {'args': {}, 'data': {'payload': 'efd2b0f28c9640be9279935c53bf2267', 'timestamp': 1719157330338}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.860', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-37e39ed1032972b37b613be2', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'efd2b0f28c9640be9279935c53bf2267', 'timestamp': 1719157330338}, 'url': 'https://postman-echo.com/post?key=15'}
2024-06-23 16:42:12.996 [INFO]: {'args': {}, 'data': {'payload': 'fe5ac4d6eb8444d887b7a3892c6cb531', 'timestamp': 1719157330335}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.969', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-325f4014444025b15082c943', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'fe5ac4d6eb8444d887b7a3892c6cb531', 'timestamp': 1719157330335}, 'url': 'https://postman-echo.com/post?key=14'}
2024-06-23 16:42:13.245 [INFO]: {'args': {}, 'data': {'payload': 'cbd2cff734104abca84e2f7c1fe3faaa', 'timestamp': 1719157330344}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.214', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-3e36086d2021d73d788bf391', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'cbd2cff734104abca84e2f7c1fe3faaa', 'timestamp': 1719157330344}, 'url': 'https://postman-echo.com/post?key=94'}
2024-06-23 16:42:13.263 [INFO]: {'args': {}, 'data': {'payload': '584eced70565498ea5459fa2f29b6f03', 'timestamp': 1719157330336}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.971', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-393c139644ba3757419aa828', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '584eced70565498ea5459fa2f29b6f03', 'timestamp': 1719157330336}, 'url': 'https://postman-echo.com/post?key=88'}
2024-06-23 16:42:13.278 [INFO]: {'args': {}, 'data': {'payload': 'f631b686407a4844bc824fff7d6df86c', 'timestamp': 1719157330334}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157332.901', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784254-4bb8d3293ec24a333cb083e4', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'f631b686407a4844bc824fff7d6df86c', 'timestamp': 1719157330334}, 'url': 'https://postman-echo.com/post?key=34'}
2024-06-23 16:42:13.343 [INFO]: {'args': {}, 'data': {'payload': '6e7f271ed5834ba0a0da1c6ae7fc6ed6', 'timestamp': 1719157330339}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.068', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-4877c6c42578e3155eae5b49', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '6e7f271ed5834ba0a0da1c6ae7fc6ed6', 'timestamp': 1719157330339}, 'url': 'https://postman-echo.com/post?key=42'}
2024-06-23 16:42:13.345 [INFO]: {'args': {}, 'data': {'payload': '4981a7d78abc47efbc937f872f618a1f', 'timestamp': 1719157330341}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.319', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-0c3fe8d2525cf9bc4e246cee', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '4981a7d78abc47efbc937f872f618a1f', 'timestamp': 1719157330341}, 'url': 'https://postman-echo.com/post?key=19'}
2024-06-23 16:42:13.598 [INFO]: {'args': {}, 'data': {'payload': '77fd13c468b94331aa2aa162d2ea2f0a', 'timestamp': 1719157330350}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.572', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-74f5586136f6d66825aa20f6', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '77fd13c468b94331aa2aa162d2ea2f0a', 'timestamp': 1719157330350}, 'url': 'https://postman-echo.com/post?key=67'}
2024-06-23 16:42:13.611 [INFO]: {'args': {}, 'data': {'payload': 'eac94643d5c84a23a92eec8021dfc16f', 'timestamp': 1719157330343}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.586', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-683970b91f7b045c313e8ced', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'eac94643d5c84a23a92eec8021dfc16f', 'timestamp': 1719157330343}, 'url': 'https://postman-echo.com/post?key=83'}
2024-06-23 16:42:13.631 [INFO]: {'args': {}, 'data': {'payload': '0cbfbaeb15434436a6f80ba0f995d167', 'timestamp': 1719157330340}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.606', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-4e5b3deb6179537f60cfbc29', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '0cbfbaeb15434436a6f80ba0f995d167', 'timestamp': 1719157330340}, 'url': 'https://postman-echo.com/post?key=37'}
2024-06-23 16:42:13.700 [INFO]: {'args': {}, 'data': {'payload': '4d7621c007f5401680fcb372ff79df14', 'timestamp': 1719157330348}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.671', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-25b5a25c1e9277c70c75fc95', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '4d7621c007f5401680fcb372ff79df14', 'timestamp': 1719157330348}, 'url': 'https://postman-echo.com/post?key=89'}
2024-06-23 16:42:13.738 [INFO]: {'args': {}, 'data': {'payload': '70c455a5ac9b4adfb4df02b4bf351064', 'timestamp': 1719157330345}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.668', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-23e6e6d157d0f98c435c8afd', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '70c455a5ac9b4adfb4df02b4bf351064', 'timestamp': 1719157330345}, 'url': 'https://postman-echo.com/post?key=89'}
2024-06-23 16:42:13.949 [INFO]: {'args': {}, 'data': {'payload': 'a8c04f51647e418bab8544be55396ac7', 'timestamp': 1719157330356}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.923', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-2327d7884dd5a72f5b6f9589', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'a8c04f51647e418bab8544be55396ac7', 'timestamp': 1719157330356}, 'url': 'https://postman-echo.com/post?key=08'}
2024-06-23 16:42:13.961 [INFO]: {'args': {}, 'data': {'payload': '12fd2f76ac9d4556928a13590f6b3baf', 'timestamp': 1719157330349}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.936', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-12a6533279cb664042e979d6', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '12fd2f76ac9d4556928a13590f6b3baf', 'timestamp': 1719157330349}, 'url': 'https://postman-echo.com/post?key=12'}
2024-06-23 16:42:14.027 [INFO]: {'args': {}, 'data': {'payload': 'cc7ee32559fa44a0a3e7efba4cca3aa5', 'timestamp': 1719157330347}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157333.959', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784255-0d33ce9c0087fb6606fb5520', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'cc7ee32559fa44a0a3e7efba4cca3aa5', 'timestamp': 1719157330347}, 'url': 'https://postman-echo.com/post?key=66'}
2024-06-23 16:42:14.056 [INFO]: {'args': {}, 'data': {'payload': '26813dcebb9e4477bb8f10f90fa6e068', 'timestamp': 1719157330354}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.024', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-6a2299991768f17b771fd2e8', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '26813dcebb9e4477bb8f10f90fa6e068', 'timestamp': 1719157330354}, 'url': 'https://postman-echo.com/post?key=87'}
2024-06-23 16:42:14.138 [INFO]: {'args': {}, 'data': {'payload': '8b993649d09047b3a46a2997db292606', 'timestamp': 1719157330351}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.067', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-5a9bcea656a9842f4fa60cb4', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '8b993649d09047b3a46a2997db292606', 'timestamp': 1719157330351}, 'url': 'https://postman-echo.com/post?key=55'}
2024-06-23 16:42:14.303 [INFO]: {'args': {}, 'data': {'payload': '60ab35fd0e764f73853f94a55d8c5ab8', 'timestamp': 1719157330363}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.276', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-7c4c97fc0caba47c2a2b36d3', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '60ab35fd0e764f73853f94a55d8c5ab8', 'timestamp': 1719157330363}, 'url': 'https://postman-echo.com/post?key=39'}
2024-06-23 16:42:14.309 [INFO]: {'args': {}, 'data': {'payload': 'd6925541a4d34312909a3863166ba0b7', 'timestamp': 1719157330355}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.285', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-29be837b56dd0827580f7d56', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'd6925541a4d34312909a3863166ba0b7', 'timestamp': 1719157330355}, 'url': 'https://postman-echo.com/post?key=52'}
2024-06-23 16:42:14.404 [INFO]: {'args': {}, 'data': {'payload': 'fdd21abf6d694101b10b124993fdf40d', 'timestamp': 1719157330360}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.379', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-0252d9992e908cfd79045723', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'fdd21abf6d694101b10b124993fdf40d', 'timestamp': 1719157330360}, 'url': 'https://postman-echo.com/post?key=51'}
2024-06-23 16:42:14.418 [INFO]: {'args': {}, 'data': {'payload': 'f0a43c25bc7d4ef684393599c46fc3f2', 'timestamp': 1719157330353}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.353', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-062fc6d30cbd811a7e022083', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'f0a43c25bc7d4ef684393599c46fc3f2', 'timestamp': 1719157330353}, 'url': 'https://postman-echo.com/post?key=47'}
2024-06-23 16:42:14.491 [INFO]: {'args': {}, 'data': {'payload': '37731f14c8c4494bab15d1b9fe7056f7', 'timestamp': 1719157330358}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.465', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-14e6a4fe1531429c5804e96c', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '37731f14c8c4494bab15d1b9fe7056f7', 'timestamp': 1719157330358}, 'url': 'https://postman-echo.com/post?key=84'}
2024-06-23 16:42:14.662 [INFO]: {'args': {}, 'data': {'payload': 'b9bcb11a361e44c4b65bd51fe5587cb7', 'timestamp': 1719157330361}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.635', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-5b1b9ca120909b2015f84a19', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'b9bcb11a361e44c4b65bd51fe5587cb7', 'timestamp': 1719157330361}, 'url': 'https://postman-echo.com/post?key=52'}
2024-06-23 16:42:14.769 [INFO]: {'args': {}, 'data': {'payload': 'd6fb402abb6f4389997ade150ee61bad', 'timestamp': 1719157330359}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.741', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-3f73cfae3e835622487927c2', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'd6fb402abb6f4389997ade150ee61bad', 'timestamp': 1719157330359}, 'url': 'https://postman-echo.com/post?key=03'}
2024-06-23 16:42:14.850 [INFO]: {'args': {}, 'data': {'payload': 'e0b482e421a84a4392936571b59046d6', 'timestamp': 1719157330364}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.820', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-14a64e4c367b49746ee78ac2', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'e0b482e421a84a4392936571b59046d6', 'timestamp': 1719157330364}, 'url': 'https://postman-echo.com/post?key=08'}
2024-06-23 16:42:14.885 [INFO]: {'args': {}, 'data': {'payload': 'b0ac2aa37e55446eb79d028df7eb55e0', 'timestamp': 1719157330369}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.634', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-60fcb8626c60cfcc36a13718', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'b0ac2aa37e55446eb79d028df7eb55e0', 'timestamp': 1719157330369}, 'url': 'https://postman-echo.com/post?key=74'}
2024-06-23 16:42:15.025 [INFO]: {'args': {}, 'data': {'payload': 'afcfd15d50c24981b67c26116b039b9d', 'timestamp': 1719157330368}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.999', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-6e4d666848df65791b0f7a6c', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'afcfd15d50c24981b67c26116b039b9d', 'timestamp': 1719157330368}, 'url': 'https://postman-echo.com/post?key=23'}
2024-06-23 16:42:15.127 [INFO]: {'args': {}, 'data': {'payload': 'b2322516fe1146b286aff9e03f97d699', 'timestamp': 1719157330365}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157335.101', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784257-3666d0ae556005041c26c61e', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': 'b2322516fe1146b286aff9e03f97d699', 'timestamp': 1719157330365}, 'url': 'https://postman-echo.com/post?key=34'}
2024-06-23 16:42:15.199 [INFO]: {'args': {}, 'data': {'payload': '086f8a925ca44f9aadd6e7a1f345ba92', 'timestamp': 1719157330370}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157335.171', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784257-51394bba4665c543246dcda9', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '086f8a925ca44f9aadd6e7a1f345ba92', 'timestamp': 1719157330370}, 'url': 'https://postman-echo.com/post?key=86'}
2024-06-23 16:42:15.203 [INFO]: {'args': {}, 'data': {'payload': '4f4f1dbbd4694bb88bc164cd0dbcf5af', 'timestamp': 1719157330367}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157334.729', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784256-1e0ded1508175fe42c5d287b', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '4f4f1dbbd4694bb88bc164cd0dbcf5af', 'timestamp': 1719157330367}, 'url': 'https://postman-echo.com/post?key=47'}
2024-06-23 16:42:15.312 [INFO]: {'args': {}, 'data': {'payload': '4655347fb957417d9bce1a0feaadd734', 'timestamp': 1719157330375}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157335.215', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784257-522e2fd4278045f830638143', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '4655347fb957417d9bce1a0feaadd734', 'timestamp': 1719157330375}, 'url': 'https://postman-echo.com/post?key=63'}
2024-06-23 16:42:15.381 [INFO]: {'args': {}, 'data': {'payload': '1e38e9a401494dc8b72a389bfb277e7f', 'timestamp': 1719157330374}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157335.350', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784257-4b0ce139044803863a58e24d', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '1e38e9a401494dc8b72a389bfb277e7f', 'timestamp': 1719157330374}, 'url': 'https://postman-echo.com/post?key=27'}
2024-06-23 16:42:15.537 [INFO]: {'args': {}, 'data': {'payload': '093ec85663f147c2bed3fb337b75ba80', 'timestamp': 1719157330372}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157335.467', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784257-684515f2459219aa7091d62d', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '093ec85663f147c2bed3fb337b75ba80', 'timestamp': 1719157330372}, 'url': 'https://postman-echo.com/post?key=66'}
2024-06-23 16:42:15.588 [INFO]: {'args': {}, 'data': {'payload': '441f6cb3fa874f04976e1081e2bcfe3c', 'timestamp': 1719157330373}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157335.562', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784257-3943b8963b3d258b182522b1', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '441f6cb3fa874f04976e1081e2bcfe3c', 'timestamp': 1719157330373}, 'url': 'https://postman-echo.com/post?key=30'}
2024-06-23 16:42:15.601 [INFO]: {'args': {}, 'data': {'payload': '8440749db5a94410a7401453b4fbdad6', 'timestamp': 1719157330377}, 'files': {}, 'form': {}, 'headers': {'host': 'postman-echo.com', 'x-request-start': 't=1719157335.574', 'content-length': '75', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', 'x-amzn-trace-id': 'Root=1-66784257-49a3f13f6b0a506010a7cd03', 'user-agent': 'python-requests/2.32.3', 'accept-encoding': 'gzip, deflate', 'accept': '*/*', 'content-type': 'application/json'}, 'json': {'payload': '8440749db5a94410a7401453b4fbdad6', 'timestamp': 1719157330377}, 'url': 'https://postman-echo.com/post?key=93'}
^C2024-06-23 16:42:18.469 [INFO]: CTRL-C pressed by user!
2024-06-23 16:42:18.507 [INFO]: Closing consumer avro-deserialiser-01 (avro-deserialiser)
2024-06-23 16:42:18.532 [INFO]: Stopping parallel consumer threads
2024-06-23 16:42:18.600 [INFO]: Stopped consumer thread: 4
2024-06-23 16:42:18.600 [INFO]: Stopped consumer thread: 3
2024-06-23 16:42:18.600 [INFO]: Stopped consumer thread: 0
2024-06-23 16:42:18.600 [INFO]: Stopped consumer thread: 1
2024-06-23 16:42:18.600 [INFO]: Stopped consumer thread: 2
2024-06-23 16:42:18.600 [INFO]: All parallel consumer threads stopped

External References

• Introducing the Confluent Parallel Consumer
• How to use the Confluent Parallel Consumer
• Confluent's GitHub Repository for the Java Parallel Consumer
• Confluent Platform
• Confluent Control Center
• confluent_kafka Python library (based on librdkafka)