v0.12.0

This release is an effort toward better stability of the JavaScript API. As you will see in the following changelog, most of the changes are refactorings, as I envisioned in #89. I completely revamped the API, removed the old ones, and introduced better ones with a more native JavaScript syntax. These are the changes:

1. Better JS API docs with all the constants, classes, and data structures, all documented here.
2. Better logging with the ability to connect the global extension logger to the Reader and/or Writer objects (of the kafka-go library) to print the internal errors by using the connectLogger parameter.
3. Better error handling by throwing exceptions (native to JS) rather than returning them (native to Go), so you can enclose them in try/catch blocks.
4. All constants are now available for import at the module level and there are lots of them for different purposes.
5. The writer and reader functions are removed, and replaced by classes that can be instantiated like this:

   import { Writer, Reader } from "k6/x/kafka";
   
   const writer = new Writer({
     brokers: ["localhost:9092"],
     topic: "my-topic",
     autoCreateTopic: true,
   });
   
   const reader = new Reader({
     brokers: ["localhost:9092"],
     topic: "my-topic",
   });

6. The produce and consume functions are now methods of their respective Writer and Reader classes:

   writer.produce({
     messages: [
       {
         key: "key",
         value: "value",
       }
     ]
   });
   
   const messages = reader.consume({limit: 10});

7. Constructors' and methods' parameters are now JSON objects, mostly.
8. All the kafka-go's WriterConfig and ReaderConfig are now combined and consolidated, and they are available as parameters to the Writer and Reader objects, so everything is customizable now. Only some of them are tested, though. So, feel free to open an issue if you find any.
9. Topic-related functions are now exposed in a Connection class. You need to instantiate the class in the init context and use its methods for listing, creating, and deleting topics.

   import { Connection } from "k6/x/kafka";
   
   const connection = new Connection({
     address: "localhost:9092"
   });
   
   connection.listTopics();
   connection.createTopic({ topic: "my-topic" });
   connection.deleteTopic("my-topic");

10. All the scripts are now updated with the new syntax.
11. A few bugs are fixed and dependencies are updated.

I hope you like the new changes and please report issues if you find any.

What's Changed

• Refactor logging by @mostafa in #90
• Export all constants to JS code by @mostafa in #91
• Export a proper constructor by @mostafa in #92
• Throw an exception instead of returning an error by @mostafa in #93
• Call a method on an object, instead of passing the object to a function by @mostafa in #95
• Pass constructor arguments as a single JSON object by @mostafa in #96
• Consolidate options by @mostafa in #97
• Create Connection class and methods for creating, deleting, and listing topics by @mostafa in #98
• Update JS API docs by @mostafa in #99
• Fix script errors by adding connection.createTopic by @mostafa in #100
• Fix header retrieval when consuming messages by @mostafa in #101
• Ignore noTLSConfig error code, because it is polluting the output log by @mostafa in #103
• Update README for the big release by @mostafa in #102
• Update dependencies by @mostafa in #104
• Add a new logo and update README by @mostafa in 5e2b4ab 37ebfb5 2ddfa1d

Full Changelog: v0.11.0...v0.12.0

v0.11.0

This release includes several changes and fixes to how the extension handles SASL and TLS configurations. It also includes a feature by @enamrik 🙏 that lets users choose their subject name strategies for naming schemas in Schema Registry. Details of what happened are available on the respective PRs.

Previously the JS API docs were in the README, but after merging #87, the docs are available separately. The docs always refer to the latest changes on the main branch, as explained here.

The release process and branching are also changed.

What's Changed

• Fix SASL and TLS issues reported in #56 and #84 (and possibly #67) by @mostafa in #86
• Add support for using different subject name strategies by @enamrik in #88
• Add API docs with typedoc by @mostafa in #87
• Lots of changes to READMEs by @mostafa in d165b4d 98102b7 7a76adb a611a18 82e8185 5036e83 6df4449

New Contributors

• @enamrik made their first contribution in #88

Full Changelog: v0.10.0...v0.11.0

v0.10.0

In this release, I refactored almost everything. Significant changes include heavy refactoring, adding tests, and fixing bugs reported by users and contributors, which I am grateful for 🙏. I also added docstrings and comments wherever I could.

You might also notice that old tags are stale because I mistakenly committed a binary file and then wanted to remove it from the history, which resulted in overwriting the history and losing some bits and pieces 🤦. Hopefully, the commit logs are intact. The lesson learned here is NEVER to USE bfg-repo-cleaner ever again. ⚠️

Also, I've mentioned in the docs that I won't guarantee backward compatibility in APIs, so please update your scripts to reflect the latest changes or things will start to fail. You can also stick with the old v0.8.0 and Docker image if you don't want the latest changes, as mentioned in #56.

What's Changed

• Add headers to produced and consumed messages by @mostafa in #46

• Use a proper Schema Registry client to fix issues in #22 by @mostafa in #52

• Implement JSONSchema serdes by @mostafa in #59

• Add missing Avro serializer and deserializer URIs to SchemaType mapping by @hildebrandttk in #62

• Revamp logging and error reporting by @mostafa in #64

• Add tests by @mostafa in #60

• Add TLS configuration to Schema Registry client by @mostafa in #69 and 88a9ba7

• Add trivy scanner by @mostafa in #72

• Export all metrics by @mostafa in #75

• Add docstring and comment to everything by @mostafa in #77

• Fix: get controller connection in Kafka cluster for creating a topic by @mostafa in #82

• Lots of fixes and changes by @mostafa in 53d0e68 2310ba1 a2ed7d9 488821c 8e1a6cf 7b8fa5e a5c402f 23d41ab c5304b2 3d06ffb 3d06ffb 4e866e5 e2b4606 af27ed0 eaa0181 b28e3ff be9412e fe35b61

Full Changelog: 832c83b...main

v0.9.0

What's Changed

• Use SSL client certs for authentication by @hildebrandttk in #31
• Switch to k6 v0.38.0 by @mostafa in 7192e1e
• Test xk6-kafka with Apache Kafka (Zookeeperless) (part of the CI pipeline) by @mostafa in #40
• Add linters and CycloneDX SBOM generation by @mostafa in #42
• Find and fix deprecated APIs in k6 by @mostafa in #41
• Grooming of the README here and there by @mostafa in d1985e2, 327c8b1, 3516c93, f8794cd, 8a9d391 and ca59839
• Update dependencies to the latest versions by @mostafa in 1d8cecc
• Refactor the code to use the new k6 API for metrics registry by @mostafa in 467ca40 and 3e3bd84
• Use TLS v1.2 as the minimum acceptable version in TLS config by @mostafa in a773097
• Tiny fix to the build pipeline to build the latest pushed tag by @mostafa in 37cc466

New Contributors

• @hildebrandttk made their first contribution in #31

Full Changelog: v0.8.0...v0.9.0

v0.8.0

What's Changed

• Add string and byte serializer by @thmshmm in #34
• Add CI workflow by @mostafa in #39 for building binaries (see attached files) and a Docker image
• Revamp README by @mostafa

New Contributors

• @thmshmm made their first contribution in #34

Full Changelog: v0.7.0...v0.8.0

Allow producing Kafka message without a key

In this release, a few things have changed:

1. The project is re-licensed to Apache 2.0, so it can be run on the cloud.
2. Thanks to @eduardowitter, PR #30 is merged that allow producing Kafka messages with no keys.
3. README is updated with some useful troubleshooting instructions.
4. A blog article is published on k6 blog to demonstrate the capabilities of the extension.

Kafka message compression support

This release adds support for message compression in Kafka. The supported compression of messages written to and read from Kafka using these compression codec:

• Gzip
• Snappy
• Lz4
• Zstd

There's a script that shows how to use the compression feature. The createTopic function is also updated to create topics and set their compression config to the desired value.

Support for Confluent's KafkaAvroSerializer/KafkaAvroDeSerializer

In this release, basic support for Confluent's KafkaAvroSerializer/KafkaAvroDeSerializer is added by @fmck3516 (#9). This helps support the Confluent wire-format for Kafka.

Also, now that there are 4 different test scripts, they're moved to script/ directory.

Bugfix release

This release contains a fix (#11) by @fmck3516 that fixes an issue (#10) with TLS config on the writer.

Feature: SASL Auth (PLAIN and SCRAM)

In this release I've added the SASL authentication feature with PLAIN and SCRAM mechanisms (no SSL yet). I also fixed some bugs and added three separate scripts that shows how to use the extension with Avro and JSON formats and how to authenticate using SASL SCRAM (sha256).

I'd be very happy to hear your feedback.