Skip to content

Commit

Permalink
Readme update
Browse files Browse the repository at this point in the history
  • Loading branch information
Mongey committed Jun 3, 2018
1 parent 637dff4 commit 58ad619
Show file tree
Hide file tree
Showing 24 changed files with 143 additions and 34 deletions.
File renamed without changes.
8 changes: 4 additions & 4 deletions GNUmakefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ GOFMT_FILES?=$$(find . -name '*.go' |grep -v vendor)
default: build

build:
go install
go build .

test:
go test -i $(TEST) || exit 1
Expand All @@ -12,9 +12,9 @@ test:

testacc:
KAFKA_BOOTSTRAP_SERVER=localhost:9092 \
KAFKA_CACERT=../ssl-ffs/secrets/snakeoil-ca-1.crt \
KAFKA_CLIENT_CERT=../ssl-ffs/secrets/kafkacat-ca1-signed.pem \
KAFKA_CLIENT_KEY=../ssl-ffs/secrets/kafkacat-raw-private-key.pem \
KAFKA_CACERT=../secrets/snakeoil-ca-1.crt \
KAFKA_CLIENT_CERT=../secrets/kafkacat-ca1-signed.pem \
KAFKA_CLIENT_KEY=../secrets/kafkacat-raw-private-key.pem \
KAFKA_SKIP_VERIFY=true \
KAFKA_ENABLE_TLS=true \
TF_ACC=1 go test $(TEST) -v $(TESTARGS) -timeout 120m
Expand Down
135 changes: 115 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,52 @@
# `terraform-provider-kafka`
[![CircleCI](https://circleci.com/gh/Mongey/terraform-provider-kafka.svg?style=svg)](https://circleci.com/gh/Mongey/terraform-provider-kafka)

A [Terraform][1] plugin for managing [Apache Kafka][2].

[![CircleCI](https://circleci.com/gh/Mongey/terraform-provider-kafka.svg?style=svg)](https://circleci.com/gh/Mongey/terraform-provider-kafka)
## Contents

# Requirements
* [Kafka 1.0.0][3]
* [Installation](#installation)
* [Developing](#developing)
* [`kafka` Provider](#provider-configuration)
* [Resources](#resources)
* [`kafka_topic`](#kafka_topic)
* [`kafka_acl`](#kafka_acl)
* [Requirements](#requirements)

## Installation

Download and extract the [latest
release](https://github.com/Mongey/terraform-provider-kafka/releases/latest) to
your [terraform plugin directory][third-party-plugins] (typically `~/.terraform.d/plugins/`)

### Developing

# Example
0. [Install go][install-go]
0. Clone repository to: `$GOPATH/src/github.com/Mongey/terraform-provider-kafka`
``` bash
mkdir -p $GOPATH/src/github.com/Mongey/terraform-provider-kafka; cd $GOPATH/src/github.com/Mongey/
git clone https://github.com/Mongey/terraform-provider-kafka.git
```
0. Build the provider `make build`
0. Run the tests `make test`
0. Start a TLS enabled kafka-cluster `docker-compose up`
0. Run the acceptance tests `make testacc`

## Provider Configuration

### Example

Example provider with SSL client authentication.
```hcl
provider "kafka" {
bootstrap_servers = ["localhost:9092"]
}
resource "kafka_topic" "logs" {
name = "systemd_logs"
replication_factor = 2
partitions = 100
config = {
"segment.ms" = "20000"
}
ca_cert_file = "../secrets/snakeoil-ca-1.crt"
client_cert_file = "../secrets/kafkacat-ca1-signed.pem"
client_key_file = "../secrets/kafkacat-raw-private-key.pem"
skip_tls_verify = true
}
```

# Provider Configuration
| Property | Description | Default |
| ---------------- | ----------------------- | ---------- |
| `bootstrap_servers` | A list of host:port addresses that will be used to discover the full set of alive brokers | `Required` |
Expand All @@ -35,19 +58,91 @@ resource "kafka_topic" "logs" {
| `sasl_username` | Username for SASL authentication. | `""` |
| `sasl_password` | Password for SASL authentication. | `""` |
# Importing Existing Topics
## Resources
### `kafka_topic`
A resource for managing Kafka topics. Increases partition count without
destroying the topic.
#### Example
```hcl
provider "kafka" {
bootstrap_servers = ["localhost:9092"]
}
resource "kafka_topic" "logs" {
name = "systemd_logs"
replication_factor = 2
partitions = 100
config = {
"segment.ms" = "20000"
"cleanup.policy" = "compact"
}
}
```
#### Properties
| Property | Description |
| ---------------- | ----------------------- |
| `name` | The name of the topic |
| `paritions` | The number of partitions the topic should have |
| `replication_factor` | The number of replicas the topic should have |
| `config` | A map of string k/v attributes |
#### Importing Existing Topics
You can import topics with the following
```sh
terraform import kafka_topic.logs systemd_logs
```
# Resources
* kafka_topic
# Planned Resources
* kafka_acl
### `kafka_acl`
A resource for managing Kafka ACLs.
#### Example
```hcl
provider "kafka" {
bootstrap_servers = ["localhost:9092"]
ca_cert_file = "../secrets/snakeoil-ca-1.crt"
client_cert_file = "../secrets/kafkacat-ca1-signed.pem"
client_key_file = "../secrets/kafkacat-raw-private-key.pem"
skip_tls_verify = true
}
resource "kafka_acl" "test" {
resource_name = "syslog"
resource_type = "Topic"
acl_principal = "User:Alice"
acl_host = "*"
acl_operation = "Write"
acl_permission_type = "Deny"
}
```
#### Properties
| Property | Description | Valid values |
| ---------------- | ----------------------- | -------------- |
| `acl_host` | A map of string k/v attributes | `*` |
| `acl_operation` | A map of string k/v attributes | `Unknown`, `Any`, `All`, `Read`, `Write`, `Create`, `Delete`, `Alter`, `Describe`, `ClusterAction`, `DescribeConfigs`, `AlterConfigs`, `IdempotentWrite` |
| `acl_permission_type` | A map of string k/v attributes | `Unknown`, `Any`, `Allow`, `Deny` |
| `acl_principal` | The number of replicas the topic should have | `*` |
| `resource_name` | The name of the resource | `*` |
| `resource_type` | The type of resource | `Unknown`, `Any`, `Topic`, `Group`, `Cluster`, `TransactionalID` |
## Requirements
* [Kafka 1.0.0][3]
[1]: https://www.terraform.io
[2]: https://kafka.apache.org
[3]: https://cwiki.apache.org/confluence/display/KAFKA/KIP-117%3A+Add+a+public+AdminClient+API+for+Kafka+admin+operations
[third-party-plugins]: https://www.terraform.io/docs/configuration/providers.html#third-party-plugins
[install-go]: https://golang.org/doc/install#install
File renamed without changes.
14 changes: 14 additions & 0 deletions examples/main.tf
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
provider "kafka" {
bootstrap_servers = ["localhost:9092"]
ca_cert_file = "../ssl-ffs/secrets/snakeoil-ca-1.crt"
client_cert_file = "../ssl-ffs/secrets/kafkacat-ca1-signed.pem"
client_key_file = "../ssl-ffs/secrets/kafkacat-raw-private-key.pem"
tls_enabled = true
skip_tls_verify = true
}

resource "kafka_topic" "syslog" {
Expand All @@ -12,3 +17,12 @@ resource "kafka_topic" "syslog" {
"retention.ms" = "86400000"
}
}

resource "kafka_acl" "test" {
resource_name = "syslog"
resource_type = "Topic"
acl_principal = "User:Alice"
acl_host = "*"
acl_operation = "Write"
acl_permission_type = "Deny"
}
12 changes: 6 additions & 6 deletions kafka/resource_kafka_acl_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,10 +92,10 @@ func testResourceACL_updateCheck(s *terraform.State) error {
const testResourceACL_initialConfig = `
provider "kafka" {
bootstrap_servers = ["localhost:9092"]
ca_cert_file = "../ssl-ffs/secrets/snakeoil-ca-1.crt"
ca_cert_file = "../secrets/snakeoil-ca-1.crt"
client_cert_file = "../secrets/kafkacat-ca1-signed.pem"
client_key_file = "../secrets/kafkacat-raw-private-key.pem"
skip_tls_verify = true
client_cert_file = "../ssl-ffs/secrets/kafkacat-ca1-signed.pem"
client_key_file = "../ssl-ffs/secrets/kafkacat-raw-private-key.pem"
}
resource "kafka_acl" "test" {
Expand All @@ -111,10 +111,10 @@ resource "kafka_acl" "test" {
const testResourceACL_updateConfig = `
provider "kafka" {
bootstrap_servers = ["localhost:9092"]
ca_cert_file = "../ssl-ffs/secrets/snakeoil-ca-1.crt"
ca_cert_file = "../secrets/snakeoil-ca-1.crt"
client_cert_file = "../secrets/kafkacat-ca1-signed.pem"
client_key_file = "../secrets/kafkacat-raw-private-key.pem"
skip_tls_verify = true
client_cert_file = "../ssl-ffs/secrets/kafkacat-ca1-signed.pem"
client_key_file = "../ssl-ffs/secrets/kafkacat-raw-private-key.pem"
}
resource "kafka_acl" "test" {
Expand Down
8 changes: 4 additions & 4 deletions kafka/resource_kafka_topic.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,24 +24,24 @@ func kafkaTopicResource() *schema.Resource {
Type: schema.TypeString,
Required: true,
ForceNew: true,
Description: "The name of the topic",
Description: "The name of the topic.",
},
"partitions": {
Type: schema.TypeInt,
Required: true,
Description: "number of partitions",
Description: "Number of partitions.",
},
"replication_factor": {
Type: schema.TypeInt,
Required: true,
ForceNew: true,
Description: "number of replicas",
Description: "Number of replicas.",
},
"config": {
Type: schema.TypeMap,
Optional: true,
ForceNew: false,
Description: "A map of string k/v attributes",
Description: "A map of string k/v attributes.",
},
},
}
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 comments on commit 58ad619

Please sign in to comment.