Skip to content

Commit

Permalink
added extension for wso2am-4.3.0 and changed folder structure
Browse files Browse the repository at this point in the history
  • Loading branch information
sevinpr committed Sep 24, 2024
1 parent 4b99abe commit 62e96f0
Show file tree
Hide file tree
Showing 45 changed files with 2,081 additions and 106 deletions.
8 changes: 0 additions & 8 deletions .idea/.gitignore

This file was deleted.

7 changes: 0 additions & 7 deletions .idea/encodings.xml

This file was deleted.

12 changes: 0 additions & 12 deletions .idea/misc.xml

This file was deleted.

6 changes: 0 additions & 6 deletions .idea/vcs.xml

This file was deleted.

76 changes: 3 additions & 73 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,77 +1,7 @@
# Treblle Data Publisher

## Overview of Extension
## Overview

The extension retrieves data from Global Synapse Handler (https://ei.docs.wso2.com/en/latest/micro-integrator/develop/customizations/creating-synapse-handlers/) in the API Gateway and creates a payload to send to Treblle. The data is added onto a queue once received and is processed by a worker thread. The worker thread sends the data asynchronously to Treblle, if the data is successfully sent, the data is removed from the queue. If the data is not successfully sent, the worker thread will attempt to send the data again, after 1 retry attempt, the event will be dropped.

## Build the source code

Execute the following command from the root directory of the project to build

```sh
mvn clean install
```

## Usage

### Configuration

- Copy the built JAR artifact and place it inside the `<gateway>/repository/components/lib` directory and start the server to load the required classes.
- Add the following configuration at the **beginning** of the `<gateway>/repository/conf/deployment.toml` to engage the API Log Handler

```toml
enabled_global_handlers = ["api_log_handler"]

[synapse_handlers]
api_log_handler.name = "TreblleHandler"
api_log_handler.class = "com.treblle.wso2publisher.handlers.APILogHandler"
```


- Add the following in `<gateway>/repository/conf/log4j2.properties` for logging purposes

> Following to enable the logs to populate under default `wso2carbon.log`. You can create a custom appender to log the entries to a separate log file.

```properties
loggers = api-log-handler, AUDIT_LOG, ...

logger.api-log-handler.name = com.treblle.wso2publisher
logger.api-log-handler.level = INFO
logger.api-log-handler.appenderRef.CARBON_LOGFILE.ref = CARBON_LOGFILE
```

- Before starting the WSO2 Server add the following environment variables.

MacOs
```
export TREBLLE_API_KEY=API-KEY
export TREBLLE_PROJECT_ID=Project-id
export TREBLLE_GATEWAY_URL="https://test.com"
export TREBLLE_QUEUE_SIZE=20000
export TREBLLE_WORKER_THREADS=1
export ADDITIONAL_MASK_KEYWORDS=testkey,Authorization,token
export TREBLLE_ENABLED_TENANT_DOMAINS=carbon.super,abc.com
```
Windows
```
set TREBLLE_API_KEY=API-KEY
set TREBLLE_PROJECT_ID=Project-id
set TREBLLE_GATEWAY_URL="https://test.com"
set ADDITIONAL_MASK_KEYWORDS=testkey,Authorization,token
set TREBLLE_QUEUE_SIZE=20000
set TREBLLE_WORKER_THREADS=1
set TREBLLE_ENABLED_TENANT_DOMAINS=carbon.super,abc.com
```

- Definitions

```
TREBLLE_API_KEY=<API Key of the Treblle Project>
TREBLLE_PROJECT_ID=<Project Id of the Treblle Project>
TREBLLE_GATEWAY_URL=<WSO2 API Manager Gateway URL>
TREBLLE_QUEUE_SIZE=<Messages queue size>
TREBLLE_WORKER_THREADS=<Number of worker threads for publishing data>
ADDITIONAL_MASK_KEYWORDS=<Masking keywords such as header names and body parameters>
TREBLLE_ENABLED_TENANT_DOMAINS<Treblle Publishing enabled tenant domains>
```
The Treblle Data Publisher extension retrieves and processes data from the [Global Synapse Handler](https://ei.docs.wso2.com/en/latest/micro-integrator/develop/customizations/creating-synapse-handlers/) in the WSO2 API Gateway. It asynchronously sends this data to Treblle for logging and monitoring. Data is queued and sent by a worker thread, with one retry allowed if the transmission fails.

This repository contains **two versions** of the extension, each with slight variations in configuration. Please refer to the specific README for the version you are interested in.
File renamed without changes.
77 changes: 77 additions & 0 deletions wso2am-3.2.0/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Treblle Data Publisher

## Overview of Extension

The extension retrieves data from Global Synapse Handler (https://ei.docs.wso2.com/en/latest/micro-integrator/develop/customizations/creating-synapse-handlers/) in the API Gateway and creates a payload to send to Treblle. The data is added onto a queue once received and is processed by a worker thread. The worker thread sends the data asynchronously to Treblle, if the data is successfully sent, the data is removed from the queue. If the data is not successfully sent, the worker thread will attempt to send the data again, after 1 retry attempt, the event will be dropped.

## Build the source code

Execute the following command from the root directory of the project to build

```sh
mvn clean install
```

## Usage

### Configuration

- Copy the built JAR artifact and place it inside the `<gateway>/repository/components/lib` directory and start the server to load the required classes.
- Add the following configuration at the **beginning** of the `<gateway>/repository/conf/deployment.toml` to engage the API Log Handler

```toml
enabled_global_handlers = ["api_log_handler"]

[synapse_handlers]
api_log_handler.name = "TreblleHandler"
api_log_handler.class = "com.treblle.wso2publisher.handlers.APILogHandler"
```


- Add the following in `<gateway>/repository/conf/log4j2.properties` for logging purposes

> Following to enable the logs to populate under default `wso2carbon.log`. You can create a custom appender to log the entries to a separate log file.

```properties
loggers = api-log-handler, AUDIT_LOG, ...

logger.api-log-handler.name = com.treblle.wso2publisher
logger.api-log-handler.level = INFO
logger.api-log-handler.appenderRef.CARBON_LOGFILE.ref = CARBON_LOGFILE
```

- Before starting the WSO2 Server add the following environment variables.

MacOs
```
export TREBLLE_API_KEY=API-KEY
export TREBLLE_PROJECT_ID=Project-id
export TREBLLE_GATEWAY_URL="https://test.com"
export TREBLLE_QUEUE_SIZE=20000
export TREBLLE_WORKER_THREADS=1
export ADDITIONAL_MASK_KEYWORDS=testkey,Authorization,token
export TREBLLE_ENABLED_TENANT_DOMAINS=carbon.super,abc.com
```
Windows
```
set TREBLLE_API_KEY=API-KEY
set TREBLLE_PROJECT_ID=Project-id
set TREBLLE_GATEWAY_URL="https://test.com"
set ADDITIONAL_MASK_KEYWORDS=testkey,Authorization,token
set TREBLLE_QUEUE_SIZE=20000
set TREBLLE_WORKER_THREADS=1
set TREBLLE_ENABLED_TENANT_DOMAINS=carbon.super,abc.com
```

- Definitions

```
TREBLLE_API_KEY=<API Key of the Treblle Project>
TREBLLE_PROJECT_ID=<Project Id of the Treblle Project>
TREBLLE_GATEWAY_URL=<WSO2 API Manager Gateway URL>
TREBLLE_QUEUE_SIZE=<Messages queue size>
TREBLLE_WORKER_THREADS=<Number of worker threads for publishing data>
ADDITIONAL_MASK_KEYWORDS=<Masking keywords such as header names and body parameters>
TREBLLE_ENABLED_TENANT_DOMAINS<Treblle Publishing enabled tenant domains>
```

File renamed without changes.
39 changes: 39 additions & 0 deletions wso2am-4.3.0/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
target/
!.mvn/wrapper/maven-wrapper.jar
!**/src/main/**/target/
!**/src/test/**/target/

### IntelliJ IDEA ###
.idea/*
.idea/modules.xml
.idea/jarRepositories.xml
.idea/compiler.xml
.idea/libraries/
*.iws
*.iml
*.ipr

### Eclipse ###
.apt_generated
.classpath
.factorypath
.project
.settings
.springBeans
.sts4-cache

### NetBeans ###
/nbproject/private/
/nbbuild/
/dist/
/nbdist/
/.nb-gradle/
build/
!**/src/main/**/build/
!**/src/test/**/build/

### VS Code ###
.vscode/

### Mac OS ###
.DS_Store
76 changes: 76 additions & 0 deletions wso2am-4.3.0/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Treblle Data Publisher

## Overview of Extension

The extension retrieves data from Global Synapse Handler (https://ei.docs.wso2.com/en/latest/micro-integrator/develop/customizations/creating-synapse-handlers/) in the API Gateway and creates a payload to send to Treblle. The data is added onto a queue once received and is processed by a worker thread. The worker thread sends the data asynchronously to Treblle, if the data is successfully sent, the data is removed from the queue. If the data is not successfully sent, the worker thread will attempt to send the data again, after 1 retry attempt, the event will be dropped.

## Build the source code

Execute the following command from the root directory of the project to build

```sh
mvn clean install
```

## Usage

### Configuration

- Copy the built JAR artifact and place it inside the `<gateway>/repository/components/lib` directory and start the server to load the required classes.
- Add the following configuration at the **beginning** of the `<gateway>/repository/conf/deployment.toml` to engage the API Log Handler

```toml
[synapse_handlers.treblle_publisher]
enabled=true
class="com.treblle.wso2publisher.handlers.APILogHandler"
```


- Add the following in `<gateway>/repository/conf/log4j2.properties` for logging purposes

> Following to enable the logs to populate under default `wso2carbon.log`. You can create a custom appender to log the entries to a separate log file.

```properties
loggers = treblle_publisher, AUDIT_LOG, ...

logger.treblle_publisher.name = com.treblle.wso2publisher
logger.treblle_publisher.level = INFO
logger.treblle_publisher.appenderRef.CARBON_LOGFILE.ref = CARBON_LOGFILE
```

- Before starting the WSO2 Server add the following environment variables.


MacOs
```
export TREBLLE_API_KEY=API-KEY
export TREBLLE_PROJECT_ID=Project-id
export TREBLLE_GATEWAY_URL="https://test.com"
export TREBLLE_QUEUE_SIZE=20000
export TREBLLE_WORKER_THREADS=1
export ADDITIONAL_MASK_KEYWORDS=testkey,Authorization,token
export TREBLLE_ENABLED_TENANT_DOMAINS=carbon.super,abc.com
```
Windows
```
set TREBLLE_API_KEY=API-KEY
set TREBLLE_PROJECT_ID=Project-id
set TREBLLE_GATEWAY_URL="https://test.com"
set ADDITIONAL_MASK_KEYWORDS=testkey,Authorization,token
set TREBLLE_QUEUE_SIZE=20000
set TREBLLE_WORKER_THREADS=1
set TREBLLE_ENABLED_TENANT_DOMAINS=carbon.super,abc.com
```

- Definitions

```
TREBLLE_API_KEY=<API Key of the Treblle Project>
TREBLLE_PROJECT_ID=<Project Id of the Treblle Project>
TREBLLE_GATEWAY_URL=<WSO2 API Manager Gateway URL>
TREBLLE_QUEUE_SIZE=<Messages queue size>
TREBLLE_WORKER_THREADS=<Number of worker threads for publishing data>
ADDITIONAL_MASK_KEYWORDS=<Masking keywords such as header names and body parameters>
TREBLLE_ENABLED_TENANT_DOMAINS<Treblle Publishing enabled tenant domains>
```

Loading

0 comments on commit 62e96f0

Please sign in to comment.