Skip to content

Commit

Permalink
Merge pull request #144 from shreelakshmijoshi/project-d
Browse files Browse the repository at this point in the history 
Update files
  • Loading branch information
@shreelakshmijoshi
shreelakshmijoshi authored Aug 13, 2024
2 parents a36e828 + 39d1d1d commit d88bc17
Show file tree
Hide file tree
Showing 8 changed files with 72 additions and 44 deletions.
3 changes: 3 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,9 @@ local.properties
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*

# nvm repository
/.nvm/

### Maven ###
target/
pom.xml.tag
Expand Down
10 changes: 10 additions & 0 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,13 @@ We follow Git Merge based workflow
1. Fork this repo
2. Create a new feature branch in your fork. Multiple features must have a hyphen separated name
3. Commit to your fork and raise a Pull Request with upstream

## Code Style
- To maintain a consistent code style maven checkstyle plugin is used : [reference link](https://maven.apache.org/plugins/maven-checkstyle-plugin/index.html)
- To inspect, analyze the code to remove common programming defects,
inculcate programming best practices PMD is being used : [reference link](https://maven.apache.org/plugins/maven-pmd-plugin/)
- The following maven command is used to generate PMD, checkstyle and Copy/Paste Detector (CPD) reports in `./target/site` folder
<br> ```mvn checkstyle:checkstyle pmd:pmd pmd:cpd```
- To resolve checkstyle issues faster please install any of the following plugins in the IDE like IntelliJ
- google-java-format : [link](https://github.com/google/google-java-format?tab=readme-ov-file)
- CheckStyle-IDEA : [link](https://plugins.jetbrains.com/plugin/1065-checkstyle-idea)
20 changes: 8 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,16 @@ allow consumer, consumer delegates to access the resource.
</p>

## Tokens, Users and Roles
- Shall be available in the docs page
## Tokens used in DX
### Tokens used in DX
- Tokens for a user could be created using DX AAA Server API : [link to the API docs](https://authorization.iudx.org.in/apis#tag/Token-APIs/operation/post-auth-v1-token). The token used in DX are :

| Token | Purpose | Users |
|:------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------|
| Identity token | | Provider, provider delegate, consumer, consumer delegate, cos admin, admin |
| Bearer token | Access token as bearer credential is generated by Keycloak by providing the client's email ID and password and then adding bearer to the access token : `bearer <access-token>` | DX AAA Server, Users |
| Access token | To get access to resource, resource group | Provider, provider delegate, consumer, consumer delegate |
| Token | Purpose | Users |
|:------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------|
| Identity token | Serves as an identifier of the user to the server to access the ACL-APD Server's capabilities | Provider, provider delegate, consumer, consumer delegate, cos admin, admin |
| Bearer token | Access token as bearer credential is generated by Keycloak by providing the client's email ID and password and then adding bearer to the access token : `bearer <access-token>` | DX AAA Server, Users |
| Access token | To get access to resource, resource group | Provider, provider delegate, consumer, consumer delegate |

## Tokens accepted in ACL APD Server
### Tokens accepted in ACL APD Server
Identity token, bearer token is used in ACL-APD. Identity token is used for user specific APIs, bearer token is used for Verify API call.
Provider, Consumers and delegates of provider and consumers are allowed to access the following APIs:

Expand Down Expand Up @@ -130,10 +129,7 @@ API docs are available [here](https://acl-apd.iudx.org.in/apis)
FAQs are available [here](https://github.com/datakaveri/iudx-acl-apd/blob/main/docs/FAQ.md)

## Contributing
We follow Git Merge based workflow
1. Fork this repo
2. Create a new feature branch in your fork. Multiple features must have a hyphen separated name
3. Commit to your fork and raise a Pull Request with upstream
Please find information about contributing [here](https://github.com/datakaveri/iudx-acl-apd/blob/main/CONTRIBUTING.md)

## License
[View License](./LICENSE)
Expand Down
21 changes: 9 additions & 12 deletions docs/Introduction.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,16 @@ allow consumer, consumer delegates to access the resource.
</p>

## Tokens, Users and Roles
- Shall be available in the docs page
## Tokens used in DX
### Tokens used in DX
- Tokens for a user could be created using DX AAA Server API : [link to the API docs](https://authorization.iudx.org.in/apis#tag/Token-APIs/operation/post-auth-v1-token). The token used in DX are :

| Token | Purpose | Users |
|:------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------|
| Identity token | | Provider, provider delegate, consumer, consumer delegate, cos admin, admin |
| Bearer token | Access token as bearer credential is generated by Keycloak by providing the client's email ID and password and then adding bearer to the access token : `bearer <access-token>` | DX AAA Server, Users |
| Access token | To get access to resource, resource group | Provider, provider delegate, consumer, consumer delegate |
| Token | Purpose | Users |
|:------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------|
| Identity token | Serves as an identifier of the user to the server to access the ACL-APD Server's capabilities | Provider, provider delegate, consumer, consumer delegate, cos admin, admin |
| Bearer token | Access token as bearer credential is generated by Keycloak by providing the client's email ID and password and then adding bearer to the access token : `bearer <access-token>` | DX AAA Server, Users |
| Access token | To get access to resource, resource group | Provider, provider delegate, consumer, consumer delegate |

## Tokens accepted in ACL APD Server
### Tokens accepted in ACL APD Server
Identity token, bearer token is used in ACL-APD. Identity token is used for user specific APIs, bearer token is used for Verify API call.
Provider, Consumers and delegates of provider and consumers are allowed to access the following APIs:

Expand Down Expand Up @@ -130,10 +129,8 @@ API docs are available [here](https://acl-apd.iudx.org.in/apis)
FAQs are available [here](https://github.com/datakaveri/iudx-acl-apd/blob/main/docs/FAQ.md)

## Contributing
We follow Git Merge based workflow
1. Fork this repo
2. Create a new feature branch in your fork. Multiple features must have a hyphen separated name
3. Commit to your fork and raise a Pull Request with upstream
Please find information about contributing [here](https://github.com/datakaveri/iudx-acl-apd/blob/main/CONTRIBUTING.md)


## License
[View License](./LICENSE)
Expand Down
59 changes: 40 additions & 19 deletions docs/SETUP-and-Installation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ could be updated in configs. Please refer [Configurations](https://github.com/da

## Dependencies
### External
| Software Name | Purpose |
|:--------------|:-------------------------------------------------------------------------------------------------------------------------------|
| PostgreSQL | For storing information related to policy, access Request based CRUD operations, approved access requests, resources and users |
| RabbitMQ | To publish auditing related data to auditing server via RMQ exchange |
| Software Name | Purpose |
|:-----------------|:---------------------------------------------------------------------------------------------------------------------------------|
| PostgreSQL | For storing information related to policy, access Request based CRUD operations, approved access requests, resources and users |
| RabbitMQ | To publish auditing related data to auditing server via RMQ exchange |
| SMTP Mail Server | To send email notifications to provider, provider delegates when access requests are created by the consumer, consumer delegates |


### Other Dependencies
Expand Down Expand Up @@ -75,10 +76,8 @@ ACL-APD user could have write permission as it publishes audit data


#### PostgresQL

- To setup PostgreSQL refer the docker files available [here](https://github.com/datakaveri/iudx-deployment/blob/master/Docker-Swarm-deployment/single-node/postgres)
- **Note** : PostgreSQL database should be configured with a RBAC user having CRUD privileges
- Schemas for PostgreSQL tables are present here - [Flyway schema](https://github.com/datakaveri/iudx-acl-apd/tree/main/src/main/resources/db/migration)

| Table Name | Purpose |
|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Expand All @@ -88,15 +87,28 @@ ACL-APD user could have write permission as it publishes audit data
| request | To store the access request related information whenever an access request is created by a consumer to request provider to create policy for a resource / resource group |
| approved_access_requests | To store approved notifications when the provider sets the notification status to granted inorder to create policy |

#### PostgresQL
#### Auditing
- Auditing is done using Immudb and Postgres DB
- To Setup immuclient for immudb please refer [here](https://github.com/datakaveri/iudx-deployment/tree/master/docs/immudb-setup)
- Schema for PostgreSQL table is present [here](https://github.com/datakaveri/iudx-resource-server/blob/master/src/main/resources/db/migration/V5_2__create-auditing-acl-apd-table.sql)
- Schema for Immudb table, index for the table is present [here](https://github.com/datakaveri/auditing-server/tree/main/src/main/resources/immudb/migration)
- To Setup immuclient for immudb please refer [immudb setup guide](https://github.com/datakaveri/iudx-deployment/tree/master/docs/immudb-setup)
- Schema for Auditing table in PostgreSQL is present here - [postgres auditing table schema](https://github.com/datakaveri/iudx-resource-server/blob/master/src/main/resources/db/migration/V5_2__create-auditing-acl-apd-table.sql)
- Schema for Immudb table, index for the table is present here - [immudb schema in DX Auditing Server](https://github.com/datakaveri/auditing-server/tree/main/src/main/resources/immudb/migration)

| Table Name | Purpose | DB |
|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|
| auditing_acl_apd | To store logged information about endpoint, caller of the endpoint, timestamp when the POST, DELETE, PUT requests respond with 200 success response | Immudb, PostgreSQL |

- User ID, endpoint and epoch time are indexed in Immudb to retrieve the logs faster

## To Replay Recording
1. We use [asciinema](https://docs.asciinema.org/) as a tool to share the terminal recordings
2. To download the tool, please visit asciinema's get started guide : [link](https://docs.asciinema.org/getting-started/)
#### Flyway Migrations
- Database flyway migrations help in updating the schema, permissions, grants, triggers etc., with the latest version
- Each flyway schema file is versioned with the format `V<number>_<number>__file-name.sql`, ex : `V1_1__init-tables.sql`
- Schemas for PostgreSQL tables are present here - [Flyway schema](https://github.com/datakaveri/iudx-acl-apd/tree/main/src/main/resources/db/migration)
- Values like DB URL, database user credentials, user and schema name could be populated in flyway.conf
- The following commands could be executed
- ``` mvn flyway:info -Dflyway.configFiles=flyway.conf``` To get the flyway schema history table
- ``` mvn flyway:migrate -Dflyway.configFiles=flyway.conf ``` To migrate flyway schema
- ``` mvn flyway:repair ``` To resolve some migration errors during flyway migration
- Please find the reference to Flyway migration [here](https://documentation.red-gate.com/fd/migrations-184127470.html)

## Installation Steps
### Maven
Expand Down Expand Up @@ -169,16 +181,24 @@ $ java ACL_APD_JAVA_OPTS -jar target/iudx.iudx.apd.acl.server-cluster-0.0.1-SNAP
`mvn clean compile exec:java@acl-apd-server`

## Logging and Monitoring
### Log4j 2
- For asynchronous logging, logging messages to the console in a specific format, Apache log4j 2 is used
- For log formatting, adding appenders, adding custom logs, setting log levels, log4j2.xml could be updated : [link](https://github.com/datakaveri/iudx-acl-apd/blob/main/src/main/resources/log4j2.xml)
- Please find the reference to [link](https://logging.apache.org/log4j/2.x/manual/index.html)

### Micrometer
- #Metrics collection

## Testing
### Unit Testing
1. Run the server through either docker, maven or redeployer
2. Run the unit tests and generate a surefire report
`mvn clean test-compile surefire:test surefire-report:report`
3. Jacoco reports are stored in `./target/`
4. A sample recording to execute unit test is available [here](https://github.com/datakaveri/iudx-acl-apd/blob/main/unitTest.cast) and could be replayed using:
`asciinema play unitTest.cast`

<br>
Here is a sample recording to execute unit test
![](../example-tutorials/unitTest.gif)
![](../example-tutorials/unitTest.svg)


### Integration Testing
Expand All @@ -196,9 +216,10 @@ Integration tests are through Postman/Newman whose script can be found from [her


### Performance Testing

- JMeter
Explain how to execute tests. #TODO

### Security Testing

Explain how to execute tests. #TODO
- For security testing, Zed Attack Proxy(ZAP) Scanning is done to discover security risks, vulnerabilities to help us address them
- A report is generated to show vulnerabilities as high risk, medium risk, low risk and false positive
- Please find the reference to ZAP : [here](https://www.zaproxy.org/getting-started/)
Binary file added example-tutorials/unitTest.gif
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions example-tutorials/unitTest.svg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion src/main/resources/log4j2.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
</Console>
<!-- <RollingFile name="RollingFile" append="false" ignoreExceptions="false">
<FileName>/tmp/iudx/rs.log</FileName>
<FilePattern>/tmp/iudx/rs.log</FilePattern>
<FilePattern>/tmp/iudx/acl-apd.log</FilePattern>
<JSONLayout compact="true" eventEol="true" stacktraceAsString="true" includeTimeMillis="true">
<KeyValuePair key="source" value="rs"/>
<KeyValuePair key="sourceURL" value="$${env:RS_URL}"/>
Expand Down

0 comments on commit d88bc17

Please sign in to comment.