Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Documentation for Catalogue Server: Expanded Setup and Usage Guide #389

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
<p align="center">
<img src="./docs/cdpg.png" width="300">
</p>

# Contributor code of conduct

Trust, respect, collaboration and transparency are core
values we believe should live and breathe within our projects. Our community welcomes participants from around the world with different experiences, unique perspectives, and great ideas to share.
When you're working with members of the community, this Code of Conduct will help steer your interactions and keep CDPG a positive, successful, and growing community.


## Our pledge
The CDPG as an open-source community is **open**, **kind**, **inquisitive**, **respectful**

- **Being open**: Members of the community are open to collaboration
- **Focusing on what is best for the community**: We're respectful of the processes set forth in the community, and we work within them
- **Acknowledging time and effort**: We're respectful of the volunteer efforts that permeate the CDPG community and we're thoughtful when addressing the efforts of others, keeping in mind that often times the labor was completed simply for the good of the community
- **Being respectful of differing viewpoints and experiences**. We're receptive to constructive comments and criticism, as the experiences and skill sets of other members contribute to the whole of our efforts
- **Showing empathy towards other community members**. We're attentive in our communications, whether in person or online, and we're tactful when approaching differing views
- **Being considerate**. Members of the community are considerate of their peers
- **Being respectful**. We're respectful of others, their positions, their skills, their commitments, and their efforts
- **Gracefully accepting constructive criticism**. When we disagree, we are courteous in raising our issues
- **Using welcoming and inclusive language**. We're accepting of all who wish to take part in our activities, fostering an environment where anyone can participate and everyone can make a difference

## Our standards
This Code of Conduct will help inform your interactions and reinforce values that contribute to a positive environment

### Expected behavior
- Be welcoming. Use inclusive language
- Be respectful of differing viewpoints and experiences
- Be receptive to constructive comments and criticism
- Show empathy towards other community members
- Be respectful of others' positions, skills, commitments, and efforts

### Inappropriate behavior
- Trolling, insulting/derogatory comments, and personal or political attacks
- The use of sexualized language or imagery. Unwelcome sexual attention or advances
- Language or imagery that encourages, glorifies, incites, or calls for violence, emotional, or physical harm against an individual or a group of people
- Public or private harassment
- Personal insults, especially those using racist, ableist, ageist, homophobic, or sexist terms
- Defaming an individual or group, or violating any trademarks or copyrighted material
- Publishing private or identifying information or non-harassing communication without explicit permission
- Dismissing or attacking inclusion-oriented requests
- Continual disruption of talks, workshops, or other events
- Offensive comments related to race/ethnicity or national origin, sex or gender, gender identity and expression, sexual orientation, disability, physical appearance, mental illness or neurotypicality, body size, caste, age, or religion
- Threats or acts of violence
- Deliberate intimidation
- Encouraging any of the above behavior

### Reporting the behavior
If you believe that someone is violating the code of conduct, or have any other concerns, please contact
our team immediately by sending an email to [email protected]. All reports will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances.


## Attribution
This Code of Conduct is adapted from the [Opensource.com](https://opensource.com/code-of-conduct), [Python code of conduct](https://policies.python.org/python.org/code-of-conduct/).
41 changes: 41 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
<p align="center">
<img src="./docs/cdpg.png" width="300">
</p>

We follow Git Merge based workflow
1. Fork this repo
2. Create a new feature branch in your fork. Multiple features must have a hyphen separated name
3. Commit to your fork and raise a Pull Request with upstream

## Code Style
- To maintain a consistent code style maven checkstyle plugin is used : [reference link](https://maven.apache.org/plugins/maven-checkstyle-plugin/index.html)
- To inspect, analyze the code to remove common programming defects,
inculcate programming best practices PMD is being used : [reference link](https://maven.apache.org/plugins/maven-pmd-plugin/)
- The following maven command is used to generate PMD, checkstyle and Copy/Paste Detector (CPD) reports in `./target/site` folder
<br> ```mvn checkstyle:checkstyle pmd:pmd pmd:cpd```
- To resolve checkstyle issues faster please install any of the following plugins in the IDE like IntelliJ
- google-java-format : [link](https://github.com/google/google-java-format?tab=readme-ov-file)
- CheckStyle-IDEA : [link](https://plugins.jetbrains.com/plugin/1065-checkstyle-idea)

## About commit message
Git commit command is used to save the changes done in the local repository after staging in git.
To add the files in staging
```
git add <file>
```
To commit the file
```
git commit -m "<subject>" -m "<description>"
```

### Commit subject
- The commit subject could be no more than 50 characters
- To maintain consistent commit messages, the subject could be in imperative tone or as a
verb. Example : `Change query` rather than `Changes made in the query`, `Changed the query`
<br> This could also reduce the size of subject
- Could contain one of [feat, fix, style, refactor, test, docs, chore]

### Commit description
- Could be detailed to explain the changes made but no more than 72 characters
- Punctuation could be omitted at the end of commit description
- Could also reference an issue or pull request
93 changes: 18 additions & 75 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@
[![Security Tests](https://img.shields.io/jenkins/build?jobUrl=https%3A%2F%2Fjenkins.iudx.io%2Fjob%2Fiudx%2520catalogue%2520%28master%29%2520pipeline%2F&label=security%20tests)](https://jenkins.iudx.io/job/iudx%20catalogue%20(master)%20pipeline/lastBuild/zap/)
[![Integration Tests](https://img.shields.io/jenkins/build?jobUrl=https%3A%2F%2Fjenkins.iudx.io%2Fjob%2Fiudx%2520catalogue%2520%28master%29%2520pipeline%2F&label=integration%20tests)](https://jenkins.iudx.io/job/iudx%20catalogue%20(master)%20pipeline/lastBuild/Integration_20Test_20Report/)

![IUDX](./docs/iudx.png)
# iudx-catalogue-server
The catalogue is [IUDXs](https://iudx.org.in) data discovery and dataset metadata publishing portal.
![IUDX](./docs/cdpg.png)
# DX Catalogue Server
## Introduction
The catalogue is [DXs](https://iudx.org.in) data discovery and dataset metadata publishing portal.
It allows data providers to publish their data *resources* by making an IUDX vocabulary annotated meta-data document describing their datasource and affiliated terminologies.
The datasources publish their data to the IUDX *Resource Server*.
It allows consumers of such data to easily discover such *resources* by means of powerful
Expand All @@ -28,81 +29,23 @@ queries and consume the data from *Resource Servers* in an automated and machine
- Scalable, service mesh architecture based implementation using open source components: Vert.X API framework, Elasticsearch for data-base
- Hazelcast and Zookeeper based cluster management and service discovery


## Live
The live running instance of the IUDX catalogue can be found [here](https://catalogue.iudx.org.in).

## API Docs
The api docs can be found [here](https://catalogue.iudx.org.in/apis).



## Get Started

### Prerequisite - Make configuration
Make a config file based on the template in `./configs/config-example.json`
- Generate a certificate using Lets Encrypt or other methods
- Make a Java Keystore File and mention its path and password in the appropriate sections
- Modify the database url and associated credentials in the appropriate sections

### Docker based
1. Install docker and docker-compose
2. Clone this repo
3. Build the images
` ./docker/build.sh`
4. Modify the `docker-compose.yml` file to map the config file you just created
5. Start the server in production (prod) or development (dev) mode using docker-compose
` docker-compose up prod `


### Maven based
1. Install java 11 and maven
2. Use the maven exec plugin based starter to start the server
`mvn clean compile exec:java@catalogue-server`

### Redeployer
A hot-swappable redeployer is provided for quick development
`./redeploy.sh`


### Keystore
The server requires certificates to be stored in Java keystore format.
1. Obtain certs for your domain using Letsencrypt. Note: Self signed certificates using openssl will also work.
2. Concat all pems into one file
`sudo cat /etc/letsencrypt/live/demo.example.com/*.pem > fullcert.pem`
3. Convert to pkcs format
` openssl pkcs12 -export -out fullcert.pkcs12 -in fullcert.pem`
4. Create new temporary keystore using JDK keytool, will prompt for password
`keytool -genkey -keyalg RSA -alias mykeystore -keystore mykeystore.ks`
`keytool -delete -alias mykeystore -keystore mykeystore.ks`
5. Make JKS, will prompt for password
`keytool -v -importkeystore -srckeystore fullcert.pkcs12 -destkeystore mykeystore.ks -deststoretype JKS`
6. Store JKS in config directory and edit the keyfile name and password entered in previous step
7. Mention keystore mount path (w.r.t docker-compose) in config.json



### Testing

### Unit tests
1. Run the tests using `mvn clean test checkstyle:checkstyle pmd:pmd`
2. Reports are stored in `./target/`


### Integration tests
Integration tests are through Rest Assured
1. Run the server through either docker, maven or redeployer
2. Run the integration tests `mvn test-compile failsafe:integration-test -DskipUnitTests=true -DintTestHost=localhost -DintTestPort=8080`
3. Reports are stored in `./target/`


# Explanation
## Understanding Onboarding Server
- The section available [here](./docs/Solution_Architecture.md) explains the components/services
used in implementing the Onboarding Server.
- To try out the APIs, import the API collection, postman environment files in postman.
- Reference : [postman-collection](https://github.com/datakaveri/iudx-catalogue-server/blob/master/src/test/resources/iudx-catalogue-server-v5.5.0.postman_collection.json), [postman-environment](https://github.com/datakaveri/iudx-catalogue-server/blob/master/src/test/resources/CAT.postman_environmentv5.5.0.json)

## Contributing
We follow Git Merge based workflow
1. Fork this repository
2. Create a new feature branch in your fork. Multiple features must have a hyphen separated name, or refer to a milestone name as mentioned in Github -> Projects
4. Commit to your fork and raise a Pull Request with upstream
# How to Guide
## Setup and Installation
Setup and installation guide is available [here](./docs/SETUP-and-installation.md)

# Reference
## API Docs
API docs are available [here](https://api.catalogue.cos.idxgh.com/apis).

## License
[View License](./LICENSE)
## FAQ
FAQs are available [here](./docs/FAQ.md)
24 changes: 24 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<p align="center">
<img src="./docs/cdpg.png" width="300">
</p>

**Thanks for reporting the vulnerability issue**! :vulcan_salute:
<br>
If you find any potential vulnerabilities in Data Exchange servers, please report it to us in a confidential way
by adding the following content in the email:

```
From: <email-ID>
To: DX Admin <[email protected]>, DX Support <[email protected]>
Subject: Vulnerability report
Attachement: <screenshots, reports, videos, etc., >
Body:
Type of the issue: <ex: SQL Injection>,
Affected files: <path to affect files>,
Prerequisities: <configurations to reproduce the issue>,
Steps: <All the steps to follow to reproduce the issue>,
Impact of the issue: <About the issue causing any further problems>,
Anything else: <links? references? anything that will give us more context about the issue>

```

Loading