Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#2161] One-Stop Config File for Code Portfolio #2192

Open
wants to merge 21 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
1b6e79b
Implement one-stop config file
georgetayqy Apr 15, 2024
a6e30e7
Merge branch 'master' into implement-one-stop-config-file
georgetayqy Apr 27, 2024
b76b29e
Update report-config.yaml file
georgetayqy Apr 27, 2024
15bbf0e
Merge branch 'master' of https://github.com/reposense/RepoSense into …
georgetayqy May 6, 2024
0e83f36
Merge branch 'master' into implement-one-stop-config-file
sopa301 May 7, 2024
65cf7dd
Update config file in cypress
sopa301 May 7, 2024
0a4b72d
Fix title name
sopa301 May 7, 2024
907e1a7
Merge branch 'master' into implement-one-stop-config-file
ckcherry23 May 12, 2024
ab3aff4
Update report-config.yaml format
georgetayqy May 12, 2024
98db5d8
Merge branch 'implement-one-stop-config-file' of https://github.com/a…
georgetayqy May 12, 2024
c20e450
Update docs for report-config.yaml
georgetayqy May 12, 2024
66cd719
Update URLs
georgetayqy May 12, 2024
18772f0
Update YAML Format
georgetayqy Jun 9, 2024
4b409f9
Merge branch 'master' of https://github.com/reposense/RepoSense into …
georgetayqy Jun 9, 2024
587f152
Update YAML config file format
georgetayqy Jun 29, 2024
cc151f4
Revert "Update YAML config file format"
georgetayqy Jul 14, 2024
645b4c7
Update Report Configuration YAML File Format
georgetayqy Jul 14, 2024
45a9708
Merge branch 'master' into implement-one-stop-config-file
ckcherry23 Jul 14, 2024
42d167d
Implement One-Stop Config File Feature
georgetayqy Jul 15, 2024
3e0d06b
Merge branch 'master' into implement-one-stop-config-file
georgetayqy Aug 9, 2024
d7f13b4
Merge branch 'master' into implement-one-stop-config-file
gok99 Aug 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions config/report-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
title: RepoSense Report
ckcherry23 marked this conversation as resolved.
Show resolved Hide resolved
repos:
- repo: https://github.com/reposense/testrepo-Delta.git
groups:
georgetayqy marked this conversation as resolved.
Show resolved Hide resolved
- group-name: code
globs:
- "**.java"
- group-name: tests
globs:
- "src/test**"
- group-name: docs
globs:
- "docs**"
- "**.adoc"
- "**.md"
branches:
- branch: master
blurb: "My project"
authors:
- author-git-host-id: fzdy1914
author-display-name: WANG CHAO
author-git-author-name: WANG CHAO
author-emails:
- [email protected]
- author-git-host-id: FH-30
author-display-name: Francis Hodianto
author-git-author-name: Francis Hodianto
author-emails:
-
ignore-glob-list:
-
ignore-authors-list:
-
2 changes: 1 addition & 1 deletion docs/ug/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ partial credit.
<div id="section-config">

**`--config CONFIG_DIRECTORY`**: Specifies that config files located in `CONFIG_DIRECTORY` should be used to customize the report.
* Parameter: `CONFIG_DIRECTORY` The directory containing the config files. Should contain a `repo-config.csv` file. Optionally, can contain an `author-config.csv` file or/and a `group-config.csv` file or/and a `report-config.json` file or/and a `blurbs.md` file.
* Parameter: `CONFIG_DIRECTORY` The directory containing the config files. Should contain a `repo-config.csv` file. Optionally, can contain an `author-config.csv` file or/and a `group-config.csv` file or/and a `report-config.yaml` file.
* Alias: `-c`
* Example: `java -jar RepoSense.jar --config ./config`

Expand Down
67 changes: 45 additions & 22 deletions docs/ug/configFiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,20 +31,20 @@ Given below are the details of the various config files used by RepoSense.
**`repo-config.csv` file contains repo-level config data.** Each row represents a repository's configuration ([example](repo-config.csv)).


| Column Name | Explanation |
|-------------|-------------|
| Repository's Location {{ mandatory }} | The `Remote Repo URL` or `Disk Path` to the Git repository e.g., `https://github.com/foo/bar.git` or `C:\Users\user\Desktop\GitHub\foo\bar` |
| Branch | The branch to analyze in the target repository e.g., `master`. Default: the default branch of the repo |
| File formats<sup>*+</sup> | The file extensions to analyze. Binary file formats, such as `png` and `jpg`, will be automatically labelled as the file type `binary` in the generated report. Default: all file formats |
| Find Previous Authors | Enter **`yes`** to utilize Git blame's ignore revisions functionality, RepoSense will attempt to blame the line changes caused by commits in the ignore commit list to the previous authors who altered those lines (if available). |
| Ignore Glob List<sup>*+</sup> | The list of file path globs to ignore during analysis for each author e.g., `test/**;temp/**`. Refer to the [_glob format_](https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob) for the path glob syntax. |
| Ignore standalone config | To ignore the standalone config file (if any) in target repository, enter **`yes`**. If the cell is empty, the standalone config file in the repo (if any) will take precedence over configurations provided in the csv files. |
| Ignore Commits List<sup>*+</sup> | The list of commits to ignore during analysis. For accurate results, the commits should be provided with their full hash. Additionally, a range of commits can be specified using the `..` notation e.g. `abc123..def456` (both inclusive). |
| Ignore Authors List<sup>*+</sup> | The list of authors to ignore during analysis. Authors should be specified by their [Git Author Name](#a-note-about-git-author-name). |
| Shallow Cloning | Enter **`yes`** to clone the repository using Git's shallow cloning functionality. This option can significantly reduce the time taken to clone large repositories. However, the option should ideally be disabled for smaller repositories where the `.git` file is smaller than 500 MB, as it would create overhead. |
| File Size Limit<sup>+</sup> | Enter a file size limit for the repository in bytes as a single number without units (for a size limit of 1MB for example, enter 1000000). This file size limit will override the default file size limit (500KB). Files exceeding the file size limit will be marked as ignored and only the file name and line count will be reflected in the report. |
| Ignore File Size Limit | Enter **`yes`** to ignore both the default file size limit and the file size limit possibly set by the user in `repo-config.csv`. |
| Skip Ignored File Analysis | Enter **`yes`** to ignore analysis of files exceeding the file size limit entirely. If file analysis is skipped, all information about the file will be omitted from the generated report. This option can significantly improve report generation time. |
| Column Name | Explanation |
|---------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Repository's Location {{ mandatory }} | The `Remote Repo URL` or `Disk Path` to the git repository e.g., `https://github.com/foo/bar.git` or `C:\Users\user\Desktop\GitHub\foo\bar` |
| Branch | The branch to analyze in the target repository e.g., `master`. Default: the default branch of the repo |
| File formats<sup>*+</sup> | The file extensions to analyze. Binary file formats, such as `png` and `jpg`, will be automatically labelled as the file type `binary` in the generated report. Default: all file formats |
| Find Previous Authors | Enter **`yes`** to utilize Git blame's ignore revisions functionality, RepoSense will attempt to blame the line changes caused by commits in the ignore commit list to the previous authors who altered those lines (if available). |
| Ignore Glob List<sup>*+</sup> | The list of file path globs to ignore during analysis for each author e.g., `test/**;temp/**`. Refer to the [_glob format_](https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob) for the path glob syntax. |
| Ignore standalone config | To ignore the standalone config file (if any) in target repository, enter **`yes`**. If the cell is empty, the standalone config file in the repo (if any) will take precedence over configurations provided in the csv files. |
| Ignore Commits List<sup>*+</sup> | The list of commits to ignore during analysis. For accurate results, the commits should be provided with their full hash. Additionally, a range of commits can be specified using the `..` notation e.g. `abc123..def456` (both inclusive). |
| Ignore Authors List<sup>*+</sup> | The list of authors to ignore during analysis. Authors should be specified by their [Git Author Name](#a-note-about-git-author-name). |
| Shallow Cloning | Enter **`yes`** to clone the repository using Git's shallow cloning functionality. This option can significantly reduce the time taken to clone large repositories. However, the option should ideally be disabled for smaller repositories where the `.git` file is smaller than 500 MB, as it would create overhead. |
| File Size Limit<sup>+</sup> | Enter a file size limit for the repository in bytes as a single number without units (for a size limit of 1MB for example, enter 1000000). This file size limit will override the default file size limit (500KB). Files exceeding the file size limit will be marked as ignored and only the file name and line count will be reflected in the report. |
| Ignore File Size Limit | Enter **`yes`** to ignore both the default file size limit and the file size limit possibly set by the user in `repo-config.csv`. |
| Skip Ignored File Analysis | Enter **`yes`** to ignore analysis of files exceeding the file size limit entirely. If file analysis is skipped, all information about the file will be omitted from the generated report. This option can significantly improve report generation time. |

<box type="info" seamless>
The Shallow Cloning option is incompatible with the "--last-modified-date" CLI flag.
Expand Down Expand Up @@ -94,11 +94,11 @@ If `author-config.csv` is not given and the repo has not provided author details

Optionally, you can provide a `group-config.csv`(which should be in the same directory as `repo-config.csv` file) to provide details on any custom groupings for files in specified repositories ([example](group-config.csv)). It should contain the following columns:

| Column Name | Explanation |
|-------------|-------------|
| Repository's Location | Same as `repo-config.csv`. Default: all the repos in `repo-config.csv` |
| Group Name {{ mandatory }} | Name of the group, e.g.,`test`. |
| Globs * {{ mandatory }} | The list of file path globs to include for specified group, e.g.,`**/test/*;**.java`. |
| Column Name | Explanation |
|----------------------------|---------------------------------------------------------------------------------------|
| Repository's Location | Same as `repo-config.csv`. Default: all the repos in `repo-config.csv` |
| Group Name {{ mandatory }} | Name of the group, e.g.,`test`. |
| Globs * {{ mandatory }} | The list of file path globs to include for specified group, e.g.,`**/test/*;**.java`. |

<sup>* **Multi-value column**: multiple values can be entered in this column using a semicolon `;` as the separator.</sup>

Expand All @@ -107,12 +107,35 @@ e.g.: `example.java` in `example-repo` can either be in the `test` group or the

<!-- ==================================================================================================== -->

## `report-config.json`
## `report-config.yaml`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also update the --config flag under 'CLI syntax reference' to include report-config.yaml instead of json? https://docs-2192-pr-reposense-reposense.surge.sh/ug/cli.html#config-c

Let's also update the 'Customize using CSV config files' heading on the 'Customizing Reports` page to include YAML or remove file types entirely. https://docs-2192-pr-reposense-reposense.surge.sh/ug/customizingReports.html#customize-using-csv-config-files

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do!


You can optionally use `report-config.json` to customize report generation by providing the following information. ([example](report-config.json))
You can also optionally use a `report-config.yaml` file to quickly define the repository information for the repositories you are interested in tracking and generate your very own code portfolio.

View this [example](report-config.yaml) for a better understanding of what repository information is required.

Note: All fields should be defined, and left blank if not used.

**Fields to provide**:
ckcherry23 marked this conversation as resolved.
Show resolved Hide resolved
* `title`: Title of the generated report, which is also the title of the deployed dashboard. Default: "RepoSense Report"
* `title`: Title of the generated report, which is also the title of the deployed dashboard. Default: "RepoSense Report".
* `repos`: A list of repositories to include for analysis.
* `repo`: The URL to your repository of interest.
* `groups`: A list of the different custom groupings.
* `group-name`: Name of the group.
* `globs`: The list of file path globs to include for specified group.
* `branches`: A list of branches to analyse for each repository.
* `branch`: The name of the branch.
* `authors`: A list of authors to analyse on the branch.
* `author-git-host-id`: Git host username of the author.
* `author-display-name`: Display name of the author.
* `author-git-author-name`: Author's Git host name.
* `author-emails`: A list of emails associated with an author.
* `file-formats`: File formats to analyze.
* `ignore-glob-list`: Folders/files to ignore, specified using the [_glob format_](https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob).
* `ignore-standalone-config`: Ignores the different standalone configuration files when analysing this branch
* `ignore-commits-list`: The list of commits to ignore during analysis. For accurate results, the commits should be provided with their full hash. Additionally, a range of commits can be specified using the `..` notation e.g. `abc123..def456` (both inclusive).
* `ignore-authors-list`: The list of authors to ignore during analysis. Authors specified in `authors` field or `author-config.csv` will be also be omitted if they are in this list. Authors should be specified by their [Git Author Name](#a-note-about-git-author-name).
* `is-shallow-cloning`: Indicates whether to clone the repository using Git's shallow cloning functionality.
* `is-find-previous-authors`: Indicates whether to find previous authors of the repository/branch.

<!-- ==================================================================================================== -->

Expand Down
3 changes: 0 additions & 3 deletions docs/ug/report-config.json

This file was deleted.

43 changes: 43 additions & 0 deletions docs/ug/report-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
title: RepoSense Report
group-details:
- repo: https://github.com/reposense/testrepo-Delta.git
groups:
- group-name: code
globs:
- "**.java"
- group-name: tests
globs:
- "src/test**"
- group-name: docs
globs:
- "docs**"
- "**.adoc"
- "**.md"
repos:
- repo: https://github.com/reposense/testrepo-Delta.git
branches:
- branch: master
authors:
- author-git-host-id: fzdy1914
author-display-name: WANG CHAO
author-git-author-name: WANG CHAO
author-emails:
- [email protected]
- author-git-host-id: FH-30
author-display-name: Francis Hodianto
author-git-author-name: Francis Hodianto
author-emails:
-
file-formats:
- override:java
- md
- fxml
ignore-glob-list:
-
ignore-standalone-config: false
ignore-commits-list:
-
ignore-authors-list:
-
is-shallow-cloning: false
is-find-previous-authors: false
3 changes: 0 additions & 3 deletions frontend/cypress/config/report-config.json

This file was deleted.

1 change: 1 addition & 0 deletions frontend/cypress/config/report-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
title: RepoSense Test Report
2 changes: 1 addition & 1 deletion src/main/java/reposense/RepoSense.java
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@
import reposense.model.BlurbMap;
import reposense.model.CliArguments;
import reposense.model.RepoConfiguration;
import reposense.model.ReportConfiguration;
import reposense.model.RunConfigurationDecider;
import reposense.model.reportconfig.ReportConfiguration;
import reposense.parser.ArgsParser;
import reposense.parser.exceptions.InvalidCsvException;
import reposense.parser.exceptions.InvalidHeaderException;
Expand Down
5 changes: 5 additions & 0 deletions src/main/java/reposense/model/BlurbMap.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ public BlurbMap() {
this.urlBlurbMap = new HashMap<>();
}

/**
* Return a copy of the mapping between the repo URL to the associated blurb.
*
* @return a {@code Map<String, String>} containing the mapping between the repo URL to the associated blurb.
*/
public Map<String, String> getAllMappings() {
return new HashMap<>(this.urlBlurbMap);
}
Expand Down
Loading
Loading