Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] update validator page #572

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed docs/assets/img/Gqwc1q9.png
Binary file not shown.
Binary file removed docs/assets/img/PEz9hbd.png
Binary file not shown.
Binary file removed docs/assets/img/YD38eTE.png
Binary file not shown.
Binary file added docs/assets/img/validator_error.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/img/validator_warning.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/blog/posts/2023-07-06-steering-group-minutes.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ categories:
- Do we need to introduce a new term? We could stick with p-seg (probabilistic) and d-seg (deterministic)
- Should we add this Derivatives output to the website?
- Where should it go? Create a PR or multiple PRs?
- Suggestion: integrate the BIDS derivatives guidelines piecemeal into the BID general derivatives spec
- Suggestion: integrate the BIDS derivatives guidelines piecemeal into the BID general derivatives specification
- Maybe not the BEPs table info - table could go to the BIDs extensions

### Imaging Neuroscience Response regarding BIDS 'badge'
Expand Down
2 changes: 1 addition & 1 deletion docs/blog/posts/2024-01-11-steering-group-minutes.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ Github project is established with many ideas but there aren\'t many

- Revisiting how BIDS is defined beyond MRI may be a path to improving
adoption within the neuroscience community. Meaning BIDS started with
MRI and some of the BIDS spec might be more complicated than could
MRI and some of the BIDS specification might be more complicated than could
have been if we had focused on all modalities from the start.

- iEEG, MEG, ...
Expand Down
4 changes: 2 additions & 2 deletions docs/extensions/specific-guidelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ a repeated pattern in generating derivatives within several imaging modalities'

1. A reference map that is used to encode spatial features and parameters is required.
There is an antecedent of this in `BIDS` with BEP23 ([see below](#BEP-23-PET-Derivatives)).
In that `BEP`, the proposed naming takes the pattern `_<suffix>ref` (for example `_boldref`, `_dwiref`...), and that solution has been suggested as a possibility in [issue #1532](https://github.com/bids-standard/bids-specification/issues/1532) of the spec repository.
In that `BEP`, the proposed naming takes the pattern `_<suffix>ref` (for example `_boldref`, `_dwiref`...), and that solution has been suggested as a possibility in [issue #1532](https://github.com/bids-standard/bids-specification/issues/1532) of the specification repository.

1. We have derived data that are no longer of the same type as the original,
but for which we would like to keep the notion of the modality from which this was derived
Expand All @@ -57,7 +57,7 @@ and MUST specify the extension and metadata that are required with the suffix.

1. The file descriptor does fall under one of the generic derivatives descriptors.

1. No other descriptor exists in the `BIDS` spec.
1. No other descriptor exists in the `BIDS` specification.
For example, `statmap` cannot be used, because it is already being used, or soon to be,
for a [different specification](https://bids-standard.github.io/stats-models/walkthrough-1.html#from-run-outputs-to-subject-inputs).

Expand Down
2 changes: 1 addition & 1 deletion docs/getting_started/folders_and_files/metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -557,7 +557,7 @@ df.to_csv('my_new_file.tsv', sep='\t')

- Create a file with the following columns
(at least, for other values see paragraph the
[BIDS spec](https://bids-specification.readthedocs.io/en/latest/03-modality-agnostic-files.html#participants-file))
[BIDS specification](https://bids-specification.readthedocs.io/en/latest/03-modality-agnostic-files.html#participants-file))
- participant_id
- age
- sex
Expand Down
2 changes: 1 addition & 1 deletion docs/getting_started/tutorials/conversion/pet.md
Original file line number Diff line number Diff line change
Expand Up @@ -997,7 +997,7 @@ and add the plasma series of data to it from our base text file:
![screenshot of unconverted sub-01_ses-testscan_recording-Manual_blood.tsv](https://i.imgur.com/VdFqh6H.png)

Whoops, these numbers aren't in SI units, so we quickly convert time
from minutes to seconds and Microcurie to Becquerel to conform to the spec.
from minutes to seconds and Microcurie to Becquerel to conform to the specification.

!!! note

Expand Down
2 changes: 1 addition & 1 deletion docs/standards/schema/schema-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@
run: optional
```

![image of template rendered in spec](../../assets/img/noncontinuous_beh_rendered_template.png)
![image of template rendered in specification](../../assets/img/noncontinuous_beh_rendered_template.png)

---

Expand Down
122 changes: 41 additions & 81 deletions docs/tools/validator.md
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Main change of the PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@effigies maybe have a quick look at the html page see if there is anything I should add for now.

Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@ that checks if a dataset is compliant with the BIDS standard.
The validator is available for use within several different environments
to best suit individual user preferences and use cases, those versions are:

- A web browser based version
- Command line version
- Docker based version
- A python library installable via pip
- [A web browser based version](https://bids-standard.github.io/bids-validator/)
- [Command line version](https://bids-validator.readthedocs.io/en/latest/user_guide/command-line.html)
- [Docker based version](https://hub.docker.com/r/bids/validator)
- A [python library](https://github.com/bids-standard/python-validator) installable from [pypi](https://pypi.org/project/bids-validator/) or [conda-forge](https://anaconda.org/conda-forge/bids-validator)

Instructions to install and use these versions can be found
within the [quickstart guide](https://github.com/bids-standard/bids-validator)
at the BIDS Validator repository.

## Data Privacy and Confidentiality

Please note that the web app is entirely browser(not server) based.
Please note that the web app is entirely browser (not server) based.
As such, there is no file uploading as part of the validation.

## Browser Version
Expand All @@ -25,7 +25,7 @@ As such, there is no file uploading as part of the validation.
It requires that you use the Chrome or Firefox browser,
since those are the only ones in which you can select a whole folder rather than individual files.

![home](../assets/img/YD38eTE.png)
![home](https://github.com/bids-standard/bids-validator/raw/main/docs/_static/web_entrypoint_dark.png#gh-dark-mode-only)

1. You can then choose the **folder** that you wish to validate

Expand All @@ -36,126 +36,91 @@ As such, there is no file uploading as part of the validation.
This response indicates that your dataset is not BIDS compliant.
Try following the suggestions listed to make the appropriate corrections.

![Error](../assets/img//PEz9hbd.png)
![Error](../assets/img/validator_error.png)

!!! warning

This response indicates that your dataset is BIDS compliant,
but there are some non-critical problems
(such as optional fields missing, or differences between participants).
(such as recommended fields missing, or differences between participants).

![Warning](../assets/img//Gqwc1q9.png)
![Warning](../assets/img/validator_warning.png)

!!! success "Success :rocket:"

This response indicates that your folder contains a valid BIDS Dataset!

![Valid](../assets/img/DPFVXOR.png)
<!-- ![Valid](../assets/img/DPFVXOR.png) -->

## Command Line Version

### Verifying a BIDS compliant data set

After [installation with npm](https://github.com/bids-standard/bids-validator)
After [installation with deno](https://bids-validator.readthedocs.io/en/latest/user_guide/command-line.html#using-the-command-line)
using the CLI is relatively simple.
Locate a bids data set similar to the one below:

```bash
user@host:~/bids-examples$ tree sub001/
user@host:~/bids-examples$ tree pet001

sub001/
├── dataset_description.json
├── participants.json
├── participants.tsv
├── README
└── sub-01
├── ses-baseline
│   ├── anat
│   │   ├── sub-01_ses-baseline_T1w.json
│   │   └── sub-01_ses-baseline_T1w.nii.gz
│   └── pet
│   ├── sub-01_ses-baseline_pet.json
│   └── sub-01_ses-baseline_pet.nii.gz
└── ses-intervention
└── ses-01
├── anat
   ├── sub-01_ses-intervention_T1w.json
   └── sub-01_ses-intervention_T1w.nii.gz
├── sub-01_ses-01_T1w.json
└── sub-01_ses-01_T1w.nii
└── pet
├── sub-01_ses-intervention_task-eyes_events.json
├── sub-01_ses-intervention_task-eyes_events.tsv
├── sub-01_ses-intervention_task-eyes_pet.json
└── sub-01_ses-intervention_task-eyes_pet.nii.gz
├── sub-01_ses-01_trc-CIMBI36_pet.json
├── sub-01_ses-01_trc-CIMBI36_pet.nii.gz
├── sub-01_ses-01_trc-CIMBI36_recording-autosampler_blood.json
├── sub-01_ses-01_trc-CIMBI36_recording-autosampler_blood.tsv
├── sub-01_ses-01_trc-CIMBI36_recording-manual_blood.json
└── sub-01_ses-01_trc-CIMBI36_recording-manual_blood.tsv
```

Now simply point the bids validator at the folder path of the subject(s) in
question:
Now simply point the bids validator at the folder path of the subject(s) in question:

```bash
user@host:~/bids-examples$ bids-validator sub001
user@host:~/bids-examples$ bids-validator pet001
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would probably prefer to give an accurate command, though I can see the argument that simple might be better here:

Suggested change
user@host:~/bids-examples$ bids-validator pet001
user@host:~/bids-examples$ deno run -ERN jsr:@bids/validator pet001

```

### Types of Feedback

!!! failure "Error"

As is the case w/ the browser or any version of the validator follow
the `[ERR]` messages and correct until bids-validator returns 0 errors after running.
As is the case with the browser or any version of the validator follow
the `[ERROR]` messages and correct until bids-validator returns 0 errors after running.

```bash
user@host:~/bids-examples$ bids-validator sub001

[email protected]

1: [ERR] Invalid JSON file. The file is not formatted according the schema. (code: 55 - JSON_SCHEMA_VALIDATION_ERROR)
./sub-01/ses-baseline/pet/sub-01_ses-baseline_pet.json
Evidence: should have property InjectedRadioactivityUnits when property InjectedRadioactivity is present
./sub-01/ses-baseline/pet/sub-01_ses-baseline_pet.json
Evidence: should have required property 'InjectedMassUnits'
...
...
... and 3 more files having this issue (Use --verbose to see them all).

Please visit https://neurostars.org/search?q=JSON_SCHEMA_VALIDATION_ERROR for existing conversations about this issue.


Summary: Available Tasks: Available Modalities:
14 Files, 5.72MB T1w
1 - Subject pet
2 - Sessions events

[ERROR] NIFTI_HEADER_UNREADABLE We were unable to parse header data from this NIfTI file. Please ensure it is not corrupted or mislabeled.
/sub-01/ses-01/anat/sub-01_ses-01_T1w.nii

If you have any questions, please post on https://neurostars.org/tags/bids.
Please visit https://neurostars.org/search?q=NIFTI_HEADER_UNREADABLE for existing conversations about this issue.
```

!!! warning

As stated with the browser version above, one may elect to ignore warnings,
but the information provided via the validator should help to pinpoint
where and how to resolve some of these warnings.
When in doubt consult the [BIDS Spec](https://bids-specification.readthedocs.io/en/latest/)
When in doubt consult the [BIDS specification](https://bids-specification.readthedocs.io/en/latest/)

```bash
user@host:~/bids-examples$ bids-validator sub001/
[WARNING] JSON_KEY_RECOMMENDED A JSON file is missing a key listed as recommended.
DatasetType
/dataset_description.json

[email protected]
GeneratedBy
/dataset_description.json

1: [WARN] Tabular file contains custom columns not described in a data dictionary (code: 82 - CUSTOM_COLUMN_WITHOUT_DESCRIPTION)
./sub-01/func/sub-01_task-balloonanalogrisktask_run-01_events.tsv
...
...
... and 38 more files having this issue (Use --verbose to see them all).
SourceDatasets
/dataset_description.json

Please visit https://neurostars.org/search?q=CUSTOM_COLUMN_WITHOUT_DESCRIPTION for existing conversations about this issue.


Summary: Available Tasks: Available Modalities:
134 Files, 411.53KB balloon analog risk task T1w
16 - Subjects inplaneT2
1 - Session bold
events


If you have any questions, please post on https://neurostars.org/tags/bids.
Please visit https://neurostars.org/search?q=JSON_KEY_RECOMMENDED for existing conversations about this issue.
```

!!! success "Success :rocket:"
Expand All @@ -164,15 +129,10 @@ user@host:~/bids-examples$ bids-validator sub001

```bash
user@host:~/bids-examples$ bids-validator sub001
[email protected]

This dataset appears to be BIDS compatible.

Summary: Available Tasks: Available Modalities:
12 Files, 218.75KB T1w
1 - Subject pet
1 - Session blood

Summary: Available Tasks: Available Modalities:
12 Files, 224 kB PET
1 - Subjects 1 - Sessions MRI

If you have any questions, please post on https://neurostars.org/tags/bids
If you have any questions, please post on https://neurostars.org/tags/bids.
```
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -137,10 +137,10 @@ nav:
- getting_started/templates/index.md
- Tools:
- tools/index.md
- tools/converters.md
- tools/validator.md
- tools/bids-apps.md
- tools/others.md
- tools/converters.md
- Datasets:
- datasets/index.md
- FAQ:
Expand Down
Loading