BEP016 finalization

[PeerHerholz]

Hello everyone (with a slight focus on @arokem, @francopestilli, @Lestropie and @oesteban), @bids-standard/maintainers, & @bids-standard/steering,

I hope you're doing fine.

With the general conventions draft being merged, I thought it might be a good time to focus on the remaining issues in BEP016 and maybe apply some sort of classification of outstanding issues into "must-be-resolved" and "would-be-nice-but-not-needed-to-advance" comparable to what we started in other BEPs (e.g. BEP038 and BEP017), to then start the merging process.

What do you think about this?

If y'all could have a look at the issues again and share your thoughts re the current status and if something/what definitely still needs to be addressed, that would be great! I'll start a list below and will keep editing it based on your comments.

Needs to be addressed

• example datasets
• filesystem structure (added by @Lestropie, issues: Debating need for advanced inheritance #70, Differentiating model fit/derived parameters #69, Diffusion derivatives suffix (dwi, model, mdp, mfp, etc.) #68, Decisions on BIDS derivatives structure #50, Filesystem paths for diffusion models #32, Major derivatives decisions #29)
• "parameter" nomenclature (added by @Lestropie)
• updating wrt main spec (added by @Lestropie)

Would be cool to address but not necessary

Thanks so much again for all your help and effort, we highly appreciated it.

[Lestropie]

Hoping that I'll finally be able to invest more time in this this year; spent all of last year persistently playing catchup after going part-time in 2022; am still not quite on top of things, but getting close.

I'll see how I go giving some kind of classification of elements that remain outstanding.

Filesystem structure

There are multiple issues attempting to describe the blocking points I encountered with respect to filesystem structure, in pursuit of external feedback. With selection of a given structure, these can be linked to a single PR that adopts conformation to a given structure and then closed.

Note that this necessarily includes the lack of adoption of the inheritance principle. So not only will there not be any upstream modifications to BIDS itself to facilitate a more advanced derivatives structure, but the sidecar metadata and the examples provided will need to be updated accordingly.

"Parameter" nomenclature

Not only is the distinction between model fit and model-derived parameters being abandoned, but also such data are being described as parametric maps, further disambiguating from the "model input parameters" that control how any given model is fit to the empirical data. So the language describing these things will need to be revised throughout, and in addition the small section bullet-pointing the distinctions between them can be removed.

Updating w.r.t. main spec

Since the original forking of this repo, the main specification has adopted all sorts of macro cleverness for specification rendering. See #71. Should not be overly difficult, but will be absolutely necessary before proposing a merge.

---

Those are probably the compulsory ones. There are lots of nice-to-have's. I'm very hesitant about specifying anything about bootstrapping without properly road-testing a solution (#38). But most of the others are I think just additions. Most likely for those it would be a matter of doing the compulsory stuff above, and then having a go at some of these while the compulsory stuff is debated.

[PeerHerholz]

Hi @Lestropie,

thanks so much for all the information and feedback, we highly appreciate it.

I added the respective points to the list in the initial comment. Could I maybe ask you for a list concerning the filesystem structure issues, just to not miss anything and add that information above? Sorry.

Thanks again.

Cheers, Peer

[Lestropie]

I would envisage a single PR that commits to a given filesystem structure, in terms of:

• A lack of nested directories
• Filename suffices
• Absence of advanced inheritance

Which would close:

• Debating need for advanced inheritance #70
• Differentiating model fit/derived parameters #69
• Diffusion derivatives suffix (dwi, model, mdp, mfp, etc.) #68
• Decisions on BIDS derivatives structure #50
• Filesystem paths for diffusion models #32
• Major derivatives decisions #29

[oesteban]

Although I admittedly find myself amongst those who saw forking the spec as a good idea, in hindsight, it has helped only with keeping discussions organized but has translated into zero consensual progress. I believe we need to change the strategy (cc/ @francopestilli for the R01 and @arokem as a steering committee member).

Instead of looking at what we need to agree on, we should focus on those things we have arrived at an agreement.

After that, I would consider the example datasets the main and unique effort.

The other conversations (#70, #69, #50, #32, #29) transcend what a separate modality can resolve and should be addressed as separate BEPs. In particular, those topics touching on advanced inheritance, model-related language (all modalities have models), and filesystem paths are clearly BIDS-level. Even if we discuss them here and reach an agreement on all (or some) of them (which I honestly doubt is realistic at this point), then it will be brought to the general approval of the BEP. BIDS would not be doing itself a favor if DWI becomes a wholly different galaxy -- meaning, I doubt such profound changes will pass if we try to sell that they are scoped within DWI.

[PeerHerholz]

Hi @Lestropie and @oesteban,

thank you very much for your replies and feedback.

@Lestropie: thanks for the list of issues, I added them to the initial comment.

@oesteban: Thank you very much for the feedback and ideas. IMHO, it's a great and important suggestion, not only for this BEP but also BEPs in general. If ok, I would like to add a bit of background information on why I started this issue (and comparable ones for BEP17, BEP38 and BEP39.

Basically, I wanted to come up with a process to streamline BEP development and respective discussion points, i.e., turning the conversations and discussions spread across multiple platforms, documents, and issues into a clear and precise list of action items that could also be voted on. Specifically, this concerns/ed a distinction between "must haves" and "would be nice but not necessary" action items to get rid of roadblocks (in combination with the potential voting system). However, I definitely understand how this could also reintroduce existing problems and/or introduce new ones.
I agree that the issues you mentioned could be dealt with via other BEPs.

However, it would be great if we as a community could find/identify a way forward. Especially considering the ever-growing number of other BEPs generally and such that focus on derivatives. In a rather selfish manner, I would like to point to the other BEPs of the Connectivity Project which are currently implementing a similar approach, although at an earlier stage (ie moving from GoogleDocs to GitHub).

Thanks again.

Best, Peer

[oesteban]

However, it would be great if we as a community could find/identify a way forward. Especially considering the ever-growing number of other BEPs generally and such that focus on derivatives. In a rather selfish manner, I would like to point to the other BEPs of the Connectivity Project which are currently implementing a similar approach, although at an earlier stage (ie moving from GoogleDocs to GitHub).

Sure, I was not suggesting to 'cancel' this fork (that's what I understand from your last comment). Removing it and moving elsewhere is more work than it is worth, and we have many conversation threads open.

My reflection was more along the lines that, at this point, it looks reasonable to objectively look back and try to identify we all agree and put it forward.

Once that is done, I think generating example data should be the first priority. When we hit an issue on a dataset that cannot be resolved with current BIDS/Derivs, then we can prioritize that conversation under the full spec venue.

I personally don't have the time to engage in deep conversations at the moment. So let's start a pull request to the bids-examples repo (or whatever option, the thing is - let's generate something tangible).

[PeerHerholz]

Hi @oesteban,

thanks for the reply!

Oh sorry, I didn't mean to suggest that. Rather, I was asking if the approach of opening a "finalization/summary issue" plus voting makes sense in general and could be kept for the other BEPs/adapted to new ones.

Thanks again.

Cheers, Peer

[oesteban]

I see, didn't see the connection to the other BEPs then.

Okay, I guess my thinking is easy to summarize:

• Identify the area of consensus within BEP016
• Examples

As I mentioned before, I'd push other topics into the "will come back to this one day" and "to the BIDS general discussion" categories.

[PeerHerholz]

Hi @oesteban,

great, thanks so much!

@Lestropie, @arokem and @francopestilli: WDYT?

Cheers, Peer

[arokem]

Thanks for kicking off this meta-issue, @PeerHerholz.

I think that I agree with Oscar's characterization that we should find the areas of consensus and implement those. We can start by aiming to revise the current state of the spec based on bids-standard/bids-extensions#26 (not currently posted as an issue on this repo, but maybe it should be).

Regarding tangible examples, I think that we already have some progress in the form of https://github.com/PeerHerholz/bids_bep16_conv, which implements conversions of the outputs of DIPY and mrtrix pipelines into BEP16-compliant derivatives, with the HBN dataset as the inputs.

I think that we can eventually use that to create an example dataset. I think that one subject/session from that dataset should suffice.

[PeerHerholz]

Thanks @arokem!

So, the next steps would entail adapting things based on bids-standard/bids-extensions#26 and then evaluate the example dataset we currently have?

[oesteban]

I would aim at something like this https://github.com/bids-standard/bids-examples/tree/master/ds000001-fmriprep

We can just say dipy as pipeline and maybe only get to realigned & denoised 4D dwis, so we don't get into models yet.

[oesteban]

BTW - I said dipy because @arokem mentioned the progress on that front, but I'd be equally enthusiastic if we go with MRTrix or others.

[Lestropie]

With respect to the larger issues around filesystem structure, it was precisely my not wanting DWI to "become a different galaxy" that led to a jam: I recognised that anything we did in that regard would be an applicable precedent for many BEPs, and so was uncomfortable proceeding on my own. My stated intention was to see what kind of feedback or consensus we could get just within BEP016, and then open it up to the wider BIDS community with DWI voxel-wise models acting as the practical use case (proposing such major structural changes purely academically likely wouldn't get very far), potentially with a poll vote if necessary. The rejection of bids-standard/bids-specification#1280 shows that it's not a trivial problem with a trivial solution.

What I don't know is whether, upon seeing data / specification under a "simpler" proposal (dwimap across the board; no advanced inheritance), people will be content with such, or instead desire something where the intrinsic hierarchical structure of the data is more faithfully preserved in the specification. Since very early on I've proceeded under presumption of the latter, and maybe my skipping straight to that step has been detrimental to overall progress.

On the topic of software, I'm cognisant that capturing bedpostx outputs will be necessary, both because it's computationally expensive and therefore appealing for distribution to prevent duplication of processing, and to evaluate proposals for representing bootstrapped model fits. So any fundamental structure of the software used for demonstrating / converting needs to be amenable to executing / making use of data from external software. Ie. If you progress right now with dipy---which I'm perfectly fine with, I'm not going to advocate for MRtrix out of personal interest---you may not want it to become too "intrinsically tied" to dipy, whatever that means.