The evolution of Nipype into Pydra

[tclose]

Title

The evolution of Nipype into Pydra

Leaders

Tom Close (GH+MM: @tclose) and Arkiev D'Souza (GH: @arkiev MM: adsouza)

Collaborators

Dorota Jarecka (GH: @djarecka MM: dorota), Chris Markiewicz (GH: @effigies, MM: markiewicz), Yibei Chen (GH: @yibeichan MM: yibeichen), Ghislain Vaillant (GH+MM: @ghisvail) and Satra Ghosh (GH+MM: @satra)

Brainhack Global 2023 Event

Brainhack Australasia

Project Description

Nipype is a Python library that provides a uniform interface to existing neuroimaging software and facilitates interaction between these packages within a single workflow. It forms the basis of widely-used pipelines such as C-PAC and fMRIPrep.

Despite Nipype's success and longevity, some limitations of its design have become apparent over time. In particular,

• the complexity of designing new task interfaces
• inability to run workflow nodes in separate software containers
• inability to split/join workflow nodes over lists generated at execution time
• difficulty following the workflow construction syntax due to the separation of nodes and connections

Therefore, at the 2018 OHBM BrainHack in Singapore, a number of Nipype's core developers sat down to start planning a rewritten 2nd generation, which eventually turned into Pydra.

In the intervening years, Pydra has matured into a fully functioning alternative to Nipype that is almost ready for production. However, it is missing the large library of tool interfaces that have been developed for Nipype over the years. Therefore, the Nipype2Pydra conversion tool has been developed to semi-automatically convert existing Nipype interfaces into Pydra syntax. Separate repositories/packages have been created for each toolkit implemented in Nipype, containing YAML specifications to guide the conversion process (e.g. pydra-freesurfer).

In this hackathon, we aim to work through the semi-converted interfaces and complete the conversion process by editing corresponding the YAML specs. Starting off with some of the most popular toolkits, we will hopefully be able to build up enough of a library of interfaces to allow popular Nipype workflows to be ported across to Pydra.

Please read the contribution guide for tips on getting started and our policies on acknowledging contributions.

Link to project repository/sources

• https://github.com/nipype/pydra
• https://github.com/nipype/nipype2pydra

Goals for Brainhack Global

• Convert as many interfaces from Nipype to Pydra as possible
• Complete the Nipype2Pydra workflow converter

Good first issues

TODO

Communication channels

https://mattermost.brainhack.org/brainhack/channels/nipype

Skills

Required: Some Python
Nice to have: experience working with neuroimaging toolkits (e.g. FSL, ANTs) but not essential
Recommended: Reasonably comfortable with git

Onboarding documentation

https://github.com/nipype/pydra/blob/master/CONTRIBUTING.md

What will participants learn?

• How to design Pydra interfaces
• Gain familiarity with neuroimaging toolkits

Data to use

This project is not focused on any specific dataset. We will be typically using dummy datasets and sample datasets from OpenNeuro, e.g. ds000114.

Number of collaborators

more

Credit to collaborators

Project collaborators are listed in the projects' Zenodo reference

Image

Leave this text if you don't have an image yet.

Type

documentation, method_development, pipeline_development

Development status

2_releases_existing

Topic

reproducible_scientific_methods

Tools

AFNI, ANTs, DIPY, FieldTrip, fMRIPrep, Freesurfer, FSL, MNE, MRtrix, Nipype, SPM

Programming language

Python

Modalities

DWI, fMRI, MRI

Git skills

1_commit_push, 2_branches_PRs, 3_continuous_integration

Anything else?

No response

Things to do after the project is submitted and ready to review.

• Add a comment below the main post of your issue saying: Hi @brainhackorg/project-monitors my project is ready!
• Twitter-sized summary of your project pitch.

[tclose]

Hi @brainhackorg/project-monitors my project is ready!

[Remi-Gau]

Reopening to be able to "list it" in the brainhack jupyter book
Planning to update this page: http://brainhack.org/brainhack_jupyter_book/projects.html