Important

This guide hasn’t been updated since January 2017 and is based on an older version of Nipype. The code in this guide is not tested against newer Nipype versions and might not work anymore. For a newer, more up to date and better introduction to Nipype, please check out the the Nipype Tutorial.

Introduction to Nipype

What is Nipype?

Nipype (Neuroimaging in Python - Pipelines and Interfaces) is an open-source, user-friendly, community-developed software package under the umbrella of NiPy. Nipype allows you to pipeline your neuroimaging workflow in an intuitive way and enables you to use the software packages and algorithms you want to use, regardless their programing language. This is possible because Nipype provides an uniform interface to many existing neuroimaging processing and analysis packages like SPM, FreeSurfer, FSL, AFNI, ANTS, Camino, MRtrix, Slicer, MNE and many more.

Nipype allows you to easily combine all those heterogeneous software packages whithin a single workflow. This procedure gives you the opportunity to pick the best algorithm there is for the problem at hand and therefore allows you to profit from the advantages of any software package you like.

Nipype is written in Python, an easy to learn and very intuitive programming language. This means that your whole neuroimaging analysis can be easily specified using python scripts. It won’t even take as many lines of code as you might fear. Nipype is very straightforward and easy to learn. As you will see, it is quite simple to combine processing steps using different software packages. Steps from previous analyses can be reused effortlessly and new approaches can be applied much faster.

You’re still concerned because you want to combine your own bash, MATLAB, R or Python scripts with Nipype? No problem! Even the creation of your own interface to your own software solution is straightforward and can be done in a rather short time. Thanks to Github, there’s also always a community standing behind you.

Nipype provides an environment that encourages interactive exploration of algorithms. It allows you to make your research easily reproducible and lets you share your code with the community.

A Short Example

Let’s assume you want to do an Analysis that uses AFNI for the Motion Correction, FreeSurfer for the Coregistration, ANTS for the Normalization, FSL for the Smoothing, Nipype for the Model Specification, SPM for the Model Estimation and SPM for the Statistical Inference. Normally this would be a hell of a mess. Switching between multiple scripts in different programming languages with a lot of manual intervention. On top of all that, you want to do your analysis on multiple subjects, preferably as fast as possible, i.e., processing several subjects in parallel. With Nipype, this is no problem! You can do all this and much more.

To illustrate the straightforwardness of an Nipype workflow and show how simply it can be created, look at the following example. This figure shows you a simplification of the analysis workflow just outlined.

The code to create an Nipype workflow that specifies the steps illustrated in the figure above and can run all the steps would look something like this:

#Import modules
import nipype
import nipype.interfaces.afni        as afni
import nipype.interfaces.freesurfer  as fs
import nipype.interfaces.ants        as ants
import nipype.interfaces.fsl         as fsl
import nipype.interfaces.nipy        as nipy
import nipype.interfaces.spm         as spm


#Specify experiment specifc parameters
experiment_dir = '~/experiment_folder'
nameofsubjects = ['subject1','subject2','subject3']

#Where can the raw data be found?
grabber = nipype.DataGrabber()
grabber.inputs.base_directory = experiment_dir + '/data'
grabber.inputs.subject_id = nameofsubjects

#Where should the output data be stored at?
sink = nipype.DataSink()
sink.inputs.base_directory = experiment_dir + '/output_folder'


#Create a node for each step of the analysis

#Motion Correction (AFNI)
realign = afni.Retroicor()

#Coregistration (FreeSurfer)
coreg = fs.BBRegister()

#Normalization (ANTS)
normalize = ants.WarpTimeSeriesImageMultiTransform()

#Smoothing (FSL)
smooth = fsl.SUSAN()
smooth.inputs.fwhm = 6.0

#Model Specification (Nipype)
modelspec = nipype.SpecifyModel()
modelspec.inputs.input_units = 'secs'
modelspec.inputs.time_repetition = 2.5
modelspec.inputs.high_pass_filter_cutoff = 128.

#Model Estimation (SPM)
modelest = spm.EstimateModel()

#Contrast Estimation (SPM)
contrastest = spm.EstimateContrast()
cont1 = ['human_faces',  [1 0 0]]
cont2 = ['animal_faces', [0 1 0]]
contrastest.inputs.contrasts = [cont1, cont2]

#Statistical Inference (SPM)
threshold = spm.Threshold()
threshold.inputs.use_fwe_correction = True
threshold.inputs.extent_fdr_p_threshold = 0.05


#Create a workflow to connect all those nodes
analysisflow = nipype.Workflow()

#Connect the nodes to each other
analysisflow.connect([(grabber     ->  realign    ),
                      (realign     ->  coreg      ),
                      (coreg       ->  normalize  ),
                      (normalize   ->  smooth     ),
                      (smooth      ->  modelspec  ),
                      (modelspec   ->  modelest   ),
                      (modelest    ->  contrastest),
                      (contrastest ->  threshold  ),
                      (threshold   ->  sink       )
                      ])

#Run the workflow in parallel
analysisflow.run(mode='parallel')

By using multicore processing, SGE, PBS, Torque, HTCondor, LSF or other plugins for parallel execution you will be able to reduce your computation time considerably. This means, that an analysis of 24 subjects where each takes one hour to process would normally take about one day, but it could be done on a single machine with eight processors in under about three hours.

Note

The code above is of course a shortened and simplified version of the real code. But it gives you a good idea of what the code would look like, and how straightforward and readable the programming of a neuroimaging pipeline with Nipype is.

Nipype’s Architecture

Nipype consists of many parts, but the most important ones are Interfaces, the Workflow Engine and the Execution Plugins.

Note

For a deeper understanding of Nipype go either to Nipype’s main homepage or read the official paper: Gorgolewski K, Burns CD, Madison C, Clark D, Halchenko YO, Waskom ML, Ghosh SS (2011) Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in Python. Front. Neuroinform. 5:13. http://dx.doi.org/10.3389/fninf.2011.00013

Interfaces

Interfaces in the context of Nipype are program wrappers that enable Nipype, which runs in Python, to run a program or function in any other programming language. As a result, Python becomes the common denominator of all neuroimaging software packages and allows Nipype to easily connect them to each other. A short tutorial about interfaces can be found on the official homepage. More practical examples will be given later in this beginner’s guide.

For a full list of software interfaces supported by Nipype go here.

Workflow Engine

The core of Nipype’s architecture is the workflow engine. It consists of Nodes, MapNodes and Workflows, which can be interconnected in various ways.

• Node: A node provides the information – parameters, filenames, etc. – that is needed by an interface to run the program properly for a particular job, whether as part of a workflow or separately.

• MapNode: A Mapnode is quite similar to a Node, but it differs because it takes multiple inputs of a single type to create a single output. For example, it might specify multiple DICOM files to create one NIfTI file.

• Workflow: A workflow (also called a pipeline), is a directed acyclic graph (DAG) or forest of graphs whose nodes are of type Node, MapNode or Workflow and whose edges (lines connecting nodes) represent data flow.

Each Node, MapNode or Workflow has (at least) one input field and (at least) one output field. Those fields specify the dataflow into and out of a Node, MapNode or Workflow. MapNodes use fields to specify multiple inputs (basically a list of input items). There they are called iterfields because the interface will iterate over the list of input items, and they have to be labeled as such to distinguish them from single-item fields.

A very cool feature of a Nipype workflow are so called iterables. Iterables allow you to run a given workflow or subgraph several times with changing input values. For example, if you want to run an analysis pipeline on multiple subjects or with an FWHM smoothing kernel of 4mm, 6mm, and 8mm. This can easily be achieved with iterables and additionally allows you to do this all in parallel (simultaneous execution), if requested.

Go to the documentation section of Nipype’s main homepage to read more about MapNode, iterfield, and iterables, JoinNode, synchronize and itersource and much more. Nonetheless, a more detailed explanation will be given in a later section of this beginner’s guide.

Note

For more practical and extended examples of Nipype concepts see Michael Waskom’s really cool Jupyter notebooks about Interfaces, Iteration and Workflows.

Execution Plugins

Plugins are components that describe how a workflow should be executed. They allow seamless execution across many architectures and make using parallel computation quite easy.

On a local machine, you can use the plugin Serial for a linear, or serial, execution of your workflow. If you machine has more than one core, you can use the Multicore plugin for parallel execution of your workflow. On a cluster, you have the option of using plugins for:

• HTCondor

• PBS, Torque, SGE, LSF (native and via IPython)

• SSH (via IPython)

• Soma Workflow

Note

Cluster operation often needs a special setup. You may wish to consult your cluster operators about which plugins are available.

To show how easily this can be done, the following code shows how to run a workflow with different plugins:

# Normally calling run executes the workflow in series
workflow.run()

# But you can scale to parallel very easily.
# For example, to use multiple cores on your local machine
workflow.run('MultiProc', plugin_args={'n_procs': 4})

# or to other job managers
workflow.run('PBS', plugin_args={'qsub_args': '-q many'})
workflow.run('SGE', plugin_args={'qsub_args': '-q many'})
workflow.run('LSF', plugin_args={'qsub_args': '-q many'})
workflow.run('Condor')
workflow.run('IPython')

# or submit graphs as a whole
workflow.run('PBSGraph', plugin_args={'qsub_args': '-q many'})
workflow.run('SGEGraph', plugin_args={'qsub_args': '-q many'})
workflow.run('CondorDAGMan')

More about Plugins can be found on Nipype’s main homepage under Using Nipype Plugins.