NCO NetCDF Operators

The NCO toolkit manipulates and analyzes data stored in netCDF-accessible formats, including DAP, HDF4, and HDF5. It exploits the geophysical expressivity of many CF (Climate & Forecast) metadata conventions, the flexible description of physical dimensions translated by UDUnits, the network transparency of OPeNDAP, the storage features (e.g., compression, chunking, groups) of HDF (the Hierarchical Data Format), and many powerful mathematical and statistical algorithms of GSL (the GNU Scientific Library). NCO is fast, powerful, and free.

What is NCO?

The netCDF Operators (NCO) comprise a dozen standalone, command-line programs that take netCDF, HDF, and/or DAP files as input, then operate (e.g., derive new data, compute statistics, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats. NCO aids analysis of gridded scientific data. The shell-command style of NCO allows users to manipulate and analyze files interactively, or with expressive scripts that avoid some overhead of higher-level programming environments.

Traditional geoscience data analysis requires users to work with numerous flat (data in one level or namespace) files. In that paradigm instruments or models produce, and then repositories archive and distribute, and then researchers request and analyze, collections of flat files. NCO works well with that paradigm, yet it also embodies the necessary algorithms to transition geoscience data analysis from relying solely on traditional (or “flat”) datasets to allowing newer hierarchical (or “nested”) datasets.

The next logical step is to support and enable combining all datastreams that meet user-specified criteria into a single or small number of files that hold all the science-relevant data organized in hierarchical structures. NCO (and no other software to our knowledge) can do this now. We call the resulting data storage, distribution, and analysis paradigm Group-Oriented Data Analysis and Distribution (GODAD). GODAD lets the scientific question organize the data, not the ad hoc granularity of all relevant datasets. The User Guide illustrates GODAD techniques for climate data analysis:

Operator	Full Name	Examples
ncap2	netCDF Arithmetic Processor	here
ncatted	netCDF ATTribute EDitor	here
ncbo	netCDF Binary Operator	here
nces	netCDF Ensemble Statistics	here
ncecat	netCDF Ensemble conCATenator	here
ncflint	netCDF FiLe INTerpolator	here
ncks	netCDF Kitchen Sink	here
ncpdq	netCDF Permute Dimensions Quickly, Pack Data Quietly	here
ncra	netCDF Record Averager	here
ncrcat	netCDF Record conCATenator	here
ncremap	netCDF REMAPer	here
ncrename	netCDF RENAMEer	here
ncwa	netCDF Weighted Averager	here

How to Contribute: Volunteer, Endorse, or Donate

The NCO project homepage points to mailing lists, discussion forums, and instructions to make contributing easy. It contains everything about NCO that you may be looking for.