The NCO toolkit manipulates and analyzes data stored in netCDF-accessible formats, including DAP, HDF4, and HDF5. It exploits the geophysical expressivity of many CF (Climate & Forecast) metadata conventions, the flexible description of physical dimensions translated by UDUnits, the network transparency of OPeNDAP, the storage features (e.g., compression, chunking, groups) of HDF (the Hierarchical Data Format), and many powerful mathematical and statistical algorithms of GSL (the GNU Scientific Library). NCO is fast, powerful, and free.
The netCDF Operators (NCO) comprise a dozen standalone, command-line programs that take netCDF, HDF, and/or DAP files as input, then operate (e.g., derive new data, compute statistics, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats. NCO aids analysis of gridded scientific data. The shell-command style of NCO allows users to manipulate and analyze files interactively, or with expressive scripts that avoid some overhead of higher-level programming environments.
Traditional geoscience data analysis requires users to work with numerous flat (data in one level or namespace) files. In that paradigm instruments or models produce, and then repositories archive and distribute, and then researchers request and analyze, collections of flat files. NCO works well with that paradigm, yet it also embodies the necessary algorithms to transition geoscience data analysis from relying solely on traditional (or “flat”) datasets to allowing newer hierarchical (or “nested”) datasets.
The next logical step is to support and enable combining all datastreams that meet user-specified criteria into a single or small number of files that hold all the science-relevant data organized in hierarchical structures. NCO (and no other software to our knowledge) can do this now. We call the resulting data storage, distribution, and analysis paradigm Group-Oriented Data Analysis and Distribution (GODAD). GODAD lets the scientific question organize the data, not the ad hoc granularity of all relevant datasets. The User Guide illustrates GODAD techniques for climate data analysis:
| Operator | Full Name | Examples |
|---|---|---|
ncap2 |
netCDF Arithmetic Processor | here |
ncatted |
netCDF ATTribute EDitor | here |
ncbo |
netCDF Binary Operator | here |
nces |
netCDF Ensemble Statistics | here |
ncecat |
netCDF Ensemble conCATenator | here |
ncflint |
netCDF FiLe INTerpolator | here |
ncks |
netCDF Kitchen Sink | here |
ncpdq |
netCDF Permute Dimensions Quickly, Pack Data Quietly | here |
ncra |
netCDF Record Averager | here |
ncrcat |
netCDF Record conCATenator | here |
ncremap |
netCDF REMAPer | here |
ncrename |
netCDF RENAMEer | here |
ncwa |
netCDF Weighted Averager | here |
The NCO project homepage points to mailing lists, discussion forums, and instructions to make contributing easy. It contains everything about NCO that you may be looking for.