Correcting netcdf file data and metadata using NCO

Sometimes we get data in netcdf files that do not really conform to the standards used by different tools that were developed for calculating standard properties of the data. Here an example is shown how the files of this type can be easily corrected using NCO and shell:



After having run the script the command `cdo infov fname` should give correct dates, minimum and maximum values (which previously was not the case): Here for comparison I've put the result of the same command on the initial version of the file:

Summation along a dimension

Suppose you have a netcdf file with temporal data of say evaporation (in meters). And you want to know how much water evaporates during a year. And also you do not want to loose time writing a program for this. Fortunately we have NCO. I did the procedure for the ERA-Interim reanalysis file downloaded from the ECMWF site. I picked the year 1985. The header of the file looks like this:


Here is the description of the procedure that I used:

1. Calculate the mean of the variable along the specified dimension (time in my case).

   ~# ncra -d time,0,729 -v e evap-1985.nc mean_evap_1985.nc
   

   Here I was not sure whether I should take care of the scale_factor and add_offset. The answer is no, because nco takes care of it for you.
2. Then, since sum = mean * ntimes, I use the mean to get the sum. Minus is because of the model convention, they consider the flux negative if it is upwards.

   ~# ncap -O -s "e=-730*e" mean_evap_1985.nc sum_evap_1985.nc
   

3. So we have our resulting field in the file sum_evap_1985.nc, which can be viewed using ncview. The field looks as shown below. I do not know how to save the legend in ncview, so the figure is not very informative. The result corresponds to the means given by ECMWF: E-P and P.
   
   
   

Data source: "ECMWF ERA-Interim data used here have been obtained from the ECMWF data server."

NCO

NCO is a very powerfull tool for editing netcdf files. In this post I will be writing some useful commands.

1. Delete a dimension of size 1 (the dimension name is level):

ncwa -a level directions_qc_amno.nc directions_qc_amno1.nc

2. Copy data from one to the other netcdf file, if possible recognize common dimensions:

ncks file1.nc file2.nc

3. Rename a dimension

ncrename -d rlat,latitude directions_qc_amno.nc

4. permute dimensions(make longitude first and the latitude second):

ncpdq -a longitude,latitude infocell.nc infocell.nc 

5. Rename the variable lat to latitude:

ncrename -v lat,latitude infocell.nc

6. Take difference of variable x in 2 files:

ncbo -v x --op_typ='-' infocell.nc infocell_1.nc infocell_diff.nc

7. Operation on variables in a file. Here the values of the variable are changed and the attributes added.

ncap -O -s "air=(air-512.81)/0.01" filename.nc temp.nc
ncap -O -s "air=short(air);air@add_offset=512.81;air@scale_factor=0.01" temp.nc filename.nc

taken from http://jisao.washington.edu/data/nco/
8. Concantenating netcdf files along the record dimension.

ncrcat runoff_file_name_pattern total_runoff_af.nc

9. Subsetting by index and by coordinate values:

ncks -d lon,0,4 -d lat,2,9,2 in.nc out.nc
ncks -d lon,10.,20. -d lat,80.,90. in.nc out.nc

Used data from:

• http://linux.die.net/man/1/ncbo
• http://nco.sourceforge.net/#RTFM