Correcting netcdf file data and metadata using NCO

Sometimes we get data in netcdf files that do not really conform to the standards used by different tools that were developed for calculating standard properties of the data. Here an example is shown how the files of this type can be easily corrected using NCO and shell:

	#!/bin/bash
	folder=.
	prefix=ANUSPLIN_latlon_stmn_
	var_name=daily_minimum_temperature
	for x in ${folder}/${prefix}*.nc
	do
	x_name=$(basename $x)
	part=$(echo $x_name | cut -d"." -f 1)
	#get the year and month from the file name
	y=$(echo $part | cut -d"_" -f 4)
	m=$(echo $part | cut -d"_" -f 5)
	#change time units
	ncatted -O -a units,time,m,c,"days since ${y}-${m}-01" $x
	# decrement the time values so they start from 0
	ncap2 -O -s "time=time-1;" $x -o $x
	#replace NaNs with a value specified in the missing_value attribute
	ncap2 -O -s "where(${var_name} != ${var_name}){${var_name}=${var_name}@missing_value;}" $x -o $x
	done

view raw changer.sh hosted with ❤ by GitHub

After having run the script the command `cdo infov fname` should give correct dates, minimum and maximum values (which previously was not the case):

	cdo infov ANUSPLIN_latlon_stmn_2005_11.nc
	-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
	1 : 2005-11-01 00:00:00 0 544680 287417 : -33.800 -10.348 10.620 : daily_minimum_temperature
	2 : 2005-11-02 00:00:00 0 544680 287417 : -38.890 -11.761 11.000 : daily_minimum_temperature
	3 : 2005-11-03 00:00:00 0 544680 287417 : -42.150 -12.516 11.690 : daily_minimum_temperature
	4 : 2005-11-04 00:00:00 0 544680 287417 : -46.670 -12.967 13.740 : daily_minimum_temperature
	5 : 2005-11-05 00:00:00 0 544680 287417 : -45.040 -13.658 13.910 : daily_minimum_temperature
	6 : 2005-11-06 00:00:00 0 544680 287417 : -45.100 -14.014 11.840 : daily_minimum_temperature
	7 : 2005-11-07 00:00:00 0 544680 287417 : -43.120 -13.841 10.800 : daily_minimum_temperature
	8 : 2005-11-08 00:00:00 0 544680 287417 : -41.860 -13.430 10.730 : daily_minimum_temperature
	9 : 2005-11-09 00:00:00 0 544680 287417 : -42.100 -13.669 11.880 : daily_minimum_temperature
	10 : 2005-11-10 00:00:00 0 544680 287417 : -41.200 -12.747 9.1600 : daily_minimum_temperature
	11 : 2005-11-11 00:00:00 0 544680 287417 : -44.540 -13.257 7.7200 : daily_minimum_temperature
	12 : 2005-11-12 00:00:00 0 544680 287417 : -43.940 -14.471 9.4400 : daily_minimum_temperature
	13 : 2005-11-13 00:00:00 0 544680 287417 : -43.750 -15.916 12.520 : daily_minimum_temperature
	14 : 2005-11-14 00:00:00 0 544680 287417 : -46.490 -17.539 8.9400 : daily_minimum_temperature
	15 : 2005-11-15 00:00:00 0 544680 287417 : -40.260 -18.271 8.3900 : daily_minimum_temperature
	16 : 2005-11-16 00:00:00 0 544680 287417 : -38.060 -18.373 8.1000 : daily_minimum_temperature

view raw cdo_infov_output.txt hosted with ❤ by GitHub

Here for comparison I've put the result of the same command on the initial version of the file:

	cdo infov ANUSPLIN_latlon_stmn_2005_11.nc
	-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
	1 : 0000-00-01 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	2 : 0000-00-02 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	3 : 0000-00-03 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	4 : 0000-00-04 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	5 : 0000-00-05 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	6 : 0000-00-06 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	7 : 0000-00-07 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	8 : 0000-00-08 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	9 : 0000-00-09 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	10 : 0000-00-10 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	11 : 0000-00-11 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	12 : 0000-00-12 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	13 : 0000-00-13 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	14 : 0000-00-14 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	15 : 0000-00-15 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	16 : 0000-00-16 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature
	17 : 0000-00-17 00:00:00 0 544680 0 : nan nan nan : daily_minimum_temperature

view raw cdo_infov_old_output.txt hosted with ❤ by GitHub