fld* operators use NaNs in calculation
Added by Ryan Love about 7 years ago
Hello,
I have a file on a regular latlon grid and am trying to get the sum of the field at every timestep. In the field the land is masked out as NaNs and I would like to obtain the sum over the ocean cells only. Unfortunately it seems the some of the fld suite of operators does not respect this (fldsum and fldmean tested) and includes the NaNs in the calculations which results in no usable data due to the propagation of NaNs. See the attached for an input file I'm trying to operate on. Is this desired behavior? If so could I put in a feature request to disable this with an environment variable or flag so that NaNs are not included in such calculations?
Cheers,
Ryan
Replies (5)
RE: fld* operators use NaNs in calculation - Added by Ryan Love about 7 years ago
cdo -V output (forgot a 2nd time now...)
Climate Data Operators version 1.7.0 (http://mpimet.mpg.de/cdo)
Compiler: gcc -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wall -pedantic -fPIC -fopenmp
version: gcc (Ubuntu 5.3.1-9ubuntu2) 5.3.1 20160220
Features: DATA PTHREADS OpenMP4 HDF5 NC4/HDF5/threadsafe OPeNDAP SZ Z UDUNITS2 PROJ.4 MAGICS CURL FFTW3 SSE2
Libraries: HDF5/1.8.16 proj/4.92 curl/7.47.0
Filetypes: srv ext ieg grb grb2 nc nc2 nc4 nc4c
CDI library version : 1.7.0
GRIB_API library version : 1.14.4
netCDF library version : 4.4.0 of Mar 29 2016 11:41:40 $
HDF5 library version : 1.8.16
SERVICE library version : 1.4.0
EXTRA library version : 1.4.0
IEG library version : 1.4.0
FILE library version : 1.8.2
RE: fld* operators use NaNs in calculation - Added by Ryan Love about 7 years ago
Also persists up to 1.8rc5
RE: fld* operators use NaNs in calculation - Added by Ryan Love about 7 years ago
Ryan Love
wrote:
Also persists up to 1.8rc5
1.8.0rc5 that is
RE: fld* operators use NaNs in calculation - Added by Ryan Love about 7 years ago
Aha! Seems CDO doesn't automatically assume NaNs are missing values but rather it explicitly checks for the 'missing_value' attribute in a variable, which in my case did not exist but was easily added using cdo setattribute,s@missing_value=NaN
RE: fld* operators use NaNs in calculation - Added by Ralf Mueller about 7 years ago
you can use cdo to do this
cdo -setctomiss,nan SALTanom.maskedNA.1-9.nc t.nc
then the fldmean works
<ram@luthien:~/local/data/cdo>
% cdo -infov -fldmean SALTanom.maskedNA.1-9.nc [Tue 2017-03-28|12:56:13]
cdo infon: Started child process "fldmean SALTanom.maskedNA.1-9.nc (pipe1.1)".
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
1 : 0000-00-00 00:00:00 1 1 0 : nan : s
2 :-0001-11-30 00:05:49 1 1 0 : nan : s
3 :-0001-11-30 00:11:38 1 1 0 : nan : s
4 :-0001-11-30 00:16:47 1 1 0 : nan : s
5 :-0001-11-30 00:22:36 1 1 0 : nan : s
6 :-0001-11-30 00:27:45 1 1 0 : nan : s
7 :-0001-11-30 00:33:34 1 1 0 : nan : s
8 :-0001-11-30 00:38:43 1 1 0 : nan : s
9 :-0001-11-30 00:44:32 1 1 0 : nan : s
cdo(2) fldmean: Processed 527850 values from 1 variable over 9 timesteps ( 0.04s )
cdo infon: Processed 9 values from 1 variable over 9 timesteps ( 0.04s )
<ram@luthien:~/local/data/cdo>
% cdo -infov -fldmean t.nc [Tue 2017-03-28|12:56:29]
cdo infon: Started child process "fldmean t.nc (pipe1.1)".
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
1 : 0000-00-00 00:00:00 1 1 0 : 0.39752 : s
2 :-0001-11-30 00:05:49 1 1 0 : 0.40151 : s
3 :-0001-11-30 00:11:38 1 1 0 : 0.41480 : s
4 :-0001-11-30 00:16:47 1 1 0 : 0.42128 : s
5 :-0001-11-30 00:22:36 1 1 0 : 0.42395 : s
6 :-0001-11-30 00:27:45 1 1 0 : 0.42305 : s
7 :-0001-11-30 00:33:34 1 1 0 : 0.42434 : s
8 :-0001-11-30 00:38:43 1 1 0 : 0.42489 : s
9 :-0001-11-30 00:44:32 1 1 0 : 0.42295 : s
cdo(2) fldmean: Processed 527850 values from 1 variable over 9 timesteps ( 0.04s )
cdo infon: Processed 9 values from 1 variable over 9 timesteps ( 0.04s )
CDO uses CF-compliant missing value declaration
float s(time, lev, lat, lon) ;
s:_FillValue = -9.e+33f ;
hth
ralf