BAWiki - User contributions [en]

Differences of Calculated Results

2022-05-16T08:25:08Z

Rosenhagen: /* Other skill definitions */

[[de:Differenzen der Berechnungsergebnisse]]
__NOAUTOLINKS__
__NOAUTOLINKTARGET__

==Introduction==

For data generated by
* [[Mathematical Models for Coastal Areas and Estuaries|mathematical models]] ([[model]] results), or
* [[Analysis of Calculated Results|analysis of calculated results]] (characteristic numbers), or
* measured data (observational data)
Various differences can be computed. Input data can be typically categorized as follows:
* '''Category K0''': <math>f(x,y,z)</math>, time-independent quantities;
* '''Category K1''': <math>f(x,y,z,t_1)</math>, time-dependent quantities, one time step;
* '''Category KC''': <math>f(x,y,z,t_i)</math>, time-dependent quantities, several discrete time steps, constant time step <math>\Delta_t</math>;
* '''Category KN''': <math>f(x,y,z,t_i)</math>, time-dependent quantities, several discrete time steps, varying time step <math>\Delta_t(i)</math>.
For geophysical data categories K1, KC and KN are of significance. Examples:
* Category K1: topography/bathymetry <math>h(x,y,z,t_1)</math> for a specific instant in time;
* Category KC: [[water level]] <math>\eta(x,y,z,t_i)</math> at discrete times <math>t_i</math> with constant time step, e. g. computed by a mathematical model;
* Category KN: tidal high water <math>\eta^{\rm{HW}}(x,y,z,t_i)</math> for times <math>t_i</math> at ''non-equidistant'' time intervals, e.g. derived from a water level time serie.

==Definitions==

* '''reference data <math>r</math>''': with respect to <math>r</math> various ''deviations'' for <math>f</math> can be evaluated. Typical data are either observational data or computational as well as analysis results for a specific (reference) state (situation);
* '''variant data <math>f</math>''': can be also either observational data or computational as well as analysis results, for which deviations shall be computed with respect to the reference state. Typically variant data are given for a different period in time (natural variation) or a different state of the system under study.
* '''valid operator 1''': <math>V(r_i)</math> returns .T. or .F., in dependence whether <math>r_i</math> is valid or invalid. Can be also applied to <math>f_i</math>.
* '''valid operator 2''': <math>V(r_I,f_i)</math> returns .T. or .F., in dependence whether <math>V(r_i)\land V(f_i)</math> is valid or invalid.
* '''integer operator 1''': <math>P(r_i)</math> returns 1 if <math>V(r_i)</math> else 0. Similar for <math>f_i</math>.
* '''integer operator 2''': <math>P(r_i,f_i)</math> returns 1 if <math>V(r_i)\land V(f_i)</math> .T. else 0.

==Requirements for the computation of differences==

The following requirements must be fulfilled by <math>r</math> and <math>f</math>:
# <math>r</math> and <math>f</math> must belong to the same category (see above);
# the number of times <math>t_i</math> must be identical for <math>r</math> and <math>f</math>;
# for data belonging to category KC constant time steps must coincide <math>\Delta t</math> for <math>r</math> and <math>f</math>;
# (physical) dimension as well as meaning must be equivalent for <math>r</math> and <math>f</math>;
# <math>r_i</math> (short for <math>r(x,y,z,t_i)</math>) as well as <math>f_i</math> (short for <math>r(x,y,z,t_i)</math>) must be valid data for the same instant <math>i</math> in time; otherwise the dervied results will become invalid.

==Computational results==

Program [[NCDELTA]] can be used to compute all subsequent results. Locations of <math>r</math> are not required to coincide with those of <math>f</math>. Values <math>r</math> are interpolated to locations of <math>f</math>, as long as the geographical distance between the different locations does not exceed <math>R^\max</math>. In case the distance ''exceeds'' that limit, no results will be computed. In such a situation an ''invalid'' result value will be generated. The follwing results can be computed using [[NCDELTA]].

===Ordinary differences===

====Difference====

A result is computed for all times (one value for time-independent data) at all locations <math>(x,y,z)</math>:
# The difference between <math>f_i</math> and <math>r_i</math> is calculated in case <math>V(r_i,f_i)</math> returns .T.:
#:<math>d_i = f_i - r_i</math>, if <math>V(r_i,f_i)</math>;
# Result will be ''invalid'', if <math>V(r_i,f_i)</math> returns .F.:
#:<math>d_i = \rm{invalid}</math> if <math>\lnot V(r_i,f_i)</math>.
Results are computed for data belonging to categories '''K0''', '''K1''', '''KC''' und '''KN''', which means for all types of data.

====Maximum difference====

Maximum difference is determined using ''absolute'' value (modulus) in combination with sign preservation:
# At first all differences <math>d_i</math> will be computed as indicated above;
# Out of all valid data [[index]] <math>i^\max</math> is determined in such a way that <math>\left|d_i\right|</math> is maximal
#:<math>d^\max = d_{i^\max}</math>
#:is equal to the ''maximum difference'' according to this definition; this value can be negative, positive or zero;
# In case all values <math>d_i</math> are invalid <math>d^\max = \rm{invalid}</math> will be set.
Computation is performed for categories '''KC''' and '''KN'''. A valid <math>d^\max</math> is obtained as long there exists at least one valid difference <math>d_i</math>. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

====Minimum difference====

Minimum difference is determined using ''absolute'' value (modulus) in combination with sign preservation:
# At first all differences <math>d_i</math> will be computed as indicated above;
# Out of all valid data index <math>i^\min</math> is determined in such a way that <math>\left|d_i\right|</math> is minimal
#:<math>d^\min = d_{i^\min}</math>
#:is equal to the ''minimum difference'' according to this definition; this value can be negative, positive or zero;
# In case all values <math>d_i</math> are invalid <math>d^\min = \rm{invalid}</math> will be set.
Computation is performed for categories '''KC''' and '''KN'''. A valid <math>d^\min</math> is obtained as long there exists at least one valid difference <math>d_i</math>. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

====Mean difference====

Mean value is computed for all valid differences:
# At first all differences <math>d_i</math> will be computed as indicated above;
# From all valid differences the mean value is computed as
#:<math>d^{\rm{mit}}=\frac{\sum_{i\in I}P(d_i)d_i}{\sum_{i\in I}P(d_i)}</math>;
# In case all <math>d_i</math> are invalid <math>d^{\rm{mit}} = \rm{invalid}</math> will be set.
Computation is performed for categories '''KC''' and '''KN'''. A valid <math>d^{\rm{mit}}</math> is obtained as long there exists at least one valid difference <math>d_i</math>. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

====Mean deviation====

Mean deviation is computed for all valid differences:
# At first all differences <math>d_i</math> will be computed as indicated above;
# From all valid differences the mean deviation is computed as
#:<math>d^{\rm{abw}}=\frac{\sum_{i\in I}P(d_i)\left|d_i\right|}{\sum_{i\in I}P(d_i)}</math>;
# In case all <math>d_i</math> are invalid <math>d^{\rm{abw}} = \rm{invalid}</math> will be set.
Computation is performed for categories '''KC''' and '''KN'''. A valid <math>d^{\rm{abw}}</math> is obtained as long there exists at least one valid difference <math>d_i</math>. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

====Root mean square error (RMSE)====

RMSE is computed for all valid differences:
# At first all differences <math>d_i</math> will be computed as indicated above;
# From all valid differences the mean deviation is computed from the well known definition for RMSE;
# In case all <math>d_i</math> are invalid RMSE = '''invalid''' will be set.
Computation is performed for categories '''KC''' and '''KN'''. A valid RMSE is obtained as long there exists at least one valid difference <math>d_i</math>. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

====Number of valid differences====

The number of valid differences may vary between different locations. As a consequence the above mentioned quantities may be computed from data sets of different size:
# At first all differences <math>d_i</math> will be computed as indicated above;
# The number of valid differences is computed as
#:<math>N_{\rm{ord}}=\sum_{i\in I}P(d_i)</math>;
Computation is performed for categories '''KC''' and '''KN'''. In program [[NCPLOT]] quantities like ''maximum difference'', ''minimum difference'', ''mean value'' as well as ''mean deviation'' can be visualised using this variable as a [[filter]].
In that way visualizations can be created where the results are shown for points only where e. g. either the maximum number of events or a specific number of events occurred.

===Data for a Taylor diagram===

Taylor diagrams provide "''a concise statistical summary of how well patterns match each other in terms of their correlation, their root-mean-square difference and the ratio of their variances''." Additional information such as [[bias]] can be added to the conventional Taylor diagram. The Taylor diagram provides a graphical framework that allows a suite of variables from a variety of (say) one or more models or reanalyses to be compared to reference data. The reference data can be observationally based (eg, [[reanalysis]]) or to another model or a control run.

Literature and further informations:
# Taylor, K. E. (2001), ''Summarizing multiple aspects of model performance in a single diagram'', Journal of Geophysical Research, 106 (D7), 7183–7192, doi: [http://dx.doi.org/10.1029/2000JD900719 http://dx.doi.org/10.1029/2000JD900719];
# [http://www-pcmdi.llnl.gov/about/staff/Taylor/CV/Taylor_diagram_primer.htm http://www-pcmdi.llnl.gov/about/staff/Taylor/CV/Taylor_diagram_primer.htm] with a short introduction by K. E. Taylor as well as links to example applications.

====Standard deviation for reference data====

Standard deviation is going to be computed for all valid reference data:
# Mean value for <math>r_i</math> is computed according to
#:<math>\bar{r}=\frac{\sum_{i\in I}P(r_i,f_i)r_i}{\sum_{i\in I}P(r_i,f_i)}</math>;
# If a valid <math>\bar{r}</math> was computed standard deviation is obtained from
#:<math>\sigma_r = \sqrt{\frac{\sum_{i\in I}P(r_i,f_i)\left(r_i-\bar{r}\right)^2}{\sum_{i\in I}P(r_i,f_i)}}</math>;
# In case all <math>P(r_i,f_i)</math> are 0 <math>\sigma_r = \rm{invalid}</math> is set.
Computation is performed for categories '''KC''' and '''KN'''. A valid <math>\sigma_r</math> is obtained as long as there exists at least one valid difference <math>V(r_i,f_i)</math>. Standard deviations for different locations may be computed from data sets of different size. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid Taylor data''.

Remark: We apply <math>P(r_i,f_i)</math> instead of <math>P(r_i)</math>. This guarantees that all Taylor data are computed for the same data set size at a specific location.

====Standard deviation for variant data====

Standard deviation is going to be computed for all valid variant data:
# Mean value for <math>f_i</math> is computed according to
#:<math>\bar{f}=\frac{\sum_{i\in I}P(r_i,f_i)f_i}{\sum_{i\in I}P(r_i,f_i)}</math>;
# If a valid <math>\bar{f}</math> was computed standard deviation is obtained from
#:<math>\sigma_f = \sqrt{\frac{\sum_{i\in I}P(r_i,f_i)\left(f_i-\bar{f}\right)^2}{\sum_{i\in I}P(r_i,f_i)}}</math>;
# In case all <math>P(r_i,f_i)</math> are 0 <math>\sigma_f = \rm{invalid}</math> is set.
Computation is performed for categories '''KC''' and '''KN'''. A valid <math>\sigma_f</math> is obtained as long as there exists at least one valid difference <math>V(r_i,f_i)</math>. Standard deviations for different locations may be computed from data sets of different size. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid Taylor data''.

Remark: We apply <math>P(r_i,f_i)</math> instead of <math>P(r_i)</math>. This guarantees that all Taylor data are computed for the same data set size at a specific location.

====Correlation====

Correlation is computed for <math>r</math> and <math>f</math>:
# Mean value <math>\bar{r}</math> for reference data is computed as indicated above;
# Mean value <math>\bar{f}</math> for variant data is computed as indicated above;
# Standard deviation <math>\sigma_r</math> for reference data is computed as indicated above;
# Standard deviation <math>\sigma_f</math> for variant data is computed as indicated above;
# Correlation <math>R</math> is given as follows
#:<math>R=\frac{\sum_{i\in I}P(r_i,f_i)\left(r_i-\bar{r}\right)\left(f_i-\bar{f}\right)}{\sigma_r\sigma_f\sum_{i\in I}P(r_i,f_i)}</math>;
# In case all <math>V(r_i,f_i)</math> are invalid <math>R = \rm{invalid}</math> is set.
Computation is performed for categories '''KC''' und '''KN'''. A valid <math>R</math> is obtained as long as there exists at least one valid difference <math>V(r_i,f_i)</math>. Correlations for different locations may be computed from data sets of different size. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid Taylor data''.

====Pattern RMS====

Pattern RMS is computed for <math>r</math> and <math>f</math>:
# Mean value <math>\bar{r}</math> for reference data is computed as indicated above;
# Mean value <math>\bar{f}</math> for variant data is computed as indicated above;
# Pattern RMS <math>E'</math> is given as follows
#:<math>E'=\sqrt{\frac{\sum_{i\in I}P(r_i,f_i)\left[\left(r_i-\bar{r}\right)\left(f_i-\bar{f}\right)\right]^2}{\sum_{i\in I}P(r_i,f_i)}}</math>;
# In case all <math>V(r_i,f_i)</math> are invalid <math>E' = \rm{invalid}</math> is set.
Computation is performed for categories '''KC''' und '''KN'''. A valid <math>E'</math> is obtained as long as there exists at least one valid difference <math>V(r_i,f_i)</math>. Correlations for different locations may be computed from data sets of different size. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid Taylor data''.

====Deviation of means (''Bias'')====

Deviation of means is computed for <math>\bar{r}</math> and <math>\bar{f}</math>; this quantity is also called ''bias'':
# Mean value <math>\bar{r}</math> for reference data is computed as indicated above;
# Mean value <math>\bar{f}</math> for variant data is computed as indicated above;
# Deviation of means <math>\bar{E}</math> is given as follows
#:<math>\bar{E}=\bar{f}-\bar{r}</math>;
# In case all <math>V(r_i,f_i)</math> are invalid <math>\bar{E} = \rm{invalid}</math> is set;
# Overall RMS <math>E</math> can be computed from <math>\bar{E}</math> and <math>E'</math> according to
#: <math>E = \sqrt{\bar{E}^2+E'^2}</math>;
Computation is performed for categories '''KC''' und '''KN'''. A valid <math>\bar{E}</math> is obtained as long as there exists at least one valid difference <math>V(r_i,f_i)</math>. Biases for different locations may be computed from data sets of different size. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid Taylor data''.

====Root mean square error (RMSE) according to Taylor====

RMSE is computed according to Taylor (2001, equation 3) from ''Pattern RMS'' and ''Bias''. Computation is performed for categories '''KC''' and '''KN'''. A valid RMSE is obtained as long as there exists at least one valid difference <math>d_i</math>. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

Definition of RMSE according to Taylor (2001) is mathematically identical to the standard equation for RMSE but computational results may differ slightly due to numerical roundoff.

====Taylor Skill 4====

Skill S4 is computed according to Taylor (2001, equation 4) from correlation and normalized variance.
Computation is performed for categories '''KC''' and '''KN'''. A valid S4 is obtained as long as there exists at least one valid difference <math>d_i</math>. S4 can be characterised as follows:
* S4 = 1.0 indicates perfect fit;
* S4 = 0.0, if correlation R = -1.0 or variance of the variant tends towards 0.0 or infinity;
* S4 is linear with respect to R (at constant variance);
* For zero variance S4 is not defined;
* Bias between variant and reference has no influence on S4.
S4 ''punishes'' deviations in ''Pattern RMS'', and is more ''tolerant'' with respect to deviations in R. See Taylor (2001, figure 10).

====Taylor Skill 5====

Skill S5 according to Taylor (2001, euqation 5) is computed from correlation and normalized variance.
Computation is performed for categories '''KC''' and '''KN'''.

In contrast to S4 deviations with respect to ''Pattern RMS'' and correlation are treated (''punished'') similarly. See Taylor (2001, figure 11).

====Number of valid reference data====

Number of valid reference data <math>r_i</math> may differ between locations:
# The number of valid reference data is given by
#:<math>N_r=\sum_{i\in I}P(r_i)</math>;
Computation is performed for categories '''KC''' und '''KN'''. This quantity is of purely informative character and is not truely required for any of the Taylor diagram data.

====Number of valid variant data====

Number of valid variant data <math>f_i</math> may differ between locations:
# The number of valid variant data is given by
#:<math>N_f=\sum_{i\in I}P(f_i)</math>;
Computation is performed for categories '''KC''' and '''KN'''. This quantity is of purely informative character and is not truely required for any of the Taylor diagram data.

====Number of valid Taylor data====

The number of valid Taylor data <math>\bar{r},\bar{f},\sigma_r,\sigma_f,R,E'</math> and <math>\bar{E}</math> may be different from location to location:
# The number of valid Taylor data is given by
#:<math>N_T=\sum_{i\in I}P(r_i,f_i)</math>;
Computation is performed for categories '''KC''' and '''KN'''. In programs like [[NCPLOT]] the above mentioned Taylor data can be filtered using this quantity. This offers the opportunity to present results for e. g. locations where all events have occurred, or for locations where only a specific number of events has occurred.

===Median and quantiles===

A prerequisite for the computation of all subsequent quantities is that all valid differences <math>d_i</math> have to be sorted in ascending order: all <math>N_{\rm{ord}}</math> valid differences <math>d_i</math> are sorted in ascending order into <math>s_j</math>, with <math>j \in [1:N_{\rm{ord}}]</math>. Subsequently <math>n:=N_{\rm{ord}}</math> will be used for sake of simplicity.

Computation is carried through if <math>n \ge 32</math> holds.

====Median====

Median is computed for all valid differences <math>d_i</math>:
# if <math>n</math> odd: <math>d_{\rm{Med}} = s_\frac{n+1}{2}</math>;
# if <math>n</math> even: <math>d_{\rm{Med}} = 0.5\left( s_{\frac{n}{2}}+s_{\frac{n}{2}+1}\right)</math>;
# in case that there are less than 32 valid data <math>d_{\rm{Med}} = \rm{invalid}</math> is set.
Computation is performed for categories '''KC''' and '''KN'''. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

====Quantile Q01====

Quantile <math>p=0.01</math> is computed for all valid differences <math>d_i</math>. In other words: we obtain a specific value
for <math>d_i</math>, which is deceeded by just 1 % of all events but exceeded by 99 % of all events:
# if <math>n \cdot p</math> integer: <math>d_{\rm{Q01}} = 0.5\left( s_{n \cdot p}+s_{n \cdot p+1}\right)</math>;
# if <math>n \cdot p</math> real: <math>d_{\rm{Q01}} = s_{\lceil n \cdot p \rceil}</math>;
# in case that there are less than 32 valid data <math>d_{\rm{Q01}} = \rm{invalid}</math> is set.
Computation is performed for categories '''KC''' and '''KN'''. <math>d_{\rm{Q01}}</math> may be computed for different data set sizes at different locations. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

====Quantile Q05====

Quantile <math>p=0.05</math> is computed for all valid differences <math>d_i</math>. In other words: we obtain a specific value
for <math>d_i</math>, which is deceeded by 5 % of all events but exceeded by 95 % of all events:
# if <math>n \cdot p</math> integer: <math>d_{\rm{Q05}} = 0.5\left( s_{n \cdot p}+s_{n \cdot p+1}\right)</math>;
# if <math>n \cdot p</math> real: <math>d_{\rm{Q05}} = s_{\lceil n \cdot p \rceil}</math>;
# in case that there are less than 32 valid data <math>d_{\rm{Q05}} = \rm{invalid}</math> is set.
Computation is performed for categories '''KC''' and '''KN'''. <math>d_{\rm{Q05}}</math> may be computed for different data set sizes at different locations. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

====Quantile Q95====

Quantile <math>p=0.95</math> is computed for all valid differences <math>d_i</math>. In other words: we obtain a specific value
for <math>d_i</math>, which is deceeded by 95 % of all events but exceeded by 5 % of all events:
# if <math>n \cdot p</math> integer: <math>d_{\rm{Q95}} = 0.5\left( s_{n \cdot p}+s_{n \cdot p+1}\right)</math>;
# if <math>n \cdot p</math> real: <math>d_{\rm{Q95}} = s_{\lceil n \cdot p \rceil}</math>;
# in case that there are less than 32 valid data <math>d_{\rm{Q95}} = \rm{invalid}</math> is set.
Computation is performed for categories '''KC''' and '''KN'''. <math>d_{\rm{Q95}}</math> may be computed for different data set sizes at different locations. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

====Quantile Q99====

Quantile <math>p=0.99</math> is computed for all valid differences <math>d_i</math>. In other words: we obtain a specific value
for <math>d_i</math>, which is deceeded by 99 % of all events but exceeded by just 1 % of all events:
# if <math>n \cdot p</math> integer: <math>d_{\rm{Q99}} = 0.5\left( s_{n \cdot p}+s_{n \cdot p+1}\right)</math>;
# if <math>n \cdot p</math> real: <math>d_{\rm{Q99}} = s_{\lceil n \cdot p \rceil}</math>;
# in case that there are less than 32 valid data <math>d_{\rm{Q99}} = \rm{invalid}</math> is set.
Computation is performed for categories '''KC''' and '''KN'''. <math>d_{\rm{Q99}}</math> may be computed for different data set sizes at different locations. During visualization [[NCPLOT]] enables filtering using ancillary variable ''Number of valid differences''.

===Other skill definitions===

====Murphy Skill 4====

Literature:
# Murphy, Allan H. (1988) "Skill Scores Based on the Mean Square Error and Their Relationship to the Correlation Coefficient". Monthly Weather Review, Dec. 1988, pages 2417 - 2424.

Computation is performed for categories '''KC''' and '''KN'''. Skill S4 according to Murphy (1988, equation 4) can be characterised as follows:
* 1.0 indicates perfect fit;
* 0.0 indicates that the mean value of the reference data models the (reference) data as good as the variant data, because both "models" show the same mean square error (MSE);
* negative skill indicates, that the mean value of the reference data models the (reference) data better than the variant data do;
* ''bias'' is taken into account.

====Willmott Skill (1981)====

Literature:
# Willmott, Cort J. (1981) "On the validation of models". Physical Geography, pages 184–194.

Computation is performed for categories '''KC''' and '''KN'''. Skill according to Willmott (1981, equation 9) can be characterised as follows:
* 1.0 indicates perfect fit;
* 0.0 is the lower limit, for example resulting from phase shift by 180°;
* ''bias'' is taken into account.

----
back to [[Pre- and Postprocessing]]
----
[[Overview]]

NCDELTA.DAT

2021-09-06T08:00:41Z

Rosenhagen:

{{FileDescription
|name_de=NCDELTA.DAT
|filetype=ncdelta.dat
|version=August 2021
|version_descr=August 2021
|significance=contains general steering data for the program [[NCDELTA]].
|filecontents=
'''steering data'''

* Block '''Program_Parameters''': general steering data
*# '''Max_Read_Storage''': maximum number of data items read from input file during each analysis cycle. This value, multiplied with either 24 (REAL Single) or 48 (REAL Double), gives in good approximation [[NCDELTA|NCDELTA's]] memory requirements (in Byte).
*# '''Max_Distance''': maximum distance in [m] up to which points with different location are compared.
*# (optional) '''Print_Modus_Classify''': parameter to steer informative printout messages in program section ''classification'':
*## '''0''' = no extensive messages printed;
*## '''1''' = more extensive messages about classified variables will be printed.
*# (optional) '''Print_Modus_Pairs''': parameter to steer informative printout messages in program section ''primary partner variables'':
*## '''0''' = no extensive messages printed;
*## '''1''' = more extensive messages about detected variable pairs will be printed;
*## '''2''' = also informations about rejected variable pairs will be printed.
*# (optional) '''Print_Modus_Define''': parameter to steer informative printout messages in program section ''definition of computational results'':
*##'''0''' = no extensive messages printed;
*## '''1''' = more extensive informations with respect to metadata of computational results will be printed.
*# (optional) '''Print_Modus_Compute''': parameter to steer informative printout messages in program section ''computation of results'':
*##'''0''' = no extensive messages printed;
*## '''1''' = more extensive messages about computational result variables will be printed.
*# (optional) '''NC_Deflate_Level''': deflate level (0,1,2,...,9) for online compression of data during write. 0 corresponds to no compression and 9 to maximum compression
*# (optional) '''NC_Cmode_Id''': ''creation mode flag'' used for [[NetCDF|CF NetCDF]] method [https://www.unidata.ucar.edu/software/netcdf/netcdf-4/newdocs/netcdf-f90/NF90_005fCREATE.html NF90_CREATE]. Typical values used in this application are:
*#* 4 = '''NF90_64BIT_OFFSET''' (NetCDF 64Bit file, no HDF - no online compression);
*#* 5 = '''NF90_CLASSIC_MODEL''' (HDF file without use of extended HDF features - with online compression).
* Block '''Data_Files''': data files
*# '''Reference_File''': file (NETCDF) with '''reference''' data (filetype [[CF-NETCDF.NC|cf-netcdf.nc]]);
*# '''Variant_File''': file (NETCDF) with '''variant''' data (filetype [[CF-NETCDF.NC|cf-netcdf.nc]]);
*# '''Result_File''': file (NETCDF) for '''result''' data (filetype [[CF-NETCDF.NC|cf-netcdf.nc]]);
*# (optional) '''Reference_Period''': start and end time of period of comparison for reference data (for synoptic data only);
*# (optional) '''Variant_Period''': start and end time of period of comparison for variant data (for synoptic data only).
* (optional) Block '''Data_Operations''': data operations
*# (optional) '''With_Ordinary_Differences''': compute (.true.) common differences (if possible) or do not compute (.false.);
*# (optional) '''With_Taylor_Diagram_Data''': compute (.true.) data for Taylor diagram (if possible) or do not compute (.false.); including skill scores according to Taylor (2001) equations 4 and 5;
*# (optional) '''With_Median''': compute (.true.) median (>= 32 events) or do not compute (.false.);
*# (optional) '''With_Percentiles_05_95''': compute (.true.) percentiles Q05, Q95 (>= 32 events) or do not compute (.false.);
*# (optional) '''With_Percentiles_01_99''': compute (.true.) percentiles Q01, Q99 (>= 51 events) or do not compute (.false.);
*# (optional) '''With_Murphy_Skill_1988''': compute (.true.) skill according to Murphy (1988) equation 4 or do not compute (.false.);
*# (optional) '''With_Skill_Willmott_1981''': compute (.true.) skill according to Willmott (1981) Index of agreement (d);
*# (optional) '''With_Original_Data''': copy (.true.) or not (.false.) input data from which ordinary differences are computed.
* (optional) Block '''Exclude_Variable_From_NCDELTA''': list of variables to be excluded from computation.
*# '''Variant_Variable''': variable from ''Variant-File''. Key can be prescribed several times. With exception of the variables listed here results will be computed for all primary variable pairs.
* (optional) Block '''Include_Only_Variable_for_NCDELTA''': list of variables for which results ar exclusively computed.
*# '''Variant_Variable''': variable from ''Variant-File''. Key can be prescribed several times. Results will be solely computed for the variables listed here.
* (optional) Block '''No_Normal_Vector_Variable''': The list of variables which are normal vectors is (mostly) automatically determined with [[NCDELTA]]. So far, unfortunately, not all of them can be safely identified based on CF metadata. With this list variables can be explicitely '''''excluded''''' from the class of normal vector variables.
*# '''Variable''': variable from ''Variant-File'' or ''Reference_File''. Key can be prescribed several times.
* (optional) Block '''Normal_Vector_Variable''': The list of variables which are normal vectors is (mostly) automatically determined with [[NCDELTA]]. So far, unfortunately, not all of them can be safely identified based on CF metadata. With this list variables can be explicitely '''''included''''' in the class of normal vector variables.
*# '''Variable''': variable from ''Variant-File'' or ''Reference_File''. Key can be prescribed several times.
* (optional) Block '''Label_Relation''': For intercomparison of ''fractioned'' variables, for which the relationship between individual ''fractions'' (sediment, tidal constituent, etc.) cannot be automatically determined. In such situations the corresponding names must be prescribed by the user. Block can be specified several times.
*# '''Variant_Label_KV''': name of a valid label coordinate variable from ''Variant_File'' or ''NONE''.
*# '''Reference_Label_KV''': name of a valid label coordinate variable from ''Reference_File'' or ''NONE''.
*# '''Relation''': ''name of a ''fraction'' in Variant_File'' ''':''' ''name of a ''fraction'' in Reference_File''. Key can be specified several times. ''NONE'' must be specified in case there exists no corresponding label coordinate variable. Colon is used as separator.

===General Remarks===
# Dictionary files used from directory '''''$PROGHOME/dic/''''':
#* '''''ncdelta_dico.dat'''''.
# Use Cases:
#* Case 1: '''Reference_File''' = simulation 1, '''Variant_File''' = simulation 2;
#* Case 2: '''Reference_File''' = measurement, '''Variant_File''' = simulation;
#* Case 3: '''Reference_File''' = measurement 1, '''Variant_File''' = measurement 2;
#: Synoptic data ''or'' data analysis results from program [[NCANALYSE]] can be used.
# General requirements:
#* Synoptic data:
#** length of time period must be identical (periods may differ), and
#** time step must be identical.
#* Analysis results ([[NCANALYSE]]):
#** length of period of data analysis must be comparable, and
#** number of events, e.g. tidal high water, must also be identical.
#* Z coordinate variables:
#: z coordinate variables, if present, must be consistent with respect to
#:* CF attribute ''positive'' all with value "down", or
#:* CF attribute ''positive'' all with value "up"
#: (see [http://cfconventions.org/ CF metadata conventions]).
# Special requirements:
#* ref. case 1:
#** same vertical structure (number of layers), and
#** identical (tracer-) fractions (if present).
#* ref. case 2:
#** identical (tracer-) fractions (if present), and
#** measured data may contain for the same (geographical) location (x,y) several measured data in different depths (z). Furthermore
#** for comparisons with measured current velocity attribute "name_id" of variable "current_velocity" must be set to 836. This requirement should be checked before execution (e.g. using ncdump). In case this condition is not fulfilled NCO tool NCATTED '''ncatted -a name_id,current_velocity,m,i,836 -O ''reference_file''.nc''' can be used to redefine "name_id" accordingly.
#* ref. case 3:
#** each (geographical) location is allowed to be present only once; locations with several stacked data positions are not allowed currently with this use case.

|nutzerprogramme=[[NCDELTA]]
|language=Fortran90
|fileform=FORMATTED
|fileaccess=SEQUENTIAL
|fileextension=.dat
|writemodules= ---
|readmodules=$PROGHOME/fortran/prg/ncdelta/*/mod_m_ncdelta_steer.f90
|contact_original=[mailto:guenther.lang@baw.de G. Lang]
|contact_maintenance=[mailto:guenther.lang@baw.de G. Lang], [mailto:susanne.spohr@baw.de S. Spohr]
|examplefile=$PROGHOME/examples/ncdelta/ncdelta.dat
}}

NCDELTA

2021-09-06T07:58:09Z

Rosenhagen:

{{ProgramDescription
|name_de=NCDELTA
|name=NCDELTA
|version=August 2021
|version_descr=August 2021
|catchwords=
postprocessor 
differences of synoptic data (with optional restriction of time period) 
differences of characteristic numbers 
differences for extensive quantities 
input data for Taylor diagram 
median, percentiles (Q01, Q05, Q95, Q99) 
skill score according to Murphy (1988) equation 4 
skill score according to Taylor (2001) equations 4 and 5 
skill score according to Willmott (1981) Index of agreement (d) 
parallelization using [http://openmp.org/wp/ OpenMP] 
[[NetCDF#Quality assurance using NetCDF attributes|automatic quality assurance (value range)]] 
automatic adjustment of number of READ data to chunk size of data 
automatic setting of WRITE chunk sizes for written variables 
Storage of the content of the ASCII input control files in [[CF-NETCDF.NC|netcdf.nc]] (as a variable) 
Storage of [https://en.wikipedia.org/wiki/MD5 MD5 hash values] of input files in [[CF-NETCDF.NC|netcdf.nc]] (as a variable) 
optional use of ''Message Passing Interface'' (MPI, [https://www.mpi-forum.org/ MPI Forum])

Acknowledgment: ''This project took advantage of netCDF software developed by UCAR/Unidata ([http://www.unidata.ucar.edu/software/netcdf/ www.unidata.ucar.edu/software/netcdf/]).''

|shortdescription=
Program computes differences for comparable variables (primary variable pairs) as well as additional statistical data derived from primary differences, but also data for Taylor diagrams (details see [[Differences of Calculated Results]]) can be computed for some types of data. Finding the primary variable pairs is done in an essentially automatical manner but can be altered to some extent by the user (see [[NCDELTA.DAT|ncdelta.dat]]). Primary differences are computed according to ''variant data'' '''minus''' ''reference data''.

Requirements for time-dependent primary variable pairs with individual constant time step:
# both data sets may contain a ''different'' number of data items (time steps), but the length of the time period to be compared must be identical, whereas the periods to be compared may differ ;
# constant time step for each data set may differ from each other, but the larger time step must an integer multiple of the smaller one.

Requirements for time-dependent primary variable pairs with variable time step:
# both data sets must contain an ''identical'' number of data items (time steps), with periods allowed to be different.

Remarks concerning the spatial location of data:
# data sets may differ with respect to their geographical location;
# areas for which data are defined must overlap to some degree;
# coordinates may be given in different coordinate systems, e. g. Gauß-Krüger and UTM;
# data for a location are compared with data for the nearest lying location, as far as the distance between the different locations does not exceed a prescribed maximum (see [[NCDELTA.DAT|ncdelta.dat]]).

Remarks concerning the comparison of extensive variables:
# for extensive variables the respective sizes of the computational cells (area, length) are taken into account.

|inputfiles=
# '''general input data''' (file type [[NCDELTA.DAT|ncdelta.dat]]);
# '''reference data''' (file type [[CF-NETCDF.NC]]);
# '''variant data''' (file type [[CF-NETCDF.NC]]);
# for [[NetCDF#Automatic quality assurance using NetCDF attributes|automatic quality assurance]] (file type [[BOUNDS.CFG.DAT|bounds_verify.dat]]).

|outputfiles=
# '''results''' (file type [[CF-NETCDF.NC]])
# (optional) informative '''printer file''' of program execution (file type ncdelta.sdr)
# (optional) '''trace''' of program execution (file type ncdelta.trc)

|methodology=
The program is subdivided into the following sections:
# read, check and print steering data prescribed by the user;
# read metadata for ''reference data'';
# read metadata for ''variant data'';
# copy metadata for reference as well as variant data to program specific data structures;
# compare metadata for substantial discrepancies (e. g. reference locations) between data sets;
# classify all reference as well as variant data;
# find primary variable pairs: each variant variable has one partner reference variable; primary computational results will be later derived from primary variable pairs;
# derive variables to be copied from reference as well as variant data file into the result file;
# compute interpolation matrices required for interpolation between reference data locations and variant data locations;
# create all metadata for the result file; essentially they stem from variables to be copied, from primary result variables, newly generated coordinate variables (time, vertical), as well as newly derived or to be copied measure and auxiliary variables;
# copy selceted variables from reference as well as variant file to the result file;
# compute all primary variables, (new) time and vertical coordinate variables, as well as weights and auxiliary variables. For primary variables optionally available axiliary variables with ''standard_name'' modifier ''status_flag'' are taken into account in case meaning ''good'' is represented by an appropriate flag (bit).
# with respect of the different skill scores used and computed by the program - references:
#* Murphy, Allan H. (1988) "Skill Scores Based on the Mean Square Error and Their Relationship to the Correlation Coefficient". Monthly Weather Review, Dec. 1988, Pages 2417 - 2424.
#* Taylor, Karl E. (2001) "Summarizing multiple aspects of model performance in a single diagram". Journal of Geophysical Research, Vol 106, No. D7, April 16, 2001, Pages 7183 - 7192.
#* Willmott, Cort J. (1981) "On the validation of models". Physical Geography, Pages 184–194.
Just in case '''HDF error''' is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only.

|preprocessor=[[BOE2NC]], [[DATACONVERT]], [[NCAGGREGATE]], [[NCANALYSE]], [[NCCHUNKIE]], [[NCCUTOUT]], [[NCRCATMAT]], [[NetCDF Operators]], [[UNK]], [[UNTRIM2007]], [[UNTRIM2]]
|postprocessor=[[DAVIT]], [[NCAUTO]], [[NCCHUNKIE]], [[NCPOLO]], [[NCPLOT]], [[NCCUTOUT]], [[NC2TABLE]], [[NCVIEW2D]], [[PLOTTS]], [[TAYLORDIAGRAM]]
|language=Fortran95
|add_software=none
|contact_original=[mailto:guenther.lang@baw.de G. Lang], [mailto:susanne.spohr@baw.de S. Spohr]
|contact_maintenance=[mailto:guenther.lang@baw.de G. Lang], [mailto:susanne.spohr@baw.de S. Spohr]
|documentation=
* Presentations (available in German only):
** 2015-07-15: [http://ewisa.baw.de/files/12835_tv12_2015_07_15_ncdelta_g_lang.pdf ''NCDELTA - Differenzen neu berechnet''].
* Template Files:
** template files available in '''$PROGHOME/examples/ncdelta'''
}}

NCDELTA

2021-09-06T07:52:57Z

Rosenhagen:

{{ProgramDescription
|name_de=NCDELTA
|name=NCDELTA
|version=July 2020
|version_descr=July 2020
|catchwords=
postprocessor 
differences of synoptic data (with optional restriction of time period) 
differences of characteristic numbers 
differences for extensive quantities 
input data for Taylor diagram 
median, percentiles (Q01, Q05, Q95, Q99) 
skill score according to Murphy (1988) equation 4 
skill score according to Taylor (2001) equations 4 and 5 
skill score according to Willmott (1981) Index of agreement (d) 
parallelization using [http://openmp.org/wp/ OpenMP] 
[[NetCDF#Quality assurance using NetCDF attributes|automatic quality assurance (value range)]] 
automatic adjustment of number of READ data to chunk size of data 
automatic setting of WRITE chunk sizes for written variables 
Storage of the content of the ASCII input control files in [[CF-NETCDF.NC|netcdf.nc]] (as a variable) 
Storage of [https://en.wikipedia.org/wiki/MD5 MD5 hash values] of input files in [[CF-NETCDF.NC|netcdf.nc]] (as a variable) 
optional use of ''Message Passing Interface'' (MPI, [https://www.mpi-forum.org/ MPI Forum])

Acknowledgment: ''This project took advantage of netCDF software developed by UCAR/Unidata ([http://www.unidata.ucar.edu/software/netcdf/ www.unidata.ucar.edu/software/netcdf/]).''

|shortdescription=
Program computes differences for comparable variables (primary variable pairs) as well as additional statistical data derived from primary differences, but also data for Taylor diagrams (details see [[Differences of Calculated Results]]) can be computed for some types of data. Finding the primary variable pairs is done in an essentially automatical manner but can be altered to some extent by the user (see [[NCDELTA.DAT|ncdelta.dat]]). Primary differences are computed according to ''variant data'' '''minus''' ''reference data''.

Requirements for time-dependent primary variable pairs with individual constant time step:
# both data sets may contain a ''different'' number of data items (time steps), but the length of the time period to be compared must be identical, whereas the periods to be compared may differ ;
# constant time step for each data set may differ from each other, but the larger time step must an integer multiple of the smaller one.

Requirements for time-dependent primary variable pairs with variable time step:
# both data sets must contain an ''identical'' number of data items (time steps), with periods allowed to be different.

Remarks concerning the spatial location of data:
# data sets may differ with respect to their geographical location;
# areas for which data are defined must overlap to some degree;
# coordinates may be given in different coordinate systems, e. g. Gauß-Krüger and UTM;
# data for a location are compared with data for the nearest lying location, as far as the distance between the different locations does not exceed a prescribed maximum (see [[NCDELTA.DAT|ncdelta.dat]]).

Remarks concerning the comparison of extensive variables:
# for extensive variables the respective sizes of the computational cells (area, length) are taken into account.

|inputfiles=
# '''general input data''' (file type [[NCDELTA.DAT|ncdelta.dat]]);
# '''reference data''' (file type [[CF-NETCDF.NC]]);
# '''variant data''' (file type [[CF-NETCDF.NC]]);
# for [[NetCDF#Automatic quality assurance using NetCDF attributes|automatic quality assurance]] (file type [[BOUNDS.CFG.DAT|bounds_verify.dat]]).

|outputfiles=
# '''results''' (file type [[CF-NETCDF.NC]])
# (optional) informative '''printer file''' of program execution (file type ncdelta.sdr)
# (optional) '''trace''' of program execution (file type ncdelta.trc)

|methodology=
The program is subdivided into the following sections:
# read, check and print steering data prescribed by the user;
# read metadata for ''reference data'';
# read metadata for ''variant data'';
# copy metadata for reference as well as variant data to program specific data structures;
# compare metadata for substantial discrepancies (e. g. reference locations) between data sets;
# classify all reference as well as variant data;
# find primary variable pairs: each variant variable has one partner reference variable; primary computational results will be later derived from primary variable pairs;
# derive variables to be copied from reference as well as variant data file into the result file;
# compute interpolation matrices required for interpolation between reference data locations and variant data locations;
# create all metadata for the result file; essentially they stem from variables to be copied, from primary result variables, newly generated coordinate variables (time, vertical), as well as newly derived or to be copied measure and auxiliary variables;
# copy selceted variables from reference as well as variant file to the result file;
# compute all primary variables, (new) time and vertical coordinate variables, as well as weights and auxiliary variables. For primary variables optionally available axiliary variables with ''standard_name'' modifier ''status_flag'' are taken into account in case meaning ''good'' is represented by an appropriate flag (bit).
# with respect of the different skill scores used and computed by the program - references:
#* Murphy, Allan H. (1988) "Skill Scores Based on the Mean Square Error and Their Relationship to the Correlation Coefficient". Monthly Weather Review, Dec. 1988, Pages 2417 - 2424.
#* Taylor, Karl E. (2001) "Summarizing multiple aspects of model performance in a single diagram". Journal of Geophysical Research, Vol 106, No. D7, April 16, 2001, Pages 7183 - 7192.
#* Willmott, Cort J. (1981) "On the validation of models". Physical Geography, Pages 184–194.
Just in case '''HDF error''' is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only.

|preprocessor=[[BOE2NC]], [[DATACONVERT]], [[NCAGGREGATE]], [[NCANALYSE]], [[NCCHUNKIE]], [[NCCUTOUT]], [[NCRCATMAT]], [[NetCDF Operators]], [[UNK]], [[UNTRIM2007]], [[UNTRIM2]]
|postprocessor=[[DAVIT]], [[NCAUTO]], [[NCCHUNKIE]], [[NCPOLO]], [[NCPLOT]], [[NCCUTOUT]], [[NC2TABLE]], [[NCVIEW2D]], [[PLOTTS]], [[TAYLORDIAGRAM]]
|language=Fortran95
|add_software=none
|contact_original=[mailto:guenther.lang@baw.de G. Lang], [mailto:susanne.spohr@baw.de S. Spohr]
|contact_maintenance=[mailto:guenther.lang@baw.de G. Lang], [mailto:susanne.spohr@baw.de S. Spohr]
|documentation=
* Presentations (available in German only):
** 2015-07-15: [http://ewisa.baw.de/files/12835_tv12_2015_07_15_ncdelta_g_lang.pdf ''NCDELTA - Differenzen neu berechnet''].
* Template Files:
** template files available in '''$PROGHOME/examples/ncdelta'''
}}