Actions

BOEWRT.DAT: Difference between revisions

From BAWiki

imported>BAWiki 2
No edit summary
imported>Schade Peter
(same icode and classes)
 
(7 intermediate revisions by 4 users not shown)
Line 2: Line 2:
|name_de=BOEWRT.DAT
|name_de=BOEWRT.DAT
|filetype=boewrt.dat  
|filetype=boewrt.dat  
|version=September 2001
|version=February 2018
|version_descr=May 2009
|version_descr=February 2018
|significance=contains measured or calculated time-series data for the usage in different programs.
|significance=contains measured or calculated time-series data in ASCII for the usage in various programs.
===Remarks about the new version===
===General remarks===
The file format is rather flexible:<br />inside the lines of data each of the time formates used by BAW is allowed.<br />Use ";" to separate datetime and physical values!<br />The date time format with two digit year should not be used further.<br />When it is used, the century after $BAWCENTURY (inclusive) is assumed. You should specify $BAWCENTURY inside your environment (DEFAULT: 1900).<br />The time zone is used only, when the used begin or end date have a defined time zone.<br />For this case MEZ is assumed for the file, when the file contains no time zone!
 
The file has a header where meta data is identified by its position in the file, the first data line e.g. contains the node number. This is the classic boewrt.dat that can be processed by classical methods of BAW's io library. Since May 2014 the new boewrt.dat header can contain key-value pairs with additional meta data, the key Fill_Value is for example followed by a value in real format. The additional meta information is only evaluated by methods of BAW's io_dataset software package.
 
Programs using the io and the io_dataset library can read the  classical as well as the new version of boewrt.dat. The data part is for both the same and rather flexible; each of the ASCII time formats used by the BAW is allowed. But the date and time format with two digit year should not be used any longer. A ";" should be used to separate date and time from the values of the quantities. <br/>
 
But the libraries interpret the meta information in a different way. <br/>  
 
===Io library preferably for classic boewrt.dat with support of environment variables===
This library can interpret environment variables. If the environment variable $BAWCRS is set to a valid value (EPSG code), the station coordinate will be transformed to that system during the reading process on the fly. If the file CRS is not valid, the coordinate is used "as is". The user should set $BAWCRS when using the software listed below as classic programs.<br />
 
If the two digit year is used, the century after $BAWCENTURY (inclusive) will be assumed. You should specify $BAWCENTURY inside your environment (DEFAULT: 1900). <br />
The time zone is used only, when the used begin or end date have a defined time zone.<br />For this case MEZ is assumed for the file, when the file contains no time zone!<br />
 
===Io_dataset library preferably for new boewrt.dat===
If the new boewrt.dat gets accessed by the io_dataset software the header has to contain more mandatory meta information, i.e. time zone, CRS and fill value, s. file-contents. The environment variables are not interpreted by io_dataset . The metadata is packed into objects of type dimension (dim), variable (var) and attribute (att) as common for netcdf files. From outside the package the data is accessed in a generic way, meaning the same interfaces are used for different file formats.
|filecontents=
|filecontents=
# nodenumber and (optional) timezone and coordinate system
===Data for classic as well as new file format===
# name of the location
# node number, time zone and coordinate reference System (CRS). This record has to be written FORMATTED with FORTRAN-Format "(I10,1X,A4,1X,A5)". Preferably you should use known abbreviations for the time zones ("CET", "CEST", "UTC"). For the coordinate reference system of the station coordinate the EPSG codes used by BAW (see [[GEOTRANSFORMER]] for example) should be used. Certain abbrevations as "GK3B" or "SPHW" are valid as well. Time zone and CRS are mandatory for the new boewrt.dat version.
# coordinates (x,y) of the location
# name of the location (written in FORTRAN format "(A)").
# number and type-codes for the physical quantities stored in the file
# coordinates (x,y and optionally z) of the station position. Z values are as default depths with positiv values in the downward direction.(unformatted with separating blanks!)
# date, time and data for each time-step stored
# number and type-codes for the physical quantities stored in the file (FORTRAN format "(I10,nI8)". Since February 2018 the same  icode can appear more than once, e.g. for several fractions of suspended load.
# date, time and data for each time-step stored. Preferably the date and time should be separated from the physical values by a ";"-character.<br/>
 
===Data for classic file format only===
# minimum and maximum time step: on failure of reading a file, these comments should be erased. Files manipulated by an editor should never contain these comments!
# FORTRAN format specifications of the lines in the data part. The FORTRAN format is used to accelerate the reading procedure of uniformly formatted records. On failure of reading a file it should be erased. Files manipulated by an editor should never contain the format.<br/>
 
===Data for new file format only===
Content of the new boewrt.dat format, the keywords refer to NODC's standard attributes and guidance tables ( http://www.nodc.noaa.gov/data/formats/netcdf/#guidancetable )
# fill value: a real number defining invalid values of quantities (mandatory)
# instrument name (optional)
# instrument comment: commenting on the instrument given by instrument name (optional)
# platform name: name of the geophysical platform hosting the instrument (optional)
# platform comment: commenting on the platform given by platform name (optional)
# valid range: defining minimal and maximal limits for values of quantities identified by BAW' icode number, e.g. 3 for waterlevel (optional)
# class: name of a sub class of a quantity, e.g. 'sum of all fractions' identified by BAW's icode number, e.g. 7 for suspended load (optional)
# measure comment: individual comment on a quantity (optional)
# line comment: optionally commenting on the values of one data line as displayed in the subsequent example, where "!" separates the quantities from the line comment "2_QMag 2_QDir":<br/>
          29.02.2004 12:00:31;    0.066  122.010    !2_QMag 2_QDir<br/>
 
===Status flag variable===
This variable is used to characterize the qualtity of individual data values. PHYDEF-code for this type of data is 3180.
During conversion using [[DATACONVERT]] a
[http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#flags flag] variable consistent with CF metadata standard is created. [http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#standard-name-modifiers Modificator] ''status_flag'' is part of the standard name of this variable. Attributes flag_masks and flag_meanings are used to describe the quality of individual data items:
* flag_mask='''1b''', flag_meaning='''''good''''': valid data, will be used by [[NCANALYSE]] and [[NCDELTA]];
* flag_mask='''2b''', flag_meaning='''''unchecked''''': quality not checked;
* flag_mask='''4b''', flag_meaning='''''estimated''''': derived from neighbouring data;
* flag_mask='''8b''', flag_meaning='''''suspect''''': doubtful or obviously wrong data.
 
|nutzerprogramme=
|nutzerprogramme=
[[EVENTFILTER]], [[EXCELENZ]], [[EXKNO]], [[FD2MET]], [[FRQWF]], [[GEOTRANSFORMER]], [[GVIEW2D]], [[HVIEW2D]], [[MESKOR]], [[ROSE]], [[TIDKEN]], [[TSCALC]], [[UTRRND]], [[VVIEW2D]], [[XTRDATA]], [[ZEITRIO]]
classic: [[EVENTFILTER]], [[EXCELENZ]], [[EXKNO]], [[FD2MET]], [[FRQWF]], [[GEOTRANSFORMER]], [[GVIEW2D]], [[HVIEW2D]], [[MESKOR]], [[ROSE]], [[TIDKEN]], [[TSCALC]], [[UNTRIM2007MONITOR]], [[UTRRND]], [[VVIEW2D]], [[XTRDATA]], [[ZEITRIO]]<br>
|language=Fortran90
new: [[DATACONVERT]]
|language=Fortran95
|fileform=FORMATTED
|fileform=FORMATTED
|fileaccess=SEQUENTIAL
|fileaccess=SEQUENTIAL
|fileextension=.dat
|fileextension=.dat
|writemodules=$PROGHOME/fortran/lib/io/*/mod_boewrt_io.f90:write_boewrt_info  
|writemodules=classic: $PROGHOME/fortran/lib/io/*/mod_boewrt_io.f90:write_boewrt_info; <br/>
|readmodules=$PROGHOME/fortran/lib/io/*/mod_boewrt_io.f90:read_boewrt_info  
new: -         
|readmodules=classic: $PROGHOME/fortran/lib/io/*/mod_boewrt_io.f90:read_boewrt_info; <br/>
new: $PROGHOME/fortran/lib/io_dataset/*/mod_io_dataset_ui.f90, mod_m_dataset_boewrt_info.f90 and mod_m_dataset_boewrt.f90.
|contact_original=[mailto:ingrid.uliczka@baw.de I. Uliczka]
|contact_original=[mailto:ingrid.uliczka@baw.de I. Uliczka]
|contact_maintenance=[mailto:ingrid.uliczka@baw.de I. Uliczka], [mailto:guntram.seiss@baw.de G. Seiß]
|contact_maintenance=[mailto:ingrid.uliczka@baw.de I. Uliczka], [mailto:guntram.seiss@baw.de G. Seiß]; [mailto:peter.schade@baw.de P. Schade]
|examplefile=$PROGHOME/examples/zeitrio/boewrt.new.dat<br />
|examplefile= classic: $PROGHOME/examples/zeitrio/boewrt.new.dat and $PROGHOME/examples/zeitrio/boewrt.dat<br/>  
$PROGHOME/examples/zeitrio/boewrt.dat  
new: $PROGHOME/examples/dataconvert/boewrt.optionalheader.dat and
$PROGHOME/examples/dataconvert/boewrt.linecomments.dat<br/>
}}
}}

Latest revision as of 09:02, 13 February 2018

Basic Information

File-Type

boewrt.dat

File-Form

FORMATTED

Version

February 2018

Description-Date

February 2018

Significance of the File

contains measured or calculated time-series data in ASCII for the usage in various programs.

General remarks

The file has a header where meta data is identified by its position in the file, the first data line e.g. contains the node number. This is the classic boewrt.dat that can be processed by classical methods of BAW's io library. Since May 2014 the new boewrt.dat header can contain key-value pairs with additional meta data, the key Fill_Value is for example followed by a value in real format. The additional meta information is only evaluated by methods of BAW's io_dataset software package.

Programs using the io and the io_dataset library can read the classical as well as the new version of boewrt.dat. The data part is for both the same and rather flexible; each of the ASCII time formats used by the BAW is allowed. But the date and time format with two digit year should not be used any longer. A ";" should be used to separate date and time from the values of the quantities.

But the libraries interpret the meta information in a different way.

Io library preferably for classic boewrt.dat with support of environment variables

This library can interpret environment variables. If the environment variable $BAWCRS is set to a valid value (EPSG code), the station coordinate will be transformed to that system during the reading process on the fly. If the file CRS is not valid, the coordinate is used "as is". The user should set $BAWCRS when using the software listed below as classic programs.

If the two digit year is used, the century after $BAWCENTURY (inclusive) will be assumed. You should specify $BAWCENTURY inside your environment (DEFAULT: 1900).
The time zone is used only, when the used begin or end date have a defined time zone.
For this case MEZ is assumed for the file, when the file contains no time zone!

Io_dataset library preferably for new boewrt.dat

If the new boewrt.dat gets accessed by the io_dataset software the header has to contain more mandatory meta information, i.e. time zone, CRS and fill value, s. file-contents. The environment variables are not interpreted by io_dataset . The metadata is packed into objects of type dimension (dim), variable (var) and attribute (att) as common for netcdf files. From outside the package the data is accessed in a generic way, meaning the same interfaces are used for different file formats.

File-Contents (in Catchwords)

Data for classic as well as new file format

  1. node number, time zone and coordinate reference System (CRS). This record has to be written FORMATTED with FORTRAN-Format "(I10,1X,A4,1X,A5)". Preferably you should use known abbreviations for the time zones ("CET", "CEST", "UTC"). For the coordinate reference system of the station coordinate the EPSG codes used by BAW (see GEOTRANSFORMER for example) should be used. Certain abbrevations as "GK3B" or "SPHW" are valid as well. Time zone and CRS are mandatory for the new boewrt.dat version.
  2. name of the location (written in FORTRAN format "(A)").
  3. coordinates (x,y and optionally z) of the station position. Z values are as default depths with positiv values in the downward direction.(unformatted with separating blanks!)
  4. number and type-codes for the physical quantities stored in the file (FORTRAN format "(I10,nI8)". Since February 2018 the same icode can appear more than once, e.g. for several fractions of suspended load.
  5. date, time and data for each time-step stored. Preferably the date and time should be separated from the physical values by a ";"-character.

Data for classic file format only

  1. minimum and maximum time step: on failure of reading a file, these comments should be erased. Files manipulated by an editor should never contain these comments!
  2. FORTRAN format specifications of the lines in the data part. The FORTRAN format is used to accelerate the reading procedure of uniformly formatted records. On failure of reading a file it should be erased. Files manipulated by an editor should never contain the format.

Data for new file format only

Content of the new boewrt.dat format, the keywords refer to NODC's standard attributes and guidance tables ( http://www.nodc.noaa.gov/data/formats/netcdf/#guidancetable )

  1. fill value: a real number defining invalid values of quantities (mandatory)
  2. instrument name (optional)
  3. instrument comment: commenting on the instrument given by instrument name (optional)
  4. platform name: name of the geophysical platform hosting the instrument (optional)
  5. platform comment: commenting on the platform given by platform name (optional)
  6. valid range: defining minimal and maximal limits for values of quantities identified by BAW' icode number, e.g. 3 for waterlevel (optional)
  7. class: name of a sub class of a quantity, e.g. 'sum of all fractions' identified by BAW's icode number, e.g. 7 for suspended load (optional)
  8. measure comment: individual comment on a quantity (optional)
  9. line comment: optionally commenting on the values of one data line as displayed in the subsequent example, where "!" separates the quantities from the line comment "2_QMag 2_QDir":
         29.02.2004 12:00:31;    0.066  122.010     !2_QMag 2_QDir

Status flag variable

This variable is used to characterize the qualtity of individual data values. PHYDEF-code for this type of data is 3180. During conversion using DATACONVERT a flag variable consistent with CF metadata standard is created. Modificator status_flag is part of the standard name of this variable. Attributes flag_masks and flag_meanings are used to describe the quality of individual data items:

  • flag_mask=1b, flag_meaning=good: valid data, will be used by NCANALYSE and NCDELTA;
  • flag_mask=2b, flag_meaning=unchecked: quality not checked;
  • flag_mask=4b, flag_meaning=estimated: derived from neighbouring data;
  • flag_mask=8b, flag_meaning=suspect: doubtful or obviously wrong data.

Programs using this Type of File

classic: EVENTFILTER, EXCELENZ, EXKNO, FD2MET, FRQWF, GEOTRANSFORMER, GVIEW2D, HVIEW2D, MESKOR, ROSE, TIDKEN, TSCALC, UNTRIM2007MONITOR, UTRRND, VVIEW2D, XTRDATA, ZEITRIO
new: DATACONVERT

Example-File

classic: $PROGHOME/examples/zeitrio/boewrt.new.dat and $PROGHOME/examples/zeitrio/boewrt.dat
new: $PROGHOME/examples/dataconvert/boewrt.optionalheader.dat and $PROGHOME/examples/dataconvert/boewrt.linecomments.dat


back to: File Descriptions


Overview