View source for Metadata of Model results in Coastal Engineering

[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]

The metadata set is a thoroughly documented package that describes the simulations and hydrodynamic model results in coastal engineering. The details included in metadata utterly summarizes different aspects of the data in information packages as identification, contact, quality, lineage, hierarchy, distribution, constraint, extent and reference. Broad recording and provision of mentioned information packages of a metadata set is a fundamental measure of quality and completeness for a metadata set. Completeness of metadata ensures proper utilization of data or their reproducibility. Furthermore, the metadata facilitate administration and search as well as segmentation and classification of the data. This page explains how to collect and provide institutional ([https://www.baw.de/EN/wasserbau/wasserbau.html BAW] internal) metadata for coastal simulation data considering the existing [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf BAW metadata profile].  

'''[1] Collecting Metadata'''

Meta-information of a model is collected in the processing chain for a simulation data. The [https://wiki.baw.de/en/index.php/UNTRIM UnTRIM] and [https://wiki.baw.de/en/index.php/UNTRIM2 UnTRIM2] modeling approach through the use of the NetCDF data format processes the metadata in either of the subsequent sub-models: 

* Hydrodynamics in utromp2007 and  SubGrid in utromp2009
* Morphodynamics in SediMorph
* Waves in the k-Model

Setting environmental parameters is a requirement and an initial step to conduct a simulation. These environmental variables are set through run and project scripts (in German: Auftragsskripte), which also comprise metadata of the respective model. The user adequately navigates these variables on the script and provides the required attributes as metadata (figure 01). Recording meta-information is a single time customization task for a project with multiple simulation conditions. Each of the subsequent methods e.g. hydrodynamics, morphodynamics and sea motion is providing and securing their relevant metadata in the processing chain from their run and project scripts. 

[[File:Run and project script.png|Run and project script ]]
Figure 01. Run and project script (in German: Auftragsskript) to insert the meta-information for a simulation.

Most of the metadata elements are acquired from the steering data of the processing environment automatically. However, there are approximately 15 customized elements relevant to the project management and contact methods the user is providing. A quality assurance aspect in this process is the validity check through comparisons of the inserted values with the existing code list as configuration data. The user provides the following elements in the project script: 

*	DMQS_PSP: Project PSP number from EWisA
*	DMQS_TITLE: Project title from EWisA
*	DMQS_SUMMARY: summary of the project in a max of 240 characters
*	DMQS_CREATOR_NAME: "<surname>, <name>, <optional_title>" of project manager
*	DMQS_CREATOR_EMAIL: email address of project manager
*	DMQS_CREATOR_ROLE:  role of project manager
*	DMQS_GEO_ID: a list for valid geographical identification located in %PROGHOME%\cfg\dmqs\geo\valid_geo_ids.dat.
*	Further elements, in case the project is funded through mFUND:
**	MFUND_PROJECT: name of the funded project
**	MFUND_ID: project funding number
For a more accurate description of a simulation file, its lineage and hierarchal information is also required to be stored. The elements relevant to the model description and hierarchy to set are:
*	DMQS_VARIANT: describes the geometrical property of a model
*	DMQS_SCENARIO: describes the time or boundary conditions and other features of a simulation e.g. wind or storm event
*	RUN: further distinguishes a simulation through its specific feature/characteristic  
*	DMQS_RUN_COMMENT: free text for an exact description of each individual model
Note: for ease and access the template files are stored as:
*	The template file for the project script is located under $PROGHOME/bin/dmqs/set_dmqs_template.sh
*	Template files for the run script are located under $PROGHOME/examples/slurm
*	List of the valid names are located under:
**	$PROGHOME/cfg/dmqs/variant/01_template.dat
**	$PROGHOME/cfg/dmqs/scenario/01_template.dat
**	$PROGHOME/cfg/dmqs/geo/valid_geo_ids.dat 
Post-processing and analyzing algorithms append their process relevant meta-information to their output NetCDF files. Furthermore, the lineage information and steering information is complemented in this process as well. Figure 02 further illustrates the post-processing chain, which automatically writes its meta-information in the NetCDF files.

[[File:Post-processing chain.png| Figure 02: post-processing chain.]]
Figure 02: post-processing chain.

'''[2] Extracting Metadata '''

An extra metadata generation tool is integrated into the visualization software [https://wiki.baw.de/en/index.php/DAVIT Davit]. This tool is developed and customized based on the BAW requirements, to convert the NetCDF CF metadata elements to ISO compliant XML metadata. Extensible Markup Language (XML) is both human and machine-readable and the standard data format for storing and exchanging of geographical metadata and attributes via the web. The exported XML metadata set is further mapped based on ISO schemas in adherence to the relevant standards listed in BAW metadata profile [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf (GDI-BAW V1.3)], which makes it compatible with a wide range of information systems. Furthermore, to ensure the validity and completeness of the exported metadata *.XML file, a new popup window appears if any of the mandatory elements are missing in the NetCDF file.

'''[3] Editing Metadata'''

In this process, it is considered that the metadata is complete and valid. However, if this is not the case, supplementary manual editing possibilities are:
# NetCDF attribute via NCO-Tool ‘ncatted’ or MATLAB ‘ncwriteatt’ command,
# XML file in a notepad e.g. Notepad++,
# InGrid editor (IGE) interface of MIS-BAW.


'''[4]	Administrating Metadata'''

The produced XML dataset is then imported in the metadata information system MIS-BAW. The user can import a dataset via either (a) InGrid editor (IGE) or (b) CSW-T interface via a curl command. The imported elements could be also further edited in IGE. In addition, IGE provides a further possibility of manual entry of a new metadata set. 


'''Acknowledgment'''

This project took advantage of NetCDF software developed by [http://www.unidata.ucar.edu/software/netcdf/ UCAR/Unidata].

===Documentation/Literature ===

Metadata of model results is a part of the project data management and quality assurance (DMQS) in waterways engineering. Further information is available on the [http://intranet.baw.de/cocoon/portal/res?t=1&ci=1&si=0&i=344&d=Aufgabe&v=DETAIL&x=DETAIL&hra=12&hi=344&hd=Aufgabe&hc=8766&sichtc=cms&r=&cid=8766&rt=&nd=Aufgabe&ni=344&uki=344&ukd=Aufgabe&uksi=&ukg=1&ucid=8766&bereich=&ber=&lim=&ord=&os=&asc=&pm=&arch=false&l=DE&s=de&su=&p=&bn=&bc=&refModule=Hauptnavigation intranet pages of the BAW]. <br> 
----
For detailed information have a look at the [https://wiki.baw.de/de/index.php/Erfassung_der_Metadaten_einer_Simulation guidelines in German].
----
back to [[Data Management and Information Systems]]
----
[[Overview]]
<br> <big>'''Under Construction'''</big> <br>