Metadata of Model results in Coastal Engineering - Revision history

127.0.0.1: The LinkTitles extension automatically added links to existing pages (https://github.com/bovender/LinkTitles).

2022-10-21T10:08:22Z

The LinkTitles extension automatically added links to existing pages (<a target="_blank" rel="nofollow noreferrer noopener" class="external free" href="https://github.com/bovender/LinkTitles">https://github.com/bovender/LinkTitles</a>).

Ak3pscha at 13:09, 15 May 2020

2020-05-15T13:09:50Z

← Older revision		Revision as of 13:09, 15 May 2020
Line 24:		Line 24:
	* DMQS_CREATOR_EMAIL: email address of project manager		* DMQS_CREATOR_EMAIL: email address of project manager
	* DMQS_CREATOR_ROLE: role of project manager		* DMQS_CREATOR_ROLE: role of project manager
	* DMQS_GEO_ID: a list for valid geographical identification located in ~~\\themis\system\akprog~~\cfg\dmqs\geo\valid_geo_ids.dat ~~($CFGDIR)~~.		* DMQS_GEO_ID: a list for valid geographical identification located in %PROGHOME%\cfg\dmqs\geo\valid_geo_ids.dat.
	* Further elements, in case the project is funded through mFUND:		* Further elements, in case the project is funded through mFUND:
	** MFUND_PROJECT: name of the funded project		** MFUND_PROJECT: name of the funded project

Arif at 19:01, 6 April 2020

2020-04-06T19:01:34Z

Arif: corrections

2020-03-11T09:33:16Z

corrections

← Older revision		Revision as of 09:33, 11 March 2020
Line 1:		Line 1:
	[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]		[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]
	~~'''This page is under construction!'''~~

	The metadata set is a thoroughly documented package that describes the simulations and hydrodynamic model results in coastal engineering. The details included in metadata utterly summaries different aspect of the data in information packages as identification, contact, quality, lineage, hierarchy, distribution, constraint, extent and reference. Completeness of metadata ensures proper utilization of data or their reproducibility. Furthermore, the metadata facilitate administration and search as well as sort and classification of the data. This page explains how to collect and provide institutional (BAW internal) metadata for coastal simulation data considering the existing [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf BAW metadata profile].		The metadata set is a thoroughly documented package that describes the simulations and hydrodynamic model results in coastal engineering. The details included in metadata utterly summaries different aspect of the data in information packages as identification, contact, quality, lineage, hierarchy, distribution, constraint, extent and reference. Completeness of metadata ensures proper utilization of data or their reproducibility. Furthermore, the metadata facilitate administration and search as well as sort and classification of the data. This page explains how to collect and provide institutional ([https://www.baw.de/EN/wasserbau/wasserbau.html BAW] internal) metadata for coastal simulation data considering the existing [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf BAW metadata profile].

	'''[1] Collecting Metadata'''		'''[1] Collecting Metadata'''
Line 48:		Line 47:
	'''[2] Extracting Metadata '''		'''[2] Extracting Metadata '''

	An extra metadata generation tool is integrated into the visualization software [https://wiki.baw.de/en/index.php/DAVIT Davit]. This tool is developed and customized based on the BAW requirements, to convert the NetCDF CF metadata elements to ISO compliant XML metadata. Extensible Markup Language (XML) is both human and machine-readable and the standard data format for storing and exchanging of geographical metadata and attributes via the web. The exported XML metadata set is further mapped based on ISO schema in adherence to the relevant standards listed in BAW metadata profile [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf (GDI-BAW V1.3)], which makes it compatible with a wide range of information systems. Furthermore, to ensure the validity and completeness of the exported metadata *.XML file, a new popup window appears if any of the ~~core elements~~ is missing in the NetCDF file.		An extra metadata generation tool is integrated into the visualization software [https://wiki.baw.de/en/index.php/DAVIT Davit]. This tool is developed and customized based on the BAW requirements, to convert the NetCDF CF metadata elements to ISO compliant XML metadata. Extensible Markup Language (XML) is both human and machine-readable and the standard data format for storing and exchanging of geographical metadata and attributes via the web. The exported XML metadata set is further mapped based on ISO schema in adherence to the relevant standards listed in BAW metadata profile [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf (GDI-BAW V1.3)], which makes it compatible with a wide range of information systems. Furthermore, to ensure the validity and completeness of the exported metadata *.XML file, a new popup window appears if any of the mandatory element is missing in the NetCDF file.

	'''[3] Editing Metadata'''		'''[3] Editing Metadata'''
Line 54:		Line 53:
	In this process, it is considered that the metadata is complete and valid. However, if this is not the case, further manual editing possibilities are:		In this process, it is considered that the metadata is complete and valid. However, if this is not the case, further manual editing possibilities are:
	# NetCDF attribute via NCO-Tool ‘ncatted’ or MATLAB ‘ncwriteatt’ command,		# NetCDF attribute via NCO-Tool ‘ncatted’ or MATLAB ‘ncwriteatt’ command,
	# XML file in a notepad e.g. ~~notepad~~++,		# XML file in a notepad e.g. Notepad++,
	# InGrid editor (IGE) interface of MIS-BAW.		# InGrid editor (IGE) interface of MIS-BAW.

Line 62:		Line 61:
	The produced XML dataset is then imported in the metadata information system MIS-BAW. The user can import a dataset via either (a) InGrid editor (IGE) or (b) CSW-T interface via a curl command. The imported elements could be also further edited in IGE. In addition, IGE provides a further possibility of manual entry of a new metadata set.		The produced XML dataset is then imported in the metadata information system MIS-BAW. The user can import a dataset via either (a) InGrid editor (IGE) or (b) CSW-T interface via a curl command. The imported elements could be also further edited in IGE. In addition, IGE provides a further possibility of manual entry of a new metadata set.

	~~For detailed information have a look at the [https://wiki.baw.de/de/index.php/Erfassung_der_Metadaten_einer_Simulation guidelines in German].~~

	'''Acknowledgment'''		'''Acknowledgment'''
Line 71:		Line 69:

	Metadata of model results is a part of the project data management and quality assurance (DMQS) in waterways engineering. Further information is available on the [http://intranet.baw.de/cocoon/portal/res?t=1&ci=1&si=0&i=344&d=Aufgabe&v=DETAIL&x=DETAIL&hra=12&hi=344&hd=Aufgabe&hc=8766&sichtc=cms&r=&cid=8766&rt=&nd=Aufgabe&ni=344&uki=344&ukd=Aufgabe&uksi=&ukg=1&ucid=8766&bereich=&ber=&lim=&ord=&os=&asc=&pm=&arch=false&l=DE&s=de&su=&p=&bn=&bc=&refModule=Hauptnavigation intranet pages of the BAW]. <br>		Metadata of model results is a part of the project data management and quality assurance (DMQS) in waterways engineering. Further information is available on the [http://intranet.baw.de/cocoon/portal/res?t=1&ci=1&si=0&i=344&d=Aufgabe&v=DETAIL&x=DETAIL&hra=12&hi=344&hd=Aufgabe&hc=8766&sichtc=cms&r=&cid=8766&rt=&nd=Aufgabe&ni=344&uki=344&ukd=Aufgabe&uksi=&ukg=1&ucid=8766&bereich=&ber=&lim=&ord=&os=&asc=&pm=&arch=false&l=DE&s=de&su=&p=&bn=&bc=&refModule=Hauptnavigation intranet pages of the BAW]. <br>
			----
			For detailed information have a look at the [https://wiki.baw.de/de/index.php/Erfassung_der_Metadaten_einer_Simulation guidelines in German].
	----		----
	back to [[Data Management and Information Systems]]		back to [[Data Management and Information Systems]]

Arif: Replaced the text

2020-03-10T16:47:00Z

Replaced the text

Show changes

Arif: first draft

2020-03-10T15:25:57Z

first draft

← Older revision		Revision as of 15:25, 10 March 2020
Line 1:		Line 1:
	[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]		[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]
	~~Model~~ results of ~~coastal waters are systematically described by their meta~~ data. ~~This information will be referenced in an internal use case~~. ~~Starting from a figure in a text document~~ the ~~reader will be directly forwarded to~~ the ~~meta~~ data ~~of the displayed model results~~.~~<br>~~		'''This page is under construction!'''
			The metadata set is a thoroughly documented package that describes the simulations and hydrodynamic model results in coastal engineering. The details included in metadata utterly summaries different aspect of the data in information packages as identification, contact, quality, lineage, hierarchy, distribution, constraint, extent and reference. Completeness of metadata ensures proper utilization of data or their reproducibility. Furthermore, the metadata facilitate administration and search as well as sort and classification of the data. This page explains how to collect and provide institutional (BAW internal) metadata for coastal simulation data considering the existing [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf \|BAW metadata profile].
	~~These guidelines explain~~ how ~~meta~~ data ~~of coastal models can be collected and provided to~~ the BAW ~~staff~~. ~~The described method is connected to the use of NetCDF files~~. ~~It starts with the first step in the processing chain with output of UGRID [[CF-NETCDF~~.NC\|~~CF-NETCDF files~~]~~], the simulation:~~

	'''1. Collecting metadata of a simulation'''		'''1. Collecting metadata of a simulation'''
Line 17:		Line 16:

	The analysis tools at the BAW compute characteristic values out of the model results and write these values again in NetCDF files. They also read the meta data of the modelling results, complement them and write it to its output file. [http://wiki.baw.de/de/index.php/Erfassung_von_Analyse-Metadaten\| Full story.].		The analysis tools at the BAW compute characteristic values out of the model results and write these values again in NetCDF files. They also read the meta data of the modelling results, complement them and write it to its output file. [http://wiki.baw.de/de/index.php/Erfassung_von_Analyse-Metadaten\| Full story.].
			[[Datei:DMQS Shellskripte.png\|820px\|thumb\|center\|Abb. 1: Shellskripte zum Erfassen der DMQS-Metadaten eines Simulationslaufes]]

	'''3. Creation of sets of meta data'''		'''3. Creation of sets of meta data'''

imported>Schade Peter: "Under Construction" entfernt.

2017-07-04T06:50:10Z

"Under Construction" entfernt.

← Older revision		Revision as of 06:50, 4 July 2017
Line 1:		Line 1:
	[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]		[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]
	~~<big>'''Under Construction'''</big> <br>~~
	Model results of coastal waters are systematically described by their meta data. This information will be referenced in an internal use case. Starting from a figure in a text document the reader will be directly forwarded to the meta data of the displayed model results.<br>		Model results of coastal waters are systematically described by their meta data. This information will be referenced in an internal use case. Starting from a figure in a text document the reader will be directly forwarded to the meta data of the displayed model results.<br>

imported>Schade Peter at 14:51, 22 March 2017

2017-03-22T14:51:25Z

← Older revision		Revision as of 14:51, 22 March 2017
Line 38:		Line 38:
	<big>'''Under Construction'''</big> <br>		<big>'''Under Construction'''</big> <br>
	===Documentation/Literature ===		===Documentation/Literature ===
	Meta data of model results is a part of the project data management und quality assurance in waterways engineering (DMQS). See further information on the [http://intranet.baw.de/cocoon/portal/res?t=1&ci=1&si=0&i=344&d=Aufgabe&v=DETAIL&x=DETAIL&hra=12&hi=344&hd=Aufgabe&hc=8766&sichtc=cms&r=&cid=8766&rt=&nd=Aufgabe&ni=344&uki=344&ukd=Aufgabe&uksi=&ukg=1&ucid=8766&bereich=&ber=&lim=&ord=&os=&asc=&pm=&arch=false&l=DE&s=de&su=&p=&bn=&bc=&refModule=Hauptnavigation intranet pages of the BAW]. ~~<br>~~ <br>		Meta data of model results is a part of the project data management und quality assurance in waterways engineering (DMQS). See further information on the [http://intranet.baw.de/cocoon/portal/res?t=1&ci=1&si=0&i=344&d=Aufgabe&v=DETAIL&x=DETAIL&hra=12&hi=344&hd=Aufgabe&hc=8766&sichtc=cms&r=&cid=8766&rt=&nd=Aufgabe&ni=344&uki=344&ukd=Aufgabe&uksi=&ukg=1&ucid=8766&bereich=&ber=&lim=&ord=&os=&asc=&pm=&arch=false&l=DE&s=de&su=&p=&bn=&bc=&refModule=Hauptnavigation intranet pages of the BAW]. <br>
	back to [[~~Data_Management_and_Information_Systems~~]]		----
			back to [[Data Management and Information Systems]]
	----		----
	[[Overview]]		[[Overview]]

imported>Schade Peter at 14:43, 22 March 2017

2017-03-22T14:43:49Z

← Older revision		Revision as of 14:43, 22 March 2017
Line 1:		Line 1:
	~~[[Category:Program Description]]~~
	[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]		[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]
	<big>'''Under Construction'''</big> <br>		<big>'''Under Construction'''</big> <br>

imported>Schade Peter: corrections

2017-03-22T14:37:09Z

corrections

← Older revision		Revision as of 10:08, 21 October 2022
Line 1:		Line 1:
	[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]		[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]

	The metadata set is a thoroughly documented package that describes the simulations and hydrodynamic model results in coastal engineering. The details included in metadata utterly summarizes different aspects of the data in information packages as identification, contact, quality, lineage, hierarchy, distribution, constraint, extent and reference. Broad recording and provision of mentioned information packages of a metadata set is a fundamental measure of quality and completeness for a metadata set. Completeness of metadata ensures proper utilization of data or their reproducibility. Furthermore, the metadata facilitate administration and search as well as segmentation and classification of the data. This page explains how to collect and provide institutional ([https://www.baw.de/EN/wasserbau/wasserbau.html BAW] internal) metadata for coastal simulation data considering the existing [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf BAW metadata profile].		The metadata set is a thoroughly documented package that describes the simulations and hydrodynamic [[model]] results in coastal engineering. The details included in metadata utterly summarizes different aspects of the data in information packages as [[identification]], contact, quality, lineage, hierarchy, distribution, constraint, extent and reference. Broad recording and provision of mentioned information packages of a metadata set is a fundamental measure of quality and completeness for a metadata set. Completeness of metadata ensures proper utilization of data or their reproducibility. Furthermore, the metadata facilitate administration and search as well as segmentation and classification of the data. This page explains how to collect and provide institutional ([https://www.baw.de/EN/wasserbau/wasserbau.html BAW] internal) metadata for coastal simulation data considering the existing [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf BAW metadata profile].

	'''[1] Collecting Metadata'''		'''[1] Collecting Metadata'''

	Meta-information of a model is collected in the processing chain for a simulation data. The [https://wiki.baw.de/en/index.php/UNTRIM UnTRIM] and [https://wiki.baw.de/en/index.php/UNTRIM2 UnTRIM2] modeling approach through the use of the NetCDF data format processes the metadata in either of the subsequent sub-models:		Meta-information of a [[model]] is collected in the processing chain for a simulation data. The [https://wiki.baw.de/en/index.php/UNTRIM UnTRIM] and [https://wiki.baw.de/en/index.php/UNTRIM2 UnTRIM2] modeling approach through the use of the [[NetCDF]] data format processes the metadata in either of the subsequent sub-models:

	* Hydrodynamics in utromp2007 and SubGrid in utromp2009		* Hydrodynamics in utromp2007 and SubGrid in utromp2009
	* Morphodynamics in SediMorph		* [[Morphodynamics]] in SediMorph
	* Waves in the k-Model		* Waves in the k-[[Model]]

	Setting environmental parameters is a requirement and an initial step to conduct a simulation. These environmental variables are set through run and project scripts (in German: Auftragsskripte), which also comprise metadata of the respective model. The user adequately navigates these variables on the script and provides the required attributes as metadata (figure 01). Recording meta-information is a single time customization task for a project with multiple simulation conditions. Each of the subsequent methods e.g. hydrodynamics, morphodynamics and sea motion is providing and securing their relevant metadata in the processing chain from their run and project scripts.		Setting environmental parameters is a requirement and an initial step to conduct a simulation. These environmental variables are set through run and project scripts (in German: Auftragsskripte), which also comprise metadata of the respective [[model]]. The user adequately navigates these variables on the script and provides the required attributes as metadata (figure 01). Recording meta-information is a single time customization task for a project with multiple simulation conditions. Each of the subsequent methods e.g. hydrodynamics, [[morphodynamics]] and sea motion is providing and securing their relevant metadata in the processing chain from their run and project scripts.

	[[File:Run and project script.png\|Run and project script ]]		[[File:Run and project script.png\|Run and project script ]]
	Figure 01. Run and project script (in German: Auftragsskript) to insert the meta-information for a simulation.		Figure 01. Run and project script (in German: Auftragsskript) to insert the meta-information for a simulation.

	Most of the metadata elements are acquired from the steering data of the processing environment automatically. However, there are approximately 15 customized elements relevant to the project management and contact methods the user is providing. A quality assurance aspect in this process is the validity check through comparisons of the inserted values with the existing code list as configuration data. The user provides the following elements in the project script:		Most of the metadata elements are acquired from the steering data of the processing environment automatically. However, there are approximately 15 customized elements relevant to the project management and contact methods the user is providing. A [[Quality Assurance\|quality assurance]] aspect in this process is the validity check through comparisons of the inserted values with the existing code list as configuration data. The user provides the following elements in the project script:

	* DMQS_PSP: Project PSP number from EWisA		* DMQS_PSP: Project PSP number from EWisA
Line 24:		Line 24:
	* DMQS_CREATOR_EMAIL: email address of project manager		* DMQS_CREATOR_EMAIL: email address of project manager
	* DMQS_CREATOR_ROLE: role of project manager		* DMQS_CREATOR_ROLE: role of project manager
	* DMQS_GEO_ID: a list for valid geographical identification located in %PROGHOME%\cfg\dmqs\geo\valid_geo_ids.dat.		* DMQS_GEO_ID: a list for valid geographical [[identification]] located in %PROGHOME%\cfg\dmqs\geo\valid_geo_ids.dat.
	* Further elements, in case the project is funded through mFUND:		* Further elements, in case the project is funded through mFUND:
	** MFUND_PROJECT: name of the funded project		** MFUND_PROJECT: name of the funded project
	** MFUND_ID: project funding number		** MFUND_ID: project funding number
	For a more accurate description of a simulation file, its lineage and hierarchal information is also required to be stored. The elements relevant to the model description and hierarchy to set are:		For a more accurate description of a simulation file, its lineage and hierarchal information is also required to be stored. The elements relevant to the [[model]] description and hierarchy to set are:
	* DMQS_VARIANT: describes the geometrical property of a model		* DMQS_VARIANT: describes the geometrical property of a [[model]]
	* DMQS_SCENARIO: describes the time or boundary conditions and other features of a simulation e.g. wind or storm event		* DMQS_SCENARIO: describes the time or boundary conditions and other features of a simulation e.g. wind or storm event
	* RUN: further distinguishes a simulation through its specific feature/characteristic		* RUN: further distinguishes a simulation through its specific feature/characteristic
	* DMQS_RUN_COMMENT: free text for an exact description of each individual model		* DMQS_RUN_COMMENT: free text for an exact description of each individual [[model]]
	Note: for ease and access the template files are stored as:		Note: for ease and access the template files are stored as:
	* The template file for the project script is located under $PROGHOME/bin/dmqs/set_dmqs_template.sh		* The template file for the project script is located under $PROGHOME/bin/dmqs/set_dmqs_template.sh
Line 40:		Line 40:
	** $PROGHOME/cfg/dmqs/scenario/01_template.dat		** $PROGHOME/cfg/dmqs/scenario/01_template.dat
	** $PROGHOME/cfg/dmqs/geo/valid_geo_ids.dat		** $PROGHOME/cfg/dmqs/geo/valid_geo_ids.dat
	Post-processing and analyzing algorithms append their process relevant meta-information to their output NetCDF files. Furthermore, the lineage information and steering information is complemented in this process as well. Figure 02 further illustrates the post-processing chain, which automatically writes its meta-information in the NetCDF files.		Post-processing and analyzing algorithms append their process relevant meta-information to their output [[NetCDF]] files. Furthermore, the lineage information and steering information is complemented in this process as well. Figure 02 further illustrates the post-processing chain, which automatically writes its meta-information in the [[NetCDF]] files.

	[[File:Post-processing chain.png\| Figure 02: post-processing chain.]]		[[File:Post-processing chain.png\| Figure 02: post-processing chain.]]
Line 47:		Line 47:
	'''[2] Extracting Metadata '''		'''[2] Extracting Metadata '''

	An extra metadata generation tool is integrated into the visualization software [https://wiki.baw.de/en/index.php/DAVIT Davit]. This tool is developed and customized based on the BAW requirements, to convert the NetCDF CF metadata elements to ISO compliant XML metadata. Extensible Markup Language (XML) is both human and machine-readable and the standard data format for storing and exchanging of geographical metadata and attributes via the web. The exported XML metadata set is further mapped based on ISO schemas in adherence to the relevant standards listed in BAW metadata profile [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf (GDI-BAW V1.3)], which makes it compatible with a wide range of information systems. Furthermore, to ensure the validity and completeness of the exported metadata *.XML file, a new popup window appears if any of the mandatory elements are missing in the NetCDF file.		An extra metadata generation tool is integrated into the visualization software [https://wiki.baw.de/en/index.php/DAVIT Davit]. This tool is developed and customized based on the BAW requirements, to convert the [[NetCDF]] CF metadata elements to ISO compliant XML metadata. Extensible Markup Language (XML) is both human and machine-readable and the standard data format for storing and exchanging of geographical metadata and attributes via the web. The exported XML metadata set is further mapped based on ISO schemas in adherence to the relevant standards listed in BAW metadata profile [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf (GDI-BAW V1.3)], which makes it compatible with a wide range of information systems. Furthermore, to ensure the validity and completeness of the exported metadata *.XML file, a new popup window appears if any of the mandatory elements are missing in the [[NetCDF]] file.

	'''[3] Editing Metadata'''		'''[3] Editing Metadata'''

	In this process, it is considered that the metadata is complete and valid. However, if this is not the case, supplementary manual editing possibilities are:		In this process, it is considered that the metadata is complete and valid. However, if this is not the case, supplementary manual editing possibilities are:
	# NetCDF attribute via NCO-Tool ‘ncatted’ or MATLAB ‘ncwriteatt’ command,		# [[NetCDF]] attribute via NCO-Tool ‘ncatted’ or [[MATLAB]] ‘ncwriteatt’ command,
	# XML file in a notepad e.g. Notepad++,		# XML file in a notepad e.g. Notepad++,
	# InGrid editor (IGE) interface of MIS-BAW.		# InGrid [[EDITOR\|editor]] (IGE) interface of MIS-BAW.


	'''[4] Administrating Metadata'''		'''[4] Administrating Metadata'''

	The produced XML dataset is then imported in the metadata information system MIS-BAW. The user can import a dataset via either (a) InGrid editor (IGE) or (b) CSW-T interface via a curl command. The imported elements could be also further edited in IGE. In addition, IGE provides a further possibility of manual entry of a new metadata set.		The produced XML dataset is then imported in the metadata information system MIS-BAW. The user can import a dataset via either (a) InGrid [[EDITOR\|editor]] (IGE) or (b) CSW-T interface via a curl command. The imported elements could be also further edited in IGE. In addition, IGE provides a further possibility of manual entry of a new metadata set.


	'''Acknowledgment'''		'''Acknowledgment'''

	This project took advantage of NetCDF software developed by [http://www.unidata.ucar.edu/software/netcdf/ UCAR/Unidata].		This project took advantage of [[NetCDF]] software developed by [http://www.unidata.ucar.edu/software/netcdf/ UCAR/Unidata].

	===Documentation/Literature ===		===Documentation/Literature ===

	Metadata of model results is a part of the project data management and quality assurance (DMQS) in waterways engineering. Further information is available on the [http://intranet.baw.de/cocoon/portal/res?t=1&ci=1&si=0&i=344&d=Aufgabe&v=DETAIL&x=DETAIL&hra=12&hi=344&hd=Aufgabe&hc=8766&sichtc=cms&r=&cid=8766&rt=&nd=Aufgabe&ni=344&uki=344&ukd=Aufgabe&uksi=&ukg=1&ucid=8766&bereich=&ber=&lim=&ord=&os=&asc=&pm=&arch=false&l=DE&s=de&su=&p=&bn=&bc=&refModule=Hauptnavigation intranet pages of the BAW]. <br>		Metadata of [[model]] results is a part of the project data management and [[Quality Assurance\|quality assurance]] (DMQS) in waterways engineering. Further information is available on the [http://intranet.baw.de/cocoon/portal/res?t=1&ci=1&si=0&i=344&d=Aufgabe&v=DETAIL&x=DETAIL&hra=12&hi=344&hd=Aufgabe&hc=8766&sichtc=cms&r=&cid=8766&rt=&nd=Aufgabe&ni=344&uki=344&ukd=Aufgabe&uksi=&ukg=1&ucid=8766&bereich=&ber=&lim=&ord=&os=&asc=&pm=&arch=false&l=DE&s=de&su=&p=&bn=&bc=&refModule=Hauptnavigation intranet pages of the BAW]. <br>
	----		----
	For detailed information have a look at the [https://wiki.baw.de/de/index.php/Erfassung_der_Metadaten_einer_Simulation guidelines in German].		For detailed information have a look at the [https://wiki.baw.de/de/index.php/Erfassung_der_Metadaten_einer_Simulation guidelines in German].

← Older revision		Revision as of 19:01, 6 April 2020
Line 1:		Line 1:
	[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]		[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]

	The metadata set is a thoroughly documented package that describes the simulations and hydrodynamic model results in coastal engineering. The details included in metadata utterly ~~summaries~~ different ~~aspect~~ of the data in information packages as identification, contact, quality, lineage, hierarchy, distribution, constraint, extent and reference. Completeness of metadata ensures proper utilization of data or their reproducibility. Furthermore, the metadata facilitate administration and search as well as ~~sort~~ and classification of the data. This page explains how to collect and provide institutional ([https://www.baw.de/EN/wasserbau/wasserbau.html BAW] internal) metadata for coastal simulation data considering the existing [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf BAW metadata profile].		The metadata set is a thoroughly documented package that describes the simulations and hydrodynamic model results in coastal engineering. The details included in metadata utterly summarizes different aspects of the data in information packages as identification, contact, quality, lineage, hierarchy, distribution, constraint, extent and reference. Broad recording and provision of mentioned information packages of a metadata set is a fundamental measure of quality and completeness for a metadata set. Completeness of metadata ensures proper utilization of data or their reproducibility. Furthermore, the metadata facilitate administration and search as well as segmentation and classification of the data. This page explains how to collect and provide institutional ([https://www.baw.de/EN/wasserbau/wasserbau.html BAW] internal) metadata for coastal simulation data considering the existing [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf BAW metadata profile].

	'''[1] Collecting Metadata'''		'''[1] Collecting Metadata'''

	Meta-information of a ~~simulation~~ is collected in the processing chain for a simulation data. The UnTRIM and UnTRIM2 modeling approach through the use of the NetCDF data format processes the metadata in either of the subsequent sub-models:		Meta-information of a model is collected in the processing chain for a simulation data. The [https://wiki.baw.de/en/index.php/UNTRIM UnTRIM] and [https://wiki.baw.de/en/index.php/UNTRIM2 UnTRIM2] modeling approach through the use of the NetCDF data format processes the metadata in either of the subsequent sub-models:

	* Hydrodynamics in utromp2007 and SubGrid in utromp2009		* Hydrodynamics in utromp2007 and SubGrid in utromp2009
Line 11:		Line 11:
	* Waves in the k-Model		* Waves in the k-Model

	Setting environmental parameters is a requirement and an initial step to conduct a simulation. These environmental variables are set through run and project scripts (in German: Auftragsskripte), which also comprise metadata of respective ~~simulation~~. The user navigates ~~adequately~~ these variables on the script and provides the required attributes as metadata (~~see~~ figure 01). Recording meta-information is a single time customization task for a project with multiple simulation conditions. Each of the subsequent methods e.g. hydrodynamics, morphodynamics and sea motion is providing and securing their relevant metadata in the processing chain from their run and project scripts.		Setting environmental parameters is a requirement and an initial step to conduct a simulation. These environmental variables are set through run and project scripts (in German: Auftragsskripte), which also comprise metadata of the respective model. The user adequately navigates these variables on the script and provides the required attributes as metadata (figure 01). Recording meta-information is a single time customization task for a project with multiple simulation conditions. Each of the subsequent methods e.g. hydrodynamics, morphodynamics and sea motion is providing and securing their relevant metadata in the processing chain from their run and project scripts.

	[[File:Run and project script.png\|Run and project script ]]		[[File:Run and project script.png\|Run and project script ]]
	Figure 01. Run and project script (Auftragsskript) to insert the meta-information for a simulation.		Figure 01. Run and project script (in German: Auftragsskript) to insert the meta-information for a simulation.

	Most of the metadata elements are acquired from the steering data of the processing environment automatically. However, there are approximately 15 customized elements relevant to the project management and contact methods the user is providing. A quality assurance aspect in this process is the validity check through comparisons of the inserted values with the existing code list as configuration data. The user provides the following elements in the project script:		Most of the metadata elements are acquired from the steering data of the processing environment automatically. However, there are approximately 15 customized elements relevant to the project management and contact methods the user is providing. A quality assurance aspect in this process is the validity check through comparisons of the inserted values with the existing code list as configuration data. The user provides the following elements in the project script:
Line 28:		Line 28:
	** MFUND_PROJECT: name of the funded project		** MFUND_PROJECT: name of the funded project
	** MFUND_ID: project funding number		** MFUND_ID: project funding number
	For a more accurate description of a simulation file, its lineage and hierarchal information is also required to be stored. The elements relevant to the ~~simulation~~ description and hierarchy to set are:		For a more accurate description of a simulation file, its lineage and hierarchal information is also required to be stored. The elements relevant to the model description and hierarchy to set are:
	* DMQS_VARIANT: describes the geometrical property of a model		* DMQS_VARIANT: describes the geometrical property of a model
	* DMQS_SCENARIO: describes the time or boundary conditions and other features of a simulation e.g. wind or storm event		* DMQS_SCENARIO: describes the time or boundary conditions and other features of a simulation e.g. wind or storm event
	* RUN: distinguishes ~~further~~ a simulation through its specific feature/characteristic		* RUN: further distinguishes a simulation through its specific feature/characteristic
	* DMQS_RUN_COMMENT: free text for an exact description of each individual ~~simulation~~		* DMQS_RUN_COMMENT: free text for an exact description of each individual model
	Note: for ease and access the template files are ~~store~~ as:		Note: for ease and access the template files are stored as:
	* The template file for the project script is located under $PROGHOME/bin/dmqs/set_dmqs_template.sh		* The template file for the project script is located under $PROGHOME/bin/dmqs/set_dmqs_template.sh
	* Template files for the run script are located under $PROGHOME/examples/slurm		* Template files for the run script are located under $PROGHOME/examples/slurm
Line 47:		Line 47:
	'''[2] Extracting Metadata '''		'''[2] Extracting Metadata '''

	An extra metadata generation tool is integrated into the visualization software [https://wiki.baw.de/en/index.php/DAVIT Davit]. This tool is developed and customized based on the BAW requirements, to convert the NetCDF CF metadata elements to ISO compliant XML metadata. Extensible Markup Language (XML) is both human and machine-readable and the standard data format for storing and exchanging of geographical metadata and attributes via the web. The exported XML metadata set is further mapped based on ISO ~~schema~~ in adherence to the relevant standards listed in BAW metadata profile [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf (GDI-BAW V1.3)], which makes it compatible with a wide range of information systems. Furthermore, to ensure the validity and completeness of the exported metadata *.XML file, a new popup window appears if any of the mandatory ~~element is~~ missing in the NetCDF file.		An extra metadata generation tool is integrated into the visualization software [https://wiki.baw.de/en/index.php/DAVIT Davit]. This tool is developed and customized based on the BAW requirements, to convert the NetCDF CF metadata elements to ISO compliant XML metadata. Extensible Markup Language (XML) is both human and machine-readable and the standard data format for storing and exchanging of geographical metadata and attributes via the web. The exported XML metadata set is further mapped based on ISO schemas in adherence to the relevant standards listed in BAW metadata profile [http://wiki-intern.baw.de/images/b/b4/Metadatenprofil_GDI-BAW-V_1.3_Technische_Doku_1.0.pdf (GDI-BAW V1.3)], which makes it compatible with a wide range of information systems. Furthermore, to ensure the validity and completeness of the exported metadata *.XML file, a new popup window appears if any of the mandatory elements are missing in the NetCDF file.

	'''[3] Editing Metadata'''		'''[3] Editing Metadata'''

	In this process, it is considered that the metadata is complete and valid. However, if this is not the case, ~~further~~ manual editing possibilities are:		In this process, it is considered that the metadata is complete and valid. However, if this is not the case, supplementary manual editing possibilities are:
	# NetCDF attribute via NCO-Tool ‘ncatted’ or MATLAB ‘ncwriteatt’ command,		# NetCDF attribute via NCO-Tool ‘ncatted’ or MATLAB ‘ncwriteatt’ command,
	# XML file in a notepad e.g. Notepad++,		# XML file in a notepad e.g. Notepad++,

← Older revision		Revision as of 14:37, 22 March 2017
Line 2:		Line 2:
	[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]		[[de:Metadaten_von_Modellergebnissen_im_Küstenwasserbau]]
	<big>'''Under Construction'''</big> <br>		<big>'''Under Construction'''</big> <br>
	Model results of coastal waters ~~can be~~ systematically described by meta data. In an internal use case ~~it will be possible to reference this information~~. Starting from a ~~picture~~ in a text document the reader will be directly forwarded to the meta data of the displayed model results.<br>		Model results of coastal waters are systematically described by their meta data. This information will be referenced in an internal use case. Starting from a figure in a text document the reader will be directly forwarded to the meta data of the displayed model results.<br>

	These guidelines explain how meta data of coastal models can be collected and provided to the BAW staff. The described method is connected to the use of NetCDF files. It starts with the first step in the processing chain with output of UGRID [[CF-NETCDF.NC\|CF-NETCDF files]], the simulation:		These guidelines explain how meta data of coastal models can be collected and provided to the BAW staff. The described method is connected to the use of NetCDF files. It starts with the first step in the processing chain with output of UGRID [[CF-NETCDF.NC\|CF-NETCDF files]], the simulation:
Line 22:		Line 22:
	'''3. Creation of sets of meta data'''		'''3. Creation of sets of meta data'''

	The visualisation tool [Davit] has been enabled to generate sets of meta data in the machine readable XML-Format and as PDF ~~document~~. The user can activate the DMQS metadata generation in Davit's screenshot ~~menue~~. ~~One~~ reduced set of meta data describing the picture ~~is generated when the picture is stored,~~ a second ~~set contains~~ information about the underlying NetCDF file. [http://wiki.baw.de/de/index.php/Erzeugung_der_Metadatensätze\| Full story.].		The visualisation tool [Davit] has been enabled to generate sets of meta data in the machine readable XML-Format and as PDF documents. The user can activate the DMQS metadata generation in Davit's screenshot menu where the output of figures is defined. A reduced set of meta data describing the picture and a second one containing information about the underlying NetCDF file are generated. [http://wiki.baw.de/de/index.php/Erzeugung_der_Metadatensätze\| Full story.].

	'''4. Editing and ~~Copying~~ of sets of meta data'''		'''4. Editing and copying of sets of meta data'''

	After simulation or analysis result files are often moved to other hard disks. ~~Permanently stored~~ data is moved into an archive. Since the meta data contains links to steering and result files, the paths in the links have to be adapted. Copying and editing is done with automatically generated scripts. [http://wiki.baw.de/de/index.php/Editieren_und_Kopieren_der_Metadatensätze\| Full story.].		After simulation or analysis result files are often moved to other hard disks. In order to store data permanently it is e.g. moved into an archive. Since the meta data contains links to steering and result files, the paths in the links have to be adapted. Copying and editing is done with automatically generated scripts. [http://wiki.baw.de/de/index.php/Editieren_und_Kopieren_der_Metadatensätze\| Full story.].

	'''5. Referencing sets of meta data in a document'''		'''5. Referencing sets of meta data in a document'''

	The reader of a dokument can get information about a ~~picture~~ displaying model results by clicking on ~~the picture~~. ~~Technically this~~ is done by a Word macro referencing the set of meta data in PDF format. ~~Picture~~ and PDF have similar names:		The reader of a dokument can get information about a figure displaying model results by clicking on it. This is technically done by a Word macro referencing the set of meta data in PDF format. Figure and PDF have similar names:
	* <path>/<picture>.png		* <path>/<picture>.png
	* <path>/<picture>.png.pdf: set of meta data for the ~~picture~~ <br>		* <path>/<picture>.png.pdf: set of meta data for the figure <br>
	[http://wiki.baw.de/de/index.php/Einbinden_von_Metadatensätzen_in_ein_Dokument\| Full story.].		[http://wiki.baw.de/de/index.php/Einbinden_von_Metadatensätzen_in_ein_Dokument\| Full story].


Line 39:		Line 39:
	<big>'''Under Construction'''</big> <br>		<big>'''Under Construction'''</big> <br>
	===Documentation/Literature ===		===Documentation/Literature ===
	Meta data of model results is a part of the project data management und quality assurance in waterways engineering (DMQS). See further information on the [http://intranet.baw.de/cocoon/portal/res?t=1&ci=1&si=0&i=344&d=Aufgabe&v=DETAIL&x=DETAIL&hra=12&hi=344&hd=Aufgabe&hc=8766&sichtc=cms&r=&cid=8766&rt=&nd=Aufgabe&ni=344&uki=344&ukd=Aufgabe&uksi=&ukg=1&ucid=8766&bereich=&ber=&lim=&ord=&os=&asc=&pm=&arch=false&l=DE&s=de&su=&p=&bn=&bc=&refModule=Hauptnavigation intranet pages of the BAW].		Meta data of model results is a part of the project data management und quality assurance in waterways engineering (DMQS). See further information on the [http://intranet.baw.de/cocoon/portal/res?t=1&ci=1&si=0&i=344&d=Aufgabe&v=DETAIL&x=DETAIL&hra=12&hi=344&hd=Aufgabe&hc=8766&sichtc=cms&r=&cid=8766&rt=&nd=Aufgabe&ni=344&uki=344&ukd=Aufgabe&uksi=&ukg=1&ucid=8766&bereich=&ber=&lim=&ord=&os=&asc=&pm=&arch=false&l=DE&s=de&su=&p=&bn=&bc=&refModule=Hauptnavigation intranet pages of the BAW]. <br> <br>
	back to [[Data_Management_and_Information_Systems]]		back to [[Data_Management_and_Information_Systems]]
	----		----
	[[Overview]]		[[Overview]]