Testseite ProgamDescription: Difference between revisions
From BAWiki
Franz Simons (talk | contribs) Created page with "{{ProgramDescription |name_de=NCCHUNKIE |name=NCCHUNKIE |version=April 2022 |version_descr=September 2022 }}" |
Franz Simons (talk | contribs) No edit summary |
||
| Line 4: | Line 4: | ||
|version=April 2022 | |version=April 2022 | ||
|version_descr=September 2022 | |version_descr=September 2022 | ||
|catchwords= | |||
data conversion<br /> | |||
postprocessor<br /> | |||
automatic adjustment of number of data READ to chunk size of input data <br /> | |||
automatic computation of chunk sizes for result variables to support orthogonal data access <br /> | |||
parrallelization (''collective'' IO) using [http://de.wikipedia.org/wiki/Message_Passing_Interface MPI]<br /> | |||
Acknowledgment: ''This project took advantage of netCDF software developed by UCAR/Unidata ([http://www.unidata.ucar.edu/software/netcdf/ www.unidata.ucar.edu/software/netcdf/]).'' | |||
|shortdescription= | |||
Program NCCHUNKIE can be used to chunk data stored in [[CF-NETCDF.NC|cf-netcdf.nc]] files: | |||
# Chunk sizes are computed automatically, and all dimensions are chunked to support orthogonal data access; | |||
# Resulting chunks sizes lie somewhere between ''Disc Block Size'' and ''Chunk Buffer Size''; | |||
# Online compression is used during storage of data (low level of compression used, level 1); | |||
# A netCDF-4 file is created (serial version creates NetCDF4 ''classic model format''); | |||
# Parameters ''cache size'' and ''cache nelems'' used within netCDF-4 API are automatically determined. | |||
Just in case '''HDF error''' is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only. | |||
}} | }} | ||
Revision as of 08:50, 4 December 2025
Basic Information
Name of Program
NCCHUNKIE
Version-Date
April 2022
Description-Date
September 2022
Catchwords
data conversion
postprocessor
automatic adjustment of number of data READ to chunk size of input data
automatic computation of chunk sizes for result variables to support orthogonal data access
parrallelization (collective IO) using MPI
Acknowledgment: This project took advantage of netCDF software developed by UCAR/Unidata (www.unidata.ucar.edu/software/netcdf/).
Short Description of Functionality
Program NCCHUNKIE can be used to chunk data stored in cf-netcdf.nc files:
- Chunk sizes are computed automatically, and all dimensions are chunked to support orthogonal data access;
- Resulting chunks sizes lie somewhere between Disc Block Size and Chunk Buffer Size;
- Online compression is used during storage of data (low level of compression used, level 1);
- A netCDF-4 file is created (serial version creates NetCDF4 classic model format);
- Parameters cache size and cache nelems used within netCDF-4 API are automatically determined.
Just in case HDF error is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only.
Input-Files
{{{inputfiles}}}
Output-Files
{{{outputfiles}}}
Methodology
{{{methodology}}}
Program(s) to run before this Program
{{{preprocessor}}}
Program(s) to run after this Program
{{{postprocessor}}}
Additional Information
Language
{{{language}}}
Additional software
{{{add_software}}}
Original Version
{{{contact_original}}}
Maintenance
{{{contact_maintenance}}}
Documentation/Literature
{{{documentation}}}
back to Program Descriptions