NCCHUNKIE: Difference between revisions
From BAWiki
Günther Lang (talk | contribs) (version date changed) |
Günther Lang (talk | contribs) (NCDVAR listed as postprocessor) |
||
| Line 3: | Line 3: | ||
|name=NCCHUNKIE | |name=NCCHUNKIE | ||
|version=January 2020 | |version=January 2020 | ||
|version_descr= | |version_descr=March 2020 | ||
|catchwords= | |catchwords= | ||
data conversion<br /> | data conversion<br /> | ||
| Line 35: | Line 35: | ||
|preprocessor=[[DATACONVERT]], [[GRIDCONVERT]], [[NCANALYSE]], [[NCAGGREGATE]], [[NCCUTOUT]], [[NCDELTA]], [[NetCDF Operators]], [[UNTRIM2007]], [[UNTRIM2]] | |preprocessor=[[DATACONVERT]], [[GRIDCONVERT]], [[NCANALYSE]], [[NCAGGREGATE]], [[NCCUTOUT]], [[NCDELTA]], [[NetCDF Operators]], [[UNTRIM2007]], [[UNTRIM2]] | ||
|postprocessor=[[NCANALYSE]], [[NCAGGREGATE]], [[NCDELTA]], [[NCCHUNKIE]], [[NCCUTOUT]], [[NC2TABLE]], [[NetCDF Operators]] | |postprocessor=[[NCANALYSE]], [[NCAGGREGATE]], [[NCDELTA]], [[NCDVAR]], [[NCCHUNKIE]], [[NCCUTOUT]], [[NC2TABLE]], [[NetCDF Operators]] | ||
|language=Fortran95 | |language=Fortran95 | ||
|add_software=--- | |add_software=--- | ||
Revision as of 14:36, 17 March 2020
Basic Information
Name of Program
NCCHUNKIE
Version-Date
January 2020
Description-Date
March 2020
Catchwords
data conversion
postprocessor
automatic adjustment of number of data READ to chunk size of input data
automatic computation of chunk sizes for result variables to support orthogonal data access
Acknowledgment: This project took advantage of netCDF software developed by UCAR/Unidata (www.unidata.ucar.edu/software/netcdf/).
Short Description of Functionality
Program NCCHUNKIE can be used to chunk data stored in cf-netcdf.nc files:
- Chunk sizes are computed automatically, and all dimensions are chunked to support orthogonal data access;
- Resulting chunks sizes lie somewhere between Disc Block Size and Chunk Buffer Size;
- Online compression is used during storage of data (low level of compression used, level 1);
- A netCDF-4 file in classic model format is created (which is a netCDF compatible HDF5 format);
- Parameters cache size and cache nelems used within netCDF-4 API are automatically determined.
Just in case HDF error is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only.
Input-Files
- No input steering data file required (interactive specification of file name);
- UGRID CF NetCDF data set (file type CF-NETCDF.NC).
Output-Files
- UGRID CF NetCDF data set (file type CF-NETCDF.NC);
- informative printer file of program execution (file type NCCHUNKIE.sdr) with informations related to program execution, required time for READ and WRITE of data, effective data transfer rates.
- trace of program execution (file type NCCHUNKIE.trc)
Methodology
Some concepts published in https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf were used.
Program(s) to run before this Program
DATACONVERT, GRIDCONVERT, NCANALYSE, NCAGGREGATE, NCCUTOUT, NCDELTA, NetCDF Operators, UNTRIM2007, UNTRIM2
Program(s) to run after this Program
NCANALYSE, NCAGGREGATE, NCDELTA, NCDVAR, NCCHUNKIE, NCCUTOUT, NC2TABLE, NetCDF Operators
Additional Information
Language
Fortran95
Additional software
---
Original Version
Maintenance
Documentation/Literature
No template files available.
back to Program Descriptions