BAWiki - User contributions [en]

Testseite ProgamDescription

2025-12-04T09:02:15Z

Simons:

{{ProgramDescription
|name_de=NCCHUNKIE
|name=NCCHUNKIE
|version=April 2022
|version_descr=September 2022
|catchwords=
data conversion 
postprocessor 
automatic adjustment of number of data READ to chunk size of input data 
automatic computation of chunk sizes for result variables to support orthogonal data access 
parrallelization (''collective'' IO) using [http://de.wikipedia.org/wiki/Message_Passing_Interface MPI] 

Acknowledgment: ''This project took advantage of netCDF software developed by UCAR/Unidata ([http://www.unidata.ucar.edu/software/netcdf/ www.unidata.ucar.edu/software/netcdf/]).''

|shortdescription=
Program NCCHUNKIE can be used to chunk data stored in [[CF-NETCDF.NC|cf-netcdf.nc]] files:
# Chunk sizes are computed automatically, and all dimensions are chunked to support orthogonal data access;
# Resulting chunks sizes lie somewhere between ''Disc Block Size'' and ''Chunk Buffer Size'';
# Online compression is used during storage of data (low level of compression used, level 1);
# A netCDF-4 file is created (serial version creates NetCDF4 ''classic model format'');
# Parameters ''cache size'' and ''cache nelems'' used within netCDF-4 API are automatically determined.

Just in case '''HDF error''' is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only.

|inputfiles=
# No input steering data file required (parameters in command line or interactive specification);
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]).

|outputfiles=
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]);
# informative '''printer file''' of program execution (file type NCCHUNKIE.sdr) with informations related to program execution, required time for READ and WRITE of data, effective data transfer rates.
# '''trace''' of program execution (file type NCCHUNKIE.trc)

|methodology=
Some concepts published in [https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf] were used.

|preprocessor=[[DATACONVERT]], [[GRIDCONVERT]], [[NCANALYSE]], [[NCAGGREGATE]], [[NCCUTOUT]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NetCDF Operators]], [[UNTRIM2007]], [[UNTRIM2]]
|postprocessor=[[NCANALYSE]], [[NCAGGREGATE]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NCCHUNKIE]], [[NCCUTOUT]], [[NC2TABLE]], [[NetCDF Operators]]
|language=Fortran95
|add_software=---
|contact_original=G. Lang
|contact_maintenance=[mailto:pos.proghome@baw.de Working group POS]
|documentation=
* [https://izw-campus.baw.de/ ''IZW-Campus''] (Podcast, available in German only)
** 2020-11-02: [https://izw-campus.baw.de/goto.php?target=cat_1832&client_id=iliasclient ''Chunking und NCCHUNKIE''];
** 2019-08-02: [https://izw-campus.baw.de/goto.php?target=cat_1831&client_id=iliasclient ''Anmerkungen zur Chunked I/O mit NetCDF-4 / HDF5''].
* Template of program execution test(s):
** No template files available.
}}

Testseite ProgamDescription

2025-12-04T09:01:35Z

Simons:

Testseite ProgamDescription

2025-12-04T09:01:09Z

Simons:

{{ProgramDescription
|name_de=NCCHUNKIE
|name=NCCHUNKIE
|version=April 2022
|version_descr=September 2022
|catchwords=
data conversion 
postprocessor 
automatic adjustment of number of data READ to chunk size of input data 
automatic computation of chunk sizes for result variables to support orthogonal data access 
parrallelization (''collective'' IO) using [http://de.wikipedia.org/wiki/Message_Passing_Interface MPI] 

Acknowledgment: ''This project took advantage of netCDF software developed by UCAR/Unidata ([http://www.unidata.ucar.edu/software/netcdf/ www.unidata.ucar.edu/software/netcdf/]).''

|shortdescription=
Program NCCHUNKIE can be used to chunk data stored in [[CF-NETCDF.NC|cf-netcdf.nc]] files:
# Chunk sizes are computed automatically, and all dimensions are chunked to support orthogonal data access;
# Resulting chunks sizes lie somewhere between ''Disc Block Size'' and ''Chunk Buffer Size'';
# Online compression is used during storage of data (low level of compression used, level 1);
# A netCDF-4 file is created (serial version creates NetCDF4 ''classic model format'');
# Parameters ''cache size'' and ''cache nelems'' used within netCDF-4 API are automatically determined.

Just in case '''HDF error''' is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only.

|inputfiles=
# No input steering data file required (parameters in command line or interactive specification);
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]).

|outputfiles=
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]);
# informative '''printer file''' of program execution (file type NCCHUNKIE.sdr) with informations related to program execution, required time for READ and WRITE of data, effective data transfer rates.
# '''trace''' of program execution (file type NCCHUNKIE.trc)

|methodology=
Some concepts published in [https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf] were used.

|preprocessor=[[DATACONVERT]], [[GRIDCONVERT]], [[NCANALYSE]], [[NCAGGREGATE]], [[NCCUTOUT]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NetCDF Operators]], [[UNTRIM2007]], [[UNTRIM2]]
|postprocessor=[[NCANALYSE]], [[NCAGGREGATE]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NCCHUNKIE]], [[NCCUTOUT]], [[NC2TABLE]], [[NetCDF Operators]]
|language=Fortran95
|add_software=---
|contact_original=G. Lang
|contact_maintenance=[mailto:pos.proghome@baw.de Working group POS]
|documentation=
* [https://izw-campus.baw.de/ ''IZW-Campus''] (Podcast, available in German only)
** 2020-11-02: [https://izw-campus.baw.de/goto.php?target=cat_1832&client_id=iliasclient ''Chunking und NCCHUNKIE''];
** 2019-08-02: [https://izw-campus.baw.de/goto.php?target=cat_1831&client_id=iliasclient ''Anmerkungen zur Chunked I/O mit NetCDF-4 / HDF5''].
* of program execution (s):
** No template files available.
}}

Testseite ProgamDescription

2025-12-04T09:00:16Z

Simons:

{{ProgramDescription
|name_de=NCCHUNKIE
|name=NCCHUNKIE
|version=April 2022
|version_descr=September 2022
|catchwords=
data conversion 
postprocessor 
automatic adjustment of number of data READ to chunk size of input data 
automatic computation of chunk sizes for result variables to support orthogonal data access 
parrallelization (''collective'' IO) using [http://de.wikipedia.org/wiki/Message_Passing_Interface MPI] 

Acknowledgment: ''This project took advantage of netCDF software developed by UCAR/Unidata ([http://www.unidata.ucar.edu/software/netcdf/ www.unidata.ucar.edu/software/netcdf/]).''

|shortdescription=
Program NCCHUNKIE can be used to chunk data stored in [[CF-NETCDF.NC|cf-netcdf.nc]] files:
# Chunk sizes are computed automatically, and all dimensions are chunked to support orthogonal data access;
# Resulting chunks sizes lie somewhere between ''Disc Block Size'' and ''Chunk Buffer Size'';
# Online compression is used during storage of data (low level of compression used, level 1);
# A netCDF-4 file is created (serial version creates NetCDF4 ''classic model format'');
# Parameters ''cache size'' and ''cache nelems'' used within netCDF-4 API are automatically determined.

Just in case '''HDF error''' is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only.

|inputfiles=
# No input steering data file required (parameters in command line or interactive specification);
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]).

|outputfiles=
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]);
# informative '''printer file''' of program execution (file type NCCHUNKIE.sdr) with informations related to program execution, required time for READ and WRITE of data, effective data transfer rates.
# '''trace''' of program execution (file type NCCHUNKIE.trc)

|methodology=
Some concepts published in [https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf] were used.

|preprocessor=[[DATACONVERT]], [[GRIDCONVERT]], [[NCANALYSE]], [[NCAGGREGATE]], [[NCCUTOUT]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NetCDF Operators]], [[UNTRIM2007]], [[UNTRIM2]]
|postprocessor=[[NCANALYSE]], [[NCAGGREGATE]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NCCHUNKIE]], [[NCCUTOUT]], [[NC2TABLE]], [[NetCDF Operators]]
|language=Fortran95
|add_software=---
|contact_original=G. Lang
|contact_maintenance=[mailto:pos.proghome@baw.de Working group POS]
|documentation=
* [https://izw-campus.baw.de/ ''IZW-Campus''] (Podcast, available in German only)
** 2020-11-02: [https://izw-campus.baw.de/goto.php?target=cat_1832&client_id=iliasclient ''Chunking und NCCHUNKIE''];
** 2019-08-02: [https://izw-campus.baw.de/goto.php?target=cat_1831&client_id=iliasclient ''Anmerkungen zur Chunked I/O mit NetCDF-4 / HDF5''].
* '''trace''' of program execution (s):
** No template files available.
}}

Testseite ProgamDescription

2025-12-04T08:59:37Z

Simons:

{{ProgramDescription
|name_de=NCCHUNKIE
|name=NCCHUNKIE
|version=April 2022
|version_descr=September 2022
|catchwords=
data conversion 
postprocessor 
automatic adjustment of number of data READ to chunk size of input data 
automatic computation of chunk sizes for result variables to support orthogonal data access 
parrallelization (''collective'' IO) using [http://de.wikipedia.org/wiki/Message_Passing_Interface MPI] 

Acknowledgment: ''This project took advantage of netCDF software developed by UCAR/Unidata ([http://www.unidata.ucar.edu/software/netcdf/ www.unidata.ucar.edu/software/netcdf/]).''

|shortdescription=
Program NCCHUNKIE can be used to chunk data stored in [[CF-NETCDF.NC|cf-netcdf.nc]] files:
# Chunk sizes are computed automatically, and all dimensions are chunked to support orthogonal data access;
# Resulting chunks sizes lie somewhere between ''Disc Block Size'' and ''Chunk Buffer Size'';
# Online compression is used during storage of data (low level of compression used, level 1);
# A netCDF-4 file is created (serial version creates NetCDF4 ''classic model format'');
# Parameters ''cache size'' and ''cache nelems'' used within netCDF-4 API are automatically determined.

Just in case '''HDF error''' is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only.

|inputfiles=
# No input steering data file required (parameters in command line or interactive specification);
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]).

|outputfiles=
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]);
# informative '''printer file''' of program execution (file type NCCHUNKIE.sdr) with informations related to program execution, required time for READ and WRITE of data, effective data transfer rates.
# '''trace''' of program execution (file type NCCHUNKIE.trc)

|methodology=
Some concepts published in [https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf] were used.

|preprocessor=[[DATACONVERT]], [[GRIDCONVERT]], [[NCANALYSE]], [[NCAGGREGATE]], [[NCCUTOUT]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NetCDF Operators]], [[UNTRIM2007]], [[UNTRIM2]]
|postprocessor=[[NCANALYSE]], [[NCAGGREGATE]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NCCHUNKIE]], [[NCCUTOUT]], [[NC2TABLE]], [[NetCDF Operators]]
|language=Fortran95
|add_software=---
|contact_original=G. Lang
|contact_maintenance=[mailto:pos.proghome@baw.de Working group POS]
|documentation=
* [https://izw-campus.baw.de/ ''IZW-Campus''] (Podcast, available in German only)
** 2020-11-02: [https://izw-campus.baw.de/goto.php?target=cat_1832&client_id=iliasclient ''Chunking und NCCHUNKIE''];
** 2019-08-02: [https://izw-campus.baw.de/goto.php?target=cat_1831&client_id=iliasclient ''Anmerkungen zur Chunked I/O mit NetCDF-4 / HDF5''].
* '''trace''' of program execution (file type NCCHUNKIE.trc)
** No template files available.
}}

Testseite ProgamDescription

2025-12-04T08:57:52Z

Simons:

{{ProgramDescription
|name_de=NCCHUNKIE
|name=NCCHUNKIE
|version=April 2022
|version_descr=September 2022
|catchwords=
data conversion 
postprocessor 
automatic adjustment of number of data READ to chunk size of input data 
automatic computation of chunk sizes for result variables to support orthogonal data access 
parrallelization (''collective'' IO) using [http://de.wikipedia.org/wiki/Message_Passing_Interface MPI] 

Acknowledgment: ''This project took advantage of netCDF software developed by UCAR/Unidata ([http://www.unidata.ucar.edu/software/netcdf/ www.unidata.ucar.edu/software/netcdf/]).''

|shortdescription=
Program NCCHUNKIE can be used to chunk data stored in [[CF-NETCDF.NC|cf-netcdf.nc]] files:
# Chunk sizes are computed automatically, and all dimensions are chunked to support orthogonal data access;
# Resulting chunks sizes lie somewhere between ''Disc Block Size'' and ''Chunk Buffer Size'';
# Online compression is used during storage of data (low level of compression used, level 1);
# A netCDF-4 file is created (serial version creates NetCDF4 ''classic model format'');
# Parameters ''cache size'' and ''cache nelems'' used within netCDF-4 API are automatically determined.

Just in case '''HDF error''' is detected during read of a data record, the program tries to reconstruct the wanted data set from adjacent (in time) records for the same variable. This type of repair works for time dependent variables only.

|inputfiles=
# No input steering data file required (parameters in command line or interactive specification);
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]).

|outputfiles=
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]);
# informative '''printer file''' of program execution (file type NCCHUNKIE.sdr) with informations related to program execution, required time for READ and WRITE of data, effective data transfer rates.
# '''trace''' of program execution (file type NCCHUNKIE.trc)

|methodology=
Some concepts published in [https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf https://support.hdfgroup.org/pubs/papers/2008-06_netcdf4_perf_report.pdf] were used.

|preprocessor=[[DATACONVERT]], [[GRIDCONVERT]], [[NCANALYSE]], [[NCAGGREGATE]], [[NCCUTOUT]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NetCDF Operators]], [[UNTRIM2007]], [[UNTRIM2]]
|postprocessor=[[NCANALYSE]], [[NCAGGREGATE]], [[NCDELTA]], [[NCDVAR]], [[NCMERGE]], [[NCCHUNKIE]], [[NCCUTOUT]], [[NC2TABLE]], [[NetCDF Operators]]
|language=Fortran95
|add_software=---
|contact_original=G. Lang
|contact_maintenance=[mailto:pos.proghome@baw.de Working group POS]
|documentation=
* [https://izw-campus.baw.de/ ''IZW-Campus''] (Podcast, available in German only)
** 2020-11-02: [https://izw-campus.baw.de/goto.php?target=cat_1832&client_id=iliasclient ''Chunking und NCCHUNKIE''];
** 2019-08-02: [https://izw-campus.baw.de/goto.php?target=cat_1831&client_id=iliasclient ''Anmerkungen zur Chunked I/O mit NetCDF-4 / HDF5''].
# '''UGRID CF NetCDF data set''' (file type [[CF-NETCDF.NC]]);
# informative '''printer file''' of program execution (file type NCCHUNKIE.sdr) with informations related to program execution, required time for READ and WRITE of data, effective data transfer rates.
# '''trace''' of program execution (file type NCCHUNKIE.trc)
}}

Testseite ProgamDescription

2025-12-04T08:55:34Z

Simons:

Testseite ProgamDescription

2025-12-04T08:54:51Z

Simons:

Testseite ProgamDescription

2025-12-04T08:53:52Z

Simons:

Testseite ProgamDescription

2025-12-04T08:52:47Z

Simons:

Testseite ProgamDescription

2025-12-04T08:52:17Z

Simons:

Testseite ProgamDescription

2025-12-04T08:51:45Z

Simons:

Testseite ProgamDescription

2025-12-04T08:51:16Z

Simons:

Testseite ProgamDescription

2025-12-04T08:50:48Z

Simons:

Testseite ProgamDescription

2025-12-04T08:50:11Z

Simons:

Testseite ProgamDescription

2025-12-04T08:49:51Z

Simons: Created page with "{{ProgramDescription |name_de=NCCHUNKIE |name=NCCHUNKIE |version=April 2022 |version_descr=September 2022 }}"

{{ProgramDescription
|name_de=NCCHUNKIE
|name=NCCHUNKIE
|version=April 2022
|version_descr=September 2022

}}