Skip to content

GitLab

Commit 3e4bf1b3 authored by PStar User Account's avatar PStar User Account
Browse files

mostly changes to use matlab native netcdf, plus mintrp2 to fill only gaps up to a certain length 

 Committer: PStar User Account <pstar@192.168.122.1>

 On branch dy113
 Changes to be committed:
   (use "git reset HEAD <file>..." to unstage)

	new file:   source/mexec_snctools/Contents.m
	new file:   source/mexec_snctools/nc_add_dimension.m
	new file:   source/mexec_snctools/nc_addvar.m
	new file:   source/mexec_snctools/nc_attget.m
	new file:   source/mexec_snctools/nc_attput.m
	new file:   source/mexec_snctools/nc_attputq.m
	new file:   source/mexec_snctools/nc_create_empty.m
	new file:   source/mexec_snctools/nc_getdiminfo.m
	new file:   source/mexec_snctools/nc_getvarinfo.m
	new file:   source/mexec_snctools/nc_global.m
	new file:   source/mexec_snctools/nc_info.m
	new file:   source/mexec_snctools/nc_infoq.m
	new file:   source/mexec_snctools/nc_infoqatt.m
	new file:   source/mexec_snctools/nc_infoqdim.m
	new file:   source/mexec_snctools/nc_padheader.m
	new file:   source/mexec_snctools/nc_varget.m
	new file:   source/mexec_snctools/nc_varput.m
	new file:   source/mexec_snctools/nc_varrename.m
	modified:   source/msubs/m_add_variable_name.m
	modified:   source/msubs/m_read_header.m
	modified:   source/msubs/m_unpack_dimnames.m
	modified:   source/msubs/m_write_variable.m
	modified:   source/mtechsas/mtnames.m
	modified:   source/unfinished/m_interp.m
	new file:   source/unfinished/mintrp2.m
parent c0f66864
Hide whitespace changes
Inline Side-by-side
Showing with 1036 additions and 41 deletions
source/mexec_snctools/Contents.m 0 → 100644
View file @ 3e4bf1b3
% mexec snctools toolbox
%
% v 1.0 bak on jc191, 24N hydro cruise 7 Feb 2020
%
% This set of .m scripts is intended to replace the snctools toolbox.
%
% Rather than edit all the mexec programs, a new set of routines has been
% written and is intended to have identical names and outputs as the
% snctools routines of the same name.
%
% Not every snctools routine has been replaced. But as far as I can tell at
% the time of writing, every snctools routine that is calleed by mexec has
% been replaced.
%
% The original snctools routines did a lot of checking, and could be very
% slow for routines called often. Many mexec programs or scripts executed
% very slowly because of time spent in snctools. Especially nc_info..
%
% The new routines don't do very much checking of calling arguments. It is
% assumed that if the routine is called from a program or mexec processing
% scripts, then the call will be properly formatted.
%
% The resulting library executes much much faster than snctools used to.
%
% The calls all use the matlab netcdf command, eg netcdf.inqVar, and so
% on. There are no residual calls (that I can find) to the old mexnc library.
% So all that is needed is the netcdf.xxxxx commands.
%
% The matlab netcdf library has commands like "ncread". These have been
% avoided, since they are often slower. Useful commands for investigating
% NetCDF files include ncinfo, ncdisp.
%
% The code was written with mexecNetCDF files in mind, so some standard
% NetCDF features are ignored. For example _FillValue is recognised on both
% input and output, but "missing_value" is not, and nor are scaling and
% offset, although these could easily be included in the relevant parts of
% the code. Handling of unlimited dimensions may fail in some places,
% because it has not been extensively tested. mexec netcdf does not use
% unlimited dimensions.
%
% When it started to be used for routine mecxec processing, it broke when
% reading a CODAS netcdf file. This is because it expected a _FillValue
% attribute, which did not exist. This glitch was fixed, but their could be
% others if reading non-mexec NetCDF files.
%
% Files in this library, each of which has some help comments
%
% Contents.m
% nc_add_dimension.m
% nc_addvar.m
% nc_attget.m
% nc_attput.m
% nc_attputq.m (alias for nc_attput)
% nc_create_empty.m
% nc_getdiminfo.m
% nc_getvarinfo.m
% nc_global.m
% nc_info.m
% nc_infoqatt.m (alias for nc_info)
% nc_infoqdim.m (alias for nc_info)
% nc_infoq.m (alias for nc_info)
% nc_padheader.m
% nc_varget.m
% nc_varput.m
% nc_varrename.m
source/mexec_snctools/nc_add_dimension.m 0 → 100644
View file @ 3e4bf1b3
function nc_add_dimension(ncfile , dimname , dimlen)
%
% version of nc_add_dimension for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% The old snctools description was
%
% % NC_ADD_DIMENSION: adds a dimension to an existing netcdf file
% %
% % USAGE: nc_add_dimension ( ncfile, dimension_name, dimension_size );
% %
% % PARAMETERS:
% % Input:
% % ncfile: path to netcdf file
% % dimension_name: name of dimension to be added
% % dimension_size: length of new dimension. If zero, it will be an
% % unlimited dimension.
% % Output:
% % none
%
%
% bak jc191 6 Feb 2020
%
%
% Note from netcdf.defDim page:
% dimlen for unlimited dimensions should be specified by the constant value for 'UNLIMITED'.
% eg
% timeDimId = netcdf.defDim(ncid,'time',netcdf.getConstant('UNLIMITED'));
%
% netcdf.getConstant('UNLIMITED') seems to be zero
%
ncid = netcdf.open(ncfile, 'WRITE' );
netcdf.reDef(ncid );
dimid = netcdf.defDim(ncid, dimname, dimlen);
netcdf.endDef(ncid );
netcdf.close(ncid );
return
source/mexec_snctools/nc_addvar.m 0 → 100644
View file @ 3e4bf1b3
function nc_addvar (ncfile , v)
%
% version of nc_addvar for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% In this version, varaiable v has Name. Dimension and Ncype
% This is called from mexec program m_add_variable name; attributes are
% added later in m_add_default_variable_attributes.
%
% The old snctools description was
%
% % NC_ADDVAR: adds a variable to a NetCDF file
% %
% % USAGE: nc_addvar ( ncfile, varstruct );
% %
% % PARAMETERS:
% % Input
% % ncfile:
% % varstruct:
% % This is a structure with four fields:
% %
% % Name
% % Nctype
% % Dimension
% % Attribute
% %
% % "Name" is just that, the name of the variable to be defined.
% %
% % "Nctype" should be
% % 'double', 'float', 'int', 'short', or 'byte', or 'char'
% % 'NC_DOUBLE', 'NC_FLOAT', 'NC_INT', 'NC_SHORT', 'NC_BYTE', 'NC_CHAR'
% %
% % "Dimension" is a cell array of dimension names.
% %
% % "Attribute" is also a structure array. Each element has two
% % fields, "Name", and "Value".
%
%
% bak jc191 6 Feb 2020
%
ncid = netcdf.open(ncfile,'WRITE');
num_dims_v = length(v.Dimension);
dimids = nan(1,num_dims_v);
for k = 1:num_dims_v
dimids(1,k) = netcdf.inqDimID(ncid,v.Dimension{k});
end
% we seem to need to flip the dimids to get the same effect as the snctools
% nc_addvar
dimids = fliplr(dimids);
netcdf.reDef(ncid);
varid = netcdf.defVar(ncid, v.Name , v.Nctype, dimids );
netcdf.endDef(ncid );
netcdf.close(ncid );
return
source/mexec_snctools/nc_attget.m 0 → 100644
View file @ 3e4bf1b3
function attval = nc_attget (ncfile , varname , attribute_name)
%
% version of nc_attget for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% Use numeric varname = -1 to get a global attribute
%
% The old snctools description was
%
% % USAGE: att_value = nc_attget(ncfile, varname, attribute_name);
% %
% % PARAMETERS:
% % Input:
% % ncfile:
% % name of netcdf file in question
% % varname:
% % name of variable in question. In order to retrieve a global
% % attribute, use NC_GLOBAL for the variable name argument.
% % Do NOT use 'global'!
% % attribute_name:
% % name of attribute in question
% % Output:
% % values:
% % value of attribute asked for. Returns the empty matrix
% % in case of an error.
%
%
% bak jc191 6 Feb 2020
%
ncid=netcdf.open(ncfile,'nowrite');
if isnumeric(varname)
% global attribute, can enter -1 in the argument
varid = netcdf.getConstant('GLOBAL'); % usually -1
else
varid = netcdf.inqVarID(ncid, varname);
end
attval = netcdf.getAtt(ncid,varid,attribute_name);
netcdf.close(ncid);
% we could use the ncattread utility, but it seems to be slower.
% if isnumeric(varname)
% % global attribute, value = -1, signified here by '/'
% varname = '/';
% end
%
% attval = ncreadatt(ncfile, varname, attribute_name);
return
source/mexec_snctools/nc_attput.m 0 → 100644
View file @ 3e4bf1b3
function nc_attput (ncfile , varname , attribute_name , attval )
%
% version of nc_attput for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% Use numeric varname = -1 to get a global attribute
%
% The old snctools description was
%
% % NC_ATTPUT: writes an attribute into a netCDF file
% % NC_ATTPUT(NCFILE,VARNAME,ATTNAME,ATTVAL) writes the data in ATTVAL to
% % the attribute ATTNAME of the variable VARNAME of the netCDF file NCFILE.
% % VARNAME should be the name of a netCDF VARIABLE, but one can also use the
% % mnemonic nc_global to specify a global attribute. Do not use 'global'.
%
%
% bak jc191 6 Feb 2020
%
ncid=netcdf.open(ncfile,'WRITE');
if isnumeric(varname)
% global attribute, can enter -1 in the argument
varid = netcdf.getConstant('GLOBAL'); % usually -1
else
varid = netcdf.inqVarID(ncid, varname);
end
netcdf.reDef(ncid); % need to enter def mode in case attribute is new
netcdf.putAtt(ncid,varid,attribute_name, attval);
netcdf.endDef(ncid);
netcdf.close(ncid);
% %
% % % we could use the ncattwrite utility, but it seems to be slower, and we
% % % are only using netcdf.xxxxx() routines.
% %
% % if isnumeric(varname)
% % % global attribute, value = -1, signified here by '/'
% % varname = '/';
% % end
% %
% % ncwriteatt(ncfile , varname, attribute_name , attval);
return
source/mexec_snctools/nc_attputq.m 0 → 120000
View file @ 3e4bf1b3
nc_attput.m
\ No newline at end of file
source/mexec_snctools/nc_create_empty.m 0 → 100644
View file @ 3e4bf1b3
function nc_create_empty (ncfile , mode)
%
% version of nc_attget for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% The old snctools description was
%
% % NC_CREATE_EMPTY: creates an empty netCDF file
% % NC_CREATE_EMPTY(NCFILE,MODE) creates the empty netCDF file NCFILE
% % with the given MODE. MODE is optional, defaulting to
% % nc_clobber_mode.
%
%
% bak jc191 6 Feb 2020
%
% In mexec, this is only called from m_create_file, with mode 'nc_clobber'
% mode is 'nc_clobber' or 'nc_noclobber'.
% is also seems that 'clobber' and 'noclobber' are ok.
% The netcdf.create help page refers to 'clobber' and 'noclobber' so translate to that
% form before call to netcdf.create.
%
if strcmp(lower(mode),'nc_noclobber'); mode = 'noclobber'; end
if strcmp(lower(mode),'nc_clobber'); mode = 'clobber'; end
ncid = netcdf.create(ncfile , mode);
netcdf.close(ncid);
return
source/mexec_snctools/nc_getdiminfo.m 0 → 100644
View file @ 3e4bf1b3
function diminfo = nc_getdiminfo(ncfile , dimname)
%
% version of nc_getdiminfo for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% In this version, the dimname is always a name, not a dimid.
% In the old snctools version, it was possible to pass in the dimid
% of an already open file.
%
% The old snctools description was
%
% % NC_GETDIMINFO: returns metadata about a specific NetCDF dimension
% %
% % DINFO = NC_GETDIMINFO(NCFILE,DIMNAME) returns information about the
% % dimension DIMNAME in the netCDF file NCFILE.
% %
% % DINFO = NC_GETDIMINFO(NCID,DIMID) returns information about the
% % dimension with numeric ID DIMID in the already-opened netCDF file
% % with file ID NCID. This form is not recommended for use from the
% % command line.
% %
% % Upon output, DINFO will have the following fields.
% %
% % Name:
% % a string containing the name of the dimension.
% % Length:
% % a scalar equal to the length of the dimension
% % Unlimited:
% % A flag, either 1 if the dimension is an unlimited dimension
% % or 0 if not.
%
%
% bak jc191 6 Feb 2020
%
ncid=netcdf.open(ncfile,'NOWRITE');
[ndims,nvars,ngatts,unlimited_dimnum] = netcdf.inq(ncid); % save the unlimited dimnum if there is one
dimid = netcdf.inqDimID(ncid , dimname);
[diminfo.Name, diminfo.Length] = netcdf.inqDim(ncid, dimid);
% check for unlimited dimension. This has not been extensively tested.
diminfo.Unlimited = 0;
if dimid == unlimited_dimnum
diminfo.Unlimited = 1;
end
netcdf.close(ncid);
return
\ No newline at end of file
source/mexec_snctools/nc_getvarinfo.m 0 → 100644
View file @ 3e4bf1b3
function vinfo = nc_getvarinfo( ncfile , varname )
%
% version of nc_getvarinfo for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% In this version, the varname is always a name, not a varid.
% In the old snctools version, it was possible to pass in the varid
% of an already open file.
%
% The old snctools description was
%
% % NC_GETVARINFO: returns metadata about a specific NetCDF variable
% %
% % VINFO = NC_GETVARINFO(NCFILE,VARNAME) returns a metadata structure VINFO about
% % the variable VARNAME in the netCDF file NCFILE.
% %
% % VINFO = NC_GETVARINFO(NCID,VARID) returns a metadata structure VINFO about
% % the variable whose netCDF variable-id is VARID, and whose parent file-id is
% % NCID. The netCDF file is assumed to be open, and in this case the file will
% % not be closed upon completion.
% %
% % VINFO will have the following fields:
% %
% % Name:
% % a string containing the name of the variable.
% % Nctype:
% % a string specifying the NetCDF datatype of this variable.
% % Unlimited:
% % Flag, either 1 if the variable has an unlimited dimension or 0 if not.
% % Dimensions:
% % a cell array with the names of the dimensions upon which this variable
% % depends.
% % Attribute:
% % An array of structures corresponding to the attributes defined for the
% % specified variable.
% %
% % Each "Attribute" element contains the following fields.
% %
% % Name:
% % a string containing the name of the attribute.
% % Nctype:
% % a string specifying the NetCDF datatype of this attribute.
% % Attnum:
% % a scalar specifying the attribute id
% % Value:
% % either a string or a double precision value corresponding to the
% % value of the attribute
%
%
% bak jc191 6 Feb 2020
%
%
% This code attempts to check for an unlimited dimension, the number of
% which is discovered from netcdf.inq . This has not been tested because
% we don't have any mexec files with unlimited dimension.
%
clear vinfo
ncid=netcdf.open(ncfile,'NOWRITE');
[ndims,nvars,ngatts,unlimited_dimnum] = netcdf.inq(ncid); % save the unlimited dimnum if there is one
varid = netcdf.inqVarID(ncid, varname);
[varname,xtype,dimids,natts] = netcdf.inqVar(ncid, varid);
% we seem to need to flip the dimids to get the same effect as the snctools
% nc_addvar
dimids = fliplr(dimids);
ndimsv = length(dimids);
vinfo.Name = varname;
vinfo.Nctype = xtype;
vinfo.Unlimited = 0;
for kd = 1:ndimsv
[dimname, dimlength] = netcdf.inqDim(ncid, dimids(kd));
vinfo.Dimension{kd} = dimname;
vinfo.Size(kd) = dimlength;
if dimids(kd) == unlimited_dimnum
vinfo.Unlimited = 1;
end
end
for ka = 1:natts
attname = netcdf.inqAttName(ncid,varid,ka-1);
[xtype,attlen] = netcdf.inqAtt(ncid,varid,attname);
vinfo.Attribute(ka).Name = attname;
vinfo.Attribute(ka).Nctype = xtype;
vinfo.Attribute(ka).Attnum = ka-1;
vinfo.Attribute(ka).Value = netcdf.getAtt(ncid,varid,attname);
end
netcdf.close(ncid);
return
source/mexec_snctools/nc_global.m 0 → 100644
View file @ 3e4bf1b3
function value = nc_global()
%
% version of nc_attput for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% The old snctools description was
%
% % NC_GLOBAL: returns enumerated constant NC_GLOBAL in netcdf.h
% %
% % USAGE: the_value = nc_global;
%
% bak jc191 6 Feb 2020
value = netcdf.getConstant('NC_GLOBAL'); % The number to use as varid global attributes. Expect it to be -1
return
source/mexec_snctools/nc_info.m 0 → 100644
View file @ 3e4bf1b3
function finfo = nc_info ( ncfile )
%
% version of nc_info for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% in tests, this is as fast as netcdf ncinfo and much fater than snctools
% nc_info
%
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% The old snctools description was
%
% % USAGE: fileinfo = nc_info ( ncfile );
% %
% % PARAMETERS:
% % Input:
% % ncfile:
% % a string that specifies the name of the NetCDF file
% % Output:
% % fileinfo:
% % A structure whose fields contain information about the contants
% % of the NetCDF file. The set of fields return in "fileinfo" are:
% %
% % Filename:
% % a string containing the name of the file.
% % Dimension:
% % an array of structures describing the NetCDF dimensions.
% % Dataset:
% % an array of structures describing the NetCDF datasets.
% % Attributes:
% % An array of structures These correspond to the global attributes.
% %
% %
% % Each "Dimension" element contains the following fields:
% %
% % Name:
% % a string containing the name of the dimension.
% % Length:
% % a scalar value, the size of this dimension
% % Unlimited:
% % Set to 1 if the dimension is the record dimension, set to
% % 0 otherwise.
% %
% %
% % Each "Dataset" element contains the following structures.
% %
% % Name:
% % a string containing the name of the variable.
% % Nctype:
% % a string specifying the NetCDF datatype of this variable.
% % Dimensions:
% % a cell array with the names of the dimensions upon which
% % this variable depends.
% % Unlimited:
% % Flag, either 1 if the variable has an unlimited dimension
% % or 0 if not.
% % Rank:
% % Array that describes the size of each dimension upon which
% % this dataset depends.
% % DataAttributes:
% % Same as "Attributes" above, but here they are the variable
% % attributes.
% %
% % Each "Attribute" or "DataAttribute" element contains the following
% % fields.
% %
% % Name:
% % a string containing the name of the attribute.
% % Nctype:
% % a string specifying the NetCDF datatype of this attribute.
% % Attnum:
% % a scalar specifying the attribute id
% % Value:
% % either a string or a double precision value corresponding to
% % the value of the attribute
% %
% %
% % The "Dataset" elements are not populated with the actual data values.
% %
% % This routine purposefully mimics that of Mathwork's hdfinfo.
% % character data remains just character data.
%
%
% bak jc191 6 Feb 2020
%
clear finfo
finfo.Filename = ncfile;
globvarnum = netcdf.getConstant('NC_GLOBAL'); % The number to use as varid global attributes. Expect it to be -1
ncid = netcdf.open(ncfile,'NOWRITE');
[numdims,numvars,numglobalatts,unlimdimid] = netcdf.inq(ncid);
for kg = 1:numglobalatts
% for global atts, use varid -1
attname = netcdf.inqAttName(ncid,globvarnum,kg-1);
[xtype,attlen] = netcdf.inqAtt(ncid,globvarnum,attname);
attrvalue = netcdf.getAtt(ncid,globvarnum,attname);
finfo.Attribute(kg).Name = attname;
finfo.Attribute(kg).Nctype = xtype;
finfo.Attribute(kg).Attnum = kg-1;
finfo.Attribute(kg).Value = attrvalue;
end
for kd = 1:numdims
[dimname, dimlen] = netcdf.inqDim(ncid,kd-1);
finfo.Dimension(kd).Name = dimname;
finfo.Dimension(kd).Length = dimlen;
finfo.Dimension(kd).Unlimited = 0;
if kd == unlimdimid; finfo.Dimension(kd).Unlimited = 1; end
end
for kv = 1:numvars
[varname,xtype,dimids,natts] = netcdf.inqVar(ncid,kv-1);
finfo.Dataset(kv).Name = varname;
finfo.Dataset(kv).Nctype = xtype;
finfo.Dataset(kv).Unlimited = 0;
finfo.Dataset(kv).Dimension = {};
finfo.Dataset(kv).Size = [];
for kd = 1:length(dimids)
finfo.Dataset(kv).Dimension = [finfo.Dataset(kv).Dimension finfo.Dimension(dimids(kd)+1).Name];
finfo.Dataset(kv).Size = [finfo.Dataset(kv).Size finfo.Dimension(dimids(kd)+1).Length];
if ismember(unlimdimid,dimids); finfo.Dataset(kv).Unlimited = 1; end
end
for ka = 1:natts
attname = netcdf.inqAttName(ncid,kv-1,ka-1);
[xtype,attlen] = netcdf.inqAtt(ncid,kv-1,attname);
attrvalue = netcdf.getAtt(ncid,kv-1,attname);
finfo.Dataset(kv).Attribute(ka).Name = attname;
finfo.Dataset(kv).Attribute(ka).Nctype = xtype;
finfo.Dataset(kv).Attribute(ka).Attnum = ka-1;
finfo.Dataset(kv).Attribute(ka).Value = attrvalue;
end
end
if isfield(finfo,'Attribute'); finfo.Attribute = finfo.Attribute(:); end
if isfield(finfo,'Dimension'); finfo.Dimension = finfo.Dimension(:); end
if isfield(finfo,'Dataset'); finfo.Dataset = finfo.Dataset(:); end
netcdf.close(ncid);
return
source/mexec_snctools/nc_infoq.m 0 → 120000
View file @ 3e4bf1b3
nc_info.m
\ No newline at end of file
source/mexec_snctools/nc_infoqatt.m 0 → 120000
View file @ 3e4bf1b3
nc_info.m
\ No newline at end of file
source/mexec_snctools/nc_infoqdim.m 0 → 120000
View file @ 3e4bf1b3
nc_info.m
\ No newline at end of file
source/mexec_snctools/nc_padheader.m 0 → 100644
View file @ 3e4bf1b3
function nc_padheader (ncfile , num_bytes)
%
% version of nc_padheader for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% The old snctools description was
%
% % NC_PADHEADER: pads the metadata header of a netcdf file
% %
% % When a netCDF file gets very large, adding new attributes can become
% % a time-consuming process. This can be mitigated by padding the
% % netCDF header with additional bytes. Subsequent new attributes will
% % not result in long time delays unless the length of the new
% % attribute exceeds that of the header.
% %
% % USAGE: nc_padheader ( ncfile, num_bytes );
%
%
% bak jc191 6 Feb 2020
%
ncid = netcdf.open(ncfile,'WRITE');
netcdf.reDef(ncid);
%
% Sets the padding to be "num_bytes" at the end of the header section.
%
netcdf.endDef(ncid,num_bytes,4,0,4);
netcdf.close(ncid);
return
source/mexec_snctools/nc_varget.m 0 → 100644
View file @ 3e4bf1b3
function vardata = nc_varget(ncfile , varname , varargin )
%
% version of nc_varget for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% varargin can be
% start, count, stride
% start begins at zero, so if start = [2 3] it will retrieve data from
% row 3, column 4;
% if start is defined and count is empty, the data are retrieved from
% the point denoted by the start offset, to the end of each row and column.
%
% When I tried it, the -1 notation for count in the snctools version did
% not work as intended.
%
% This version handles _FillValue but not missing values or scaling or
% offset.
%
% The old snctools description was
%
% % DATA = NC_VARGET(NCFILE,VARNAME) retrieves all the data from the
% % variable VARNAME in the netCDF file NCFILE.
% %
% % DATA = NC_VARGET(NCFILE,VARNAME,START,COUNT) retrieves the contiguous
% % portion of the variable specified by the index vectors START and
% % COUNT. Remember that SNCTOOLS indexing is zero-based, not
% % one-based. Specifying a -1 in COUNT means to retrieve everything
% % along that dimension from the START coordinate.
% %
% % DATA = NC_VARGET(NCFILE,VARNAME,START,COUNT,STRIDE) retrieves
% % a non-contiguous portion of the dataset. The amount of
% % skipping along each dimension is given through the STRIDE vector.
% %
% % NCFILE can also be an OPeNDAP URL if the proper java SNCTOOLS
% % backend is installed. See the README for details.
% %
% % NC_VARGET tries to be intelligent about retrieving the data.
% % Since most general matlab operations are done in double precision,
% % retrieved numeric data will be cast to double precision, while
% % character data remains just character data. %
%
%
% bak jc191 6 Feb 2020
%
args = varargin(:);
start = [];
count = [];
stride = [];
switch length(args)
case 1
start = args{1};
case 2
start = args{1};
count = args{2};
case 3
start = args{1};
count = args{2};
stride = args{3};
otherwise
end
ncid = netcdf.open(ncfile,'NOWRITE');
varid = netcdf.inqVarID(ncid,varname);
% is there is start but no count, then calculate count to retrieve from
% start to the end of each dimension
[varname,xtype,dimids,natts] = netcdf.inqVar(ncid, varid);
if isempty(count) & ~isempty(start)
% we seem to need to flip the dimids to get the same effect as the snctools
% nc_addvar
dimids = fliplr(dimids);
ndimsv = length(dimids);
clear dimsize
for kd = 1:ndimsv
[dimname, dimlength] = netcdf.inqDim(ncid, dimids(kd));
dimsize(kd) = dimlength;
end
count = dimsize(:)'-start;
end
% the snctools version seems to retrieve every element if there is start but no count.
%
% I couldn't see anywhere where mexec calls nc_varget with start but no
% count, so the above difference should not matter.
start = fliplr(start);
count = fliplr(count);
stride = fliplr(stride);
if isempty(start)
vardata = netcdf.getVar(ncid, varid );
elseif isempty(count)
fprintf(2,'\n%s\n','Warning, nc_varget has been called with start but no count');
fprintf(2,'%s\n','The bahaviour of this version is different than the behaviour of the snctools version');
fprintf(2,'%s\n\n','Suggest the code be rewritten to state exactly what ''count'' should be');
error('error in nc_varget');
% if you reinstate the line of code below you will get precisely one data
% value, at location given by 'start'
vardata = netcdf.getVar(ncid, varid, start );
elseif isempty(stride)
vardata = netcdf.getVar(ncid, varid, start, count );
else
vardata = netcdf.getVar(ncid, varid, start, count, stride);
end
if length(dimids) == 1
vardata = vardata(:); % techsas files have one dimension which is time; snctools nc_varget makes this a column vector
else
vardata = vardata'; % this is necessary to mimic the action of the old nc_varget;
% the above line will only work on 2-D vars, ie 1xN, Mx1, MxN.
% mexec is not presently set up to handle more than 2-D vars.
% mexec files have at least three dimension parameters: n_unity, nrows1, ncols1.
% mexec variables are always 2-D. MxN 1xN or Mx1.
end
% now handle _FillValue
switch ( xtype )
case { 6 } % 6 is double
try
fillvalue = netcdf.getAtt(ncid, varid, '_FillValue');
vardata(vardata == fillvalue) = NaN;
catch
% If there is no fillvalue defined, take no action
end
otherwise
end
netcdf.close(ncid);
return
source/mexec_snctools/nc_varput.m 0 → 100644
View file @ 3e4bf1b3
function nc_varput(ncfile , varname , vardata , varargin )
%
% version of nc_varput for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% varargin can be
% start, count, stride
% start begins at zero, so if start = [2 3] it will retrieve data from
% row 3, column 4;
% if start is defined and count is empty, the data are retrieved from
% the point denoted by the start offset, to the end of each row and column.
%
% If subsetting is used, ie start, count, stride, then the size of vardata
% must exactly match the number of elements implied by the subsetting.
% There is no checking that this matches, the program will just crash if it
% things don't match.
%
% When I tried it, the -1 notation for count in the snctools version did
% not work as intended.
%
% This version handles _FillValue but not missing values or scaling or
% offset.
%
% The old snctools description was
%
% % NC_VARPUT: Writes data into a netCDF file.
% %
% % NC_VARPUT(NCFILE,VARNAME,DATA) writes the matlab variable DATA to
% % the variable VARNAME in the netCDF file NCFILE. The main requirement
% % here is that DATA have the same dimensions as the netCDF variable.
% %
% % NC_VARPUT(NCFILE,VARNAME,DATA,START,COUNT) writes DATA contiguously,
% % starting at the zero-based index START and with extents given by
% % COUNT.
% %
% % NC_VARPUT(NCFILE,VARNAME,DATA,START,COUNT,STRIDE) writes DATA
% % starting at the zero-based index START with extents given by
% % COUNT, but this time with strides given by STRIDE. If STRIDE is not
% % given, then it is assumes that all data is contiguous.
%
%
% bak jc191 6 Feb 2020
%
args = varargin(:);
start = [];
count = [];
stride = [];
switch length(args)
case 1
start = args{1};
case 2
start = args{1};
count = args{2};
case 3
start = args{1};
count = args{2};
stride = args{3};
otherwise
end
ncid = netcdf.open(ncfile,'WRITE');
varid = netcdf.inqVarID(ncid,varname);
% If there is start but no count, then calculate count to retrieve from
% start to the end of each dimension
[varname,xtype,dimids,natts] = netcdf.inqVar(ncid, varid);
if isempty(count) & ~isempty(start)
% calculate the size of count to ensure writing from start to the end
% of each dimension
% we seem to need to flip the dimids to get the same effect as the snctools
% nc_varput
dimids = fliplr(dimids);
ndimsv = length(dimids);
clear dimsize
for kd = 1:ndimsv
[dimname, dimlength] = netcdf.inqDim(ncid, dimids(kd));
dimsize(kd) = dimlength;
end
count = dimsize(:)'-start;
end
% I couldn't see anywhere where mexec calls nc_varget with start but no
% count, so the above difference should not matter.
start = fliplr(start);
count = fliplr(count);
stride = fliplr(stride);
% now handle _FillValue
switch ( xtype )
case { 6 } % 6 is double
fillvalue = netcdf.getAtt(ncid, varid, '_FillValue');
vardata(isnan(vardata)) = fillvalue;
otherwise
end
% Now transpose; mexec files are always 2-D
if length(dimids) == 2
vardata = vardata'; % this is necessary to mimic the action of the old nc_varget;
% the above line will only work on 2-D vars, ie 1xN, Mx1, MxN.
% mexec is not presently set up to handle more than 2-D vars.
% mexec files have at least three dimension parameters: n_unity, nrows1, ncols1.
% mexec variables are always 2-D. MxN 1xN or Mx1.
end
if isempty(start)
netcdf.putVar(ncid, varid, vardata );
elseif isempty(count)
fprintf(2,'\n%s\n','Warning, nc_varget has been called with start but no count');
fprintf(2,'%s\n','The bahaviour of this version is different than the behaviour of the snctools version');
fprintf(2,'%s\n\n','Suggest the code be rewritten to state exactly what ''count'' should be');
error('error in nc_varget');
% I don't know what happens in putvar if there is start but not count
% getvar reads only one point.
netcdf.putVar(ncid, varid, start, vardata );
elseif isempty(stride) % assume it is 1
% k1 = start(1)+ (1:count(1));
% k2 = start(2)+ (1:count(2));
netcdf.putVar(ncid, varid, start, count, vardata );
else
% k1 = start(1)+1+stride(1)*(0:count(1)-1);
% k2 = start(2)+1+stride(2)*(0:count(2)-1);
netcdf.putVar(ncid, varid, start, count, stride, vardata);
end
netcdf.close(ncid);
return
\ No newline at end of file
source/mexec_snctools/nc_varrename.m 0 → 100644
View file @ 3e4bf1b3
function nc_varrename (ncfile, old_name , new_name)
%
% version of nc_varrename for mexec to replace snctools version
% the aim is to replace snctools nc_ calls with faster versions that make
% simple native matlab netcdf calls.
%
% rather than changing every mexec program, the snctools library can be
% replaced with this library
%
% The old snctools description was
%
% % NC_VARRENAME: renames a NetCDF variable.
% %
% % NC_VARRENAME(NCFILE,OLD_VARNAME,NEW_VARNAME) renames a netCDF variable from
% % OLD_VARNAME to NEW_VARNAME.
%
%
% bak jc191 6 Feb 2020
%
ncid=netcdf.open(ncfile,'write');
netcdf.reDef(ncid); % reDef and endDef may happen automatically, but safer to keep them in the code than leave them out
varid = netcdf.inqVarID(ncid, old_name);
netcdf.renameVar(ncid, varid, new_name);
netcdf.endDef(ncid);
netcdf.close(ncid);
return
\ No newline at end of file
source/msubs/m_add_variable_name.m
View file @ 3e4bf1b3
......@@ -18,26 +18,7 @@ if nargin < 3
end
if exist('ncinfo','file') == 2 % jc191 use matlab netcdf if available
% make metadata look the same as the old snctools nc_info
% dm is the matlab data structure; dr is the reconstructed one.
clear dm dr
dm = ncinfo(ncfile.name);
dr.Filename = ncfile.name;
dr.Attribute = dm.Attributes; % don't need in add_variable_name
dr.Dimension = dm.Dimensions(:);
for kl = 1:length(dm.Variables);
dr.Dataset(kl).Name = dm.Variables(kl).Name;
dr.Dataset(kl).Dimension = dm.Variables(kl).Dimensions.Name;
dr.Dataset(kl).Nctype = find(strcmp(dm.Variables(kl).Datatype,{'' 'char' '' '' '' 'double' ''}));
end
if isfield(dr,'Dataset'); dr.Dataset = dr.Dataset(:); end
metadata = dr;
else
metadata = nc_infoqdim(ncfile.name); %refresh metadata % command before jc191
end
metadata = nc_infoqdim(ncfile.name); %refresh metadata
ncfile.metadata = metadata;
dimnames = m_unpack_dimnames(ncfile);
......
source/msubs/m_read_header.m
View file @ 3e4bf1b3
......@@ -11,27 +11,7 @@ ncfile = m_resolve_filename(ncfile);
ncfile = m_ismstar(ncfile); % check it is an mstar file
if exist('ncinfo','file') == 2 % jc191 use matlab netcdf if available
% make metadata look the same as the old snctools nc_info
% dm is the matlab data structure; dr is the reconstructed one.
clear dm dr
dm = ncinfo(ncfile.name);
dr.Filename = ncfile.name;
dr.Attribute = dm.Attributes; % don't need in add_variable_name
dr.Dimension = dm.Dimensions(:);
for kl = 1:length(dm.Variables);
dr.Dataset(kl).Name = dm.Variables(kl).Name;
dr.Dataset(kl).Dimension = dm.Variables(kl).Dimensions.Name;
dr.Dataset(kl).Nctype = find(strcmp(dm.Variables(kl).Datatype,{'' 'char' '' '' '' 'double' ''}));
end
if isfield(dr,'Dataset'); dr.Dataset = dr.Dataset(:); end
metadata = dr;
else
metadata = nc_infoqatt(ncfile.name); %refresh metadata % command before jc191
end
metadata = nc_infoqatt(ncfile.name); %refresh metadata
ncfile.metadata = metadata;
var_names = m_unpack_varnames(ncfile);
dimnames = m_unpack_dimnames(ncfile);
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment