Commit 8f6d0b5b authored by thopri's avatar thopri
Browse files

added unit testing to setup.py script

parent 118b1c65
......@@ -21,15 +21,15 @@ Steps to take to install PyNEMO, creating a specific conda virtual environment i
- Create conda environment for PyNEMO::
$ cd to/PyNEMO/directory
$ conda env create -f environment_pynemo.yml
$ conda env create -f pynemo_37.yml
- Activate the new virtual environment::
$ source activate pynemo_env
$ source activate pynemo3
- Install Jave JRE (outside scope of this readme) and link libjvm.dylib to LD_LIBRARY_PATH variable::
- Install Jave JDK (outside scope of this readme) and link Java Home to conda environment::
$ export LD_LIBRARY_PATH=/path/to/java/library/folder/containing/libjvm.dylib:$LD_LIBARY_PATH # see notes below
$ export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-13.0.2.jdk/Contents/Home # see notes below
- Install PyNEMO::
......@@ -39,45 +39,39 @@ Steps to take to install PyNEMO, creating a specific conda virtual environment i
This should result in PyNEMO being installed in the virtual environment, and can be checked by entering::
$ pynemo -v
$ pynemo -h
Resulting in a help usage prompt::
$ usage: pynemo -g -s <namelist.bdy>
$ usage: pynemo [-g] -s -d <namelist.bdy>
-g (optional) will open settings editor before extracting the data
-s <bdy filename> file to use
-d (optional) will download CMEMS data using provided bdy file
The virtual environment can be deactivated to return you to the normal prompt by typing::
$ conda deactivate
$ conda deactivate
To reactivate, the following needs to be typed::
$ source activate pynemo_env
$ source activate pynemo3
To use PyNEMO, the following command is entered: (the example will run an benchmarking test)::
$ pynemo -s /path/to/namelist/file (e.g. PyNEMO/inputs/namelist_remote.bdy)
**Additional NOTES**
For Macbook Pro 2015, macOS Mojave and Java SDK 13 and JRE 8 the following path for the libjvm library should be correct::
/Library/Java/JavaVirtualMachines/jdk-13.0.1.jdk/Contents/Home/lib/server
Other commands include -d which downloads the specified CMEMS data in the namelist bdy file.::
Resulting in the following command: (this will be different for different java versions and operating systems)::
$ pynemo -d /PyNEMO/inputs/namelist_cmems.bdy
$ export LD_LIBRARY_PATH=/Library/Java/JavaVirtualMachines/jdk-13.0.1.jdk/Contents/Home/lib/server:$LD_LIBRARY_PATH
For an iMac 2013, macOS Catalina and JRE 8 only the followinng path was found to be correct::
/Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/lib/server
With the following command being required to set the environment variable::
**Additional NOTES**
$ export LD_LIBRARY_PATH=/Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/lib/server:$LD_LIBRARY_PATH
The above path for Java Home was valid for a Macbook Pro 2015 with macOS Catalina and Java SDK 13.0.2
however for different java versions, operating systems etc this may be different
The conda environment creation command has not yet been tested. The yml document (can be opened using text editor) gives a list of all the modules and their versions that are required for PyNEMO so a environment can be constructed using this document as reference (or if you use pip!)
The conda environment yaml file has been tested with Miniconda 3.7 and found to install the enironment correctly.
**Update** conda environment yaml file has been tested (and works!) on a Macbook Pro 2015 and iMac 2013 running Anaconda 3.7 and Miniconda 3.7 respectively.
Contribution guidelines
-----------------------
......@@ -101,6 +95,16 @@ The PyNEMO module can be tested using the bench marking namelist bdy file in the
:width: 800
:alt: Example BDY coords output
Unit Tests
-------------------
To test operation of the PyNEMO module, running the PyTest script in the unit tests folder will perform a range tests on different child grids,
checking the interpolation of the source data on to the child grid. To do this the following command is required::
$ pytest -q unit_tests/unit_test.py
The command should be run from the main PyNEMO directory to enable modules to be imported correctly.
Who do I talk to?
-----------------
......
......@@ -18,7 +18,7 @@ def main():
setup_file = ''
mask_gui = False
try:
opts, dummy_args = getopt.getopt(sys.argv[1:], "hsdg", ["help", "setup=", "download_cmems=", "mask_gui"])
opts, dummy_args = getopt.getopt(sys.argv[1:], "hs:dg", ["help", "setup=", "download_cmems=", "mask_gui"])
except getopt.GetoptError:
print("usage: pynemo -g -s -d <namelist.bdy> ")
sys.exit(2)
......@@ -41,6 +41,7 @@ def main():
t1 = time.time()
print("CMEMS download time: %s" % (t1 - t0))
sys.exit(0)
if setup_file == "":
print("usage: pynemo [-g] -s <namelist.bdy> ")
sys.exit(2)
......
......@@ -11,7 +11,7 @@ import os
# generate test data by import test gen script and executing main function
# TODO: Maybe simplify this, as this import imports other scripts and is abit clunky.
import unit_tests.test_gen as tg
from pynemo.unit_tests import test_gen as tg
gen_data = tg._main()
# if a non zero is return than the grid and data generation has failed.
if gen_data != 0:
......@@ -19,7 +19,7 @@ if gen_data != 0:
# run PyNEMO with test data
# generate list of namelist.bdy files to run
namelist_files = glob.glob('unit_tests/namelist*')
namelist_files = glob.glob('pynemo/unit_tests/namelist*')
for n in namelist_files:
# run each of the namelist files
stdout, stderr = Popen(['pynemo', '-s', n], stdout=PIPE, stderr=PIPE,
......@@ -34,7 +34,7 @@ for n in namelist_files:
# perform tests
def test_temp():
test_files = glob.glob('unit_tests/test_outputs/*bdyT*')
test_files = glob.glob('pynemo/unit_tests/test_outputs/*bdyT*')
if len(test_files) == 0:
raise Exception('DONT PANIC: no temperature test files found')
for t in test_files:
......@@ -47,7 +47,7 @@ def test_temp():
assert abs(temp_[temp_ != 0.0].min() - 15) <= 0.001
def test_salinty():
test_files = glob.glob('unit_tests/test_outputs/*bdyT*')
test_files = glob.glob('pynemo/unit_tests/test_outputs/*bdyT*')
if len(test_files) == 0:
raise Exception('DONT PANIC: no salinity test files found')
for t in test_files:
......@@ -63,7 +63,7 @@ def test_salinty():
# U and V and SSH tests are required. e.g. ln_dyn2d is set to true.
def test_ssh():
test_files = glob.glob('unit_tests/test_outputs/*bdyT*')
test_files = glob.glob('pynemo/unit_tests/test_outputs/*bdyT*')
if len(test_files) == 0:
raise Exception('DONT PANIC: no SSH test files found')
for t in test_files:
......@@ -76,7 +76,7 @@ def test_ssh():
assert abs(ssh_[ssh_ != 0.0].min() - 1.0) <= 0.001
def test_U():
test_files = glob.glob('unit_tests/test_outputs/*bdyU*')
test_files = glob.glob('pynemo/unit_tests/test_outputs/*bdyU*')
if len(test_files) == 0:
raise Exception('DONT PANIC: no U current test files found')
for t in test_files:
......@@ -89,7 +89,7 @@ def test_U():
assert abs(U_[U_ != 0.0].min() - 0.5) <= 0.001
def test_V():
test_files = glob.glob('unit_tests/test_outputs/*bdyV*')
test_files = glob.glob('pynemo/unit_tests/test_outputs/*bdyV*')
if len(test_files) == 0:
raise Exception('DONT PANIC: no V current test files found')
for t in test_files:
......@@ -103,16 +103,16 @@ def test_V():
# clean up test I/O
def test_rm_out():
files = glob.glob('unit_tests/test_outputs/*')
files = glob.glob('pynemo/unit_tests/test_outputs/*')
for f in files:
os.remove(f)
files = glob.glob('unit_tests/test_outputs/*')
files = glob.glob('pynemo/unit_tests/test_outputs/*')
assert len(files) == 0
def test_rm_in():
files = glob.glob('unit_tests/test_inputs/*')
files = glob.glob('pynemo/unit_tests/test_inputs/*')
for f in files:
os.remove(f)
files = glob.glob('unit_tests/test_inputs/*')
files = glob.glob('pynemo/unit_tests/test_inputs/*')
assert len(files) == 0
File moved
......@@ -30,18 +30,18 @@
!------------------------------------------------------------------------------
! grid information
!------------------------------------------------------------------------------
sn_src_hgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_src_hgr_zps.nc'
sn_src_zgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_src_zgr_zps.nc'
sn_dst_hgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_offset_dst_hgr_zps.nc'
sn_dst_zgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_offset_dst_zgr_zps.nc'
sn_src_msk = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/mask.nc'
sn_bathy = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_offset_dst_bathy.nc'
sn_src_hgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_src_hgr_zps.nc'
sn_src_zgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_src_zgr_zps.nc'
sn_dst_hgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_offset_dst_hgr_zps.nc'
sn_dst_zgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_offset_dst_zgr_zps.nc'
sn_src_msk = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/mask.nc'
sn_bathy = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_offset_dst_bathy.nc'
!------------------------------------------------------------------------------
! I/O
!------------------------------------------------------------------------------
sn_src_dir = '/Users/thopri/Projects/PyNEMO/unit_tests/unit_tests.ncml' ! src_files/'
sn_dst_dir = '/Users/thopri/Projects/PyNEMO/unit_tests/test_outputs'
sn_src_dir = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/unit_tests.ncml' ! src_files/'
sn_dst_dir = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_outputs'
sn_fn = 'unit_test_offset' ! prefix for output files
nn_fv = -1e20 ! set fill value for output files
nn_src_time_adj = 0 ! src time adjustment
......
......@@ -30,18 +30,18 @@
!------------------------------------------------------------------------------
! grid information
!------------------------------------------------------------------------------
sn_src_hgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_src_hgr_zps.nc'
sn_src_zgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_src_zgr_zps.nc'
sn_dst_hgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_dst_hgr_zps.nc'
sn_dst_zgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_dst_zgr_zps.nc'
sn_src_msk = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/mask.nc'
sn_bathy = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_dst_bathy.nc'
sn_src_hgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_src_hgr_zps.nc'
sn_src_zgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_src_zgr_zps.nc'
sn_dst_hgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_dst_hgr_zps.nc'
sn_dst_zgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_dst_zgr_zps.nc'
sn_src_msk = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/mask.nc'
sn_bathy = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_dst_bathy.nc'
!------------------------------------------------------------------------------
! I/O
!------------------------------------------------------------------------------
sn_src_dir = '/Users/thopri/Projects/PyNEMO/unit_tests/unit_tests.ncml' ! src_files/'
sn_dst_dir = '/Users/thopri/Projects/PyNEMO/unit_tests/test_outputs'
sn_src_dir = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/unit_tests.ncml' ! src_files/'
sn_dst_dir = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_outputs'
sn_fn = 'unit_test_orth' ! prefix for output files
nn_fv = -1e20 ! set fill value for output files
nn_src_time_adj = 0 ! src time adjustment
......
......@@ -30,18 +30,18 @@
!------------------------------------------------------------------------------
! grid information
!------------------------------------------------------------------------------
sn_src_hgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_src_hgr_zps.nc'
sn_src_zgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_src_zgr_zps.nc'
sn_dst_hgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_rot_dst_hgr_zps.nc'
sn_dst_zgr = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_rot_dst_zgr_zps.nc'
sn_src_msk = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/mask.nc'
sn_bathy = '/Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/test_rot_dst_bathy.nc'
sn_src_hgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_src_hgr_zps.nc'
sn_src_zgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_src_zgr_zps.nc'
sn_dst_hgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_rot_dst_hgr_zps.nc'
sn_dst_zgr = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_rot_dst_zgr_zps.nc'
sn_src_msk = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/mask.nc'
sn_bathy = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/test_rot_dst_bathy.nc'
!------------------------------------------------------------------------------
! I/O
!------------------------------------------------------------------------------
sn_src_dir = '/Users/thopri/Projects/PyNEMO/unit_tests/unit_tests.ncml' ! src_files/'
sn_dst_dir = '/Users/thopri/Projects/PyNEMO/unit_tests/test_outputs'
sn_src_dir = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/unit_tests.ncml' ! src_files/'
sn_dst_dir = '/Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_outputs'
sn_fn = 'unit_test_rotated' ! prefix for output files
nn_fv = -1e20 ! set fill value for output files
nn_src_time_adj = 0 ! src time adjustment
......
......@@ -6,7 +6,7 @@ occurs, the results are then plotted in a figure showing original (blue points)
The source coordinate grid is also plotted (green).
"""
import unit_tests.gen_tools as gt
from pynemo.unit_tests import gen_tools as gt
# TODO: remove hard coded file names and directories.
# TODO: organise the variables better, (maybe in a single dict?)
......@@ -21,8 +21,8 @@ def _main():
max_dep = 100
min_dep = 10
z_end_dim = 1
h_fname = 'unit_tests/test_inputs/test_src_hgr_zps.nc'
z_fname = 'unit_tests/test_inputs/test_src_zgr_zps.nc'
h_fname = 'pynemo/unit_tests/test_inputs/test_src_hgr_zps.nc'
z_fname = 'pynemo/unit_tests/test_inputs/test_src_zgr_zps.nc'
grid_h1 = gt.set_hgrid(dx,dy,jpi,jpj)
grid_z1 = gt.set_zgrid(grid_h1,jpk,max_dep,min_dep,z_end_dim)
write_coord_H = gt.write_coord_H(h_fname,grid_h1)
......@@ -42,14 +42,14 @@ def _main():
min_dep = 10
z_end_dim = 1
sf = 10
h_fname = 'unit_tests/test_inputs/test_dst_hgr_zps.nc'
z_fname = 'unit_tests/test_inputs/test_dst_zgr_zps.nc'
h_fname = 'pynemo/unit_tests/test_inputs/test_dst_hgr_zps.nc'
z_fname = 'pynemo/unit_tests/test_inputs/test_dst_zgr_zps.nc'
grid_h2 = gt.set_hgrid(dx,dy,jpi,jpj,zoffx,zoffy,sf)
grid_z2 = gt.set_zgrid(grid_h2,jpk,max_dep,min_dep,z_end_dim)
write_coord_H = gt.write_coord_H(h_fname,grid_h2)
write_coord_Z = gt.write_coord_Z(z_fname,grid_h2,grid_z2)
# write bathy files (constant bathy)
bathy_fname = 'unit_tests/test_inputs/test_dst_bathy.nc'
bathy_fname = 'pynemo/unit_tests/test_inputs/test_dst_bathy.nc'
bathy = gt.write_bathy(bathy_fname,grid_h2,grid_z2)
if write_coord_H + write_coord_Z + bathy == 0:
print("Org child grid generation successful!")
......@@ -60,8 +60,8 @@ def _main():
origin = (8,8)
# rotate grid
rot_h_fname = 'unit_tests/test_inputs/test_rot_dst_hgr_zps.nc'
rot_z_fname = 'unit_tests/test_inputs/test_rot_dst_zgr_zps.nc'
rot_h_fname = 'pynemo/unit_tests/test_inputs/test_rot_dst_hgr_zps.nc'
rot_z_fname = 'pynemo/unit_tests/test_inputs/test_rot_dst_zgr_zps.nc'
grid_rot = grid_h2.copy()
grid_rot['latt'], grid_rot['lont'] = gt.rotate_around_point(grid_h2['latt'],grid_h2['lont'],theta,origin)
grid_rot['latu'], grid_rot['lonu'] = gt.rotate_around_point(grid_h2['latu'], grid_h2['lonu'], theta, origin)
......@@ -70,7 +70,7 @@ def _main():
write_coord_H = gt.write_coord_H(rot_h_fname,grid_rot)
write_coord_Z = gt.write_coord_Z(rot_z_fname,grid_rot,grid_z2)
# write bathy files (constant bathy)
bathy_fname = 'unit_tests/test_inputs/test_rot_dst_bathy.nc'
bathy_fname = 'pynemo/unit_tests/test_inputs/test_rot_dst_bathy.nc'
bathy = gt.write_bathy(bathy_fname,grid_rot,grid_z2)
if write_coord_H + write_coord_Z + bathy == 0:
print("Rotated child grid generation Successful!")
......@@ -87,14 +87,14 @@ def _main():
min_dep = 10
z_end_dim = 1
sf = 10
h_fname = 'unit_tests/test_inputs/test_offset_dst_hgr_zps.nc'
z_fname = 'unit_tests/test_inputs/test_offset_dst_zgr_zps.nc'
h_fname = 'pynemo/unit_tests/test_inputs/test_offset_dst_hgr_zps.nc'
z_fname = 'pynemo/unit_tests/test_inputs/test_offset_dst_zgr_zps.nc'
grid_h3 = gt.set_hgrid(dx,dy,jpi,jpj,zoffx,zoffy,sf)
grid_z3 = gt.set_zgrid(grid_h2,jpk,max_dep,min_dep,z_end_dim)
write_coord_H = gt.write_coord_H(h_fname,grid_h3)
write_coord_Z = gt.write_coord_Z(z_fname,grid_h3,grid_z3)
# write bathy files (constant bathy)
bathy_fname = 'unit_tests/test_inputs/test_offset_dst_bathy.nc'
bathy_fname = 'pynemo/unit_tests/test_inputs/test_offset_dst_bathy.nc'
bathy = gt.write_bathy(bathy_fname,grid_h3,grid_z3)
if write_coord_H + write_coord_Z + bathy == 0:
print("Offset child grid gneration successful!")
......@@ -104,7 +104,7 @@ def _main():
# grid_h3['lont'],grid_h1['latt'],grid_h1['lont'])
# write boundary files (constant parameters)
out_fname = 'unit_tests/test_inputs/output_boundary' #drop file extension
out_fname = 'pynemo/unit_tests/test_inputs/output_boundary' #drop file extension
params_t = {'param1': {'name':'thetao','const_value':15.0,'longname':'temperature','units':'degreesC'},
'param2': {'name':'so','const_value':35.0,'longname':'salinity','units':'PSU'},
'param3': {'name': 'zos', 'const_value': 1.0, 'longname': 'sea surface height', 'units': 'metres'}
......@@ -121,7 +121,7 @@ def _main():
print('Boundary file generation successful!')
#write_mask
mask_fname = 'unit_tests/test_inputs/mask.nc'
mask_fname = 'pynemo/unit_tests/test_inputs/mask.nc'
mask = gt.write_mask(mask_fname,grid_h1,grid_z1)
if mask == 0:
print('Mask file generation successful!')
......
......@@ -2,22 +2,22 @@
<ns0:aggregation type="union">
<ns0:netcdf>
<ns0:aggregation dimName="time" name="temperature" type="joinExisting">
<ns0:scan location="file://Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/" regExp=".*T\.nc$" />
<ns0:scan location="file://Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/" regExp=".*T\.nc$" />
</ns0:aggregation>
</ns0:netcdf>
<ns0:netcdf>
<ns0:aggregation dimName="time" name="zonal_velocity" type="joinExisting">
<ns0:scan location="file://Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/" regExp=".*U\.nc$" />
<ns0:scan location="file://Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/" regExp=".*U\.nc$" />
</ns0:aggregation>
</ns0:netcdf>
<ns0:netcdf>
<ns0:aggregation dimName="time" name="meridian_velocity" type="joinExisting">
<ns0:scan location="file://Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/" regExp=".*V\.nc$" />
<ns0:scan location="file://Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/" regExp=".*V\.nc$" />
</ns0:aggregation>
</ns0:netcdf>
<ns0:netcdf>
<ns0:aggregation dimName="time" name="sea_surface_height" type="joinExisting">
<ns0:scan location="file://Users/thopri/Projects/PyNEMO/unit_tests/test_inputs/" regExp=".*T\.nc$" />
<ns0:scan location="file://Users/thopri/Projects/PyNEMO/pynemo/unit_tests/test_inputs/" regExp=".*T\.nc$" />
</ns0:aggregation>
</ns0:netcdf>
</ns0:aggregation>
......
......@@ -38,7 +38,7 @@ setup(
keywords='Oceanography, NEMO',
packages=['pynemo','pynemo.tests','pynemo.gui','pynemo.utils','pynemo.tide','pynemo.reader'],
packages=['pynemo','pynemo.tests','pynemo.gui','pynemo.utils','pynemo.tide','pynemo.reader','pynemo.unit_tests'],
install_requires=['netCDF4>=1.1.9','scipy','numpy','matplotlib', 'basemap', 'thredds_crawler', 'seawater'],
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment