Tests

From cclib

Contents 1 Unit tests 2 Regression tests 3 Test data 4 Doc tests 5 Source code upload policy 6 Ensuring source code quality

Unit tests

In order to check whether our parsers extract information in the correct format, with the correct units, we have unit tests that parse a series of basic data files (see below) of the same calculation undertaken with different programs. Running testall.py in the test directory runs the whole test suite, but it is also possible to individually run the tests for GeoOpts (testGeoOpt.py), Single Point calculations, and so on.

Note that no change should be commited to the repository if it increases the number of failed tests (unless you are adding new tests, of course).

Regression tests

Regression tests ensure that bugs, once fixed, stay fixed. That is, for every bug found in our parsers, we should add a regression test and then fix the bug. This process is simplified by regression.py in the test directory.

regression.py runs through all of our test data, both the basic data and the real life log files, uses 'ccopen()' to guess its type (checks for mistakes) and open it, parses it (catches any errors), and runs any relevant regression tests (catches any failures).

New regression tests are added by creating a function testMyFileName_out following the examples at the start of regression.py.

Test data

The test directory (not included in the release version at the moment) contains all of the tests that help keep cclib working, and keep us sane. In general we use two types of data files for testing:

1. 'basic' data files are stored in folders like "basicJaguar6.4" and are b3lyp/sto-3g calculations on 1,4-divinylbenzene (dvb) with C2h symmetry. These jobs (a geometry optimisation, a single point calculation (one restricted and another unrestricted), frequency calculation, a TD-DFT calculation, and any variants of these which break the parser such as symmetry/nosymmetry) are run for each parser. These data files are stored in SVN and may be included in future releases.
2. real-life parser-breaking files are stored in folders like "Jaguar6.4". These data files are *not* stored in SVN as they are often massive but are stored on the web server and downloaded using a shell script (contained in the data directory). These files are also available as a download from the File Release page on Sourceforge (only updated every so often).

Doc tests

Doc tests are a nice Python invention for unit testing individual functions. To run the doctests in a particular file, you need to run the script. For example, "python gaussianparser.py" runs the doctests contained in gaussianparser.py. To run all of the doctests at once, you need to install a testing tool like nose, and then use the following command (note that many errors may be due to missing libraries like BioPython):

> "C:\Program Files\Python24\Scripts\nosetests.exe" cclib --with-doctest -e test* -v
ERROR
ERROR
Doctest: cclib.bridge.cclib2openbabel.makeopenbabel ... ok
ERROR
ERROR
Doctest: cclib.parser.adfparser.ADF.normalisesym ... ok
Doctest: cclib.parser.gamessparser.GAMESS.normalise_aonames ... ok
Doctest: cclib.parser.gamessparser.GAMESS.normalisesym ... ok
Doctest: cclib.parser.gamessukparser.GAMESSUK.normalisesym ... ok
Doctest: cclib.parser.gaussianparser.Gaussian.normalisesym ... ok
Doctest: cclib.parser.jaguarparser.Jaguar.normalisesym ... ok
Doctest: cclib.parser.logfileparser.Logfile.float ... ok
Doctest: cclib.parser.utils.PeriodicTable ... ok
Doctest: cclib.parser.utils.convertor ... ok
ERROR
ERROR
......

Source code upload policy

As a sort of guide for developers who are commiting source code revisions to the SVN repository, we recommend the following:

• Run the tests before commiting (at least testall.py)
• If tests that previously passed now no longer do (when we have a more complete and stable release, this will read "If any tests fail"), and you don't have time to fix things before commiting, commits your changes to a branch as follows:

svn copy https://svn.sourceforge.net/svnroot/cclib/trunk https://..../branches/brokenadfparser
       -m "Informative log message about why you're branching"
# change directory into the 'top' of your working copy where setup.py is
svn switch https://svn.sourceforge.net/svnroot/cclib/branches/brokenadfparser
# (note that this preserves the local modifications, only now these modifications
   are to the branch instead of the trunk)
svn commit # (commits the local modifications to the branch)

• As soon as the tests that previously passed are passed again, merge the changes and remove the branch (this should be within a revision or two of the branch):

svn log --stop-on-copy # (to find the revision when the branching took place)
svn switch https://...../mytrunk
svn merge --dry-run -r123:HEAD https://..../mybranch
svn merge -r123:HEAD https://..../mybranch
svn commit -m "Merging 123:HEAD of mybranch into trunk"
svn remove https://..../mybranch

Ensuring source code quality

To ensure source code quality, or at least consistency, we use the following tools/metrics:

• Cheesecake index