Tests

Unit tests
In order to check whether our parsers extract information in the correct format, with the correct units, we have unit tests that parse a series of basic data files (see below) of the same calculation undertaken with different programs. Running testall.py in the test directory runs the whole test suite, but it is also possible to individually run the tests for GeoOpts (testGeoOpt.py), Single Point calculations, and so on.

Note that no change should be commited to the repository if it increases the number of failed tests (unless you are adding new tests, of course).

Regression tests
Regression tests ensure that bugs, once fixed, stay fixed. That is, for every bug found in our parsers, we should add a regression test and then fix the bug. This process is simplified by regression.py in the test directory.

regression.py runs through all of our test data, both the basic data and the real life log files, uses 'ccopen' to guess its type (checks for mistakes) and open it, parses it (catches any errors), and runs any relevant regression tests (catches any failures).

New regression tests are added by creating a function testMyFileName_out following the examples at the start of regression.py.

Test data
The test directory (not included in the release version at the moment) contains all of the tests that help keep cclib working, and keep us sane. In general we use two types of data files for testing:
 * 1) 'basic' data files are stored in folders like "basicJaguar6.4" and are b3lyp/sto-3g calculations on 1,4-divinylbenzene (dvb) with C2h symmetry. These jobs (a geometry optimisation, a single point calculation (one restricted and another unrestricted), frequency calculation, a TD-DFT calculation, and any variants of these which break the parser such as symmetry/nosymmetry) are run for each parser. These data files are stored in SVN and may be included in future releases.
 * 2) real-life parser-breaking files are stored in folders like "Jaguar6.4". These data files are *not* stored in SVN as they are often massive but are stored on the web server and downloaded using a shell script (contained in the data directory). These files are also available as a download from the File Release page on Sourceforge (only updated every so often).

Doc tests
Doc tests are a nice Python invention for unit testing individual functions. To run the doctests in a particular file, you need to run the script. For example, "python gaussianparser.py" runs the doctests contained in gaussianparser.py. To run all of the doctests at once, you need to install a testing tool like nose, and then use the following command (note that many errors may be due to missing libraries like BioPython): > "C:\Program Files\Python24\Scripts\nosetests.exe" cclib --with-doctest -e test* -v ERROR ERROR Doctest: cclib.bridge.cclib2openbabel.makeopenbabel ... ok ERROR ERROR Doctest: cclib.parser.adfparser.ADF.normalisesym ... ok Doctest: cclib.parser.gamessparser.GAMESS.normalise_aonames ... ok Doctest: cclib.parser.gamessparser.GAMESS.normalisesym ... ok Doctest: cclib.parser.gamessukparser.GAMESSUK.normalisesym ... ok Doctest: cclib.parser.gaussianparser.Gaussian.normalisesym ... ok Doctest: cclib.parser.jaguarparser.Jaguar.normalisesym ... ok Doctest: cclib.parser.logfileparser.Logfile.float ... ok Doctest: cclib.parser.utils.PeriodicTable ... ok Doctest: cclib.parser.utils.convertor ... ok ERROR ERROR ......

Source code upload policy
As a sort of guide for developers who are commiting source code revisions to the SVN repository, we recommend the following:

svn copy https://svn.sourceforge.net/svnroot/cclib/trunk https://..../branches/brokenadfparser -m "Informative log message about why you're branching" svn switch https://svn.sourceforge.net/svnroot/cclib/branches/brokenadfparser are to the branch instead of the trunk) svn commit # (commits the local modifications to the branch)
 * Run the tests before commiting (at least testall.py)
 * If tests that previously passed now no longer do (when we have a more complete and stable release, this will read "If any tests fail"), and you don't have time to fix things before commiting, commits your changes to a branch as follows:
 * 1) change directory into the 'top' of your working copy where setup.py is
 * 1) (note that this preserves the local modifications, only now these modifications

svn log --stop-on-copy # (to find the revision when the branching took place) svn switch https://...../mytrunk svn merge --dry-run -r123:HEAD https://..../mybranch svn merge -r123:HEAD https://..../mybranch svn commit -m "Merging 123:HEAD of mybranch into trunk" svn remove https://..../mybranch
 * As soon as the tests that previously passed are passed again, merge the changes and remove the branch (this should be within a revision or two of the branch):

Ensuring source code quality
To ensure source code quality, or at least consistency, we use the following tools/metrics:
 * Cheesecake index