Why Tests?

Our main reason to have tests is to ensure we do not loose existing functionality when adding new functionality. Another reason, hopefully getting less important, is to figure out what influences bugs had, respectively what is influenced by a bugfix.

Possible Examples of Tests

The following list is not complete and likely never will. It is more to give you an idea, what could be tested:

• readers and writers for data (Generating data, writing data to file. Reading the file and writing it out again should produce the same file as before).
• calculating link travel times based on events
• events-generation, e.g. from the transport simulation
• calculation of plans' scores
• are parameters correctly reflected in the execution (e.g. when replanning?)
• ...

Framework for Tests

We currently use JUnit 3.8 for our tests. JUnit is integrated in many IDEs, e.g. in eclipse. JUnit uses three main concepts:

• Tests: the effective Tests, implemented as methods in a class. The method-name must start with "test".
• TestCases: group several tests, implemented as a class containing severeal test-methods
• TestSuites: group several testcases or other testsuites

A small example of a test and testcase:

 public class MyTests extends TestCase {
     protected void setUp() throws Exception {
         super.setUp();
         // your code here...
     }
     protected void tearDown() throws Exception {
         super.tearDown();
         // your code here...
     }
     public final void testOne() {
         // your code here...
         assertEquals(expected, actual);
     }
     public final void testTwo() {
         // your code here...
         assertNotNull(someObject);
     }
 }

This code defines two tests (testOne, testTwo). Additionally, there are two methods setUp() and tearDown(), which are automatically called by JUnit when executing the TestCase. Make sure to call Gbl.reset() in tearDown() when you use the class Gbl or Config within your tests, so that the following TestCases are not influenced by your TestCase.
JUnit offers many different assert-statements which should be used to verify the results of your tests. Do not use the standard assert() offered by Java, as this must especially be enabled to be executed!

Tests for MATSim-T

Currently, all our test-cases are stored in the directory test on our regular MATSim project which is hosted at sourceforge.
To run the tests, start JUnit with the class org.matsim.AllTests. In eclipse, select the class and then choose Run > Run As > JUnit Test.
A few tests take quiet some time to finish, we call them "expensive tests" or "extended tests". If you want to skip those tests when running the tests locally (e.g. before committing your own changes), start with the class org.matsim.SimpleTests.
The testing-project has a similar package structure as the main project. For each package, there is a TestSuite called AllTests.java. Such a TestSuite lists all TestCases in the same package as well as TestSuites of subpackages.
An example TestSuite is given here:

public class AllTests {
    public static Test suite() {
        TestSuite suite = new TestSuite("Test for org.matsim.mypackage");
        suite.addTestSuite(MyTests.class);
        suite.addTest(org.matsim.mypackage.subpackage.AllTests.suite());
        if (TestDepth.getDepth() == TestDepth.extended) {
            suite.addTestSuite(MyExpensiveTests.class); 
        }
        return suite;
    }
}

In the example above you can see how you can define if your own tests should always run ("simple test"; nothing special to do, this is the regular case) or if it is an expensive tests which should only be run when all tests, including the extended ones, should be run (query TestDepth.getDepth()).
If you write an AllTests.java for a new package, make sure to add it to the TestSuite of the parent package! Otherwise the tests will be ignored when the main AllTests or SimpleTests is started.

Guidelines for Writing Your Own Tests for MATSim

We offer a default TestCase, org.matsim.testcases.MatsimTestCase, from which you can inherit and which provides some default functionality used in most tests. Create a new class that inherits from MatsimTestCase and start writing your own tests. A good starting point to write your own tests is to check out the current tests from cvs and have a look at them.
MatsimTestCase provides the following functionality:

• calls Gbl.reset() in setUp() to give you a clean environment for your test
• creates an output-directory for your test, so your output is separated from the output of other tests.
• MatsimTestCase.getOutputDirector() gives you the path to the output directory, including the trailing slash
• a method loadConfig(String filename) with which you can load a config-file, e.g. from within your setUp() of test-method. This method overwrite the output directory of the Controler, usually specified in the config-file, with the correct path.

When you write your own tests, try to follow these guidelines as good as possible:

• your TestCase inherits from org.matsim.testcases.MatsimTestCase
• when you implement the method setUp(), the first call in the method is super.setUp()
• the output of your tests is inside the directory specified by MatsimTestCase.getOutputDirectory()
• input for your tests (e.g. config-files, special plans, ...) are inside test/input/ organized according to the package-name, class-name and name of the test. MatsimTestCase offers the method getInputDirectory() to faciliate the reading in of test-input.
• you load your config-files with loadConfig(String filename) to ensure the config knows about the correct output directory.

Automatic Run of the Tests

The tests are automatically run every night. When you log in on matsim.org, you'll find a link in the left navigation area ("developer info") where you can look at the nightly status of the tests. If errors or failures occur, an email is sent to Marcel Rieser who will then inform people as necessary.