4. Regression Testing*

Regression testing is crucial to the model development process. Regression tests in Matmodlab are special purpose problems that serve several purposes. Most notably, component tests for the core capabilities of Matmodlab and verification and validation (V&V) of material models. In the first role, problems are fast running and exercise specific features of Matmodlab in a unit-test type fashion. In the second, material models are exercised through specific paths with known, or expected outcomes. Each type of test is supported through the core.test.TestBase. New tests are created as subclasses of TestBase.

4.1. Basic Examples

Standard Material Test

from matmodlab import *

path_table_id = "path_table"

class TestPathTable(TestBase):
    def __init__(self):
        self.runid = path_table_id
        self.keywords = ["fast", "feature", "path", "table", "builtin"]

def run_path_table():

    runid = path_table_id
    logger = Logger(runid)
    path = """0E+00 0E+00 0E+00 0E+00 0E+00 0E+00 0E+00
              1E+00 1E-01 0E+00 0E+00 0E+00 0E+00 0E+00
              2E+00 0E+00 0E+00 0E+00 0E+00 0E+00 0E+00"""
    driver = Driver("Continuum", path, kappa=0.0, path_input="table",
                    step_multiplier=10.0, cfmt="222222", cols=range(7),
                    tfmt="time", logger=logger)
    parameters = {"K":1.350E+11, "G":5.300E+10}
    material = Material("elastic", parameters, logger=logger)
    mps = MaterialPointSimulator(runid, driver, material, logger=logger, d=d)

if __name__ == "__main__":

A test is created by subclassing TestBase. Minimally, a test defines runid and keywords attributes. The runid attribute is used internally for test identification and keywords for test filtering and organization. The module containing the test must also define a run_<runid> function (where <runid> is replaced with the actual runid of the test) to run the actual simulation. For each test so defined, Matmodlab expects the existence of a base file <runid>.base_exo containing the expected, or baseline, results. Matmodlab also expects, on exercising run_<runid>, the creation of the results file <runid>.exo. At the completion of the test, <runid>.exo is compared to <runid>.base_exo and differences (if any) determined by Other Command Line Tools.

4.2. Command-Line Interface

The test subcommand of mml gathers, runs, and analyzes tests. To run tests with Matmodlab, be sure that matmodlab/bin is on your path and execute:

$ mml test

mml test will create a results directory TestResults.<platform>, where <platform> is is the machine platform (as determined by Python’s sys.platform) on which the tests are being run. The following files and directories will be produced by mml test in the TestResults.<platform>, directory

$ ls
TestsResults.darwin completed_tests.db mmd/ summary.html

completed_tests.db is a database file containing information on all completed tests and summary.html is an html summary file, viewable in any web browser.

mml test searches for test specification files in the matmodlab/tests directory and directories in the tests section of the configuration file. Test files are python files whose names match (?:^|[\\b_\\.-])[Tt]est. Matmodlab supports

mml test Options

The full list of options to mml test is:

usage: mml test [-h] [-k K] [-K K] [-X] [-j J] [--no-tear-down] [--html]
                [--overlay] [-E] [-l] [--rebaseline]
                [sources [sources ...]]

mml test: run the matmodlab tests. By default, tests are found in
MML_ROOT/tests and any other directories and/or files found in the tests group
of the MATMODLABRC file (if any).

positional arguments:
  sources         [Optional] directores and/or files to find matmodlab tests.
                  The default directories will not be searched.

optional arguments:
  -h, --help      show this help message and exit
  -k K            Keywords to include [default: ]
  -K K            Keywords to exclude [default: ]
  -X              Do not stop on test initialization failure (tests that fail
                  to initialize will be skipped) [default: False]
  -j J            Number of simutaneous tests to run [default: ]
  --no-tear-down  Do not tear down passed tests on completion [default: ]
  --html          Write html summary of results (negates tear down) [default: ]
  --overlay       Create overlays of failed tests with baseline (negates tear
                  down) [default: ]
  -E              Do not use matmodlabrc configuration file [default: False]
  -l              List tests and exit [default: False]
  --rebaseline    Rebaseline test in PWD [default: False]

4.3. TestBase API

class TestBase

Instances of the TestBase represent individual tests. The class is intended to be used as a base class, with specific tests being implemented by concrete subclasses. The class implements the interface needed by mml test to allow it to drive the test, and methods that the test code can use check for and report various kinds of failure. Each instance of TestBase will run a single test.

Required Attributes of TestBase


List of keywords identifying the test. Each test must define one of long, medium, fast.


The test identifier.

Definable Attributes of TestBase


Base result file name [default: runid.base_exo]


Other Command Line Tools diff file [default: tests/base.exodiff]

Useful Read-Only Attributes of TestBase


The directory in which the test will be run

4.4. Methods

As described in Basic Examples, minimally, a test subclasses TestBase and defines a runid and keywords, Matmodlab will set up the test, run, and perform post processing. Optionally, a test may define the following methods.

TestBase.setup(*args, **kwargs)

Test setup. Minimally, setup should check for existence of needed files and create the test directory.

TestBase.pre_hook(*args, **kwargs)

Called before each test is run and after setup. The base pre_hook performs a no-op.


Run the test. Set test.status to one of FAILED_TO_RUN, FAILED, DIFFED, PASSED


Tear down the test. The standard tears down the test by removing the test directory (if test passed).

Parameters:force (int) – Force tear down even if test failed
TestBase.post_hook(*args, **kwargs)

Run after test is run. The standard post_hook performs a no-op.


Make the test directory TestBase.test_dir