## Performance testing

BAT comes with a test suite which performs benchmark and performance tests. The test suite is run for each release and the results can be found here.

The test suite comprises of several types of tests which each come
with a set of sub-tests. So far, only tests for the MCMC are
implemented. Each test returns a status which is one of the following:

good | the test is successful |

acceptable | the test is successful but at least one sub-test result
deviates from the expectation to a level which is still
acceptable |

bad | the test is not successful, at least one sub-test result
deviates from the expectation to a level which is not
acceptable |

fatal | the test is not successful, something is seriously wrong |

unknown | the outcome of the test is not known |

off | test is not performed |

Each test first summarizes the settings used and then lists the sub-tests with the results and allowed deviations (tolerances). The tests implemented so far are:

Function1D

An analytically known 1-dimensional function is sampled
from and the sampled (marginalized) distribution is compared to the
function. Several quantities are calculated for the distribution and
the function and compared. These are:

- the auto-correlation for each parameter,
- the chi2-value,
- the Kolmovorov-Smirnov probability,
- the mean value,
- the mode,
- the variance,
- the 10% to 90% quantiles.

Function2D

An analytically known 2-dimensional function is sampled
from and the sampled (marginalized) distribution is compared to the
function. Several quantities are calculated for the distribution and
the function and compared. These are:

- the auto-correlation for each parameter,
- the chi2-value.

Varying parameters

An analytically known 1-dimensional function
is sampled from and the sampled (marginalized) distribution is
compared to the function. Several quantities are calculated for the
distribution and the function and compared. One parameter of the MCMC,
e.g., the lag or the number of iterations, is varied.