This C++ version of BAT is still being maintained, but addition of new features is unlikely. Check out our new incarnation, BAT.jl, the Bayesian analysis toolkit in Julia. In addition to Metropolis-Hastings sampling, BAT.jl supports Hamiltonian Monte Carlo (HMC) with automatic differentiation, automatic prior-based parameter space transformations, and much more. See the BAT.jl documentation.

Results of performance testing for BAT version 0.4.2

Back to | overview for 0.4.2 | all versions |

Test "1d_poisson_2"

Results
Status good
CPU time 64.77 s
Real time 64.8 s
Plots 1d_poisson_2.ps
Log 1d_poisson_2.log

Settings
N chains 10
N lag 10
Convergence true
N iterations (pre-run) 1000
N iterations (run) 10000000

Plots
Auto-correlation for the parameter.Distribution from MCMC and analytic function.Distribution from MCMC and analytic function in log-scale.
Difference between the distribution from MCMC and the analytic function. The one, two and three sigma uncertainty bands are colored green, yellow and red, respectively.Pull between the distribution from MCMC and the analytic function. The Gaussian has a mean value of 0 and a standard deviation of 1 (not fitted).Summary of subtest values.

Subtest Status Target Test Uncertainty Deviation [%] Deviation [sigma] Tol. (Good) Tol. (Flawed) Tol. (Bad)
correlation par 0 off 0 0.1689 0.01682 - -10.04 0.3 0.5 0.7
chi2 good 100 101.2 14.14 1.175 -0.08306 42.43 70.71 98.99
KS good 1 0.8676 0.95 -13.24 0.1394 0.95 0.99 0.9999
mean good 2.999 3 0.0005388 0.01012 -0.5632 0.001616 0.002694 0.003771
mode good 2 2.025 0.1291 1.25 -0.1936 0.3873 0.6455 0.9037
variance good 2.993 3.053 0.5917 1.993 -0.1008 1.775 2.958 4.142
quantile10 good 1.1 1.1 0.1291 0.03725 -0.003174 0.3873 0.6455 0.9037
quantile20 good 1.534 1.535 0.1291 0.05886 -0.006996 0.3873 0.6455 0.9037
quantile30 good 1.914 1.915 0.1291 0.04951 -0.007339 0.3873 0.6455 0.9037
quantile40 good 2.285 2.286 0.1291 0.01826 -0.003232 0.3873 0.6455 0.9037
quantile50 good 2.674 2.675 0.1291 0.01166 -0.002416 0.3873 0.6455 0.9037
quantile60 good 3.106 3.106 0.1291 0.001824 -0.0004389 0.3873 0.6455 0.9037
quantile70 good 3.616 3.616 0.1291 0.002368 -0.0006633 0.3873 0.6455 0.9037
quantile80 good 4.28 4.28 0.1291 -0.001504 0.0004988 0.3873 0.6455 0.9037
quantile90 good 5.324 5.324 0.1291 0.004842 -0.001997 0.3873 0.6455 0.9037

Subtest Description
correlation par 0 Calculate the auto-correlation among the points.
chi2 Calculate χ2 and compare with prediction for dof=number of bins with an expectation >= 10.
Tolerance good: |χ2-E[χ2]| < 3 · (2 dof)1/2,
Tolerance acceptable: |χ2-E[χ2]| < 5 · (2 dof)1/2,
Tolerance bad: |χ2-E[χ2]| < 7 · (2 dof)1/2.
KS Calculate the Kolmogorov-Smirnov probability based on the ROOT implemention.
Tolerance good: KS prob > 0.05,
Tolerance acceptable: KS prob > 0.01
Tolerance bad: KS prob > 0.0001.
mean Compare sample mean, <x>, with expectation value of function, E[x].
Tolerance good: |<x> -E[x]| < 3 · (V[x]/n)1/2,
Tolerance acceptable: |<x> -E[x]| < 5 · (V[x]/n)1/2,
Tolerance bad: |<x> -E[x]| < 7 · (V[x]/n)1/2.
mode Compare mode of distribution with mode of the analytic function.
Tolerance good: |x*-mode| < 3 · V[mode]1/2,
Tolerance acceptable: |x*-mode| < 5 · V[mode]1/2 bin widths,
Tolerance bad: |x*-mode| < 7 · V[mode]1/2.
variance Compare sample variance s2 of distribution with variance of function.
Tolerance good: 3 · V[s2]1/2,
Tolerance acceptable: 5 · V[s2]1/2,
Tolerance bad: 7 · V[s2]1/2.
quantile10 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile20 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile30 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile40 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile50 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile60 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile70 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile80 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile90 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.