This C++ version of BAT is still being maintained, but addition of new features is unlikely. Check out our new incarnation, BAT.jl, the Bayesian analysis toolkit in Julia. In addition to Metropolis-Hastings sampling, BAT.jl supports Hamiltonian Monte Carlo (HMC) with automatic differentiation, automatic prior-based parameter space transformations, and much more. See the BAT.jl documentation.

Results of performance testing for BAT version 0.4.2

Back to | overview for 0.4.2 | all versions |

Test "1d_poisson_5"

Results
Status good
CPU time 67.82 s
Real time 67.85 s
Plots 1d_poisson_5.ps
Log 1d_poisson_5.log

Settings
N chains 10
N lag 10
Convergence true
N iterations (pre-run) 1000
N iterations (run) 10000000

Plots
Auto-correlation for the parameter.Distribution from MCMC and analytic function.Distribution from MCMC and analytic function in log-scale.
Difference between the distribution from MCMC and the analytic function. The one, two and three sigma uncertainty bands are colored green, yellow and red, respectively.Pull between the distribution from MCMC and the analytic function. The Gaussian has a mean value of 0 and a standard deviation of 1 (not fitted).Summary of subtest values.

Subtest Status Target Test Uncertainty Deviation [%] Deviation [sigma] Tol. (Good) Tol. (Flawed) Tol. (Bad)
correlation par 0 off 0 0.1321 0.01311 - -10.07 0.3 0.5 0.7
chi2 good 99 106.1 14.07 7.14 -0.5024 42.21 70.36 98.5
KS good 1 1 0.95 -2.691e-06 2.833e-08 0.95 0.99 0.9999
mean good 6 6 0.0007713 0.003614 -0.2811 0.002314 0.003856 0.005399
mode good 5 5.031 0.1576 0.6231 -0.1976 0.4729 0.7881 1.103
variance good 5.996 6.116 1.037 1.999 -0.1155 3.112 5.187 7.262
quantile10 good 3.151 3.151 0.1576 0.01088 -0.002174 0.4729 0.7881 1.103
quantile20 good 3.902 3.902 0.1576 0.006648 -0.001646 0.4729 0.7881 1.103
quantile30 good 4.517 4.517 0.1576 0.01289 -0.003694 0.4729 0.7881 1.103
quantile40 good 5.091 5.092 0.1576 0.01005 -0.003248 0.4729 0.7881 1.103
quantile50 good 5.671 5.671 0.1576 0.007496 -0.002697 0.4729 0.7881 1.103
quantile60 good 6.293 6.293 0.1576 0.0004349 -0.0001736 0.4729 0.7881 1.103
quantile70 good 7.007 7.007 0.1576 -0.005175 0.0023 0.4729 0.7881 1.103
quantile80 good 7.908 7.907 0.1576 -0.0106 0.005316 0.4729 0.7881 1.103
quantile90 good 9.277 9.278 0.1576 0.002756 -0.001622 0.4729 0.7881 1.103

Subtest Description
correlation par 0 Calculate the auto-correlation among the points.
chi2 Calculate χ2 and compare with prediction for dof=number of bins with an expectation >= 10.
Tolerance good: |χ2-E[χ2]| < 3 · (2 dof)1/2,
Tolerance acceptable: |χ2-E[χ2]| < 5 · (2 dof)1/2,
Tolerance bad: |χ2-E[χ2]| < 7 · (2 dof)1/2.
KS Calculate the Kolmogorov-Smirnov probability based on the ROOT implemention.
Tolerance good: KS prob > 0.05,
Tolerance acceptable: KS prob > 0.01
Tolerance bad: KS prob > 0.0001.
mean Compare sample mean, <x>, with expectation value of function, E[x].
Tolerance good: |<x> -E[x]| < 3 · (V[x]/n)1/2,
Tolerance acceptable: |<x> -E[x]| < 5 · (V[x]/n)1/2,
Tolerance bad: |<x> -E[x]| < 7 · (V[x]/n)1/2.
mode Compare mode of distribution with mode of the analytic function.
Tolerance good: |x*-mode| < 3 · V[mode]1/2,
Tolerance acceptable: |x*-mode| < 5 · V[mode]1/2 bin widths,
Tolerance bad: |x*-mode| < 7 · V[mode]1/2.
variance Compare sample variance s2 of distribution with variance of function.
Tolerance good: 3 · V[s2]1/2,
Tolerance acceptable: 5 · V[s2]1/2,
Tolerance bad: 7 · V[s2]1/2.
quantile10 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile20 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile30 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile40 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile50 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile60 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile70 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile80 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile90 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.