This C++ version of BAT is still being maintained, but addition of new features is unlikely. Check out our new incarnation, BAT.jl, the Bayesian analysis toolkit in Julia. In addition to Metropolis-Hastings sampling, BAT.jl supports Hamiltonian Monte Carlo (HMC) with automatic differentiation, automatic prior-based parameter space transformations, and much more. See the BAT.jl documentation.

Results of performance testing for BAT version 0.4.2

Back to | overview for 0.4.2 | all versions |

Test "1d_squared"

Results
Status good
CPU time 55.85 s
Real time 55.9 s
Plots 1d_squared.ps
Log 1d_squared.log

Settings
N chains 10
N lag 10
Convergence true
N iterations (pre-run) 1000
N iterations (run) 10000000

Plots
Auto-correlation for the parameter.Distribution from MCMC and analytic function.Distribution from MCMC and analytic function in log-scale.
Difference between the distribution from MCMC and the analytic function. The one, two and three sigma uncertainty bands are colored green, yellow and red, respectively.Pull between the distribution from MCMC and the analytic function. The Gaussian has a mean value of 0 and a standard deviation of 1 (not fitted).Summary of subtest values.

Subtest Status Target Test Uncertainty Deviation [%] Deviation [sigma] Tol. (Good) Tol. (Flawed) Tol. (Bad)
correlation par 0 off 0 0.0418 0.004149 - -10.07 0.3 0.5 0.7
chi2 good 100 115.1 14.14 15.13 -1.07 42.43 70.71 98.99
KS good 1 0.996 0.95 -0.3984 0.004194 0.95 0.99 0.9999
mean good 0 -0.001731 0.002837 - 0.6102 0.00851 0.01418 0.01986
mode good 1.442e-07 0.2 0.541 1.387e+08 -0.3697 1.623 2.705 3.787
variance good 80 81.64 8.628 2.055 -0.1906 25.88 43.14 60.39
quantile10 good -12.17 -12.18 0.2108 0.05016 0.02896 0.6325 1.054 1.476
quantile20 good -8.515 -8.517 0.2108 0.02171 0.008768 0.6325 1.054 1.476
quantile30 good -5.47 -5.471 0.2108 0.009296 0.002412 0.6325 1.054 1.476
quantile40 good -2.683 -2.684 0.2108 0.02871 0.003654 0.6325 1.054 1.476
quantile50 good 1.036e-14 -0.0008524 0.2108 -8.225e+12 0.004043 0.6325 1.054 1.476
quantile60 good 2.683 2.679 0.2108 -0.1393 0.01773 0.6325 1.054 1.476
quantile70 good 5.47 5.473 0.2108 0.04889 -0.01269 0.6325 1.054 1.476
quantile80 good 8.515 8.514 0.2108 -0.009083 0.003669 0.6325 1.054 1.476
quantile90 good 12.17 12.17 0.2108 0.004766 -0.002751 0.6325 1.054 1.476

Subtest Description
correlation par 0 Calculate the auto-correlation among the points.
chi2 Calculate χ2 and compare with prediction for dof=number of bins with an expectation >= 10.
Tolerance good: |χ2-E[χ2]| < 3 · (2 dof)1/2,
Tolerance acceptable: |χ2-E[χ2]| < 5 · (2 dof)1/2,
Tolerance bad: |χ2-E[χ2]| < 7 · (2 dof)1/2.
KS Calculate the Kolmogorov-Smirnov probability based on the ROOT implemention.
Tolerance good: KS prob > 0.05,
Tolerance acceptable: KS prob > 0.01
Tolerance bad: KS prob > 0.0001.
mean Compare sample mean, <x>, with expectation value of function, E[x].
Tolerance good: |<x> -E[x]| < 3 · (V[x]/n)1/2,
Tolerance acceptable: |<x> -E[x]| < 5 · (V[x]/n)1/2,
Tolerance bad: |<x> -E[x]| < 7 · (V[x]/n)1/2.
mode Compare mode of distribution with mode of the analytic function.
Tolerance good: |x*-mode| < 3 · V[mode]1/2,
Tolerance acceptable: |x*-mode| < 5 · V[mode]1/2 bin widths,
Tolerance bad: |x*-mode| < 7 · V[mode]1/2.
variance Compare sample variance s2 of distribution with variance of function.
Tolerance good: 3 · V[s2]1/2,
Tolerance acceptable: 5 · V[s2]1/2,
Tolerance bad: 7 · V[s2]1/2.
quantile10 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile20 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile30 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile40 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile50 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile60 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile70 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile80 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile90 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.