This C++ version of BAT is still being maintained, but addition of new features is unlikely. Check out our new incarnation, BAT.jl, the Bayesian analysis toolkit in Julia. In addition to Metropolis-Hastings sampling, BAT.jl supports Hamiltonian Monte Carlo (HMC) with automatic differentiation, automatic prior-based parameter space transformations, and much more. See the BAT.jl documentation.

Results of performance testing for BAT version 0.9

Back to | overview for 0.9 | all versions |

Test "1d_poisson_9"

Results
Status good
CPU time 21.29 s
Real time 21.53 s
Plots 1d_poisson_9.ps
Log 1d_poisson_9.log

Settings
N chains 10
N lag 10
Convergence true
N iterations (pre-run) 1000
N iterations (run) 10000000

Plots
Auto-correlation for the parameter.Distribution from MCMC and analytic function.Distribution from MCMC and analytic function in log-scale.
Difference between the distribution from MCMC and the analytic function. The one, two and three sigma uncertainty bands are colored green, yellow and red, respectively.Pull between the distribution from MCMC and the analytic function. The Gaussian has a mean value of 0 and a standard deviation of 1 (not fitted).Summary of subtest values.

Subtest Status Target Test Uncertainty Deviation [%] Deviation [sigma] Tol. (Good) Tol. (Acceptable) Tol. (Bad)
correlation par 0 off 0 0.1235 0.01227 - -10.07 0.3 0.5 0.7
chi2 good 96 85.58 13.86 -10.86 0.7523 41.57 69.28 96.99
KS good 1 0.9998 0.95 -0.02463 0.0002592 0.95 0.99 0.9999
mean good 10 10 0.00101 0.005717 -0.5659 0.00303 0.00505 0.007071
mode good 9 9.15 0.1826 1.667 -0.8216 0.5477 0.9129 1.278
variance good 9.997 10.2 1.614 2.041 -0.1264 4.843 8.071 11.3
quantile10 good 6.217 6.218 0.1826 0.01561 -0.005316 0.5477 0.9129 1.278
quantile20 good 7.287 7.287 0.1826 0.0007739 -0.0003089 0.5477 0.9129 1.278
quantile30 good 8.132 8.132 0.1826 -0.001312 0.0005846 0.5477 0.9129 1.278
quantile40 good 8.904 8.905 0.1826 0.008808 -0.004296 0.5477 0.9129 1.278
quantile50 good 9.669 9.669 0.1826 -0.00264 0.001398 0.5477 0.9129 1.278
quantile60 good 10.48 10.48 0.1826 0.008023 -0.004604 0.5477 0.9129 1.278
quantile70 good 11.39 11.39 0.1826 0.008536 -0.005324 0.5477 0.9129 1.278
quantile80 good 12.52 12.52 0.1826 0.005876 -0.00403 0.5477 0.9129 1.278
quantile90 good 14.21 14.21 0.1826 0.006451 -0.005021 0.5477 0.9129 1.278

Subtest Description
correlation par 0 Calculate the auto-correlation among the points.
chi2 Calculate χ2 and compare with prediction for dof=number of bins with an expectation >= 10.
Tolerance good: |χ2-E[χ2]| < 3 · (2 dof)1/2,
Tolerance acceptable: |χ2-E[χ2]| < 5 · (2 dof)1/2,
Tolerance bad: |χ2-E[χ2]| < 7 · (2 dof)1/2.
KS Calculate the Kolmogorov-Smirnov probability based on the ROOT implemention.
Tolerance good: KS prob > 0.05,
Tolerance acceptable: KS prob > 0.01
Tolerance bad: KS prob > 0.0001.
mean Compare sample mean, <x>, with expectation value of function, E[x].
Tolerance good: |<x> -E[x]| < 3 · (V[x]/n)1/2,
Tolerance acceptable: |<x> -E[x]| < 5 · (V[x]/n)1/2,
Tolerance bad: |<x> -E[x]| < 7 · (V[x]/n)1/2.
mode Compare mode of distribution with mode of the analytic function.
Tolerance good: |x*-mode| < 3 · V[mode]1/2,
Tolerance acceptable: |x*-mode| < 5 · V[mode]1/2 bin widths,
Tolerance bad: |x*-mode| < 7 · V[mode]1/2.
variance Compare sample variance s2 of distribution with variance of function.
Tolerance good: 3 · V[s2]1/2,
Tolerance acceptable: 5 · V[s2]1/2,
Tolerance bad: 7 · V[s2]1/2.
quantile10 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile20 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile30 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile40 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile50 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile60 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile70 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile80 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.
quantile90 Compare quantile of distribution from MCMC with the quantile of analytic function.
Tolerance good: |q_{X}-E[q_{X}]|<3·V[q]1/2,
Tolerance acceptable: |q_{X}-E[q_{X}]|<5·V[q]1/2,
Tolerance bad: |q_{X}-E[q_{X}]|<7·V[q]1/2.