tddlog results
The release run of tddlog against the systems people use for each kind of problem: ProbLog, PITA and aspmc on logic programs, Ace, Merlin, pyAgrum, pgmpy, libDAI and TensorInference.jl on Bayesian and Markov networks, ProvSQL on probabilistic databases, and Dice, Alea.jl, Roulette, Geni, noDice, Psi, Genfer, SPPL, Pluck, Storm and PRISM on probabilistic programs. Each system ran single-threaded, in the same job and the same lanes of one machine as the systems it is compared with, and every answer of tddlog's is checked against a reference. Where another system is faster, the findings say so.
A plot counts the instances each system answers within a time limit, on a log scale from 0.2 s to the suite's limit: a line further up and to the left is better, and a run that takes less than 0.2 s counts at 0.2 s, since differences of a few milliseconds of start-up are not compared. A plot draws every system over all of its instances, an instance a system cannot express or was not given counted as unanswered, and its legend gives how many instances each system answers and how many it expresses. Under the plot of a family's problems, a small plot for each cluster of systems with similar expressive capabilities covers the instances all of them express, with every system that expresses all of those. The findings compare tddlog with another system on the instances both answer: where each is faster, by how much, and their total times over those instances. An instance that both finish in under 0.1 s counts neither as a win nor as a loss. The README compares by total time instead, a run that does not finish counting as the time limit. The numbers come from the same records as the README's and docs/benchmarks.md's.