Probabilistic Circuits

Bayesian Model Combination.

Chow-Liu Tree

Weighted ensemble of probabilistic circuits.

A BitCircuit with parameters attached to the elements

A probabilistic conjunction node (multiplication node)

Root of the plain probabilistic circuit node hierarchy

A probabilistic inner node

A probabilistic leaf node

A probabilistic literal node

A probabilistic disjunction node (summation node)

Root of the probabilistic circuit node hierarchy

Evaluate a probabilistic circuit as a function

Root of region graph node hierarchy

Options for SamplePSDD.

A shared probabilistic multiplcation node

Root of the shared probabilistic circuit node hierarchy

A shared probabilistic inner node

A shared probabilistic leaf node

A shared probabilistic literal node

A shared probabilistic summation node

A plain structured probabilistic conjunction node

Root of the plain structure probabilistic circuit node hierarchy

A plain structured probabilistic inner node

A plain structured probabilistic leaf node

A plain structured probabilistic literal leaf node, representing the positive or negative literal of its variable

A plain structured probabilistic disjunction node

Base.read(file::AbstractString, ::Type{C}) where C <: ProbCircuit

Reads circuit from file; uses extension to detect format type, for example ".psdd" for PSDDs.

Base.write(file::AbstractString, circuit::ProbCircuit)

Writes circuit to file; uses file name extention to detect file format.

Base.write(files::Tuple{AbstractString,AbstractString}, circuit::StructProbCircuit)

Saves circuit and vtree to file.

Print edges and vertices of a ChowLiu tree

Get the vtree corresponding to the argument, or nothing if the node has no vtree

EVI(pc, data)

Computes the log likelihood data given full evidence. Outputs $\log{p(x)}$ for each datapoint.

Maximum a-posteriori queries

MAR(pc, data) = marginal

Computes Marginal log likelhood of data. MAR is juat an alias for marginal. See docs for marginal for more details.

Compute the Minimum Spanning Tree (MST) of graph g with weights weights, with constraints such that included_edges should be included while excluded_edges should be excluded.

apply_entropy_reg_cpu(bc::BitCircuit; log_params::Vector{Float64}, edge_counts::Vector{Float64}, total_data_counts::Float64, pseudocount::Float64 = 0.1, entropy_reg::Float64 = 0.0)::Vector{Float64}

Add entropy regularization to a deterministic (see LogicCircuits.isdeterministic) probabilistic circuit. alpha is a hyperparameter that balances the weights between the likelihood and the entropy: rgmax{ heta} L( heta) = llmean( heta) + alpha * entropy( heta).

apply_entropy_reg_gpu(bc::BitCircuit; log_params, edge_counts, total_data_counts::Float64, pseudocount::Float64 = 0.1, entropy_reg::Float64 = 0.0)

Add entropy regularization to a deterministic (see LogicCircuits.isdeterministic) probabilistic circuit. alpha is a hyperparameter that balances the weights between the likelihood and the entropy: rgmax{ heta} L( heta) = llmean( heta) + alpha * entropy( heta).

Makes sure the sum nodes does not have too many children. Makes balanced sums of sums to reduce children count.

Makes sure input nodes don't have too many parents. Makes a dummy sum node for each input per partition. Then nodes corresponding to the partition use the dummy node as their children instead of the input node. This way instead of numnodesroot * numnodesleaf, we would have numnodesroot parents nodes.

Returns an increasing sorted right-linear vtree.

Constructs a SamplePSDD BMC with q*t combinations, each with n models.

Return a vertex sequence to traverse the tree `g', where children are accessed before parent.

Compute the Chow-Liu Tree given a binary dataset. Automatically convert to categorical dataset if specified by num_vars' andnumcats'. If `numtrees` is greater than 1, the algorithm returns the top-K maximum spanning trees with respect to the pairwise_MI weights. Reference: Listing all the minimum spanning trees in an undirected graph http://www.nda.ac.jp/~yamada/paper/enum-mst.pdf

Construct literal nodes given variable var

Construct decision nodes given primes and subs

Construct decision nodes conditiond on different distribution

Compile a psdd circuit from clt and vtree

Construct true nodes given variable var

Pick the edge with maximum flow

Pick the edge randomly

Creates an ensemble of n SamplePSDD-generated probabilistic circuits, with v the total number of variables in the data, ϕ the logic constraints, D the data to be learned and k the maximum number of primes to be sampled.

Keyword arguments for sample_psdd are passed down. Optionally, the function takes keyword argument vtree_bias, which samples more left (value closer to 0.0) or right-leaning (value closer to 1.0) vtrees. If a negative value is given, sample uniformly distributed vtrees.

Weights are computed by the given strategy. These can be any one of the following: 1. :likelihood for likelihood weighting; 2. :uniform for uniform weights; 3. :em for Expectation-Maximization; 4. :stacking for mixture model Stacking;

estimate_parameters!(pc::ProbCircuit, data; pseudocount::Float64, entropy_reg::Float64 = 0.0)

Maximum likelihood estimation of a ProbCircuit's parameters given data

estimate_parameters!(spc::SharedProbCircuit, data; pseudocount::Float64, entropy_reg::Float64 = 0.0)

Maximum likelihood estimation of a SharedProbCircuit's parameters given data.

bagging support: If spc is a SharedProbCircuit and data is an array of DataFrames with the same number of "components", learn each circuit with its corresponding dataset.

estimate_parameters_cpu(bc::BitCircuit, data, pseudocount; weights = nothing, entropy_reg::Float64 = 0.0)

estimate_parameters_em!(pc::ProbCircuit, data; pseudocount::Float64, entropy_reg::Float64 = 0.0, exp_update_factor::Float64 = 0.0, update_per_batch::Bool = false)

One epoch of Expectation maximization (EM) parameter learning for circuits. Useful when having missing data or non-deterministic circuits.

• entropy_reg: Entropy Regularization
• update_per_batch: Whether to update circuit paramters per batch (using the ParamBitCircuit's paramters)

estimate_parameters_em_multi_epochs!(circuit::ProbCircuit, train_data)

Runs multiple epochs of EM for circuit. It will run on CPU/GPU based on where train_data is.

Arguments:

• circuit:
• train_data: training data. If want gpu, move to gpu before calling this to_gpu(train_data).

Keyword arguments:

• valid_data=nothing: validation data, should be same device as train_data
• test_data=nothing: test data, data, should be same device as train_data
• entropy_reg::Float64 = 0.0: Entropy regularization
• exp_update_factor_start::Float64 = 0.1:
• exp_update_factor_end::Float64 = 0.9:
• em_warmup_iters=100 : number of EM Warm up iterations
• em_finetune_iters=100: number of EM fine tune iterations
• verbose=false: verbose or not
• verbose_log_rate=1: how often to print info
• save_path=nothing: path of the file to save the partially trained circuit
• save_rate=20: how often to save the circuit to file

estimate_parameters_em_per_batch!(pc::ProbCircuit, data; pseudocount::Float64, entropy_reg::Float64 = 0.0, exp_update_factor = 0.0)

One epoch of Expectation maximization (EM) parameter learning for circuits. Useful when having missing data or non-deterministic circuits.

• entropy_reg: Entropy Regularization
• update_per_batch: Whether to update circuit paramters per batch (using the ParamBitCircuit's paramters)

estimate_parameters_gpu(bc::BitCircuit, data, pseudocount; weights = nothing, entropy_reg::Float64 = 0.0)

estimate_parameters_sgd!(pc::ProbCircuit, data; lr::Float64 = 0.01, reuse_values = nothing, reuse_flows = nothing, reuse = (nothing, nothing))

One epoch SGD to learn paramters of a ParamBitCircuit. It will run on CPU/GPU depending on where the data is located.

• lr: Learning Rate
• pbc: The ParamBitCircuit
• 'data': training data
• 'resue_*`: Allows reusing of allocated memory to avoid extra memory allocation between epochs. If training for only one epoch don't need to specify them.

estimate_parameters_sgd!(pc::ProbCircuit, data; lr::Float64 = 0.01, reuse_values = nothing, reuse_flows = nothing, reuse = (nothing, nothing))

One epoch SGD to learn paramters of the pc. It will run on CPU/GPU depending on where the data is located.

• lr: Learning Rate
• pc: The ProbCircuit
• 'data': training data
• 'resue_*`: Allows reusing of allocated memory to avoid extra memory allocation between epochs. If training for only one epoch don't need to specify them.

estimate_single_circuit_parameters!(pc::ProbCircuit, data; pseudocount::Float64, component_idx::Integer = 0, entropy_reg::Float64 = 0.0)

Maximum likelihood estimation of a single circuit (for example, ProbCircuit or one component of SharedProbCircuit) parameters given data

Returns a fully factorized circuit reusing leaf nodes from L.

Returns a structured probabilistic circuit compiled from a binary decision diagram.

Calculate CPT of child conditioned on parent from dis_cache

init_marginal(data, reuse, num_nodes; Float=Float32)

Initialize values from the data (data frames)

init_marginal_cpu(data, reuse, num_nodes; Float)

Initialize values from the cpu data (data frames)

init_marginal_gpu(data, reuse, num_nodes; Float=Float32)

Initialize values from the gpu data

Is the node a multiplication?

Is the node a summation?

Calculate KL divergence calculation for pcs that are not necessarily identical

Learns a structured probabilistic circuit consistent with a binary decision diagram ϕ.

learn a Chow-Liu tree from training set train_x, with Laplace smoothing factor α, specifying the tree root by clt_root return a CLT

Learning from data a structured-decomposable circuit with several structure learning algorithms

Learn structure of a single structured decomposable circuit

Learn structure of a single structured decomposable circuit from missing data.

Missing feature are denoted by missing. Median Imputation is used by default for initial structure (set impute_method for other options).

Given a circuit, learns a mixture of structure decomposable circuits based on that circuit

learn_circuit_mixture(pc, data; num_mix=5, pseudocount=1.0, maxiter=20)

Learns the weights of the Ensemble by Expectation-Maximization.

Learns the weights of the Ensemble by the likelihood value of data D.

Learns the weights of the Ensemble by Stacking, with k as the number of folds in k-fold.

learn_strudel(train_x; num_mix = 5, init_maxiter = 10, em_maxiter=20)

Learn a mixture of circuits See "Strudel: Learning Structured-Decomposable Probabilistic Circuits. arxiv.org/abs/2007.09331

Learn a vtree from clt, with strategy (close to) linear or balanced

Loads an ensemble from disk.

log_likelihood(pc, data)

Compute the likelihood of the PC given the data

log_likelihood_avg(pc, data)

Compute the likelihood of the PC given the data, averaged over all instances in the data

Compute the likelihood of the PC given each individual instance in the data

Find the MAP child value and node id of a given decision node

map_prob(root::ProbCircuit, data::Union{Real,Missing}...)
map_prob(root::ProbCircuit, data::Union{Vector{Union{Bool,Missing}},CuVector{UInt8}})
map_prob(circuit::ProbCircuit, data::DataFrame)
map_prob(circuit::ParamBitCircuit, data::DataFrame)

The most likely world probability that agrees with the provided data"

Missing values should be denoted by missing in the data.

The most likely world that agrees with the provided data for all nodes

Compute marginals on the GPU

Compute MAP probabilities on the CPU (SIMD & multi-threaded)

CUDA kernel for circuit evaluation

marginal(circuit::SharedProbCircuit, data::DataFrame, weights::Union{AbstractArray, Nothing} = nothing; component_idx = 0)::AbstractVector

Computes marginals for one circuit in a SharedProbCircuit.

• component_idx: index of the SharedProbCircuit component to comptue marginals on.

marginal(root::ProbCircuit, data::Union{Real,Missing}...)
marginal(root::ProbCircuit, data::Union{Vector{Union{Bool,Missing}},CuVector{UInt8}})
marginal(circuit::ProbCircuit, data::DataFrame)
marginal(circuit::ParamBitCircuit, data::DataFrame)::AbstractVector
marginal(circuit::SharedProbCircuit, data::DataFrame, weights::Union{AbstractArray, Nothing}; component_idx)

Evaluate marginals of the circuit bottom-up for given input(s).

Missing values should be denoted by missing in the data.

Outputs $\log{p(x^o)}$ for each data point.

Evaluate the probabilistic circuit bottom-up for a given input and return the marginal probability value of all nodes

Compute the marginal and flow of each node

When marginals of nodes have already been computed, do a downward pass computing the marginal flows at each node

Pass marginal flows down the layers of a bit circuit on the GPU

Evaluate marginals of the layers of a parameter bit circuit on the CPU (SIMD & multi-threaded)

CUDA kernel for passing marginal flows down circuit

Compute marginals on the GPU

Compute marginals on the CPU (SIMD & multi-threaded)

CUDA kernel for circuit evaluation

marginal_log_likelihood(pc, data)
marginal_log_likelihood(pc, data, weights::DataFrame)
marginal_log_likelihood(pc, data, weights::AbstractArray)
marginal_log_likelihood(pc, data::Vector{DataFrame})

Compute the marginal likelihood of the PC given the data

marginal_log_likelihood_avg(pc, data)

Compute the marginal likelihood of the PC given the data, averaged over all instances in the data. If the data is weighted then does a weighted average.

max_a_posteriori(root::ProbCircuit, data::Union{Bool,Missing}...)
max_a_posteriori(root::ProbCircuit, data::Union{Vector{<:Union{Bool,Missing}},CuVector{UInt8}})
max_a_posteriori(circuit::ProbCircuit, data::DataFrame)
max_a_posteriori(pbc::ParamBitCircuit, data; Float=Float32)

Evaluate maximum a-posteriori state of the circuit for given input(s).

Outputs the states, and the corresponding probabilities (in log domain).

Metis top down method

Mode of the distribution

Get the list of multiplication nodes in a given circuit

Multiply nodes into a single circuit

How many components are mixed together in this shared circuit?

Count the number of parameters in the circuit

Count the number of parameters in the node

Compute pairwise Mutual Information given binary/categorical data.

Get the parameters associated with a sum node

Get parent vector of a tree

Parse a clt from given file

Calculate the probability of the logic formula given by LC for the PC

random_region_graph(X::AbstractVector{Var}, depth::Int = 5, replicas::Int = 2, num_splits::Int = 2)

• X: Vector of all variables to include; for the root region
• depth: how many layers to do splits
• replicas: number of replicas or paritions (replicas only used for the root region; for other regions only 1 parition (inner nodes), or 0 parition for leaves)
• num_splits: number of splits for each parition; split variables into random equaly sized regions

region_graph_2_pc(node::RegionGraph; num_nodes_root, num_nodes_region, num_nodes_leaf, balance_childs_parents)

• num_nodes_root: number of sum nodes in the root region
• num_nodes_leaf: number of sum nodes per leaf region
• num_nodes_region: number of in each region except root and leaves

sample(pc::ProbCircuit, num_samples)
sample(pc::ProbCircuit, num_samples, evidences)

Sample states from the probabilistic circuit distribution. Also can do conditional sampling if evidence is given (any subset of features).

Samples a single combination uniformly following Robert Floyd's algorithm.

Samples primes for the ⊤ case.

Samples a partial partition.

Samples primes for partial partition.

Samples a PSDD from a BDD ϕ and vtree V with at most k elements in each disjunction node.

Samples an element from a Binomial distribution with p=0.5.

Save file as a .esbl ensemble file format.

Get the list of summation nodes in a given circuit

Sum nodes into a single circuit

Convert an array of samples into a vector of dataframes

uniform_parameters!(pc::ProbCircuit; perturbation::Float64 = 0.0)

Assign parameters of ProbCircuit so that we have a Uniform distribution.

function update_pc_params_from_pbc!(pc::ProbCircuit, bc, params; exp_update_factor = 0.0)

During parameter learning pc's paramters are not updated automatically, and only the corresponding ParamBitCircuit's paramters update. This method updates the paramters of pc using the corresponding ParamBitCircuit.

Note: This is mostly for internal use.

update_pc_params_from_pbc!(pc::ProbCircuit, pbc; exp_update_factor = 0.0)

During parameter learning pc's paramters are not updated automatically, and only the corresponding ParamBitCircuit's paramters update. This method updates the paramters of pc using the corresponding ParamBitCircuit.

Note: This is mostly for internal use.

update_pc_params_from_pbc!(pc::SharedProbCircuit, bc, params, component_idx; exp_update_factor = 0.0)

During parameter learning pc's paramters are not updated automatically, and only the corresponding BitCircuit's paramters update. This method updates the paramters of pc using the corresponding BitCircuit.

Note: This is mostly for internal use.

Pick the variable with maximum sum of mutual information

Pick the variable randomly

wmc_chavira(root::LogicCircuit; varprob::Function)::Float

Compute weighted model count in the context of a chavira encoding (default negative literals to 1) Probability of each variable is given by varprob Function which defauls to 1/2 for every variable.

zoo_jpc(name)

Loads JPC file with given name from model zoo. See https://github.com/UCLA-StarAI/Circuit-Model-Zoo.

zoo_psdd(name)

Loads PSDD file with given name from model zoo. See https://github.com/UCLA-StarAI/Circuit-Model-Zoo.

zoo_spn(name)

Loads SPN file with given name from model zoo. See https://github.com/UCLA-StarAI/Circuit-Model-Zoo.