A Dynamic Approach to MPE and Weighted MAX-SAT
Tian Sang1, Paul Beame1, and Henry Kautz2
1Department of Computer Science and Engineering
University of Washington, Seattle, WA 98195
2Department of Computer Science
University of Rochester, Rochester, NY 14627
The problem of Most Probable Explanation (MPE)
arises in the scenario of probabilistic inference:
finding an assignment to all variables that has the
maximum likelihood given some evidence.
consider the more general CNF-based MPE prob-
lem, where each literal in a CNF-formula is asso-
ciated with a weight. We describe reductions be-
tween MPE and weighted MAX-SAT, and show
that both can be solved by a variant of weighted
model counting. The MPE-SAT algorithm is quite
competitive with the state-of-the-art MAX-SAT,
WCSP, and MPE solvers on a variety of problems.
Constraint Satisfaction Problems(CSP) havebeen the subject
of intensive study; many real-world domains can be formal-
ized by CSP models and solved by either complete or incom-
plete reasoning methods. Beyond classic CSP, where a solu-
tion must satisfy all hard constraints, some CSP models are
capable of handling both hard and soft constraints. The def-
inition of constraints and the measurement of the quality of
a solution vary from model to model, and the goal is usually
to find a best solution to the constraints, rather than simply
a solution. For example, the constraints can be associated
with probability, cost, utility, or weight; the goal can be to
minimize costs of violated constraints, or to maximize the
likelihood of a variable assignment, etc. However, all fit in
a general framework for soft constraints, namely, semi-ring
based CSPs [Bistarelli et al., 1997]. In this paper, we focus
on two specific models: MPE and weighted MAX-SAT.
MAX-SAT extends SAT to the problem of finding an as-
signment that maximizes the number of satisfied clauses, in
case the formula is unsatisfiable. Weighted MAX-SAT ex-
tends MAX-SAT by adding a weight to each clause, with
the goal of finding an assignment that maximizes the sum
of weights of satisfied clauses.
MAX-SAT problems are solved by either incomplete local
tive search.For most MAX-SAT problems, local search
methodsexhibitbetterspeed andscaling thancompletemeth-
ods. Local search methods, unfortunately, do not provide a
proof that the returned solution is optimal.
MAX-SAT and weighted
The success of modern complete SAT solvers has inspired
SAT algorithms. Recent developments include using unit
propagation for strong bound computation [Li et al., 2005;
2006]; adapting local consistency methods developed for
CSP to MAX-SAT [de Givry et al., 2003]; and using fast
bounds[Aloul et al., 2002].
In probabilistic reasoning, the problem of Most Probable
Explanation (MPE) is to find an assignment to all variables
that has the maximum likelihood given the evidence. Exact
methods for MPE on probability distributions represented by
Bayesian networks include well-known methods such as the
Join Tree algorithm [Jensen et al., 1990], as well as a recent
branch-and-bound algorithm, AND/OR Tree Search [Mari-
nescu and Dechter, 2005]. Since solving MPE exactly is NP-
Hard, local search algorithms have been introduced for ap-
proximation [Park, 2002]. In this paper, we consider MPE
on CNF formulas with weighted literals, where the goal is to
find a solution with maximum product or sum of its literal
weights. This CNF-based MPE problem is strictly more gen-
eral than MPE for discrete Bayesian networks, because any
discrete Bayesian network can be converted to a CNF with
weightedliterals whosesize is linear inthe size ofconditional
probabilitytables (CPTs) of the network[Sang et al., 2005b].
MPE on CNF can be viewed as a special case of Weighted
Model Counting (WMC) [Sang et al., 2005b], and is likely
easier than WMC because we may apply branch-and-bound
based pruning techniques to reduce the search space. We
choose Cachet [Sang et al., 2004], a state-of-the-art model
counting system, as a platform on which to build our
MPE solver, and we extend pruning techniques for sub-
problems with components. As a result, we present MPE-
SAT, a decomposition-based branch-and-bound algorithm
that works on top of WMC and prunes the search space ef-
fectively. Furthermore, we are able to use the dtree algorithm
of[Huang and Darwiche, 2003; Darwiche, 2002]to boost the
performance on problems with good decomposability.
In general, MPE and weighted MAX-SAT illustrate two
complementary ways of representing soft constraints: either
having weight on variables or having weight on constraints
(clauses). Although they have apparently different represen-
tations of weight and goals, they can be converted to each
other by adding auxiliary variables or clauses, possibly at
techniques than UP, has serious difficulties on many prob-
lems. MPE-SAT is efficient because it takes advantage of
decomposition: the problems often decompose after a few
instantiations when dtree heuristic is used. Toolbar fails on
most problems, while PB2 can solve problems with small op-
timal values (≤ 4) but fails on most problems with large op-
timal values (≥ 5). We guess that in general pseudo-Boolean
solvers become very inefficient for MAX-SAT when the op-
timal value is reasonably large, where it is hard to prove both
SAT and UNSAT.
MPE and weighted MAX-SAT are complementary represen-
tations for problems with soft constraints and we have de-
scribed natural reductions between them. To solve these op-
timization tasks efficiently, we have presented an algorithm
MPE-SAT which incorporates various techniques with a dy-
namic bounding, caching (a form of dynamic programming),
and clause-learning. As a result, our approach is quite com-
petitive with other solvers on a wide range of problem do-
mains from MPE and MAX-SAT and significantly outper-
forms each on at least one of the domains.
[Aloul et al., 2002] F.
Applications of Satisfiability Testing, 2002.
Aloul, A. Ramani,I.Markov, and
PBS: A backtrack search pseudo-boolean
In Proc. 5th International Conference on Theory and
[Bacchus et al., 2003] F. Bacchus, S. Dalmao, and T. Pitassi. Value
elimination: Bayesian inference via backtracking search. In Un-
certainty in Artificial Intelligence UAI-2003, pages 20–28, 2003.
[Bistarelli et al., 1997] S. Bistarelli, U. Montanari, and F. Rossi.
Semiring-based constraint satisfaction and optimization. Journal
of the ACM, 44(2):201–236, 1997.
[Borchers and Furman, 1999] B. Borchers and J. Furman. A Two-
Phase Exact Algorithm for MAX-SAT and Weighted MAX-SAT.
Journal of Combinatorial Optimization, 2(4):299-306, 1999.
[Brglez and Fujiwara, 1985] F. Brglez and H. Fujiwara.
tral netlist of 10 combinational benchmark designs and a special
translator in fortran. In Proc. ISCAS, 1985.
[Chavira and Darwiche, 2005] M. Chavira and A. Darwiche. Com-
piling Bayesian networks with local structures. In Proc. 19th
[Darwiche, 2002] A. Darwiche. A compiler for deterministic de-
composable negation normal form. In Proc. 18th AAAI, 2002.
[Darwiche, 2004] A. Darwiche. New advances in compiling cnf
into decomposable negation normal form. In Proc. 16th Euro-
pean Conference on Artificial Intelligence, 2004.
[de Givry et al., 2003] S. de Givry, J. Larrosa, P. Meseguer, and
T. Schiex. Solving max-sat as weighted csp. In 9th Principles
and Practice of Constraint Programming, 2003.
[de Givry et al., 2005] S. de Givry, M. Zytnicki, F. Heras, and
J. Larrosa. Existential arc consistency: Getting closer to full arc
consistency in weighted csps. In Proc. 19th IJCAI, 2005.
[Huang and Darwiche, 2003] J. Huang and A. Darwiche.
structure-based variable ordering heuristic for sat. In Proc. 18th
[Jensen et al., 1990] F. V. Jensen, S.L. Lauritzen, and K.G. Olesen.
Bayesian updating in recursive graphical models by local com-
putation. Computational Statistics Quarterly, 4:269–282, 1990.
[Li et al., 2005] C. M. Li, F. Manya, and J. Planes. Exploiting unit
propagation to compute lower bounds in branch and bound max-
sat solvers. In Proc. 11th Principles and Practice of Constraint
[Li et al., 2006] C. Li, F. Manya, and J. Planes.
joint Inconsistent Subformulas for Computing Lower Bounds for
Max-SAT. In Proc. 21st AAAI, 2006.
[Marinescu and Dechter, 2005] R. Marinescu and R. Dechter.
And/or branch-and-bound for graphical models. In Proc. 19th
[Moskewicz et al., 2001] M. Moskewicz, C. Madigan, Y. Zhao,
L. Zhang, and S. Malik. Chaff: Engineering an efficient SAT
solver. In Proc. 38th Design Automation Conference, 2001.
[Park, 2002] J. Park. Using weighted MAX-SAT engines to solve
MPE. In Proc. 18th AAAI, 2002.
[Sang et al., 2004] T. Sang, F. Bacchus, P. Beame, H. Kautz, and
T. Pitassi. Combining component caching and clause learning for
effective model counting. In Proc. 7th International Conference
on Theory and Applications of Satisfiability Testing, 2004.
[Sang et al., 2005a] T.Sang, P.Beame, and H. Kautz. Heuristics for
fast exact model counting. In Proc. 8th International Conference
on Theory and Applications of Satisfiability Testing, 2005.
[Sang et al., 2005b] T. Sang, P. Beame, and H. Kautz. Performing
Bayesian inference by weighted model counting. In Proc. 20th
[Smith et al., 2005] A. Smith, A. Veneris, M. Ali, and A. Viglas.
Fault diagnosis and logic debugging using boolean satisfiability.
In IEEE Transactions on Computer-Aided Design of Integrated
Circuits and Systems, volume 24, 2005.
[Zhang et al., 2001] L. Zhang, C. F. Madigan, M. H. Moskewicz,
and S. Malik. Efficient conflict driven learning in a boolean satis-
fiability solver. In International Conference on Computer Aided