A Multiple Instance Learning Strategy for Combating Good Word Attacks on Spam Filters.

Full text

Page 1

Journal of Machine Learning Research 8 (2008) 1115-1146Submitted 9/07; Revised 2/08; Published 6/08
A Multiple Instance Learning Strategy for Combating Good Word
Attacks on Spam Filters
Zach Jorgensen
Yan Zhou
Meador Inge
School of Computer and Information Sciences
University of South Alabama
Mobile, AL 36688, USA
ZDJORGEN@JAGUAR1.USOUTHAL.EDU
ZHOU@CIS.USOUTHAL.EDU
WMI601@JAGUAR1.USOUTHAL.EDU
Editor: Lyle Ungar
Abstract
Statistical spam filters are known to be vulnerable to adversarial attacks. One of the more common
adversarial attacks, known as the good word attack, thwarts spam filters by appending to spam
messages sets of “good” words, which are words that are common in legitimate email but rare in
spam. We present a counterattack strategy that attempts to differentiate spam from legitimate email
in the input space by transforming each email into a bag of multiple segments, and subsequently
applying multiple instance logistic regression on the bags. We treat each segment in the bag as
an instance. An email is classified as spam if at least one instance in the corresponding bag is
spam, and as legitimate if all the instances in it are legitimate. We show that a classifier using our
multiple instance counterattack strategy is more robust to good word attacks than its single instance
counterpart and other single instance learners commonly used in the spam filtering domain.
Keywords: spam filtering, multiple instance learning, good word attack, adversarial learning
1. Introduction
It has been nearly thirty years since the first email spam appeared on the Arpanet. Today, to most
end users, spam does not seem to be a serious threat due to the apparent effectiveness of current
spam filters. Behind the scenes, however, is a seemingly never-ending battle between spammers
and spam fighters. With millions of email users, profit-driven spammers have great incentives to
spam. With as little as 0.001% response rate, a spammer could potentially profit $25,000 on a
$50 product (Carpinter and Hunt, 2006). Over the years, spammers have grown in sophistication
with cutting-edge technologies and have become more evasive. The best evidence of their growing
effectiveness is a recent estimate of over US $10 billion worldwide spam-related cost (Jennings,
2005). In this paper, we target one of the adversarial techniques spammers often use to circumvent
existing spam filters.
Adversarial attacks on spam filters have become an increasing challenge to the anti-spam com-
munity. The good word attack (Lowd and Meek, 2005b) is one of the techniques most frequently
employed by spammers. This technique involves appending sets of so-called “good words” to spam
messages. Good words are words that are common to legitimate emails (also called ham) but rare
in spam. Spam messages injected with such words are more likely to appear legitimate and bypass
spam filters. So far, relatively little research has been done to investigate how spam filters might be
c ?2008 Zach Jorgensen, Yan Zhou and Meador Inge.

Page 2

JORGENSEN, ZHOU AND INGE
trained to account for such attacks. This paper presents a possible defense strategy using multiple
instance learning that has shown promising results in our experiments.
Multiple instance (MI) learning (Dietterich et al., 1997) differs from single instance supervised
learning in that an example is represented by a set, or bag, of instances rather than as just a single
instance. The bag is assigned a class label (either positive or negative) based on the instances it
contains; however, the instances within the bag are not necessarily labeled. Classic MI learning
assumes that a bag is positive if at least one instance in the bag is positive, and negative if all
instances are negative. Therefore, the goal of multiple instance learning is to learn a classification
function that accurately maps a given bag to a class. Formally, let B = {B1,...,Bi,...,Bm} be a set
of bags where Bi= {X1i,X2i,...,Xji} is the ithbag and X1i,X2i,...,Xjiare the j instances contained
in bag Bi. If B is a training set, then every Bi∈ B also has a class label ci∈C = {positive,negative}
associated with it. The training process, using B as input, yields a binary classification function
f(Bi) : B →C that maps a bag to a class label.
OurspamfilteringstrategyadoptstheclassicalMIassumption, whichstatesthatabagispositive
if at least one of its instances is positive, and negative if all instances are negative. We treat each
email as a bag of instances. Thus, an email is classified as spam if at least one instance in the
corresponding bag is spam, and as legitimate if all the instances in it are legitimate. The idea is that
by splitting an email into multiple instances, a multiple instance learner will be able to recognize the
spam part of the message even if the message has been injected with good words. Our experimental
results show that a multiple instance learner, combined with an appropriate technique for splitting
emails into multiple instance bags, is more robust to good word attacks than its single instance
counterpart and other single instance learners that are commonly used in the spam filtering domain.
The remainder of this paper is organized as follows. First, we discuss recent research that
has motivated our work. Next, we formalize the spam filtering problem as a multiple instance
learning problem and explain our proposed counterattack strategy in more detail. Following that,
wepresentourexperimentalresultstodemonstratetheeffectivenessofourfilteringstrategy. Finally,
we conclude our work and discuss future directions.
2. Related Work
Our work is primarily motivated by recent research on adversarial learning (Dalvi et al., 2004;
Lowd and Meek, 2005a; Kolter and Maloof, 2005). Dalvi et al. (2004) consider classification to
be a game between classifiers and adversaries in problem domains where adversarial attacks are
expected. They model the computation of the adversary’s optimal strategy as a constrained opti-
mization problem and approximate its solution based on dynamic programming. Subsequently, an
optimal classifier is produced against the optimal adversarial strategy. Their experimental results
demonstrate that their game-theoretic approach outperforms traditional classifiers in the spam filter-
ing domain. However, in their adversarial classification framework, they assume both the classifier
and the adversary have perfect knowledge of each other, which is unrealistic in practice.
Instead of assuming the adversary has perfect knowledge of the classifier, Lowd and Meek
(2005a) formalized the task of adversarial learning as the process of reverse engineering the clas-
sifier. In their adversarial classifier reverse engineer (ACRE) framework, the adversary aims to
identify difficult spam instances (the ones that are hard to detect by the classifier) through member-
ship queries. The goal is to find a set of negative instances with minimum adversarial cost within a
polynomial number of membership queries.
1116

Page 3

COMBATING GOOD WORD ATTACKS ON SPAM FILTERS
Newsome et al. (2006) emphasize the point that the training data used to build classifiers for
spam filtering and the similar problem of internet worm detection is, to a large extent, controlled by
an adversary. They describe and demonstrate several attacks on the generators of such classifiers
in which the adversary is able to significantly impair the learning of accurate classifiers by manip-
ulating the training data, even while still providing correct labels for the training instances. The
attacks involve inserting features, in a specific manner, into one or both classes of the training data
and are specifically designed to cause a significant increase in false positives or false negatives for
the resulting classifier. They conclude that the generation of classifiers for adversarial environments
should take into account the fact that training data is controlled by an adversarial source in order to
ensure the production of accurate classifiers.
Barreno et al. (2006) explore possible adversarial attacks on machine learning algorithms from
multiple perspectives. They present a taxonomy of different types of attacks on machine learning
systems. An attack is causative if it targets the training data, and is exploratory if it aims to discover
information through, for example, offline analysis. An attack is targeted if it focuses on a small
set of points, and is indiscriminate if it targets a general class of points. An integrity attack leads
to false negatives, and an availability attack aims to cause (machine learning) system dysfunction
by generating many false negatives and false positives. They also discuss several potential defenses
against those attacks, and give a lower bound on the adversary’s effort in attacking a na¨ ıve learning
algorithm.
A practical example of adversarial learning is learning in the presence of the good word attack.
Lowd and Meek (2005b) present and evaluate several variations of this type of attack on spam filters.
They demonstrate two different ways to carry out the attack: passively and actively. Active good
word attacks use feedback obtained by sending test messages to a spam filter in order to determine
which words are “good”. The active attacks were found to be more effective than the passive
attacks; however, active attacks are generally more difficult to perform than passive attacks because
they require user-level access to the spam filter, which is not always possible. Passive good word
attacks, on the other hand, do not involve any feedback from the spam filter, but rather, guesses
are made as to which words are considered good. Three common ways for passively choosing
good words are identified. First, dictionary attacks involve selecting random words from a large
collection of words, such as a dictionary. In testing, this method did not prove to be effective; in
fact, it actually increased the chances that the email would be classified as spam. Next, frequent
word attacks involve the selection of words that occur most often in legitimate messages, such as
news articles. This method was more effective than the previous one, but it still required as many
as 1,000 good words to be added to the original message. Finally, frequency ratio attacks involve
the selection of words that occur very often in legitimate messages but not in spam messages. The
authors’ tests showed that this technique was quite effective, resulting in the average spam message
being passed off as legitimate by adding as few as 150 good words to it. Preliminary results were
also presented that suggested that frequent retraining on attacked messages may help reduce the
effect of good word attacks on spam filters.
Webb et al. (2005) also examined the effectiveness of good word attacks on statistical spam
filters. They present a “large-scale evaluation” of the effectiveness of the attack on four spam filters:
na¨ ıve Bayes, support vector machine (SVM), LogitBoost, and SpamProbe. Their experiments were
performed on a large email corpus consisting of around a million spam and ham messages, which
they formed by combining several public and private corpora. Such a large and diverse corpus
more closely simulates the environment of a server-level spam filter than a client-level filter. The
1117

Page 4

JORGENSEN, ZHOU AND INGE
experimental results show that, on normal email, that is, email that has not been modified with
good words, each of the filters is able to attain an accuracy as high as 98%. When testing on
“camouflaged messages”, however, the accuracies of the filters drop to between 50% and 75%. In
their experiments, spam emails were camouflaged by combining them with portions of legitimate
messages. They experimented with camouflaged messages containing twice as much spam content
as legitimate content, and vice versa. They also proposed and demonstrated a possible solution to
the attack. By training on a collection of emails consisting of half normal and half camouflaged
messages, and treating all camouflaged messages as spam, they were able to improve the accuracy
of the filters when classifying camouflaged messages.
Our counterattack strategy against good word attacks is inspired by work in the field of multiple
instance (MI) learning. The concept of MI learning was initially proposed by Dietterich et al. (1997)
for predicting drug activities. The challenge of identifying a drug molecule that binds strongly to
a target protein is that a drug molecule can have multiple conformations, or shapes. A molecule
is positive if at least one of its conformations binds tightly to the target, and negative if none of
its conformations bind well to the target. The problem was tackled with an MI model that aims to
learn axis-parallel rectangles (APR). Later, learning APR in the multiple instance setting was further
studied and proved to be NP-complete by several other researchers in the PAC-learning framework
(Auer, 1997; Long and Tan, 1998; Blum and Kalai, 1998).
Several probabilistic models: Diverse Density (DD) (Maron and Lozano-P´ erez, 1998) and its
variation EM-DD (Zhang and Goldman, 2002), and multiple instance logistic regression (MILR)
(Ray and Craven, 2005), employ a maximum likelihood estimation to solve problems in the MI
domain. The original DD algorithm searches for the target concept by finding an area in the feature
space with maximum diverse density, that is, an area with a high density of positive points and a low
density of negative points. The diverse density at a point in the feature space is defined to measure
probabilistically how many different positive bags have instances near that point, and how far the
negative instances are from that point. EM-DD combines EM with the DD algorithm to reduce the
multiple instance learning problem to a single-instance setting. The algorithm uses EM to estimate
the instance in each bag which is most likely to be the one responsible for the label of the bag.
The MILR algorithm presented by Ray and Craven (2005) is designed to learn linear models in a
multiple instance setting. Logistic regression is used to model the posterior probability of the label
of each instance in a bag, and the bag level posterior probability is estimated by using softmax to
combine the posterior probabilities over the instances of the bag. Similar approaches with different
combining functions are presented by Xu and Frank (2004).
Manysingle-instancelearningalgorithmshavebeenadaptedtosolvethemultipleinstancelearn-
ing problem. For example, Wang and Zucker (2000) propose the lazy MI learning algorithms,
namely Bayesian-kNN and citation-kNN, which solve the multiple instance learning problem by us-
ing the Hausdorff distance to measure the distance between two bags of points in the feature space.
Chevaleyre and Zucker (2001) propose the multi-instance decision tree ID3-MI and decision rule
learner RIPPER-MI by defining a new multiple instance entropy function and a multiple instance
coverage function. Other algorithms that have been adapted to multiple instance learning include
the neural network MI-NN (Ramon and Raedt, 2000), DD-SVM (Chen and Wang, 2004), MI-SVM
and mi-SVM (Andrews et al., 2003), multi-instance kernels (G¨ artner et al., 2002), MI-Ensemble
(Zhou and Zhang, 2003), and MI-Boosting (Xu and Frank, 2004).
In this paper, we demonstrate that a counterattack strategy against good word attacks, developed
in the framework of multiple instance learning, can be very effective, provided that a single instance
1118

Page 5

COMBATING GOOD WORD ATTACKS ON SPAM FILTERS
can be properly transformed into a bag of instances. We also explore several possible ways to
transform emails into bags of instances. Our experiments also verify earlier observations, discussed
in other works (Lowd and Meek, 2005b; Webb et al., 2005), that retraining on emails modified
during adversarial attacks may improve the performance of the filters against the attack.
3. Problem Definition
Consider a standard supervised learning problem with a set of training data D = {< X1,Y1>,...,<
Xm,Ym>}, where Xiis an instance represented as a single feature vector, Yi= C(Xi) is the target
value of Xi, where C is the target function. Normally, the task is to learn C given D. The learning
task becomes more difficult when there are adversaries who could alter some instance Xiso that
Xi→ X?
X?
with good words. So, ∆Xirepresents a set of good words added to a spam message by the spammer.
There are two cases that need to be studied separately:
iand cause Yi→ Y?
i=Xi+∆Xi. In the case of spam filtering, an adversary can modify spam emails by injecting them
i, where Yi?= Y?
i. Let ∆Xibe the difference between Xiand X?
i, that is,
1. the filter is trained on normal emails, that is, emails that have not been injected with good
words, and tested on emails which have been injected with good words;
2. both the training and testing sets contain emails injected with good words.
In the first case, the classifier is trained on a clean training set. Predictions made for the altered
test instances are highly unreliable. In the second case, the classifier may capture some adversarial
patterns as long as the adversaries consistently follow a particular pattern.
In both cases, the problem becomes trivial if we know exactly how the instances are altered;
we could recover the original data and solve the problem as if no instances were altered by the
adversary. In reality, knowing exactly how the instances are altered is impossible. Instead, we seek
to approximately separate Xiand ∆Xiand treat them as separate instances in a bag. We then apply
multiple instance learning to learn a hypothesis defined over a set of bags.
4. Multiple Instance Bag Creation
We now formulate the spam filtering problem as a multiple instance binary classification problem
in the context of adversarial attacks. Note that the adversary is only interested in altering positive
instances, that is, spam, by injecting sets of good words that are commonly encountered in negative
instances, that is, legitimate emails, or ham. We propose four different approaches to creating multi-
ple instance bags from emails. We call them split-half (split-H), split-term (split-T), split-projection
(split-P), and split-subtraction (split-S). We will now discuss each of these splitting methods, in
turn. Later, in Section 8, we investigate and discuss possible weaknesses of some of these splitting
methods.
4.1 Split-H
The first and simplest splitting method that we considered, which we call split-half (split-H), in-
volves splitting an email down the middle into approximately equal halves. Formally, let B =
{B1,...,Bi,...,Bm} be a set of bags (emails), where Bi= {Xi1,Xi2} is the ithbag, and Xi1, Xi2are
1119