ATARiS: Computational quantification of
gene suppression phenotypes from multi-
sample RNAi screens
Running Title: A novel approach for analysis of RNAi screens
Keywords: RNAi screening, cancer, computational, analysis, gene phenotype, RNAi reagents
Diane D. Shao1,2,*, Aviad Tsherniak1,*, Shuba Gopal1, Barbara A. Weir1, Pablo Tamayo1,
Nicolas Stransky1,3, Steven E. Schumacher1,4, Travis I. Zack1,4,7, Rameen Beroukhim1,2,4,5, Levi
A. Garraway1,2,5, Adam A. Margolin1,6, David E. Root1, William C. Hahn1,2,5,†, Jill P. Mesirov1,†
* These authors contributed equally.
1Broad Institute of Harvard and M.I.T., 7 Cambridge Center, Cambridge, Massachusetts
Departments of 2Medical Oncology and 4Cancer Biology, Dana-Farber Cancer Institute,
Boston, MA 02215, USA
3Current affiliation: Blueprint Medicines, Cambridge, MA 02142, USA
5Departments of Medicine, Brigham and Women’s Hospital and Harvard Medical School,
Boston, MA 02115, USA
6Current affiliation: Sage Bionetworks, Seattle, WA 98109, USA
7Program in Biophysics, Harvard University, Boston, MA 02115, USA
Jill P. Mesirov, Ph.D.
Broad Institute of MIT and Harvard
7 Cambridge Center
Cambridge MA 02142
phone: 617-714-7070 fax: 617-714-8991
William C. Hahn, M.D., Ph.D.
Dana-Farber Cancer Institute
450 Brookline Avenue, Dana 1538
Boston, MA 02215 USA
Genome-scale RNAi libraries enable the systematic interrogation of gene function. However, the
interpretation of RNAi screens is complicated by the observation that RNAi reagents designed
to suppress the mRNA transcripts of the same gene often produce a spectrum of phenotypic
outcomes due to differential on-target gene suppression or perturbation of off-target transcripts.
Here we present a computational method, Analytic Technique for Assessment of RNAi by
Similarity (ATARiS), that takes advantage of patterns in RNAi data across multiple samples in
order to enrich for RNAi reagents whose phenotypic effects relate to suppression of their
intended targets. By summarizing only such reagent effects for each gene, ATARiS produces
quantitative, gene-level phenotype values, which provide an intuitive measure of the effect of
gene suppression in each sample. This method is robust for datasets that contain as few as ten
samples and can be used to analyze screens of any number of targeted genes. We used this
analytic approach to interrogate RNAi data derived from screening more than 100 human
cancer cell lines and identified HNF1B as a transforming oncogene required for the survival of
cancer cells that harbor HNF1B amplifications. ATARiS is publicly available at
RNAi screening is a powerful approach that facilitates the systematic assessment of the effect
of gene suppression on cell phenotypes such as cell death or the activity of a signaling pathway.
The development and availability of genome-scale RNAi libraries provide the tools to identify
new pathway components and context-specific cancer dependencies (Luo et al. 2008; Cheung
et al. 2011; Kittler and Pelletier 2008; Hirsch 2010). Technological and analytical advances will
provide further opportunities for the application and interpretation of functional screens.
For screens in mammalian cells, a short interfering RNA (siRNA) is introduced into cells either
directly as a duplex or by expression of a short hairpin RNA (shRNA) that is processed into
active siRNA. This siRNA is designed to specifically degrade mRNA transcripts of
complementary sequence to reduce the expression of gene products (Elbashir et al. 2001; Root
et al. 2006). In practice, these reagents exhibit a variable degree of suppression of the targeted
gene, and may also suppress genes other than the intended target (Jackson et al. 2003; 2006;
Birmingham et al. 2006). Here we refer to a reagent’s phenotypic effects resulting from
suppression of unintended genes as off-target effects. Analytical approaches to identify specific
types of off-target effects in siRNA (e.g., seed sequence similarity) have been previously
developed (Marine et al. 2012; Sigoillot et al. 2012). However, we currently lack the ability to
systematically characterize both the on-target and off-target effects of siRNAs.
To identify candidate genes that produce a desired phenotype based upon imperfect reagents,
multiple distinct RNAi reagents targeting each gene are often screened (Echeverri et al. 2006;
Cullen 2006). Analyzing data from multiple reagents per gene has the potential to (1) increase
the power to detect candidate genes, and (2) decrease the false-positive rates. For example,
the “frequency approach” considers a gene a candidate in a sample if several of its reagents
induce a desired effect. This effect is usually measured by deviation from the experimental, or
de facto, negative control effects (Chung et al. 2008; Bard et al. 2006; Müller et al. 2005). A
variation of this approach is to assign a gene score by using a simple function, such as the
average, of a few reagents with the most desired effects (Marcotte et al. 2012). More recent
methods for scoring genes in individual samples, such as ‘redundant siRNA activity’ (RSA,
König et al. 2007) and ‘strictly standardized mean difference’ (SSMD, Zhang et al. 2007; 2011)
have further decreased false-positive rates. For each sample, they consider the phenotypes
produced by reagents for all of the screened genes simultaneously (RSA) or by all reagents for
each gene separately (SSMD).
As RNAi screens are being performed in increasing numbers of samples (Cheung et al. 2011;
Brough et al. 2011; Collinet et al. 2010; Marcotte et al. 2012), a common analytical approach
has been to segregate samples into two pre-defined classes in order to identify genes with
differential effects. By summarizing data within each class, aberrant reagent effects in individual
samples are less likely to impact the final result. The “second best” method assigns scores to
genes based on each gene’s second most differentially scoring reagent between classes
(Cheung et al. 2011), requiring – similar to the “frequency approach” – that favorable genes
have at least two high-scoring reagents. Alternatively, RNAi Gene Enrichment Ranking (RIGER)
ranks all the reagents by their differential effects and generates a gene-level score for each
gene based on the rank distribution of its reagents (Luo et al. 2008; Barbie et al. 2009),
analogous to RSA. However, the requirement of two pre-defined classes can limit full
interrogation of the data.
Currently, RNAi analysis methods do not attempt to assess the performance of individual
reagents. Thus, there is an opportunity to further improve analysis of RNAi data by harnessing
the statistical information across many samples to identify and avoid data from problematic
reagents when determining gene-level effects. An analogous approach is employed by dChip (Li
and Hung Wong 2001) and RMA (Irizarry et al. 2003), two widely used methods for mRNA
abundance quantification in microarray data. Given a set of samples, these algorithms quantify
a probe set’s overall abundance level in each sample from a set of multiple, distinct,
complementary probes. In the case of RNAi data, one must also consider additional factors
such as off-target effects thought to exist for a subset of reagents, the dramatically greater
biological variability, and the possibility of multiple phenotypic effects for a single gene (e.g., due
to different levels of on-target gene suppression).
Here we introduce ATARiS (Analytic Technique for Assessment of RNAi by Similarity), a novel
computational approach to the quantification of gene-specific suppression phenotypes. ATARiS
uses patterns in the data from multi-sample RNAi screens to estimate the performance of
individual RNAi reagents targeting each gene and generates a per-gene value for each sample
that quantifies the phenotypic effect of gene suppression. We used data from two recent large-
scale shRNA screens of 102 and 72 cancer cell lines (Cheung et al. 2011; Marcotte et al. 2012),
respectively, to demonstrate the performance of ATARiS. We integrated ATARiS-generated
gene phenotype values with copy-number and gene expression data to uncover novel cancer
dependencies, including the identification of a novel amplified oncogene, HNF1B. We are
making ATARiS publicly available (http://broadinstitute.org/ataris) in hopes of aiding current
RNAi screening efforts.
ATARiS is a computational method to assess gene suppression effects in each sample of multi-
sample RNAi screens that include at least two RNAi reagents (siRNA or shRNA) designed to
target each gene. Our method uses only data from reagents determined to have primarily on-
target effects, discarding data from reagents with off-target effects. To identify on-target
reagents, we noted that in an RNAi library, reagents are designed to target distinct sequences.
Thus, it is unlikely that any two reagents – including those targeting the same gene – will
suppress the same set of off-target genes. We therefore concluded that when RNAi reagents
designed to target the same gene behave similarly across the screened samples, the observed
effects are likely due to suppression of the intended gene rather than off-target suppression. For
each gene in a screen, ATARiS identifies sets of reagents with similar behavior across all
samples in order to produce two types of results:
1. A gene solution that summarizes the observed effects produced by identified on-target
reagents into quantitative values across all screened samples (the value for an individual
sample is called a phenotype value). We account for potential multiple phenotypic
outcomes after suppression of a given gene, possibly due to different degrees of gene
suppression, by allowing for multiple solutions comprised of disjoint sets of consistent
2. A consistency score for each RNAi reagent that represents the confidence that its
observed phenotypic effects are the result of on-target gene suppression. ATARiS
assigns higher consistency scores to reagents whose profiles (i.e., the observed effect of
that reagent in every screened sample) exhibit higher correlation to a larger number of
reagent profiles within the same solution.
We give a general description of ATARiS here (see also Supplementary Fig. 1), and provide
technical details in Methods. Figure 1a-d summarizes the different approaches used by ATARiS
and current RNAi analysis methods.
First, to construct a gene solution for a given gene G, ATARiS considers the observed data of
all RNAi reagents designed to target G. The data for each reagent r are median-centered, as we
are interested in reagents whose relative effects across the samples are similar. For each
sample s, ATARiS calculates a value cs that summarizes the effects produced by all the
reagents targeting G in s. We refer to the vector c of all cs values as the consensus profile. To
estimate cs, ATARiS models each data point xr,s (i.e., the observed effect induced by reagent r
in sample s) as a product of two unknown quantities: er, representing the relative magnitude of
the effects of reagent r, and cs. ATARiS estimates the values for er and cs by minimizing an L1-
norm objective function using the method of alternating minimizations (Csiszar and Tusnady
1984). An L1 norm makes the optimization more robust to outliers, which are common in this
type of data.
Next, ATARiS iteratively refines the considered set of reagents by evaluating the similarity of
each reagent profile to the consensus profile. If, for any reagent profile, the significance of the
Spearman correlation (calculated using an empirical null distribution) is lower than a pre-defined
threshold, the reagent whose profile is least similar to the consensus profile is discarded from
further analysis. ATARiS then repeats the process of computing a consensus profile and
discarding dissimilar reagents until either only one reagent remains — in which case no gene
solution is generated — or until all remaining reagents have profiles significantly similar to the
consensus profile. The consensus profile c for the retained reagents is then used as a gene
solution for that gene, and we refer to its elements cs as the gene’s phenotype values. The
entire process is then repeated for any remaining reagents not yet contributing to a solution until
no more solutions are found. A greedy approach to refinement, rather than an exhaustive one,
allows scaling to larger numbers of reagents per gene.
After generating all gene solutions for gene G, ATARiS computes a consistency score for each
of its reagents. The consistency score for reagent r is based on the negative log10 of the
integrated p-values of the Spearman correlation coefficients of r’s profile to each of the other
reagent profiles within the same solution. Thus, the consistency score may be interpreted as a
p-value, i.e., a consistency score of 1.3 corresponds to –log10 (p-value of 0.05). For RNAi
reagents that do not participate in any solution, all reagents targeting G are considered in
computing the consistency score. Thus, even for a reagent that is excluded from a solution
depending on the pre-defined threshold, ATARiS still provides an assessment of the confidence
in its functional effects.
Application of ATARiS to data derived from multi-
sample shRNA screens
To test and validate ATARiS, we primarily used the data produced by Project Achilles – a
dataset produced from massively parallel screening of 102 cancer cell lines with a genome-
scale pooled shRNA library targeting more than 11,000 human genes with an average of 5
shRNAs per gene (Cheung et al. 2011). The final abundance of each shRNA after propagation
of the cell line was determined with respect to the initial reference shRNA pool to assess cellular
dependency on each shRNA’s target (i.e., shRNAs that target essential genes will be depleted).
See Methods for a full description of additional data pre-processing and normalization steps.
The resulting dataset is available as Supplementary Data 1.
The application of ATARiS to this dataset yielded gene solutions for 7,250 genes, and
incorporated data from 49.5% of the screened shRNA reagents (Supplementary Data 2) when
using a 0.15 significance threshold. With this threshold, we would expect ATARiS to generate
solutions, on average, for 15% of the genes using randomly permuted data. Supplementary
Figure 2 shows the distribution of the number of gene solutions identified for varying thresholds.
In our dataset, 6,233 genes had one associated gene solution, 1,017 genes had two or more
solutions, and 3,955 genes had no solutions (Supplementary Table 1). We illustrate the type of
correlated reagent data that becomes incorporated into a gene solution by using an example
gene BRAF (Fig. 1e). A consistency score was generated for every screened shRNA, including
those that do not participate in any gene solution (Supplementary Data 3).
Influence of dataset size and biological context on
Since most RNAi screens currently do not include as many samples as Project Achilles, we
assessed the robustness of ATARiS on simulated cases where data from fewer samples were
available. We generated 100 random subsets of the Achilles dataset for each sample size of 10,
20,…, 100 cell lines. We used only the data for genes with solutions from analyzing the full 102-
sample Achilles dataset, which served as a reference dataset for these analyses. For the
subsets of size 10, 30, and 50 cell lines, we found that the median percentage of genes with
solutions is 67%, 80%, and 85%, respectively (Fig. 2a). This increase corresponds to an overall
increase in the number of genes with solutions as the sample size increases (Supplementary
Fig. 3). In addition, we observed a substantial overlap in the shRNAs that are used for a gene
solution in the reference dataset with the shRNAs used in the subsets of size 10, 30, and 50 cell
lines (median percentages 75%, 85% and 89%, respectively, Supplementary Fig. 4). The
robustness of shRNA choice for gene solutions resulted in generally robust values for gene
phenotype values even at small sample size. For example, ATARiS results from subsets of as
few as ten cell lines showed that more than 75% of gene phenotype values deviated from their
respective gene phenotype values in the reference dataset by less than one standard deviation
(Fig. 2b). We observed similarly robust results when running ATARiS on datasets after the
addition of random noise (Supplementary Fig. 5). We concluded that we can use ATARiS to
identify consistent shRNAs, and thus to determine gene phenotype values, even for small
numbers of screened samples.
We evaluated whether ATARiS results are robust when applied to samples screened
independently of Project Achilles by (1) simulating independent screening datasets, and (2)
analyzing results from a set of independently performed genome-scale pooled shRNA viability
screens (Marcotte et al. 2012). To simulate independent datasets, we generated 100 pairs of
random disjoint subsets (i.e., no overlapping samples) from the full Achilles dataset for each
sample size of 10, 20,…, 50 cell lines. For pairs of subsets of size 10, 30, and 50 cell lines, the
median percentage of shRNAs that were used to generate solutions in both disjoint subsets of
samples are 71%, 80% and 84% respectively (Fig. 2c). Next, we analyzed the RNAi screening
data from Marcotte et al. (2012), comprised of 72 cancer cell lines screened using a comparable
shRNA library (See Methods for details). ATARiS found relatively fewer gene solutions in this
data (as a fraction of the number of genes targeted), consistent with its having fewer samples
and higher homogeneity in cell lineages (Supplementary Table 3). For genes that have a
solution in both datasets, we found that the shRNAs targeting those genes are more likely to
participate in a solution in both datasets than in one dataset but not in the other (odds ratio =
2.1; 95% confidence interval [1.96, 2.26]; p-value < 2.2x10-16; Fisher’s exact test). Furthermore,
the Pearson correlation coefficient for ATARiS shRNA consistency scores in the two datasets is
0.46 (95% confidence interval [0.45, 0.47]; p-value < 2.2x10-16). Together, these observations
suggested that ATARiS produces robust results between independent screens.
To account for the fewer number of solutions when smaller subsets of cell lines are analyzed,
we hypothesized that gene solutions can be identified primarily for genes whose suppression
yields phenotypic variation across samples. To test this hypothesis, we determined the
frequency of finding a BRAF solution when we apply ATARiS to sets of BRAF wild-type cell
lines (which are expected to exhibit similar dependence on BRAF) versus when the set contains
an equal number of BRAF wild-type and mutant cell lines (mutant lines are much more
dependent on BRAF relative to wild-type). In the latter case, a BRAF solution was found in 98%
of runs using only 10 cell lines (5 wild-type, 5 mutant), whereas using as many as 26 wild-type
cell lines alone (26 wild-type, 0 mutant) yielded solutions in only 82% of runs (Fig. 2d). Thus, the
reduced number of solutions at smaller sample size likely reflects the reduced overall
heterogeneity amongst a few samples as compared to the full sample set.
Validation of shRNA consistency scores
ATARiS shRNA consistency scores are intended to reflect our confidence in the specificity of
each reagent. However, validation is challenging since the currently accepted standards for
evaluating reagent performance, i.e., immunoblotting and quantitative RT-PCR for on-target
gene suppression, cannot assess off-target effects whereas ATARiS consistency scores
attempt to encompass both on- and off-target aspects. Since a greater degree of target gene
suppression does not necessarily amplify functional outcome, and effective on-target gene
suppression does not equate to lack of off-target effects, we did not expect high correlation
between immunoblotting results and ATARiS scores. We expected, however, that shRNAs with
high consistency scores have some degree of detectable on-target gene suppression in order to
produce correlated profiles.
We validated consistency scores on a few selected genes – BRAF, PIK3CA, KRAS, and MYC –
chosen for their importance in cancer, availability of reagents to assess the expression of these
genes, and known functional effect of their shRNAs in a subset of Project Achilles cell lines
(Cheung et al. 2011). We introduced individual shRNAs into the A549 cancer cell line and
performed immunoblotting on cell lysates to determine changes at the protein level. For BRAF,
we observed that ATARiS consistency scores are high for shRNAs that reduce BRAF protein
levels (Fig. 3a). We note that for shBRAF-3 and shBRAF-4, which have similar consistency
scores but different degrees of protein suppression, 40% protein suppression may be sufficient
to produce functional effects, and the effects may not be enhanced by increased protein
suppression. For PIK3CA, only two shRNAs (shPIK3CA-1 and shPIK3CA-2) have high
consistency scores and both result in increased suppression of PIK3CA protein levels (Fig. 3b).
Our interpretation for the low consistency score of shPIK3CA-3, which effectively suppresses
PIK3CA at the protein level, is that it may also have significant off-target effects. For KRAS and
MYC, the effects of expressing individual shRNAs on protein levels also agreed with ATARiS
consistency scores (Supplementary Fig. 6).
To test whether consistency scores reflect on-target gene suppression for many more genes,
we compared ATARiS consistency scores to gene suppression assessed by qRT-PCR for 9,050
of the shRNAs from the screening library (manuscript in preparation, data available on request).
We found that shRNAs with significantly high consistency scores (corresponding to FDR < 0.1)
suppress target gene mRNA levels to a greater degree than other shRNAs targeting the same
gene (p-value = 0.003, χ2 test; Supplementary Fig. 7). We therefore concluded that genes with
high consistency scores are likely to have a functionally relevant degree of gene suppression.
Gene phenotype values from Achilles dataset
represent biological dependencies
We first validated individual ATARiS gene phenotype values, representing degree of
dependency on each gene in the Achilles dataset, by assessing whether they recapitulate
known dependencies for the oncogenes BRAF, PIK3CA, and KRAS. For each oncogene, we
calculated the area under Receiver Operating Characteristic curve (AUC) statistic (Mason and
Graham 2002) to measure the degree to which the gene phenotype values discriminate
between cell lines harboring a mutation versus those without mutation. We confirmed that cell
lines harboring a mutation have significantly lower phenotype values for the respective gene,
i.e., are more sensitive to gene suppression (Fig. 4a; p-value < 0.01; Mann-Whitney test). To
show that our phenotype values may be meaningful for individual cell lines, we performed low-
throughput viability assays on cell lines that span a range of KRAS phenotype values. We
introduced two KRAS-specific shRNA or a control shRNA into three KRAS wild-type and three
KRAS mutant cell lines and measured cell proliferation/viability after six days using an ATP-
luminescence assay. Indeed, the cell lines most sensitive to KRAS suppression were the ones
that received the lowest KRAS phenotype values (Fig. 4b). Thus, we affirmed that ATARiS
phenotype values reflect the relative effects of gene suppression between individual samples.
We reasoned that if ATARiS solutions are meaningful, then we should be able to “rediscover”
the above oncogenic dependencies. Two-class comparisons between groups of cell lines with
defined properties are currently a common application of this type of RNAi proliferation screen
data, so we defined classes based on mutation status for each of BRAF, PIK3CA, and KRAS to
identify differentially required genes (see Supplementary Data 4). For each analysis, we
calculated the mean difference between mutant versus wild-type classes for each gene solution
and estimated p-values from an empirically calculated null distribution by class permutation.
KRAS, BRAF, and PIK3CA are each ranked 1st for being differentially required in their
respective mutant class and remained significant after Benjamini-Hochberg adjustment for
multiple hypothesis testing (Benjamini and Hochberg 1995) (q-value < 0.25, Supplementary
Table 2). The fact that each class comparison yielded statistically meaningful results lends more
validity to ATARiS phenotype values overall.
Finally, we sought to show that ATARiS phenotype values are valid for more than the specific
oncogenes described above by defining classes using recurrent genomic alterations. Since
commonly amplified or deleted regions in cancer are believed to include drivers that require
unique cellular networks, we reasoned that more genes should be differentially essential when
classes are defined by significant genomic alteration than when defined randomly. We defined
significantly amplified and deleted peaks based on application of the Genomic Identification of
Significant Targets in Cancer (GISTIC) method (Beroukhim et al. 2007) to copy number data
from the Cancer Cell Line Encyclopedia (CCLE, Barretina et al. 2012), a large collection of
genomically annotated cancer cell lines, of which 76 were screened in Project Achilles (see
Supplementary Data 5). For each peak present in at least six Achilles cell lines, we defined two
classes based on the peak’s presence or absence (101 total peaks), and calculated the
difference in means between classes for every ATARiS gene solution to identify differentially
essential genes (see Methods). Only 5% of analyses using randomly defined classes yielded
more than 4 significantly differential genes while 16% of analyses using GISTIC peaks do. We
showed that significantly more essential genes were in classes defined by GISTIC peaks
compared to random classes (p-value = 6x10-6, Wilcoxon rank sum; Fig. 4c; see also
Supplementary Fig. 8a), supporting the idea that ATARiS gene phenotype values likely reflect
Our analysis of the Achilles data results in multiple gene solutions for 9% of genes. To
determine whether the supplementary solutions are also meaningful, we repeated the analysis
using GISTIC peaks, as described above, after removing the first solution found for each gene
(1,030 solutions remaining). Indeed, we confirmed that even in this case, more significant
solutions (FDR < 0.25) were identified when the analysis is performed using significant genomic
alterations compared to random permutation of these alterations across samples (p-value =
0.0041; Wilcoxon rank sum; see also Supplementary Fig. 8b), suggesting that the multiple
solutions for each gene may have biological relevance.
ATARiS phenotype values enable novel approaches to
In Figure 4c and Supplementary Table 2, we showed that ATARiS gene phenotype values could
be effectively used for two-class comparisons, the focus of current analytic methods for multi-
sample RNAi data. Additionally, ATARiS per-sample phenotype values expand the repertoire of
downstream analyses from comparison between two classes to a range of additional
possibilities. These include integrated analysis with other types of genomic data, e.g., gene
expression, mutations, genome copy number, which provide quantitative information for genes
in each sample. We describe illustrative examples here.
Using phenotype values to identify genomic predictors of gene dependency. Genes that control
the cell cycle G1 restriction point are commonly altered in the cancer genome. Thus, we
focused on ATARiS solutions for E2F transcription factors, well characterized in checkpoint
regulation, to determine whether we could identify known (and unknown) genetic alterations
related to E2F activation. We used an annotated sample feature list that includes significant
amplification and deletion peaks, cell lineage, mutation, and copy number alterations of major
oncogenes/tumor suppressors (see Methods; Supplementary Data 5). As expected, using the
E2F1 ATARiS solution, we found that RB1 loss was one of the most highly associated features
with E2F1 dependence (Supplementary Fig. 9). In addition, when we examined E2F3, we found
that E2F3 dependence is significantly associated with an E2F3-containing amplification peak
6p22, a MYC-containing amplification peak at 8q24.1, as well as RB1 copy-number loss (Fig.
5a) — all mechanisms that lead to E2F3 activation (Leone et al. 2001; Dyson 1998; Oeggerli et
al. 2006). Similar analyses can be applied to other gene phenotype scores to elucidate genomic
relationships with functional data.
Identifying functional relationships between gene phenotype scores. We evaluated the potential
of using correlations between different gene solutions to yield functionally meaningful gene
relationships. In particular, we focused on the ATARiS solution for Cyclin D1 (CCND1) since this
gene serves as critical mediator between the mitogenic pathway and cell cycle progression. The
genes whose solutions most significantly correlated to CCND1 solution included a gene that
encodes the CCND1 binding partner CDK6 (no solution was available for the other cyclin D
binding partner CDK4) and also members of the mitogenic pathway, including KRAS and RAF1
(Musgrove et al. 2011; Liu et al. 1995, Fig. 5b). Thus, we showed that using statistical
relationships between ATARiS phenotype values allowed us to assess functional gene
Identifying novel cancer-associated genes by integrating datasets on a per-sample basis.
Currently, identification of novel cancer genes using RNAi data primarily consists of intersecting
candidate gene lists separately derived from RNAi analysis and copy-number or expression
data (Cheung et al. 2011; Garraway et al. 2005). Gene expression data is a powerful tool that
has been integrated with copy-number data in a sample-specific manner to identify cancer
drivers (Bussey et al. 2006). Since ATARiS enables integration of functional data with other
datasets on a per-sample basis, we reasoned that we could identify cancer drivers by looking
for genes that are essential in samples where the gene is highly expressed. Thus, we
independently calculated the correlation between each gene solution and the corresponding
gene’s expression values across 83 Project Achilles cell lines for which expression microarrays
are available (Supplementary Table 4). We noted that previously reported cancer dependencies
or oncogenes such as PAX8 (Cheung et al. 2011; Li et al. 2011), BCL2L1 (Beroukhim et al.
2010), E2F3 (Oeggerli et al. 2006), and MYB (Ramsay 2008) are significantly essential in
samples that express the gene highly (Fig. 5c; Supplementary Table 5). Furthermore, we
evaluated whether oncogenes, a subset of genes that might be expected to be essential in
highly expressed cell lines, are enriched in our results. We determined that the list of known
amplified oncogenes reported in Beroukhim et al. (2010) was significantly over-represented
towards the top of our list (p-value = 5.38e-06; Wilcoxon rank sum). For comparison, as
expected, known tumor suppressors were not enriched (p-value = 0.84). These results
suggested that other statistically significant genes from this analysis might contribute to
malignant transformation, in particular the top gene, HNF1B.
Characterization of HNF1B dependency
We found that cell lines that express high levels of HNF1B required HNF1B expression for
proliferation/survival (Fig. 5c). A common mechanism for increased gene expression in cancer
is genomic amplification; thus we also analyzed which genes involved in recurrent, focal
genomic amplifications specifically scored as dependent in those samples (See Supplementary
Methods). HNF1B again ranked at the top of this analysis (Supplementary Table 6), suggesting
that HNF1B was one target of this amplification. We note that HNF1B is amplified in 23 percent
of all cancers (http://broadinstitute.org/tumorscape).
Characterization of HNF1B dependency was made straightforward by directly applying ATARiS
results to reveal which shRNAs are on-target and, specifically, which samples show higher
dependency. We confirmed that the two HNF1B-specific shRNAs receiving the highest
consistency scores suppressed HNF1B levels as assessed by immunoblotting (Fig. 6a).
Furthermore, exogenous expression of HNF1B in cells expressing a HNF1B 3’UTR-specific
shRNA (shHNF1B-1) abrogated the cell death induced by expressing the HNF1B 3’UTR-
specific shRNA alone (Fig. 6b), confirming that the observed shRNA effects were specific. We
used a panel of cell lines to confirm that HNF1B protein expression was correlated to HNF1B
phenotype values (Supplementary Fig. 10). Finally, we used cell lines that expressed high levels
of HNF1B to confirm that they were indeed sensitive to HNF1B suppression by the two HNF1B-
specific shRNAs as compared to control shRNA. For comparison, we showed that DLD-1 and
an immortalized cell line HA1E (Hahn et al. 1999), neither of which harbor amplifications
involving HNF1B nor express high levels of the gene, are insensitive to HNF1B suppression
(Fig. 6c). ATARiS phenotype values allowed us to identify two additional cell lines SLR-21 and
786-O that had low phenotype values for HNF1B, but for which we did not have corresponding
copy-number data. We confirmed that these cell lines were also dependent on HNF1B and had
corresponding genomic copy-number gain (Fig. 6c, Supplementary Fig. 11).
To determine whether HNF1B expression is essential for tumor maintenance in vivo, we
performed xenograft experiments by implanting HT29 colon cancer cells subcutaneously after
expression of control or HNF1B-specific shRNA (Fig. 6d). In the initial two weeks, the xenografts
with suppressed HNF1B showed marked growth impairment (n=3; p<0.01, one-tailed Student’s
t-test). Four weeks post-injection, their growth increased, likely due to re-activation of HNF1B
expression (Fig. 6d), suggesting that HNF1B expression was critical for growth.
Finally, we sought to determine whether expression of HNF1B transforms human cell lines.
Specifically, we introduced HNF1B or LacZ cDNA into HA1EM cells, which are immortalized,
non-tumorigenic human embryonic kidney cells that are transformed upon addition of
oncogenes AKT or IKKε (Boehm et al. 2007). Expression of HNF1B conferred the ability for
anchorage independent growth, a marker of cell transformation (Fig. 6e). Together these
observations – that HNF1B is amplified in human cancers, transforms immortalized cells, and is
essential for those cancer cell lines that harbor increased HNF1B copy number – provide strong
evidence that HNF1B is an oncogene.
One key advance of ATARiS lies in the ability to distinguish reagents with on-target effects and
reject reagents with significant off-target effects by mining patterns across multi-sample
screens. ATARiS reagent consistency scores may be interpreted as a p-value that estimates the
confidence in each reagent, and thus enables the use of ATARiS in the selection of reagents for
validation studies and as an aid in the design and refinement of RNAi libraries. Based on our
work with these shRNAs, we anticipate that we lack more than 1 effective shRNA for a fraction
of targeted genes. ATARiS will allow us to interrogate these situations and to develop improved
libraries in the future. For example, one might create additional shRNA reagents for genes that
lack solutions or create sub-libraries only containing shRNAs involved in ATARiS solutions. As
RNAi libraries include more reagents per gene, and as screens include more samples, the
ability of ATARiS to correctly identify on-target reagents will also improve.
ATARiS gene phenotype values are an inherently different metric than previous gene scores for
RNAi. Existing methods determine gene candidates in a manner that is dependent on a user-
defined desired phenotype whereas ATARiS aims to summarize the data available for each
gene in an unbiased way. For example, with a “frequency approach,” it is theoretically possible
for a single gene in a sample to be a candidate for both a positive and a negative phenotype,
whereas the ATARiS phenotype value provides a single metric for that gene. For methods such
as RSA and SSMD, gene scores are influenced by the distribution of reagents towards or away
from a desired phenotype. In comparison, ATARiS attempts to determine the best subset of
reagents that describe the actual gene-level effect. Furthermore, it incorporates information
across all screened samples instead of using data from each sample independently. Another
major difference between ATARiS and previous work is that gene phenotype values are relative
to the samples screened instead of absolute, as is the case in RSA and SSMD. Finally, unlike
RNAi analysis methods for two-class comparisons such as RIGER (Barbie et al. 2009) and
“second best,” (Cheung et al. 2011) which are primarily used to determine a single value
representing each gene’s differential effect across classes, ATARiS phenotype values describe
the effect of each gene in each individual sample.
ATARiS is analogous to the approaches used by methods such as RMA (Irizarry et al. 2003)
and dChip (Li and Hung Wong 2001) for microarray data analysis in that gene scores are
evaluated by incorporating multiple probes/reagents and excluding problematic ones. It is
similar to dChip specifically in that a multiplicative model is fit to the set of probes/reagents.
However, ATARiS differs from both methods in order to account for the unique attributes of
RNAi data. For example, while most microarray probes are assumed to generally agree, the
majority of RNAi reagents do not. Supplementary Figure 12 shows how the correlation
coefficients between data from shRNA pairs targeting the same gene are only marginally higher
than the coefficients from random shRNA pairs. Thus, ATARiS implements an empirical null
distribution to determine correlations that are significantly above background. Another difference
is that ATARiS considers multiple solutions for each gene, as varying degrees of gene
suppression by distinct reagents may produce different effect profiles across samples.
We identified HNF1B as an oncogene by examining the correlation between each gene’s
expression and ATARiS gene solution in a sample-specific manner. On the other hand, when
we examined the correlation between measurements of each individual shRNA and
corresponding gene expression values, we found HNF1B shRNAs spread throughout the
ranked results: one ranked in the top 10 shRNAs, two in the top 500, and two ranked
considerably lower. In the same way that using co-regulated sets of genes, rather than
individual genes, can increase the signal in transcription profiling data (Subramanian et al.
2005), ATARiS uses multiple shRNAs, enriching for on target effects, to increase the signal from
individual RNAi reagents. Furthermore, experimental validation of HNF1B was made
straightforward by using ATARiS consistency scores to predict the shRNA reagents driving the
cellular phenotype and by using gene phenotype scores to identify specific samples to examine.
HNF1B is located near the known oncogene ERBB2. However, our observations indicate that
HNF1B independently induces cell transformation. It remains possible that HNF1B may
cooperate with ERBB2 to drive transformation in a manner analogous to what has been
observed for YAP1 and CIAP1, which reside in a single amplicon in hepatocellular cancer
(Zender et al. 2006). HNF1B has previously been described as an essential gene in ovarian
clear cell carcinoma where it is highly expressed (Tsuchiya et al. 2003), and genome-wide
association studies have associated SNPs in the HNF1B locus with risk for prostate and
endometrial cancers (Spurdle et al. 2011; Schumacher et al. 2011), although HNF1B may also
be epigenetically inactivated in certain contexts (Terasawa et al. 2006). Developmentally,
HNF1B is required for visceral endoderm formation (Barbacci et al. 1999) and proper
development of the genitourinary tract (Bellanne-Chantelot et al. 2005; Ryffel 2001), but
appears to be dispensable in adult tissue (Verdeguer et al. 2009), making it a reasonable
candidate for therapeutic targeting.
Although high-throughput shRNA viability screens are discussed here, ATARiS can be applied
to any screen where multiple, redundant reagents produce different observed outcomes in
multiple samples. For example, ATARiS would apply to a screen that uses siRNA reagents or
measures a phenotype other than viability. A similar approach can potentially be applied to
small molecule screening where multiple target-specific compounds are assayed across
different samples. In addition, since ATARiS analyzes the data of each gene independently, it
can be effectively applied to screens that target a small number of genes as long as multiple
samples are screened. One such example is validation screens, where screeners have prior
expectation that the reagents screened will produce an effect. By contrast, methods such as
RIGER (Barbie et al. 2009) construct a null distribution from all screened reagents, requiring
many reagents to be screened while the majority of them may have no effect on the measured
Despite the fact that parallel screens continue to grow in size, we recognize that not all datasets
will be as large as Achilles. When the sample size is small, the number of genes with solutions
decreases. This is due to the loss of genetic heterogeneity between samples and reduced
statistical power to discriminate true effects from noise. The user-defined significance threshold
allows for tailoring to specific applications. In our analyses, we chose a relatively lax threshold
(0.15) such that greater numbers of gene solutions will be available for analysis. As our
significance calculations are based on an empirical null distribution, a more stringent threshold
will provide increased confidence in the solutions generated (i.e., lower false-positive rate; see
also Supplementary Figure 2). Furthermore, the compatibility of results from independent
datasets suggests that investigators screening single or few samples may use ATARiS results
(such as consistency scores) derived from larger datasets to improve their ability to assess
reagent performance and gene effects in their screened samples.
We hope that by providing a foundation for interpreting RNAi gene suppression effects as
quantifiable values in individual samples, we will aid functional genomics in reaching its full
potential. We illustrated a number of analytic methods that are enabled by ATARiS, and note
that investigators have already begun to map phenotype-based gene networks (Amit et al.
2009; Horn et al. 2011). Nevertheless, much remains to be explored. ATARiS results from the
Achilles and Marcotte et al. datasets will be useful for deeper analysis, but we also believe that
the application of ATARiS to other screening datasets, large and small, will yield novel insights.
ATARiS is available online at http://broadinstitute.org/ataris.
Given the measurements of phenotypic effects produced by a set of RNAi reagents designed to
target the same gene G, ATARiS generates a consensus profile that represents the effect of
suppressing G in each screened sample relative to the other samples. Let n denote the number
of screened samples and p denote the number of reagents targeting G for which the
measurements are given. Let X denote a p × n matrix with each element xi, j representing the
observed phenotypic effect produced by reagent i in sample j. As we are only interested in
finding the relative effects of gene suppression, we median-center each row of X to obtain
X*= X − μ1n
1's of length n.
Let c denote a vector of length n representing the consensus profile for X* and let e denote a
vector of length p consisting of a relative effect size for each RNAi reagent. ATARiS models
each measurement xi, j
phenotypic effect cj, such that an approximation for X* is given by ˆX*= ecT, and we set
max(e)=1 for identifiability. We can then formulate the problem of finding the values for e and
T where μ is a vector of length p such that μi= median(xi,*) and 1n is a vector of
* as a product of its corresponding (unknown) relative effect size ei and
c as the following optimization problem:
1 subject to max(e)=1, where A1=
This criterion, which can also be seen as a rank-1 matrix factorization problem, although not
convex, is bilinear in c and e (i.e., with c fixed, it is linear in e and vice versa). To optimize it we
use the following iterative algorithm of alternating minimizations (Csiszar and Tusnady 1984):
We begin by initializing c with the mean values of X* in each sample:
We then update e and c repeatedly until convergence:
e ← argmineX*− ecT
c ← argmincX*− ecT
The elements of e and c are updated in an element-wise manner, i.e.,
ei← argminˆ e
for j ∈1, ? , n.
*− ˆ ecj
for i ∈1, ? , p
cj← argminˆ c
*− eiˆ c
for j ∈1, ? , n.
Each such assignment can be viewed as a problem of finding a weighted median, which can be
We cease iterating when a decrease of less than 1% in X*− ecT
and Marcotte et al. datasets, we found that convergence almost always occurs after fewer than
20 iterations. As this optimization problem is not convex, we are not guaranteed to find a global
minimum. To test the performance of the optimization in practice, we ran it multiple times with
random initialization values and found that the variations in the parameters estimated are
1 is observed. For the Achilles
minimal. Finally, to identify the solution we set e ←
max(e)⋅e and c ← max(e)⋅c.
Refinement of RNAi reagent subset
For each gene, ATARiS tries to identify subsets of its RNAi reagents that produce similar effects
across the screened samples. Given a set of reagents RG targeting gene G, we iteratively refine
RG until we identify a subset ˆRG⊆ RG that consists of reagents whose profiles (i.e., effects
across the samples) are all similar to the consensus profile computed for ˆRG. We then consider
ˆRG to be a consistent set and use its consensus profile as a gene solution, as described in the
main text. We begin by computing a consensus profile for the reagent set RG
evaluate the following criteria to determine whether RG
1. For each reagent r ∈RG
profile (xr,1,xr,2,? ,xr,n) and the consensus profile c must be greater than the 85th
percentile of the corresponding Spearman correlation coefficients similarily generated
from data of random reagent sets of size RG
depending on the user's desired confidence and properties of the data. See also
Supplementary Fig. 2 for an analysis of the influence of this threshold on the number of
2. All the reagents in RG
er≥ 0.3,∀r ∈RG
magnitudes, avoiding the inclusion of reagents whose effects are mainly due to noise
(assuming that noise magnitudes are similar across reagents).
If either criterion is not fulfilled we remove one reagent from the set RG
1. If any reagent r ∈ RG
effect magnitude er.
2. Otherwise, we discard the reagent r ∈RG
coefficient between its profile and the consensus profile c.
*= RG. We then
is a consistent set of reagents:
*, the Spearman correlation coefficient ρr between the reagent
*. (Note: The threshold can be adjusted
* must have a relative effect size er of at least 0.3, i.e.,
*. We therefore favor reagents whose effects have comparable
* as follows:
* does not satisfy criterion (2), we discard the one with the lowest
*with the lowest Spearman correlation
The refinement process is repeated until RG
in which case we conclude that there is no solution to the set RG of reagents. Our refinement
algorithm is greedy so that it is scalable and can be used to analyze RNAi screens performed
using reagent libraries that have a large number of reagents per gene.
* is consistent or until it consists of only one reagent,
Assignment of consistency scores
We determine a consistency score for each RNAi reagent of a given gene G based on its
similarity to other reagents targeting G. For a reagent r that is part of a consistent set ˆRG (and
hence was used to generated a gene solution), we determine the similarity of its profile
(xr,1,xr,2,? ,xr,n) to the profiles of all other reagents in ˆRG by computing the corresponding
Spearman correlation coefficients. We estimate a p-value for each correlation coefficient based
on an empirical null distribution of Spearman correlation coefficients of random pairs of reagent
profiles. We combine the p-values associated with r into a single significance estimate, p-valuer
using Stouffer’s method (Stouffer et al. 1949; Whitlock 2005). The consistency score of r is
defined as –log10 (p-valuer
For reagents that were not used to generate a gene solution we proceed as above to estimate a
consistency score and a p-value but use the set of all reagents that target gene G.
Analysis of Project Achilles dataset
Genome-scale pooled shRNA screens to identify genes essential for proliferation in 102 cancer
cell lines were performed using a lentivirally delivered pool of 54,020 shRNAs targeting 11,217
genes (Cheung et al. 2011). Each cell line was infected in quadruplicates and propagated for at
least 16 population doublings. The abundance of shRNA constructs was measured by
microarray hybridization and raw .CEL files from custom Affymetrix barcode arrays were
processed with a modified version of dCHIP software. ShRNAs that had an overlap of more
than 3 base pairs to other screened shRNAs were removed (n=679). The log2 fold change in
shRNA abundances for each cell line at the conclusion of the screening relative to the initial
plasmid DNA reference pool was calculated (Cheung et al. 2011). The log2 fold change data
were then normalized by a robust Z-score normalization (i.e., centering around the median and
scaling by the Median Absolute Deviation). The median value was used to collapse data from
replicates. The resulting dataset is provided as Supplementary Data 1. Each data point
represents the abundance of one shRNA construct within one cell line as compared to the initial
abundance of that shRNA construct in the initial plasmid DNA pool. The ATARiS analysis
ignored data for shRNAs targeting non-human genes (n=4) and genes targeted by only one
Analysis of Marcotte et al. dataset
Marcotte et al. (2012) performed genome-wide pooled shRNA screens to identify genes
essential for cancer cell survival and proliferation in 72 breast, pancreatic, and ovarian cancer
cell lines. They used a lentiviral shRNA library targeting ~16,000 genes with 78,432 shRNAs, of
which 50,981 shRNAs were also used in the Achilles screens. We obtained shRNA-level shARP
(shRNA Activity Ranking Profile) scores for all the shRNAs and cell lines screened through the
COLT-Cancer database (Koh et al. 2012) and considered them to represent the observed
phenotypic effects. We computed a robust Z-score for each cell line separately and discarded
data for two cell lines (OVCA1369_TR, HPDE) that showed aberrant score distributions. We ran
ATARiS on the normalized values using the same parameters used for the analysis of the
Achilles dataset. ATARiS found gene solutions for 8406 (54.4%) of the genes using data from
29731 (39.2%) of the shRNAs (Supplementary Table 3, Supplementary Data 6, 7).
All cancer cell lines were cultured in RPMI-1640 (Invitrogen) supplemented with 10% FBS
(Sigma) and 1% streptomycin and penicillin. HA1E and HA1EM immortalized lines were
cultured in alpha-MEM (Invitrogen) supplemented with 10% FBS.
Lentivirus containing shRNA targeting BRAF, PIK3CA, KRAS, MYC, HNF1B and controls
targeting GFP or LacZ for validation of ATARiS consistency scores were purchased directly
from The RNAi Consortium (Root et al. 2006) for gene suppression validation studies. Lentivirus
for KRAS and HNF1B dependency experiments were produced as previously described (Barbie
et al. 2009). See Supplementary Table 6 for detailed shRNA identities. Cells were infected in
media containing 8ug/ml polybrene and 1:10 dilution of virus. Infected cells were selected with
2ug/ml puromycin for 48 hours.
Low-throughput assessment of cell viability
Cells were replated at 50,000 cells/well post-infection and post-selection in triplicate in 12-well
plates post-selection with puromycin. Wells were counted 4 days later by ViaCell.
Cell lysates collected 72 hours post-infection were run on 4-12% Bis-Tris gel (Invitrogen
NuPAGE) and transferred to nitrocellulose membrane for immunoblotting. Primary antibodies
were obtained from Santa Cruz (KRAS sc-30, BRAF sc-5284, MYC sc-764, HNF1B sc-7411, ß-
actin sc-1615) and Cell Signaling (PI3Kinase 110alpha #4255). Immunoblots for BRAF and
PIK3CA protein were visualized by infrared imaging (LICOR). Quantification was performed by
ImageJ software (http://rsb.info.nih.gov/ij).
For each 2-class comparison, ATARiS gene phenotype values were used to calculate a mean
for each class for each gene solution. The difference of means between the classes was used
as a scoring metric and p-values were estimated based on a null distribution generated by
50,000 class label permutations. Q-values were generated by the Benjamini-Hochberg method
(Benjamini and Hochberg 1995).
Annotation of cell line genomic features
We constructed a matrix of genomic features for cell lines that had matched genomic data from
the Cancer Cell Line Encyclopedia (CCLE, Barretina et al. 2012). As previously described,
features include mutational status, tumor tissue lineage, regions of recurrent copy-number gain
or loss (derived from GISTIC), and combined gene mutation and copy number amplification (for
oncogenes) or combined mutation and copy number deletion (for tumor suppressors). GISTIC
regions were assessed across all available CCLE cell lines, of which 76 were screened in
Achilles. Amplification and deletion of specific genes were defined by relative log fold copy-
number value greater than 0.25 or less than -0.25, respectively. All data are represented as
binary values, with 1 representing presence of the indicated feature in the sample. Refer to
Supplementary Data 5 for full feature matrix.
Anchorage Independent Growth Assay
HA1EM cells infected with lentiviral expression plasmid pLX-304 with desired genes were
selected for 5 days in 10ug/ml blasticidin. Cells were seeded in triplicate at 2.5x104 cells per well
in 0.4% top agar (Difco) in 6-well plates. Bottom agar was 0.6% agar (Difco) supplemented with
20% FBS. Macroscopic images were collected of each well and colonies were counted using
HT29 infected with lentiviral plasmid PLKO.1 shHNF1B-1 or shControl were expanded for 4
days before subcutaneous implantation into immunocompromised mice (Taconic, CrTac:NCr-
Foxn1nu). 2x106 cells were implanted into each of three sites per mouse. Tumor growth was
monitored every two weeks by digital caliper measurement of tumor diameter. Approximate
cross-sectional area was calculated.
ATARiS can be run online on user-provided data through the GenePattern computational
genomics suite (Reich et al. 2006)
(http://broadinstitute.org/ataris). The website also includes all datasets used to obtain the results
described in this manuscript.
accessible on the ATARiS website
We thank the Broad Institute Project Achilles investigators for the data from RNAi pooled
screening, and the Broad-Novartis Cancer Cell Line Encyclopedia investigators for providing cell
lines for validation. We thank D. Nijhawan, J. Rosenbluh, D. Barbie, and F. Verdeguer for
helpful discussions. We thank L. Solomon and L. Gaffney for assistance with figure design. This
work was supported by grants from the NIH/NCI (RC2 CA148268, P01 CA050661, U54
CA112962, R01 CA109467), the H.L. Snyder Medical Foundation, and the Ivy Research
A.T., D.D.S., S.G., B.A.W., A.A.M., P.T., D.E.R., W.C.H., and J.P.M. conceptualized the
W.C.H. and J.P.M. directed the work.
A.T. implemented the method.
D.D.S. performed validation experiments.
A.T., D.D.S., B.A.W., S.G., N.S., S.E.S., T.I.Z., R.B. performed computational analyses.
N.S., L.A.G., S.G. and D.E.R. contributed data.
A.T., D.D.S., W.C.H. and J.P.M. wrote the manuscript.
B.A.W., P.T., A.A.M., N.S., and D.E.R. edited the manuscript.
Competing Financial Interests
W.C.H. and R.B. are consultants for Novartis Pharmaceuticals.
Figure 1: ATARiS accounts for patterns in RNAi reagent data in order to quantify
phenotypic effect of gene suppression in each sample.
(a-d) Hypothetical phenotypic data from four RNAi reagents, all designed to target the same
gene, in five independent samples from two classes A and B. (a) Samples 1, 2, and 3 each
have at least two reagents that score below a desired threshold (purple dotted line), thus,
according to “frequency approach” methods, this gene may be a "hit" in those samples. (b) A
line connecting each reagent’s effects across the samples reveals additional information.
Specifically, we note that it is possible (as in this scenario) that different shRNAs drive the
determination of hits in each sample when samples are each analyzed separately as in (a). (c)
For each reagent, the difference between its mean values in class A and class B is shown,
reducing much of the noise from individual samples. Reagents 3 and 4 both show differential
effects between the classes and would suggest that two-class-based analytic methods select
this gene as a hit. (d) ATARiS phenotype values for each of the screened samples. Phenotype
values represent relative gene-level effects in each individual sample by incorporating
information from trends across all samples, favoring reagents that produce correlated effects
(i.e., reagents 1 and 2 from (b)). If the user chooses to assess whether differential effects exist
between class A and B, this example would show no significant difference by avoiding
uncorrelated reagents 3 and 4. (e) Real data from the Achilles dataset for shRNAs targeting
BRAF. Median-normalized screening data across 102 samples are displayed as barplots in
sample order of ascending BRAF phenotype value. Boxed numbers display -log10 p-value of the
Spearman correlation coefficient for the two shRNA labeled in the corresponding row and
column. Red, shRNAs with correlated effects that are incorporated into the BRAF gene solution.
Figure 2: Influence of dataset size and context on ATARiS results.
(a, b) Robustness of ATARiS for datasets of smaller sample size. ATARiS results from 100 sets
of randomly selected samples for each indicated sample size was compared to ATARiS results
from the full 102-sample Achilles dataset. We determined the percentage of genes with a
solution in the Achilles dataset results that are also represented in results from fewer samples
(a). For 100 randomly selected genes, we also compared phenotype values in each sample to
the values generated by ATARiS for the corresponding sample when using the full 102-sample
dataset (b). Standard deviations are based on phenotype values across all 102 samples for
each gene independently. For each gene, differences between phenotype values from smaller
datasets compared to the full dataset are depicted in standard deviation units. (c) The
robustness of shRNA selection by ATARiS is demonstrated by simulating independent
screening datasets. One hundred pairs of disjoint sets of samples were randomly generated for
the sample sizes indicated. Each set was independently analyzed by ATARiS. For each pair of
sets, the overlap in shRNA used to generate solutions was determined. Boxplot displays the
size of the overlap for each pair as a fraction of the average number of shRNAs used in the
analysis of each set. P-value < 2.2x10-16 for all results, χ2 test of independence. (d) ATARiS
was used to analyze sets of samples that harbor either wild-type BRAF only, or an equal
number of samples that harbor wild-type and mutant BRAF. One hundred randomly generated
sets of samples were analyzed for each sampling size. The percentage of sets for which a
BRAF solution was found is shown.
Figure 3: ATARiS consistency scores are associated with on-target gene suppression.
Consistency scores computed by ATARiS and corresponding protein suppression levels by
immunoblotting are shown for shRNAs targeting (a) BRAF and (b) PIK3CA. A higher
consistency score represents greater confidence that the effects produced by the shRNA are
due to suppression of the target gene. Immunoblotting for the effect of each shRNA compared
to control shRNA was performed in cell line A549 and percent suppression compared to control
shRNA was calculated based upon quantification by ImageJ software. Shading of axis labels
correspond to data bars of the same type. *Asterisks indicate reagents used in the gene’s
ATARiS gene solution.
Figure 4: ATARiS gene phenotype values reflect biological dependencies.
(a) Correspondence between gene mutation status and ATARiS phenotype values for BRAF,
PIK3CA, and KRAS. Each vertical bar represents a single screened sample, colored by
mutation status. In each plot, samples are ordered by increasing phenotype values. AUC, area
under receiver operating characteristic curve. P-value, assessed by Mann-Whitney test. (b)
Low-throughput validation of the relationship between gene phenotype scores and gene
dependency. Six cell lines infected with shKRAS were counted four days post-selection to
determine cell number relative to infection with control shRNA. Immunoblots were performed
using lysates from each sample collected at 2-days post-selection and stained using primary
antibodies from Santa Cruz Biotechnology KRAS (sc-30) or actin (sc-1615). Immunoblot lanes
correspond to bars in the graph directly above. Horizontal bar orders all cell lines with known
KRAS mutation status in increasing order by ATARiS phenotype value, with validated samples
marked by corresponding triangles. Grey, KRAS wild-type. Black, KRAS mutant. Error bars, ±
one standard deviation (n=3). n.s., non-specific band. (c) Genes differentially required in sample
classes defined by recurrent amplification or deletion peaks. Recurrent genomic peaks were
identified by GISTIC analysis across genomic data for samples from the Cancer Cell Line
Encyclopedia. For each peak existing in at least six samples screened in Project Achilles
(n=101), two classes of samples were defined based on presence or absence of the peak.
Genes that are differentially required in samples harboring the peak as compared to samples
that do not (FDR < 0.25) were determined. The distribution of the number of significantly
differential genes is shown. For comparison, the same analysis was performed using classes
defined by random permutation of peak assignments.
Figure 5: ATARiS phenotype values enable phenotype-based analyses for biological
(a) Identifying genomic predictors of dependency on E2F3. Genomic features are shown
ranked by their correspondence to E2F3 phenotype values as measured by area under receiver
operating characteristic curve (AUC). Amplification and deletion peaks, as determined by
GISTIC, are denoted by “Amp” and “Del,” respectively. Columns correspond to individual cell
lines. (b) Correlations between gene phenotype value profiles to CCND1 gene solution. Gene
solutions are ranked by their similarity to the CCND1 gene solution using Pearson correlation
coefficient. P-values were generated by permutation of sample labels. (c) Identifying significant
cancer genes by integrating expression data and phenotype values. Gene solutions are ranked
by increasing Pearson correlation coefficient between the solution and expression data for the
corresponding gene. Thus, genes that are essential in samples with high expression and less
essential in samples with low expression are more negatively correlated, and receive higher
ranks. P-values were calculated from a null distribution derived by permutation of sample labels.
Numbers following gene names in (b, c) indicate gene solution number (see Supplementary
Data). FDR, false discovery rate. Red, previously reported gene dependencies in cancer.
Figure 6: Characterizing the role of HNF1B in cancer
(a) Immunoblot of HNF1B after expression of five independent shRNA designed to target
HNF1B. The two shRNA incorporated into the ATARiS solution, which also have the highest
consistency scores, are indicated by asterisks. (b) Cell viability upon exogenous expression of
HNF1B or GFP in an HNF1B-sensitive cell line OE33 with stable integration of doxycycline-
inducible expression of shHNF1B-1. Each bar in the graph corresponds to the immunoblot lane
directly below. (c) Relative viability of a panel of cell lines upon suppression of control or two
HNF1B-specific shRNAs. Cell lines with high levels of HNF1B are shown in bold text. Each bar
in the graph corresponds to the immunoblot lane directly below. Each boxed image derives from
a separately exposed gel, as the HNF1B-amplified samples express much higher endogenous
levels of HNF1B (Supplementary Fig. 10). Data for HT29 is shown in panel a. (d) HNF1B
sensitive cell line HT29 expressing shHNF1B-1 or shControl was implanted subcutaneously into
immunocompromised mice. ShHNF1B-1 was used for all experiments since it has potent effects
and is specific for HNF1B, as shown in panels a and b. Tumor volume was monitored bi-weekly
and lysates were collected pre-implantation and from tumors formed at 4 weeks. * indicates p-
values < 0.05, ** indicates p-value < 0.01 (one-tailed Student’s t-test) (e) HNF1B or LacZ was
expressed in HA1EM cells and anchorage independent growth determined. Representative
photos shown after six weeks. Error bars, ± one standard deviation (n=3).
Amit I, Garber M, Chevrier N, Leite AP, Donner Y, Eisenhaure T, Guttman M, Grenier JK, Li W, Zuk O, et
al. 2009. Unbiased reconstruction of a mammalian transcriptional network mediating pathogen
responses. Science 326: 257–263.
Barbacci E, Reber M, Ott MO, Breillat C, Huetz F, Cereghini S. 1999. Variant hepatocyte nuclear factor 1
is required for visceral endoderm specification. Development 126: 4795–4805.
Barbie D, Tamayo P, Boehm J, Kim S, Moody S, Dunn I, Schinzel A, Sandy P, Meylan E, Scholl C, et al.
2009. Systematic RNA interference reveals that oncogenic KRAS-driven cancers require TBK1.
Bard F, Casano L, Mallabiabarrena A, Wallace E, Saito K, Kitayama H, Guizzunti G, Hu Y, Wendler F,
Dasgupta R, et al. 2006. Functional genomics reveals genes involved in protein secretion and Golgi
organization. Nature 439: 604–607.
Barretina J, Caponigro G, Stransky N, Venkatesan K, Margolin AA, Kim S, Wilson CJ, Lehár J, Kryukov
GV, Sonkin D, et al. 2012. The Cancer Cell Line Encyclopedia enables predictive modelling of
anticancer drug sensitivity. Nature 483: 603–607.
Bellanne-Chantelot C, Clauin S, Chauveau D, Collin P, Daumont M, Douillard C, Dubois-Laforgue D,
Dusselier L, Gautier JF, Jadoul M, et al. 2005. Large Genomic Rearrangements in the Hepatocyte
Nuclear Factor-1 (TCF2) Gene Are the Most Frequent Cause of Maturity-Onset Diabetes of the
Young Type 5. Diabetes 54: 3126–3132.
Benjamini Y, Hochberg Y. 1995. Controlling the false discovery rate: a practical and powerful approach to
multiple testing. Journal of the Royal Statistical Society Series B.
Beroukhim R, Getz G, Nghiemphu L, Barretina J, Hsueh T, Linhart D, Vivanco I, Lee JC, Huang JH,
Alexander S, et al. 2007. Assessing the significance of chromosomal aberrations in cancer:
methodology and application to glioma. PNAS 104: 20007–20012.
Beroukhim R, Mermel CH, Porter D, Wei G, Raychaudhuri S, Donovan J, Barretina J, Boehm JS, Dobson
J, Urashima M, et al. 2010. The landscape of somatic copy-number alteration across human cancers.
Nature 463: 899–905.
Birmingham A, Anderson EM, Reynolds A, Ilsley-Tyree D, Leake D, Fedorov Y, Baskerville S, Maksimova
E, Robinson K, Karpilow J, et al. 2006. 3' UTR seed matches, but not overall identity, are associated
with RNAi off-targets. Nature Methods 3: 199–204.
Boehm J, Zhao J, Yao J, Kim S. 2007. Integrative genomic approaches identify IKBKE as a breast cancer
Brough R, Frankum JR, Sims D, Mackay A, Mendes-Pereira AM, Bajrami I, Costa-Cabral S, Rafiq R,
Ahmad AS, Cerone MA, et al. 2011. Functional Viability Profiles of Breast Cancer. Cancer Discov 1:
Bussey KJ, Chin K, Lababidi S, Reimers M, Reinhold WC, Kuo W-L, Gwadry F, Ajay, Kouros-Mehr H,
Fridlyand J, et al. 2006. Integrating data on DNA copy number with gene expression levels and drug
sensitivities in the NCI-60 cell line panel. Mol Cancer Ther 5: 853–867.
Cheung HW, Cowley GS, Weir BA, Boehm JS, Rusin S, Scott JA, East A, Ali LD, Lizotte PH, Wong TC, et
al. 2011. Systematic investigation of genetic vulnerabilities across cancer cell lines reveals lineage-
specific dependencies in ovarian cancer. PNAS 108: 12372–12377.
Chung N, Zhang XD, Kreamer A, Locco L, Kuan P-F, Bartz S, Linsley PS, Ferrer M, Strulovici B. 2008.
Median absolute deviation to improve hit selection for genome-scale RNAi screens. Journal of
biomolecular screening : the official journal of the Society for Biomolecular Screening 13: 149–158.
Collinet C, Stöter M, Bradshaw CR, Samusik N, Rink JC, Kenski D, Habermann B, Buchholz F, Henschel
R, Mueller MS, et al. 2010. Systems survey of endocytosis by multiparametric image analysis. Nature
Csiszar I, Tusnady G. 1984. Information geometry and alternating minimization procedures. Statistics and
Decisions Supplement: 205–237. http://research.microsoft.com/apps/pubs/default.aspx?id=64832.
Cullen BR. 2006. Enhancing and confirming the specificity of RNAi experiments. Nature Methods 3: 677–
Dyson N. 1998. The regulation of E2F by pRB-family proteins. Genes & development.
Echeverri C, Beachy P, Baum B, Boutros M. 2006. Minimizing the risk of reporting false positives in large-
scale RNAi screens. Nature Methods.
Elbashir SM, Harborth J, Lendeckel W, Yalcin A, Weber K, Tuschl T. 2001. Duplexes of 21-nucleotide
RNAs mediate RNA interference in cultured mammalian cells. Nature 411: 494–498.
Garraway L, Widlund H, Rubin M, Getz G. 2005. Integrative genomic analyses identify MITF as a lineage
survival oncogene amplified in malignant melanoma. Nature.
Hahn WC, Counter CM, Lundberg AS, Beijersbergen RL, Brooks MW, Weinberg RA. 1999. Creation of
human tumour cells with defined genetic elements. Nature 400: 464–468.
Hirsch AJ. 2010. The use of RNAi-based screens to identify host proteins involved in viral replication.
Future Microbiol 5: 303–311.
Horn T, Sandmann T, Fischer B, Axelsson E. 2011. Mapping of signaling networks through synthetic
genetic interaction analysis by RNAi. Nature Methods.
Irizarry RA, Bolstad BM, Collin F, Cope LM, Hobbs B, Speed TP. 2003. Summaries of Affymetrix
GeneChip probe level data. Nucleic Acids Res 31.
Jackson AL, Bartz SR, Schelter J, Kobayashi SV, Burchard J, Mao M, Li B, Cavet G, Linsley PS. 2003.
Expression profiling reveals off-target gene regulation by RNAi. Nature Biotechnology 21: 635–637.
Jackson AL, Burchard J, Schelter J, Chau BN, Cleary M, Lim L, Linsley PS. 2006. Widespread siRNA
“off-target” transcript silencing mediated by seed region sequence complementarity. RNA 12: 1179–
Kittler R, Pelletier L. 2008. Systems biology of mammalian cell division. Cell Cycle.
Koh JLY, Brown KR, Moffat J, 6. 2012. COLT-Cancer: functional genetic screening resource for essential
genes in human cancer cell lines. Nucleic Acids Research 40: D957–D963.
König R, Chiang C-Y, Tu BP, Yan SF, Dejesus PD, Romero A, Bergauer T, Orth A, Krueger U, Zhou Y, et
al. 2007. A probability-based approach for the analysis of large-scale RNAi screens. Nature Methods
Leone G, Sears R, Huang E, Rempel R, Nuckolls F, Park CH, Giangrande P, Wu L, Saavedra HI, Field
SJ, et al. 2001. Myc requires distinct E2F activities to induce S phase and apoptosis. Mol Cell 8:
Li C, Hung Wong W. 2001. Model-based analysis of oligonucleotide arrays: model validation, design
issues and standard error application. Genome Biology 2: RESEARCH0032.
Li CG, Nyman JE, Braithwaite AW, Eccles MR. 2011. PAX8 promotes tumor cell growth by
transcriptionally regulating E2F1 and stabilizing RB protein. Oncogene.
Liu JJ, Chao JR, Jiang MC, Ng SY, Yen JJ, Yang-Yen HF. 1995. Ras transformation results in an
elevated level of cyclin D1 and acceleration of G1 progression in NIH 3T3 cells. Mol Cell Biol 15:
Luo B, Cheung HW, Subramanian A, Sharifnia T, Okamoto M, Yang X, Hinkle G, Boehm JS, Beroukhim
R, Weir BA, et al. 2008. Highly parallel identification of essential genes in cancer cells. PNAS 105:
Marcotte R, Brown KR, Suarez F, Sayad A, Karamboulas K, Krzyzanowski PM, Sircoulomb F, Medrano
M, Fedyshyn Y, Koh JLY. 2012. Essential Gene Profiles in Breast, Pancreatic, and Ovarian Cancer
Cells. Cancer Discov 2: 172–189.
Marine S, Bahl A, Ferrer M, Buehler E. 2012. Common seed analysis to identify off-target effects in siRNA
screens. J Biomol Screen 17: 370–378.
Mason SJ, Graham NE. 2002. Areas beneath the relative operating characteristics (ROC) and relative
operating levels (ROL) curves: Statistical significance and interpretation. Q J R Meteorol Soc 128:
Musgrove EA, Caldon CE, Barraclough J, Stone A, Sutherland RL. 2011. Cyclin D as a therapeutic target
in cancer. Nature Reviews Cancer 11: 558–572.
Müller P, Kuttenkeuler D, Gesellchen V, Zeidler MP, Boutros M. 2005. Identification of JAK/STAT
signalling components by genome-wide RNA interference. Nature 436: 871–875.
Oeggerli M, Schraml P, Ruiz C, Bloch M, Novotny H, Mirlacher M, Sauter G, Simon R. 2006. E2F3 is the
main target gene of the 6p22 amplicon with high specificity for human bladder cancer. Oncogene 25:
Ramsay R. 2008. MYB function in normal and cancer cells. Nature Reviews Cancer.
Reich M, Liefeld T, Gould J, Lerner J, Tamayo P, Mesirov JP. 2006. GenePattern 2.0. Nat Genet 38:
Root DE, Hacohen N, Hahn WC, Lander ES, Sabatini DM. 2006. Genome-scale loss-of-function
screening with a lentiviral RNAi library. Nature Methods 3: 715–719.
Ryffel GU. 2001. Mutations in the human genes encoding the transcription factors of the hepatocyte
nuclear factor (HNF)1 and HNF4 families: functional and pathological consequences. J Mol
Endocrinol 27: 11–29.
Schumacher FR, Berndt SI, Siddiq A, Jacobs KB, Wang Z, Lindstrom S, Stevens VL, Chen C, Mondul
AM, Travis RC, et al. 2011. Genome-wide association study identifies new prostate cancer
susceptibility loci. Human Molecular Genetics 20: 3867–3875.
Sigoillot FD, Lyman S, Huckins JF, Adamson B. 2012. A bioinformatics method identifies prominent off-
targeted transcripts in RNAi screens. Nature Methods.
Spurdle AB, Thompson DJ, Ahmed S, Ferguson K, Healey CS, O'Mara T, Walker LC, Montgomery SB,
Dermitzakis ET, Fahey P, et al. 2011. Genome-wide association study identifies a common variant
associated with risk of endometrial cancer. Nat Genet 43: 451–454.
Stouffer SA, Suchman EA, DeVinney LC, Star SA, Williams RMJ. 1949. The American Soldier, Vol. 1:
Adjustment during Army Life. 1st ed. Princeton U. Press.
Subramanian A, Tamayo P, Mootha VK, Mukherjee S, Ebert BL, Gillette MA, Paulovich A, Pomeroy SL,
Golub TR, Lander ES, et al. 2005. From the Cover: Gene set enrichment analysis: A knowledge-
based approach for interpreting genome-wide expression profiles. 102: 15545–15550.
Terasawa K, Toyota M, Sagae S, Ogi K, Suzuki H, Sonoda T, Akino K, Maruyama R, Nishikawa N, Imai
K, et al. 2006. Epigenetic inactivation of TCF2 in ovarian cancer and various cancer cell lines. Br J
Cancer 94: 914–921.
Tsuchiya A, Sakamoto M, Yasuda J, Chuma M, Ohta T, Ohki M, Yasugi T, Taketani Y, Hirohashi S. 2003.
Expression profiling in ovarian clear cell carcinoma: identification of hepatocyte nuclear factor-1β as
a molecular marker and a possible molecular target for therapy of ovarian clear cell carcinoma. The
American journal of pathology 163: 2503–2512.
Verdeguer F, Le Corre S, Fischer E, Callens C, Garbay S, Doyen A, Igarashi P, Terzi F, Pontoglio M.
2009. A mitotic transcriptional switch in polycystic kidney disease. Nat Med 16: 106–110.
Whitlock MC. 2005. Combining probability from independent tests: the weighted Z-method is superior to
Fisher's approach. Journal of Evolutionary Biology 18: 1368–1373.
Zender L, Spector MS, Xue W, Flemming P, Cordon-Cardo C, Silke J, Fan S-T, Luk JM, Wigler M,
Hannon GJ, et al. 2006. Identification and Validation of Oncogenes in Liver Cancer Using an
Integrative Oncogenomic Approach. Cell 125: 1253–1267.
Zhang XD, Ferrer M, Espeseth AS, Marine SD, Stec EM, Crackower MA, Holder DJ, Heyse JF, Strulovici
B. 2007. The use of strictly standardized mean difference for hit selection in primary RNA
interference high-throughput screening experiments. Journal of biomolecular screening : the official
journal of the Society for Biomolecular Screening 12: 497–509.
Zhang XD, Santini F, Lacson R, Marine SD, Wu Q, Benetti L, Yang R, McCampbell A, Berger JP, Toolan
DM, et al. 2011. cSSMD: Assessing Collective Activity for Addressing Off-Target Effects in Genome-
Scale RNA Interference Screens. Bioinformatics.
Relative phenotype score
Class AClass B
Difference between class A and B
Reagent 1 Reagent 2 Reagent 3 Reagent 4
10 2030 4050 60708090 100
Percent shRNA shared between disjoint subsets
Number of cell lines per sampling
Percent of runs containing a BRAF solution
Wild type only
1:1 mutant : wild type
Number of cell lines per sampling
Number of cell lines per sampling
Number of cell lines per sampling
Percent genes with a solution
(compared with full 102-sample data)
Standard deviation from phenotype values
derived from full 102-sample data
* Part of a gene solution
2.5 Consistency score
% Protein suppressionConsistency score
% Protein suppression
% Protein suppression
Classes defined by GISTIC peaks
Classes defined by random peak permutation
Number of genes differentially required between classes
ATARiS phenotype value
ATARiS phenotype value
ATARiS phenotype value
AUC = 0.897
p value = 2.36e-5
AUC = 0.823
p value = 1.78e-3
AUC = 0.813
p value = 2.45e-6
Mutation / copy number alteration
Lung Small Cell
Rank Solution CorrelationFDR
1 CCL8_1 0.590 0.010
2 KRAS_2 0.566 0.010
3 64748_1 0.565 0.010
4 KRAS_1 0.557 0.010
5 CRX_1 0.543 0.010
6 RPS11_1 0.543 0.010
7 Gpr12_1 0.538 0.012
8 CDK6_2 0.530 0.012
9 RAF1_1 0.524 0.010
10 HIATL1_1 0.516 0.012
RankSolutionp value FDR
1 HNF1B_1 2.00E-05 0.075
2 PAX8_1 2.00E-05 0.075
3 E2F3_1 4.00E-05 0.075
3 ELF3_1 4.00E-05 0.075
5 SOX10_1 6.00E-05 0.075
5 HIST1H4D_1 6.00E-05 0.075
7 NGEF_1 8.00E-05 0.086
8 FERMT1_1 1.00E-04 0.094
9 BCL2L1_1 1.40E-04 0.096
9 ASL_1 1.40E-04 0.096
9 POLE3_1 1.40E-04 0.096
12 MYB_1 3.40E-04 0.213
13 MPP6_1 4.00E-04 0.225
14 PITX3_1 4.20E-04 0.225
15 HNF4A_1 4.60E-04 0.230
E2F3 phenotype value
CCND1 phenotype value
0 Download full-text
α - HNF1B
α - actin
α - HNF1B
α - actin
Week 0 Week 2 Week 4
Tumor size (mm2
α - HNF1B
α - actin
Number of colonies