Content uploaded by Marc Helbling
Author content
All content in this area was uploaded by Marc Helbling
Content may be subject to copyright.
Measuring Party Positioning and Issue Salience with Media-Data:
Characteristics and Research Questions
Marc Helbling
Department of political science
University of Zurich
helbling@ipz.uzh.ch
Anke Tresch
Department of political science
University of Geneva
anke.tresch@unige.ch
Abstract: The aim of this paper is to explore the characteristics of media data when
we measure party positions and issue salience. We thereby contribute in developing
new methodologies to analyze political parties and in operationalizing new theoreti-
cal models of political competition. Following other studies that have compared in-
dicators based on different data, we seek to highlight the characteristics of media
data in comparison with the more traditional manifesto- and expert-data. We will
show that media-data measure similar dimensions regarding positions, but salience
constructs that are different from other data. Contrary to our expectation, media data
do not better reflect how ordinary citizens perceive political parties. We however
have reasons to believe that media-data are a finer-grained instrument and more sen-
sitive than other data to explore short-term changes, diverging positions over sub-
issues and intra-party dissent.
Paper prepared for presentation at the annual meeting of the Swiss Political Science
Association, University of St. Gallen, January 8, 2009
Introduction1
Over the past twenty-five years, methodological research on the measurement of political par-
ties' policy positions has been continually growing. This scholarly interest in developing new
methodologies to locate political parties in policy and/or ideological spaces is motivated by
the need to operationalize a range of new and fairly sophisticated theoretical models of politi-
cal competition (Laver 2001: 6). Some of these models are not only concerned with issue po-
sitions, but also with issue salience, that is the relative importance of particular issues to some
parties. The underlying idea is the assumption that party competition is not mainly a direct
confrontation of opposing positions on the same issues, but that parties compete by emphasiz-
ing those issues on which they hold comparative advantages (e.g., Budge and Farlie 1983).
There is a wide variety of methods to generate data on party positions and issue salience, but
one can draw a basic distinction between survey data and document-driven data (Keman
2007: 77). Among the former, expert judgments are certainly the typical example, among the
latter, the coding of party manifestos is the dominant approach. Expert surveys and party
manifestos have become standard techniques to estimate party positions and issue salience,
and both approaches have given rise to flourishing networks of international research collabo-
ration. More recently, scholars have made some attempts at comparing these two approaches
systematically by cross-validating their respective measures and indicators (e.g., Marks et al.
2007; Netjes and Binnema 2007; Ray 2007; Steenbergen and Marks 2007). Most often, these
cross-validations have focused on the left-right dimension or on the issue of European integra-
tion.
In this paper, we concentrate on European integration and introduce media data as an addi-
tional source for the measurement of party positions and issue salience. While the use of me-
dia data has a long tradition in social movement research, the media have been an underused
data source in the analysis of party positions (see however Kriesi et al. 2008, Statham et al.
2008). Recently, however, eminent party specialists such as Peter Mair (2006: 162) have
compellingly argued that the literature on European integration has
"relied perhaps too heavily on standardized quantitative variables that can be used
directly in quite highly-abstracted cross-national research. One example of this is
1 Previous versions of this paper were presented at the staff seminar of the Department of Political Science,
University of Geneva and in the Social Statistic Speakers Series of the Department of Political Science, McGill
University, Montreal. We thank seminar participants for valuable comments and suggestions.
the research effort that has been made to identify party positions on Europe […]
In some cases, this research has relied on the sort of crude but easily accessible
data provided by expert judgments (Ray 1999; Hooghe et al 2004), while in others
it has been based on analyses of the contents of party programs […] What is
really needed here, however, particularly given that this is a new and often ex-
ploratory avenue of research, is a much more systematic, inductive, and largely
bottom-up comparison of political discussions at the national level – whether as
revealed in parliamentary debates, or in contests surrounding European referen-
dums, or in the ebb and flow of the arguments used in national election campaigns
[…]".
Media data are a means to do exactly this, and to examine how Europe actually plays in po-
litical party competition at the national level. In a similar vein, Netjes and Binnema (2007: 42,
48) call for the cross-validation of traditional salience measures based on expert surveys and
party manifestos with "a 'harder' measurement of salience, utilizing content analysis of na-
tional and EP election campaigns in the printed media". While Kriesi (2007: 92) has made a
first step in this direction using media data and party manifestos, he neither reports his results
in detail nor does he offer a systematic comparison between various methods and data
sources.
Against this background, the aim of this paper is to introduce content coding of the media as
an alternative method for the estimation of party positions and issue salience. We concentrate
on the issue of European integration mainly for practical reasons such as the availability of
empirical data and the comparability with previous studies on the validity of indicators de-
rived from different methods. Moreover, as the above-cited statement by Peter Mair illus-
trates, a growing literature is concerned with how national parties adapt to European integra-
tion, and it is therefore important to think about the characteristics and comparative advan-
tages of different indicators used to measure party positions and issue salience in this particu-
lar policy field.
This paper is structured as follows. We first introduce two different techniques to derive indi-
cators of party positions and issue salience from the media—the core sentence approach and
political claims analysis. In relation with manifesto- and expert-data we then discuss the char-
acteristics of media data and formulate on this basis some arguments in which situation and
for which research questions media-data are more suitable than the traditionally used data.
3
Our analyses show that media-data capture similar information about party positions as other
indicators, but that they measure different underlying constructs when it comes to issue sali-
ence. Contrary to our expectation, media-data do not better reflect how ordinary citizens per-
ceive political parties. We however have reasons to believe that media-data are a finer-grained
instrument and more sensitive than other data to explore short-term changes, positions over
sub-issues and intra-party dissent.
Media data
Over the last two decades, newspapers have become a primary data source in various study
fields (e.g., Ferree et al. 2002; Koopmans et al. 2005; Koopmans 2007; Kriesi et al. 1995;
Kriesi et al. 2008; Trenz 2005; de Vreese 2003). There is little doubt in the literature that the
mass media constitute the most important arena for public debates on politically relevant is-
sues (see Ferree et al. 2002; Bennett et al. 2004). An analysis of newspaper articles allows us
to analyze the statements of political actors as they can be perceived by their opponents, pol-
icy makers and the broader audience.
We rely on two different approaches: the core sentence approach and political claims analysis.
Both methods rest on newspapers to generate indicators of party positions and issue salience
and can be characterized as "relational content analysis" (Kleinnijenhuis and Pennings 2001:
163). Another commonality of these two approaches is that their unit of analysis is not the
article but, as we will see, elements within articles, namely core sentences or political claims.
Having two kinds of data at our disposition, we are in a better position to generalize our ar-
guments and to make sure that our findings do not depend on specific coding methods. In the
following, we mainly discuss those aspects of the two methods that directly concern the
measurement of party positions and issue salience. Additional facets regarding the coding of
variables that are not relevant for our purposes will be left out. For extensive methodological
discussions see Kleinnijenhuis et al. (1997) for the core sentence approach and Koopmans and
Statham (1999) for the political claims analysis.
Core sentence approach (CSA)
The core (or nuclear) sentence approach has its origins in early theoretical elaborations by
Wittgenstein (1984 [1921]) and was first practically implemented in concrete coding instruc-
tions by Osgood (1956) and Axelrod (1976). Recently, it has been revived for the analysis of
4
party systems (Kleinnijenhuis et al. 1997). Additionally, Franzosi (2004: 60f.) has provided
theoretical and empirical evidence that the method – which he calls ‘story grammars’ – is a
useful device for the social sciences in general. ‘Core sentences’ represent an inductive ap-
proach that aims to capture the full complexity of the political debate without imposing strong
theoretical expectations (e.g. in terms of previously fixed category schemas) on the data.
The basic idea of this method comes in the notion that the content of every written document
can be described as a network of relationships between objects. To analyze party positions,
for example, every relationship between 'political objects' (i.e. between a political actor and a
political issue) that appears in the text is coded. Each sentence is reduced to its most basic
structure (the so-called ‘core sentence'), indicating only its subject (political actor) and its
object (issue), as well as the direction of the relationship between the two. In substantive
terms, such a core sentence represents an actor’s opinion on an issue.
Let us give a short, illustrative example of what core sentences are: “Party X supports the
European Constitution but opposes EU accession of Turkey”. This grammatical sentence con-
sists of two core sentences. While the subject is the same in both sentences, its object and di-
rection change (see Table 1). It appears that a core sentence is always embedded in a gram-
matical sentence and that a grammatical sentence might consist of several core sentences.
Table 1: Examples of core sentences
Core sentence Subject Direction Object
1 Party X +1 European Constitution
2 Party X -1 EU accession of Turkey
Political claims analysis (PCA)
PCA, developed by Ruud Koopmans and Paul Statham (1999), is based on protest event
analysis as well as political discourse analysis and has also been inspired by Franzosi’s idea to
use the structure of linguistic grammar to code contentious events (Koopmans and Statham
1999; Koopmans et al. 2005: 254-265). The method has been developed out of the deficiency
that protest event analysis is too “protest-centric” and does not account for more routine and
conventional action forms (Koopmans and Statham 1999: 204-205). Integrating political dis-
course analysis into protest event analysis and combining approaches that have so far been
5
considered as competing paradigms within the social movements field enables researchers to
study two social phenomena at the same time and to relate collective mobilization variables to
discursive and institutional contextual variables.
Especially with regard to positions, PCA shares many ideas of CSA while using a different
vocabulary: ‘claimant’ instead of ‘subject’; ‘issue of claim’ instead of ‘object’ (see Table 2).
The specific vocabulary however indicates one crucial difference between the two methods.
PCA is not simply interested in positions but mainly in claims. Instances of claim-making
must be the result of purposive strategic actions of the claimant and refer to an ongoing or
concluded physical or verbal action in the public sphere. In other words, simple attributions of
attitudes or opinions would not be coded. The above-presented example of core sentences
would not qualify as claims-making. It must become clear that the claimant has intentionally
undertaken an action. Such intentions or actions are mostly indicated by verbs such as
‘stated’, ‘demanded’, ‘criticized’ etc. (Koopmans et al. 2005: 258). Accordingly, if we replace
the verb in our example, the sentence would be coded: “Party X has decided to support the
European Constitution but to oppose EU accession of Turkey”. Contrary to the logic of CSA,
this grammatical sentence does not consist of two claims but of one claim with two issues.
According to the specific research question, this claim might constitute one observation (with
regard to the appearance of actors, for example) or two observations (with regard to an actor's
positions).
Table 2: Example of a political claim
Claim Claimant Issue 1 (Direction) Issue 2 (Direction)
1 Party X European Constitution (+1) EU accession of Turkey (-1)
Media coverage as a source for measuring party positions and issue salience
Media data in comparison
In this section, we discuss the characteristics of media data by directly relating them to two of
the most widespread data sources, namely party manifestos and expert surveys, along the ar-
guments presented by Marks et al. (2007: 26-27).2 In contrast to them and many others, we
2 We only discuss characteristics that are presented by Marks et al. (2007: 26-27) and concern the data them-
selves. Of course, further aspects of manifesto- and expert-data are discussed in the literature, but they rather
pertain to the way the data are collected. For example, a source of contestation is that the coding of party mani-
festos relies on a priori fixed, thematic categories, which might become inappropriate over time because the
6
deliberately speak of ‘characteristics’ and not of ‘advantages’ and ‘disadvantages’ of different
data sources as their appropriateness always depends on the specific research questions one
wants to address.
Table 3: Characteristics of media-, manifesto- and expert-data
Media coverage Party Manifestos Expert surveys
- Mass-mediated positions/
salience - Self-declared positions/ sali-
ence - Reputational positions/salience
- Short and long time periods,
any points in time (during elec-
tion campaign/debates)
- Long time periods, specific
points in time (before elec-
tions)
- Long time periods, specific
points in time (dates of sur-
veys)
- Information on intra-party
dissent - No information on intra-party
dissent - Information on degree of intra-
party dissent, but not on devi-
ant factions / individuals
- Retrospective analysis possible - Retrospective analysis possible - No retrospective analysis/
temporal constraints
- Document-driven data - Document-driven data - Subjective/reputational data
- Reliance on one clear criteria
(media coverage), but some
possible conflation between
declarations and behavior
- Reliance on one clear criteria
(manifestos), separating party
declarations and actual behav-
ior
- Reliance on diverse sources of
information, risk of conflating
declarations and behavior
- Flexibility, but analysis con-
strained by what appears in the
media
- Flexibility, but analysis con-
strained by what appears in
party programs
- Much flexibility: any topic can
be studied
- Data aggregation necessary to
generate indicators - Data aggregation necessary to
generate indicators - Direct quantification possible
- Small parties are less visible in
the media - Small parties often have
shorter manifestos - Experts have less knowledge
about small parties
- Very time-consuming and
personnel-intensive - Somewhat time-consuming
and personnel-intensive - Little time-consuming and
personnel-intensive
Source: Information on manifesto- and expert-data a3dapted from Marks et al. (2007: 26-27)
As shown by Table 3, media data share many of the characteristics of manifesto data, but they
differ from each other in mainly three respects. First, an important feature of media data is
that they reflect how political parties appear in the publicly visible, mass-mediated political
debate—during or between election campaigns. In other words, party positions and issue sali-
ence are coded as they can be perceived by the wider public, whereas manifestos mirror self-
relevance of certain issues may change. This is however a problem related to a decision taken by the Compara-
tive Manifestos Project. Additionally, we ignore specific reliability problems as they are also more related to the
way data are collected than to the nature of the data itself.
7
declared positions and issue emphases defined by political parties themselves. Yet the overall
relevance of party manifestos can be questioned because few voters actually read them (Kriesi
et al. 2008: 66-67).
Second, as manifestos are published at the beginning of an election campaign, they cannot
capture the dynamics of public debates and might miss important topics that come up in the
course of or between election campaigns. Short-term changes in issue positions or salience
can therefore not be measured. While manifesto-data are available for only one specific point
in time per legislature, media-data can be gathered over both shorter and longer time periods
in order to study the ups and downs of particular issues and to track cross-temporal changes of
party positions. Whereas parties decide by themselves how much importance they attach to
various issues in their manifestos, the salience of issues in the public debate is very much
shaped by the agenda setting strategies of other actors as well as by exogenous events such as
economic crises or natural catastrophes. Third, manifestos provide no information on intra-
party dissent. To the extent that electoral programs are official, authoritative statements stra-
tegically designed to put a party in a positive light, they generally do not touch on sensitive
issues and present a party as a coherent unity. Media data, for their part, not only give infor-
mation on the party as such, but also on positions of individual party members or deviant
groups and factions. Thus, intra-party divisions are made public and appear in the media.
Based on news value theory (e.g. Galtung and Holmboe Ruge 1965), it might even be argued
that the media are particularly interested in covering intra-party heterogeneity as conflict in-
creases the newsworthiness of a message. As a consequence, the media may tend to focus on
controversial topics and to overemphasize party divisions. This should be kept in mind when
interpreting analyses of issue salience.
To some extent, intra-party heterogeneity can also be captured by expert surveys. Yet, while
expert surveys sometimes measure the degree of intra-party dissent, they generally fail to
identify dissenting voices and offer no information on deviant (groups of) actors. Such con-
troversies within political parties may also affect experts’ evaluations of their overall position
on a policy issue. In fact, experts may exclusively refer to the dominant opinion of the party
leadership when locating a party and completely discard the opinions of factions, or they may
report some mean value accounting for diverging positions within a party. This possibility
points to one important characteristic of expert data: they rely on subjective judgments sup-
posedly derived from diverse sources of information (e.g. Budge 2000, see however Steen-
8
bergen and Marks 2007 for a reply). While some experts may refer to the positions of party
leaders, others may think of the views of party activists or the opinions of party electorates,
and all may simultaneously rely on simple rhetoric and on actual behavior. This makes these
data inappropriate to analyze the impact of preferences on behavior (Marks et al. 2007). In
addition, it is difficult for experts to assess party positions and issue salience in the past.
Given that expert surveys can hardly be used retroactively, they only provide a snapshot and
cannot detect any changes within short periods of time either.
In contrast, expert surveys not only offer an “attractive combination of economy and access”
(Benoit and Laver 2006: 75), but also much flexibility since information can be gathered on
any topic according to the needs of the researcher, whereas the analysis of the media and of
party manifestos is constrained by what appears in theses sources. Typically, minor or more
technical issues are unlikely to get reported by the media or to be included in electoral pro-
grams, but experts may be available to locate political parties on such issues. Similarly, expert
surveys are a convenient source for direct quantification. While experts can answer a simple
question on whether a party is rather for or against the European integration process, positions
on such general issues can hardly be found in the media or in party manifestos. They rather
reflect positions on sub-issues such as monetary policy or EU enlargement, which then have
to be aggregated by the researcher into a more general indicator of positions towards the
European integration process. All three data sources however share the common characteristic
that measurements of positions – on large issue categories or their sub-dimensions – tend to
be more reliable for large (governing) parties than for smaller (opposition) parties at the ex-
tremes of the left-right scale. While experts often have not enough information and knowl-
edge about small parties, they generally get less coverage in the media and have shorter elec-
toral manifestos than larger parties, thus leaving the researcher with an insufficient number of
coded observations.
The last two aspects – the reliability problem for small parties and the problem of generating
general indicators – point to another particularity of media coding: data collection is much
more time-consuming and personnel-intensive than in the other two approaches. To make
valid and reliable statements about party positions and issue salience, a huge number of arti-
cles has to be coded. Contrary to a ‘quasi sentence’ in a manifesto that can be considered as
representing the dominant opinion within a party, we cannot know whether one particular
core sentence or political claim in a newspaper article reflects the general opinion of a party.
9
Similarly, we do not know from one core sentence or claim whether the respective issue is
central to the general debate or concerns a rather minor sub-issue. This insecurity can only be
overcome with a large number of observations.
This problem is aggravated by the need to code different newspapers at the same time. News-
papers generally have a distinctive ideological position represented in their editorial line,
which might influence their selection of political information (e.g. Page 1996). To correctly
represent the public debate, it is therefore advisable to code several newspapers with different
editorial profiles. It is quite common to code either a left-wing and a right-wing newspaper or
a qualitative newspaper and a tabloid. Koopmans and Statham (1999: 207) have however
shown that there are no significant differences in the positioning of political actors between
quality papers and tabloids. There are, however, differences with regard to the amount of cov-
erage of particular issues as the coverage of tabloids is much more limited and concise
(Koopmans et al. 2005: 261). Although newspapers cover some issues more extensively than
others and tend to grant more visibility to the positions of political actors that fit into their
editorial line, they generally do not distort reported positions (e.g. Hagen 1993). In other
words, the measurement of political positions should be similar across newspapers while the
salience of issues may vary according to the type of paper (tabloid vs. quality paper) and/or
their editorial line (left- vs. right-wing paper).
When to use media data
While media-data share many of the characteristics of expert- and, especially, of manifesto-
data, media-data seem to be more suitable for certain research questions than the other two
approaches. Let us briefly highlight three aspects:
Generally, we think that the choice of an indicator for particular research questions matters
more for the measurement and analysis of issue salience than of party positions. In our view,
parties are unlikely to change their positions over short periods of time, but for strategic rea-
sons they might choose to abandon some issues over the course of a political debate and to
stress other, more promising issues instead. Moreover, as argued above, while the media can
be expected to truthfully report the positions of political actors, their selection behavior
probably affects the salience of issues. As a consequence, we expect indicators of party posi-
tions derived from media coverage to be closely related to indicators based on other data
sources, whereas the various indicators of issue salience should be less strongly related.
10
More specifically, we believe that media-data are more useful when the analysis focuses on
public debates and aims at comparing the supply side (what parties offer) with the demand
side (what citizens require and how they perceive positions). Given that most citizens learn
about politics from the media (see Robertson 1976) and that the media have the power to in-
fluence how political actors and issues are perceived (e.g. Iyengar and Kinder 1987), we
should observe that indicators generated from mass survey-data are closer related to media-
than to manifesto- and expert-data.
Finally, we contend that media-data are a more sensitive and finer-grained instrument to
measure party positions and issue salience as they are not restricted to a single point in time
every four to five years, but allow studying the dynamics of actual debates. It is therefore pos-
sible to analyze short-period effects. Moreover, debates can be grasped much more appropri-
ately because media coverage offers information on intra-party dissent. Provided that news-
papers are coded without an a priori fixed coding scheme, information can be gathered on all
possible sub-issues. The problem is that we cannot empirically test these aspects by compar-
ing media data with other data as the latter simply do not provide information on positions of
party factions, positions on sub-issues or changes over relatively short periods of time.4 If,
however, we are able to show with media-data that dissenting voices and short-time changes
really exist and occur, we can make a strong case for a more generalized use of media-data in
the study of party positions.
Data and operationalization
Our media-data based on CSA stem from the comparative research project “National Political
Change in a Globalizing World” coordinated by Hanspeter Kriesi from the University of Zu-
rich and Edgar Grande from the Ludwig-Maximilians-University of Munich (Kriesi et al.
2008).5 This project studies party competition in the context of national election campaigns in
six Western European countries (Austria, France, Germany, the Netherlands, Switzerland, and
the United Kingdom) and covers, in each country, three campaigns from the 1990s and early
2000s, and one from the mid-1970s (for an overview, see Dolezal 2008: 57). The research
team content analyzed all articles (except for commentaries) related to the electoral contest or
4 Expert surveys are a partial exception as some of them enquire about intra-party dissent and positions towards
several sub-issues.
5 This project was financed by the German Research Foundation (SFB 536 – Project 5C) and by the Swiss Na-
tional Science Foundation (Project Nr. 1214-68010.02).
11
to politics in general for the two months prior to the four elections in one quality paper and
one tabloid for each country.6 The headlines, the lead and the first paragraph of the selected
articles from a sample of issues in case of quality papers and from all issues in case of tab-
loids were coded sentence by sentence. Actors were coded according to their party member-
ship, and represent a total of 36 different parties (for a list of parties per country, see Dolezal
2008: 69-70). Issues were captured with a fine-grained, open coding scheme and later aggre-
gated into twelve broader issue categories, one of which is European integration. The direc-
tion of a relationship between actors and issues was quantified on a scale ranging from +1 to -
1 (with three intermediary positions), and the position of a political party on Europe is there-
fore computed by averaging all core sentences involving this party and the issue of European
integration. Issue salience is given by the frequency with which a given political party takes a
position on European integration, relative to the total of all its statements.
The comparative research project “The Transformation of Political Mobilization and Com-
munication in European Public Spheres” (Europub.com) provides us with media-data based
on PCA.7 In this project directed by Koopmans and Statham (2002), political claims pertain-
ing to agriculture, monetary politics, immigration, troops deployment, retirement and pen-
sions, education, and European integration were analyzed in four newspapers – were available
two quality papers, one regional paper and one tabloid – in seven Western European countries
(France, Germany, Italy, the Netherlands, Spain, Switzerland, and the United Kingdom) for
the years 1990, 1995, 2000-2002.8 In contrast to media data from CSA, media data from PCA
is not tied to national election campaigns, but studies political contestation in general. Based
on a sampling scheme, articles in the international, national and economic sections of each
newspaper were retrieved and coded in full length.9 From this large dataset, we extracted all
political claims made by actors with a party affiliation on topics related to European integra-
tion. Overall, representatives from 60 different political parties are included in this dataset.
6 The following newspapers were selected: Die Presse and Kronenzeitung in Austria, The Times and the Sun in
the UK, Le Monde and Le Parisien in France, Süddeutsche Zeitung and Bild in Germany, NRC Handelsblad and
Algemeen Dagblad in the Netherlands, and Neue Zürcher Zeitung and Blick in Switzerland.
7 This project was financed by the European Commission in the context of its 5th framework program (HPSE-
CT2001-00046). The Federal Office for Education and Research funded the Swiss part of this study (BBW
00.0455). More information is available at http://europub.wzb.eu.
8 The newspapers that were chosen are: Le Monde, Le Figaro, Ouest France, L’Humanité in France, Süd-
deutsche Zeitung, Frankfurter Allgemeine Zeitung, Leipziger Volkszeitung, Bild-Zeitung in Germany, La Repub-
blica, Il Corriere della Sera, Il Mattino, La Nazione in Italy, De Volkskrant, Algemeen Dagblad, Leeuwarder
Courant, De Telegraaf in the Netherlands, El Pais, Abc, La Vanguardia, El Mundo in Spain, Neue Zürcher Zei-
tung, Journal de Genève/Le Temps, Blick, Le Matin in Switzerland, The Guardian, The Times, The Scotsman,
The Sun in the United Kingdom.
9 For more information on sampling schemes and coding rules, see Koopmans (2002).
12
Positions were coded on a scale ranging from +1 to -1, and we obtained our indicator of party
positions by calculating an average for each party for the period 2000-2002 and for 1995 re-
spectively. However, we refrained from computing an indicator of issue salience for different
reasons. For one, and contrary to CSA, each political claim is only coded once, that is posi-
tions that repeatedly appear in one (or several) newspaper article(s) are not duplicated. Sec-
ond, political claims were coded with a closed coding scheme, and while some issue catego-
ries such as agriculture were narrowly defined, the issue of European integration was a very
broad category. As a result, European integration appeared to be the most salient issue for
(almost) all parties in all countries. While such an indicator can be validly used to compare
the relative salience of different parties in the dataset, it cannot be used in comparison with
other data sources.
We contrast party positions and issue salience derived from these two sets of media-data to
measurements obtained from the more commonly used party manifestos, expert and mass
surveys. We have no space here to present the respective datasets in detail and restrict the
discussion to the operationalization of positions and issue salience. In the manifesto dataset
(Budge et al. 2001; Klingemann et al. 2006) there are two-EU related categories, labeled
“European Integration: Positive” (per108) and “European Integration: Negative” (per110).
Whereas the first category contains all favorable mentions to the EU in general, desirability of
expanding the EU and/or of increasing its competence, as well as the desirability to join the
EU (or to remain a member country), the second category refers to exactly the opposite and
also includes opposition to specific European policies. In order to measure party positions, we
follow McDonald and Mendes (2001: 94) who advise researchers to privilege ratio measures
(rather than subtractive measures) when the intention is to locate a party on a given policy
dimension regardless of salience. We therefore operationalize party positions on the pro/anti-
European dimension as the percentage of positive statements relative to the total of positive
and negative mentions of European integration. Given that the manifesto approach is based on
saliency theory, all data entries are percentages (standardized by the total number of quasi-
sentences in a given manifesto), and we obtain our indicator of salience by summing up the
pro- and anti-EU scores (see Netjes and Binnema 2007: 41).
13
To measure party positioning and salience based on expert-data we rely on data collected by
Ray (1999) for the year 1996 and by the Chapel Hill group for the year 2002.10 To measure
party positions, both surveys asked experts to evaluate the overall orientation of the party
leadership towards European integration on a seven-point scale (ranging from 1 ‘strongly op-
posed’ to 7 ‘strongly in favor’). Issue salience, for its part, is captured by a question asking
experts about the relative importance of European integration in the party’s public stance in
the year of the surveys (5-point scale, ranging from 1 ‘no importance, never mentioned by the
party’ to 5 ‘the most important issue for the party’).
To derive party positions and issue salience from mass surveys we rely on the European Elec-
tion Study (EES) from the years 1994 and 1999 (van der Eijk et al. 2002). To get a measure-
ment of party positions, we resort to a question asking respondents to place the views of a list
of political parties on a scale ranging from 1 'European unification has already gone too far' to
10 'European unification should be pushed further'. To tap issue salience, we rely on a ques-
tion asking respondents to evaluate, on a four-point scale and for a list of parties, the impor-
tance of European integration as compared to other important topics in their home country.
Contrary to the manifesto-, expert- and mass-survey data, our media-data are only available
for a restricted number of Western European countries. To augment the number of observa-
tions, we include two periods of time. As it appears in Table 4, for the CSA-, Manifesto- and
National Survey-data we have chosen national elections in the mid-1990s and at the beginning
of the new century, as the PCA-data were collected for these two periods. EES- and expert-
data are also available for these two periods. As there are no expert and EES-data for Switzer-
land, we exclude this country from our analyses. A further difficulty concerns the number of
parties: in order to directly compare the various position and salience indicators we have to
make sure that our results are not distorted by composition effects. Therefore we include only
parties in our analyses that are part of all dataset. Table 5a lists all 31 parties that are included
when we compare the CSA-indicators with other indicators, whereas we find in Table 5b the
45 parties for the respective analyses with the PCA-indicators.
10 These datasets are available from http://www.unc.edu/~gwmarks
14
Table 4: Time periods and political parties
Dataset Countries Period 1 Period 2
Austria 1999 2002
France 1995 2002
Netherlands 1998 2002
United Kingdom 1997 2001
Germany 1998 2002
Italy 1996 2001
CSA, Manifestos
Spain 1996 2000
PCA All countries 1995 2000-2002
EES All countries 1994 1999
Experts All countries 1996 2002
Abbreviations: Indicators: Core Sentence Approach (CSA), European Election Survey (EES),
Political Claims Analysis (PCA).
Table 5a: Parties included for comparison with CSA-Indicators
Countries Period 1 Period 2
Austria FPÖ, GA, SPÖ FPÖ, GA, ÖVP, SPÖ
France FN, PS, RPR, UDF FN, PS, UDF, Verts
Netherlands D’66, GL, PvdA, VVD CDA, D’66, GL
United Kingdom Labour, Tories Labour, Tories
Germany CDU, FDP CDU, FDP, SPD
Table 5b: Parties included for comparison with PCA-Indicators
Countries Period 1 Period 2
France PS, RPR, UDF PCF, PS, UDF, Verts
Netherlands D’66, PvdA, VVD CDA, D’66, GL, PvdA, SP, VVD
United Kingdom Labour, Tories Labour, LDP, SNP, Tories, UKIP
Germany CDU, FDP, Grüne, PDS, SPD CDU, CSU, FDP, Grüne, PDS, SPD
Italy FI AN, FI, LN, PCI, RC
Spain PP CiU, IU, PP, PSOE
Political parties: (A) Freiheitliche Partei Österreichs (FPÖ, Freedom Party of Austria), Die Grüne Alternative
(GA, The Green Alternative), Österreichische Volkspartei (ÖVP, Austrian People’s Party), Sozialdemokratische
Partei Österreichs (SPÖ, Social Democratic Party of Austria); (F) Front National (FN, National Front), Parti
Communiste Français (PCF, French Communist Party), Parti Socialiste (PS, Socialist Party), Rassemblement
pour la République (RPR, Rally for the Republic), Union pour la Démocratie Française (UDF, Union for French
Democracy), Les Verts (Verts, The Greens); (NL) Christen-Democratisch Appel (CDA, Christian Democratic
Appeal), Democraten’66 (D’66, Democrats 66), GroenLinks (GL, Green Left), Partij van de Arbeid (PvdA,
Labour Party), Socialistische Partije (SP, Socialist Party), Volkspartij voor Vrijheid en Democratie (VVD, Peo-
ple’s Party for Freedom and Democracy); (D) Christlich-Demokratische Union (CDU, Christian Democratic
15
Union), Christlich-Soziale Union (CSU, Christian Social Union), Freie Demokratische Partei (FDP, Free De-
mocratic Party), Bündnis 90/Die Grünen (Grüne, Alliance 90/The Greens), Partei des demokratischen Sozialis-
mus (PDS, Party of Democratic Socialism), Sozialdemokratische Partei Deutschlands (SPD, Social Democratic
Party of Germany); (I) Forza Italia (FI, Forwards Italy), Lega Nord (LN, Northern League), Alleanza Nazionale
(AN, National Alliance), Partito Comunisto Italiano (PCI, Italian Communist Party), Rifondazione Comunista
(RC, Newly Founded Communists); (E) Convergencia i Unio (CiU, Covergence and Union), Izguierda Unido
(IU, United Left), Partido Popular (PP, Popular Party), Partido Socialista Obrero Español (PSOE, Spanish So-
cialist Workers’ Party).
Exploring the potential advantages of media-data
Convergent validity of party positions and issue salience
First of all we like to test our hypothesis according to which different indicators for party po-
sitions are closer related than the respective indicators for salience. To confirm this argument
we have to cross-validate our data to find out whether or not the various indicators measure
the same underlying dimensions. According to Bollen (1989: 184), “validity is concerned
with whether a variable measures what it is supposed to measure.” We closely follow the lead
of Marks et al. (2007), Ray (2007) and Netjes and Binnema (2007) who assess indicators of
party position and issue salience, among others, in terms of their convergent and discriminant
validity.
Convergent validity involves the comparison of alternative measures of the same concept and
discriminant validity the comparison of measures of different concepts (Ray 2007: 12). Valid
measures of the same underlying construct are empirically associated and are therefore similar
and ‘converge’. Two common instruments to assess convergent and discriminant validity are
correlation tests and exploratory factor analysis. While correlation tests simply enable us to
observe whether indicators converge or not, factor analysis allows us in addition to discrimi-
nate among different dimensions. In other words, if there is more than one underlying con-
struct, factor analysis helps us show to which dimension an indicator belongs and how
strongly it correlates with the respective dimension.
It is always difficult to judge whether two indicators are closely related or not. There is no
general standard to distinguish high from low correlations, for example. The best way to as-
sess coefficients that come out of correlation tests or factor analyses is to compare them with
other similar tests. In our discussion of the validity tests we will therefore consistently com-
pare our results with those from Marks et al. (2007), Ray (2007) and Netjes and Binnema
(2007). Of course, the comparisons have to be taken with a grain of salt as these other studies
partly use different cases and indicators.
16
Table 6 reports the correlations of our six indicators for party positions. In the first part we
run the correlations with the CSA- and in the second part with the PCA-data. All coefficients
are positive and highly significant. On average the indicators correlate at the level of respec-
tively 0.639 and 0.601. This is clearly above the average coefficient of 0.531 in Ray’s (2007:
19) convergent validity test. His analysis displays the highest correlations between the two
perceptual measures of expert- and mass-surveys and between manifesto- and expert-data. In
our analysis we get very high correlations between manifesto- and expert-data, expert- and
CSA-data and manifesto- and PCA-data.
As most indicators correlate at a high level and the strength of correlations does not depend
on the indicators, we can assume that there is no strong method effect and all variables reflect
the same underlying dimension. This assumption is confirmed in our factor analysis in Table
7. All indicators display very high coefficients that are very similar to those in Ray’s (2007:
20) and Marks et al. (2007: 26) studies. The resulting factors explain respectively 74 and 70
per cent of the variance while 61 per cent and between 67 and 73 per cent of the variance is
explained in Ray’s (2007: 20) and Marks et al. (2007: 26) factor analyses. In both factor
analyses we get the highest factor loadings for the manifesto data followed by the expert-, the
media- and the EES-data.
Table 6: Convergent validity of positions I: correlations
CSA-cases PCA-cases
EES
Experts
CSA
EES
Experts
PCA
Experts 0.541***
(31)
Experts 0.643***
(45)
CSA
0.401*
(31)
0.714***
(31)
PCA
0.478***
(45)
0.592***
(45)
Manifesto
0.654***
(31)
0.878***
(31) 0.648***
(31)
Manifesto
0.521***
(45)
0.650***
(45) 0.722***
(45)
Levels of significance: * p<0.05, ** p<0.01, ***p<0.001
Notes: N in parentheses
Abbreviations: Core Sentence Approach (CSA), European Election Study (EES),
Political Claims Analysis (PCA)
17
Table 7: Convergent validity of positions II: factor analysis
CSA-cases PCA-cases
Factor 1 Factor 1
Manifesto 0.939 Manifesto 0.869
Expert 0.928 Expert 0.864
CSA 0.807 PCA 0.836
EES 0.741 EES 0.779
Eigenvalue 2.942 Eigenvalue 2.807
Explained Variance 74% Explained Variance 70%
N 31
N
45
Notes: principal component analysis, varimax rotation
Abbreviations: Core Sentence Approach (CSA), European Election Study (EES),
Political Claims Analysis (PCA)
Let us now test the convergent validity of the four dataset for which we have generated an
indicator of salience.11 We clearly see in Table 8 that with one exception all coefficients are
very low and not significant. In Netjes and Binnema’s (2007: 45) comparison of salience indi-
cators, the coefficients are slightly higher and highly significant. Compared to the different
analyses of party positions, it clearly appears that salience indicators are much less correlated.
This also becomes clear when we run a factor analysis. While the three indicators used in Net-
jes and Binnema’s (2007: 45) study load on the same factor but at a lower level than the coef-
ficients for positions in Ray (2007) and Marks et al. (2007) and explaining only 49 per cent of
the variance, we even get two factors that explain respectively 35 per cent and 29 per cent of
the variance (see Table 9). The two variables with the highest loadings are the manifesto data
in factor 1 and the CSA media-data in factor 2. This finding suggests that the parties stress
different issues in their party manifestos than during mass-mediated election campaigns. Or,
put in other words, newspapers do not seem to cover the various issues in the same way as
parties do it in their manifestos.
11 As we have explained above, an indicator of salience cannot be generated with PCA media-data.
18
Table 8: Convergent validity of salience I: correlations
CSA-cases
EES
Experts
CSA
Experts
0.087
(31)
CSA -0.061
(31)
0.188
(31)
Manifesto
0.352
(31)
0.622***
(31)
-0.071
(31)
Levels of significance: * p<0.05, ** p<0.01, ***p<0.001
Notes: N in parentheses
Abbreviations: Core Sentence Approach (CSA), European Election Study (EES),
Political Claims Analysis (PCA)
Table 9: Convergent validity of salience II: factor analysis
Notes: principal component analysis, varimax rotation
CSA-cases
Factor 1 Factor 2
Manifesto 0.891 -0.195
EES 0.463 -0.544
CSA 0.15 0.845
Experts 0.849 0.285
Eigenvalue 1.757 1.125
Explained Variance 44% 28%
N 31
Abbreviations: Core Sentence Approach (CSA), European Election Study (EES),
Political Claims Analysis (PCA)
Comparing demand- and supply-side
Another advantage of media data that is sometimes put forward is that they supposedly reflect
much better than other data how ordinary people perceive political parties as they do not read
party manifestos and do not know parties the same way as experts (Kriesi et al. 2008: 66-67).
To analyze public debates and to directly compare demand- and supply sides of political
19
struggles it might therefore be useful to use media-data. To know whether media data better
reflect how people perceive political parties we simply have to compare the correlation coef-
ficients between indicators based on survey data with the other indicators.
Contrary to our expectation in the first columns of Table 6 we see that the EES data are both
times closer related to manifesto- and expert-data than to media-data. One could argue that it
is problematic to compare data that are not collected at the same moment. We therefore con-
ducted the same analysis with data from national surveys that have been collected during the
same election periods as the CSA- and the manifesto-data (results not shown here). But even
the indicators based on the national surveys correlated at a higher level with manifesto- than
with media-data.
As we already know, media data diverge from other data much more with regard to salience
than positioning. It might therefore be that the hypothesis of a close relationship between me-
dia- and survey-data holds for salience indicators. As it appears in the first column of Table 8,
however, no such strong relationship can be detected. The highest correlation coefficient is
the one between manifesto- and EES-data that is however significant only at the 0.1 level.
Especially the generally strong correlations with the manifesto-data indicates that people
(even if they do not read party manifestos) perceive much more a party's long term strategies
than positions in short term struggles. Another result that points towards such a conclusion is
the slightly stronger correlation between survey- and PCA- than between survey- and CSA-
data. As PCA data have been coded for ‘periods of routine politics’, it might be that the posi-
tions taken in these periods are closer related to the long-term strategies of parties than the
positions taken during the specific periods of election campaigns.
Observing heterogeneity
The last aspect we are interested in concerns the question whether media-data are a finer-
grained and more sensitive to explore short-term changes, diverging positions over sub-issues
and intra-party dissent. Unfortunately, we have not yet been able to investigate these aspects
with our own media-data, but some studies have already (at least partly) shown that diverging
voices within a party and over sub-issues exist whereas changing positions over short periods
of time have not yet been subject of empirical analyses. For instance, analyzing party families
in six Western European countries, Helbling et al. (2008) confirm that the extremist parties at
20
the right and the left generally strongly oppose the European integration process while the
established Social democrats and conservative parties take a rather ambivalent position and
are internally divided. Since European integration consists of economic, political and cultural
projects, it has been argued and empirically shown that parties take diverging positions on the
different dimensions (see Helbling et al. 2008; Hooghe et al. 2004; Marks 2004: 241-343).
For example, parties of the center-left are often said to be in favor of more integration if par-
ticular projects focus on ‘market-regulation’. Social democrats are in favor of diminishing
cultural boundaries, but simultaneously are afraid of losing social achievements at the national
level. On the other side of the political spectrum, those on the right increase their opposition
to integration when the process concerns more than a simple economic and monetary union.
Liberal conservative parties often support further integration as long as it mainly concerns
market liberalization, and oppose further integration when political or cultural aspects are
concerned (Kriesi 2007: 86-87).
Conclusion
The aim of this paper was to introduce the content coding of media coverage as an alternative
approach to study party positions and issue salience in the field of European integration.
Whereas the use of media-data has a long tradition in social movement research on conten-
tious politics, party experts have rarely relied on the media to measure party positions about
policies or the salience of issues although some eminent scholars have recently called for a
move in this direction. To do so, we presented two different techniques to derive indicators of
party positions and issue salience from the media: the core sentence approach and political
claims analysis. Both methods focus on (the direction of) relationships between actors and
issues and therefore produce relational data, which can be collected at any moment on any
issue covered by the media during an undetermined period of time according to the needs of
the researcher.
The results from our correlation and factor analyses provide evidence for convergent validity
between our indicators of party positions derived from media-data and more established
measures based on party manifestos, experts and mass surveys. In other words, our media
indicators seem to capture largely the same information about party positions as other meas-
ures. This finding underlines that media-data constitute a valid alternative to more traditional
and widely used data sources. Although different methods produce similar results on political
parties' general location on the European integration issue, each has its particular strengths
21
and weaknesses. Depending on the specific research question, one method might be prefer-
able to the others. In the case of media-data, we believe that its particular strength lies in the
possibility to explore short-term changes, diverging positions on sub-issues and intra-party
dissent. To answer such research questions, media-data have more to offer than other ap-
proaches because they are more sensitive and finer-grained. In such a case, the enormous and
time-consuming coding effort may be justified by more detailed and balanced research find-
ings. If, in contrast, one is primarily interested in political parties' general policy positions, a
less costly and more accessible method such as expert surveys may be advisable.
The indicators to measure issue salience must be chosen even more carefully. In fact, our
findings indicate that the salience indicators derived from various data sources measure dif-
ferent constructs and cannot be used interchangeably. We expected that media data better re-
flect how ordinary citizens perceive political parties and which importance they attach to spe-
cific issues. As it turned out however, this is not true. It seems that ordinary people perceive
much more a party's long-term strategies. Nonetheless, we think that media data are more
appropriate when public debates are at the core of interest and when we aim at exploring how
parties compete with each other (see Netjes and Binnema 2007: 40). Since media data allow
capturing short-term changes, we are in a position to explore how parties react to the agenda
setting strategies of other parties and how they change their priorities in the aftermath of ex-
ternal events.
We have mentioned some of the studies that have already used data derived from media ac-
cording to the two techniques we have presented (Kriesi 2007; Kriesi et al. 2008; Koopmans
2007; Koopmans et al. 2005). Further applications of media data are however necessary that
explore in detail the research questions we have outlined in this paper. By doing so, we will
gain further insights into the strengths and weaknesses of media data. Even though none of
the various sources of data we have discussed in this paper has a monopoly of truth, it is pos-
sible to generalize about specific research questions for which some indicators are more ap-
propriate than others.
22
References
Axelrod, Robert. 1976. Structure of Decision: The Cognitive Maps of Political Elites. Prince-
ton, NJ: Princeton University Press.
Bennett, Lance W., Vicor W. Pickard, David P. Iozzi, Carl L. Schroeder, Taso Lagos and Ev-
ans C. Caswell. 2004. “Managing the Public Sphere: Journalistic Construction of the
Great Globalization Debate.” Journal of Communication 54(3): 437-455.
Benoit, Kenneth and Michael Laver. 2006. Party Policy in Modern Democracies. London:
Routledge.
Bollen, Kenneth A. 1989. Structural Equations with Latent Variables. New York: John Wiley
and Sons.
Budge, Ian. 2000. "Expert judgements of party policy positions: Uses and limitations in po-
litical research." European Journal of Political Research 37 (1): 103-13.
Budge, Ian and Dennis Farlie. 1983. "Party Competition - Selective Emphasis or Direct Con-
frontation? An Alternative View with Data." In Western European Party Systems.
Continuity and Change, ed. H. Daalder and P. Mair. Beverly Hills: Sage, 267-305.
Budge, Ian, Hans-Dieter Klingemann, Andrea Volkens and Judith Bara. 2001. Mapping Pol-
icy Preferences: Estimates for Parties, Electors, and Governments, 1945-1998. Ox-
ford: Oxford University Press.
De Vreese, Claes H. 2003. Framing Europe: Television News and European Integration. Am-
sterdam: Aksant.
Dolezal, Martin. 2008. "The design of the study: the distinguishing characteristics of our ap-
proach." In West European Politics in the Age of Globalization, ed. H. Kriesi, E.
Grande, R. Lachat, M. Dolezal, S. Bornschier and T. Frey. Cambridge: Cambridge
University Press, 53-74.
Eijk, Cees van der, Mark Franklin, Klaus Schönbach, Hermann Schmitt, Holli Semetko, et al.
2002. European Election Study 1999: Design, Implementation and Results (Computer
File and Codebook). Amsterdam: Steinmetz Archives.
Ferree, Myra Marx, William Gamson, Jürgen Gerhards, J. and Dieter Rucht. 2002. Shaping
Abortion Discourse. Democracy and the Public Sphere in Germany and the United
States. Cambridge: Cambridge University Press.
Franzosi, Roberto. 2004. From Words to Numbers: Narrative, Data, and Social Science.
Cambridge: Cambridge University Press.
Galtung, Johan and Marie Holmboe Ruge. 1965. "The Structure of Foreign News." Journal of
Peace Research 2 (1):64-91.
Hagen, Lutz M. 1993. "Opportune Witnesses: An Analysis of Balance in the Selection of
Sources and Arguments in the Leading German Newspapers' Coverage of the Census
Issue." European Journal of Communication 8 (3):317-43.
Helbling, Marc, Dominic Höglinger and Bruno Wüest. 2008. “How Political Parties Frame
the European Integration Process.” Article manuscript under review.
Hooghe, Liesbet, Gary Marks and Carole J. Wilson. 2004. “Does left/right structure party
positions on European integration?” In European Integration and Political Conflict,
ed. Gary Marks and Marco R. Steenbergen. Cambridge: Cambridge University Press.
Iyengar, Shanto and Donald R. Kinder. 1987. News That Matters: Television and American
Opinion. Chicago/London: University of Chicago Press.
Keman, Hans. 2007. "Experts and manifestos: Different sources - Same results for compara-
tive research?" Electoral Studies 26 (1):76-89.
Kleinnijenhuis, Jan and Paul Pennings. 2001. "Measurement of party positions on the basis of
party programmes, media coverage and voter perceptions." In Estimating the Policy
Position of Political Actors, ed. M. Laver. London: Routledge, 162-182.
23
Kleinnijenhuis, Jan, Jan de Ridder and Ewald M. Rietberg. 1997. “Reasoning in Economic
Discourse: An Application of the Network Approach to the Dutch Press.” In Text
Analysis for the Social Sciences: Methods for Drawing Statistical Inferences from
Texts and Transcripts, ed. Carl W. Roberts. Mahwah, NJ: Lawrence Erlbaum Associ-
ates. 191-209.
Klingemann, Hans-Dieter, Andrea Volkens, Judith Bara, Ian Budge and Michael Macdonald.
2006. Mapping Policy Preferences II: Estimates for Parties, Electors, and Govern-
ments in Eastern Europe, the European Union and the OECD, 1990-2003. Oxford:
Oxford University Press.
Koopmans, Ruud. 2002. Codebook for the analysis of political mobilisation and communica-
tion in European public spheres. Mimeo: available from http://europub.wz-berlin.de.
Koopmans, Ruud. 2007. “Who Inhabits the European Public Sphere? Winners and Losers,
Supporters and Opponents in Europeanised Political Debates.” European Journal of
Political Research 46(2): 183-210.
Koopmans, Ruud and Paul Statham. 1999. “Political Claims Analysis: Integrating Protest
Event and Political Discourse Approaches.” Mobilization 4(1): 40-51.
Koopmans, Ruud and Paul Statham. 2002. The Transformation of Political Mobilisation and
Communication in European Public Spheres: A Research Outline. Mimeo: available
from http://europub.wz-berlin.de.
Koopmans, Ruud, Paul Statham, Marco Giugni and Florence Passy. 2005. Contested Citizen-
ship: Immigration and Cultural Diversity in Europe. Minneapolis and London: Uni-
versity of Minnesota Press.
Kriesi, Hanspeter. 2007. "The Role of European Integration in National Election Campaigns."
European Union Politics 8 (1): 83-108.
Kriesi, Hanspeter, Edgar Grande, Romain Lachat, Martin Dolezal, Simon Bornschier and
Timotheos Frey. 2008. West European Politics in the Age of Globalization. Cam-
bridge: Cambridge University Press.
Kriesi, Hanspeter, Ruud Koopmans, Jan Willem Duyvendak and Marco G. Giugni. 1995. New
Social Movements in Western Europe. Minneapolis: University of Minnesota Press.
Laver, Michael. 2001. "Why should we estimate the policy positions of political actors?" In
Estimating the Policy Position of Political Actors, ed. M. Laver. London: Routledge,
3-9.
Mair, Peter. 2006. "Political Parties and Party Systems." In Europeanization: New Research
Agendas, ed. P. Graziano and M. Vink. Basingstoke: Palgrave Macmillan.
Marks, Gary. 2004. "Conclusion: European Integration and Political Conflict.” In European
Integration and Political Conflict, ed. G. Marks and M. Steenbergen. Cambridge: Cam-
bridge University Press.
Marks, Gary, Liesbet Hooghe, Marco R. Steenbergen and Ryan Bakker. 2007. "Crossvalidat-
ing data on party positioning on European integration." Electoral Studies 26 (1): 23-
38.
McDonald, Michael D. and Silvia M. Mendes. 2001. "The policy space of party manifestos."
In Estimating the Policy Position of Political Actors, ed. M. Laver. London:
Routledge, 90-114.
Netjes, Catherine E. and Harem A. Binnema. 2007. "The salience of the European integration
issue: Three data sources compared." Electoral Studies 26 (1): 39-49.
Osgood, Charles E. (1956). Evaluative Assertion Analysis. Litera 3: 47-102.
Page, Benjamin I. 1996. "The Mass Media as Political Actors." Political Science & Politics
29 (1):20-4.
24
Ray, Leonard. 2007. "Validity of measured party positions on European integration: Assump-
tions, approaches, and a comparison of alternative measures." Electoral Studies 26 (1):
11-22.
Ray, Leonard. 1999. "Measuring party orientations towards European integration: Results
from an expert survey." European Journal of Political Research 36 (6): 283-306.
Roberston, Davis. 1976. A Theory of Party Competition. London: John Wiley and Sons.
Statham, Paul, Ruud Koopmans and Anke Tresch. 2008. Political Party Contestation: Emerg-
ing Eurosceptics or Constructive Critics? Unpublished Ms. Bristol: University of Bris-
tol, Department of Sociology.
Steenbergen, Marco R. and Gary Marks. 2007. "Evaluating expert judgements." European
Journal of Political Research 46 (3): 347-66.
Trenz, Hans-Jörg. 2005. Europa in den Medien. Die Europäische Integration im Spiegel nati-
onaler Öffentlichkeit. Frankfurt am Main: Campus.
Wittgenstein, Ludwig. 1984 [1921]. Tractatus Logico-Philosophicus. Frankfurt am Main:
Suhrkamp.
25
