Conference PaperPDF Available

Game or measurement? Algorithmic transparency and the Klout score

Authors:
Game or measurement?
Algorithmic transparency and the Klout score
Devin Gaffney
Oxford Internet Institute
University of Oxford
1 St. Giles
Oxford, OX1 3JS
United Kingdom
devin.gaffney@oii.ox.ac.uk
Cornelius Puschmann
Berlin School of Library and Information Science
Humboldt-Universität zu Berlin
Unter den Linden 6
10099 Berlin
Germany
puschmann@ibi.hu-berlin.de
ABSTRACT
Klout 1is an Internet service that claims to measure an
individuals influence by aggregating information from a va-
riety of social media platforms. We briefly examine Klouts
approach to measuring influence and argue that the Klout
scores lack of algorithmic transparency undermines its sta-
tus as a trustworthy metric. We further argue that Klout
and similar services “gamify” the notion of influence in ways
that encourage competitive behavior among users in ways
which are detrimental to the quality of measurement in a
scientific sense, instead encouraging a gamified notion of in-
fluence. We conclude that only an approach to measure-
ment that is based on transparency has a real chance of
gaining trust, and that only a widely trusted metric may
serve as a reliable indicator of influence. Beyond the case
of Klout, context, theoretical soundness, algorithmic trans-
parency, and user agency are general issues that must be
dealt with for any entity attempting to measure influence in
online social networks.
Keywords
Klout, Social Media, Influence Measurement
1. INTRODUCTION
A flurry of news articles have recently explicated the im-
portance of Klout scores as a point of judgment in cases
ranging from access to VIP areas of a club to job offers from
potential employers [2, 7, 11]. This to be expected – what is
the utility of a service that purports to definitively measure
influence unless these scores are employed beyond Klouts
domain as a proxy for social media authority? While this is
perhaps not surprising, it is problematic, and Klout exem-
plifies many of the issues that arise when relying on metrics
provided by a commercial third party, and belies a grow-
1http://www.klout.com
ing dependence on such metrics both within and outside of
academia [5].
2. INFLUENCE AS CONTEXT-DEPENDENT
AND MUTABLE
Individuals, communities, and publics interacting online do
not form a homogenous whole. It is essential for researchers
to keep in mind that context matters, and that what ap-
plies to one set of people may not apply to another. As an
example, [8] noted distinct audience practices surrounding
different influential Twitter accounts in Tunisia and Egypt
in 2011. Even the most exhaustive works surrounding so-
cial media influence research stops short of claiming causal-
ity or predictive power, despite the strong indications they
find [5, 1]. Influence is a profoundly contextually-bounded
notion – what defines it in one community may differ dras-
tically from another communitys definition. Klouts ap-
proach to influence assumes an ability to reduce the contex-
tual complexity provided by seemingly unambiguous signals
(retweets, likes, follower/friend-relations among users) into
a one-dimensional, 0-100 index score [3]. While this total-
ization is of obvious importance when seeking to establish a
commercial standard, it simplifies the meaning of these signs
and inevitably raises questions of agency and legitimacy.
3. LEGITIMATELY MEASURING INFLU-
ENCE?
Successful operationalization of influence is a precondition to
proposing a scientifically sound measurement – what makes
someone influential to whom has to be systematically de-
scribed before it can be quantified. Another condition is
that what is being measured should not be influenced by
the process of measurement, i.e. it should be observed un-
der natural conditions. Finally, subjectivity must be ad-
dressed: Who is measuring? Who is being measured? What
is the purpose of the measurement and what potential for
misuse exists? While counting the number of cookies in a
jar can be regarded as value-neutral in the sense of being
reproducible, reliable and not bound to a single context of
analysis or intention, measuring influence immediately raises
questions of how the particular judgment was reached. The
Klout score renders the context of analysis irrelevant and
encourages systematic comparisons of scores among users –
a defining, game-like feature of the platform. The score and
the subjective quality of influence thus lose sight of one an-
other and the score becomes an entity of its own, more akin
to a currency than to a metric.
4. ALGORITHMIC TRANSPARENCY
A major component to the legitimacy of analytical methods
is the degree to which they are accessible for scrutiny and
reproduction. In the case of Google, the analytic methods it
developed were laid out in general terms in conjunction with
its operationalization [9, 4]. While PageRank has certainly
changed drastically in the intervening decade owing to the
various business directions the service has gone in, it seems
that the initial transparency of the algorithm paved the way
for users to establish a relationship of trust to the search en-
gine. For obvious competitive reasons, not all startups are
in a position to publish white papers documenting their al-
gorithms to the general public. But disclosing the details of
their system would be especially relevant for a service that
measures something as subjective and mutable as influence,
a parameter that relies on widespread acceptance to be con-
sidered authoritative. Furthermore, transparency is not a
switch – it is a gradient. Though Klout may be at risk by
disclosing a trade secret in the short run, moving towards
a more transparent model in the long term would under-
pin Klouts status as a reliable metric. In the contemporary
model, without generalized knowledge about the system, it
is impossible to trust Klout, leaving room for speculation
about the soundness of their approach, and academics rely-
ing on it are on very shaky grounds [10].
5. USER AGENCY BEYOND THE SCORE
Because the kind of measurement delivered by Klout is so
obviously not value-neutral, it is crucial to provide users
with a meaningful form of agency beyond seeking to raise
their score to ever-higher levels. While many users cer-
tainly wouldnt use all the bells and whistles available to
modify what defines influence in their niche, those who do
will be given a much better metric. Necessarily, this elimi-
nates the universal relative comparison of scores, which does
nothing to improve analytical knowledge and everything to
encourage competition. In many ways, Klout has already
jumped the analytical shark by shifting their service away
from the core metrics available for several years and to-
wards a much more gamified experience via achievements
(awards for various activities on the site) and perks (cross-
promotional events with partner goods and services) [6]. In
doing this, Klout effectively de-emphasizes the importance
of its analytics service, and cheapens the value such a system
could have for its users.
6. REFERENCES
[1] E. Bakshy, J. M. Hofman, W. A. Mason, and D. J.
Watts. Everyones an Influencer: Quantifying
Influence on Twitter. Hong Kong, China, February
9-12 2011. WSDM11.
[2] A. Blow. Can your Klout score land you a job?
http://www.usatodayeducate.com/staging/index.php/career/can-
your-klout-score-land-you-a-job, April 25
2012.
[3] d. boyd, S. Golder, and G. Lotan. Tweet, Tweet,
Retweet: Conversational Aspects of Retweeting on
Twitter. In System Sciences (HICSS), 2010 43rd
Hawaii International Conference on, pages 1–10,
January 2010.
[4] S. Brin and L. Page. The Anatomy of a Large-Scale
Hypertextual Web Search Engine. In Seventh
International World-Wide Web Conference (WWW
1998), 1998.
[5] M. Cha, H. Haddadi, F. Benevenuto, and K. P.
Gummadi. Measuring User Influence in Twitter: The
Million Follower Fallacy. pages 10–17, Washington,
DC, May 23-26 2010. Proceedings of the Fourth
International AAAI Conference on Weblogs and Social
Media.
[6] S. Deterding, D. Dixon, R. Khaled, and L. Nacke.
From Game Design Elements to Gamefulness:
Defining ”Gamification”. Tampere, Finland,
September 28-30 2011. MindTrekˆa ˘
A´
Z11.
[7] B. Landman. Are You a V.I.P.? Check Your Klout
Score.
http://www.nytimes.com/2011/11/20/fashion/klout-
scores-sort-out-social-media-stars.html, November 18
2011.
[8] G. Lotan, E. Graeff, M. Ananny, D. Gaffney, I. Pearce,
and danah boyd. The Revolutions Were Tweeted:
Information Flows during the 2011 Tunisian and
Egyptian Revolutions. International Journal of
Communications, 5:1375–1405, 2011.
[9] L. Page, S. Brin, R. Motwani, and T. Winograd. The
PageRank Citation Ranking: Bringing Order to the
Web. Technical Report 1999-66, November 1999.
[10] D. Quercia, J. Ellis, L. Capra, and J. Crowcroft. In
the Mood for Being Influential on Twitter. Boston,
MA, USA, October 9-11 2011. In proceeding of:
PASSAT/SocialCom 2011, Privacy, Security, Risk and
Trust (PASSAT), 2011 IEEE Third International
Conference on and 2011 IEEE Third International
Confernece on Social Computing (SocialCom).
[11] S. Stevenson. What Your Klout Score Really Means.
http://www.wired.com/epicenter/2012/04/ff klout/all/1,
April 24 2012.
... However, they have received several critics and have been the focus of several controversies regarding how the measurements are computed or the effect that spam-bots might have on the algorithms. As most of the commercial measures do not publicly state how scores are computed, they are not accessible for scrutiny or reproduction, which might compromise their trustworthiness [Gaffney and Puschmann, 2012]. ...
... Considering that there is no consensus on what means to be an influential user [del Campo-Ávila et al., 2013;Gaffney and Puschmann, 2012], this work aims at analysing the human perception of influence. The presented metric was compared not only to several commercial metrics, but also to a human assessment of user influence. ...
Article
Full-text available
People's influence has been the subject of study of several social and humanities disciplines. Lately, the study of user's influence in micro-blogging platforms arises as an important issue. Although social influence or prestige can be defined as the potential or ability of an individual to engage others in a certain act, or to induce others to behave in a particular manner, there is no global consensus on what means to be an influential user. This work aims at shedding some light on how to assess user influence by proposing a novel metric of user influence based on analysing user behaviour regarding both content-based and topological factors. The metric does not only consider each user individually, but also aims at assessing the interactions with his/her neighbourhood. The statistical analysis performed confirmed that only analysing the topological factors is not sufficient for accurately assessing the influence of users. Instead the published content and its influence over the neighbourhood of users has to be also analysed. A comparison with a human assessment of user influence showed that the factors considered by the proposed metric are truly relevant for assessing people's influence.
... While companies, governments, and other stakeholders previously had to rely on vague forecasts, the promise of these new approaches is ultimately to curb human unpredictability through information. The traces created by the users of social media platforms are harvested, bought, and sold; an entire commercial ecosystem is forming around social data, with analytics companies and services at the helm (Burgess & Bruns, 2012; Gaffney & Puschmann, 2012, and Chapter 5 in this volume). Yet, while the data in social media platforms is sought after by companies, governments, and scientists, the users who produce it have the least degree of control over " their " data. ...
Article
Our paper approaches Twitter through the lens of “platform politics” (Gillespie, 2010), focusing in particular on controversies around user data access, ownership, and control. We characterise different actors in the Twitter data ecosystem: private and institutional end users of Twitter, commercial data resellers such as Gnip and DataSift, data scientists, and finally Twitter, Inc. itself; and describe their conflicting interests. We furthermore study Twitter’s Terms of Service and application programming interface (API) as material instantiations of regulatory instruments used by the platform provider and argue for a more promotion of data rights and literacy to strengthen the position of end users.
Conference Paper
Full-text available
Son yıllarda işletmeler rekabet avantajı yaratmada kritik rol oynayan yetenekli çalışanları işe almak için çeşitli yöntemler kullanmaktadır. Bunlar içerisinde en önemlisi, sosyal medyanın etkin kullanımıdır. Başta Amerika Birleşik Devletleri olmak üzere işe alımlarda adayların sosyal medya puanı referans kabul edilmekte ve bu puanı değerlendirebilmek için adayların klout puanı incelenmektedir. Klout puanı, bir kişinin sosyal medya sitelerindeki popülerliğini ve etkisini göstermektedir. Sosyal medyanın işe alımlardaki etkinliğinin artmasıyla birlikte, klout puanının da ülkemizde yakın bir zamanda işe alımlarda etkili olacağı değerlendirilmektedir. Bu çalışmada, klout puanı hakkında bilgi verilmiş ve klout puanının işe alımlarda etkili olup olamayacağı sorgulanmıştır. In recent years, businesses use a variety of methods to recruit talented employees which play a critical role in creating competitive advantage. The effective use of social media is one of the most important of these methods. Particularly in the United States, applicants' social media scores are accepted as reference in recruitment and applicants' klout scores are analyzed in order to assess this score. Klout score shows a person's popularity and effectiveness at the social media sites. With the increasing effectiveness of social media in recruitment, klout score is considered to be effective in recruitment in our country in the near future. In this study, information was given about klout score and whether it will be effective or not in recruitment was questioned.
Chapter
Die Weitergabe und Verbreitung von Informationen zählen zu den beliebtesten Aktivitäten in den sozialen Medien. Zahlreiche Nutzungsoptionen (z. B. Posting, Sharing, Retweeting, Reblogging) ermöglichen das schnelle Teilen von Neuigkeiten in unterschiedlichen Formaten. Dabei erfüllen Weitergabe und Verbreitung von Informationen für die User wichtige soziale und kommunikative Funktionen, die über den Kerneffekt der Informationsdiffusion häufig hinausgehen. In unserem Beitrag geben wir auf Basis aktueller Literatur einen Überblick über typische Erklärungsmodelle für Informationsdiffusion einerseits und beschreiben andererseits Motive für die Weitergabe und Verbreitung von Informationen in und mithilfe von sozialen Medien. Wir skizzieren zudem den Einfluss von Netzwerkstrukturen und Informationstypen auf und Barrieren bei der Informationsdiffusion.
Chapter
Die Weitergabe und Verbreitung von Informationen zählen zu den beliebtesten Aktivitäten in den sozialen Medien. Zahlreiche Nutzungsoptionen (z. B. Posting, Sharing, Retweeting, Reblogging) ermöglichen das schnelle Teilen von Neuigkeiten in unterschiedlichen Formaten. Dabei erfüllen Weitergabe und Verbreitung von Informationen für die User wichtige soziale und kommunikative Funktionen, die über den Kerneffekt der Informationsdiffusion häufig hinausgehen. In unserem Beitrag geben wir auf Basis aktueller Literatur einen Überblick über typische Erklärungsmodelle für Informationsdiffusion einerseits und beschreiben andererseits Motive für die Weitergabe und Verbreitung von Informationen in und mithilfe von sozialen Medien. Wir skizzieren zudem den Einfluss von Netzwerkstrukturen und Informationstypen auf und Barrieren bei der Informationsdiffusion.
Article
Full-text available
This article details the networked production and dissemination of news on Twitter during snapshots of the 2011 Tunisian and Egyptian Revolutions as seen through information flows—sets of near-duplicate tweets—across activists, bloggers, journalists, mainstream media outlets, and other engaged participants. We differentiate between these user types and analyze patterns of sourcing and routing information among them. We describe the symbiotic relationship between media outlets and individuals and the distinct roles particular user types appear to play. Using this analysis, we discuss how Twitter plays a key role in amplifying and spreading timely information across the globe.
Conference Paper
Full-text available
Recent years have seen a rapid proliferation of mass-market consumer software that takes inspiration from video games. Usually summarized as "gamification", this trend connects to a sizeable body of existing concepts and research in human-computer interaction and game studies, such as serious games, pervasive games, alternate reality games, or playful design. However, it is not clear how "gamification" relates to these, whether it denotes a novel phenomenon, and how to define it. Thus, in this paper we investigate "gamification" and the historical origins of the term in relation to precursors and similar concepts. It is suggested that "gamified" applications provide insight into novel, gameful phenomena complementary to playful phenomena. Based on our research, we propose a definition of "gamification" as the use of game design elements in non-game contexts.
Conference Paper
Full-text available
Twitter - a microblogging service that enables users to post messages ("tweets") of up to 140 characters - supports a variety of communicative practices; participants use Twitter to converse with individuals, groups, and the public at large, so when conversations emerge, they are often experienced by broader audiences than just the interlocutors. This paper examines the practice of retweeting as a way by which participants can be "in a conversation." While retweeting has become a convention inside Twitter, participants retweet using different styles and for diverse reasons. We highlight how authorship, attribution, and communicative fidelity are negotiated in diverse ways. Using a series of case studies and empirical data, this paper maps out retweeting as a conversational practice.
Conference Paper
Full-text available
In this paper we investigate the attributes and relative influence of 1.6M Twitter users by tracking 74 million diffusion events that took place on the Twitter follower graph over a two month interval in 2009. Unsurprisingly, we find that the largest cascades tend to be generated by users who have been influential in the past and who have a large number of followers. We also find that URLs that were rated more interesting and/or elicited more positive feelings by workers on Mechanical Turk were more likely to spread. In spite of these intuitive results, however, we find that predictions of which particular user or URL will generate large cascades are relatively unreliable. We conclude, therefore, that word-of-mouth diffusion can only be harnessed reliably by targeting large numbers of potential influencers, thereby capturing average effects. Finally, we consider a family of hypothetical marketing strategies, defined by the relative cost of identifying versus compensating potential "influencers." We find that although under some circumstances, the most influential users are also the most cost-effective, under a wide range of plausible assumptions the most cost-effective performance can be realized using "ordinary influencers"---individuals who exert average or even less-than-average influence.
Conference Paper
Full-text available
Directed links in social media could represent anything from intimate friendships to common interests, or even a passion for breaking news or celebrity gossip. Such directed links determine the flow of information and hence indicate a user's influence on others—a concept that is crucial in sociology and viral marketing. In this paper, using a large amount of data collected from Twit- ter, we present an in-depth comparison of three mea- sures of influence: indegree, retweets, and mentions. Based on these measures, we investigate the dynam- ics of user influence across topics and time. We make several interesting observations. First, popular users who have high indegree are not necessarily influential in terms of spawning retweets or mentions. Second, most influential users can hold significant influence over a variety of topics. Third, influence is not gained spon- taneously or accidentally, but through concerted effort such as limiting tweets to a single topic. We believe that these findings provide new insights for viral marketing and suggest that topological measures such as indegree alone reveals very little about the influence of a user.
Article
The importance of a Web page is an inherently subjective matter, which depends on the readers interests, knowledge and attitudes. But there is still much that can be said objectively about the relative importance of Web pages. This paper describes PageRank, a mathod for rating Web pages objectively and mechanically, effectively measuring the human interest and attention devoted to them. We compare PageRank to an idealized random Web surfer. We show how to efficiently compute PageRank for large numbers of pages. And, we show how to apply PageRank to search and to user navigation.
Conference Paper
In this paper, we present Google, a prototype of a large-scale search engine which makes heavy use of the structure present in hypertext. Google is designed to crawl and index the Web efficiently and produce much more satisfying search results than existing systems. The prototype with a full text and hyperlink database of at least 24 million pages is available at http://google.stanford.edu/ To engineer a search engine is a challenging task. Search engines index tens to hundreds of millions of web pages involving a comparable number of distinct terms. They answer tens of millions of queries every day. Despite the importance of large-scale search engines on the web, very little academic research has been done on them. Furthermore, due to rapid advance in technology and web proliferation, creating a web search engine today is very different from 3 years ago. This paper provides an in-depth description of our large-scale web search engine - the first such detailed public description we know of to date. Apart from the problems of scaling traditional search techniques to data of this magnitude, there are new technical challenges involved with using the additional information present in hypertext to produce better search results. This paper addresses this question of how to build a practical large-scale system which can exploit the additional information present in hypertext. Also we look at the problem of how to effectively deal with uncontrolled hypertext collections, where anyone can publish anything they want.
Are You a V.I.P.? Check Your Klout Scoreklout- scores-sort-out-social-media-stars.html
  • B Landman
B. Landman. Are You a V.I.P.? Check Your Klout Score. http://www.nytimes.com/2011/11/20/fashion/klout- scores-sort-out-social-media-stars.html, November 18 2011.
Can your Klout score land you a job? http://www.usatodayeducate.com/staging/index.php/career/canyour-klout-score-land-you-a-job
  • A Blow
A. Blow. Can your Klout score land you a job? http://www.usatodayeducate.com/staging/index.php/career/canyour-klout-score-land-you-a-job, April 25 2012.
What Your Klout Score Really Means
  • S Stevenson
S. Stevenson. What Your Klout Score Really Means. http://www.wired.com/epicenter/2012/04/ff klout/all/1, April 24 2012.