Journal of the Association for Information Science and Technology

This paper provides the first historical analysis of the relationship between collaboration and scientific impact, using three indicators of collaboration (number of authors, number of addresses, and number of countries) and including articles published between 1900 and 2011. The results demonstrate that an increase in the number of authors leads to an increase in impact--from the beginning of the last century onwards--and that this is not simply due to self-citations. A similar trend is also observed for the number of addresses and number of countries represented in the byline of an article. However, the constant inflation of collaboration since 1900 has resulted in diminishing citation returns: larger and more diverse (in terms of institutional and country affiliation) teams are necessary to realize higher impact. The paper concludes with a discussion of the potential causes of the impact gain in citations of collaborative papers.

A content analysis of the research of library and information science (LIS) from 1965 to 1985 is reported. The aim is to find out how international research in LIS is distributed over topics, and what approaches and methods have been used to investigate these topics. The study samples consist of 142, 359, and 449 full-length research articles published in 1965, 1975, and 1985, respectively, in core LIS journals. The proportion of library and information service activities, and information storage and retrieval among the topics of the research articles was each 25% to 30% through the years. There was very little research on methodology (1%–8%), information seeking (6%–8%), and scientific communication (5%–7%). The proportion of empirical research strategies was high (49%–56%) with survey method (20%–23%) as the single most important method. A conceptual research strategy (mainly verbal argumentation) was employed in 23%–29% of the articles and system analysis, description and design in 10%–15%. The most remarkable changes from 1965 to 1985 are the loss of interest in methodology and in the analysis of LIS and the change of interest in information storage and retrieval from classification and indexing (from 22% to 6%) to retrieval (from 4% to 13%). Cross-tabulations of article topics with research strategies and approaches are presented.

Using Scopus data, we construct a global map of science based on aggregated journal-journal citations from 1996-2012 (N of journals = 20,554). This base map enables users to overlay downloads from Scopus interactively. Using a single year (e.g., 2012), results can be compared with mappings based on the Journal Citation Reports at the Web-of-Science (N = 10,936). The Scopus maps are richer at both the local and global levels because of their greater coverage, including, for example, the arts and humanities. The base maps can be interactively overlaid with journal distributions in sets downloaded from Scopus, for example, for the purpose of portfolio analysis. Rao-Stirling diversity can be used as a measure of interdisciplinarity in the sets under study. Maps at the global and the local level, however, can be very different because of the different levels of aggregation involved. Two journals, for example, can both belong to the humanities in the global map, but participate in different specialty structures locally. The base map and interactive tools are available online (with instructions) at http://www.leydesdorff.net/scopus_ovl

MELIBEA is a Spanish database that uses a composite formula with eight weighted conditions to estimate the effectiveness of Open Access mandates (registered in ROARMAP). We analyzed 68 mandated institutions for publication years 2011-2013 to determine how well the MELIBEA score and its individual conditions predict what percentage of published articles indexed by Web of Knowledge is deposited in each institution's OA repository, and when. We found a small but significant positive correlation (0.18) between MELIBEA score and deposit percentage. We also found that for three of the eight MELIBEA conditions (deposit timing, internal use, and opt-outs), one value of each was strongly associated with deposit percentage or deposit latency (immediate deposit required, deposit required for performance evaluation, unconditional opt-out allowed for the OA requirement but no opt-out for deposit requirement). When we updated the initial values and weights of the MELIBEA formula for mandate effectiveness to reflect the empirical association we had found, the score's predictive power doubled (.36). There are not yet enough OA mandates to test further mandate conditions that might contribute to mandate effectiveness, but these findings already suggest that it would be useful for future mandates to adopt these three conditions so as to maximize their effectiveness, and thereby the growth of OA.

This brief communication presents preliminary findings on automated Twitter accounts distributing links to scientific papers deposited on the preprint repository arXiv. It discusses the implication of the presence of such bots from the perspective of social media metrics (altmetrics), where mentions of scholarly documents on Twitter have been suggested as a means of measuring impact that is both broader and timelier than citations. We present preliminary findings that automated Twitter accounts create a considerable amount of tweets to scientific papers and that they behave differently than common social bots, which has critical implications for the use of raw tweet counts in research evaluation and assessment. We discuss some definitions of Twitter cyborgs and bots in scholarly communication and propose differentiating between different levels of engagement from tweeting only bibliographic information to discussing or commenting on the content of a paper.

Using three years of the Journal Citation Reports (2011, 2012, and 2013), indicators of change are developed involving only changes of more than one standard deviation above average. Two forms of change are distinguished: monotonic increases or decreases of citation intensity (cited or citing) and critical revisions of the a posteriori (2013) prediction in 2012 on the basis of 2011 data. Changes can be studied at the level of journals using the margin totals of entropy production along the row or column vectors, but also at the level of links among groups of journals by importing the transition matrices into network analysis and visualization programs (and using community-finding algorithms). The latter analysis improves on the former by being more fine-grained. Changes are found in the giant component representing the major disciplinary structures; and significant changes at the level of specialties are indicated in 52 components which are mostly on the applied side of the sciences and engineering.

We compare the network of aggregated journal-journal citation relations provided by the Journal Citation Reports (JCR) 2012 of the Science and Social Science Citation Indexes (SCI and SSCI) with similar data based on Scopus 2012. First, global maps were developed for the two sets separately; sets of documents can then be compared using overlays to both maps. Using fuzzy-string matching and ISSN numbers, we were able to match 10,524 journal names between the two sets; that is, 96.3% of the 10,930 journals contained in JCR or 51.2% of the 20,553 journals covered by Scopus. Network analysis was then pursued on the set of journals shared between the two databases and the two sets of unique journals. Citations among the shared journals are more comprehensively covered in JCR than Scopus, so the network in JCR is denser and more connected than in Scopus. The ranking of shared journals in terms of indegree (that is, numbers of citing journals) or total citations is similar in both databases overall (Spearman's rho > 0.97), but some individual journals rank very differently. Journals that are unique to Scopus seem to be less important--they are citing shared journals rather than being cited by them--but the humanities are covered better in Scopus than in JCR.

The contribution of this article is twofold. First, we present Indexing by latent Dirichlet allocation (LDI), an automatic document indexing method. Many ad hoc applications, or their variants with smoothing techniques suggested in LDA-based language modeling, can result in unsatisfactory performance as the document representations do not accurately reflect concept space. To improve document retrieval performance, we introduce a new definition of document probability vectors in the context of LDA and present a novel scheme for automatic document indexing based on LDA. Second, we propose an Ensemble Model (EnM) for document retrieval. EnM combines basic indexing models by assigning different weights and attempts to uncover the optimal weights to maximize the mean average precision. To solve the optimization problem, we propose an algorithm, which is derived based on the boosting method. The results of our computational experiments on benchmark data sets indicate that both the proposed approaches are viable options for document retrieval.

An extensive analysis of the presence of different altmetric indicators provided by Altmetric.com across scientific fields is presented, particularly focusing on their relationship with citations. Our results confirm that the presence and density of social media altmetric counts are still very low and not very frequent among scientific publications, with 15%-24% of the publications presenting some altmetric activity and concentrating in the most recent publications, although their presence is increasing over time. Publications from the social sciences, humanities and the medical and life sciences show the highest presence of altmetrics, indicating their potential value and interest for these fields. The analysis of the relationships between altmetrics and citations confirms previous claims of positive correlations but relatively weak, thus supporting the idea that altmetrics do not reflect the same concept of impact as citations. Also, altmetric counts do not always present a better filtering of highly cited publications than journal citation scores. Altmetrics scores (particularly mentions in blogs) are able to identify highly cited publications with higher levels of precision than journal citation scores (JCS), but they have a lower level of recall. The value of altmetrics as a complementary tool of citation analysis is highlighted, although more research is suggested to disentangle the potential meaning and value of altmetric indicators for research evaluation.

As the number of scientific journals has multiplied, journal rankings have become increasingly important for scientific decisions. From submissions and subscriptions to grants and hirings, researchers, policy makers, and funding agencies make important decisions with influence from journal rankings such as the ISI journal impact factor. Typically, the rankings are derived from the citation network between a selection of journals and unavoidably depend on this selection. However, little is known about how robust rankings are to the selection of included journals. Here we compare the robustness of three journal rankings based on network flows induced on citation networks. They model pathways of researchers navigating scholarly literature, stepping between journals and remembering their previous steps to different degree: zero-step memory as impact factor, one-step memory as Eigenfactor, and two-step memory, corresponding to zero-, first-, and second-order Markov models of citation flow between journals. We conclude that a second-order Markov model is slightly more robust, because it combines the advantages of the lower-order models: perturbations that remain local and citation weights that depend on journal importance. However, the robustness gain comes at the cost of requiring more data, because the second-order Markov model requires citation data from twice as long a period.

We present an analysis on data citation practices based on the Data Citation Index from Thomson Reuters. This database launched in 2012 aims to link data sets and data studies with citations received from the rest of their citation indexes. Funding bodies and research organizations are increasingly demanding the need of researchers to make their scientific data available in a reusable and reproducible manner, aiming to maximize the allocation of funding while providing transparency on the scientific process. The DCI harvests citations to research data from papers indexed in the Web of Science. It relies on the information provided by the data repository as data citation practices are inconsistent or inexistent in many cases. The findings of this study show that data citation practices are far from common in most research fields. Some differences have been reported on the way researchers cite data: while in the areas of Science and Engineering and Technology data sets were the most cited, in Social Sciences and Arts and Humanities data studies play a greater role. 88.1 percent of the records have received no citations, but some repositories show very low uncitedness rates. While data citation practices are rare in most fields, they have expanded in disciplines such as Crystallography or Genomics. We conclude by emphasizing the role that the DCI could play in encouraging the consistent, standardized citation of research data - a role that would enhance its value as a means of following the research process from data collection to publication.

Since its creation in 1991, arXiv has become central to the diffusion of research in a number of fields. Combining data from the entirety of arXiv and the Web of Science (WoS), this paper investigates (a) the proportion of papers across all disciplines that are on arXiv and the proportion of arXiv papers that are in the WoS, (b) elapsed time between arXiv submission and journal publication, and (c) the aging characteristics and scientific impact of arXiv e-prints and their published version. It shows that the proportion of WoS papers found on arXiv varies across the specialties of physics and mathematics, and that only a few specialties make extensive use of the repository. Elapsed time between arXiv submission and journal publication has shortened but remains longer in mathematics than in physics. In physics, mathematics, as well as in astronomy and astrophysics, arXiv versions are cited more promptly and decay faster than WoS papers. The arXiv versions of papers - both published and unpublished - have lower citation rates than published papers, although there is almost no difference in the impact of the arXiv versions of both published and unpublished papers.

This article introduces the Multidimensional Research Assessment Matrix of scientific output. Its base notion holds that the choice of metrics to be applied in a research assessment process depends upon the unit of assessment, the research dimension to be assessed, and the purposes and policy context of the assessment. An indicator may by highly useful within one assessment process, but less so in another. For instance, publication counts are useful tools to help discriminating between those staff members who are research active, and those who are not, but are of little value if active scientists are to be compared one another according to their research performance. This paper gives a systematic account of the potential usefulness and limitations of a set of 10 important metrics including altmetrics, applied at the level of individual articles, individual researchers, research groups and institutions. It presents a typology of research impact dimensions, and indicates which metrics are the most appropriate to measure each dimension. It introduces the concept of a meta-analysis of the units under assessment in which metrics are not used as tools to evaluate individual units, but to reach policy inferences regarding the objectives and general setup of an assessment process.

Over the last few years, several major scientific fraud cases have shocked the scientific community. The number of retractions each year has also increased tremendously, especially in the biomedical field, and scientific misconduct accounts for approximately more than half of those retractions. It is assumed that co-authors of retracted papers are affected by their colleagues' misconduct, and the aim of this study is to provide empirical evidence of the effect of retractions in biomedical research on co-authors' research careers. Using data from the Web of Science (WOS), we measured the productivity, impact and collaboration of 1,123 co-authors of 293 retracted articles for a period of five years before and after the retraction. We found clear evidence that collaborators do suffer consequences of their colleagues' misconduct, and that a retraction for fraud has higher consequences than a retraction for error. Our results also suggest that the extent of these consequences is closely linked with the ranking of co-authors on the retracted paper, being felt most strongly by first authors, followed by the last authors, while the impact is less important for middle authors.

As a follow-up to the highly-cited authors list published by Thomson Reuters in June 2014, we analyze the top-1% most frequently cited papers published between 2002 and 2012 included in the Web of Science (WoS) subject category "Information Science & Library Science." 798 authors contributed to 305 top-1% publications; these authors were employed at 275 institutions. The authors at Harvard University contributed the largest number of papers, when the addresses are whole-number counted. However, Leiden University leads the ranking, if fractional counting is used. Twenty-three of the 798 authors were also listed as most highly-cited authors by Thomson Reuters in June 2014 (http://highlycited.com/). Twelve of these 23 authors were involved in publishing four or more of the 305 papers under study. Analysis of co-authorship relations among the 798 highly-cited scientists shows that co-authorships are based on common interests in a specific topic. Three topics were important between 2002 and 2012: (1) collection and exploitation of information in clinical practices, (2) the use of internet in public communication and commerce, and (3) scientometrics.

We analyze international co-authorship relations in the Social Science Citation Index 2011 using all citable items in the DVD-version of this index. Network statistics indicate four groups of nations: (i) an Asian-Pacific one to which all Anglo-Saxon nations (including the UK and Ireland) are attributed; (ii) a continental European one including also the Latin-American countries; (iii) the Scandinavian nations; and (iv) a community of African nations. Within the EU-28 (including Croatia), eleven of the EU-15 states have dominant positions. Collapsing the EU-28 into a single node leads to a bi-polar structure between the US and EU-28; China is part of the US-pole. We develop an information-theoretical test to distinguish whether international collaborations or domestic collaborations prevail; the results are mixed, but the international dimension is more important than the national one in the aggregated sets (this was found in both SSCI and SCI). In France, however, the national distribution is more important than the international one, while the reverse is true for most European nations in the core group (UK, Germany, the Netherlands, etc.). Decomposition of the USA in terms of states shows a similarly mixed result; more US states are domestically oriented in SSCI, whereas more internationally in SCI. The international networks have grown during the last decades in addition to the national ones, but not by replacing them.

Using the referencing patterns in articles in Cognitive Science over three decades, we analyze the knowledge base of this literature in terms of its changing disciplinary composition. Three periods are distinguished: (1) construction of the interdisciplinary space in the 1980s; (2) development of an interdisciplinary orientation in the 1990s; (3) reintegration into "cognitive psychology" in the 2000s. The fluidity and fuzziness of the interdisciplinary delineations in the different visualizations can be reduced and clarified using factor analysis. We also explore newly available routines ("CorText") to analyze this development in terms of "tubes" using an alluvial map, and compare the results with an animation (using "visone"). The historical specificity of this development can be compared with the development of "artificial intelligence" into an integrated specialty during this same period. "Interdisciplinarity" should be defined differently at the level of journals and of specialties.

Google Scholar has been well received by the research community. Its promises of free, universal and easy access to scientific literature as well as the perception that it covers better than other traditional multidisciplinary databases the areas of the Social Sciences and the Humanities have contributed to the quick expansion of Google Scholar Citations and Google Scholar Metrics: two new bibliometric products that offer citation data at the individual level and at journal level. In this paper we show the results of a experiment undertaken to analyze Google Scholar's capacity to detect citation counting manipulation. For this, six documents were uploaded to an institutional web domain authored by a false researcher and referencing all the publications of the members of the EC3 research group at the University of Granada. The detection of Google Scholar of these papers outburst the citations included in the Google Scholar Citations profiles of the authors. We discuss the effects of such outburst and how it could affect the future development of such products not only at individual level but also at journal level, especially if Google Scholar persists with its lack of transparency.

Normalization of citation scores using reference sets based on Web-of-Science Subject Categories (WCs) has become an established ("best") practice in evaluative bibliometrics. For example, the Times Higher Education World University Rankings are, among other things, based on this operationalization. However, WCs were developed decades ago for the purpose of information retrieval and evolved incrementally with the database; the classification is machine-based and partially manually corrected. Using the WC "information science & library science" and the WCs attributed to journals in the field of "science and technology studies," we show that WCs do not provide sufficient analytical clarity to carry bibliometric normalization in evaluation practices because of "indexer effects." Can the compliance with "best practices" be replaced with an ambition to develop "best possible practices"? New research questions can then be envisaged.

We prove that Ochiai similarity of the co-occurrence matrix is equal to cosine similarity in the underlying occurrence matrix. Neither the cosine nor the Pearson correlation should be used for the normalization of co-occurrence matrices because the similarity is then normalized twice, and therefore over-estimated; the Ochiai coefficient can be used instead. Results are shown using a small matrix (5 cases, 4 variables) for didactic reasons, and also Ahlgren et al.'s (2003) co-occurrence matrix of 24 authors in library and information sciences. The over-estimation is shown numerically and will be illustrated using multidimensional scaling and cluster dendograms. If the occurrence matrix is not available (such as in internet research or author co-citation analysis) using Ochiai for the normalization is preferable to using the cosine.

The Internet Movie Database (IMDb) is one of the most-visited websites in the world and the premier source for information on films. Like Wikipedia, much of IMDb's information is user contributed. IMDb also allows users to voice their opinion on the quality of films through voting. We investigate whether there is a connection between this user voting data and certain economic film characteristics. To this end, we perform distribution and correlation analysis on a set of films chosen to mitigate effects of bias due to the language and country of origin of films. We show that production budget, box office gross, and total number of user votes for films are consistent with double-log normal distributions for certain time periods. Both total gross and user votes are consistent with a double-log normal distribution from the late 1980s onward, while for budget, it extends from 1935 to 1979. In addition, we find a strong correlation between number of user votes and the economic statistics, particularly budget. Remarkably, we find no evidence for a correlation between number of votes and average user rating. As previous studies have found a strong correlation between production budget and marketing expenses, our results suggest that total user votes is an indicator of a film's prominence or notability, which can be quantified by its promotional costs.

Assuming that meaning cannot be communicated, we extend Shannon's mathematical theory of communication by defining mutual redundancy as a positional counterpart of the relational communication of information. Mutual redundancy indicates the surplus of meanings that can be provided to the exchange of information in reflexive communications. In the three-dimensional case (e.g., a Triple Helix of university-industry-government relations), mutual redundancy is equal to mutual information [R(xyz) = T(xyz)]; but when the dimensionality is even, the sign is different. We generalize to the measurement in N dimensions. Using Luhmann's social-systems theory and/or Giddens' structuration theory, mutual redundancy can be provided with an interpretation in the sociological case: different meaning-processing structures code and decode with other algorithms. A surplus of ("absent") options can then be generated that add to the redundancy. Luhmann's "functional (sub)systems" of expectations or Giddens' "rule-resource sets" are positioned mutually, but can be coupled operationally in events or "instantiated" in actions. Shannon-type information is generated by the mediation. The "structures" are re-positioned towards one another as sets of (potentially counterfactual) expectations.

F1000 recommendations have been validated as a potential data source for research evaluation, but reasons for differences between FFa scores and citations remain to be explored. By linking 30640 publications in F1000 to citations in Scopus, we investigated the effect of research level and article type on the internal consistency of assessment results based on citations and FFa scores. It turns out that research level has little impact, while article type has big effect on the differences. The two measures are significantly different for two groups: non-primary research or evidence-based research publications are more highly cited rather than highly recommended, however, translational research or transformative research publications are more highly recommended by faculties but gather relatively lower citations. It is logical because citation routines are usually practiced by academic authors while the potential for scientific revolutions and the suitability for clinical practice of an article should be investigated from the practitioners' points of view. We conclude with a policy relevant recommendation that the application of bibliometric approaches in research evaluation procedures could include the proportion of three types of publications: evidence-based research, transformative research, and translational research. The latter two types are more suitable to be assessed through peer review.

A few weeks ago, Thomson Reuters published a list of the highly cited researchers worldwide (highlycited.com). Since the data is freely available for downloading and includes the names of the researchers' institutions, we produced a ranking of the institutions on the basis of the number of highly cited researchers per institution.

Using Web-of-Science data, portfolio analysis in terms of journal coverage can be projected on a base map for units of analysis such as countries, cities, universities, and firms. The units of analysis under study can be compared statistically across the 10,000+ journals. The interdisciplinarity of the portfolios can be measured using Rao-Stirling diversity or Zhang et al.'s (in press) improved measure 2D3. At the country level we find regional differentiation (e.g., Latin-American or Asian countries), but also a major divide between advanced and less-developed countries. Israel and Israeli cities outperform other nations and cities in terms of diversity. Universities appear to be specifically related to firms when a number of these units are exploratively compared. The instrument is relatively simple and straightforward, and one can generalize the application to any document set retrieved from WoS. Further instruction is provided online at http://www.leydesdorff.net/portfolio.