ArticlePDF Available

Welcome to the Tidyverse

Authors:
Hadley Wickham
Hadley Wickham
Hadley Wickham
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Mara Averick
Mara Averick
Jennifer Bryan
Jennifer Bryan
Jennifer Bryan
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Winston Chang
Winston Chang
Winston Chang
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Show all 24 authorsHide
Download full-text PDF
Read full-text
Download citation
Available via license: CC BY 4.0
Content may be subject to copyright.
Welcome to the Tidyverse
Hadley Wickham1, Mara Averick1, Jennifer Bryan1, Winston Chang1,
Lucy D’Agostino McGowan8, Romain François1, Garrett Grolemund1,
Alex Hayes12, Lionel Henry1, Jim Hester1, Max Kuhn1, Thomas Lin
Pedersen1, Evan Miller13, Stephan Milton Bache3, Kirill Müller2,
Jeroen Ooms14, David Robinson5, Dana Paige Seidel10, Vitalie Spinu4,
Kohske Takahashi9, Davis Vaughan1, Claus Wilke6, Kara Woo7, and
Hiroaki Yutani11
1RStudio 2cynkra 3Redbubble 4Erasmus University Rotterdam 5Flatiron Health 6Department
of Integrative Biology, The University of Texas at Austin 7Sage Bionetworks 8Department of
Biostatistics, Johns Hopkins Bloomberg School of Public Health 9Chukyo University, Japan 10
Department of Environmental Science, Policy, & Management, University of California, Berkeley 11
LINE Corporation 12 University of Wisconsin, Madison 13 None 14 University of California, Berkeley
DOI: 10.21105/joss.01686
Software
•Review
•Repository
•Archive
Editor: Karthik Ram
Reviewers:
•@ldecicco-USGS
•@jereyhanson
Submitted: 09 August 2019
Published: 21 November 2019
License
Authors of papers retain
copyright and release the work
under a Creative Commons
Attribution 4.0 International
License (CC-BY).
Summary
At a high level, the tidyverse is a language for solving data science challenges with R code. Its
primary goal is to facilitate a conversation between a human and a computer about data. Less
abstractly, the tidyverse is a collection of R packages that share a high-level design philosophy
and low-level grammar and data structures, so that learning one package makes it easier to
learn the next.
The tidyverse encompasses the repeated tasks at the heart of every data science project:
data import, tidying, manipulation, visualisation, and programming. We expect that almost
every project will use multiple domain-specic packages outside of the tidyverse: our goal
is to provide tooling for the most common challenges; not to solve every possible problem.
Notably, the tidyverse doesn’t include tools for statistical modelling or communication. These
toolkits are critical for data science, but are so large that they merit separate treatment. The
tidyverse package allows users to install all tidyverse packages with a single command.
There are a number of projects that are similar in scope to the tidyverse. The closest is perhaps
Bioconductor (Gentleman et al., 2004; Huber et al., 2015), which provides an ecosystem of
packages that support the analysis of high-throughput genomic data. The tidyverse has similar
goals to R itself, but any comparison to the R Project (R Core Team, 2019) is fundamentally
challenging as the tidyverse is written in R, and relies on R for its infrastructure; there is
no tidyverse without R! That said, the biggest dierence is in priorities: base R is highly
focussed on stability, whereas the tidyverse will make breaking changes in the search for
better interfaces. Another closely related project is data.table (Dowle & Srinivasan, 2019),
Wickham et al., (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686 1
which provides tools roughly equivalent to the combination of dplyr, tidyr, tibble, and readr.
data.table prioritises concision and performance.
This paper describes the tidyverse package, the components of the tidyverse, and some of the
underlying design principles. This is a lot of ground to cover in a brief paper, so we focus on
a 50,000-foot view showing how all the pieces t together with copious links to more detailed
resources.
Tidyverse package
The tidyverse is a collection of packages that can easily be installed with a single “meta”-
package, which is called “tidyverse”. This provides a convenient way of downloading and
installing all tidyverse packages with a single R command:
install.packages("tidyverse")
The core tidyverse includes the packages that you’re likely to use in everyday data analyses,
and these are attached when you attach the tidyverse package:
library(tidyverse)
#> -- Attaching packages --------------------------- tidyverse 1.2.1 --
#> v ggplot2 3.2.1 v purrr 0.3.3
#> v tibble 2.1.3 v dplyr 0.8.3
#> v tidyr 1.0.0 v stringr 1.4.0
#> v readr 1.3.1 v forcats 0.4.0
#> -- Conflicts ------------------------------ tidyverse_conflicts() --
#> x dplyr::filter() masks stats::filter()
#> x dplyr::lag() masks stats::lag()
This is a convenient shortcut for attaching the core packages, produces a short report telling
you which package versions you’re using, and succinctly informs you of any conicts with
previously loaded packages. As of tidyverse version 1.2.0, the core packages include dplyr
(Wickham et al., 2019a), forcats (Wickham, 2019a), ggplot2 (Wickham, 2016), purrr (Henry
& Wickham, 2019), readr (Wickham & Hester, 2018), stringr (Wickham, 2019b), tibble
(Müller & Wickham, 2018), and tidyr (Wickham & Henry, 2019).
Non-core packages are installed with install.packages("tidyverse"), but are not at-
tached by library(tidyverse). They play more specialised roles, so will be attached by the
analyst as needed. The non-core packages are: blob (Wickham, 2018a), feather (Wickham,
2019c), jsonlite (Ooms, 2014), glue (Hester, 2018), googledrive (D’Agostino McGowan &
Bryan, 2019), haven (Wickham & Miller, 2018), hms (Müller, 2018), lubridate (Spinu, Grole-
mund, & Wickham, 2018), magrittr (Bache & Wickham, 2014), modelr (Wickham, 2018b),
readxl (Wickham & Bryan, 2019), reprex (Bryan, Hester, Robinson, & Wickham, 2019), rvest
(Wickham, 2019d), and xml2 (Wickham et al., 2019b).
The tidyverse package is designed with an eye for teaching: install.packages("tidyver
se") gets you a “batteries-included” set of 87 packages (at time of writing). This large set
of dependencies means that it is not appropriate to use the tidyverse package within another
package; instead, we recommend that package authors import only the specic packages that
they use.
Wickham et al., (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686 2
Components
How do the component packages of the tidyverse t together? We use the model of data
science tools from “R for Data Science” (Wickham & Grolemund, 2017):
Every analysis starts with data import: if you can’t get your data into R, you can’t do data
science on it! Data import takes data stored in a le, database, or behind a web API, and
reads it into a data frame in R. Data import is supported by the core readr (Wickham &
Hester, 2018) package for tabular les (like csv, tsv, and fwf).
Additional non-core packages, such as readxl (Wickham & Bryan, 2019), haven (Wickham
& Miller, 2018), googledrive (D’Agostino McGowan & Bryan, 2019), and rvest (Wickham,
2019d), make it possible to import data stored in other common formats or directly from the
web.
Next, we recommend that you tidy your data, getting it into a consistent form that makes
the rest of the analysis easier. Most functions in the tidyverse work with tidy data (Wickham,
2014), where every column is a variable, every row is an observation, and every cell contains a
single value. If your data is not already in this form (almost always!), the core tidyr (Wickham
& Henry, 2019) package provides tools to tidy it up.
Data transformation is supported by the core dplyr (Wickham et al., 2019a) package. dplyr
provides verbs that work with whole data frames, such as mutate() to create new variables,
filter() to nd observations matching given criteria, and left_join() and friends to
combine multiple tables. dplyr is paired with packages that provide tools for specic column
types:
•stringr for strings.
•forcats for factors, R’s categorical data type.
•lubridate (Spinu et al., 2018) for dates and date-times.
•hms (Müller, 2018) for clock times.
There are two main tools for understanding data: visualisation and modelling. The tidyverse
provides the ggplot2 (Wickham, 2016) package for visualisation. ggplot2 is a system for
declaratively creating graphics, based on The Grammar of Graphics (Wilkinson, 2005).
You provide the data, tell ggplot2 how to map variables to aesthetics, what graphical primitives
to use, and it takes care of the details. Modelling is outside the scope of this paper, but is part
of the closely aliated tidymodels (Kuhn & Wickham, 2018) project, which shares interface
design and data structures with the tidyverse.
Finally, you’ll need to communicate your results to someone else. Communication is one
of the most important parts of data science, but is not included within tidyverse. Instead,
we expect people will use other R packages, like rmarkdown (Allaire et al., 2018) and shiny
(Chang, Cheng, Allaire, Xie, & McPherson, 2019), which support dozens of static and dynamic
output formats.
Wickham et al., (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686 3
Surrounding all these tools is programming. Programming is a cross-cutting tool that you
use in every part of a data science project. Programming tools in the tidyverse include:
•purrr (Henry & Wickham, 2019), which enhances R’s functional programming toolkit.
•tibble (Müller & Wickham, 2018), which provides a modern re-imagining of the vener-
able data frame, keeping what time has proven to be eective, and throwing out what
it has not.
•reprex (Bryan et al., 2019), which helps programmers get help when they get stuck by
easing the creation of reproducible examples.
•magrittr (Bache & Wickham, 2014), which provides the pipe operator, %>%, used
throughout the tidyverse. The pipe is a tool for function composition, making it easier
to solve large problems by breaking them into small pieces.
Design principles
We are still working to explicitly describe the unifying principles that make the tidyverse
consistent, but you can read our latest thoughts at https://design.tidyverse.org/. There is one
particularly important principle that we want to call out here: the tidyverse is fundamentally
human centred. That is, the tidyverse is designed to support the activities of a human data
analyst, so to be eective tool builders, we must explicitly recognise and acknowledge the
strengths and weaknesses of human cognition.
This is particularly important for R, because it’s a language that’s used primarily by non-
programmers, and we want to make it as easy as possible for rst-time and end-user program-
mers to learn the tidyverse. We believe deeply in the motivations that lead to the creation of
S: “to turn ideas into software, quickly and faithfully” (Chambers, 1998). This means that we
spend a lot of time thinking about interface design, and have recently started experimenting
with surveys to help guide interface choices.
Similarly, the tidyverse is not just the collection of packages — it is also the community of
people who use them. We want the tidyverse to be a diverse, inclusive, and welcoming com-
munity. We are still developing our skills in this area, but our existing approaches include
active use of Twitter to solicit feedback, announce updates, and generally listen to the com-
munity. We also keep users apprised of major upcoming changes through the tidyverse blog,
run developer days, and support lively discussions on RStudio community.
Acknowledgments
The tidyverse would not be possible without the immense work of the R-core team who
maintain the R language and we are deeply indebted to them. We are also grateful for the
nancial support of RStudio, Inc.
References
Allaire, J., Xie, Y., McPherson, J., Luraschi, J., Ushey, K., Atkins, A., Wickham, H., et al.
(2018). rmarkdown: Dynamic documents for R. Retrieved from https://rmarkdown.rstudio.
com
Bache, S. M., & Wickham, H. (2014). magrittr: A forward-pipe operator for R. Retrieved
from https://CRAN.R-project.org/package=magrittr
Wickham et al., (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686 4
Bryan, J., Hester, J., Robinson, D., & Wickham, H. (2019). reprex: Prepare reproducible
example code via the clipboard. Retrieved from https://CRAN.R-project.org/package=reprex
Chambers, J. M. (1998). Programming with data: A guide to the S language. Springer.
Chang, W., Cheng, J., Allaire, J., Xie, Y., & McPherson, J. (2019). shiny: Web application
framework for R. Retrieved from https://CRAN.R-project.org/package=shiny
D’Agostino McGowan, L., & Bryan, J. (2019). googledrive: An interface to google drive.
Retrieved from https://CRAN.R-project.org/package=googledrive
Dowle, M., & Srinivasan, A. (2019). data.table: Extension of ‘data.frame‘. Retrieved from
https://CRAN.R-project.org/package=data.table
Gentleman, R. C., Carey, V. J., Bates, D. M., Bolstad, B., Dettling, M., Dudoit, S., Ellis,
B., et al. (2004). Bioconductor: Open software development for computational biology and
bioinformatics. Genome Biology,5(10), R80.
Henry, L., & Wickham, H. (2019). purrr: Functional programming tools. Retrieved from
https://CRAN.R-project.org/package=purrr
Hester, J. (2018). glue: Interpreted string literals. Retrieved from https://CRAN.R-project.
org/package=glue
Huber, W., Carey, V. J., Gentleman, R., Anders, S., Carlson, M., Carvalho, B. S., Bravo, H.
C., et al. (2015). Orchestrating high-throughput genomic analysis with Bioconductor. Nature
Methods,12(2), 115–121. Retrieved from http://www.nature.com/nmeth/journal/v12/n2/
full/nmeth.3252.html
Kuhn, M., & Wickham, H. (2018). tidymodels: Easily install and load the ’tidymodels’
packages. Retrieved from https://CRAN.R-project.org/package=tidymodels
Müller, K. (2018). hms: Pretty time of day. Retrieved from https://CRAN.R-project.org/
package=hms
Müller, K., & Wickham, H. (2018). tibble: Simple data frames. Retrieved from https:
//CRAN.R-project.org/package=tibble
Ooms, J. (2014). The jsonlite package: A practical and consistent mapping between JSON
data and R objects. arXiv:1403.2805 [stat.CO]. Retrieved from https://arxiv.org/abs/1403.
2805
R Core Team. (2019). R: A language and environment for statistical computing. Vienna,
Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/
Spinu, V., Grolemund, G., & Wickham, H. (2018). lubridate: Make dealing with dates a little
easier. Retrieved from https://CRAN.R-project.org/package=lubridate
Wickham, H. (2014). Tidy data. The Journal of Statistical Software,59.
Wickham, H. (2016). ggplot2: Elegant graphics for data analysis. useR. Springer.
Wickham, H. (2018a). blob: A simple s3 class for representing vectors of binary data (’blobs’).
Retrieved from https://CRAN.R-project.org/package=blob
Wickham, H. (2018b). modelr: Modelling functions that work with the pipe. Retrieved from
https://CRAN.R-project.org/package=modelr
Wickham, H. (2019a). forcats: Tools for working with categorical variables (factors). Re-
trieved from https://CRAN.R-project.org/package=forcats
Wickham, H. (2019b). stringr: Simple, consistent wrappers for common string operations.
Retrieved from https://CRAN.R-project.org/package=stringr
Wickham, H. (2019c). Feather: R bindings to the feather API. Retrieved from https://CRAN.
R-project.org/package=feather
Wickham et al., (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686 5
Wickham, H. (2019d). rvest: Easily harvest (scrape) web pages. Retrieved from https:
//CRAN.R-project.org/package=rvest
Wickham, H., & Bryan, J. (2019). readxl: Read excel les. Retrieved from https://CRAN.
R-project.org/package=readxl
Wickham, H., François, R., Henry, L., & Müller, K. (2019a). dplyr: A grammar of data
manipulation. Retrieved from https://CRAN.R-project.org/package=dplyr
Wickham, H., & Grolemund, G. (2017). R for data science: Import, tidy, transform, visualize,
and model data. O’Reilly Media, Inc.
Wickham, H., & Henry, L. (2019). tidyr: Tidy messy data. Retrieved from https://CRAN.
R-project.org/package=tidyr
Wickham, H., & Hester, J. (2018). readr: Read rectangular text data. Retrieved from
https://CRAN.R-project.org/package=readr
Wickham, H., Hester, J., & Ooms, J. (2019b). xml2: Parse XML. Retrieved from https:
//CRAN.R-project.org/package=xml2
Wickham, H., & Miller, E. (2018). haven: Import and export SPSS, Stata, and SAS les.
Retrieved from https://CRAN.R-project.org/package=haven
Wilkinson, L. (2005). The grammar of graphics. Berlin, Heidelberg: Springer-Verlag. doi:10.
1007/0-387-28695-0
Wickham et al., (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686 6
... Illustrations (unless stated otherwise), analysis of variance (ANOVA), and Tukey's Honest Significant Difference tests were performed using R Statistical Software (v4.4.2; R Core Team, 2024) with the R Packages tidyvere (v2.0.0; Wickham et al., 2019) and rstatix (v0.7.2; Kassambara, 2023). ...
Formate addition enhanced hydrogen production by Thermococcus paralvinellae when grown on brewery wastewater
Article
Full-text available
  • Mar 2025
The hyperthermophilic archaeon Thermococcus paralvinellae produces H 2 when grown on carbohydrates or protein with increased H 2 production when cultures are grown on formate. This study examined the use of brewery wastewater as a feedstock for H 2 production, the addition of formate to enhance H 2 production, and the activities of hydrogenases and formate hydrogenlyase under varying growth conditions as markers of performance. T. paralvinellae was grown at 80°C on maltose only (a model brewery wastewater), formate only, and maltose plus formate media as well as brewery wastewater with and without the addition of formate. Growth rates were higher on formate only medium than on maltose only and brewery wastewater only media. H 2 yield per cell was higher in all media containing formate relative to those without formate. Hydrogenase and formate hydrogenlyase specific activities were not affected by the presence of formate and were largely consistent across all growth conditions. Growth rates were consistent in media containing 0.05 to 2.5% (wt/vol) maltose only, but total H 2 production doubled from medium containing 0.05% maltose to 0.5% maltose and remained unchanged at higher maltose concentrations. Cells grown in a 2 L N 2 flushed batch bioreactor at 80°C on brewery wastewater with and without formate showed no difference in growth rates but the amount of H 2 in the headspace was six times higher when formate was present. However, the amount of H 2 produced by cells grown on brewery wastewater plus formate peaked in mid-logarithmic growth phase and then decreased to amounts produced by cells without formate addition by late logarithmic growth phase. When the bioreactor was run as a chemostat, the addition of formate to brewery wastewater led to a 12-fold increase in the amount of H 2 present in the headspace that was sustained over time relative to growth without formate addition. Therefore, T. paralvinellae grows on brewery wastewater as its sole source of organic carbon and produces biohydrogen at a steady rate in a pilot-scale bioreactor, and H 2 production is enhanced by formate addition.
View
View
View
View
View
View
View
View
Glove and instrument handling in small animal oncological surgeries: a survey
Article
  • Mar 2025
  • J SMALL ANIM PRACT
Objectives To evaluate veterinary surgeons' awareness of the potential for surgical gloves and instruments to act as vectors for tumour seeding in small animal oncological surgery and to assess the use of specific protocols to mitigate this risk. Materials and Methods A 21‐question survey was developed and distributed to small animal veterinary surgeons, focusing on practices related to glove and instrument handling during oncological surgeries. The survey targeted veterinary surgeons who regularly performed oncological procedures, and was analysed using descriptive statistics and Pearson's chi‐square analysis. Results A total of 194 veterinary surgeons participated. Most respondents (89%) reported changing gloves and instruments during oncological surgeries to avoid tumour seeding. Surgeons with advanced qualifications and those working in referral hospitals were more likely to implement these practices. Additionally, surgeons with a higher oncological caseload were more likely to follow protocols for wound protection. The majority (74%) of respondents believed that there was a risk of neoplastic cells on gloves or instruments, and 98% expressed a strong interest in evidence‐based guidelines. Clinical Significance This study highlights a significant awareness among veterinary surgeons regarding the risk of tumour seeding through surgical gloves and instruments. There is a clear willingness among practitioners to adopt new guidelines and improve practices based on emerging evidence, indicating a potential shift towards more stringent protocols in small animal oncological surgeries.
View
A chromosome-level genome assembly of the varied leaved jewelflower, Streptanthus diversifolius, reveals a recent whole genome duplication
Article
  • Mar 2025
The Streptanthoid complex, a clade of primarily Streptanthus and Caulanthus species in the Thelypodieae (Brassicaceae) is an emerging model system for ecological and evolutionary studies. This complex spans the full range of the California Floristic Province including desert, foothill, and mountain environments. The ability of these related species to radiate into dramatically different environments makes them a desirable study subject for exploring how plant species expand their ranges and adapt to new environments over time. Ecological and evolutionary studies for this complex have revealed fascinating variation in serpentine soil adaptation, defense compounds, germination, flowering, and life history strategies. Until now a lack of publicly available genome assemblies has hindered the ability to relate these phenotypic observations to their underlying genetic and molecular mechanisms. To help remedy this situation, we present here a chromosome-level genome assembly and annotation of Streptanthus diversifolius, a member of the Streptanthoid Complex, developed using Illumina, Hi-C, and HiFi sequencing technologies. Construction of this assembly also provides further evidence to support the previously reported recent whole genome duplication unique to the Thelypodieae. This whole genome duplication may have provided individuals in the Streptanthoid Complex the genetic arsenal to rapidly radiate throughout the California Floristic Province and to occupy commonly inhospitable environments including serpentine soils.
View
Orchestrating high-throughput genomic analysis with Bioconductor
Article
Full-text available
  • Jan 2015
  • Br J Pharmacol
Bioconductor is an open-source, open-development software project for the analysis and comprehension of high-throughput data in genomics and molecular biology. The project aims to enable interdisciplinary research, collaboration and rapid development of scientific software. Based on the statistical programming language R, Bioconductor comprises 934 interoperable packages contributed by a large, diverse community of scientists. Packages cover a range of bioinformatic and statistical applications. They undergo formal initial review and continuous automated testing. We present an overview for prospective users and contributors.
View
Tidy data
Article
Full-text available
  • Sep 2014
  • J STAT SOFTW
It’s often said that 80% of the effort of analysis is in data cleaning. Despite the amount of time it takes, there has been little research on how to clean data well. This paper attempts to tackle a small, but important, subset of data cleaning: data “tidying”. “Tidy” data is easy to manipulate, model and visualise, and has a specific structure: variables are stored in columns, observations in rows, and one type of experimental unit per file. This structure makes it easy to tidy messy data, because only a small set of tools are needed to deal with a large number of messy data sets. Tidy data is only useful in conjunction with tidy tools, tools that input and output tidy data. Tidy tools for manipulation, modelling, and visualisation are described, and the notion of tidy tools is used to critique existing messy tools.
View
Bioconductor: Open Software Development for Computational Biology and Bioinformatics
Article
Full-text available
  • Sep 2004
The Bioconductor project is an initiative for the collaborative creation of extensible software for computational biology and bioinformatics. The goals of the project include: fostering collaborative development and widespread use of innovative software, reducing barriers to entry into interdisciplinary scientific research, and promoting the achievement of remote reproducibility of research results. We describe details of our aims and methods, identify current challenges, compare Bioconductor to other open bioinformatics projects, and provide working examples.
View
The Grammar of Graphics
Book
  • Jan 2005
This book was written for statisticians, computer scientists, geographers, researchers, and others interested in visualizing data. It presents a unique foundation for producing almost every quantitative graphic found in scientific journals, newspapers, statistical packages, and data visualization systems. While the tangible results of this work have been several visualization software libraries, this book focuses on the deep structures involved in producing quantitative graphics from data. What are the rules that underlie the production of pie charts, bar charts, scatterplots, function plots, maps, mosaics, and radar charts? Those less interested in the theoretical and mathematical foundations can still get a sense of the richness and structure of the system by examining the numerous and often unique color graphics it can produce. The second edition is almost twice the size of the original, with six new chapters and substantial revision. Much of the added material makes this book suitable for survey courses in visualization and statistical graphics. From reviews of the first edition: "Destined to become a landmark in statistical graphics, this book provides a formal description of graphics, particularly static graphics, playing much the same role for graphics as probability theory played for statistics." Journal of the American Statistical Association "Wilkinson’s careful scholarship shows around every corner. This is a tour de force of the highest order." Psychometrika "All geography and map libraries should add this book to their collections; the serious scholar of quantitative data graphics will place this book on the same shelf with those by Edward Tufte, and volumes by Cleveland, Bertin, Monmonier, MacEachren, among others, and continue the unending task of proselytizing for the best in statistical data presentation by example and through scholarship like that of Leland Wilkinson." Cartographic Perspectives "In summary, this is certainly a remarkable book and a new ambitious step for the development and application of statistical graphics." Computational Statistics and Data Analysis About the author: Leland Wilkinson is Senior VP, SPSS Inc. and Adjunct Professor of Statistics at Northwestern University. He is also affiliated with the Computer Science department at The University of Illinois at Chicago. He wrote the SYSTAT statistical package and founded SYSTAT Inc. in 1984. Wilkinson joined SPSS in a 1994 acquisition and now works on research and development of visual analytics and statistics. He is a Fellow of the ASA. In addition to journal articles and the original SYSTAT computer program and manuals, Wilkinson is the author (with Grant Blank and Chris Gruber) of Desktop Data Analysis with SYSTAT.
View
View
View
View
View
The jsonlite Package: A Practical and Consistent Mapping Between JSON Data and R Objects
Article
  • Mar 2014
A naive realization of JSON data in R maps JSON arrays to an unnamed list, and JSON objects to a named list. However, in practice a list is an awkward, inefficient type to store and manipulate data. Most statistical applications work with (homogeneous) vectors, matrices or data frames. Therefore JSON packages in R typically define certain special cases of JSON structures which map to simpler R types. Currently there exist no formal guidelines, or even consensus between implementations on how R data should be represented in JSON. Furthermore, upon closer inspection, even the most basic data structures in R actually do not perfectly map to their JSON counterparts and leave some ambiguity for edge cases. These problems have resulted in different behavior between implementations and can lead to unexpected output. This paper explicitly describes a mapping between R classes and JSON data, highlights potential problems, and proposes conventions that generalize the mapping to cover all common structures. We emphasize the importance of type consistency when using JSON to exchange dynamic data, and illustrate using examples and anecdotes. The jsonlite R package is used throughout the paper as a reference implementation.
View
View