Introduction to R

The examples in this workshop use the baby names data provided by the governments of the United States and the United Kingdom. A cleaned and merged version of these data is in dataSets/babyNames.csv.

R knows the directory it was started in, and refers to this as the "working directory". Since our workshop examples are in the Rintro folder in your Downloads folder, we should all take a moment to set that as our working directory.

getwd() # what is my current working directory?
# setwd("~/Desktop/Rintro") # change directory

Note that "~" means "my home directory" but that this can mean different things on different operating systems. You can also use the Files tab in Rstudio to navigate to a directory, then click "More -> Set as working directory".

We can a set the working directory using paths relative to the current working directory. Once we are in the "Rintro" folder we can navigate to the "dataSets" folder like this:

getwd() # get the current working directory
setwd("dataSets") # set wd to the dataSets folder
getwd()
setwd("..") # set wd to enclosing folder ("up")

> getwd() # get the current working directory
[1] "/home/izahn/Documents/Work/Classes/IQSS_Stats_Workshops/R/Rintro"
> setwd("dataSets") # set wd to the dataSets folder
> getwd()
[1] "/home/izahn/Documents/Work/Classes/IQSS_Stats_Workshops/R/Rintro/dataSets"
> setwd("..") # set wd to enclosing folder ("up")
>

It can be convenient to list files in a directory without leaving R

list.files("dataSets") # list files in the dataSets folder

> list.files("dataSets") # list files in the dataSets folder
[1] "babyNames.csv"
>

In order to read data from a file, you have to know what kind of file it is. The table below lists the functions that can import data from common file formats.

The purpose of this exercise is to practice reading data into R. The data in "dataSets/babyNames.csv" is moderately tricky to read, making it a good data set to practice on.

1. Open the help page for the read_csv function. How can you limit the number of rows to be read in?
2. Read just the first 10 rows of "dataSets/babyNames.csv". Notice that the "Sex" column has been read as a logical (TRUE/FALSE).
3. Read the read_csv help page to figure out how to make it read the "Sex" column as a character. Make adjustments to your code until you have read in the first 10 rows with the correct column types. "Year" and "Name.length" should be integer (int), "Count" and "Percent" should be double (dbl) and everything else should be character (chr).
4. Once you have successfully read in the first 10 rows, read the whole file, assigning the result to the name baby.names.

It is always a good idea to examine the imported data set–usually we want the results to be a data.frame

## we know that this object will have mode and length, because all R objects do.
mode(baby.names)
length(baby.names) # number of columns

## additional information about this data object
class(baby.names) # check to see that test is a data.frame
dim(baby.names) # how many rows and columns?
names(baby.names) # or colnames(baby.names)
str(baby.names) # more details
glimpse(baby.names) # details, more compactly

> ## we know that this object will have mode and length, because all R objects do.
> mode(baby.names)
[1] "list"
> length(baby.names) # number of columns
[1] 7
> 
> ## additional information about this data object
> class(baby.names) # check to see that test is a data.frame
[1] "tbl_df"     "tbl"        "data.frame"
> dim(baby.names) # how many rows and columns?
[1] 1966001       7
> names(baby.names) # or colnames(baby.names)
[1] "Location"    "Year"        "Sex"         "Name"       
[5] "Count"       "Percent"     "Name.length"
> str(baby.names) # more details
Classes 'tbl_df', 'tbl' and 'data.frame':	1966001 obs. of  7 variables:
 $ Location   : chr  "England and Wales" "England and Wales" "England and Wales" "England and Wales" ...
 $ Year       : int  1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 ...
 $ Sex        : chr  "F" "F" "F" "F" ...
 $ Name       : chr  "sophie" "chloe" "jessica" "emily" ...
 $ Count      : num  7087 6824 6711 6415 6299 ...
 $ Percent    : num  2.39 2.31 2.27 2.17 2.13 ...
 $ Name.length: int  6 5 7 5 6 6 9 7 3 5 ...
 - attr(*, "spec")=List of 2
  ..$ cols   :List of 7
  .. ..$ Location   : list()
  .. .. ..- attr(*, "class")= chr  "collector_character" "collector"
  .. ..$ Year       : list()
  .. .. ..- attr(*, "class")= chr  "collector_integer" "collector"
  .. ..$ Sex        : list()
  .. .. ..- attr(*, "class")= chr  "collector_character" "collector"
  .. ..$ Name       : list()
  .. .. ..- attr(*, "class")= chr  "collector_character" "collector"
  .. ..$ Count      : list()
  .. .. ..- attr(*, "class")= chr  "collector_double" "collector"
  .. ..$ Percent    : list()
  .. .. ..- attr(*, "class")= chr  "collector_double" "collector"
  .. ..$ Name.length: list()
  .. .. ..- attr(*, "class")= chr  "collector_integer" "collector"
  ..$ default: list()
  .. ..- attr(*, "class")= chr  "collector_guess" "collector"
  ..- attr(*, "class")= chr "col_spec"
> glimpse(baby.names) # details, more compactly
Observations: 1,966,001
Variables: 7
$ Location    <chr> "England and Wales", "England and Wales", "En...
$ Year        <int> 1996, 1996, 1996, 1996, 1996, 1996, 1996, 199...
$ Sex         <chr> "F", "F", "F", "F", "F", "F", "F", "F", "F", ...
$ Name        <chr> "sophie", "chloe", "jessica", "emily", "laure...
$ Count       <dbl> 7087, 6824, 6711, 6415, 6299, 5916, 5866, 582...
$ Percent     <dbl> 2.3942729, 2.3054210, 2.2672450, 2.1672444, 2...
$ Name.length <int> 6, 5, 7, 5, 6, 6, 9, 7, 3, 5, 7, 5, 7, 4, 4, ...
>

Usually data read into R will be stored as a data.frame

• A data.frame is a list of vectors of equal length
  • Each vector in the list forms a column
  • Each column can be a differnt type of vector
  • Typically columns are variables and the rows are observations

A data.frame has two dimensions corresponding the number of rows and the number of columns (in that order)

You can extract subsets of data.frames using slice to select rows by number and filter to select rows that match some condition. It works like this:

## make up some example data
(example.df <- data.frame(id  = rep(letters[1:4], each = 4),
                          t   = rep(1:4, times = 4),
                          var1 = runif(16),
                          var2 = sample(letters[1:3], 16, replace = TRUE)))

## rows 2 and 4
slice(example.df, c(2, 4))

## rows where id == "a"
filter(example.df, id == "a")

## rows where id is either "a" or "b"
filter(example.df, id %in% c("a", "b"))

> ## make up some example data
> (example.df <- data.frame(id  = rep(letters[1:4], each = 4),
+                           t   = rep(1:4, times = 4),
+                           var1 = runif(16),
+                           var2 = sample(letters[1:3], 16, replace = TRUE)))
   id t       var1 var2
1   a 1 0.19158254    c
2   a 2 0.46921828    c
3   a 3 0.71092483    c
4   a 4 0.81892913    c
5   b 1 0.17894751    b
6   b 2 0.94742350    c
7   b 3 0.24051714    c
8   b 4 0.41236477    c
9   c 1 0.66573374    c
10  c 2 0.72515137    b
11  c 3 0.66923824    b
12  c 4 0.41101666    b
13  d 1 0.06401198    c
14  d 2 0.34580213    b
15  d 3 0.44477036    c
16  d 4 0.12253790    b
> 
> ## rows 2 and 4
> slice(example.df, c(2, 4))
  id t      var1 var2
1  a 2 0.4692183    c
2  a 4 0.8189291    c
> 
> ## rows where id == "a"
> filter(example.df, id == "a")
  id t      var1 var2
1  a 1 0.1915825    c
2  a 2 0.4692183    c
3  a 3 0.7109248    c
4  a 4 0.8189291    c
> 
> ## rows where id is either "a" or "b"
> filter(example.df, id %in% c("a", "b"))
  id t      var1 var2
1  a 1 0.1915825    c
2  a 2 0.4692183    c
3  a 3 0.7109248    c
4  a 4 0.8189291    c
5  b 1 0.1789475    b
6  b 2 0.9474235    c
7  b 3 0.2405171    c
8  b 4 0.4123648    c
>

slice and filter are used to extract rows. select is used to extract columns

select(example.df, id, var1)
select(example.df, id, t, var1)

> select(example.df, id, var1)
   id       var1
1   a 0.19158254
2   a 0.46921828
3   a 0.71092483
4   a 0.81892913
5   b 0.17894751
6   b 0.94742350
7   b 0.24051714
8   b 0.41236477
9   c 0.66573374
10  c 0.72515137
11  c 0.66923824
12  c 0.41101666
13  d 0.06401198
14  d 0.34580213
15  d 0.44477036
16  d 0.12253790
> select(example.df, id, t, var1)
   id t       var1
1   a 1 0.19158254
2   a 2 0.46921828
3   a 3 0.71092483
4   a 4 0.81892913
5   b 1 0.17894751
6   b 2 0.94742350
7   b 3 0.24051714
8   b 4 0.41236477
9   c 1 0.66573374
10  c 2 0.72515137
11  c 3 0.66923824
12  c 4 0.41101666
13  d 1 0.06401198
14  d 2 0.34580213
15  d 3 0.44477036
16  d 4 0.12253790
>

You can also conveniently select a single column using $, like this:

example.df$t

> example.df$t
 [1] 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
>

Data manipulation commands can be combined:

filter(select(example.df,
              id,
              var1),
       id == "a")

> filter(select(example.df,
+               id,
+               var1),
+        id == "a")
  id      var1
1  a 0.1915825
2  a 0.4692183
3  a 0.7109248
4  a 0.8189291
>

In the previous example we used == to filter rows where id was "a". Other relational and logical operators are listed below.

You can modify data.frames using the mutate() function. It works like this:

example.df

## modify example.df and assign the modified data.frame the name example.df
example.df <- mutate(example.df,
       var2 = var1/t, # replace the values in var2
       var3 = 1:length(t), # create a new column named var3
       var4 = factor(letters[t]),
       t = NULL # delete the column named t
       )

## examine our changes
example.df

> example.df
   id t       var1 var2
1   a 1 0.19158254    c
2   a 2 0.46921828    c
3   a 3 0.71092483    c
4   a 4 0.81892913    c
5   b 1 0.17894751    b
6   b 2 0.94742350    c
7   b 3 0.24051714    c
8   b 4 0.41236477    c
9   c 1 0.66573374    c
10  c 2 0.72515137    b
11  c 3 0.66923824    b
12  c 4 0.41101666    b
13  d 1 0.06401198    c
14  d 2 0.34580213    b
15  d 3 0.44477036    c
16  d 4 0.12253790    b
> 
> ## modify example.df and assign the modified data.frame the name example.df
> example.df <- mutate(example.df,
+        var2 = var1/t, # replace the values in var2
+        var3 = 1:length(t), # create a new column named var3
+        var4 = factor(letters[t]),
+        t = NULL # delete the column named t
+        )
> 
> ## examine our changes
> example.df
   id       var1       var2 var3 var4
1   a 0.19158254 0.19158254    1    a
2   a 0.46921828 0.23460914    2    b
3   a 0.71092483 0.23697494    3    c
4   a 0.81892913 0.20473228    4    d
5   b 0.17894751 0.17894751    5    a
6   b 0.94742350 0.47371175    6    b
7   b 0.24051714 0.08017238    7    c
8   b 0.41236477 0.10309119    8    d
9   c 0.66573374 0.66573374    9    a
10  c 0.72515137 0.36257569   10    b
11  c 0.66923824 0.22307941   11    c
12  c 0.41101666 0.10275416   12    d
13  d 0.06401198 0.06401198   13    a
14  d 0.34580213 0.17290107   14    b
15  d 0.44477036 0.14825679   15    c
16  d 0.12253790 0.03063448   16    d
>

Now that we have made some changes to our data set, we might want to save those changes to a file.

# write data to a .csv file
write_csv(example.df, path = "example.csv")

# write data to an R file
write_rds(example.df, path = "example.rds")

# write data to a Stata file
library(haven)
write_dta(example.df, path = "example.dta")

In addition to importing individual datasets, R can save and load entire workspaces

ls() # list objects in our workspace
save.image(file="myWorkspace.RData") # save workspace 
rm(list=ls()) # remove all objects from our workspace 
ls() # list stored objects to make sure they are deleted

> ls() # list objects in our workspace
 [1] "a"                "baby.names"       "comet"           
 [4] "comet.plot"       "example.df"       "filter"          
 [7] "fit"              "nwbuilding"       "orig.search.path"
[10] "x"                "X"                "y"               
[13] "Y"                "z"                "Z"               
> save.image(file="myWorkspace.RData") # save workspace 
> rm(list=ls()) # remove all objects from our workspace 
> ls() # list stored objects to make sure they are deleted
character(0)
>

Load the "myWorkspace.RData" file and check that it is restored

load("myWorkspace.RData") # load myWorkspace.RData
ls() # list objects

> load("myWorkspace.RData") # load myWorkspace.RData
> ls() # list objects
 [1] "a"                "baby.names"       "comet"           
 [4] "comet.plot"       "example.df"       "filter"          
 [7] "fit"              "nwbuilding"       "orig.search.path"
[10] "x"                "X"                "y"               
[13] "Y"                "z"                "Z"               
>

Read in the "babyNames.csv" file if you have not already done so, assigning the result to baby.names.

1. Filter baby.names to show only names given to at least 3 percent of boys.
2. Create a column named "Proportion" equal to Percent divided by 100.
3. Filter baby.names to include only names given to at least 3 percent of Girls. Save this to a Stata data set named "popularGirlNames.dta")

The summarize function can be used to calculate statistics by grouping variable. Here is how it works.

summarize(group_by(example.df, id), mean(var1), sd(var1))

> summarize(group_by(example.df, id), mean(var1), sd(var1))
# A tibble: 4 × 3
      id `mean(var1)` `sd(var1)`
  <fctr>        <dbl>      <dbl>
1      a    0.5476637  0.2787990
2      b    0.4448132  0.3493286
3      c    0.6177850  0.1405077
4      d    0.2442806  0.1805739
>

You can group by multiple variables:

summarize(group_by(example.df, id, var3), mean(var1), sd(var1))

> summarize(group_by(example.df, id, var3), mean(var1), sd(var1))
Source: local data frame [16 x 4]
Groups: id [?]

       id  var3 `mean(var1)` `sd(var1)`
   <fctr> <int>        <dbl>      <dbl>
1       a     1   0.19158254         NA
2       a     2   0.46921828         NA
3       a     3   0.71092483         NA
4       a     4   0.81892913         NA
5       b     5   0.17894751         NA
6       b     6   0.94742350         NA
7       b     7   0.24051714         NA
8       b     8   0.41236477         NA
9       c     9   0.66573374         NA
10      c    10   0.72515137         NA
11      c    11   0.66923824         NA
12      c    12   0.41101666         NA
13      d    13   0.06401198         NA
14      d    14   0.34580213         NA
15      d    15   0.44477036         NA
16      d    16   0.12253790         NA
>

#+END_SRC

Earlier we learned how to write a data set to a file. But what if we want to write something that isn't in a nice rectangular format, like the output of summary? For that we can use the sink() function:

sink(file="output.txt", split=TRUE) # start logging
print("This is the summary of example.df \n")
print(summary(example.df))
sink() ## sink with no arguments turns logging off

> sink(file="output.txt", split=TRUE) # start logging
> print("This is the summary of example.df \n")
[1] "This is the summary of example.df \n"
> print(summary(example.df))
 id         var1              var2              var3       var4 
 a:4   Min.   :0.06401   Min.   :0.03063   Min.   : 1.00   a:4  
 b:4   1st Qu.:0.22828   1st Qu.:0.10301   1st Qu.: 4.75   b:4  
 c:4   Median :0.42857   Median :0.18527   Median : 8.50   c:4  
 d:4   Mean   :0.46364   Mean   :0.21711   Mean   : 8.50   d:4  
       3rd Qu.:0.67966   3rd Qu.:0.23520   3rd Qu.:12.25        
       Max.   :0.94742   Max.   :0.66573   Max.   :16.00        
> sink() ## sink with no arguments turns logging off
>

1. Calculate the total number of children born.
2. Filter the data to extract only Massachusetts (Location "MA"), and calculate the total number of children born in Massachusetts.
3. Group and summarize the data to calculate the number of children born each year.
4. Calculate the average number of characters in baby names (using the "Name.length" column).
5. Group and summarize to calculate the average number of characters in baby names for each location.

Thanks to classes and methods, you can plot() many kinds of objects:

plot(example.df$var4)

Thanks to classes and methods, you can plot() many kinds of objects:

plot(select(example.df, id, var1))

Thanks to classes and methods, you can plot() many kinds of objects:

plot(select(example.df, id, var4))

• Please take a moment to fill out a very short feedback form
• These workshops exist for you – tell us what you need!
• http://tinyurl.com/R-intro-feedback

• IQSS workshops: http://projects.iq.harvard.edu/rtc/filter_by/workshops
• IQSS statistical consulting: http://dss.iq.harvard.edu
• Software (all free!):
  • R and R package download: http://cran.r-project.org
  • Rstudio download: http://rstudio.org
  • ESS (emacs R package): http://ess.r-project.org/
• Online tutorials
  • http://www.codeschool.com/courses/try-r
  • http://www.datacamp.org
  • http://swirlstats.com/
  • http://r4ds.had.co.nz/
• Getting help:
  • Documentation and tutorials: http://cran.r-project.org/other-docs.html
  • Recommended R packages by topic: http://cran.r-project.org/web/views/
  • Mailing list: https://stat.ethz.ch/mailman/listinfo/r-help
  • StackOverflow: http://stackoverflow.com/questions/tagged/r