Author: Nicholas Maxwell, Erin Buchanan
Date: 2021-12-06

# Sentence Recall Example

#### Nicholas Maxwell, Erin Buchanan

#### 2021-12-06

# Libraries and Data

Please see manuscript for a long description of the following data. We will load the example data, and you can use the `?` with the dataset name to learn more about the data.

    library(lrd)
    data("sentence_data")
    head(sentence_data)
    #>   Sub.ID Trial.ID                                      Sentence
    #> 1      1        1                           This is a sentence.
    #> 2      1        2                           Woo more sentences!
    #> 3      1        3     This is thing that the participant typed.
    #> 4      1        4             This is another example sentence.
    #> 5      1        5 Okay this is the final thing that they typed.
    #> 6      2        1                           This is a sentence.
    #>                                        Response Condition
    #> 1                            This is a sentence         a
    #> 2                           Woo more sentences!         a
    #> 3 This is the thing that the participant typed.         a
    #> 4             This is another example sentence.         a
    #> 5   Okay this is the final thing they typed.            a
    #> 6                             This is sentence.         b
    #?sentence_data

# Data Cleanup

Scoring in `lrd` is case sensitive, so we will use `tolower()` to lower case all correct answers and participant answers.

    sentence_data$Sentence <- tolower(sentence_data$Sentence)
    sentence_data$Response <- tolower(sentence_data$Response)

# Score the Data

You should define the following:

- data = dataframe of participant responses
- responses = column name of the participant answers
- key = column name of the answer key
- key.trial = column name of the trial id code
- id = column name of the participant id number
- id.trial = column name of the trial id within the participant data
- cutoff = the Levenshtein distance value you want to use for scoring (0 no changes exactly the same, higher numbers allow more variance in the word)
- flag = calculate z scores for outliers (TRUE/FALSE)
- group.by = column name(s) for grouping variables
- token.split = a value to split the tokens (i.e., words) in the sentence

Note that the answer key can be in a separate dataframe, use something like `answer_key$answer` for the key argument and `answer_key$id_num` for the trial number. Fill in `answer_key` with your dataframe name and the column name for those columns after the `$`.

    sentence_ouptut <-
      prop_correct_sentence(data = sentence_data,
                            responses = "Response",
                            key = "Sentence",
                            key.trial = "Trial.ID",
                            id = "Sub.ID",
                            id.trial = "Trial.ID",
                            cutoff = 1,
                            flag = TRUE,
                            group.by = "Condition",
                            token.split = " ")

    str(sentence_ouptut)
    #> List of 3
    #>  $ DF_Scored     :'data.frame':  30 obs. of  11 variables:
    #>   ..$ Trial.ID        : int [1:30] 1 1 1 1 1 1 2 2 2 2 ...
    #>   ..$ Sub.ID          : int [1:30] 1 2 3 4 5 6 2 3 4 1 ...
    #>   ..$ Sentence        : chr [1:30] "this is a sentence." "this is a sentence." "this is a sentence." "this is a sentence." ...
    #>   ..$ Responses       : chr [1:30] "this is a sentence" "this is sentence" "this is a sentence" "this is a sentence" ...
    #>   ..$ Condition       : chr [1:30] "a" "b" "a" "b" ...
    #>   ..$ Answer          : chr [1:30] "this is a sentence" "this is a sentence" "this is a sentence" "this is a sentence" ...
    #>   ..$ Proportion.Match: num [1:30] 1 0.75 1 1 1 ...
    #>   ..$ Shared.Items    : chr [1:30] "this is a sentence" "this is sentence" "this is a sentence" "this is a sentence" ...
    #>   ..$ Corrected.Items : chr [1:30] NA NA NA NA ...
    #>   ..$ Omitted.Items   : chr [1:30] NA "a" NA NA ...
    #>   ..$ Extra.Items     : chr [1:30] NA NA NA NA ...
    #>  $ DF_Participant:'data.frame':  6 obs. of  5 variables:
    #>   ..$ Condition          : chr [1:6] "a" "b" "a" "b" ...
    #>   ..$ Sub.ID             : int [1:6] 1 2 3 4 5 6
    #>   ..$ Proportion.Correct : num [1:6] 0.978 0.802 0.832 0.646 0.599 ...
    #>   ..$ Z.Score.Group      : num [1:6] 0.916 1.155 0.151 -0.577 -1.067 ...
    #>   ..$ Z.Score.Participant: num [1:6, 1] 1.564 0.355 0.559 -0.72 -1.039 ...
    #>   .. ..- attr(*, "scaled:center")= num 0.75
    #>   .. ..- attr(*, "scaled:scale")= num 0.145
    #>  $ DF_Group      :'data.frame':  2 obs. of  4 variables:
    #>   ..$ Condition: chr [1:2] "a" "b"
    #>   ..$ Mean     : num [1:2] 0.803 0.698
    #>   ..$ SD       : num [1:2] 0.1908 0.0903
    #>   ..$ N        : int [1:2] 3 3

# Output

We can use `DF_Scored` to see the original dataframe with our new scoring columns - also to check if our answer key and participant answers matched up correctly! First, each sentence is stripped of punctuation and extra white space within the function. The total number of tokens, as split by `token.split` are tallied for calculating `Proportion.Match.` Then, the tokens are matched using the Levenshtein distance indicated in `cutoff`, as with the cued and free recall functions. The key difference in this function is how each type of token is handled. The `Shared.Items` column includes all the items that were matched completely with the original answer (i.e., a `cutoff` of 0). The tokens not matched in the participant answer are then compared to the tokens not matched from the answer key to create the `Corrected.Items` column. This column indicates answers that were misspelled but within the cutoff score and were matched to the answer key (i.e., “th” for the, “ths” for this). The non-matched items are then separated into `Omitted.Items` (i.e., items in the answer key not found in the participant answer), and `Extra.Items` (i.e., items found in the participant answer that were not found in the answer key). The `Proportion.Match` is calculated by summing the number of tokens matched in `Shared.Items` and `Corrected.Items` and dividing by the total number of tokens in the answer key. The `DF_Participant` can be used to view a participant level summary of the data. Last, if a grouping variable is used, we can use `DF_Group` to see that output.

    #Overall
    sentence_ouptut$DF_Scored
    #>    Trial.ID Sub.ID                                      Sentence
    #> 1         1      1                           this is a sentence.
    #> 2         1      2                           this is a sentence.
    #> 3         1      3                           this is a sentence.
    #> 4         1      4                           this is a sentence.
    #> 5         1      5                           this is a sentence.
    #> 6         1      6                           this is a sentence.
    #> 7         2      2                           woo more sentences!
    #> 8         2      3                           woo more sentences!
    #> 9         2      4                           woo more sentences!
    #> 10        2      1                           woo more sentences!
    #> 11        2      6                           woo more sentences!
    #> 12        2      5                           woo more sentences!
    #> 13        3      2     this is thing that the participant typed.
    #> 14        3      3     this is thing that the participant typed.
    #> 15        3      1     this is thing that the participant typed.
    #> 16        3      6     this is thing that the participant typed.
    #> 17        3      4     this is thing that the participant typed.
    #> 18        3      5     this is thing that the participant typed.
    #> 19        4      2             this is another example sentence.
    #> 20        4      1             this is another example sentence.
    #> 21        4      6             this is another example sentence.
    #> 22        4      3             this is another example sentence.
    #> 23        4      4             this is another example sentence.
    #> 24        4      5             this is another example sentence.
    #> 25        5      1 okay this is the final thing that they typed.
    #> 26        5      2 okay this is the final thing that they typed.
    #> 27        5      3 okay this is the final thing that they typed.
    #> 28        5      4 okay this is the final thing that they typed.
    #> 29        5      5 okay this is the final thing that they typed.
    #> 30        5      6 okay this is the final thing that they typed.
    #>                                       Responses Condition
    #> 1                            this is a sentence         a
    #> 2                              this is sentence         b
    #> 3                            this is a sentence         a
    #> 4                            this is a sentence         b
    #> 5                      this thing is a sentence         a
    #> 6                            this is a sentence         b
    #> 7                            woo more sentences         b
    #> 8                            woo more sentecnes         a
    #> 9                                more sentences         b
    #> 10                           woo more sentences         a
    #> 11                               more sentences         b
    #> 12                        woohoo more sentences         a
    #> 13                            this is the thing         b
    #> 14             the participant typed this thing         a
    #> 15 this is the thing that the participant typed         a
    #> 16                         this thing was typed         b
    #> 17                         this thing was typed         b
    #> 18                      this sentence was typed         a
    #> 19            this is an extra example sentence         b
    #> 20             this is another example sentence         a
    #> 21                       this is anothr example         b
    #> 22               tis is another xample sentence         a
    #> 23                       this is anothr example         b
    #> 24                          this is another one         a
    #> 25      okay this is the final thing they typed         a
    #> 26      okay this is the final thing they typed         b
    #> 27       okay this is the last thing they typed         a
    #> 28                      ok this is the last one         b
    #> 29                         this is th final one         a
    #> 30                      ok this is the last one         b
    #>                                          Answer Proportion.Match
    #> 1                            this is a sentence        1.0000000
    #> 2                            this is a sentence        0.7500000
    #> 3                            this is a sentence        1.0000000
    #> 4                            this is a sentence        1.0000000
    #> 5                            this is a sentence        1.0000000
    #> 6                            this is a sentence        1.0000000
    #> 7                            woo more sentences        1.0000000
    #> 8                            woo more sentences        0.6666667
    #> 9                            woo more sentences        0.6666667
    #> 10                           woo more sentences        1.0000000
    #> 11                           woo more sentences        0.6666667
    #> 12                           woo more sentences        0.6666667
    #> 13     this is thing that the participant typed        0.5714286
    #> 14     this is thing that the participant typed        0.7142857
    #> 15     this is thing that the participant typed        1.0000000
    #> 16     this is thing that the participant typed        0.4285714
    #> 17     this is thing that the participant typed        0.4285714
    #> 18     this is thing that the participant typed        0.2857143
    #> 19             this is another example sentence        0.8000000
    #> 20             this is another example sentence        1.0000000
    #> 21             this is another example sentence        0.8000000
    #> 22             this is another example sentence        1.0000000
    #> 23             this is another example sentence        0.8000000
    #> 24             this is another example sentence        0.6000000
    #> 25 okay this is the final thing that they typed        0.8888889
    #> 26 okay this is the final thing that they typed        0.8888889
    #> 27 okay this is the final thing that they typed        0.7777778
    #> 28 okay this is the final thing that they typed        0.3333333
    #> 29 okay this is the final thing that they typed        0.4444444
    #> 30 okay this is the final thing that they typed        0.3333333
    #>                                Shared.Items Corrected.Items
    #> 1                        this is a sentence            <NA>
    #> 2                          this is sentence            <NA>
    #> 3                        this is a sentence            <NA>
    #> 4                        this is a sentence            <NA>
    #> 5                        this is a sentence            <NA>
    #> 6                        this is a sentence            <NA>
    #> 7                        woo more sentences            <NA>
    #> 8                                  woo more            <NA>
    #> 9                            more sentences            <NA>
    #> 10                       woo more sentences            <NA>
    #> 11                           more sentences            <NA>
    #> 12                           more sentences            <NA>
    #> 13                        this is thing the            <NA>
    #> 14         this thing the participant typed            <NA>
    #> 15 this is thing that the participant typed            <NA>
    #> 16                         this thing typed            <NA>
    #> 17                         this thing typed            <NA>
    #> 18                               this typed            <NA>
    #> 19                 this is example sentence            <NA>
    #> 20         this is another example sentence            <NA>
    #> 21                          this is example          anothr
    #> 22                      is another sentence      tis xample
    #> 23                          this is example          anothr
    #> 24                          this is another            <NA>
    #> 25  okay this is the final thing they typed            <NA>
    #> 26  okay this is the final thing they typed            <NA>
    #> 27        okay this is the thing they typed            <NA>
    #> 28                              this is the            <NA>
    #> 29                            this is final              th
    #> 30                              this is the            <NA>
    #>                       Omitted.Items  Extra.Items
    #> 1                              <NA>         <NA>
    #> 2                                 a         <NA>
    #> 3                              <NA>         <NA>
    #> 4                              <NA>         <NA>
    #> 5                              <NA>        thing
    #> 6                              <NA>         <NA>
    #> 7                              <NA>         <NA>
    #> 8                         sentences    sentecnes
    #> 9                               woo         <NA>
    #> 10                             <NA>         <NA>
    #> 11                              woo         <NA>
    #> 12                              woo       woohoo
    #> 13           that participant typed         <NA>
    #> 14                          is that         <NA>
    #> 15                             <NA>         <NA>
    #> 16          is that the participant          was
    #> 17          is that the participant          was
    #> 18    is thing that the participant sentence was
    #> 19                          another     an extra
    #> 20                             <NA>         <NA>
    #> 21                         sentence         <NA>
    #> 22                             <NA>         <NA>
    #> 23                         sentence         <NA>
    #> 24                 example sentence          one
    #> 25                             that         <NA>
    #> 26                             that         <NA>
    #> 27                       final that         last
    #> 28 okay final thing that they typed  ok last one
    #> 29            okay thing that typed          one
    #> 30 okay final thing that they typed  ok last one

    #Participant
    sentence_ouptut$DF_Participant
    #>   Condition Sub.ID Proportion.Correct Z.Score.Group Z.Score.Participant
    #> 1         a      1          0.9777778     0.9160221           1.5637499
    #> 2         b      2          0.8020635     1.1547005           0.3553233
    #> 3         a      3          0.8317460     0.1508220           0.5594568
    #> 4         b      4          0.6457143    -0.5773503          -0.7199254
    #> 5         a      5          0.5993651    -1.0668441          -1.0386793
    #> 6         b      6          0.6457143    -0.5773503          -0.7199254

    #Groups
    sentence_ouptut$DF_Group
    #>   Condition      Mean         SD N
    #> 1         a 0.8029630 0.19084127 3
    #> 2         b 0.6978307 0.09026826 3