Introduction
The lrd package provides a set of tools for quickly and accurately processing participant from various types of recall tests (e.g., cued-recall, free-recall, and sentences), while also being able to control for minor errors in participant responses. To simplify use, we have created an R Shiny application that can be used to run this package having to download or install R. A general user guide for this application is provided below.
The application can be accessed here . Source code for both the R package and the application have been made available on GitHub. This page will walk you through using the Shiny to score cued-recall, free-recall, and sentence data. A video walkthrough covering both the package and Shiny is available here.
Overview of Application
The lrd application is set up as a series of tabs. When you first load the application, you’ll be directed to the Information Tab. From here, you can use the menu on the left to select the tab for the appropriate scoring task. For all scoring tasks, data will need to be structured in long-format (i.e., each participant observation constitutes one row in the dataset). The Arrange Data tab can be used first to convert wide-format into the appropriate format for scoring. lrd supports most file types, however only one header row is supported. If you run into problems uploading your data, try uploading it as a .csv file. Finally, all tabs provide options for downloading scored output for importing into R, SPSS, Excel, etc.
Cued-Recall Tab
Uploading a Cued-Recall Dataset
When scoring cued-recall data, the upload file needs to be arranged in long format and must contain the following columns: A unique participant identifier, participant responses, trial number for each recall trial. Additionally, cued-recall scoring requires an answer key and trial numbers for each key item. These can either be uploaded as a separate file or these columns can be part of the original upload file. Finally, the upload file can also contain other columns (e.g., those denoting experimental conditions, demographics, etc.), which can be selected using the “group by” option.
Scoring Cued-Recall Data
After uploading your dataset and answer key, you will need to choose the cutoff value used for scoring. Next, you will need to select the cutoff percentage used for scoring. The scoring cutoff ranges from 0 (most strict, must be spelled exactly correct according to the answer key) to 5 (least strict and many things will match, so choose this carefully). This number is the Levensthein Distance between the participant response and key item and represents the total number of insertions, deletions, and substitutions one might need to convert the given response to the answer key. Therefore, a 1 represents one letter difference, while 5 represents 5 changes. Finally, checking the box below the scoring cutoff will tell lrd to flag participant outliers (defined as any recall proportions that are 3 standard deviations above or below the mean).
Once all of the appropriate settings have been selected, clicking “Score Your Data” will begin the scoring process. Scored output will then be displayed in below in the “Scored Output” panel. This panel allows you to preview your dataset to ensure that all columns imported correctly. Next, the “Summarized Output” panel displays correct recall proportions for at the participant level. Outliers will be flagged if the option for this was selected during set up. If a grouping variable was selected, recall proportions at the group level will also be displayed. Finally, the bottom panel displays recall proportions plotted as a function of the grouping variable. If no grouping variable is selected, this panel will display a histogram of recall proportions.
Free-Recall Tab
Uploading a Free-Recall Dataset
As with cued-recall scoring, free-recall data will also need to be in long-format. Data upload follows the same general procedure. For simplicity, we suggest uploading the response data and answer key as two separate files. The response data will need to contain at minimum the participant responses and a unique participant identifier. The answer key file should contain only one column (consisting of the key items). Any grouping variables should be attached to the file containing the participant responses.
Scoring Free-Recall Data
Free recall data is scored using the same procedure as cued-recall. To begin, you will need to select the scoring cutoff. You also have the option to flag participant outliers. If your upload data contains a column denoting the order in which items were recalled, this column can be selected using the “position answered” box to be used for creating serial position curves. Note that this column is automatically generated if you use the Arrange Data tab to convert your data from wide to long format.
Scored output is then previewed in the “Scored Output” panel. Additional panels show item and group level proportions of correct recall and plots showing recall proportions as a function of grouping variable (if selected) or a histogram of participant level responses. Finally, serial position curves, probability of first response plots, and conditional probability curves are displayed in the final three panels.
Scoring Free-Recall Data from Multiple Lists
The Free-Recall tab above can only be used to score free-recall datasets where all items are come from the same study list. For scenarios where more than one study is used, the Multiple Free-Recall tab should be used to score responses. Like single list free-recall scoring, the Multiple Free-Recall tabs requires that the uploaded data be arranged in long format. Further, both the answer key and dataset will need to contain a column denoting unique list identifiers. Data upload follows the same general process as used in the standard free-recall tab, and the Scoring Set Up box contains the same inputs for selecting the appropriate columns for each argument (e.g., participant responses, participant identifiers, group options, etc.) as well as two additional inputs for selecting the scoring key and dataset list identifiers (answer key trial ID column and participant trial ID column). This tab returns all outputs from the Free-Recall tab, including scoring summaries, response frequencies, serial position curves, probability of first response plots, and conditional probability curves
Sentence-Recall Tab
Uploading a Sentence-Recall Dataset
For scoring sentence recall, data upload closely follows the process used when scoring cued-recall data. The data will need to be arranged in long format such that each row corresponds to one participant response. Moreover, the response column should be structured such that each cell contains the full participant response (i.e., each cell contains a full sentence.) Example data illustrating the format for the upload data is available here. Finally, this data will need to contain all of the required columns from the cued-recall tab. The answer key can either be included as a column with the upload data or it can be uploaded as a separate file.
Scoring Sentence-Recall Data
When scoring sentence data, in addition to selecting the cutoff criteria, you will need to select the delimiter for sentence tokens (i.e., the character(s) separating each word within the sentences). This can be selected using the box below the scoring cutoff selection. Note that by default, this argument defaults to a single space. If a different character separates words, you will need to delete this space before typing a different character.
Scored output is then previewed in the “Scored Output” panel. In addition to showing whether or not the sentence was correctly recalled, this function will also return any tokens omitted from the answer key and any extra words included in the response. Finally, the “Summarized Output” panel displays participant level and group level (if specified) recall proportions. Plots can be viewed using the graph panel at the bottom.
