Why use RTutor for interactive tutorials if there is RStudio's learnr?

There are nowadays different options to create interactive R tutorials that allow users to type in code, automatically check their solution and get hints if they are stuck. Two options are RStudio’s learnr package and my RTutor package.

Obviously, RStudio has a great track record for creating and continously supporting awesome software and R packages. So unless there are compelling reasons, it probably makes sense to go with RStudio’s learnr. Below are three potential reasons to consider RTutor. The may be relevant or not for your application.

1. Larger Multi-Chunk Exercises

I created and use RTutor mainly for longer interactive problem sets in university courses and to replicate research articles in an interactive fashion. The key idea is that different code chunks are bundled into exercises and will be solved sequentially.

By default the results of earlier chunks will be availble in later chunks in the same exercise. For example, if some data set is loaded in the first chunk of an exercise, it can be used in all later chunks.

In contrast, in learnr each code chunk is independent and one has not access to the results of earlier chunks. Here is an interesting discussion on whether such access should be possible or not. JJ Allaire also states compelling counter-arguments:

The biggest downside to allowing environment sharing is that the exercises now cannot in principle be run on an “exercise farm”. The package doesn’t currently take advantage of this but it would be nice to swap this in underneath all the existing tutorials later.

A secondary downside is that users must now execute the chunks in order (obviously in an instructor-led classroom setting this isn’t such a big problem).

So it is not clear whether in the future, learnr will allow sharing of the results of previous chunks. The mentioned secondary downside that the chunks must be solved in order, can be alleviated in two ways in RTutor. First, one can set the chunk option optional=TRUE which means that chunk does not need to be solved and the results will not be available in future chunks. Second, it is by default not possible to access chunks from previous exercises and consequently one can start with a new exercise without having completely solved the previous exercise.

So I guess if you mainly construct longer exercises where chunks build upon each other, RTutor might currently be the more convenient option for you than learnr.

2. Ability to solve inside RStudio

Both learnr and RStudio allow to show and solve a problems set in a Shiny app in the web browser. The designs differ, but I would say that there are no huge differences. An advantage of learnr is that you can more flexible change the default design and e.g. easily include interactive chunks into markdown based slides.

An advantage of RTutor is that you can also solve problem sets by editing an RMarkdown file inside RStudio. RTutor creates a RStudio Addin that allows you to check your solution. You can also type hint() in the console to get a hint for the chunk you are currently working on. For R beginners a Shiny app is probably the better option. Yet, for my Master courses where students solve several RTutor problem sets, I always let students solve the problem sets inside RStudio. The advantage is simply that students also work with the tools that they can use outside the problem set: RStudio and RMarkdown files.

3. Automatic Default Hints and Solution Checkers

I tried to design RTutor with the goal in mind to minimize the need to write custom code checkers and custom hints. While it is possible to customize code checkers and hints for each chunk (and each expression inside a chunk), I find that there is often no need for it.

Consider an example chunk of an RTutor solution file that asks the user to enter a specific dplyr pipe:

#< task
df = data.frame(a=rep(1:3, length=10), x=runif(10))
#>
df %>% 
  filter(a==2) %>%
  arrange(x)

This is essentially just a sample solution, where the code in the #<task ... #> block will be already shown to the user. If the user wants a hint, RTutor analyses the currently entered code and tries to adapt the hint.

For example, assume the user has entered so far no code on her own. Then the following automatically generated hint will be shown:

My solution consists of a chain of the form:

df??? %>%
  filter??? %>%
  arrange???

The ??? may stand for some function arguments wrapped in () that you must figure out.

If instead the user has entered the following code

df = data.frame(a=rep(1:3, length=10), x=runif(10))
df %>%
  filter(a==3) %>%
  arrange(x)

the following automatic hint is generated

In your following chain, I can detect wrong results already after the second element 'filter':

df %>%
   filter(a == 3) %>% !! WRONG RESULTS !!
   arrange(x) %>%

There is surely always scope for improvement, but personally I often stay with the defaults without customization. While I don’t have much experience with learnr my feeling from reading the documentation is that currently there is more need for customization of code checks and hints.

Other aspects

Distributing and Hosting Problem Sets

Both learnr and RTutor problem sets can be easily distributed to students to be solved on their own computer or be hosted in the web on shinyapps.io or in an RStudio Cloud project.

I let students solve the problem sets on their own computer in all of my courses. Of course, it is some work to get R, RStudio and the relevant packages installed, but students should learn to install relevant software anyways. Even though students solve the problem set on their own computer, they have to hand in the solution and it will be partly relevant for their total grade. For this purpose, RTutor allows students to generate a submission file. This will be uploaded on a course management system, like Moodle. Bulk downloading all submissions and a little script allows me then automatic grading.

In principle, you could also decide to host the problem sets on your own Shiny server. But then you have think about security. If you want to do that, I would recommend to use learnr rather than RTutor. Unless you know pretty well what your are doing security-wise, it would probably also be a good idea to look at RStudio’s commercial offerings. In learnr’s documentation it is stated:

You can use various features of RStudio Connect and Shiny Server Pro to run tutorials within a resource and/or filesystem sandbox.

I was daddling a bit with RAppArmor based secure evaluation for RTutor. Yet, probably the ideal environment would be something that spins separate Docker containers, as is described in this Datacamp blog entry. Possible, this can be nicely implement using ShinyProxy. But I have not yet looked into details.

Community size, Long term support

Ok, perhaps some of the reasons above may want you to try RTutor. But are not the larger community and the backing by RStudio strong arguments for learnr? I guess that is definitely the case. But how important these arguments are for you, depends on your preferences and your application.

I can only say that I plan to use continue using RTutor in my courses and we currently expand RTutor usage in our department. I also have regularly updated it with new features and plan to do so in future, but not neccessarily at a fast pace. And of course, you can always file a Github issue for a feature request. RTutor also has some Github stars, so also other people seem to use it.

In addition I don’t think that there is a big lock-in effect. Even if one uses RTutor now and possibly wants to switch to newer learnr versions in the future, it should not be a lot of work to convert your tutorials. In the end both are based on RMarkdown files.