Over the past few months, we’ve been running a beta test on our new feature—the Dream Point Booster. Teachers often tell us that guessing behaviour is a major frustration with digital education tools: what good is practice, even with the best programs, if students are just clicking through without really engaging the content? The Dream Point Booster was designed to address that challenge, to boost student accuracy while discouraging and demotivating guessing, all without taking the fun out of the game. We’re really excited to share some early results from the beta.
What is the Dream Point Booster?
The Dream Point Booster is a meter that is tied to a students (recent) question accuracy. At it’s most basic, the more accurate students are when they answer questions, the higher the arrow will climb and when they get questions wrong, the arrow will dip.
Depending on what zone the arrow is in (indicated by the four colours), each question will have a multiplier attached to it. These range from 1x in the red, to 5x in the orange, 10x in the green and 15x in the blue. The higher your multiplier, the faster buildings and upgrades will complete, reveries will be trained, and new resources can be harvested.
In the classroom, this shows up as more purposeful and intentional practice. Students who keep their average accuracy “high” (over 50%) are rewarded with faster progression. On the other hand, those who begin to guess experience a slow-down in game play that encourages them to slow down and refocus—making the practice time teachers allocate more meaningful.
Now, we took careful steps to make sure that it is possible (and without herculean effort) for students to be able to climb back up, should they fall down to the orange or red. Every zone on the booster represents a range of accuracy and takes multiple correct (or incorrect) questions to move out of it. That said, we also made sure that they need to keep up the effort once they get into the blue and green in order to stay there. That means it’s easier to climb up the farther down you are, and you’ll drop faster the higher you are.
Accuracy and Guessing
Okay, let’s get into some data. First up, the graph below shows the average accuracy of the players in the beta test before and after the Dream Point Booster was turned on.
Let’s start by focusing on the four bars on the left, above the Average DS Accuracy label. For the beta test, the Booster was only enabled in Dreamscape, so for the analysis, I focused on player’s accuracy only on questions they answered while playing Dreamscape.
The first two yellow bars represent the overall accuracy of everyone in the beta test before and after, while the two green next to it are just those students who have answered at least 100 questions after the Booster was enabled. As you can see, the overall accuracy increased pretty dramatically for both groups. Overall, it increased by about 6%, and for those with 100+ questions, it increased almost 8%! The 100+ group went from accuracy near 50% (which is the minimum of where we want students) to nearing 60%.
Note: A really cool thing (in my opinion) about the Booster is that it’s a very easy visual check for both you and your students on their accuracy. As stated in our blog post on fidelity, we recommend 25 questions a week and staying above 50% accuracy. 50% accuracy is indicated by the start of the green zone on the Booster. So as long as students are staying in the green or blue, they’re on track.
On the right side of the graph, these four blocks look at the percent of questions that we flag as being likely guessed. Likely guessed questions are questions that are answered extremely fast in comparison to what we normally expect. There are different thresholds for the independent/stand alone questions and the passage questions.
We were really pleased to see a very noticeable drop in the percent of questions that players likely guessed. The overall group had a lower starting average than those with the 100+ questions, but after the Booster was turned on, both groups are now much closer together.
It’s clear from this data, that the Booster is having the effect we wanted on student accuracy and guessing. But how do the educators and students feel about it?
Educator and Student Feedback
We’ve asked all the educators and students who have been participating in the beta for their feedback and have received a lot of it. Let’s see what they think:
First up: Their overall impression. We asked students to rate us on a scale from Hate it to Love it. And they love it. We also had many comments from them expressing the same.
Next up, we asked students to answer a few true false questions. The first two here are kind of interesting as a pair. First up, students overwhelming answered that the introduction of the Booster made them try harder (which is fantastic). But, they were pretty evenly split on whether it affected how they played the game. This one puzzled me for a bit, and I’m still not entirely sure of the difference. It may be that they separate the questions from the game and considering them two distinct parts.
The third set on the graph were focused on a concern we had. We were/are moving cautiously with it’s implementation because we didn’t want the focus on accuracy (even though it doesn’t say that word) to make students worried or anxious when answering. We worried that they may feel afraid to answer if they didn’t know in case the arrow would drop. But, as you can see on the bar, most students didn’t feel this at all. On the flip side, when we asked the educators the same thing, they were very split, as you can see on the second answer on the graph below.
We also asked the educators a few other questions. Like the students, they agreed that the introduction of the Booster seemed to be making their students try harder on the questions. And most of them could tell a difference in how their students played (even if their students couldn’t tell). And, lastly, all of them thought the Booster is a good addition to the game.
What’s next?
We still have a few more tweaks before we’ll be ready to release the Dream Point Booster more widely, but it will be coming to everyone soon. While it’s first iteration is focused on Dreamscape, questions answered over in Drift do affect the position of the arrow. That means it will also be eventually arriving in Drift.
In the meantime, if you’re interested in testing out the Dream Point Booster in your classroom(s), you can fill out this form. Keep an eye on your email afterwards, as I’ll reach out with some additional questions.