Practice Experience Redesign
Boosting practice experience through gamification
Overview
When you learn something new, how do you make sure you really solidify the acquired knowledge?
​
If you said 'putting in practice' then you get why this case study is so relevant. In this project, we wanted to make learning on Oppia more hands-on. We noticed users weren't really getting into the practice activities, so we looked into how to make them more fun. Our solution? Adding some gamification to spice things up and make learning a blast!
Role
In this project I was the Principal Product Designer, defining product requirements, user stories, research plans, wireframes, low to high fidelity prototypes and handoff.
I've worked collaboratively with various stakeholders over a period of 4 months in 2023.
Background
Oppia is an educational platform that offers learning adventures through story-based lessons. It's mainly designed for kids aged 7 to 14, especially those from underserved communities. These lessons interactive stories where characters tackle real-life problems, while introducing basic math concepts.
Additionally, Oppia has a feature where learners can practice the skills they have learned in the lessons, through quick quizzes. It's about giving them an opportunity to solidify the content, while we assess their mastery of their skills.
Problem statement
Firstly, we did a detailed heuristic analysis of the existing practice feature. Points of friction in the user journey were identified, such as the lack of integration between lessons, practice, and review, which resulted ihn significant challenges regarding user adoption of the practice functionality:
Problem
The separation between lessons, practice, and review in different tabs caused confusion and lack of engagement.
Associated Heuristic
Flexibility and efficiency of use / System status visibility / User control and freedom
The users didn't understand when and how to use the practical feature, and often they didn't even know it existed.
Error prevention / System status visibility / User control and freedom
Problem
The practice feature didn't take full advantage of its gamification potential, and had an aesthetically unappealing design, making it less interesting for users.
Additionally, it wasn't clear to students that by doing these practice sessions, their learning was being measured, and that they should continue practicing continuously until reaching 100% mastery of the subject.
Associated Heuristic
Consistency and standards /
Aesthetic and Minimalist Design
System status visibility/
User control and freedom
Goals and Tasks
So, the goal of this project was to redefine the entire practice feature, making it more accessible and engaging for users, and increasing user engagement and retention. To accomplish this, our proposed tasks were:
Incorporating gamification elements to increase user interest and retention.
Developing a robust and understandable domain measurement logic that aligns with gamification strategies.
​Additionally, there are related tasks on our radar, which will be covered in a separate case study, such as:
​
-
Streamlining the discovery of practice features for skills learned in lessons and integrating practice seamlessly into the learning process (e.g., suggesting practice immediately after lessons).
-
Implementing progress visualization on the learner dashboard.
Design Process
In order to design a solution that truly motivated students in the practical experience, we needed to understand what actually motivates these children. What are the gamification elements that truly deliver value to them? We conducted qualitative research with 6 children aged between 9 and 14 years old to gain a better understanding.
​
Among other insights, we discovered that what motivates users the most in the engagement journey of games/apps are the possibilities of reaching levels and accumulating points/earning rewards. Based on this, more than 50 user stories were defined for this project by this designer, which shaped the solution.
Ok, but how does this work?
To explain the proposed solution, let's take a step back and clarify how the learning logic works in the practice session. Essentially, when a student completes Lesson 1, they will begin a practice session on Skill 1. Each practice session consists of 10 questions, each related to a concept from Skill 1, as follows:
The student has the option to:
​
-
Attempt answering the question as many times as needed.
-
If they make a mistake on the first try, they can try again
-
They can view a hint that will help them answer the questions
-
If unable to answer, view the solution or
-
Skip to the next question.
​
However, as a requirement of the feature, to properly measure if the student has learned all the concepts of Skill 1, they must answer the questions on these concepts on the first attempt, without looking at a hint.
Solution 1
For this solution, we proposed that students earn points for the questions they answer correctly. However, we wouldn't want them to not earn points for answering the question correctly after viewing a hint or on subsequent attempts. After all, even if they didn't get it right on the first try, they should still be rewarded for answering correctly at some point, right?
​
So for example, an easy question is worth 8 points, medium 10, and difficult 12. However, if the student gets the answer wrong on their first attempt, the question becomes worth 6, 8, and 10 points, respectively. Additionally, if the student looks at a hint, they lose more points. Check-out the mid-fidelity prototype:
When a student answers correctly on their second try, they lose 1 point. So instead of getting 10 points for a Medium question, they get 9 points.
At the end of the session, the student can view the total questions, the total of questions answered correctly, and their total of points.
Next, they can see how much their mastery has progressed, and view their level in that skill progressing from beginner to intermediate.
The idea behind this solution was to motivate students to answer the questions on the first attempt, without indiscriminately using hints, or just trying repeatedly, testing the answers. It was also a way to encourage them to progress in mastering the skill.
​
Furthermore, at the end of the session, they would see a progress bar, indicating whether they were beginners, intermediates, or advanced students. The idea was for them to understand that by answering a question without using a hint and on the first attempt, it would elevate their mastery towards becoming intermediate and eventually advanced students in that skill.
Usability Test and Encountered Issues
Moderated virtual tests were conducted with users aged 9-14. Many user stories were validated, but a few issues were encountered, such as:
The solution didn't effectively convey to users that their mastery increases only when they answer correctly on the first try.
​
Users didn't comprehend the term "mastery," only grasping the concept of progress.
​
Users felt demotivated by losing points for incorrect answers or when utilizing a hint.
​
There was a lack of understanding regarding the meanings of "beginner," "intermediate," and "advanced"
Solution 2 - How did I addressed these issues?
The second proposed solution was based on rewarding users with something extra - in this case, a star - instead of deducting points when they make mistakes or use hints. This resolves the issue of discouraging them by losing points and, on the contrary, motivates them further because they earn a star, something beyond just points.
When a student answers correctly on their first attempt, for the first time, they see a tooltip. This ensures that they have contextual onboarding for the feature.
When the user finishes a practice session, they see the total of points and stars earned for that session. The stars will float to the progress bar, which will increase according to the number of stars.
Furthermore, at the end of the solution, the student can clearly understand that the stars are what drive their progression through the levels, since we show visually stars floating so the progress bar rise.
So now they know that even if they use hints or make mistakes on the first attempt, they'll still earn points, but to level up, they need to earn stars. And of course! I renamed levels to 1, 2, and 3 which was clear for them to understand.
​
In conclusion, which for them means:
"The more questions I answer without using a hint and on the first attempt -> the more stars I earn -> the higher my level increases"
For us, means:
The more questions the student answers without using a hint on the first attempt -> the more it signifies they've learned the content -> until they reach what we consider 100% mastery for that skill.
So why keep the points, if they don't count for their mastery?
Wee keep the points because we still want to reward them even when they didn't answer correctly on their first try. The only way they don't earn points is if they view the solution or skip the question. Additionally, the points will be used for other things to be designed, such as:
-
Unlocking bonus lessons teaching more advanced material
-
Unlocking additional short stories about the characters in the lesson
-
Turning on profile/avatar/lesson-player customization options (e.g. background)
-
Badges on user's profile page
Final results
100%
completion rate
93%
conversion rate
4.7 of 5
satisfaction rate
1.3 of 5
difficulty level
Final Considerations and Limitations
-
The study was conducted with a limited number of users, and further testing with a broader audience may be necessary to confirm the results.
-
The long-term effectiveness of the solution still needs to be evaluated.
-
The impact of gamification on student learning and knowledge retention is a topic that requires additional research.
​
Next steps:
​
-
Implementing the new solution on the Oppia platform
-
Monitoring its impact on student engagement and learning, such as % of returning learners engaging in practice sessions
-
Exploring other gamification methods to make the learning experience even more engaging and effective.
-
Conducting research to evaluate the impact of gamification on long-term knowledge retention
What have I learned from this project?
A fundamental lesson that this project has taught me is that, while there is often a logical sequence behind certain user stories, such as our example of skill or concept mastery, it is not always essential to explicitly expose this process or logic to the user, as it can become unnecessarely complex.
​
Instead, we should adapt our tools to fit the users' context, allowing them to achieve the same end result in a more organic and intuitive manner. This approach promotes a truly user-centered experience, taking into account not only their immediate needs but also their specific contexts, prior knowledge, and motivations.
​
Thanks for reading!
​
