Making Student Learning Proficiency Actionable

The Problem

Carnegie Mellon Open Learning Initiative's learning platform,Torus, estimates a student's proficiency for individual learning objectives using evidence from their course activity, including correctness, attempts, and activity completion.

But research showed that students struggled to use that information. Learning objectives were easy to miss, students received little context about what proficiency meant, and receiving a proficiency level didn't help them understand what to do next.

The Opportunity

Research showed students already evaluate their learning using signals like assessment results, practice, feedback, effort, and conversations with instructors and peers.

They also think about proficiency more broadly in terms of application, confidence, and improvement. This created an opportunity to make Torus proficiency more useful without presenting it as a complete picture of learning:

The Outcome

Open full-resolution image

I designed a responsive student experience that:

• Surfaces learning objectives in the course experience
• Explains what proficiency means in context
• Uses growth-oriented language and iconography
• Connects proficiency to relevant review and practice
• Works across desktop, mobile, light, and dark modes

In final testing, 4/4 students could correctly explain learning proficiency and identify what their proficiency level meant.

I synthesized findings across previous OLI student research, proficiency studies, learning-science literature, and design research before validating the direction through new prototype testing.

The research revealed an important distinction between how students think about learning and what Torus proficiency actually represents.

Students described proficiency in terms of applying concepts, explaining ideas, practicing, gaining confidence, and improving over time.

Torus proficiency is more specific: it's a calculated estimate based on evidence from course activity.

1. 01

   Learning Is Broader Than the Calculation

   Students use many signals to evaluate their learning—assessments, practice, feedback, effort, and confidence. Proficiency should be one useful signal, not a complete picture of learning.

2. 02

   Guidance, Not Just a Label

   Students didn’t just want a proficiency level. They wanted to know what it meant, why they received it, what to review, and what to do next.

3. 03

   Proficiency Belongs in the Learning Flow

   Students wanted learning objectives and proficiency present throughout the course itself—not tucked away in another analytics destination.

Making Proficiency Visible and Understandable

Open full-resolution image

I designed a Learning Objective Introduction that introduces students to the objectives they are about to encounter in a section.

I also embedded learning objectives and proficiency directly into lessons and added an expandable explanation where students encounter proficiency.

The experience explains what proficiency represents, how course activity contributes evidence, and why estimates become more reliable as students complete additional activities.

Designing for Growth Without Misrepresenting the Data

One of the most important design challenges was communicating proficiency without making students feel defined by a category.

Research warned that simplistic proficiency systems could discourage students or communicate fixed ability.

Through workshops with learning experts and SMEs, I explored terminology and iconography before arriving at four states:

1. 01

   Not Enough Information

   There isn't yet enough activity evidence to estimate proficiency.

2. 02

   Beginning Proficiency

   Available evidence suggests the student is starting to apply the objective.

3. 03

   Growing Proficiency

   Evidence shows the student has applied the objective and can keep strengthening consistency.

4. 04

   Strong Proficiency

   Evidence indicates the student is likely to apply the objective in different contexts.

The language describes what the available evidence suggests while reinforcing that students can continue learning. Icons provide an additional cue without relying on color alone.

Turning Proficiency Into Action

Understanding the estimate was only half the problem. Students also needed to know what to do with it.

I designed a Learning Summary that separates objectives into:

Learning Objectives You're Applying

Objectives where the available evidence indicates stronger proficiency.

Recommended Review

Objectives where additional review or practice may be useful.

Students can move directly from an objective to relevant course content, practice activities, or an explanation from the existing AI Learning Assistant.

Designing for a Flexible Learning Platform

The student experience also created an authoring challenge. Torus courses don't follow a single structure, so the system couldn't assume where an objective introduction or learning summary should appear.

Working with engineering and learning experts, I designed them as insertable course elements.

Authors choose where these moments make pedagogical sense, allowing the same student experience to work across different course structures.

This also allowed the interface to remain flexible while the underlying proficiency model continued to evolve.

I used Figma Make and Cursor to build interactive prototypes and tested the evolving experience with students.

Testing focused on whether students could:

• Find proficiency
• Explain what it represented
• Interpret their current proficiency level
• Understand what they could do next

I iterated on terminology, hierarchy, iconography, explanations, and recommendations based on those sessions.

Before

After

Final testing showed a significant improvement in comprehension:

• 4/4

  Students correctly explained learning proficiency

• 4/4

  Students identified their proficiency level and explained what that estimate meant

• In development

  Final designs moving into implementation in Torus

More importantly, the experience shifts proficiency from an isolated calculation into a useful part of the learning experience.

1. 01

   Before

   Learning Objective → Proficiency

2. 02

   After

   Learning Objective → Proficiency → Understanding → Action