Motivation Is the Product: What I'd Measure If I Were Building School Software

Every few years a new wave of education software promises to fix school, and every few years the results come back roughly the same. Justin Reich spent a decade studying those waves at MIT and summarized them in a book whose title gives away the ending: Failure to Disrupt. The technologies were not bad. They were aimed at the wrong constraint.

The constraint is not access to instruction. Instruction has been free and abundant since roughly 2010. The constraint is whether a fourteen-year-old will sit down and do the hard part when nobody is making them. Alpha School, one of the few places that has tried to build a whole school around software, puts it more bluntly than any researcher would: ed-tech is ten percent of the solution, and motivated students are the other ninety. I think that is roughly right, and I think most people who build learning products read it as a slogan when it is actually a specification.

What “engagement” measures

Open the analytics dashboard of almost any learning product and you will find the same metrics: daily active users, session length, streaks, lessons started, minutes on task. These are attention metrics. They were invented by companies whose business is attention, and they measure exactly what those companies care about: did the person come back, and how long did they stay.

None of them tell you whether anyone learned anything. When a Carnegie Mellon team studied a psychology MOOC, they estimated that the learning benefit of extra practice activities was more than six times that of extra video watching or page reading. Watching is what the dashboard counts. Doing is what worked. Worse, attention metrics reward the wrong behavior. A student who spends forty minutes on a lesson they already understand scores beautifully on time-on-task. A student who clears a mastery check in six minutes and goes outside looks like a churn risk. The metric punishes the outcome you want.

The massive open online course era ran this experiment at scale. Reich and Ruipérez-Valiente looked at 565 runs of Harvard and MIT courses on the edX platform and found completion rates that fell from about five percent in 2013-14 to about three percent in 2017-18 (The MOOC pivot). The content was excellent. The engagement dashboards were full of green. Almost nobody finished.

What the scarce input actually is

Ryan and Deci’s work on self-determination theory is the most-cited account of why people do hard things voluntarily, and it comes down to three needs: autonomy, competence, and relatedness (Ryan & Deci, 2000). Kids do hard work when they chose it, when they can feel themselves getting better at it, and when someone they respect notices. A 2021 meta-analysis of 344 samples and more than 223,000 students found the same thing from the other direction: motivation driven by rewards or the threat of punishment was unrelated to performance or persistence, while motivation rooted in personal value was the type most strongly tied to persistence.

Read that list against a typical school day. Autonomy: none, the schedule is fixed. Competence: undetectable, because everyone moves on together whether they got it or not. Relatedness: one teacher, thirty students, and a grading pile. It is not surprising that motivation is scarce. The system is designed to consume it.

Now read the same list against what a piece of software could do. Autonomy is a product decision: what does the student get to choose? Competence is a measurement decision: can the student see, in real time, that they are getting better? Relatedness is a workflow decision: what does the adult in the room see, and when?

That is the whole job. Translate three psychological needs into mechanics.

The three numbers I would put on the wall

If I were building the software for a school, I would throw out the attention dashboard and replace it with three numbers.

Mastery, checked later. Not “lesson complete.” A unit counts when the student clears it at ninety percent or better, and it stays counted only if they can still clear it two weeks later without warning. The retest is the whole point. It is the difference between learning and cramming, and it is the one thing a classroom of thirty can never afford to do.

Time returned. Alpha calls this Timeback, and it is the best single idea in the model. If a student finishes the day’s academics in ninety minutes, the software should say so, and the school should let them leave the room. The freed time is the reward. It is also the most honest measure of whether the software works: a product that is genuinely more efficient gives time back, and a product that is merely engaging takes it.

Completed loops. A loop is something the student did that touched the real world and came back with a response: a sale, a published post, a shipped feature, a demo in front of strangers. Count the loops. Count how long each one took. A student who has closed twelve loops in a semester has learned something no test can measure, and the software can measure the count.

Everything else (streaks, badges, minutes, logins) is either an input to these three or a distraction from them.

What the screen shows at eight in the morning

Metrics are abstract until you decide what the first screen looks like. Here is mine.

At eight in the morning the student sees three things. The first is what they have not mastered yet, ordered by what is blocking the most downstream work, with an honest estimate of how long it will take. The second is the time they will have back if they finish. The third is the loop they are in the middle of: the customer they promised to call, the post that is due, the prototype they said they would demo on Friday.

Nothing on that screen is a feed. Nothing autoplays. Nothing is designed to keep them there. The whole screen is designed to get them off it as fast as possible, having done something hard.

The guide sees a different screen. Not a gradebook. A short list of the students whose mastery checks slipped on the retest, whose returned time has been shrinking, and who have not closed a loop in two weeks. Those are the three signals that a kid is about to disengage, and they show up in the data days before they show up in the room. The guide’s job is motivation and character, and the software’s job is to tell them where to spend it.

The objection, and why it is right

The obvious objection is that this is a lot of measurement, and measurement is exactly what people hate about school. Grades, rankings, standardized tests, the whole apparatus of sorting.

The objection is right about the apparatus and wrong about measurement. A grade is a measurement whose purpose is to sort students and move everyone on together. A mastery check is a measurement whose purpose is to tell one student precisely what they have not learned yet, and to hold them there until they have. Same instrument, opposite intent. The first produces a number that goes on a transcript. The second produces a next step.

I wrote about this once under a title that was harsher than it needed to be: grades measure compliance, not mastery. I still believe that. The fix is not less measurement. It is measurement that serves the student instead of the institution.

The part that is not software

I want to be honest about the limit of this argument. You can build the three numbers, the eight-a.m. screen, and the guide’s queue, and a kid who has no reason to care will still not care. Software can remove the friction that kills motivation. It cannot supply the motivation.

That comes from the loops. A student who has to call a real customer tomorrow morning finds a reason to finish the math today. The afternoon gives the morning its point. That is why the two halves of the model cannot be separated, and why the most important product decision is not on the learning side at all. It is this: what real thing does this student get to do with the time we give back?

Build that, measure those three numbers, and get out of the way.