Why completion rates tell you almost nothing

Completion rates are easy to collect, simple to explain and dangerously easy to overvalue. A member can reach the end of a lesson, guess every answer and leave no better prepared than when they arrived. Another can learn something important, stop before the final screen and be counted as a failure.

For credit unions, community banks and financial institutions, the question is not whether people finished. It is whether the programme helped them understand a decision, build realistic confidence and choose an appropriate next step.

Why completion became the default

Completion is attractive because it is unambiguous. It fits neatly into a campaign report and makes different programmes appear comparable. It also asks very little of the platform: record a final event, divide it by starts and display a percentage.

The problem is that completion combines very different experiences. It does not distinguish careful learning from rapid tapping. It does not reveal which ideas caused difficulty. It does not show whether confidence is well founded. And it says nothing about what the member was ready to do afterwards.

Keep completion as an operational measure. Use it to find broken journeys or unexpected drop-off. Do not present it to a board as proof that financial education worked.

1. Knowledge gain

Knowledge gain measures the change between what a member understands before and after a learning experience. This does not require a long exam. A small number of well-designed questions can establish a baseline and test the decision the lesson is meant to support.

The questions should assess application, not memory. Asking someone to identify the lower total borrowing cost in a short scenario is more useful than asking them to repeat a definition. Report the change at programme level and by topic so the average result does not hide a weak concept.

2. Confidence calibration

Confidence matters because people act on what they believe they can do. But confidence on its own can be misleading. A member can feel certain and still be wrong, which is why confidence should always be read alongside knowledge.

The useful signal is whether confidence and understanding move into better alignment. When both rise, the member is more likely to be genuinely ready. When confidence rises but accuracy remains flat, the content may be producing reassurance without capability. When knowledge improves but confidence stays low, the learner may need practice, examples or human support.

3. Topic-level friction

An overall score hides where people struggle. Track incorrect answers, repeated attempts, requests for explanation and exits by topic. Together, these create a practical heat map of member need.

That heat map can inform much more than the lesson library. If many members misunderstand a fee, a fraud signal or an affordability concept, the finding can improve member communications, front-line training and service design. Financial education becomes a listening channel, not only a publishing channel.

4. Consented readiness

A learning journey can reveal that a member is ready to explore a next step, but that does not make every completion a lead. The signal should be specific, relevant and controlled by the member.

Useful readiness events include choosing to open a calculator, saving a checklist, asking for more information or booking a conversation after an appropriate lesson. Record the consent attached to that action and keep it visible when the signal reaches a CRM or another team.

This creates better context for staff without compromising the educational relationship. It also produces a more meaningful measure than a generic click-through rate because the action follows demonstrated learning and an explicit choice.

5. Supported outcomes

The final measure asks what happened after the learning journey. Did the member return and continue? Did a staff conversation take place? Did the member complete a relevant task, access support or make a better-informed choice? The right outcome depends on the programme and should be agreed before launch.

Avoid claiming causation that the data cannot prove. A lesson may contribute to an outcome without being the only reason it happened. Report the connection honestly: what the member learned, what they chose next and what the institution observed afterwards.

How to read the measures together

No single metric tells the whole story. A strong reporting view connects participation, learning quality and responsible follow-up.

• High completion and low knowledge gain usually indicates content that is easy to finish but ineffective.

• High knowledge gain and low confidence suggests the learner understands more but still needs practice or reassurance.

• Repeated friction on one topic points to a content, communication or service-design problem worth investigating.

• High readiness with weak consent records is a governance failure, not a commercial success.

• Strong learning results with no supported next step may mean the programme is helping but the wider member journey is disconnected.

What belongs in the board pack

A useful board report is short enough to read and specific enough to challenge. Lead with reach and knowledge gain. Show confidence calibration, the highest-friction topics and the member-selected next steps. Make consent visible wherever a signal moves beyond the learning environment.

Then add one brief paragraph on what the team learned and what it will change in the next reporting period. This closes the loop. The board is not simply looking backwards at activity; it can see how evidence is improving the programme.

• Are members more capable after the programme than before it?

• Which topics or decisions require more support?

• Are next-step signals relevant, proportionate and consented?

• Which member groups are not being reached?

• What changed because of the evidence?

Start with the outcome, then design the lesson

Before building content, write down the decision the member should be better able to make and the evidence that would demonstrate progress. That simple discipline improves the lesson, the data model and the board report at the same time.

Completion can tell you whether the journey functioned. Knowledge gain, confidence, friction, consented readiness and supported outcomes tell you whether the journey mattered.