August 16, 2026

The defect is never in the course. It's in the package.

Three published runs, three authoring tools, five years apart — and the same failure each time. A pass mark somebody set deliberately, silently overridden by an export setting nobody reviews.

There is a step in every eLearning build that nobody reviews.

Not the storyboard. Not the script, the visual design, the assessment items, or the accessibility pass. Those all get looked at, sometimes by several people. The step nobody reviews is the one right at the end, where the finished course is exported into a package and handed to the LMS.

We have now published four runs against courses we did not author. In three of them, the most consequential defect lived in that step and nowhere else.

What that looks like in practice

In one course, an author wrote five scenario-based quiz questions with explanatory feedback on both the correct and incorrect branches — genuinely good work, better than most commercial training. They set a pass mark of 80%.

The export was configured to report completion and incompletion only. No quiz was registered for reporting. A learner who scored zero and a learner who never opened the quiz produce an identical record in the LMS.

In another, built in a different tool five years earlier, the author set a pass mark of 8 out of 10 on the graded question. Both the completion criterion and the success criterion were then set to view, at 75% of slides. The pass mark exists. Nothing consults it.

In a third, there was no scoring of any kind. Fifteen knowledge checks, every correct answer declared in client-side JavaScript, no page in the course reporting a score or a status, and a completion certificate issued for reaching the last page.

Why it survives review

Because in the authoring tool, everything works.

The quiz runs. The feedback appears. The pass mark sits in a settings panel looking authoritative. A reviewer clicks through the preview, sees a functioning course, and approves it — correctly, on the evidence available to them.

The defect only exists in the exported artifact. That artifact is the one thing nobody opens and the only thing your learners ever receive.

The part that should worry you

None of this shows up in your completion data. It cannot: the data is produced by the thing that is broken.

A course that reports completion for viewing content produces a 94% completion rate. So does a course that reports completion for passing a real assessment. The rate does not distinguish them, and nothing in the LMS does either.

If the course is mandatory, the record you are keeping — and reporting upward, and retaining for years — is a record of what the package decided to observe. In three of our four published runs, that was attendance.

What to do about it

You do not need a vendor to find out. Ask whoever administers your LMS six questions about your mandatory courses:

  1. Does the course report a score, or only a status?
  2. What condition triggers completion — passing, viewing a percentage, or reaching the last page?
  3. Is there a pass mark, and is anything evaluating it?
  4. Can a learner advance without attempting the assessment?
  5. Where are the correct answers stored?
  6. Who last verified any of the above?

If they can answer all six, you have nothing to worry about and it cost you one conversation. If they cannot, you have found something worth knowing before somebody else asks.

The runs referenced here are published in full, with the evidence and a ranked remediation plan for each. Publishers are blinded — we publish findings, not organizations.

← All writing