Manufacturing

Quality inspection.
Lighting first.

Visual inspection is one of the genuinely mature applications of machine learning in industry. It also fails for reasons that have almost nothing to do with the model: inconsistent illumination, parts that move, and a defect class with eleven examples.

THE PHYSICAL SETUP IS THE PROJECT

A vision system that works in the pilot cell and fails on the line has usually not encountered a harder problem — it has encountered a different one. Ambient light changing through the day. A part presented at a slightly different angle. A camera slowly drifting out of focus from vibration. Each one shifts the input distribution away from what the model was trained on, and the model degrades without announcing it.

Controlled illumination, consistent fixturing and a fixed camera position remove most of that variance for a fraction of what is spent on model development. Most experienced practitioners in this field will tell you the same thing, and it remains the most commonly skipped step because it is a mechanical engineering job in the middle of a software project.

The second reality is class imbalance. A well-run line produces very few defects, which is the point — and it means the interesting class is rare. Anomaly detection trained on normal parts is frequently more practical than classification trained on defect types, and augmenting a handful of real defect images does not substitute for having seen the failure mode.

WHAT WE BUILD AROUND THE MODEL

Drift detection on the input

Monitoring the images themselves, not just the predictions. Brightness, focus and part position tracked against the training distribution, so a dirty lens or a moved fixture is caught as a data alert rather than as a mysterious accuracy decline weeks later.

A defect library that grows

Every confirmed defect captured, labelled and added to the evaluation set. Over a year this becomes the most valuable asset in the system, and it only exists if capturing it is part of the operator workflow.

Tunable operating point

False rejects cost yield; false accepts cost a customer complaint or a recall. The threshold between them is a business decision that changes by product and by customer, and it must be adjustable by quality staff without a redeploy.

Operator review that closes the loop

Borderline cases routed to a person whose decision feeds back as a label. This is what makes the system improve rather than plateau, and it is a workflow design problem more than a technical one.

MEASURED PROPERLY

Overall accuracy is close to meaningless on an imbalanced line — a model that passes everything scores well and catches nothing. What matters is recall on each defect class separately, false reject rate at the chosen operating point, and the cost-weighted combination of the two using your actual scrap and complaint costs.

Rare defect classes need reporting individually and honestly, including the confidence interval. A class with eleven training examples has a recall estimate with an interval so wide it barely constrains anything, and presenting it as a point number is the kind of overstatement that ends badly at the first escape.

FROM CAPTURE TO REVIEW

INSPECTION PATH

Controlled capture

fixed light, fixture, focus
Input drift check

against the training distribution
Inference

on the line, no internet
Operating point

set by quality, not by us
Accept or reject

cost-weighted, per product
Operator review

borderline cases become labels

Solid: the inspection path Dashed: the review loop that grows the defect library

Monitoring the images, not only the predictions, is what catches a dirty lens before it becomes a mysterious accuracy decline.

QUESTIONS

QUESTIONS — 3

More than you have, almost always. Hundreds per class is comfortable; dozens is workable with anomaly-detection framing; a handful means you are validating a concept rather than deploying a classifier. We would rather set that expectation up front than discover it at acceptance testing.

Generally yes. Inspection models are small and run on modest edge hardware at frame rates well beyond most line speeds. Where it gets tight is very high-speed lines or very high-resolution imaging, and that is a sizing question we answer before committing.

No, and it should not. The model runs on the line, on the plant network. Training and retraining can happen elsewhere and be deployed as an artefact, which is the same controlled-media pattern used in air-gapped environments.

Keep reading

Evaluating AI Systems

→

30-minute technical call · no deck