Scoping a Computer Vision Quality Inspection Pilot
A computer vision inspection pilot succeeds when defect definitions, image capture conditions and labelled samples are agreed before model work begins. Fix the camera position, lighting and line speed, and collect good and defective samples from real production across shifts. Decide early how the cost of a false reject compares with a missed defect, and design the operator workflow for every case the system flags.
What should a pilot cover and what should it leave out?
Limit the pilot to one product family, one inspection station and a small number of defect types. A narrow scope makes the labelling work manageable and shows quickly whether the approach fits the production process. Products with many variants, frequent design changes or unstable material supply usually belong in a later phase.
Write the scope as a short statement: which product, which station, which defect classes, and what happens to the output. This statement is the reference for every later decision about accuracy targets, hardware and integration.
How do you define defects clearly enough to label them?
A defect definition must be precise enough that two trained inspectors would classify the same part the same way. For each defect class, write a rule in plain language, show reference photos of acceptable and unacceptable parts, and list the borderline cases. Where the rule depends on a measurement, state the measurement method and tolerance.
Involve the quality team and experienced operators in this step. Disagreements found here are cheaper to resolve on paper than after hundreds of images have been labelled. Keep a change log for definitions, so that labels made under an earlier rule can be identified and corrected.
What image capture conditions must be fixed?
Image quality depends on conditions that the factory controls more than the software does. Fix the camera position, distance, focus, lens, lighting, background and the part’s orientation. Changes to any of these can make a model that worked in the test look unreliable on the line.
Capture sample images under the same conditions the production system will use, including each shift, the lighting at different times of day and the variation in incoming material. Record the capture settings with each image so that you can trace problems back to a setting or a station.
How many labelled samples do you need and how should they be collected?
The right number of samples depends on the number of defect classes, their natural frequency and the variation in the product. Rare defects need deliberate effort to collect, because a random sample may contain very few of them. Plan the collection so that good parts and each defect class are represented across the conditions described above.
Keep a separate test set that is not used to train the model. Label it independently where possible, and check the labels on a sample to confirm the agreement between labellers. Measure the results on the test set and on new production images after the pilot starts.
How should you weigh false rejects against missed defects?
Every inspection system makes two kinds of error. A false reject sends a good part for rework or scrap, which costs time and material. A missed defect lets a faulty part continue, which can lead to customer complaints or safety risks. The acceptable balance between these two depends on the product and the consequence of each error.
- Agree which error is more costly for each defect class before setting any threshold.
- Measure both error types on the test set and on live production, not only overall accuracy.
- Decide who can change the threshold and how each change is recorded.
- Plan a manual check of a sample of accepted parts, so that missed defects are found over time.
How should operators work with the system?
Design the workflow before the pilot starts. Define what the operator sees when the system flags a part, how they confirm or override the result, and where an override is recorded. Operators should be able to explain why a part was flagged, otherwise they will stop trusting the system.
Create a feedback loop so that operator decisions can be reviewed and fed back into the labelled set. Escalate unclear cases to a quality supervisor, and keep a log of the cases the system could not classify. These logs are often the most useful input for improving the next version.
Frequently asked questions
Can a pilot use the cameras we already have?
Sometimes. Check resolution, mounting position, lighting and frame rate against the defects you need to detect, using sample images before you decide.
Does the system replace the inspectors?
A pilot should be designed to support inspectors, with people reviewing flagged parts and overrides. Any change to that role should be a separate decision, based on measured results.
What should we decide before the pilot ends?
Agree the success measures at the start, such as the error rates on each side, the review workload and the cases that need escalation. Use those measures to decide whether to extend, adjust or stop the pilot.
