Computer Vision from Prototype to Production

A computer vision demo can look convincing after a few days. A dependable vision product may require months of work across data quality, evaluation, deployment, monitoring, and user experience.

The gap exists because a demonstration answers one question: can the model recognise examples similar to those already collected? Production introduces a harder question: can the complete system remain useful when images, environments, users, and operational priorities change?

Arthur Sedek approaches computer vision as a full product lifecycle rather than an isolated training exercise.

Define the operational target

A model objective should connect directly to a decision. Detection, segmentation, classification, pose estimation, and tracking produce different forms of information. The correct task is the one that provides enough detail for the downstream workflow.

For example, classification may confirm that an object is present, while segmentation may be required to measure its area or boundary. Choosing a more detailed task increases annotation and computation costs, so the additional information should create clear value.

Define acceptable errors with users and domain experts. Missing a critical object may be more costly than creating a false alert. Evaluation and thresholds should reflect that asymmetry.

Treat the dataset as a product

The dataset is a maintained asset with versions, owners, quality standards, and documented scope. Images should represent expected environments, devices, lighting, viewpoints, object scales, and difficult edge cases.

Annotation guidance must be precise. Ambiguous boundaries, inconsistent class definitions, and missing objects create a ceiling on model quality. A review process should measure agreement between annotators and identify classes that need clearer definitions.

Splits should prevent leakage. Frames from the same video, images from the same site, or near-duplicate captures should not be divided randomly across training and testing sets. Grouped splits provide a more realistic estimate of generalisation.

Select metrics that reflect use

A single aggregate score can hide serious weaknesses. Evaluation should be broken down by class, object size, environment, capture device, and other relevant conditions.

Useful measures may include precision and recall, intersection over union, average precision, calibration, tracking stability, inference latency, and memory usage. Product metrics can include review time, avoided manual effort, or the rate of correct downstream actions.

Error analysis is more valuable than leaderboard optimisation. Review false positives, false negatives, boundary errors, and low-confidence cases in context. Look for recurring patterns that suggest better data, revised labels, or a different task formulation.

Design the inference pathway

The deployment target shapes model development. Cloud inference can provide flexible scaling and centralised management. Edge inference can reduce latency, bandwidth use, and exposure of sensitive imagery. Hybrid designs can perform immediate processing locally and send selected outputs to central services.

The inference pipeline should define preprocessing, model execution, postprocessing, thresholding, and output schemas explicitly. Export formats such as ONNX can improve portability, but converted models must be tested for numerical and task-level consistency.

Performance testing should use the intended hardware and realistic input streams. Average latency alone is insufficient. Measure tail latency, throughput under load, startup time, and resource use.

Monitor what the model cannot see

Production monitoring is difficult because ground truth is often delayed or absent. Teams can still track useful signals:

• changes in image characteristics and input distributions;
• confidence and prediction distributions;
• class frequency and object-size patterns;
• rates of human correction or rejection;
• processing failures and latency;
• disagreement between model versions.

These signals do not prove that accuracy has changed, but they identify where investigation and new labelling should be prioritised.

Build an active learning loop carefully

Active learning can focus annotation effort on uncertain, novel, or operationally important examples. Selection should seek useful coverage, not only low confidence. Repeatedly sampling one difficult pattern may create a narrow dataset while missing other emerging conditions.

Every new training round should preserve dataset lineage and use a stable evaluation set. Otherwise, apparent improvement may come from changing the test rather than improving the model.

Make correction part of the interface

Users need an efficient way to inspect and correct predictions. Their feedback can improve immediate outcomes and create valuable training data, but only if it is stored with context and quality controls.

Interfaces should distinguish model suggestions from confirmed results. Confidence should be communicated in a way that supports action without implying certainty. For high-impact decisions, review requirements should be explicit.

Production is a continuous process

Computer vision quality is not fixed at deployment. Cameras change, environments evolve, new object types appear, and operational priorities shift.

A durable system has repeatable paths for collecting evidence, diagnosing failures, updating data, training models, validating releases, and rolling back safely. The model is one versioned component inside that process.

This lifecycle perspective turns computer vision from an impressive prototype into a dependable capability.