Why high validation accuracy is not enough for a vision model

What did Kodanda examine?

Our work included fine-tuning pretrained vision detectors and crop classifiers, evaluating them against several recorded-data sources, and running experimental software on NVIDIA Jetson hardware. These are development studies. They are not project flight or field trials.

One classifier experiment used 9,124 training crops and 2,476 internal-validation crops. Sources were separated between these roles. Its reported internal top-1 classification accuracy was 99.27%. Top-1 accuracy asks whether the highest-scoring class label is correct for each crop.

That was a useful result on the internal task. It was not the acceptance result for the larger pipeline.

What did the additional checks reveal?

The classifier-equipped pipeline passed five of six development checks, but failed on another positive-example source. Across 47 Halmstad multirotor frames, the existing detector pipeline recorded 31 successful frames; after adding the classifier, six remained successful.

Here, a successful frame required a retained detection to meet the evaluation's localization criterion. This is a pipeline-level frame count, not crop-classification accuracy. It should not be plotted as though 99.27% and six out of 47 were the same metric on interchangeable samples.

The classifier reduced false alarms in other development checks. Both facts matter: there was useful learning, and there was a material regression. We rejected the candidate as a general replacement.

The 47-frame slice is small, and frames from a source are correlated. It does not support a population-wide failure rate. It does show that high internal classification accuracy did not guarantee preservation of the existing pipeline's behaviour on that inspected source.

Does running on embedded hardware prove reliability?

No. The same experiment had a deployment configuration that passed its measured Jetson timing gate while still failing the accuracy checks above. Timing establishes how long the measured computation took under its recorded conditions. Correctness establishes whether the outputs met the defined task.

Neither result, separately or together, establishes behaviour in flight. Hardware, data source and evaluation procedure must stay attached to the claim.

Can a rejected experiment be useful research?

Yes, when it answers a specific question with evidence and reports the contrary findings. Here the lesson is that an apparently strong component score can coexist with a pipeline regression. Publishing only the high score or only the rejection would omit useful information.

This is a bounded development observation, not a claim that all crop classifiers fail, that one architecture is universally superior, or that a particular intervention will work elsewhere. A different evaluation could produce a different outcome.

What remains unproven?

The inspected development checks do not establish independent generalization, operational effectiveness or integrated autonomous flight. Reused development data cannot become an independent test by changing its label. Protected evaluation roles remain separate.

The dataset creators supplied the underlying research data; Kodanda's contribution here is its own experimental evaluation and this analysis. This note includes no dataset imagery, model weights, training recipes or operational implementation instructions.

Evidence and availability

These are first-party Kodanda observations from the August 2026 crop-classifier experiment. This note was checked against its saved training-result manifest and development-gate result on 15 September 2026. Raw artifacts are not publicly released with this note, and the result has not been independently reproduced. This limits what a reader can verify independently.

Related public material: Kodanda Research and Technology. This note reports a learning from development work; it is not a peer-reviewed paper. Version 1.0 / Published 15 September 2026.