Local AI vs Cloud AI: Decisions Behind FitWardrobe

FitWardrobe classifies clothing from user photos. There were two viable architectures: a cloud endpoint with a larger model, or an on-device model that runs locally. Neither is a moral position. Both had real trade-offs I had to weigh.

The default answer for anyone who has read enough tech Twitter is 'on-device, obviously.' The honest answer is 'it depends on what breaks first when the model is wrong.' I forced myself to write down what each version would look like on a bad day, not a good one.

I found the worst-case latency issue the night before a demo. The app would sometimes just stop responding on a weak connection. The median numbers looked fine. The tail didn't. That sealed the decision more than any benchmark did.

I ran benchmarks. The cloud model was more accurate - measurably, not marginally. The on-device model was faster - not by a small amount, by a factor that changed whether the interaction felt instant or laggy. The cost difference mattered less than I expected.

Then I asked the questions the benchmark couldn't answer. What happens on a subway with no signal? What does a user assume we're doing with their bedroom photos? How do we roll back a bad model version? What's our story when accuracy drops on darker fabrics - a known failure mode for most vision models?

On-device won on three axes and lost on one.

• Latency: fast enough to feel instant. The cloud version sometimes felt laggy. For a swipe-heavy interface, that gap matters.
• Privacy: 'your closet never leaves your phone' is a real claim I could put in the App Store description without a lawyer. That mattered more than I expected in early feedback.
• Cost: on-device has no per-call cost at scale. That mattered less than the other two, but it made the solo-builder economics trivial.
• Accuracy: cloud was better. I accepted the gap and added a "not right? tap to fix" button so every mistake became a feedback loop, not a dead end.

On-device shipped. The thing I gave up - accuracy - became a training signal through the correction button. The axis I lost on was the one that compounded.

The 'tap to correct' affordance turned model mistakes into a labelled dataset I use to retrain the classifier periodically. The very axis I gave up became a source of future improvement.

That was the accidental lesson: the way you handle the model being wrong is often more product-defining than the accuracy number itself.

The accuracy trade-off was the uncomfortable one. My engineering instinct hated shipping the worse number. What convinced me was writing the bad-day scenarios side by side. A cloud outage on Monday morning is a worse product than a slightly-less-accurate local model on Monday morning.

If I had another AI product tomorrow, I'd run the four-axis exercise on day one, before I picked a model. Not because the axes are magic. Because the argument you have while running them is where the real product decision lives.