Lightweight CNNs Ranked: EfficientNetV2-S Wins Accuracy, EfficientNet-B0 Balances Performance and Efficiency

Comparing lightweight convolutional neural networks has historically been challenging because researchers typically evaluate models using different training procedures, input formats, and pretrained weights. This inconsistency makes it difficult to fairly assess which architectures truly perform best. A new study on arXiv tackles this problem by establishing a standardized benchmark across seven leading CNN designs.

Unified Testing Framework#

The researchers evaluated their architectures—including EfficientNet and MobileNet variants—on three image classification benchmarks: CIFAR-10, CIFAR-100, and Tiny ImageNet. Crucially, all models were trained and tested under a single, consistent protocol. The evaluation recorded not just accuracy metrics but also practical efficiency measures: parameter counts, storage footprint in FP32 format, and multiply-accumulate operations (MACs).

Performance Leaders#

EfficientNetV2-S emerged as the most accurate model, achieving top-1 accuracy of 97.57% on CIFAR-10, 86.98% on CIFAR-100, and 78.73% on Tiny ImageNet. Yet raw accuracy alone doesn't tell the full story in resource-constrained settings.

EfficientNet-B0 proved more practical for many deployments. It stayed within 0.85 percentage points of EfficientNetV2-S's performance while using only about 21% as many parameters and requiring roughly 14% of the MACs on Tiny ImageNet. This favorable balance—strong accuracy with substantially lower computational cost—makes it a compelling reference point for practitioners weighing performance against efficiency demands.

Extreme Efficiency Option#

For ultra-low-resource environments, MobileNetV3-Small offers an additional trade-off. This model consumes approximately 40% of EfficientNet-B0's parameters and 15% of its MACs while maintaining competitive accuracy, making it suitable for deployment on severely constrained hardware.

The Pretrained Weight Advantage#

The benchmark also examined whether starting with ImageNet-pretrained weights, rather than random initialization, provides meaningful gains. The pretrained advantage varied significantly by dataset: it was largest on CIFAR-100 and Tiny ImageNet but substantially smaller on CIFAR-10. This finding suggests that for smaller, simpler datasets, pretrained initialization matters less than practitioners might assume.