New method makes neural network training faster by smartly choosing which images to label

Training deep neural networks (DNNs) often requires labeling new data—a laborious and expensive task. Selecting which inputs matter most before labeling them can save significant time and resources, but existing diversity-based selection methods tend to be slow and difficult to scale.

The Challenge with Current Methods#

Most diversity-based input selection approaches aim to identify the most informative subset of data for labeling. However, these techniques are computationally demanding and struggle with large datasets, making them impractical for real-world applications where speed matters.

A More Efficient Alternative#

Researchers have developed Concept-Based Diversity (CBD), a new efficiency-focused metric that harnesses Vision-Language Models (VLMs) to measure image diversity. The method works by extracting semantic concepts from images and using those representations to identify how different the inputs are from one another. Remarkably, CBD produces results that correlate strongly with Geometric Diversity (GD), a well-established diversity metric, while requiring only a fraction of the computation time.

Combining Diversity with Uncertainty#

The team created a hybrid selection strategy by pairing CBD with Margin, a straightforward uncertainty measurement. This combination leverages both diversity (which inputs are different from each other) and uncertainty (which predictions the model is least confident about).

Performance and Scalability#

Testing across multiple DNN architectures, datasets, budgets, and leading baseline methods showed consistent gains: the CBD-based approach outperformed all competing methods while maintaining speed comparable to simple uncertainty-only techniques. Critically, even on large-scale datasets like ImageNet, the method remains fast enough to be practical in scenarios requiring frequent or repeated selection rounds. This scalability advantage sets it apart from earlier hybrid approaches that slowed down significantly with bigger datasets.