Until 2020, measuring a convolutional neural network was never well understood. That is until researchers set out to answer an important question:

Is there a principled method to scale up ConvNets, so they achieve better accuracy and efficiency?

And in the process, they accomplished two feats which changed the direction of deep learning:

1. Discovered a novel scaling method called compound scaling.

2. Created a new family of SOTA architectures called EfficientNet.

Now, back to the original question: how do we measure the size of a ConvNet?

By looking at three factors:

1. Resolution (dimensions of its inputs)

2. Width (number of feature maps)

3. Depth (number of layers in the network)

All three factors — depth, width, and resolution — impact the accuracy and efficiency of your network. Ideally, you want to maximize all of these factors and accomplish the following:

• Keep the baseline model architecture, i.e., keep the operations in each layer fixed.

• Leave the memory footprint of your model constrained to some target hardware.

• Keep the number of FLOPs below some predefined threshold.

But there's a catch…

Scaling only one network dimension (width, depth, or resolution) improves accuracy, but accuracy rapidly decreases. For better accuracy and efficiency, you must balance all dimensions during ConvNet scaling.

This brings us to the first novelty: compound scaling. This method uniformly scales a baseline network's width, depth, and resolution in a principled way. But none of this matters unless we have a good baseline network to begin with…

The second novelty: the EfficientNetB0 architecture.

This baseline architecture was found using neural architecture search (NAS). In a nutshell, NAS has three core building blocks that can be categorized regarding search space, search strategy/algorithm, and evaluation strategy. A set of operations that define search space that specifies the structure of a network. The search strategy determines how the NAS algorithm experiments with different neural networks.

Your evaluation strategy will determine how you'll evaluate the performance of the resulting network. The researchers found the EfficientNetB0 architecture by optimizing for both accuracy and flops. The researchers used compound scaling from these baseline architectures to build a family of 8 models: EfficientNetB0 to EfficientNetB7.

This family of models shattered previous performance on ImageNet and crowned the EfficientNet family as the most powerful CNN architecture! In this post, you'll see the EfficientNetB0 architecture using SuperGradients. You'll use it to classify cars into one of the top 50 car brands in the USA.

This notebook will show you how to use the EfficientNetB0 architecture to classify cars into one of the top 50 car brands in the U.S.

Dive right into the code here.