Train and quantize the CIFER10 model on Google Colaboratory and perform inference on an edge device (RaspberryPi 3)

overview

Google Colaboratory environment settings

Mounting Google Drive and installing the library

Importing required libraries

Check the version of each module

• Keras version: 2.15.0
• TensorFlow version: 2.15.0
• scikit-learn version: 1.2.2
• matplotlib version: 3.7.1
• NumPy version: 1.25.2

Loading and preprocessing the CIFAR-10 dataset

Model selection and construction

Model training: Experiment for 2 epochs

Compile and train the model.
First, set Epoch to 2 and try learning under conditions that are not expected to produce very high accuracy.

• Learning log excerpt

Visualizing training results

Model evaluation and prediction

• Total inference time: 3.62 seconds
• Inference time per image: 0.000362 seconds
• Inference speed: 2762.24 frames per second
• Test accuracy: 51.85%
• Correct predictions: 5185/10000
• Accuracy: 0.5185
• Precision: 0.5460565026512546
• Recall: 0.5185
• F1 Score: 0.5121520088704362
• Number of parameters: 14982474

Model evaluation metrics: English version

The following formula shows how the evaluation index is calculated:

• Accuracy

• Precision

• Recall rate

• F1 Score

Model evaluation index: Japanese version

The following formula shows how the evaluation index is calculated:

• Accuracy

• Precision

• Recall rate

• F1 Score

Visualization of prediction results

The number of correct answers is 5185/10000 as shown above, but let's check it in image format.

It displays random images from the test data and compares the predictions with the correct answer.

Model training: Experiment for 20 epochs

• Learning log excerpt

Visualizing training results

Plot the accuracy and loss during the training process.

Model evaluation and prediction

The performance of the model is evaluated using test data and prediction results are output.

• Total inference time: 3.44 seconds
• Inference time per image: 0.000344 seconds
• Inference speed: 2910.39 frames per second
• Test accuracy: 79.88%
• Correct predictions: 7988/10000
• Accuracy: 0.7988
• Precision: 0.8079718674357317
• Recall: 0.7988
• F1 Score: 0.8000619454577802
• Number of parameters: 14982474

Visualization of prediction results

It displays random images from the test data and compares the predictions with the correct answer.

Comparison of Epoch2 and Epoch20

• Accuracy

The accuracy for Epoch 2 was 51.85%, which indicates that the model is not predicting randomly, but is almost half accurate.
In Epoch 20, the accuracy improved to 79.88%, indicating that the model is now able to classify more accurately.

• Precision

Precision indicates the percentage of correctly classified positive examples.
In Epoch 2, it was 0.5461, but in Epoch 20 it improved to 0.8080.
This indicates that the model is now able to predict positive examples more accurately.

• Recall

Recall indicates the proportion of correctly classified positive examples.
In Epoch 2, it was 0.5185, but in Epoch 20, it improved to 0.7988.
This indicates that the model is now able to predict many positive examples without missing any.

• F1 Score

The F1 score is the harmonic mean of Precision and Recall, and is a balanced evaluation metric.
In Epoch 2, it was 0.5122, but in Epoch 20, it improved to 0.8001.
This indicates that the model has a more balanced performance overall.

• comprehensive evaluation

Comparing Epoch2 and Epoch20, we can see that the Epoch20 model is clearly superior. As training continues, the model's accuracy, inference speed, precision, recall, and F1 score all improve.
This is likely due to the model learning more data and optimizing its parameters.

• OTHERS supplementary information

• Model training process

In the early epochs, the model is still in the process of finding the optimal weights, so accuracy and other metrics tend to be low. By increasing the number of epochs, the model will learn the features of the data better, adjust the parameters, and improve performance.

• Overfitting concerns

As the number of epochs increases, the risk of overfitting to the training data increases. Currently, the results of epoch 20 show no signs of overfitting, but evaluation on the validation dataset is important for continued training.

• Hyperparameter Tuning

Tuning hyperparameters such as learning rate and batch size can affect model performance, so finding the right hyperparameters is important.

Overall, we can see that increasing the number of epochs significantly improves the accuracy of the model, indicating that the learning process is progressing smoothly.

Saving a Keras model

This block saves a trained Keras model in HDF5 format (.h5).

Use the model_name variable to define the full path of the model file and use model.save to save the model.

Converting the model to TensorFlow Lite

This block converts a saved Keras model to the TensorFlow Lite format.

First, load the model using tf.keras.models.load_model.

Next, create a TFLiteConverter object and configure it to convert the model using quantization (converter.optimizations = [tf.lite.Optimize.DEFAULT]).

Finally, we convert the model and store the converted data in the tflite_model variable.

Loading and running inference on a TensorFlow Lite model

Load the converted TensorFlow Lite model and run inference on a single test image.

1.Create a TFLiteInterpreter object, specify the model path, and allocate memory for the tensors.
2. Next, we get the details of the input and output tensors. We prepare the input data by reshaping the test image and converting it to float32 format.
3. Set the input tensor to point to the prepared data and use interpreter.invoke to run inference.
4. Finally, take the output data and get the predicted class based on the maximum value of the output.

Performance evaluation of TensorFlow Lite models

We evaluate the performance of the TensorFlow Lite model on the entire test dataset.

1. Import the libraries required to calculate the evaluation metrics. Prepare the input data by getting the input shape from the model.
2. Create a list, predictions, to store the predictions for all test images. Start recording the time and iterate over all test images.
3. In a loop, prepare the input data for each image, set the input tensor, run inference, get the output, and add the prediction to the predictions list.
4. After iterating through all the images, record the time. Convert the predictions and test labels into 1D arrays for easier comparison.
5. Use functions in sklearn.metrics to calculate various evaluation metrics such as accuracy, precision, recall, and F1 score.
6. Finally, we calculate the total inference time, inference time per image, and FRAMES rate (FPS), and output the results along with the number of parameters in the model.

• Test accuracy: 79.78%
• Correct predictions: 7978/10000
• Accuracy: 0.7978
• Precision: 0.8069027338990966
• Recall: 0.7978
• F1 Score: 0.7990733190498165
• Number of parameters: 14982474

Visualizing the forecast

This block visualizes the test image and its predicted and true labels.

1. Create a figure with subplots using Matplotlib, defining the number of rows and columns to display the image.
2. Iterate through all test images and display them using imshow to see if the predicted classes match the true classes.
3. If there is a match, we will mark the title with a blue circle (○) to indicate that it is a correct prediction.
4. If there is a mismatch, we mark the title with a red cross (x) to indicate an incorrect prediction. We turn off axis labels for each subplot and apply a tight layout for better organization.

Model size comparison

Displays the size of the trained model and calculates the compression ratio for H5 and TFLite formats.

• Size of H5 model: 171.59 MB
• Size of TFLite model: 14.36 MB
• Compression rate: 11.95 times

Comparison of model performance and size before and after quantization

• Test accuracy

• Before quantization: 79.88%
• After quantization: 79.78%

Although there is a slight decrease in test accuracy, the performance remains roughly the same, indicating that quantization does not significantly affect the accuracy of the model.

• Correct predictions

• Before quantization: 7988/10000
• After quantization: 7978/10000

The number of correctly predicted values is also roughly the same, indicating that quantization has only a small effect on the model's performance.

• Accuracy

• Before quantization: 0.7988
• After quantization: 0.7978

Although the accuracy has decreased slightly, it is still at a high level.
This confirms that the quantized model has a level of accuracy that is practically acceptable.

• Precision

• Before quantization: 0.8079718674357317
• After quantization: 0.8069027338990966

The precision is also roughly the same, and we can see that the quantized model still maintains high accuracy.
This demonstrates that quantization maintains accuracy without increasing false positives.

• Recall (recall rate)

• Before quantization: 0.7988
• After quantization: 0.7978

The recall is also roughly the same, with quantization having only a small effect.
The model still maintains its ability to correctly detect many positive examples.

• F1 Score

• Before quantization: 0.8000619454577802
• After quantization: 0.7990733190498165

The F1 score also dropped slightly, but remains at a high level.
This shows that the quantized model exhibits well-balanced performance.

• Number of parameters

• Before quantization: 14,982,474
• After quantization: 14,982,474

no change.

• The number of parameters remains the same before and after quantization, but the format of each parameter changes due to quantization.
  This reduces memory usage and shrinks the model size.

• Summary

• Comparing the models before and after quantization, the model after quantization is significantly compressed, resulting in a significant reduction in model size.
  On the other hand, it can be seen that there is almost no change in accuracy or OTHERS evaluation indicators.
  This demonstrates that quantization is an effective method for reducing memory usage and improving storage and transfer efficiency while largely maintaining model performance.
  Quantization is especially useful for deployment and operation in resource-limited environments.

• Supplement

There are several other options for quantizing the model.

• Go to TensorFlow tf.lite.Optimize details page

About saved models

If it works properly, xxx.h5 and xxx.tflite will be saved under the Colab Notebooks directory in Gdrive.

Download this to your local PC.
Example: ./Desktop/models/vgg_16_20230714_021101.tflite

Inference operation with Raspberry Pi + USB Camera

• environment

• Device: Raspberry Pi 3 Model B+
• Camera: USB Camera (no specific resolution is specified as the inference code includes resizing processing)
• IP Address:
  Use the ifconfig command to find out the IP address of your Raspberry Pi in advance.
  Example: 169.254.112.21

• Data Transfer

Transfer the quantized model created above to the Raspberry Pi
scp -r ./Desktop/models/vgg_16_20230714_021101.tflite
pi@169.254.112.216:/home/pi/vgg16/

• inference

• requirements.txt

• Installing required modules

pip install -r requirements.txt

• Inference code (tf_inference.py)

The image from the USB camera will be displayed, and the inference results (class names) will be displayed.

Afterword

We develop such AI applications and models, introduce and evaluate SoCs and accelerators for AI, and develop applications.

If you are interested, please Inquiry.