Object Detection on CIFAR 10 Image Dataset using Keras

Machine Learning

Data Structures

Algorithms

Problem Solving

C Programming

Python Programming

Web Dev

Home >

Machine Learning >

Object Detection on CIFAR 10 Image Dataset using Keras

Post author: Raunak

Post published: May 30, 2021

Post category: Machine Learning / Projects / Python Programming

Post comments: 0 Comments

CIFAR-10 is a dataset of images compiled by researchers at the University of Toronto, Canada. The dataset consists of 60,000 images ranging from classes like automobiles, animals, to birds. This dataset has 6000 images in each of its 10 classes. The dataset is divided into 50,000 training images and 10,000 testing images. The images are in the colour red, green and blue, measuring 32×32 pixel squares each.

You can read more about this dataset at https://www.cs.toronto.edu/~kriz/cifar.html .

This dataset can be accessed easily through keras. The common way is to use “load_data” function as follows”

tf.keras.datasets.cifar10.load_data()

On jupyter notebook, the simplest way to import the dataset is to run the following command:

from keras.datasets import cifar10

First we need to import the libraries we might intend to use.

We will discuss the importance of these libraries further as we use them. The next step is to load the dataset, perform basic resizing and normalizing the data.

x_train, y_train, x_test and y_test are a tuple of NumPy arrays. x_train and x_test are arrays of grayscale images with shapes (50000, 32, 32, 3) and (10000, 32, 32, 3) respectively. y_train and y_test are the arrays of labels 0-9 for training and testing data respectively.

Both x_train and x_test are further divided by 255 because we need 32-bit float type data and the dataset’s original data type is unsigned integers between 0-255.

In the above piece of code, we change the shapes of images because we require the images to be grayscale. Because it is a 32×32 image, the reshape command had the values 32,32 and because it is containing RGB images, the numeral 3 signifies that in the reshape command.

to_categorical is a feature that allows the conversion of integers in an array to binary values. For example, in this dataset, the labels range from 0-9. These are integers. When to_categorical is applied to this array, the output for them will be a matrix and would look something like this:

Now, we will design the main program where we select the number of filters we require, padding, activation function, etc.

In the above piece of code, we have added multiple filters with different tools. Let’s understand them one at a time.

There are three layers in a neural network – an input layer, hidden layer and an output layer. The input layer and output layer, as the name suggests, are exposed to the outer world. In our case, the images in the dataset are the input layer and the prediction will be the output. The code above is an abstraction that is not visible to the outer world.

Here, an activation function is used. An activation function is a way to introduce non-linearity in the model where the unwanted or irrelevant information is stopped and only the required information is allowed to pass on to the next layer. There are a number of different activation functions. In this code, we have used two of them, namely – ReLU and Softmax.

You can learn more about activation functions at https://helloml.org/introduction-to-activation-functions/ .

ReLU stands for Rectified Linear Unit. In this function, the result is always zero for all negative inputs. This means that for any negative input x, the output is 0.0.

The softmax activation function is basically a bunch of sigmoid functions put together. For example, the image of a truck can be classified into the following classes – an automobile and a truck, as per the classes in this dataset. Now, the softmax function will give a probability score between these possible classes. When multiple classes are involved, we use the softmax function for classification. Remember that the sum of all the probabilities will always be equal to 1.

Padding is a cushion of zeros on the border pixels of an image. When we need an output same as the input, we use padding.

Consider this example, if the number of pixels is 5 in the input image and the filter size is 3, we can follow a simple calculation: (n-f)+1. In this example, our n = 5, f = 3, therefore, output will be (5-3)+1 = 3. The output will be 3 pixels. To know what will be the padding value p, the calculation can be done as (n+2p-f)+1. In our case, if we want the output to be equal to the value of n, we will use p=1. So, (n+2p-f)+1 = (5+2*1-3)+1 = 5.

To ensure that the output image has the same dimensions as the input image, the ‘same’ padding is used.

Maxnorm is a way to ensure that the incoming weights are below a particular threshold. This is a form of regularization. It is basically a regularization technique that enforces the weight vector magnitude to not exceed a certain limit.

Dropout is also a way to avoid overfitting. We use dropout in the forward propagation to drop randomly selected neurons. This can help in achieving higher accuracy and lower losses while training the dataset.

We use flatten when we want our input to be in a one-dimensional format.

Next thing is to choose the number of epochs for our program. An Epoch is basically the number of times the entire input will be passed on to the training network. In other words, we can say that one epoch refers to one learning cycle. The higher is the epoch value, the better chances of eliminating errors but the longer time it takes. We need to take care of the fact that there might be a possibility of overfitting with large epochs. Though this won’t generally happen, one can always see how many epochs resulted in overfitting the data.

SGD was used which in each iteration picks random data points from the entire dataset, thus saving time. I used two layers each of 32 filters, 64 filters, and 128 filters. In addition to that, there was one layer each of 512 filters and 1024 filters, all with filter size 3×3. categorical_crosentropy was used as loss for classification with stochastic gradient descent for optimizer and accuracy for the metrics while compiling the CNN. batch_size=32 means that 50,000 images from the dataset will be divided into 32 parts before the program compiles once we have set the number of epochs.

For this dataset, I had trained the model on various layers of CNN, different epochs and other techniques. Below is the learning I got from different models.

Method

Accuracy

2-layer CNN without SGD

<30% (Epoch = 50)

3-layer CNN with SGD

Around 70% (Epoch = 30)

4-layer CNN without SGD

Around 35% (Epoch = 50)

8-layer CNN with SGD

Around 80% (Epoch = 75)

Around 70% (Epoch = 10)

10-layer CNN with SGD (2 layers each of 32, 64, 128, 512 and 1024 filters)

Around 68% (Epoch = 50)

Because I trained the dataset on a hardware device that is not very computationally powerful, it took a lot of time but definitely with more epochs and better layering of CNN, one can achieve 90%+ accuracy in this dataset.

Check the code at https://github.com/raunak977/cifar10 .

I encourage those who try this to share the output they receive in the comments below.

Hope you enjoyed this article. For more such amazing articles check out hello ML . If you would like to improve this article or report something incorrect, please do let us know in the comments below.

If you have any copyright infringement claims, kindly send a mail to [email protected] . We deal with plagiarism very strictly and if our authors are found to be involved in plagiarism, we will take appropriate action against them after reviewing the claim.