A Convolutional Neural Network From Scratch

This project was made for educational purposes to practice the skills and knowledge gained during the deep learning course and build my own convolutional neural network using minimum number of packages (numpy, pandas, matplotlib, scipy).

This neural network is for classifications tasks and it was mostly built for digit dataset from kaggle (train.csv, test.csv files).

Usage

Open main.py file in any notebook or ide.

test = ConvolutionalNeuralNetwork([('conv', [1, 5, 5, 3], 'relu'),         # 64x1x28x28 -> 64x3x24x24
                                   ('conv', [3, 5, 5, 3], 'relu'),         # 64x3x24x24 -> 64x3x20x20
                                   ('pool', [2, 2, 'max']),                # 64x3x20x20 -> 64x3x10x10
                                   ('flatten', []),                        # 64x3x10x10 -> 64x300
                                   ('full_conn', [300, [30, 20], 10,
                                                      'classification',
                                                       True, 'gd',
                                                      'leaky_relu'])       # 64x300 -> 64x10
                                       ])

test.cosmetic(progress_bar=False, loss_display=True, loss_graphic = False, iterations= 20)

test.train(train_batches, test_batches, 0.05, 3)

As you can see, you may choose layer type, convolution filter size, the number of filters, the number of inputs, outputs, hidden layers and number of neurons for every layer, gradient descent algorithm, activation function, alpha parameter etc.

Note: This implementation of a neural network is scalable, unlike other user implementations.

However, be careful when you increase the number of layers and neurons, as due to the high losses, the learning process becomes less controllable.

On digit image dataset CNN perfoms well ( about 90 accuracy ), but such architecture is not adapted for real tasks, because convolution operation and pooling are more complex operations than matrix multiplication. So, the training process takes a lot of time.

Neural Net Architecture

 ('conv', [1, 5, 5, 3]), ('conv', [3, 5, 5, 3]), ('pool', [2, 2]),  ('flatten'), ('full_conn',[300, [30, 20], 10])

progress_bar and loss_display

loss_graphic

To-Do List

• add regularization (look for new project - PyCandle)
• make it more robust for large number of layers and neurons (look for new project - PyCandle)
• make it faster (cupy, numba.njit)
• make class more pytorch-like (look for new project - PyCandle)
• add the ability to save and load model parameters
• add Batch Normalization (look for new project - PyCandle)
• add DropOut (look for new project - PyCandle)

References

• very clear explanation of how convolutional neural network is made

• nice 3Blue1Brown playlist about neural networks

• 3Blue1Brown video about convolution

• cool playlist to understand the whole process

• very important playlist about propagation in CNN

• one more explanation

• interesting github project on CNN too