Edoardo Gabriele Barp

The neural network was trained separately using Keras with Tensorflow backend. Its structure consists of 3 hidden layers using respectively Relu, Tanh and Tanh as activation, and finally softmax for the output. It was trained on the notorious MNIST dataset for 50 epochs using sgd as optimizer, cross entropy as loss and dropout layers in between every hidden layers.

It attains an accuracy of 95% and loss of 0.15 on the testing data.

It's interesting to see that, even though the network has very good results on the MNIST dataset, it seems to easily get numbers confused in here. Neural networks (especially such small ones) will try to fit in a much stricter way, which in many cases (as you can try for yourself below) will make high jump in prediction for rather small strokes, and often strokes that to a human would not matter (e.g. slightly longer/smaller length of 1).

In theory, this could be solved by adding many more layers in an attempt to gain "abstraction" and therefore learn "patterns" instead of similarity, but new issues would arise from this such as training time, data requirements and overfitting. This is where CNN excels, thanks to their very different structures.