But can we do better and go over 93%? Let's add another the FullyConnected layer and connect them using activation layers to remove non-linearity to cover the complexity of the task. Consider the following code for it:
arch = @mx.chain mx.Variable(:data) => mx.FullyConnected(num_hidden=128) => mx.Activation(act_type=:relu) => mx.FullyConnected(num_hidden=64) => mx.Activation(act_type=:relu) => mx.FullyConnected(num_hidden=10) => mx.SoftmaxOutput(mx.Variable(:label))nnet = mx.FeedForward(arch, context = mx.cpu())mx.fit(nnet, mx.ADAM(), train_data_provider, eval_data = validation_data_provider, n_epoch = 250, callbacks = [mx.speedometer()]);
The preceding model achieves as high as 97.62% accuracy! The output for it is ...
