Today we wanted to add a short follow up on our previous post on artificial neural networks. The example which we gave of training an ANN on XOR function probably appears to be very trivial for the readers. Today we want to show another example that may highlight how neural networks can be used to recognize a simple pattern. This example is taken from Amanda Rao and Srinivas book Neural Networks (link at the end of the post). In their book they give an example of training a single hidden layer neural network to differentiate the capital letter T from the letter L. Either of the letters can be rotated 90, 180, or 270 degrees and the model should still be able to pick out the letter T.

With the above problem defined we can visualize it as below:

The top four figures represent the letter T while the bottom four represent the letter L. Now, to give these patterns to a neural network we need to put numbers to the patters. A logical way is to assign the number 1 to a cell in a pattern if it is shaded and 0 otherwise. Each cell will represent an input node in our neural network. Below is how we can represent each pattern with a combination of 1s and 0s.

Now we can set up a neural network in the workbook that we previously showed you how to build. We will set up an ANN with a single hidden layer with three nodes and a single output node. The output node will equal 1 if the model thinks the pattern it is presented with is one of four possible cases of the letter T and 0 if it is L. There will be 9 input nodes to input each pattern.

The front of the sheet should look like this:

The DataInput tab should look like below:

Once we train the network by clicking the macro button the network converges smoothly to the MSE minimum. Below is the mean squared errors profile as a function of epochs

In the DataInput tab we now have a column for the model output and we can see it identifies the letter T correctly in each case:

The final model with weights is presented below:

Hopefully this example shows the strength of neural networks to recognize patterns. This is a toy example of an actual application of neural networks. In real world large scale models are trained to recognize hand writing by similar methods. An image is split into pixels and the pixel colour is identified by a value. Each pixel is an input into the neural network. After training the neural network can differentiate a hand written letter a from b for example. Hopefully this example gives more insight than the XOR function we presented in our previous post.

Some Useful Resources:

Example taken from Neural Networks by M. Ananda Rao and J. Srinivas http://www.amazon.com/Neural-Networks-Applications-Ananda-Rao/dp/1842651315