Convolutional Neural Networks in Artificial Intelligence

In today’s society, artificial intelligence (AI) is a rapidly growing industry with an infinite number of applications. AI is seen throughout the world and in the palms of almost every human in a modern society. We see these applications through everyday technologies such as virtual assistants within smartphones, movie streaming websites, music streaming services, e-commerce websites, and even web mapping applications on our phones. These are all examples of high level functioning artificial intelligence. These forms of AI, however, still all use a simple concept called pattern matching. It might not be apparent at first, but each and every one of these examples use some type of pattern recognition. Siri, the personal assistant on the Apple Inc. platform, utilizes voice recognition. In order for one’s voice to be recognized, the user must speak certain phrases into the microphone multiple times for the software to see the pattern in the user’s voice. Movie streaming websites, such as Netflix, track and analyze the patterns in genres that the user chooses from and suggests relative films television shows. Amazon, a popular e-commerce website, analyzes the types of purchases and browsing history within the site in order to propose similar purchases. Google Maps, a web mapping application, tracks the user’s location throughout a period of time and recognizes the everyday routine and pattern in which they travel and previews an estimated time of arrival to each location without the user even having to input an address. These systems of artificial intelligence all stem from the concept of Artificial Neural Networks (ANN).

Artificial Neural Networks are sets of algorithms that are designed to recognize patterns and work to mimic the biological structure of the human brain. They are a system of algorithms that are modeled to function and replicate the same way that humans tend to learn. Artificial Neural Networks consist of of an input layer, an output layer, and a series of hidden layers in between the two that interprets the input for the output layer to use later on. Each of the hidden layers can be thought of as a sort of “directed graph”. Each graph consists of a set of “nodes” and a set of  “connections” (edges) that connect each pair of nodes. Then, each node is left with the task of performing some kind of simple computation, and each connection conveys a signal from one node to another. ANN’s with deeper layers are able to identify more and more features in the given input in order to resolve pattern matching. An example of how this would work would be to give an image to the input layer. The input layer would then pass the image through to the hidden layers to analyze and deconstruct. One layer might be able to analyse the colors and the next layer could look at the shapes within the image. With that being said, the more layers that are implemented and the more data that is inputted, the more accurate the output will become in terms of pattern matching.

Simple pattern matching is merely the baseline of what artificial neural networks are capable of. ANN’s are able to pattern match on a much higher level such as classification, clustering, function approximation, and even forecasting. Classification is the assignment of each object to a specific “class” (one of many predetermined groups). This is essential in a number of areas such as image and speech recognition. Clustering is similar to classification in the way that it groups together objects that are relatively similar. However, they differ because classification assigns to predefined and predetermined groups. Clustering allows the assignment of objects to undefined groups by looking at the set of objects, analyzing their likeness, sorting them into their respective groups.  Function approximation is the task of learning or constructing a function that generates approximately the same outputs from input vectors as the process being modeled. Forecasting allows for the ANN to predict a future outcome on the basis of an event’s history.

Deep-learning networks are distinguished from the single-hidden-layer artificial neural networks by the number of layers between the input and output layers. Deep learning is qualified when there are more than one hidden layer in the ANN.

Abstract

Artificial Intelligence (AI) is an ever expanding industry in today’s society. Many people do not realize that they might even own a piece of technology that utilizes artificial intelligence or use artificial intelligence in their everyday lives. Many applications of AI start in the form of artificial neural networks. These networks can span from simple one-layered computations to complex multi-layered data interpretation. This paper discusses the premises of artificial neural networks, its higher level form in deep learning, an example of a more complex form of neural networks, and the applications of deep learning.

 Introduction

In today’s society, artificial intelligence (AI) is a rapidly growing industry with an infinite number of applications. AI is seen throughout the world and in the palms of almost every human in a modern society. We see these applications through everyday technologies such as virtual assistants within smartphones, movie streaming websites, music streaming services, e-commerce websites, and even web mapping applications on our phones. These are all examples of high level functioning artificial intelligence. At an abstract level, these forms of AI are able to sustain some level of pattern matching. It might not be apparent at first, but each and every one of these examples use some type of pattern recognition. Siri, the personal assistant on the Apple Inc. platform, utilizes voice recognition. In order for one’s voice to be recognized, the user must speak certain phrases into the microphone multiple times for the software to see the pattern in the user’s voice. Movie streaming websites, such as Netflix, track and analyze the patterns in genres that the user chooses from and suggests relative films television shows. Amazon, a popular e-commerce website, analyzes the types of purchases and browsing history within the site in order to propose similar purchases. Google Maps, a web mapping application, tracks the user’s location throughout a period of time and recognizes the everyday routine and pattern in which they travel and previews an estimated time of arrival to each location without the user even having to input an address. These systems of artificial intelligence all stem from the concept of Artificial Neural Networks (ANN).

Artificial Neural Networks Overview

Artificial Neural Networks are sets of algorithms that are designed to recognize patterns and work to mimic the biological structure of the human brain. They are a system of algorithms that are modeled to function and replicate the same way that humans tend to learn. Artificial Neural Networks consist of of an input layer, an output layer, and a series of hidden layers in between the two that interprets the input for the output layer to use later on. Each of the hidden layers can be thought of as a sort of “directed graph”. Each graph consists of a set of “nodes” and a set of  “connections” (edges) that connect each pair of nodes. Each connection conveys a signal from one node to another, labeled by a number called the “weight” indicating the extent to which a signal is amplified or diminished by a connection. Then, each node is left with the task of performing some kind of simple computation, and each connection conveys a signal from one node to another. ANN’s with deeper layers are able to identify more and more features in the given input in order to resolve pattern matching. An example of how this would work would be to give an image to the input layer. The input layer would then pass the image through to the hidden layers to analyze and deconstruct. One layer might be able to analyse the colors and the next layer could look at the shapes within the image. With that being said, the more layers that are implemented and the more data that is inputted, the more accurate the output will become in terms of pattern matching.

Deep Learning

A standard ANN only consists of an input layer, one hidden layer, and an output layer. Deep-learning networks are distinguished from the single-hidden-layer artificial neural networks by the number of layers between the input and output layers. Deep learning is qualified when there are more than one hidden layer in the ANN. The addition of more hidden layers allows for more complex and accurate functions. The learning of multiple levels of representation and abstraction helps to understand data within images, audio, and text.

Deep Learning Training

In order to receive accurate results, the ANN must be “trained”. The initial weights of the ANN are set randomly. After multiple sets of inputs, the weights of the ANN will be adjusted so that each node can carry out a specific computation accurately. Through a series of tests and various inputs, the ANN is able to be trained so that it can be calibrated spit out an accurate output.

Backpropagation Overview

The Back Propagation algorithm used in Artificial Neural Networks was invented in the late 1980s. In the training of a neural network, the initial results are usually incorrect. The first series of inputs are used to detect errors within the network so that it can alter the parameters of its algorithm. This can be similarly thought of as calibrating the scope of a rifle. The user fires off a sequence of rounds while adjusting the scope after each shot. Eventually, the scope will be adjusted to the point where the user can accurately place each shot. The Back Propagation algorithm will take the error of each input and adjust the neural networks parameters so that it computes the arithmetic sequences with less and less errors.

Fundamental Equations of Back Propagation

(Insert Figure (29) of Neural Networks and Deep Learning Book PDF)

We define the error of a specified neuron j in layer l by this equation. δl (lower case delta) can be defined as the vector of errors within layer l. (delta C) is known to be the cost function which calculates the difference between the initial network output and its expected output. (delta Z) is known to be the weighted input of each node.

(Insert Figure (BP1) of  Neural Networks and Deep Learning Book PDF)

This equation allows us to see the errors in the output layer of the network. The first expression (delta C / delta a), where (delta a) is the vector of outputs from the network, measures how fast the cost is changing as a function of the jth output activation. The second expression ( (lower case sigma)’ (z) ) measures how fast the activation function (lower case sigma) is changing at (z). The activation function expresses the output of a specific node with the given input or vector of inputs.

(Insert Figure (BP2))

 The equation given above allows us to find the errors in the succeeding layer. (w^(l+1)^T represents the the transpose of the weight matrix w^(l+1) for the (l+1)th layer. The second part of the expression, ⊙σ′(zl), is the Hadamard product, which allows us to move the error backward through the activation function and gives us the error in the weighted input of layer l. When we combine equation (x) and (y) we are able to find errors in all of the layers within the network.

Recurrent Neural Networks

One type of an evolved ANN would be a recurrent neural network (RNN). RNN’s are similar to standard ANN’s in the way that they take input and spit out input. However, the way that RNN’s process the given input is much different. When a recurrent neural network receives information, it keeps that information in a loop until a desired result. Each loop recognizes the data passed from the previous loop and builds its information on its past history. ANN’s differ because none of the information is retained from the previous input and each new given input is unaffected by the history of past data. RNN’s operate mainly on vectors, a resizable strings of inputs. Typically, they accept an input vector and give out an output vector. ANN’s, on the other hand, can only operate on a fixed-sized input and gives only a fixed-sized output. A standard ANN also only has a fixed amount of layers than compute data in between the input and output. The output vectors of an RNN are not only influenced by the input, but also the entire history of inputs that have been inserted. The computations of the hidden layer, in theory, require at least two iterations. The first iteration will take the newly desired input, and the next iteration uses the output of the first iteration. One example of how this would work is through character prediction. We could give the network a long series of letter characters and ask it to give an output of the most likely character to come after. This allows us to string together phrases or sentences one letter at a time. RNN usage allows for much more flexible and complex computations as opposed to the standard one-layered artificial neural network.

Convolutional Neural Networks

Another type of an ANN is a Convolutional Neural Network (CNN). These neural networks are most commonly used for the fields of image recognition and classification, which will be explained in further detail later on. CNN’s work through two main steps: the convolutional step and the pooling step.

Convolutional Step

The purpose of the convolutional step is to take in an image and deconstruct the image into useable data. An image is essentially made up of a matrix of pixels, with each pixel having a different integer value. In order to extract data from the image, a smaller matrix of pixels is applied to the input and scans over the image in small portions at a time. This smaller matrix of pixels is called a “filter”. The dimensions of the filter matrix vary in size and pixel values. This filter is applied to the top left most corner of the input and slides over to the right in a serpentine manner. Eventually, the filter matrix will output a filtered “map”, which in sense is another matrix that contains only the desired data all while preserving the spatial relationship between pixels. The variation of assigned pixel values within the filter matrix allows the filter to detect certain aspects of an image. For example, a filter could extract the top horizontal edge, the bottom horizontal edge, the left vertical edge, or the right vertical edge of a shape within the input image. This function is helpful in being able to scan the landscape of a given satellite image for agricultural and weather purposes. The convolutional step must use a series of filters in order to utilize accurate data for the pooling layer.

Pooling Step

Once the important data from the convolutional layer has been outputted, it is then inserted into a pooling layer. The task of the pooling layer is to simply the information it is given. After the size of the filtered map is calculated, the pooling layer prepares a condensed feature map of a proportionally smaller size. Then, the highlighted features that are given from the series of feature maps are saved and the irrelevant information is discarded. A commonly used procedure for pooling is max-pooling. This method targets the largest element in a given window and outputs it into the smaller matrix. These smaller matrices now contain the target data and passes them onto the fully connected layer.

Fully Connected Layer

Now that we have all of the important data given from the convolutional layer and pooling layer, the fully connected layer is able to classify the original given input image. Before the fully connected layer can function properly, a fixed amount of possible outputs must be defined. These outputs all have their own uniquely predefined qualities. Once this is implemented, the fully connected layer is able to see what input correlates to the proper output. For example, if we were to give the input layer an image of a dog sitting in a field and defined the possible outputs as a cat, a bird, a dog, and a fish. The convolutional and pooling layers would then scrape the entire image down the most essential details of the animal. Then, the fully connected layer would analyse the characteristics of the animal and send it to its respective output node, resulting in a dog.

Radial Basis Function Neural Network

A Radial Basis Function Neural Network is another type of ANN that is most commonly used in clustering. Unlike the previous two networks, this type only consists of three layers: an input layer, an output layer, and a hidden layer. The hidden layer contains an x amount of nodes an one bias node. The nodes of the input layer are fully connected to the nodes of the hidden layer except for the one bias node, as depicted in the figure below. All of the hidden layer nodes, however, are completely and fully connected to the output nodes.

The computations of the middle layer are given by the following Gaussian function:

Here we express c as the center and o as the width of the ith node within the hidden layer. As we can see from the preceding figure, x represents the input so that ||x-c|| is shown to be the net input to the ith node in the middle layer. All of this is contained within exp(…) which is the expected value. The bias node is a sort of “prototype” that is used to compare the input vector to a prototype vector. The point of this is to compare similarities for the output layer and eventually classification. The output layer z, contains an m number of possible class results.

Get Help With Your Essay

If you need assistance with writing your essay, our professional essay writing service is here to help!
Find out more about our Essay Writing Service

This equation shows that the output layer must compute the weighted sum of the outputs of the middle layer in order for each original output can be categorized into its respective class. In figure 4, we can see that there are a series of nodes within the output layer. Each one of these nodes represents a category as to which we are classifying each input from the input vector. When the middle layer is doing its computations, it decides how similar the input is to the category of an output node by way of decimal value. Each output node has a series of qualities that have already been defined before the inputs are applied. At the output stage, the most similar of inputs are linked to their most similar output nodes so that an accurate statement can be established.

Applications of Deep Learning

Simple pattern matching is merely the baseline of what artificial neural networks are capable of. ANN’s are able to pattern match on a much higher level such as classification, clustering, function approximation, and even forecasting. Classification is the assignment of each object to a specific “class” (one of many predetermined groups). This is essential in a number of areas such as image and speech recognition. Clustering is similar to classification in the way that it groups together objects that are relatively similar. However, they differ because classification assigns to predefined and predetermined groups. Clustering allows the assignment of objects to undefined groups by looking at the set of objects, analyzing their likeness, sorting them into their respective groups.  Function approximation is the task of learning or constructing a function that generates approximately the same outputs from input vectors as the process being modeled. Forecasting allows for the ANN to predict a future outcome on the basis of an event’s history.

Conclusion

Artificial neural networks are a concept that is the foundation of many complex technologies used in everyday lives. ANN’s are modeled after the biological human brain and are designed to learn in the same fashion as any other human. The multi-layered version of ANN’s are called deep neural networks. Deep learning is extremely versatile and have countless applications that have yet to be implemented.

References

• Schmidhuber, Jürgen. “Deep Learning in Neural Networks: An Overview.” Neural Networks, Pergamon, 13 Oct. 2014, www.sciencedirect.com/science/article/pii/S0893608014002135.