A Study About Artificial Neural Networks Computer Science Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

In this term paper I gave brief introduction of artificial neural network , the previous work on the topic by different people, attributes of the neural network that why they are used.The basic components of the artificial neural networks, some of the flow charts & application of artifiacial neural network.


An artificial neural network is a system based on the operation of biological neural networks, in other words, is an emulation of biological neural system. It is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. It consists of an interconnected group of artificial neurons and processes information using a connectionist approach to computation. The development of artificial neural networks has been marked by periods of considerable optimism and others of disillusionment . A realistic assessment of the potential of artificial neural networks is attempted and some of the unrealistic expectations which has grown around a new and developing subject are dispelled . The question whether to use artificial neural networks to solve a particular problem is a matter of judgement on the part of the designer responsible for the project . The neural network would be a suitable candidate f or use if significant advantages in important are as such as cost , speed of operation , reliability , ease of maintenance, ease of initial development, ease of deployment and modification can be shown to exist .

As neural network applications are still in the early stages of development many practical prob1ems are likely to be pioneering applications and it will not be possible to rely on established precedents as a guide. There will therefore be some element of risk in the choice of a neural network should the application fail . The consequences Of electing to use neural networks for new applications should neural networks scheme fail to provide the required degree of performance would involve costs such as the loss of time and development costs . The underlying reason for using a artificial neural network in preference to other likely methods of solution is that there is an expectation that it will be able to provide a rapid solution to a non trivial problem. Depending on the type of problem being considered t here are often satisfactory alternative proven methods capable of providing a fast assessment of the situation.


Artificial Neural Networks now have a relatively long history and a correspondingly large amount of literature exists on their properties and development. It would be unrealistic to attempt to provide an extensive coverage of the literature . The study is accordingly restricted to some of the most significant historical markers in the development of ANN'S and these are given below. The references and events outlined in t his section cover only what are believed to be major developments and turning points in ANN research .

A. Mccullock and Pitts

The first significant paper on artificial neural networks is generally considered to be that of McCullock and Pitts in 1943 . This paper outlined some concepts concerning how biological neurons could be expected to operate The neuron models proposed were modelled by simple arrangements of hardware which attempted to mimic the performance of the single neural cell .


The book 'The Organisation of Behaviour' written by Hebb i n 1949 formed t h e basis of 'Hebbian Learning' which forms an important part of ANN theory today. The basic concept under lying 'Hebbian Learning' is the principle that every time a neural circuit is used, the pathway is strengthened .

1.Availability of digital computers

About this time of neural network development the digital computer became more widely available and its availability proved to be of great practical value in the further investigation of ANN performance.

C.John Von N e u

In 1958 Neumann wrote a book 'The Computer and the Brain' in which he proposes modelling the brain performance by items of hardware available at that time .

D.Frank Rosenblatt

Rosenblatt constructed neuron models i n hardware during 1957. These models ultimately resulted in the concept of the Perceptron . This was an important development and the underlying concept is still in wide use today.

E.Bernard Widrow and Marcian Hoff

There searchers Bernard Widrow and Marcian Hoff were responsible for the development of first the ADALINE and then the MADALINE networks. The name ADALINE comes from 'ADAptive LINEar complier' and the name MADALINE comes from 'Multiple ADALINE' respectively. Much was made of the potential scope Of the technique but this was succeeded by a period of disillusionment.

F.Marvin Minsky and Seymour Pappert

In 1969 Marvin Minsky and Seymour Pappert published an influential book 'Perceptrons' which showed that the perceptron as developed by Rosenblatt had serious limitations. This very thorough work was well researched and showed that the Perceptron in the form it had at the time suffered from Severe limitations. The essence of the book Perceptrons was the assumption that the inability of the perceptron to be able to handle the 'exclusive or' function was a common feature shared by all neural networks. AS a result of this assumption interest in neural networks was greatly reduced. The overall effect of the book was to reduce the amount of research work on neural networks for the next 10 years. The book served to dampen the unrealistically high expectations previously held for ANN'S. Despite the reduction in ANN research , a number of peoples till persisted in ANN research work.


G.John Hopfield

After 10 years in the doldrums John Hopfield produced a paper in 1982 which showed that the ANN had potential for successful operation and showed how this could be developed. This paper was timely as it marked a second beginning for t h e ANN. While Hopfield is the name frequently associated with the resurgence in interest in ANN it probably represented the culmination of many peoples work in the field. From this time onwards the field of neural computing began t o expand and now there is world wide enthusiasm as well as a growing number of important practical applications .


The resurgence of interest in artificial neural networks over the last few years is due to a number of f actors .One of the continuing and significant driving forces in neural network research has been the desire of many diverse groups which include neuro physicians, engineers , psychiatrists , psychologists and biologists to gain an understanding of the workings and behaviour of the brain .

Some of the valuable features of artificial neural networks which distinguish this method of computation from other algorithm based methods of computation are now considered.

A.The generalisation capabilty of neural networks

One of the most important if not the most important at tribute of neural networks is the ability to generalise . By this is meant the ability of a neural network to succesfully interpret data which it has not previously encountered and to provide a sensible result. It is this property which sets the neural network in a different category to a system such as a look up table where it is necessary to store all of the information likely to be required for reference on future OCCaSSlOns

Naturally there are limits to the generalising ability of any network and it is essential for the network t o have been trained on information which is closely related to that on which the network is expected to generalise. In the sense of generalisation it is helpful to consider the network as an interpolater within a multidimensional space. It is very different to the normal process of interpolation which is carried out by means of an algorithm where as here the information has been learnt by the trained network.

'The extrapolation ability of a net-work beyond the boundaries of the training information provided is non existant . The network has no way of having this information.

B.Parallelism in approach

Convential programable computers which operate in a serial manner as proposed by von Neumann appear to have reached a performance plateau based on the limits set by t h e switching speed of t h e present component elements. One way of overcoming the barrier imposed by component speed limitations is t o develop a different form of computer which is able to operate in a non serial manner. Such a computer would enable operations to be performed in parallel rather than having to wait for one operation to be completed before another operation can commence. Computers based on these principles are being developed but at the present time neither the computers or suitable programs are widely available.

An alternative method of increasing the speed of calculations is to utilise a neural network which works in a simulated parallel manner and is not limited by the serial requirements of the normal program. There is a slight paradox at the present time in that the parallel functioning neural network is implemented by means of the normal serial computer. The neural network which is an intrinsically parallel functioning device is forced by the limitations of current computers in to what is a serial functioning mode of operation . This situation will in all probability be rectified when parallel operating computers and appropriate programs become more generally available .

C.Distributed memory

One of the advantages claimed for a neural network is the fact that the memory is distributed over a large number of components within the network. This has been likened to the performance of the biological brain where a degree of damage may be sustained without an immediate reduction in ability. The loss of neurons within the brain during the natural process of aging does not result in an immediately obvious loss in the organisms performance.

The robust nature of the neural network is obtained as the result of many parallel paths being available for use. In the case of the artificial neural network in the interests of training efficiency there is an imperitive to reduce the number of neurons existing in any network. The reduction in neuron numbers while it improves the training performance results in a reduction in the robustness of the network to damage.

D.Neural networks exhibit intelligent behaviour

The trained artificial neural networks are sometimes said to exhibit intelligent behavior. Statements of this type raise the question as to what is meant by intelligent behaviour in these circumstances? In the case of a living organisium it could perhaps be said to be behaviour which improves the organisms chances of survival by finding food, finding a mate or possibly eluding the evil designs of a predator .

None of these conditions fit the case of a neural network and some other measure must be adopted. Possibly [Turings] concept of an inquirer being unable to tell in a 'blind' type interogation if the unseen respondant is a person or a machine may be a suitable criterion . At the present time practical neural networks are very specialised and any interogation of this kind would need to be strictly limited to the networks restricted area of expertise. It could be said that the limitation of the inquiry is only a matter of degree because even an interogation of an individual must also be limited to those matters within the individuals range of experience.

E.Learning not programming

The neural network is not programmed during training but actually learns the in formation being presented t o it. A t the present time it does not appear to be possible to readilly interpret the information stored within the trained neural network system. Understanding of this information may become possible in the future when a more mature understanding of the operation of neural network theory is developed.

Conventional computers do not have the ability to learn and consequently do not have the ability to generalise to accomodate previously unseen data.

F.General attributes of neural networks

In the real life situation there are many areas where expertise is required in order to accomplish the desired result . Many of these areas of interest involve assessment of almost continuous operations such as , the supervision of a power system operating condition , or the testing for explosives of baggage belonging to embarking air travellers .Work of this kind when done by human beings is highly repetitive and consequently boring factors which lead to operator in attention with an in advertent lowering of surveillance standards .I n addition the cost of keeping a sufficiently large crew of trained staff' on hand to continuously monitor processes such as this is extremely expensive. The neural network has the potential in some situations to provide an alternative method which is tireless, continuous, reliable and inexpensive replacement of personal for routine work of this kind.

The artificial neural network may be relied on to undertake suitable designated tasks in a systematic manner a t a speed which could not be attained or maintained by human operator . Such advantages naturally involve development costs and very likely some continuous maintenance costs .


The range of types of neural networks which have been developed is large and it is desirable to attempt to establish some kind of relationship between the various kinds of networks. There are a number of ways in which neural networks may be categorised based on characteristics such as,

-The method of training adopted, directed or non-directed

-Whether after training feedback or non feedback operation is involved,

-The type of training algorithm employed,

In order to consider the operation of artificial neural networks it is first necessary to introduce some of the terms used . This will be done in the following section .


The neuron forms the node at which connections with other neurons in the network occur. Like the biological network, the neuron in the artificial network is also central to network operation as much of the activity on the system occurs at the neuron. Althougth the in finitely more successful biological neural network neurons are not arranged in any geometric pattern those in the electronic network are generally arranged in one or more layers which contain neurons performing a similar function . Depending on t h e type of neural network being considered, connections may or may not exist between neurons within the layer in which they are located . For example in the Back Propagation Network there are no connections between the neurons in the same layer but in the case of the Hopfield Network every neuron is connected t o all neurons in the layer .


In the trained artificial neural network the intelligence of the network is stored in the values of the connections existing between the neurons. In artificial neural network terminology the values of the connections between the neurons are generally referred to as weights. hidden layers also take part in producing output when training is complete and the network is being interrogated. The number of hidden layers provided is problem dependant. There may be advantages in providing several hidden layers but additional layers may mean a marked increase in the training time taken.


In contrast to expert systems which incorporate a knowledge base, neural networks do not have such a collection of information. They need to be trained for a given problem or situation so that the weights will then contain the required information. An example of the iterative procedure necessary during training is given in the percepron training flow diagram One of the ways of classifying training procedures into two categories is whether directed training or non directed training is employed and these methods are now considered.

A.Supervised training or directed training

When employing directed training it is necessary to include among the set of data presented to the neural network the result or answer corresponding to each particular set of data. The data set is repeatedly presented to the neural network until the network output corresponds closely enough to the result in the data set. Should the difference between the network output and the 'Target Value' exceed the permitted tolerance the network training process is repeated. The algorithm being used for the network training causes further adjustment to occur and the process is repeated until the tolerance is acceptable .

In supervised training training data set contains a collection of information representing the input pattern vector together with the target value or desired output. This set of training data is presented to the network repeatedly until the difference between the target output and the actual output of the network reaches a certain predetermined value. The perceptron training flow diagram given in Fig: uses directed or supervised training and the procedure for obtainingthe difference value may be seen in the figure .

After each evaluation of the training algorithm the difference value between the actual output and the target output contained in the training set is compared with the permitted error value. If the difference is equal to or less than the error value when the process is stopped then the network is considered to be trained . Should the difference be greater than the permitted error value the set of training data is again presented to the training algorithm and the connection weights of the neurons changed until the output satisfies the criterion .

B.Unsupervised training or non- directed training

In t he case of non directed training the target value, or answer, is not provided and the information in the training data set is continuously presented until some convergence criteria is satisfied

In unsupervised training the output pattern for a given input pattern is not provided. The neural network constructs internal models that capture regularities in input pattern .

For any problem it is necessary to provide the neural network with training data which covers the extent of the problem. This data will need t o include sufficient information so that the problem is unambiguous. The training set is repeatedly applied to the neural network until some specified training criteria is met

C.Selection of training data

The selection of training data is a crucial and difficult task with conflicting requirements which must be satisfied in the best way possible .


The first requirement is that there is sufficient training data presented to the network so that an unambiguous output is able t o be obtained. This requires the careful selection of parameters for the particular problem. In some cases the availability of sufficient suitable data may be limited and this can pose a difficult task in itself .

The elimination of any data which does not c ontribute to the performance of the trained network is also required . This may be done automatically by eliminating any parameters which are not participating in the training process. Caution must also be exercised in this respect as some data may seem to be contributing little to the solution but when eliminated the network performance is significantly degraded.

D.Termination of training

The decision when to stop neural. network training is a difficult one. If the problem is a relatively simple one it is easy to say when the answers provided during training and testing fall within certain predetermined error limits .

However significant practical problems are rarely as clear cut as this. I f an RMS error criterion is used this does not eliminate relatively large errors occurring with individual training sets even when acceptably small RMS errors are obtained. In some cases this may not matter but in many instances the occasional grossly in correct answer may be unacceptable. Also if nice answers are provided always the problem may be a simple one and perhaps simpler more conventional alternative methods are available in preference to a neural network.

E.Feed forward and feedrack networks(result of training)

Feed Forward Networks are networks which after completion of the training process provide answers to queries as the result of a single forward pass of the data through the network. However during the network training phase a feed forward network may be traversed many times before the training criteria are satisfied and the network is considered properly trained . Once the training process is complete interrogation of the feed forward trained network requires only a single pass .

The training process for feedback networks is similar to that of feed forward networks but differs during the interrogation process. Even when training is complete a feedback network will require a series of passes during the interrogation in order to provide an answer to a query.

An example of a feed forward neural network is the Back Propagation network while the Hopfield network provides an example of a feedback network.


The development of computer-aided manufacturing systems has evolved to the phase of computer integrated manufacturing (CIM). It has been proposed that the next phase will be that of intelligent manufacturing systems (IMS). As a trend, manufacturing systems are demanding more and more flexibility in product design, process planning, scheduling, process control, and quality assurance. This may be achieved by building intelligent systems that can adapt to changes in their environment.

The application area of neural networks in manufacturing is surprisingly broad. It covers nearly all of the fields spreading from the design phase through process planning, scheduling and process monitoring to quality assurance. This section provides a comprehensive survey of neural network applications in manufacturing.

A. Design

Retrieval of old product designs that meet current requirements on geometrical and/or technical information is a problem that is often encountered in batch manufacturing systems. Venugopal and Narendran modeled the design retrieval system as a human associative memory and used a Hopfield network to develop a design retrieval system. The system was verified with test cases on rotational as well as non-rotational parts. The results show that neural network methodology is a promising tool for the development of practical design retrieval systems. The use of neural networks for design data retrieval was also studied by Kamarthi et al.. Instead of Hopfield networks, a back-propagation network was used. The result is also promising.

Kumara suggested an associative memory based modeling procedure for conceptual design. The motivation for their research stems from the following discussion.

It is possible that the designer may be aware of the structures that satisfy a particular set of functions. In his memory the designer may have stored the representations of a number of physical devices (design solutions). Given a (a set of) functional requirement(s) the designer will be able to identify a structure or a set of structures that will satisfy the required function(s) and hence by associating these structures with physical devices stored in his memory, he can selectively retrieve those designs. It must be noted that the physical devices could be from different domains. After having retrieved the designs, the designer mutates them to come up with new and creative design solutions.

The author used a back-propagation network for modelling the associative memory. A design model based on associative memory is also proposed to capture the conceptual design process (Fig:3).

The application of neural networks in design also has been studied by quite a few other researchers. Among them, Coyne and Postmus explored the application of neural networks to simple spatial reasoning in computer-aided design

Arai and Iwata suggested a four-layer neural network to connect lower level items to upper level ones on the design specification step of the conceptual design phase. Kim etal. applied a neural network approach for engineering drawing with geometrical constraints. Dhingra and Rao examined a new conceptual framework for solving design optimization problems based on a neural computing paradigm. Another interesting study was conducted by Chovan and Waldron. The authors proposed a cognitive model of the transformation from perceived form to function based on findings from a behavioral study of expert mechanical designers when they were reading two

Fig:3 Design model based on associative memory

dimensional mechanical drawings. The model was simulated using an ART network and was exercised and compared with findings from the behavioral study. The top-down and bottom-up reasonings exhibited by the subjects can be easily represented in the ART network. Their results show that ART networks might be useful for representing the behavioral system. The evidence demonstrated in their paper may provide useful information for the application of neural networks as the research community works toward the development of intelligent computer-aided systems for engineering design.

B. Process Planning

The first attempt to use neural network techniques in process planning might be that of Osakada et al.. The authors applied neural network techniques to an expert system for the process planning of cold forging in order to increase the consultation speed and to provide more reliable results. A three-layer neural network is constructed to relate the shapes of rotationally symmetric products to their forming methods. The shapes of the products are transformed into 16x 16 black and white pixels and are given to the input layer of the neural network. The back-propagation algorithm is employed. After training, the network is able to determine the forming methods for the products which are exactly the same or slightly different from those used as training examples. To exploit the self-learning ability, the authors further applied neural network techniques to the prediction of the most probable number of forming steps by considering the shape complexity and material property, the prediction of the die fracture and surface defect in the formed product, and the generation of rules from the knowledge acquired from an FEM (Finite Element Mod eling) simulation. It is found that the prediction of the most probable number of forming steps can be made successfully and the FEM results are represented better by the neural network than by statistical methods.

Hwang and Henderson applied a perceptron network in feature recognition, which is the first step in automated process planning (i.e., to interpret the design data from a CAD model). The goal of feature recognition is to convert a low-level representation such as face, edge, vertex to a semantically higher feature-based model. The network training is accomplished by manually presenting exemplars of features the user considers important in an engineering analysis (for example, manufacturing-related features for process planning). Their results show that the neural network approach took less time in feature recognition than other traditional approaches.

Knapp and Wang, applied neural network techniques for the automatic acquisition of process planning knowledge. In their approach, two cooperating neural networks are utilized. The primary network is a three-layer backpropagation network. The second fixed-weight network utilizes the MAXNET architecture. Parts to be planned are decomposed into machining features such as slots, holes, and planes. Each feature type is associated with a set of characterizing attributes such as dimensions and tolerances. Each feature is represented by a vector whose elements identify the feature type and its attribute values. This vector forms the input pattern to the primary network. The network responds to the presentation of a feature vector by activating certain output nodes, corresponding to the proposal of particular machining operations. The response of the network is trained using example process plans and the back-propagation learning algorithm. The second network is used to force a decision between competing operation alternatives. Its output is then fed back to the input layer of the primary network to provide a context for deciding the next operation in the machining sequence. The part is presented to the neural network feature by feature, the network then generates a sequence of operations for machining each feature of the part independently (global sequencing of operation across all features is not considered).

Process planning is knowledge-intensive in nature. Neural networks, being a useful knowledge acquisition tool, are expected to play an important role in process planning. However, process planning is a challenging task due to the interdependencies among the steps in a plan. It is not clear how neural networks can reason about the causal relationships among the entities in a process plan. Compared with symbolic systems (e.g., knowledge-based expert systems), neural networks are less effective in representing structured, contextual knowledge . Therefore, neural networks should be integrated with symbolic systems in order to solve process planning problems

C. Quality Assurance

Quality is the single most important factor in determining market share. There are two different approaches to quality assurance, namely, reactive quality assurance and proactive quality assurance. Reactive tools include sampling plans, lot acceptance determination, scrap or rework analysis, and so forth. Proactive strategy requires. an emphasis on physical cause-effect knowledge, risk analysis, experience, and judgment to justify action. Neural networks have been used for both reactive and proactive quality assurances. Reactive quality assurance is strongly related to monitoring and diagnostics. Therefore, neural networks can also play an important role in reactive quality assurance, especially where high processing and classification capabilities are required. Barschdorff discussed the application of neural network techniques for the quality control of electric drive motors. Sixteen spectral features of the motors’ vibration were input to a three-layer neural network. The network was able to recognize production faults, such as unbalance of the rotor, nonconducting winding connections, magnetic field failures, failures on commutators or bearings, loose parts, etc. The classifying ability of the network was compared with the results obtained by different pattern recognition algorithms. It was found that the performance of the neural network was superior.

Smith reported the use of back-propagation neural networks in quality control in an injection molding corporation (Orscheln Industries, Moberly, Missouri). Injection molding is a process with many variations in raw materials, machinery conditions and ambient conditions. It also has a temporal aspect where line conditions change during operation, affecting the end product. Neural networks are especially applicable when the data considered do not follow a known distribution or pattern, and, hence, are well suited for the quality control of injection molding. The results show that the neural network approach is comparable to other quality control methods, including control charts and statistical techniques, in goodness of output for quality control. An advantage of the neural network approach is the convenience of learning to establish the relationships directly, rather than through analysis and assumptions. Using a single network to monitor multiple products and/or quality parameters is an additional advantage.

Neural network techniques also can be used in proactive quality assurance. Schmerr et al. provided an innovative approach for planning robust design experiments through the use of neural networks. Robust design is a cost-effective technique for achieving high quality and reliability. The basis of the approach was to train a neural network on a set of tuples where each tuple corresponds to a Taguchi experiment along with its observed product response. Once trained, the neural network can probe the entire parameter space of design parameter settings, equivalent to performing a full factorial experimental design. With this information, a designer can identify alternative settings and search for optimal designs. The neural network approach was compared with analytical modelbased approach. It was found that the neural network had remarkable capabilities for generalization when trained on the same sparse array of experiments as used in complementary Taguchi analysis.

The research of neural networks for quality assurance has actually be implemented in manufacturing practice. CTS Electronics of Texas has used a neural network system to detect defective loudspeaker assembly lines. Ford Motor Company used neural network techniques to check car paint finishes. Motorola Incorporated used artificial vision based on neural network techniques for the quality inspection of their chips.


Foo and Takefuji developed the Integral Linear Programming Neural Network (ILPNN) and used it to solve jobshop scheduling problems. In a job-shop scheduling problem, the resources are typically machines and the jobs are the basic tasks that need to be accomplished using the machines. Each task may consist of several subtasks related by certain precedence restrictions. This problem can be formulated as a linear programming problem. The cost function to be minimized is defined as a sum of the starting times of all jobs subject to compliance with precedence constraints. The problem is solved using a linear programming network.

Vaithyanathan and Ignizio investigated the use of neural networks for solving certain types of large-scale, resource constrained scheduling problems. Their work was focused on dynamic resource constrained scheduling problems. Such problems, so characteristic of real world situations, involve the determination of a schedule subject not only to limited resources but also to sudden, unforeseen changes. They first decomposed the problem to be solved into a series of multidimensional knapsack models and established an equivalent neural network model for each particular representation. Then, they developed an approach that ultimately served to solve the original problem by extending the work of Hopfield and Tank. Their approach, to a great extent, avoided common neural network difficulties such as instability and local minima.

In addition to Hopfield networks, feedforward backpropagation networks can also be used in solving scheduling problems. Yih er al. provided a hybrid method that combines back-propagation neural network, simulation, and semi-Markov optimization to solve the crane scheduling problem. The crane scheduling problem occurs in a circuit board production line where one overhead crane is used to transport jobs through a line of sequential chemical process tanks. Because chemical processes are involved in this production system, any mistiming or misplacing will result in defective jobs. The proposed method consists of three phases: data collection, optimization, and generalization. Training data are purified using an operations research method (semi-Markov optimization). The neural network is used in the building of the decision making model. The resulted system performed better than the human scheduler from whom the models were formulated.

Other neural network approaches to scheduling problems reported in the literature include: time table scheduling, real-time scheduling, multiple-job scheduling, assembly scheduling , robot scheduling, a stochastic neural network (Gaussian machine) for scheduling, and an intelligent scheduling system (ISS) for flexible manufacturing systems where neural networks and expert systems are used to generate good schedule .

The application of neural networks in scheduling has been studied by many researchers. In some cases, the neural network approach holds significant advantages. In others, the usefulness and effectiveness of the neural network approach is debatable. However, at the very least, neural networks present a legitimate alternative to the more conventional methods for

Fig:4 Incremental learning and synthesis approach for process modelling and control

scheduling. Since the scheduling problem is one of immense importance, it is certain that considerable efforts will continue in applying neural networks in scheduling.


In this I only completed till the applications of the artificial neural network. But I have I didn’t include the problems related to it.However artificial neural network are widely used in manufacturing .thus it quit useful thing.