A neural network also known as an artificial neural network provides a unique computing architecture whose potential has only begun to be tapped. They are used to address problems that are intractable or cumbersome with traditional methods. These new computing architectures are radically different from the computers that are widely used today. ANN’s are massively parallel systems that rely on dense arrangements of interconnections and surprisingly simple processors (Cr95, Ga93).
Artificial neural networks take their name from the networks of nerve cells in the brain. Although a great deal of biological detail is eliminated in these computing models, the ANN’s retain enough of the structure observed in the brain to provide insight into how biological neural processing may work (He90).
Neural networks provide an effective approach for a broad spectrum of applications. Neural networks excel at problems involving patterns, which include pattern mapping, pattern completion, and pattern classification (He95). Neural networks may be applied to translate images into keywords or even translate financial data into financial predictions (Wo96).
Neural networks utilize a parallel processing structure that has large numbers of processors and many interconnections between them. These processors are much simpler than typical central processing units (He90). In a neural network, each processor is linked to many of its neighbors so that there are many more interconnections than processors. The power of the neural network lies in the tremendous number of interconnections (Za93).
ANN’s are generating much interest among engineers and scientists. Artificial neural network models contribute to our understanding of biological models. They also provide a novel type of parallel processing that has powerful capabilities and potential for creative hardware implementations, meets the demand for fast computing hardware, and provides the potential for solving application problems (Wo96).
Neural networks excite our imagination and relentless desire to understand the self, and in addition, equip us with an assemblage of unique technological tools. But what has triggered the most interest in neural networks is that models similar to biological nervous systems can actually be made to do useful computations, and furthermore, the capabilities of the resulting systems provide an effective approach to previously unsolved problems (Da90).
Neural network architectures are strikingly different from traditional single-processor computers. Traditional Von Neumann machines have a single CPU that performs all of its computations in sequence (He90). A typical CPU is capable of a hundred or more basic commands, including additions, subtractions, loads, and shifts. The commands are executed one at a time, at successive steps of a time clock. In contrast, a neural network processing unit may do only one, or, at most, a few calculations. A summation function is performed on its inputs and incremental changes are made to parameters associated with interconnections. This simple structure nevertheless provides a neural network with the capabilities to classify and recognize patterns, to perform pattern mapping, and to be useful as a computing tool (Vo94).
The processing power of a neural network is measured mainly be the number of interconnection updates per second. In contrast, Von Neumann machines are benchmarked by the number of instructions that are performed per second, in sequence, by a single processor (He90). Neural networks, during their learning phase, adjust parameters associated with the interconnections between neurons. Thus, the rate of learning is dependent on the rate of interconnection updates (Kh90).
Neural network architectures depart from typical parallel processing architectures in some basic respects. First, the processors in a neural network are massively interconnected. As a result, there are more interconnections than there are processing units (Vo94). In fact, the number of interconnections usually far exceeds the number of processing units. State-of-the-art parallel processing architectures typically have a smaller ratio of interconnections to processing units (Za93). In addition, parallel processing architectures tend to incorporate processing units that are comparable in complexity to those of Von Neumann machines (He90). Neural network architectures depart from this organization scheme by containing simpler processing units, which are designed for summation of many inputs and adjustment of interconnection parameters.
The two primary attractions that come from the computational viewpoint of neural networks are learning and knowledge representation. A lot of researchers feel that machine learning techniques will give the best hope for eventually being able to perform difficult artificial intelligence tasks (Ga93).
Most neural networks learn from examples, just like children learn to recognize dogs from examples of dogs (Wo96). Typically, a neural network is presented with a training set consisting of a group of examples from which the network can learn. These examples, known as training patterns, are represented as vectors, and can be taken from such sources as images, speech signals, sensor data, and diagnosis information (Cr95, Ga93).
The most common training scenarios utilize supervised learning, during which the network is presented with an input pattern together with the target output for that pattern. The target output usually constitutes the correct answer, or correct classification for the input pattern. In response to these paired examples, the neural network adjusts the values of its internal weights (Cr95). If training is successful, the internal parameters are then adjusted to the point where the network can produce the correct answers in response to each input pattern (Za93).
Because they learn by example, neural networks have the potential for building computing systems that do not need to be programmed (Wo96). This reflects a radically different approach to computing compared to traditional methods, which involve the development of computer programs. In a computer program, every step that the computer executes is specified in advance by the network. In contrast, neural nets begin with sample inputs and outputs, and learns to provide the correct outputs for each input (Za93).
]]>
The neural network approach does not require human identification of features. It also doesn’t require human development of algorithms or programs that are specific to the classification problem at hand. All of this will suggest that time and human effort can be saved (Wo96). There are drawbacks to the neural network approach, however. The time to train the network may not be known, and the process of designing a network that successfully solves an applications problem may be involved. The potential of the approach, however, appears significantly better than past approaches (Ga93).
Neural network architectures encode information in a distributed fashion. Typically the information that is stored in a neural network is shared by many of its processing units. This type of coding is in stark contrast to traditional memory schemes, where particular pieces of information are stored in particular locations of memory. Traditional speech recognition systems, for example, contain a lookup table of template speech patterns that are compared one by one to spoken inputs. Such templates are stored in a specific location of the computer memory. Neural networks, in contrast, identify spoken syllables by using a number of processing units simultaneously. The internal representation is thus distributed across all or part of the network. Furthermore, more than one syllable or pattern may be stored at the same time by the same network (Ze93).
Neural networks have far-reaching potential as building blocks in tomorrow’s computational world. Already, useful applications have been designed, built, and commercialized, and much research continues in hopes of extending this success (He95).
Neural network applications emphasize areas where they appear to offer a more appropriate approach than traditional computing has. Neural networks offer possibilities for solving problems that require pattern recognition, pattern mapping, dealing with noisy data, pattern completion, associative lookups, and systems that learn or adapt during use (Fr93, Za93). Examples of specific areas where these types of problems appear include speech synthesis and recognition, image processing and analysis, sonar and seismic signal classification, and adaptive control. In addition, neural networks can perform some knowledge processing tasks and can be used to implement associative memory (Kh90). Some optimization tasks can be addressed with neural networks. The range of potential applications is impressive.
The first highly developed application was handwritten character identification. A neural network is trained on a set of handwritten characters, such as printed letters of the alphabet. The network training set then consists of the handwritten characters as inputs together with the correct identification for each character. At the completion of training, the network identifies handwritten characters in spite of the variations (Za93).
Another impressive application study involved NETtalk, a neural network that learns to produce phonetic strings, which in turn specify pronunciation for written text. The input to the network in this case was English text in the form of successive letters that appear in sentences. The output of the network was phonetic notation for the
JVZoo Product Feed
