Implementation of Artificial Neural Network

Ø Implementation of Artificial Neural Network:
To obtain the full benefit of inherent parallelism found within Artificial Neural Network architectures we need to realize Artificial Neural Networks. ANNs can be realized in number of ways. But for the best results and real-life applications, however, ANNs need to be implemented as analog, digital or hybrid(analog/digital) hardware. It is a challenging task to make neurocomputing viable to implementing ANNs, verify their usefulness, and then produce the neurocomputers in cost effective manner. Various ANNs implementing techniques depending upon the requirements of the application such as speed, memory etc., can be classified as shown below.
Ø Software Implementation of ANN:
Software Implementation of Neural Network is done by writing software program. Such a simulation can be run on personal computers, mainframes and supercomputers, etc.
In lieu of attempting to arrange synthetic nerve cells and synapses into new computer architecture, the neuron and the processes can be simulated in mathematical form by transfer functions based on control theory (derived from empirical data) and solved serially on a conventional computer.
Such a approach was taken by Neuraltech with the introduction of Plato/Aristotle, the first commercial neural net development software package, this system runs on the IBM PC/AT.

Ø Hardware implementation of Artificial Neural Networks:
Hardware implementation of Artificial Neural Networks can be divided in three major classes:
1. Electronic Implementation
2. Optical Implementation
3. Molecular/Chemical Implementation

1) Electronic Implementation:
Electronic Artificial Neural Network implementations involve bus-oriented processor, coprocessor, Charge-Coupled Devices (CCD), Very Large Scale Integration (VLSI), Ultra Large Scale Integration(ULSI) circuit designs. In generals, this includes electronic hardware that is designed specifically for Artificial Neural Network implementations.
An infinity gain operational amplifier with strong negative feedback form the cell body part, synaptic strength (weight value) is proportional to the conductance Gj. The feedback resistance RF provide here a proportionality factor common for all weights. The processed signals and functions of the node (neuron) can be either analog, digital, or of a combined type. The analog computation is not only more plausible in biological Neural Networks, but constitutes the fundamental form of processing for nonprogrammable neural hardware.
Conventional resistors can be used to produce synaptic connections in the form of weights. The central problem of implementation of ANNs is to make weights that are continuously adjustable, preferably in response to an analog control signal.
The two methods that can be used are-
To make weights continuously adjustable.
2. To make weights that are digitally programmable.
Following are the examples of ANN developed using Electronic Implementation Technique.
Y.Le Cun has developed chip which works on an example to perform parallel threshold convolutions to representation of Handwritten Digits[7].
ETANN (Electrically Trainable Analog Neural Networks) ASCI chip 80170NX which works for selected Harmonic Elimination (SHE) in PWM Controller was developed by Intel.
ANNA chip (Analog Neural Network Arithmetic And Logic unit) is a Neural Network with 136000 connections for recognition of Handwritten Digits has been implemented using mixed analog/digital Neural Network chip was developed by AT&T Bell Labs.

2. Optical Implementation :
Connectivity and size in Electronic Artificial Neural Network implementation poses serious problems. Optical Neural Network gives solution of these problems. Interconnecting neurons with light beams requires no insulation between signal paths. Furthermore, the signal paths can be made in three dimensions. That way Optical Neural Networks offer tremendous advantage in speed and interconnect density.
The most prevalent computation in all ANN algorithms is in the vector-matrix form. This is obvious in the case of weighted linear input combination(WLIC) units. The potentially high density and inherent speed of optical devices makes them candidates for ANN implementations. Parallel implementation of multiplications may be achieved by use of an array of photodetectors, each having a variable sensitivity. This is achieved through an optically implemented mask. The combination of the two yields a variable sensitivity photodetector array(VSPD). Several optical designs use this form of optical signal modification to implement networks.
Optical neural networks configurations being divided in to two categories.
(1) Electro optical matrix multiplier.
(2) Holographic correlator.

2.1 Electro optical matrix multiplier:
Electro optical neural networks provide a means for performing the matrix multiplication in parallel. A system multiplying a three element of input vector by a 3´3 matrix and produce a three element NET vector. On left a column of light sources allows a beam of light to fall on the optical mask. Each light uniformly illuminates a single row of weight mask. For example Light 1 illuminates w11,w12,w13. The weight mask used is a photographic film in which the transmittance of each square is proportional to weight. On the right side a row of photodetector arranged in such a way that focus of light from each column of mask on to a corresponding photodetector. Light impinging on a photo detector D1 is the sum of product of light intensities multiplied by the transmittances for column 1.
This matrix multiplication is performed in a parallel. With a suitable high-speed light emitting diodes (LEDs) and photo detectors. This entire vector matrix multiplication can take place in less then nano-second and at the same time speed is independent of the size of the array.
In this simple version, the weights are fixed; they can be changed by substituting the different optical masks. One promising method uses a liquid crystal light valve in place of photographic film. This permits the weights to be adjusted electronically in a few microseconds. Liquid- crystal light valve is suitable for binary weights, but lacks the stability and contrast needed for continuously variable weights.
The photo detector output is fed back to drive the corresponding light inputs. A threshold activation function can be provided by using electronics circuitry following each photodetector.
Bidirectional Associative Memories(BAMs) can be realized in the same way. Nakano, K., has developed a model of associative memory called an Associatron using this principle.

2.2 Holographic Correlators :
Holographic techniques has potential for expanding information storage capacity and overcoming the power consumption drawback associated with Electro Optical Matrix Multiplier technique. Holography is an associative memory and there are analogies between the holography and the way information is stored in the brain. In holography, a pattern is reconstructed by illuminating the hologram with the reference beam that was used during recording. Thus a hologram is a form of associative memory. If neurons are designed to both receive and transmit light, photorefractive crystals can be used to interconnects large networks, and 108 to 1010 interconnection per cubic centimeter are possible.
The Holographic correlators store reference images in either a thin or volume hologram and retrieve them in coherently illuminated feedback loop.
An input image, which may be noisy or in- complete, is applied to the system and is simultaneously correlated optically with all of the stored reference images. These correlations are thresholded and fed back to the input, where the strongest correlation reinforces the input image. The enhanced image passes around the loop repeatedly, approaching the strong image more closely on each pass, until the system stabilizes on the desired image.

Image recognition is the most common application for optical correlator. The configuration of optical image recognition is shown in figure. Here the input to the system is an image formed by illuminating transparency with laser beam. This image is applied to the beam splitter, which passes it to the threshold device. The image is then reflected from the threshold device, passes back to the beam splitter, and then to lens 1, which focuses it on to the first hologram.
The first hologram contains several stored images. The image is correlated with each of them, producing patterns of light. The brightness of these patterns varies with the degree of correlation, a measure of similarity between the two images. Lens 2 and mirror 1 project these image correlations on to a pinhole array, where they are spatially separated. From the pinhole array, the multiple light patterns pass to mirror 2, through lens 3 and then are applied to second hologram, which has the same stored image as the first hologram. Lens 4 and mirror 3 then produce a superposition of multiple correlated images on the backside of the threshold device.
The threshold device is the key to the operation of this system. Its front surface reflects most strongly that pattern that is brightest on its real surface. The stored image most like the input image has the highest correlation; hence it is the brightest and will be strongly reflected from the front surface. This reinforced reflected image goes through the beam splitter where it reenters the loop for further enhancement. Eventually the system will converge on the stored pattern most like the input pattern.

3. Molecular/Chemical Implementation :

We know that because of several limitations in silicon technology, conventional digital computers do not now do well enough under certain aspects of computations such as,
1. Context-dependent computations.
2. Use of associative(content addressable) memories.
3. Responsiveness to natural language commands.
4. Understanding of and translation among the natural languages.
5. Making good guesses from partial knowledge of a situation.
6. Inventing new knowledge, etc.
For improvement one looks to advances in digital computer designs or to alternative materials and systems and styles of computation. The desirability of chemically based computers has to be measured according to the likelihood that such designs, if realizable, would help with any of the computations as mentioned above. The major rationale for building molecular electronic devices is to achieve advances in computational densities and speeds.
Molecular electronics may be used to develop a new breed of analog computers for applications such as pattern recognition, correlations, and context dependent decision making etc.
Two approaches can be used for building Molecular scale electronic devices.
1. One approach would depend upon synthetic materials, such as conducting polymers or charge transfer salts. For example polyacetylene is the best-known conducting polymer. It conducts electrically less when it is in pure state, and its conductivity increases greatly when it is doped by addition of minute quantities of any one of a number of elements or inorganic compounds. Even when doped, however, conductivity falls within the range of conductivities of semiconducting materials, not those of good conductors like copper or silver. Several other organic conducting polymers have also been synthesized for example polysulfernitride.
2. The other approach would use biological polymers. An approach, which has been proposed to build molecular electronic elements is to use proteins to build the structural frameworks for integrated circuits. Non-protein molecules may then be attached chemically to proteins which from the conductors, diodes, transistors, and other elements that make an integrated circuits.
Molecular-scale integrated circuits will be much more compact then existing chips available at much lower cost per element. Clark Mobarry and Aaron Lewis have implemented a neural network using photo activated conducting biological materials.