JP2003504765A - neural network - Google Patents

Abstract

Abstract: A method for recognizing, identifying or classifying a spectrum, image or other input data sequence using a pulse-coupled neural network (PCNN). The PCNN outputs a time signature of the binary barcode data, which has a one-to-one correspondence with the input. This is then captured by a recognition engine, such as another neural network, which identifies based on a predetermined known reference time signature of the barcode data. Thus, the recognition process is simpler and faster than the recognition engine operating directly on a reference spectrum or image.

Description

Detailed Description of the Invention

The present invention relates to neural networks used for pattern recognition or identification. Among other things, the present invention can be used to recognize or identify or classify spectra produced by spectroscopic analysis of a sample. The present invention can also be used in a wide variety of image processing and pattern recognition applications.

It is known to perform identification of molecular species within a sample by studying the spectra produced by spectroscopic analysis of the sample. For example, Raman spectroscopy and infrared absorption spectroscopy have been used for this purpose. An example of a Raman spectroscopy device is
It is shown in European Patent Application No. 0543578.

However, the identification of compounds present is still essentially the task of a professional spectrographer, especially where it is desirable to identify compounds present in trace compounds and / or mixtures. The generated spectrum has to be compared with a known reference spectrum and the identified general features.

Recent studies of the visual cortex [1, 2] have largely highlighted the role of temporal processing using synchronous oscillations to identify objects. Several oscillatory neural network models have been developed using schemes for image processing. However, most of these methods use either the harmonic oscillator model or the phase oscillator model, and by themselves ignore the dendrite process.

A known neural network used for image processing is the pulse coupled neural network (PCNN), which is a computer model cortex.
ortex) Neural network (C-MCNN). It typically includes a two-dimensional, horizontally connected array of integrating and firing neurons. It can receive input generated by an imaging device, such as a charge coupled device, with one neuron connected to each pixel of the imaging device. In one embodiment of the present invention, Eckhorn [1
] 'S original neural network model has been modified according to the suggestions of Johnson et al. [2] for segmenting images.

One aspect of the invention provides a method for recognizing, identifying or classifying a spectrum, image or other input data sequence, the method comprising: a spectrum, image or data sequence in a pulse-coupled neural network. Pre-processing the input data that represents, thereby generating a time signature corresponding to the input data, and recognizing the time signature based on a predetermined time signature of a known reference spectrum, image or data sequence. Applying to a recognition engine for identifying or classifying.

The recognition engine can use a library of temporal signatures in the form of look-up tables. Alternatively, the recognition engine may itself be another neural network such as a multi-layer perceptron (MLP). It is of course known that such a recognition engine is used directly on the original spectral data, but if this recognition is performed with the time signature generated by the PCNN, it will be processed fairly quickly and easily. I understood.

A second aspect of the present invention is to provide at least one spectral, image or data sequence representation in at least a first pulse coupled neural network.
A spectrum, image or other input data sequence comprising the steps of processing one input data and thereby producing either an output barcode corresponding to the input data or an output data representing such a barcode. Provide a method for recognizing, identifying or classifying.

The output data representing the barcode can be used as a unique identifier for the input pattern or data sequence. Alternatively, it can be sent to a recognition engine for performing recognition of an input pattern or data sequence, as described above.

[0010]   Embodiments of the present invention will be described by way of example with reference to the accompanying drawings.

FIG. 1 of the accompanying drawings shows an example of a PCNN neuron consisting of the following three parts:

(1) The input unit (tree tree) includes image pixel inputs (real values) as well as feedback (binary) inputs from surrounding neurons.

(2) Linking and feeding merged to calculate internal activation U
ng) branch. Offset period "1" from the linking synapse to the integrated signal
Is added to the integrated signal from the feeding synapse,
Generate a membrane voltage U. Although linking branches may be omitted in some embodiments of the invention, it is not preferred.

(3) A pulse or spike generator that compares the threshold Θ with U. If U> theta, PCNN output is reset to the maximum value V _theta equals 1, output becomes zero, theta is reduced exponentially with time constant alpha _theta otherwise.

[0015]   PCNN can be implemented by repeating the following equations.

[0016]

[Equation 1]

[0017]

[Equation 2]

[0018]

[Equation 3]

[0019]

[Equation 4]

[0020]

[Equation 5]

Where S is the stimulus input (pixel intensity), F and L are the feeding and linking inputs to the neuron, and Y is the output. Other parameters include three potentials V and damping constants α, respectively associated with F, L, and Θ, where β is a linking conversion constant. The interconnections in M and W are local Gaussian (depending on the distance between neurons).

When a digital image is applied as input to such a PCNN network,
The network groups image pixels based on spatial proximity and intensity similarity, such as Kuntimad [3]. For example, in the case of time series, a two-dimensional input function can be mapped to a one-dimensional output function. Spatial averaging of the neural response gives a temporal output signal that is shown to be effective as a coding mechanism. The time signal G [n] is the number "on pixel" at each iteration, as calculated in equation (6).

[0023]

[Equation 6]

This changes over time as the iteration progresses, and is sometimes referred to as the “time signature” or “icon”. PCNN is the input image and its time signature (
1) to correspond to the icon).

Referring to FIG. 4, a simple example of the present invention is to use multiple luminance (or gray levels)
Receives input 10. These may represent the detector output on Raman or other spectroscopic devices according to European patent application 0543578. In such a device, the spectrum to be analyzed can be dispersed through a detector and an output (ie input 10) is produced for each pixel (or group of pixels) in the dispersed spectrum. This was captured in a computer and processed as follows, as described in the European patent application.

Each input 10 in FIG. 4 supplies one input to one neuron 11 of the PCNN array 12. Each neuron 11 is as shown in FIG. 1, and the input 10 corresponds to the input S. The connections between the neurons 11 are not shown in FIG. 4 for the sake of clarity.

Each neuron has an output 13 (corresponding to output Y in FIG. 1). From these outputs, the time signature TS is derived according to equation (6). This is taken into the recognition engine 20.

Since the PCNN makes one-to-one correspondence between an image and its time signature (icon), it is preferable to utilize this characteristic to create a database from the reference Raman spectra of known compounds.

The above-mentioned PCNN unit is merely a pre-processing device, and it is necessary for the recognition engine 20 to perform some task-specific post-processing. In the simple case, the recognition engine may include a library of time signatures (eg, look-up table format) generated from spectra of known compounds. If an unknown compound is identified, a similar time signature is generated from its spectrum to determine the closest match in the library.

Alternatively, post-processing in recognition engine 20 can be performed by a multi-layer perceptron (MLP) classifier. This is a feedforward / backpropagation neural network.

A database of known compound icons is used for training / testing the MLP neural network classifier. The icon generated from the Raman spectrum of any material in the database using PCNN can then be matched by the MLP classifier with the corresponding signature in the large database for identification.

The method described above uses several trace compounds, namely pentaerythritol tetranitrate (PETN), dimethyldinitrobutane (DMNB), 2,4.
6-trinitrotoluene (TNT), 4 mononitrotoluene (4MNT), 3,
It has been used to distinguish 4-dinitrotoluene (34DNT), heroin, and cocaine [6,7]. Typical Raman spectra for these materials are about 244n
Excited at m and 633 nm.

Each Raman image needs to be repeated a significant number of times before a periodic pattern is observed in the time signature. Therefore, a sampled or gated time signature is used that covers a representative part of the cycle (100 iterations).
. Different temporal signatures were obtained for each Raman image of 16 different PCNN parameter sets (a, β, and V).

For excitation at 244 nm and 633 nm, feedforward neural networks were trained using 80 or 55 signatures, respectively, and an architecture of 30 input neurons, 10 hidden neurons, and 3 output neurons. Each training data is a gated PCN
It consists of N-time signature amplitude and output data (required output format corresponding to compound). The remaining signatures (32 or 25 respectively) were used to test MLPs.

The results of this test show a high success rate (> 98.3%) for identification.

Therefore, the identification process using a neural network avoids the consultation of a large list of combined Raman band / Raman peak intensities and saves time. It also saves laboratories from extensive training on Raman spectroscopy of these materials. Therefore, identification can be performed even if not an expert. The results of our preliminary tests in this study demonstrate the effectiveness of the hybrid neural network, i.e. its applicability in other similar areas.

The use of the MLP classifier with the time signature allows the identification to be performed much easier and faster than if the MLP classifier was constructed directly from the Raman spectrum of the reference sample.

A preferred technique for the generation of barcodes from (1) still images and (2) data sequences, such as spectra generated by spectroscopic analysis, will now be described.

Techniques for Generating Still Image Binary Bar Codes Described Below (FIG. 2a)
Needs to use two PCNNs, namely PCNN # 1 and PCNN # 2. PCNN # 1 is used to generate temporal signatures from still images [5-7]. This time signature is then gray-coded (g
rey-coded). PCNN # 2 is then used to generate a binary barcode from the Gray coded time signature [8]. Alternatively, PCNN # 2 can be used to obtain the time signature output and then feed it back (dotted line in Figure 2a) to obtain the corresponding gray coded image. This feedback can be repeated any number of times and the PCNN
The parameters of # 2 can be changed with each iteration.

A. 1 Generation of time signature from still image Step 1: Set parameters (attenuation parameter, threshold value, potential) of PCNN # 1 and number of repetitions.

Step 2: To generate a time signature (also called ICON),
Image (Fig. 2b) (directly or pre-processed) is applied to PCNN # 1-Fig. 2c. P
In the case of a specific parameter set for CNN # 1, input / output IC to PCNN # 1
There is a 1: 1 correspondence with ON.

A. 2 Generation of Gray Level Bar Code (GBC) from ICON Step 3: Set the number of color levels and select the order of color levels. The order set determines the end-use sensitivity. Convert the ICON to a gray level bar coded image (Fig. 2d).

A. 3 Generation of Binary Bar Code from GBC Step 4: PCNN # 2 Parameters (Attenuation Parameter, Threshold, Potential), Number of Iterations N (if N = 1, proceed to step 3a; otherwise, step 3b Go to)).

Step 5a: Generate a binary barcode in the first iteration and generate 1D BBC (
The image (Fig. 2d) is applied to PCNN # 2 to obtain Fig. 2e1). PCN
In the case of the specific parameter set for N # 2, there is a 1: 1 correspondence between the input to PCNN # 2 and this output binary barcode.

Step 5b: Present the image (FIG. 2d) to PCNN # 2 to generate a binary barcode for the 2D BBC (FIG. 2e2). In the case of the specific parameter set for PCNN # 2, the input to PCNN # 2 and the output binary barcode set correspond to each other in a 1: 1 relationship.

Note: Step 4 ′: As an alternative to Step 4, the parameters of PCNN # 2 can be set to generate the time signature of FIG. 2d. This time signature is fed back for Gray coding (return to step 3). This process (feedback option) can be repeated any number of times. However, the final step is 4 to 5 in the case of BBC generation.

For a given data sequence (spectrum, etc.), given as either a pair of data (x, y) (where (x, y) εR) or a complex number (x + jy) εZ described below. The technique for generating a binary barcode (Fig. 3a) requires the use of one PCNN. This PCNN is used to generate a binary barcode directly from the Gray coded data sequence. Alternatively, this PCNN can be used to obtain the temporal signature output and then feed it back (dotted line in FIG. 3a) to obtain the corresponding gray coded image. This feedback can be repeated any number of times and the PCNN
The parameters of can be changed with each iteration.

B. 1 Generation of Gray Level Bar Code (GBC) from Data Sequence Step 1: Set the number of color levels and select the order of color levels. The order set determines the end-use sensitivity. Convert the data pair sequence (Fig. 3b) to a gray level bar coded image (Fig. 3c).

B. 2 Binary Bar Code Generation from GBC Step 2: PCNN Parameters (Attenuation Parameter, Threshold, Potential),
Number of iterations N (if N = 1, go to step 3a. Otherwise, go to step 3b
Go to)).

Step 3a: Generate a binary barcode in the first iteration and generate 1D BBC (
The GBC (Fig. 3c) is provided to the PCNN to obtain Fig. 3d1).

Step 3b: Apply the GBC (FIG. 3c) to the PCNN to generate a binary barcode for the 2D BBC (FIG. 3d2). In the case of a specific parameter set for PCNN, the input and output binary bar code set to PCNN # 2 is 1:
1 corresponds.

Note: Step 2 ': As an alternative to Step 2, the parameters of PCNN are
Can be configured to generate time signatures of This time signature is fed back for Gray coding (return to step 1).
This process (feedback option) can be repeated any number of times. However, the final step is 2-3 in the case of BBC generation.

The techniques described above (FIGS. 2 and 3) can be used to generate the actual barcode that is itself displayed. Alternatively, the data ((
For example, a binary string) can be generated.

In one form of the invention, such a binary barcode (or corresponding data stored in some way) is simply used as a unique identifier for the original image or data sequence. Alternatively, as in FIG. 4, the data corresponding to the barcode can be provided to a recognition engine for recognizing and identifying unknown input patterns or data sequences.

To date, this new method has been used to (a) identify trace compounds from Raman spectra, (b) monitor cardiac cycle (ECG recordings), and (c) condition power transformers from transfer function spectra. Surveillance has been demonstrated in three different applications. The individual input spectra are provided to the PCNN in the form of gray level images. An example is the explosive Raman spectrum (trace compound), dimethyldinitrobutane (DMNB), used as input, and the corresponding 8-bit gray level produced by PCNN after iterations 1, 2, 7, and 8. Images and binary images were generated. There is a one-to-one correspondence between the barcode-like PCNN output and the corresponding input image. This feature is exploited in spectral recognition techniques. These barcodes are
It can be used alone (one repeat only) or in combination (multiple repeats), as described below.

The first iterative PCNN output is converted to a binary codeword and stored in a look-up table (LUT). For spectroscopic analysis, the codeword is obtained from the corresponding spectrum and its Hamming distance from all codewords in the LUT is calculated. The spectrum corresponding to the minimum Hamming distance gives the identity of the input. The tie at the minimum Hamming distance is the other PCNN.
It is solved by repeating this procedure using the codeword generated from the iteration. Thailand did not occur in the current study, and the recognition success rate was 100%.

Alternatively, in such studies, the MLP classifier can be used in the barcode of the recognition engine 20. Another faster possibility is to use known n-set RAM-based neural networks. The memory-based architecture of RAM-based neural networks offers many important features, including single-shot learning and fast recall times. The architecture is basically several lookup tables (LUTs) in parallel, each LUT corresponding to a weight matrix in a feedforward neural network. The speed of recognition and success rate are very promising.

Hybrid Neural Network (PCNN preprocessor and n sets of RAM
This technique, including base neural networks), has been successfully used for purposes of cognitive, diagnostic, and condition monitoring tasks. It is easier to recognize and offers better performance than any other existing technique. Test results demonstrate that this method is fast and robust and therefore suitable for real-time applications.

[0059]   This new technique has been demonstrated in several examples, including different applications.   (A) Power transformer condition monitoring from transfer function spectrum,   (B) identification of compounds from infrared spectra,   (C) Identification of trace compounds (anesthetics and explosives) from Raman spectra,   (D) monitoring the cardiac cycle, and   (E) Identification of pulsar profile.

As discussed above, various neural networks (PCNN, MLP
, N sets of RAMs) are implemented as software within the computer. However, the PCNN can be particularly advantageously implemented as a dedicated integrated circuit.

References The following references are incorporated herein. [1] R. Eckhorn, E.I. Reitboeck, M .; Amdt, and P.M. W. “Feature Linking Via Syn” by Dicke
Chronization amon distributed asssem
blies: Simulations of results from Ca
t Visual Cortex "Neural Comp. , Vol. Two, two
93-307, 1990.

[2] J. L. "Pulse-coupled neu" by Johnson
ral nets: translation, rotation, scal
e, distortion and intensity signal in
variance for images "App. Opt. , Vol. 33
, 6239-6253, 1994.

[3] G. "Pulse-coupled neural" by Kuntimad
al network for image processing "PhD
d dissertation, University of Arabama
, USA, 1995.

[4] T. Lindblad and J. M. "Image by Kinser
Processing using Pulse-Coupled Neur
al Networks "(Springer-Verlag, London)
1998).

[5] Linblad, T .; Pladgett M., et al. L. , And Kins
er J. Edited by "Proceedings of the Nint
h Workshop on Virtual Intelligence / D
dynamic Neural Networks "Proc. OF SPIE,
3728, 328-332 (1999).

United Kingdom Patent Application No. 9915788.5 (1999), to which this application claims priority
July 7, 2000) and No. 9922008 (September 20, 1999) are also incorporated herein by reference.

[Brief description of drawings]

[Figure 1]   FIG. 3 is a schematic diagram of PCNN neurons.

Figure 2a   It is a block diagram which shows the barcode-coding technique of a still image.

Figure 2b   It is a figure which shows an example of a still image.

[Fig. 2c]   FIG. 3 shows a time signature (TS) or icon of the image of FIG. 2b.

[Fig. 2d]   FIG. 3 shows the gray level barcode of FIG. 2c.

[Fig. 2e1]   It is a figure which shows the corresponding one-dimensional binary barcode (BBC-1D).

[Fig. 2e2]   It is a figure which shows the corresponding two-dimensional binary barcode (BBC-2D).

FIG. 3a   It is a block diagram which shows the barcode-coding technique of a data sequence.

FIG. 3b   It is a figure which shows an example of the data sequence as a (x, y) plot.

[Fig. 3c]   FIG. 4 shows the gray level barcode of FIG. 3b.

[Fig. 3d1]   It is a figure which shows the corresponding one-dimensional binary barcode (BBC-1D).

[Fig. 3d2]   It is a figure which shows the corresponding two-dimensional binary barcode (BBC-2D).

[Figure 4]   It is a block diagram which shows a recognition process.

─────────────────────────────────────────────────── ─── Continued front page    (81) Designated countries EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, I T, LU, MC, NL, PT, SE), JP, US (72) Inventor Harry Coomer Shamshire Ragho             Poops             Mauritius Reduit University               Of Mauritius Department             Of Electrical and Elect             Ronic Engineering Faculty             Engineering of             ) F-term (reference) 5B057 CH20 DA11 DC36                 5L096 HA09 HA11 JA11

Claims (8)

[Claims] 1. Recognizing spectra, images or other input data sequences,
In a method for identifying or classifying, preprocessing input data representing a spectrum, image or data sequence in a pulse-coupled neural network and generating a time signature corresponding to the input data, the time signature comprising: Applying to a recognition engine for recognizing, identifying or classifying based on a known reference spectrum, a predetermined temporal signature of an image or a data sequence. 2. Recognizing spectra, images or other input data sequences,
A method for identifying or classifying comprises at least one step of processing input data representing a spectrum, an image or a data sequence in at least a first pulse-coupled neural network, thereby responding to the input data. A method characterized by producing either an output barcode or output data representing such a barcode. 3. The method of claim 2, wherein the output barcode is in binary format. 4. The method according to claim 2 or 3, characterized in that the first pulse-coupled neural network is further processed by a second pulse-coupled neural network. . 5. The method of claim 4, wherein the first pulse-coupled neural network produces a time signature and the second pulse-coupled neural network outputs the output barcode or output barcode. A method characterized by producing data representing. 6. The method of claim 4 or claim 5, wherein the output barcode is in binary format and includes an intermediate step of generating a gray level barcode or data representing a gray level barcode. And how to. 7. The method according to claim 2, wherein the output bar code or data representing the output bar code is a predetermined reference spectrum, an image or a bar code of a data sequence. A method comprising applying a recognition engine for recognizing, identifying or classifying based on a bar code or data representing a predetermined bar code. 8. Method according to claim 1 or 7, characterized in that the recognition engine is a further neural network trained by a known reference spectrum, image or data sequence.