JPH0478974A - Visual recognizing device - Google Patents

Abstract

PURPOSE:To recognize an object while considering a condition for decision by adjusting a neural network in one part of plural deciding means by a data corresponding to the first mode of a control signal and inputting picture element signals to the adjusted neural network corresponding to the second mode. CONSTITUTION:An input means 21 extracts the picture element values of respective color components separated from a color video signal in an objective area, and plural deciding means 23 input data for identifying the picture element signals to the neural network and output a code signal from the neural network. A control means 22 adjusts the neural network in one part of the plural deciding means 23 by the data corresponding to the first mode of the control signal, and controls the plural deciding means 23 so as to input the picture element values to the adjusted neural network corresponding to the second mode of the control signal, and an output means 24 outputs a recognized result according to the code signal outputted from the deciding means 23. Thus, the device itself is made optimum while considering the condition for decision corresponding to the object.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a visual recognition device using a multilayer neural network that can classify, discriminate, and measure objects.

[Prior art] In recent years, image processing technology has been applied to various industries, and images of objects such as industrial products and agricultural and marine products are input and processed by a computer system to classify and distinguish the objects. Visual recognition devices that measure and measure information are rapidly becoming popular.

Visual recognition devices in the field of image processing technology mainly binarize monochrome video signals related to objects, and classify and discriminate objects based on the amount of binarized information. Also,
In recent years, color images with a large amount of information have been used to perform highly accurate recognition.

Conventional visual recognition devices extract a target area based on the red (R), green (G), and blue (B) values of a color image captured by a color camera, or extract specific conditions such as the reflection characteristics of the target. It is configured to identify a plurality of target areas using algorithms, computer programs, etc. according to image conditions.

[Problems to be Solved by the Invention] However, the above-mentioned conventional visual recognition device judges an image using an algorithm and a computer program depending on the conditions specific to the object and the image state. In addition, as the number of discrimination conditions for classifying and distinguishing objects increases, it becomes difficult to create algorithms and computer programs. Another problem is that the values of undefined values in the computer program must be determined by trial and error.

An object of the present invention is to provide a visual recognition device using a neural network that can optimize itself by considering judgment conditions depending on the object.

[Means for Solving the Problem] The above-mentioned object of the present invention is to provide an input means for extracting pixel values of each color component separated from a color video signal of a target area, and a neural network for inputting data for identifying pixel values. a plurality of determination means that input code signals from the neural network and output them from the neural network; a control means that controls the plurality of determination means using control signals and pixel values; and an output that outputs recognition results based on the code signals output from the determination means. The control means outputs the control signal to the plurality of determination means, adjusts some neural networks of the plurality of determination means with the data according to the first mode of the control signal, and outputs the control signal to the plurality of determination means. This is achieved by a visual recognition device, characterized in that it is configured to input pixel values into a neural network that is adjusted according to a second mode of the invention.

[Operation] The input means extracts pixel values of each color component separated from the color video signal of the target area, and the plurality of determination means inputs data for identifying pixel values to a neural network and outputs a code signal from the neural network. The control means adjusts some of the neural networks of the plurality of determination means using data according to the first mode of the control signal, and outputs pixels to the neural network adjusted according to the second mode of the control signal. A plurality of determination means are controlled to input values, and the output means outputs a recognition result based on the code signal output from the determination means.

[Example] Hereinafter, an example of the visual recognition device of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a visual recognition device in this embodiment.

The visual recognition device shown in FIG. 1 includes an image input section 21 as an input means, and a determination module control section 22 as a control means.
, a plurality of determination modules 23 as a plurality of determination means, and a recognition result code output control section 24 as an output means.

Next, the operation of the above visual recognition device will be explained.

The image input unit 21 inputs a video signal of an object output from a color camera (not shown) and separates it into red (R), which is color information as each color component.

Create each pixel value of green (G) and blue (B).

The determination module control unit 22 selects red (R), which is color information.
, green (G), and blue (B) and an operation mode signal S as a control signal.

The recognition result code output control unit 24 outputs red (
Receives the recognition result code from each determination module 23 that outputs a recognition result code based on the values of R), green (G), and blue (B), and performs recognition from the determination module 23 selected by the determination module control unit 22. Output the result code.

Figure 2 shows color information of red (R) and green (G).

This shows how R, G, and B, which represent the values of blue (B), are distributed to three axes, and each axis is divided into NR7NG and NB (here, NR1NG and NB are positive values, respectively). represents an integer). In addition, color information R, G, B
Values are normalized from 0.0 to 1.0.

The determination module control unit 22 shown in FIG. 1 selects one determination module 23 for each of the divided small rectangular regions shown in FIG. 2. That is, color information R, G,
The control signal to the determination module 23 corresponding to the B value is turned on, and the control signal to the determination module 23 that does not correspond to the color information R, G, B value is turned off.

Therefore, when the color information R, G, and B values are divided as shown in FIG. 1, N×N×NB determination modules 23 are required.

FIG. 3 shows the configuration of the above-mentioned determination module 23.

The determination module 23 includes a recognition processing section 25, a sample data storage section 26, a sample data creation section 27, and a recognition result code output section 28.

The recognition processing unit 25 described above is configured by a multilayer neural network (details will be described later).

The sample data storage unit 26 includes the sample data creation unit 2
7 (details will be described later).

The sample data creation unit 27 receives an operation mode signal, a control signal, and color information R from the above-described determination module control unit 22.
, G, and B values to specify one or more representative partial regions of interest of the object by the sample image;
Each pixel value of color information R, G, and H within the designated target area and a recognition result teacher signal for the designated target area, that is, sample data are created.

Furthermore, when dividing the target region into a plurality of regions, a different recognition result teacher signal is given to each region to create similar data.

FIG. 4 shows the data format created by the sample data creation section 27.

As shown in FIG. 4, pixel values of color information R, G, and B are stored in area A, and recognition result teacher signal values for identifying the recognition results are stored in area B, respectively.

The recognition result teacher signal value is expressed as a binary value of "0" or "1", and when recognizing different areas, a different binary pattern is given. Therefore, the number of elements of the binary pattern of the recognition result teacher signal value can be changed depending on the number of regions to be classified.

When the operation mode of the operation mode signal as a control signal to the determination module 23 is the learning mode as the first mode, each pixel value R, G, B of the sample data stored in the sample data storage section 26 is recognized. The recognition result teacher signal value is input to the recognition processing unit 25 as a teacher signal.
5 and outputs the recognition result teacher signal value.

Further, when the operation mode of the operation mode signal is the recognition processing mode as the second moto, the color information R, G, and B values input to the determination module 23 are input to the recognition processing section 25, and the above-mentioned learning mode is input. A recognition result signal is output to the recognition result code output unit 28 by the multilayer neural network in the recognition processing unit 25 adapted in the above. The recognition result code output unit 28 converts this recognition result signal into a recognition result code and outputs it.

The determination module 23 outputs the above recognition result code and control signal as an output signal.

FIG. 5 shows the configuration of the recognition processing section 25 described above.

As shown in FIG. 5, the recognition processing section 25 is composed of a multilayer perceptron type neural network, which is a multilayer neural network imitating a biological neural network.

Next, the configuration and operation of the recognition processing section 25 will be explained.

The neural network constituting the recognition processing unit 25 includes an input layer 29 that receives input signals from the determination module control unit 22 and the sample data storage unit 26, respectively;
An output layer 31 that outputs a recognition result signal, an intermediate layer 30 located between the input layer 29 and the output layer 31, and an output layer 3.
The error signal calculating section 32 calculates an error signal based on both the output signal of No. 1 and the teacher signal.

Each of the input layer 29, intermediate layer 30, and output layer 31 described above each has a plurality of elements 33 called neurons (details will be described later).

Between each element 33 of the input layer 29 and each element 33 of the intermediate layer 30, and between each element 33 of the intermediate layer 30 and each element 3 of the output layer 31
3 is connected by a connection 34 that has a plastic weight, that is, the strength can be changed.

However, within each of the input layer 29, intermediate layer 30, and output layer 31, the elements 33 are independent from each other and are not coupled.

In FIG. 5, the intermediate layer 30 is composed of one layer, but the intermediate layer 30 can also be composed of a plurality of layers.

Note that even when the intermediate layer 30 is composed of a plurality of layers, the coupling between the layers, the structure and function of the element are the same as when it is composed of one layer.

FIG. 6 shows the above-mentioned element 33.

The element 33 is a multi-input-one-output element configured to output a single signal in response to a plurality of inputs.

The jth element is represented by element 33, where j is a positive integer,
Assuming that the input signal of the element 33□ is y, the weight for the input signal y is w, 1, the threshold value is θ5, and the characteristic function is f1, the output value is y, then the output signal is y.

is expressed by the following equation (■), and the characteristic function f (z) is expressed by the following equation (■).

y, = (Σ y, w+4-θ1)...■f
(x)=1/ (1+exp (-I))
-■ The recognition processing unit 25 shown in FIG. 5 adapts the neural network to output an appropriate output signal y in response to the input signal y1 when the operation mode signal S is the learning mode.

Setting and changing the values of weights w and H, which are necessary to adapt the neural network, will be explained below using the error backpropagation learning method.

Each pixel value (R), (G
), (B) are given as input signals to the input layer 29, and the binary pattern of the recognition result teacher signal is given to the error signal calculation unit 32, the input layer 29, intermediate layer 30 and Each element 33 of the output layer 31 sequentially calculates an output value, and the error signal calculation unit 32 calculates an error signal based on the output signal of each element 33 of the output layer 31 and the recognition result teacher signal.

As the error signal, values calculated by the following equations (1) and (2) can be considered.

ε. = Σ (y' L, -t,,,)
2...■ε=Σε.

...■ Here, the i-th element 33. of the binary pattern which is the teaching signal of the selected sample data number C. jl, c
1. The i-th element 33 of the output layer 31 for the input signal of the selected sample data number C. The output signal value of yol
,. Then, ε shown in equation (■). represents the error signal value for sample data number C of the selected input signal and output signal.

Further, ε shown in equation (2) represents the sum of error signals for all sample data stored in the sample data storage section 26.

In order to minimize the sum of error signals (hereinafter referred to as error) ε expressed by the above formula Find the differential and minimize the error ε.

Next, the total of the input layer 29, intermediate layer 30, and output layer 31 is
m layers, that is, the intermediate layer 30, where m is a positive integer, are (m −2
) layer will be explained.

First, if the input layer 29, intermediate layer 30, and output layer 31 are sequentially numbered, and the last layer 31 is the m-th layer, then between the layer 31 and the previous (m-1)th layer 30, The amount of modification of the connection 34 is expressed by the following formula (■).

ΔW″−′ z(1)= aδm , y m −1
+bΔw" t, l (1-1)...■δ1 (y' +, ti, e) y"+,
(1y"i,c)...■ Here, 6w", , (t) is the i-th element 33. of the m-th layer at time t. and the jth element 33 of the (m-1)th layer. , y'' is the output signal value of the i-th element 33 of the m-th layer for the input signal of the selected sample data number C, V m 1, , e is the selected sample data The first (
In the output signal value of the j-th element 33 of the m-1) layer, a represents a learning rate constant (slope rate), and b represents a constant for suppressing vibration at the time of convergence in the steepest slope method.

The amount of modification of the threshold value θ of each element 33 of the output layer 31 of the m-th layer is expressed by the following equation.

Δθ', (1) = -aδ' +bΔθ1t (+-1)...■
Here, Δθ” i (1) represents a modified value of the threshold value θ1 of the first element 331 of the m-th layer at time t.

In addition, the coupling 3 between the (m-1)th layer and the (m-2)th layer
The correction amount of 4 is expressed by the following formula (■).

△W″'-2() (1) − -aδ”-' + V-2*・ +bΔW”-2h, H(+-1) ・・・0
6m-1゜y"-1□. (1y"-' L.) x Σδ"'I Wm-' 1. + (1)
"'■Here, Δw"-2*, , (0 is time t
The third element 33 of the (m-1)th layer in . and number (
(+) is the correction value of the coupling 34 between the first element 33 of the m-th layer and the (m-th -1) Modified value of the coupling 34 between the jth element 33 of the layer, y''~23. is the first (
m-2) the th element 33 of the layer. represents the output signal value of

Further, the amount of modification of the threshold value θ1 of each element 33 of the (m-1)th layer is expressed by the following equation 0.

Δθ"-' + (1) = -aδ"-', 10bΔθffi'1l-1)
-@1 Here, Δθ'''-' I (1) represents the modified value of the threshold value θ of the j-th element 33 of the (m-1)th layer at time t.

Furthermore, from the connection 34 between the (m-2)th layer and the (m-3)th layer to the connection 34 between the second layer and the first layer, the above formulas ■, ■, and ■ are similarly applied. Operates recursively and sequentially.

In this way, all the connections 34 of the hierarchical network in the recognition processing unit 25 are corrected for all the sample data stored in the sample data storage unit 26, and this process is further performed until the value of the error signal in the above formula (■) converges. By repeating the process until the recognition process is performed until the recognition process is performed, a recognition process that is compatible with all the sample data, that is, a connection state between the elements 33 that is compatible with all the sample data is obtained.

Next, the operation when the operation mode signal S is the recognition processing mode will be explained.

When the operation mode signal S is the recognition processing mode, each pixel value (R) (G) (B) created by the image input unit 21
is processed by the recognition processing unit 25 through the determination module control unit 22.
is input to the input layer 29 of.

Each element 33 of the input layer 29 of the recognition processing section 25 calculates an output signal value according to the above equations (2) and (2).

Similarly, each element 33 of the intermediate layer 30 and output layer 31 is expressed by the formula
, (2), the output signal value is calculated based on the output signal value of the input layer 29 and the intermediate layer 30 and the value of the coupling 34.

The recognition result code output section 28 outputs the sample data storage section 2 based on the output signal value (output signal vector) of the output layer 31.
The closest one among the recognition result teacher signal vectors of the binary patterns stored in 6 is selected, and a recognition result code signal as a code signal for the binary pattern is output.

As described above, each determination module 23 receives the operation mode signal, control signal (consisting of a selection signal), and color information R, G, and B values from the determination module control unit 22, and
It operates according to the control signal and outputs the recognition result code signal and the input control signal.

The above output signals output from each determination module 23 are input to the recognition result code output control section 24.

The recognition result code output control unit 24 controls each determination module 2.
3, the recognition result code signal from the determination module 23 whose selection signal is on is output as the recognition result code, and the recognition result code signals from other determination modules are ignored.

As described above, the determination module control unit 22 selects one determination module 23 for the color information R, G, and B values. Also, in the learning mode, the judgment module 2
The multi-layer neural network configuring 3 is adapted to the sample data in the sample data storage section 26.

The above operation is performed on color information R and G within the entire target area.

If this is done for the B value, the sample data corresponding to each determination module 23 will be applied to all determination modules 23.
It is possible to adapt all the determination modules 23 by adapting each determination module 23 to each corresponding sample data created for and corresponding to the sample data. After all the judgment modules 23 are adapted, the color information R, G, B
The corresponding determination module 23 is selected by the determination module control unit 22 by giving the value. Then, the multilayer neural network constituting the selected determination module 23 outputs an appropriate recognition result code signal, and obtains an appropriate recognition result code from the determination module 23. In addition, since the space of color information R, G, and B values is divided into small spaces and one judgment module 23 is given to each small space, one judgment is required compared to the case where recognition is performed with one judgment module without dividing. The amount of learning data per module is reduced.

Further, in the learning mode, each determination module can operate in parallel.

[Effects of the Invention] An input means for extracting pixel values of each color component separated from a color video signal of a target area, inputting data for identifying pixel values to a neural network, and outputting a code signal from the neural network. It is equipped with a plurality of determination means, a control means for controlling the plurality of determination means using pixel values and a control signal, and an output means for outputting a recognition result using a code signal output from the determination means, and the control means is configured to control Outputting a signal to a plurality of determination means, adjusting some neural networks of the plurality of determination means according to the first mode of the control signal using data, and adjusting the neural network according to the second mode of the control signal. Since it is configured to input pixel values into the network, it is possible to extract target regions and identify multiple regions even for unknown images without creating algorithms and computer programs in advance for that image. It is possible to perform learning at high speed, and as a result, it is possible to recognize the target object by considering the determination conditions.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the configuration of one embodiment of the visual recognition device of the present invention, FIG. 2 is a diagram for explaining a method of dividing color information space, and FIG. 3 is the visual recognition device shown in FIG. 1. FIG. 4 is a block diagram showing an example of the configuration of the determination module of FIG. FIG. 6 is a diagram showing an example of the structure of the recognition processing section of the module. FIG. 6 is a diagram showing elements in each layer of the recognition processing section shown in FIG. 21... Image input unit, 22... Judgment module control unit, 23... Judgment module, 24... Recognition result code output control unit, 25... Recognition processing unit, 26... Sample data storage 27... Sample data creation unit, 28... Recognition result code output unit, 29... Input layer, 30... Intermediate layer, 31... Output layer, 32... Error signal calculation unit , 33...element, 34...coupling. Figure 1 Figure 5

Claims (1)

[Claims] an input means for extracting pixel values of each color component color-separated from a color video signal of a target area; and a plurality of determinations for inputting data for identifying the pixel values into a neural network and outputting a code signal from the neural network. means, a control means for controlling the plurality of determination means using a control signal and the pixel value, and an output means for outputting a recognition result using the code signal output from the determination means, the control means comprising: , outputting the control signal to the plurality of determination means, adjusting a neural network of some of the plurality of determination means according to the first mode of the control signal, and adjusting a neural network of a part of the plurality of determination means according to the first mode of the control signal; A visual recognition device, characterized in that it is configured to input the pixel value to the adjusted neural network according to a mode.