JPS62115600A - Pattern recognition equipment by color video - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is capable of photographing the front from a moving vehicle, for example, and identifying road signs installed along the road, as well as images with colors such as signs. The present invention relates to a pattern recognition device using color images that enables pattern recognition.

[Background technology] For example, the scenery in front of the vehicle is photographed, road signs installed along the road the vehicle is traveling on, lanes drawn on the road surface, etc. are identified and recognized, and the recognition results are then Various methods have been proposed to notify the driver of the problem and have him/her perform driving assistance operations. In order to perform this recognition operation, the existence of signs, lanes, etc. is recognized by pattern recognition from images taken by, for example, a television camera, and the recognition results are notified to the driver. It constitutes a device that has been developed.

Conventionally known pattern recognition means mainly perform image processing corresponding to shading, and such pattern recognition means are effective in specified places such as inside a factory where lighting conditions are good. However, it is extremely difficult to handle images of the natural world that are affected by light and shade and have shadows, and to recognize images of specific patterns that exist within these images of the natural world.

When recognizing such image patterns, the effects of brightness and shadows described above can be effectively eliminated by handling color information rather than shading as described above, but color image data The amount of image data is three times that of grayscale image data, and therefore the processing time for this image data increases significantly.

Therefore, it is difficult to perform pattern recognition processing in real time to recognize the presence of a target pattern image from a photographed color image.

That is, it is extremely difficult to photograph road signs placed along the road from a moving vehicle and perform a sign recognition operation that informs the driver of the contents of the road signs.

As a pattern recognition means for color images, means such as those shown in Japanese Patent Application Laid-open No. 59-53979 and Japanese Patent Application Laid-open No. 59-53980 have been considered, but the means shown here The method integrates analog data corresponding to each color, and therefore it is difficult to quickly recognize complicated patterns. Also,
As a means of recognizing road signs, Japanese Patent Application Laid-open No. 59-1
Although a means as shown in Japanese Patent Application No. 27200 has been proposed, the means shown here cannot eliminate the effects of brightness and shadow in the natural world as described above.

[Problems to be Solved by the Invention] This invention has been made in view of the above points. It is an object of the present invention to provide a pattern recognition device u using color images, which can reliably recognize images without being affected by color images, and can particularly effectively recognize image patterns including colors.

[Means for Solving the Problems] That is, in the pattern recognition device using color images according to the present invention, a color image i word of three primary colors is generated based on a color image photographed by, for example, a color television camera. This digital color image data is used as address data and is supplied to a memory that constitutes the color processing means. A logical value specified for each address address of the characteristic color is stored and set, and the logical value is read out based on the address data.Then, the read data from this memory is The image of the specified color is stored in the image memory as extracted image data, and a pattern matching operation is performed on the image data stored in the image memory and a preset pattern to determine the color image of the photographed color image. It identifies and recognizes video patterns specified from among them.

[Operation] In the pattern recognition device configured as described above,
The color pattern of the image to be extracted is stored in the memory constituting the color identification means, and the digital color image data of each of the three primary colors is read out from this memory as address data. . Therefore, when the color image data matches the color pattern to be extracted, for example, logic "1" data is read from the memory, and therefore this read data corresponds to the characteristic color. Begins to express images. That is, an image of a color pattern to be extracted is stored in the image memory, and by comparing this image data with a reference pattern, it becomes possible to identify and recognize the image of the desired pattern. In such a recognition operation, an image pattern specified from the photographed video is extracted based on the color pattern, so if there is a difference in brightness or darkness in the photographed video, or if there is a shadow etc. Even if there is such a thing, its features can be reliably extracted, and it will also be possible to effectively recognize patterns that exist in the natural world, such as road signs.

[Embodiments of the invention] Hereinafter, one embodiment of the present invention will be described with reference to the drawings.

FIG. 1 shows its configuration, and this embodiment shows an example of a case where a road sign set along a road is recognized from a moving vehicle such as a car.

That is, this device includes a color television camera 11 that photographs the scenery in front of the vehicle, including road signs. As shown in FIG. 2, this television camera 11 is installed on the front of a moving vehicle such as a car, and recognizes especially road signs set on the left side of the road. In this case, a color television camera 11 is set on the front left side of the vehicle to effectively photograph the left side of the road on which the vehicle is traveling.

The video signal photographed by the color television camera 11 is supplied to the color processing circuit 12, and in this color processing circuit 12, the video signal photographed by the television camera 11 is detected within the specified detection area. Characteristic colors that identify color images are extracted. That is, this color processing circuit 12 extracts only image data of a characteristic color specified from a photographed color image, and outputs a 1-bit serial signal for the characteristic color. Then, this serial extracted image data is stored and set in the image memory 13.

Control of the color processing circuit 12 and image memory 3 is executed by an arithmetic control circuit 14, and the arithmetic control circuit 14 is provided with a data memory 15 for storing programs and data. Further, this arithmetic control circuit 14 is supplied with detection signals from, for example, a vehicle speed sensor IB that detects the running speed of the vehicle, a steering angle sensor 18, etc., and detects images taken by the television camera 11. It is designed to correct deviations, distortions, etc., and the pattern recognition results etc. executed by this arithmetic control circuit 14 are outputted to the output device 17. This output device 17 outputs pattern recognition results to this vehicle 10, for example.
Specifically, the system notifies the driver of the recognition result of the road sign using a sign lamp, video, synthesized voice, etc., so that driving assistance operations can be performed.

FIG. 3 shows the portion corresponding to the color processing circuit 12 in more detail, and shows the color television camera 1 in more detail.
The analog video signals of the three primary colors red (R), green (G), and blue (B) from A/D converter 211.21
2. Each signal is converted into a 6-bit digital signal in 213, and a path buffer 2 having a tri-state function is provided.
21, 222.223. The output signals from the pass buffers 221 to 223 are supplied as address data to the memory cell 23, and the memory cell 23 is composed of, for example, 262114Xn bits (n is the number of required characteristic colors). . The n-bit output data from this memory cell 23 is supplied to the image memory 13 as image data.

In this case, a data signal is supplied from the arithmetic control circuit 14 to the memory cell 23 via the pass buffer 24 via the address data, and further via the pass buffer 25. A data signal is supplied to the memory cell 23.

FIG. 4 is for explaining the memory cell 23 portion in more detail. Addresses AO to 5 correspond to blue digital data BO to B5, and addresses A6 to Al correspond to blue digital data BO to B5.
The green digital data GO to G5 correspond to the addresses A12 to A17, and the red digital data RO to R5 correspond to the addresses A12 to A17, respectively.

Here, if the characteristic color of the extracted detection area is red, which corresponds to the color pattern forming the frame of a road marking, for example, MR>M□ and MR>MB... (1)
For each address that satisfies
”.

That is, the arithmetic control circuit 14 controls the path buffers 221 to 221.
223 to the off state, and the path buffers 24 and 2
Turn on 5. Then, the path buffer 24 outputs an address that satisfies the above formula (1), sets the data line of the pass buffer 25 to "1" for this address, and writes " 1
” is stored. For example, if the address AO-A17 of this memory cell 23 satisfies the above equation (1), data "1" is set. Also, data rOJ is set for .

In this way, the data is set from the first address to the last address of the address data of 6 bits corresponding to red, green, and blue, totaling 18 bits, so as to satisfy the above formula (1). After setting the data to , turn off the path buffers 24 and 25, and
223 to the on state.

That is, when the scenery in front of the vehicle 10 is photographed with the television camera 11 as described above, the signals of the three primary colors of the color photographed image are each converted into digital data, and are stored in the memory cell 23 of the color processing circuit 12 as address data. is supplied to Then, when address data based on color image data that matches the set characteristic color is input, a data output of logic "1" is obtained from the memory cell 23. The output data from this memory cell 23 is sequentially supplied to the image memory 13, and image data corresponding to the characteristic color of the detection area is extracted and this image data is written to the image memory 13. It is something that will be remembered.

Here, if the characteristic color to be detected in the detection area is blue, which is the color of the characters drawn within the frame of the road sign, then MB > MR and MB > MO...・(
For addresses that satisfy 2), logic "1" may be set in the same manner as above. Furthermore, when there are a large number of characteristic colors to be detected in the detection area, for example n, the number of memory cells may be multiplied by n and data may be set for these memory cells in the same manner as described above.

The characteristic color image data stored in the image memory 13 obtained in this way is used for pattern recognition, and in this case, the reference pattern data of the object to be recognized is the data memory 15 (; Memory settings have been made.

Then, this reference pattern and the pattern data based on the image data stored and set in the image memory 13 are subjected to a pattern matching operation in the arithmetic control circuit 14, and an image recognition operation is executed.

Figure 5 shows the flow of the recognition operation as described above. For example, when the ignition switch of a car is turned on, power is supplied to this pattern recognition device and this control routine is started. Ru. First, initial settings are performed in step 101, in which data settings for the memory cells 23 of the color processing circuit 12 are performed.

For example, when recognizing a road sign, red, which is the color pattern of the frame of this road sign, and blue, which is the color pattern of the characters drawn within this frame, are selected as characteristic colors, and the color pattern corresponding to this characteristic color is selected as the characteristic color. Data is set in the memory cell 23 in a state that satisfies equations (1) and (2).

Next, based on the detection signal from the vehicle speed sensor 16, it is determined in step 102 whether the vehicle is in a running state and a vehicle speed is present. If it is determined that the vehicle speed is high, the process proceeds to step 103 and the television camera 11
Allow users to input images taken with . As described above, this image is converted into digital color image data, a characteristic color is extracted from this color image data, and the image data of the characteristic color is stored and set in the image memory 13.

The image data stored in the image memory 13 in this way is
In step 104, the pattern is compared with the reference pattern stored in the data memory 15, and in step 105, it is determined whether or not a recognition target exists in the photographed image. in particular,
First, the image data of the characteristic color red is compared with the reference pattern corresponding to the shape of the frame of the road sign stored in the data memory 15, and if there is image data of red corresponding to this frame, it is determined that there is a frame. Do this. Then, the content of the road sign is recognized by performing pattern matching between the feature-exempt image data corresponding to characters and the reference pattern of characters and figures. For example, in the red round bar there is a ``50''.
If it is determined by pattern matching that the characters "" are drawn, the speed limit of this road is r50km/h.
J, and this recognition result is used in step 1.
The driver is notified at 0B.

In conventional color image processing devices and pattern recognition devices, analog signals of three primary colors are converted into digital signals, and then the digital signals are stored in image memories corresponding to each of the three primary colors. Then, various color calculations are executed. Therefore, the memory capacity for calculation requires at least four sets of three sets of image memories corresponding to each of the three primary colors and a memory for calculation. For example, if the gradation of each color is 64 gradations (6 bits) and the number of vertical and horizontal pixels is 256 pixels x 256 pixels, the total memory capacity will be 1,572,864 bits. Also, looking at the calculation time, in current image processing devices, the inter-image calculation requires at least 1/30 seconds, and for example, when performing a calculation that satisfies the above formula (1), rMR>Mal. operation, [MR>MB
A total of 3 steps are required for the calculation of J, calculations of equations (1) and (2), and (1/30)X3-1/10 (seconds) are required.

On the other hand, in the device shown in the above embodiment, the memory capacity is the capacity corresponding to 18 bits of the address of the memory cell 23 used in the color processing circuit 12, and the capacity of the image memory I3 of 256 x 256 bits. This requires a total of 327,680 bits.

The calculation time is the total time of the access time of the memory cell 23 and the time of storing the result in the image memory 3. If the access time of each of the memories 23 and 13 is set to 50 ns, for example, image capture All calculation results are stored with a delay time of 100 ns, which is required.

If the color processing circuit 12 only performs color processing that satisfies conditional expressions (1) and (2), this color processing circuit can be realized by the circuit shown in FIG. That is, this circuit is composed of high-speed comparators 31 to 34, an AND circuit 35 to which the output signals of the comparators 31 to 34 are selectively combined, and a blue circuit. On the other hand, outputs that meet conditional expressions (1) and (2), respectively, are output from AND circuits 35 and 36.

Furthermore, this color processing circuit 12 can also be configured as shown in FIG. That is, the matrix switches 41 to 43 are controlled by the control arithmetic circuit 14, so that outputs satisfying various conditional expressions can be obtained in response to input image data of each color of R, G, and B. It is something that

In this case, in the initial setup step of the color processing routine,
It is only necessary to set each matrix switch.

[Effects of the Invention] As described above, in the pattern recognition device according to the present invention, R and G can be detected from a photographed color video signal.

Digital color image data corresponding to each of the three primary colors of B is formed, and characteristic colors are extracted from this color image data based on a conditional expression set according to the color pattern of the object to be identified. Memory settings will now be made to the image memory. Then, a slam matching operation is executed based on the image pattern stored in the image memory, and pattern recognition is executed in very real time. Furthermore, since the color pattern matching of the characteristic colors is first performed according to the color image, the effects of brightness, shadow, etc. of the target landscape are effectively eliminated, for example. Recognition operations such as determining the content of road signs from images of objects such as road signs can also be effectively carried out, and for example, it will be very effective in realizing devices that identify and recognize road signs from moving vehicles. It is something that demonstrates the.

[Brief explanation of drawings]

FIG. 1 is a block diagram illustrating a pattern recognition device according to an embodiment of the present invention, FIG. 2 is a diagram showing an example of a specific implementation means of the above embodiment, and FIG. 3 is a color processing circuit portion of the above device. 4 is a block diagram illustrating the memory cell portion constituting the color processing circuit, FIG. 5 is a flowchart illustrating the operation flow of the device, and FIGS.
Each figure is a circuit configuration diagram illustrating another example of the color processing circuit. DESCRIPTION OF SYMBOLS 10... Vehicle, 11... Color television camera, 12... Color processing circuit, 13... Image memory, 14
...Arithmetic control circuit, 15...Memory (data),
IG...vehicle speed sensor, 17...output device, 18 steering angle sensor, 23...memory cell. Applicant's Representative Patent Attorney Takehiko Suzue Figure 3 Figure 5 Figure 6 Figure 7

Claims (5)

[Claims] (1) A means for generating color-separated color image signals of the three primary colors of red, green, and blue based on the captured color image, and an A/D that converts each of the color image signals of the three primary colors into digital image data. a conversion means, a color processing means for extracting image data corresponding to the characteristic color from the color image data converted into the digital signal, in which conditions for specifying the characteristic color of the target to be identified are set; an image memory for storing characteristic color data extracted from the image data as image data; a means for comparing patterns of the image data stored in the image memory with preset pattern data to identify and recognize patterns involving colors; A pattern recognition device using a color image, comprising: a pattern recognition device using a color image, which is capable of identifying and recognizing the existence of an image pattern of a specified color and a specified shape in the captured color image. (2) The color processing means is provided with a memory to which the three primary color image data is supplied as address data, and in which characteristic data of a specified color for identifying a color image pattern to be recognized is stored and set; 2. The pattern recognition device using color images according to claim 1, wherein characteristic color image data corresponding to the color feature is output based on the address data. (3) The memory constituting the color processing means is configured to store and set logical values respectively specified at addresses corresponding to the color image data of the three primary colors expressing the characteristic color, 3. The pattern recognition device using color images according to claim 2, wherein the specified logical value data is read out and output as characteristic color image data in a state in which the base address data expresses the characteristic color. (4) The color identification means is configured by a logic circuit that is supplied with the color image data of the three primary colors and outputs image data that satisfies the conditions of the characteristic color. Pattern recognition device using color images as described in Section 1. (5) The color identification means is configured by a matrix switch that is supplied with color image data of the three primary colors and outputs image data that satisfies the conditions of the characteristic color. The described color image pattern recognition device.