JP4023330B2 - Image identification device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image identification device.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there has been known an image identification device that recognizes an object by color as described in, for example, Japanese Patent Laid-Open No. 4-369087.
[0003]
[Patent Document 1]
Japanese Patent Laid-Open No. 4-369087
[Problems to be solved by the invention]
The conventional image identification device has a problem that the identification accuracy is not high and a problem that it is easy to erroneously detect when the object is dark.
[0005]
The present invention has been made to solve such a conventional technical problem, and an object of the present invention is to provide an image identification device that has high color identification accuracy and can accurately identify a color even when an object is dark. And
[0006]
[Means for Solving the Problems]
The invention according to claim 1 includes an image input unit and a calculation processing unit, and the calculation processing unit identifies a desired target image from the original image input to the image input unit based on a preset set color. In the image identification device, the arithmetic processing unit stores in advance R0, G0, and B0 that are RGB values of the set color, and sets RGB ratios C = R0 / G0, D = G0 / B0, from the RGB values. E = B0 / R0 is obtained, and for each pixel of the original image, the RGB values H1, R1, G1, I = G1 / B1, J = B1 / R1 are obtained from R1, G1, B1, which are RGB values of the pixel. The ratios K1 = H / C, L1 = I / D, and M1 = J / E, which are ratios between the set RGB ratio and the original RGB ratio, are obtained, and each value of the ratios K1, L1, and M1 is 1 or less. In this case, K2 = K1, L2 = L1, M2 = M1, and the ratio K1, If each value of 1 and M1 exceeds 1, K2 = 1 / K1, L2 = 1 / L1, M2 = 1 / M1, and K2 × L2 × M2, which is the product of the ratios K2, L2, and M2, is obtained. The value is set as an evaluation value N, and the pixels having RGB values R1, G1, and B1 at which the evaluation value N is equal to or greater than a predetermined threshold are extracted as target pixels, and the target is selected according to the distribution of the target pixels. It has the identification means which identifies an image, It is characterized by the above-mentioned.
[0007]
The invention of claim 2 includes an image input unit and a calculation processing unit, and the calculation processing unit identifies a desired target image based on a preset color set in advance from the original image input to the image input unit. In the image identification device, the arithmetic processing unit stores in advance R0, G0, and B0 that are RGB values of the set color, and sets RGB ratios C = R0 / G0, D = G0 / B0, from the RGB values. E = B0 / R0 is obtained, and for each pixel of the original image, the RGB values H1, R1, G1, I = G1 / B1, J = B1 / R1 are obtained from R1, G1, B1, which are RGB values of the pixel. Meanwhile, the brightness information T of the pixel is obtained, and ratios K1 = H / C, L1 = I / D, and M1 = J / E, which are ratios of the set RGB ratio and the original RGB ratio, are obtained, and the ratio When the values of K1, L1, and M1 are 1 or less, K2 = K1, L2 = L , M2 = M1, and if the values of the ratios K1, L1, M1 exceed 1, K2 = 1 / K1, L2 = 1 / L1, M2 = 1 / M1, and the ratios K2, L2, M2 The product K2 × L2 × M2 is obtained and the value is set as the product S, the product S × T of the product S and the brightness information T is set as the evaluation value U, and the evaluation value U is equal to or greater than a predetermined threshold value. The pixel having the RGB values R1, G1, B1 and the brightness information T to be extracted is extracted as a target pixel, and an identification unit for identifying the target image according to the distribution of the target pixel is provided. is there.
[0008]
Here, as the brightness information T, F = R1 + G1 + B1, which is the sum of the RGB values of the pixels, can be used.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a schematic view of a robot 1 incorporating the image identification device according to the first embodiment of the present invention. A video camera 3 is attached to the nose of the head 2 of the robot 1. The eye part of the robot 1 is an infrared light receiving unit 4. A microphone 5 is attached to the ear of the robot 1 so that sounds and voices can be input.
[0010]
The head 2 is attached to the body 6 of the robot 1. The head 2 is attached via an actuator with two degrees of freedom and can be moved left and right and up and down.
[0011]
Four legs 7 are attached to the body 6 of the robot 1. The leg 7 is also attached via an actuator, can walk, and can control its own posture. In FIG. 1, 8 is a speaker and 9 is a discipline button. The drive mechanism of the robot 1 is a general one, and details are not shown.
[0012]
Video information output from the video camera 3 at the tip of the nose of the robot 1 is sent as an original image to an internal CPU 11 (to be described later). The CPU 11 performs image recognition processing and the like, and reflects this in the operation and the like. For example, if a ball of a favorite color is rolling, it is set so that it can be identified and chased, or if there is something that you do not like, it will be evacuated.
[0013]
FIG. 2 shows a block diagram of the control system of the robot 1. The operation control of the robot 1 is performed by executing a software program pre-installed by the CPU 11, but here, the arithmetic processing of the CPU 11 will be described separately for each function.
[0014]
Video signals Y (luminance), U (color difference), V (color difference) and synchronization signals Vs, Hs are output from the video camera 3 to the image input port 12 of the CPU 11. The CPU 11 captures data from the image input port 12 in synchronization with the synchronization signals Vs and Hs from the video camera 3 and performs image data processing in the image data processing unit 13.
[0015]
The image data processing unit 13 performs processing such as converting input YUV data into RGB values. The conversion of this YUV data into RGB values is as follows.
[0016]
[Expression 1]
R = Y + 1.402V
G = Y−0.714V−0.344U
B = Y + 1.772U
The image data processing unit 13 obtains an RGB value for each pixel with respect to the YUV data of each pixel from the video camera 3.
[0017]
Next, based on the RGB values for each pixel, the overall control unit 14 determines whether this pixel has the color of the identification target.
[0018]
When the number of pixels having the color of the identification object is detected in the image data at a predetermined value or more (details of this method will be described later), it is determined that the identification object exists in front of the video camera 3, Correspondingly, the behavior pattern of the robot 1 is determined and the operation control unit 15 is instructed. In order to realize the instructed action pattern, the operation control unit 15 controls the operation of each unit drive motor 21 and the sound generation unit 22, moves the robot 1 with the head 2, walks with the legs 7, and makes a roaring voice a speaker. The operation such as taking out from 8 is performed.
[0019]
The image identification process by the overall control unit 14 follows the procedure of the flowchart of FIG. For each desired large number of objects, an RGB value of a color to be identified and a ratio obtained by combining the RGB values are obtained in advance. These values are stored in the CPU 11 (step S1).
[0020]
As an example 1, when the object is a yellow tennis ball, R0 = 176, G0 = 189, B0 = 80, and the ratios C, D, and E of the respective colors are as follows.
[0021]
[Expression 2]
C = R0 / G0 = 176 / 189≈0.931
D = G0 / B0 = 189 / 80≈2.36
E = B0 / R0 = 80 / 176≈0.455
When an image signal of a certain object is input from the video camera 3 to the CPU 11, the CPU 11 calculates a ratio of RGB combinations of the input pixels. Here, it is assumed that the RGB signals are R1 = 130, G1 = 143, and B1 = 61. Then, the ratios H, I, and J of each color are obtained by the following calculation (step S2).
[0022]
[Equation 3]
H = R1 / G1 = 130 / 143≈0.909
I = G1 / B1 = 143 / 61≈2.34
J = B1 / R1 = 61 / 130≈0.469
Then, first ratios K1 = H / C, L1 = I / D, and M1 = J / E between C-H, DI, and E-J corresponding to each combination are obtained. Then, the divisor and the dividend are set so that the first ratios K1, L1, and M1 do not exceed 1 and the second ratios K2, L2, and M2 are obtained (step S3). For example, with respect to K1, if H> C, K2 = 1 / K1, and if H ≦ C, the second ratio K2 is determined as K2 = K1. The same applies to the second ratios L2 and M2 between DI and EJ.
[0023]
Thus, in the above-described example 1 (yellow tennis ball), the second ratios K2, L2, and M2 between C-H, D-I, and E-J are obtained as follows.
[0024]
[Expression 4]
K2 = K1 = H / C = 0.909 / 0.931≈0.976
L2 = L1 = I / D = 2.34 / 2.36≈0.991
M2 = 1 / M1 = E / J = 0.455 / 0.469≈0.970
Next, these values are multiplied to obtain an evaluation value N (step S4).
[0025]
[Equation 5]
N = K2 × L2 × M2
= 0.976 × 0.991 × 0,970≈0.938
Next, Example 2 will be described. In this example, the RGB values of the pixels are R1 = 70, G1 = 172, and B1 = 125. In this case, a method for obtaining each value of K2, L2, M2, and N will be described below.
[0026]
First, the ratios H, I, and J of each color are as follows.
[0027]
[Formula 6]
H = R1 / G1 = 70 / 172≈0.407
I = G1 / B1 = 172 / 125≈1.38
J = B1 / R1 = 125 / 70≈1.79
Then, the ratio of C—H, D—I, and E—J corresponding to each combination is obtained. At this time, the divisor and the dividend are determined so that the ratio does not exceed 1.
[0028]
[Expression 7]
K2 = K1 = H / C = 0.407 / 0.931≈0.437
L2 = L1 = I / D = 1.38 / 2.36≈0.585
M2 = 1 / M1 = E / J = 0.455 / 1.79≈0.254
These values are multiplied to obtain an evaluation value N.
[0029]
[Equation 8]
N = K2 × L2 × M2
= 0.437 x 0.585 x 0.254 ≒ 0.0649
Here, when the threshold value S1 is set to 0.7 in advance, since the evaluation value N of the pixel of Example 1 is larger than the threshold value, the pixel is included in the image of the desired object. (Steps S4, S6, S7). The determination in this case is referred to as “OK determination”.
[0030]
On the other hand, since the evaluation value N of the pixel of Example 2 is smaller than the threshold value, the pixel is recognized as a pixel that is not included in the image of the desired object (steps S4, S6, and S8). The determination in this case is referred to as “NG determination”.
[0031]
Since the method described above is calculated based on each ratio of RGB, it is not affected by luminance. Therefore, good image identification is possible regardless of whether the image is bright or dark. In addition, since the evaluation value N is obtained by multiplying the ratio of each element, the color difference can be obtained as a large numerical difference, and identification with high accuracy is possible.
[0032]
Next, a second embodiment of the present invention will be described. The functional configuration of the image identification apparatus of the second embodiment is the same as that of the first embodiment. Since the processing up to the step of obtaining the evaluation value N in the first embodiment is common in the processing functions in the overall control unit 14, the subsequent processing functions will be described in detail below.
[0033]
The overall control unit 14 in the present embodiment includes the brightness component when the S / N of the obtained image signal is not good as in the case where an inexpensive camera such as a pet robot is used as the image input means. More specifically, it has a function of evaluating with a second evaluation value obtained by multiplying the evaluation value N by the brightness information T.
[0034]
As the brightness information T, a luminance signal Y can be used. Specifically, the evaluation value N is multiplied by the normalized value of the input luminance signal Y value. That is, when the maximum value of the luminance signal Y is YMAX = 255, the second evaluation value P is multiplied by Y / 255,
[Equation 9]
P = N × (Y / 255)
(Step S5), and this value is compared with a preset threshold value S2 (step S6).
[0035]
In this case, the closer the second evaluation value P is to 1, the closer the color is to be identified. That is, an OK determination is made with a signal that is bright and has been photographed with good S / N (steps S6 and S7). The value obtained when the color signal is dark and the color signal is buried in the noise is lowered, and the ambiguous value is canceled (steps S6 and S8).
[0036]
Therefore, since only the good S / N color signal is determined to be OK, it is possible to identify with high accuracy even for a dark image.
[0037]
As the luminance signal Y, the input Y value may be used as described above. When the input from the video camera 3 is an RGB signal, Y may be obtained by calculation. Further, the value of R1 + G1 + B1 may be used as the brightness information T.
[0038]
The CPU 11 determines the flow of the entire control program based on the identification result obtained by this method, and performs operation control.
[0039]
First, it is determined from the color identification result for each pixel obtained by the above-described processing whether or not a desired object is included in the original image. An example of this determination process will be described.
[0040]
When the determination that the color of the desired object is the color (the above-described OK determination) is obtained in four pixels adjacent vertically and horizontally, it is determined that the desired object exists in front of the camera.
[0041]
In addition, when the determination that the color of the desired object is the color is obtained independently in each of the plurality of pixel areas, the area having the larger number of continuous pixels in the vertical and horizontal directions is determined as the desired object.
[0042]
This determination process should be appropriately set according to the performance of the CPU 11 mounted in the image identification apparatus and the number of pixels of the image receiving element of the camera 3. In any of the processing methods, it is only necessary to set so as to identify a region as a target object according to the distribution of the target pixel extracted by the OK determination on the image.
[0043]
When the CPU 11 identifies a desired object in the original image as described above, the CPU 11 controls the operation of the robot so as to execute a pre-stored gesture corresponding to the object. For example, the robot is walked or the neck is rotated by controlling the motor 21 of each joint. Also, an instruction is given to the sound generator 22 to make a cry from the speaker 8. With this control, it is possible to cause the robot to follow a favorite object or to retreat from a dangerous object.
[0044]
The technical idea of the present invention includes the following image identification processing method and image identification processing program.
[0045]
(1) An image identification method for inputting an original image to a computer and identifying a desired target image from the input original image based on a preset set color, and R0, G0 which are RGB values of the set color , B0 in advance, a step of obtaining set RGB ratios C = R0 / G0, D = G0 / B0, E = B0 / R0 from the RGB values, and RGB of each pixel of the original image The original RGB ratios H = R1 / G1, I = G1 / B1, and J = B1 / R1 are obtained from the values R1, G1, and B1, and the ratio K1 = H, which is the ratio between the set RGB ratio and the original RGB ratio. / C, L1 = I / D, M1 = J / E, and when the values of the ratios K1, L1, and M1 are 1 or less, K2 = K1, L2 = L1, M2 = M1, and the ratio K1 , L1 and M1 exceed 1 when K2 = / K1, L2 = 1 / L1, and M2 = 1 / M1, and the product of the ratios K2, L2, and M2, K2 × L2 × M2, and setting the value as the evaluation value N; and the evaluation value N An image identification method comprising: extracting the pixels having RGB values R1, G1, and B1 that are equal to or greater than a predetermined threshold as target pixels; and identifying the target image according to a distribution of the target pixels .
[0046]
(2) An image identification method in which an original image is input to a computer and a desired target image is identified from the input original image based on a preset set color, and R0 and G0 which are RGB values of the set color , B0 in advance, a step of obtaining set RGB ratios C = R0 / G0, D = G0 / B0, E = B0 / R0 from the RGB values, and RGB of each pixel of the original image While obtaining the original RGB ratio H = R1 / G1, I = G1 / B1, J = B1 / R1 from the values R1, G1, and B1, the brightness information T of the pixel is obtained, and the set RGB ratio and the original RGB ratio are obtained. Ratios K1 = H / C, L1 = I / D, and M1 = J / E, which are ratios to the RGB ratio, are obtained. When the values of the ratios K1, L1, and M1 are 1 or less, K2 = K1, L2 = L1, M2 = M1, and the ratios K1, L1, M When each value of 1 exceeds 1, K2 = 1 / K1, L2 = 1 / L1, M2 = 1 / M1, and K2 × L2 × M2, which is the product of the ratios K2, L2, and M2, is obtained and the value is obtained. Is a product S, a product S × T of the product S and the brightness information T is set as an evaluation value U, and RGB values R1, G1, B1, and the evaluation value U are equal to or greater than a predetermined threshold An image identification method comprising: extracting the pixel having brightness information T as a target pixel; and identifying the pixel as the target image according to a distribution of the target pixel.
[0047]
(3) A step of storing the RGB values R0, G0, B0 of the set color in the storage device in advance, and setting RGB ratios C = R0 / G0, D = G0 / B0, E = B0 / R0 from the RGB values. Obtaining the original RGB ratio H = R1 / G1, I = G1 / B1, J = B1 / R1 from the R1, G1, and B1 RGB values of the pixel for each pixel of the original image When ratios K1 = H / C, L1 = I / D, and M1 = J / E, which are ratios of the RGB ratio and the original RGB ratio, are obtained, and when the values of the ratios K1, L1, and M1 are 1 or less, When K2 = K1, L2 = L1, M2 = M1, and each value of the ratios K1, L1, M1 exceeds 1, K2 = 1 / K1, L2 = 1 / L1, M2 = 1 / M1, K2 × L2 × M2, which is the product of the ratios K2, L2, and M2, is calculated and the value is evaluated N Extracting the pixels having RGB values R1, G1, and B1 with the evaluation value N equal to or greater than a predetermined threshold as target pixels, and extracting the target image according to the distribution of the target pixels. An image identification program for causing a computer to execute the identifying step.
[0048]
(4) R0, G0, B0, which are RGB values of the set color, are stored in the storage device in advance, and the set RGB ratios C = R0 / G0, D = G0 / B0, E = B0 / R0 from the RGB values. Determining the original RGB ratio H = R1 / G1, I = G1 / B1, J = B1 / R1 from R1, G1, and B1, which are RGB values of the pixel, for each pixel of the original image, Pixel brightness information T is obtained, and ratios K1 = H / C, L1 = I / D, and M1 = J / E, which are ratios of the set RGB ratio and the original RGB ratio, are obtained, and the ratios K1, L1, When each value of M1 is 1 or less, K2 = K1, L2 = L1, and M2 = M1, and when each value of the ratios K1, L1, and M1 exceeds 1, K2 = 1 / K1, L2 = 1 / L1, M2 = 1 / M1, and K2 is the product of the ratios K2, L2, and M2 L2 × M2 is obtained, the value is set as the product S, the product S × T of the product S and the brightness information T is set as the evaluation value U, and the evaluation value U is equal to or greater than a predetermined threshold value. An image identification program for causing a computer to execute a step of extracting the pixel having values R1, G1, B1 and the brightness information T as a target pixel and a step of identifying the pixel as the target image according to the distribution of the target pixel.
[0049]
【The invention's effect】
As described above, according to the present invention, since the threshold value for identification is obtained based on the ratio of each RGB signal, it is possible to identify an image without being affected by luminance, and as a result, whether it is bright or dark. Image identification can be performed satisfactorily. Further, since the ratio of each element is multiplied, a color difference is obtained as a large numerical difference, and identification with high accuracy is possible.
[0050]
Further, according to the present invention, it is possible to cancel evaluation based on ambiguous image information having a poor S / N in a dark part of an image, and to improve the reliability of an image identification result.
[Brief description of the drawings]
FIG. 1 is a perspective view of a robot equipped with an image identification device according to an embodiment of the present invention.
FIG. 2 is a functional block diagram of the image identification apparatus of the embodiment.
FIG. 3 is a flowchart of image identification processing according to the embodiment.
[Explanation of symbols]
1 Robot 3 Video Camera 11 CPU
12 Image input port 13 Image data processing unit 14 Overall control unit 15 Operation control unit

Claims (2)

An image input unit and an arithmetic processing unit;
The arithmetic processing unit is an image identification device that identifies a desired target image based on a preset color from an original image input to the image input unit,
The arithmetic processing unit includes:
R0, G0, B0 which are RGB values of the set color are stored in advance, and set RGB ratios C = R0 / G0, D = G0 / B0, E = B0 / R0 are obtained from the RGB values,
For each pixel of the original image, the original RGB ratio H = R1 / G1, I = G1 / B1, J = B1 / R1 is obtained from R1, G1, and B1 that are RGB values of the pixel,
A ratio K1 = H / C, L1 = I / D, and M1 = J / E, which are ratios of the set RGB ratio and the original RGB ratio,
When the values of the ratios K1, L1, and M1 are 1 or less, K2 = K1, L2 = L1, M2 = M1,
When the values of the ratios K1, L1, and M1 exceed 1, K2 = 1 / K1, L2 = 1 / L1, M2 = 1 / M1,
K2 × L2 × M2, which is the product of the ratios K2, L2, and M2, is calculated and the value is set as an evaluation value N;
Extracting the pixels having the RGB values R1, G1, and B1 at which the evaluation value N is equal to or greater than a predetermined threshold as target pixels;
An image identification apparatus comprising: identification means for identifying the target image according to the distribution of the target pixels. An image input unit and an arithmetic processing unit;
The arithmetic processing unit is an image identification device that identifies a desired target image based on a preset color from an original image input to the image input unit,
The arithmetic processing unit includes:
R0, G0, B0 which are RGB values of the set color are stored in advance, and set RGB ratios C = R0 / G0, D = G0 / B0, E = B0 / R0 are obtained from the RGB values,
For each pixel of the original image, the original RGB ratio H = R1 / G1, I = G1 / B1, J = B1 / R1 is obtained from the RGB values R1, G1, and B1 of the pixel, while the brightness of the pixel Seeking information T,
A ratio K1 = H / C, L1 = I / D, and M1 = J / E, which are ratios of the set RGB ratio and the original RGB ratio,
When the values of the ratios K1, L1, and M1 are 1 or less, K2 = K1, L2 = L1, M2 = M1,
When the values of the ratios K1, L1, and M1 exceed 1, K2 = 1 / K1, L2 = 1 / L1, M2 = 1 / M1,
K2 × L2 × M2, which is the product of the ratios K2, L2, and M2, is determined and the value is set as the product S;
The product S × T of the product S and the brightness information T is defined as an evaluation value U,
Extracting the pixels having the RGB values R1, G1, B1 and the brightness information T for which the evaluation value U is equal to or greater than a predetermined threshold as target pixels;
An image identification apparatus comprising: identification means for identifying the target image according to the distribution of the target pixels.

Images