JP3464182B2 - Processing color images - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color image processing method.

[0002]

2. Description of the Related Art In robot vision, for example, it is an important task to recognize the characteristics of a scene quickly and accurately without being influenced by the environment. Detecting edges in an image is essential to extract scene features. As a method of detecting an edge in a black-and-white image, that is, a grayscale image, a difference method using a first derivative or a second derivative is known. These methods try to detect an edge by finding the difference in shade between neighboring pixels because the shade of the image changes greatly at the edge of the image. In addition, a method of obtaining an edge by a frequency domain filter process using a Fourier transform has been proposed.

On the other hand, in recent years, various edge detection methods using a color image acquired by a CCD camera have been proposed. When using a color image, not only can an edge be detected with higher accuracy than in a grayscale image, but a larger amount of information can be used to identify a reflective boundary on the object surface, etc.
It is possible to add countermeasures that were difficult for grayscale images. A conventional edge detection method using a color image is R
It is general to perform the integration process after performing the process on the three images divided into the respective GB components.

[0004]

However, all of the conventional edge detection methods using a color image basically use a spatial operator for obtaining a local differential, and the detection accuracy is higher than that of a grayscale image. However, there is a problem in that the amount of calculation increases as the amount of information increases and the processing time increases. It should be noted that even in the case of processing with a grayscale image, at present, a considerable amount of calculation is required, and the amount of calculation is drastically increased when a color image is used. In addition, there is a problem that real-time processing is difficult without dedicated hardware.

In addition, the conventional edge detection method has a problem that it is easily affected by noise, and nevertheless, fluctuations in illumination and noise in the image pickup system cannot be avoided in a real environment. Also, especially with conventional differentiation, the gradient of the edge (differential strength)
Therefore, there is also a problem that it is difficult to detect a boundary portion having a small difference between two colors, that is, it is difficult to detect a subtle color boundary.

Therefore, the present invention solves all such problems, has a small amount of calculation, can be processed by a general-purpose computer, is not easily affected by noise, and easily has a boundary portion with a small difference between two colors. The purpose is to be able to detect.

[0007]

SUMMARY OF THE INVENTION To achieve this object, the present invention uses the color hysteresis of the color of each pixel in a color image.
The color histogram is plotted on the togram space.
Information on the frequency of each color plotted in space is extracted and
The color extracted less frequently than other colors,
-Specify the position where the color exists in the image, and
The detected position is detected as an edge of the color image or an edge of a point in focus.

That is, the present invention was made by obtaining the knowledge that the edge portion of a color image has a small number of pixels of the same color. A color histogram having frequency information is created and the frequency information is extracted. By detecting the extracted infrequent portion as an edge, it is possible to detect the edge with a small amount of calculation and without requiring dedicated hardware. Moreover,
Since the focus is on a small number of areas on the color histogram, which is the edge part, the effect of the edge gradient (differential strength) is less than that of the conventional method that uses spatial differentiation, so that a boundary with a small difference between the two colors can be ensured. It has the advantage that it can be detected. Moreover, there is an advantage that even if there is noise such as illumination fluctuation during image capturing, the frequency of a small number of parts is not affected. Note that the infrequently detected part of the above method corresponds to the in-focus portion of the image, so it is necessary to detect the in-focus portion of the image as described above at the same time as the edge detection. You can

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A color histogram plots, for example, RGB values of each pixel forming a color screen in an orthogonal space of RGB. In the color histogram used in the present invention, each plotted point has information on the number of pixels, that is, frequency. Note that the color histogram may use a color system other than RGB.

When a scene with many artificial objects such as a room is photographed, walls and various planes constituting various devices usually have the same color distribution, and therefore, they are densely present on the color histogram. Focusing on the frequency, which is the information on the number of pixels, the frequency becomes high. Hereinafter, this portion will be referred to as a “multi-region”. On the other hand, the boundary between the surfaces, the boundary between the light reflection portion and the shadow, and the phase portion with a large density change such as a cylindrical object are only scattered sparsely on the color histogram. It is considered that the above-mentioned frequency is low. Hereinafter, this portion will be referred to as a "minority region". Replacing a color image with the frequency of each color value contained in the image can be roughly divided into a majority region and a minority region.

Pixels near the boundary of two colors in the color histogram space are considered to have both color components. For example, FIG. 2 shows an example of an edge that is a boundary between red and yellow in the xy plane. This edge is here in the direction of the y-axis. When the color histogram is created, the edges of FIG. 2 are shown as in FIG. In the color histogram of FIG. 3, points A and G indicate red and yellow on both sides of the edge, respectively, and points B to F are boundary colors and have both red and yellow color components. To orange. In FIG. 3, the size of each point represents the frequency thereof, and red and yellow, that is, points A and G, on both sides of the edge are multiple areas. On the other hand, the edge, that is, the points B to F is a small area.

FIGS. 4 and 1 show other examples of an image of an actual article and its color histogram. FIG. 4 shows an example of a color image of two cylindrical bodies arranged side by side. One of the cylindrical bodies has a red color at one end and a yellow color at the other end. The other cylindrical body is configured such that one end exhibits a green color and the other end exhibits a blue color. FIG. 1 shows a color histogram for a portion surrounded by a frame FL in FIG. In FIG. 1, the four colors of the cylinder are displayed as four clusters each extending from the origin, and these clusters correspond to a large number of regions. On the other hand, the boundary parts of the red part and the yellow part, and the green part and the blue part are scattered between the clusters representing the red part and the yellow part and between the clusters representing the green part and the blue part, respectively. Applicable to the area.

According to the present invention, frequency information is extracted from the created color histogram, and the extracted part having a low frequency is detected as an edge. Therefore, there is an advantage that the calculation amount is small, and the processing can be performed by a general-purpose computer without requiring dedicated hardware. Next, the influence of noise in the present invention will be described. Noise in a color image including an edge portion is mainly random noise of a CCD camera or first noise due to fluctuation of illumination, and second noise generated when data is transferred from the CCD camera to a memory of a computer. is there.

5 and 6 relate to the boundary portion shown in FIG.
As for the green color component of GB, the luminance distribution along the x direction similar to FIG. 2 is illustrated. In the figure, points A to G correspond to points A to G in FIG. The broken line in the figure
The error range of the color of each pixel is shown. FIG. 5 shows the error caused by the above-mentioned first noise. On the other hand, FIG. 6 shows an error caused by the above-mentioned second noise.

Focusing on the point D, which is an edge point,
The error ΔNg1 corresponding to the random noise of the CCD camera and the first noise due to the fluctuation of the illumination is from other points A to C.
It occurs in the same manner as the point and the points E to G. On the other hand, CCD
The error ΔNg2 corresponding to the second noise generated when the data is transferred from the camera to the memory of the computer corresponds to the brightness value which should originally be transferred to the memory corresponding to the point C or the point E corresponds to the point D. It is generated by being transferred to the memory, and is large at point D,
It gets smaller as you move away from the point. Therefore, a large error (ΔNg1 + ΔNg2) occurs at point D. The same applies to the red and blue components of RGB.

From the above, the features of the edge portion of the color image are summarized as follows. (1) The number of pixels having color information of the edge portion is smaller than the number of pixels of the entire image, and even if the edge portion has a color change due to noise, it belongs to a small number area. (2) The color range that the pixels in the edge part can take is wider than the color range in the other part, and since many areas on both sides of the edge have similar color information, there is little color change even if there is a change due to noise. .

That is, in the edge portion that belongs to a small number of areas in the color histogram space, the pixel at the edge point has a wide range of color change and changes due to noise.
Very few pixels have the same color. Therefore, according to the present invention, the edge can be detected by detecting the small number area on the color histogram. However, the small number of areas may include pixel noise that should originally exist in the large number of areas, depending on the shooting conditions. Hereinafter, a method for stably detecting this small number area will be described.

As shown in FIG. 7, the color image acquired at time t is f ^(t), and this f ^(t) is plotted on the color histogram space as h ^(t) . This h ^(t)
Has RGB coordinate values and frequency values at the coordinates. In f ^(t) , if the color of which frequency of h ^(t) is only 1 is left and two or more colors are deleted, the image of the portion having the smallest number of regions is obtained. This portion represents an edge portion, but due to the influence of noise, pixels that should originally correspond to many areas are also included.

Therefore, in order to remove the pixels that should correspond to the large number of regions, first, the color value of each pixel of f ^(t) is replaced with the frequency obtained from h ^(t) . As a result, a grayscale screen showing the frequency of the color of each pixel is obtained. Hereinafter, this image is referred to as a frequency image. Letting this frequency image be g ^(t) , the average g _mean of n frequency screens obtained from {f ^(t) , t = 1, 2, ..., N} is expressed by the following equation.

[0020]

[Equation 1] Even if a pixel, which should originally be in a large number of areas on a certain image, is detected as a small number of areas due to color change due to noise, if the data of a plurality of images is arithmetically averaged, the influence of the noise on the pixel is almost zero. It disappears, and this pixel is thereby detected as the original multiple area. On the other hand, when noise is added to the pixels in the minority area, the frequency of the color discolored by the noise in the edge portion is also small, so the frequency of the pixels in that portion remains small, and is therefore detected as the minority area.

When the influence of noise can be ignored, a small number of areas may be detected with only one frequency image. However, as a practical matter, although the above equation is suitable for looking at the distribution of the entire frequency, it may not always be suitable for detecting a small number of areas. Therefore,
The addition result is divided by a value m that is less than or equal to the number of added sheets n, and only the lower bits are used. That is, t = 1 in the frequency image g ^(t)
Let G be the total of ~ n

[0022]

[Equation 2] Since it can be written as
_m is the number of bits of the frequency image g ^(t) is s, and
When 0 <m ≦ n, it can be expressed by the following equation.

G _m = G / m [G / m <2 ^S −1] g _m = 2 ^S −1 [G / m ≧ 2 ^S −1] The dark part in the obtained image g _m is a small area. Represent Based on the above, in order to detect the edge, binarization is performed with a value determined by the combination of m and n. In a normal contrast image, the frequency of pixels at the edge is 2
Since it is often the following, this threshold value is set to n / m + 1 to 2
The edge can be detected by setting the value in the range of n / m and binarizing it.

For example, when the image memory has 8 bits, only the lower 8 bits are used after the addition result is divided by m. Note that when a plurality of frequency images are added, the pixel value may exceed 8 bits even if divided by m. In that case, the maximum value (25
5) is assigned. When m = 1, when the obtained 8-bit grayscale image is binarized with a value of 2n, the 0 pixel portion becomes n.
It corresponds to the minority part where the average frequency is less than 2 during a single image capture.

The above method of dividing the value of G by the value m which is less than or equal to the number n of additions and using the lower bits is an effective means for displaying the frequency information in the image memory with a limited number of bits. The minority region can be detected by directly examining the value of G. For example, in the above detection example, G
The value of may be binarized by 2n.

[0026]

Example An image was captured in 256 gradations of RGB. Therefore, from the above formula, the formula representing the minority region of the histogram addition image is as follows in the range where G / m does not exceed 255 (0xFF).

[0027]

[Equation 3] FIG. 8 shows an example of a color image captured by adding and averaging 36 sheets by natural light from a window in a daytime laboratory.
This image is intended for an LSI product package, and is displayed as a grayscale image in black and white in FIG.

From the same scene, n = 25, m =
The detected image g _m obtained under the condition 1 is shown in FIG. In FIG. 9, the brighter the image, the higher the frequency. If the frequency exceeds 255, it is displayed as 255. For comparison, the result obtained by using the Sobel filter, which is one of the conventional difference methods, for each of the RGB images of FIG.
FIG. 10 shows the result of conversion into the Y signal of the Q color system and the inversion processing. In FIG. 10, the darker the portion, the greater the difference result.

From these results, in the method using the Sobel filter, the magnitude of the difference varies depending on the density difference between the colors on both sides of the edge, so that it is necessary to devise the binarization threshold for the detection result. It has been found that the method of the present invention can detect the edge without being affected by the density difference on both sides of the edge because the small number of frequencies is detected. It was also found that the method using the Sobel filter does not detect a subtle color change portion, whereas the method of the present invention detects it with a little higher frequency and with a spread.

Next, the accuracy of detection was examined. That is, as shown in FIG. 11, an image obtained by averaging 36 sheets of color covers of a magazine (CQ publisher "Transistor Technology" magazine) in a daytime laboratory was obtained. However, here
1 is displayed as a grayscale image in black and white. The illumination at this time was a combination of natural light from the window and light from a fluorescent lamp. Also,
Based on the present invention, for the same scene, n = 25, m
The detection image g _m was obtained under the condition of = 1. This detected image g _m
Is shown in FIG.

FIG. 13 (a) shows an enlarged view of the small characters at the top of FIG. Further, FIG. 13B shows a processing result by the conventional Sobel filter for the same portion. From this FIG. 13, it can be seen that according to the method of the present invention, even fine characters can be clearly detected with pixel accuracy. This is because according to the present invention, particularly when the detection target is a line drawing, not both sides of the line are detected, but the line itself is regarded as a minority region and detected.

Next, based on the present invention, it was confirmed that not only the edge detection but also other information in the image can be detected. As shown in FIG. 14, a color image in which a landscape in the laboratory during the daytime was taken by averaging 36 sheets was obtained. However, here, FIG. 14 is displayed as a grayscale image in black and white. The illumination at this time was the light of the fluorescent lamp and the natural light from the window, and the focus was adjusted to the bird doll in the center of the image. This Figure 1
FIG. 15 shows an image in which four scenes are captured at n = 25 and displayed at m = 4. From FIG. 15, it can be seen that not only the edge but also a curved surface having a large density change is detected.
FIG. 16 shows the same scene captured at n = 25 and displayed at m = 1. In FIG. 16, a blurred region in the rear and a flat surface portion having a small curvature are detected as a large number of regions. On the other hand, the edges, the surface of a bird doll that is a curved surface with a large curvature, the specular reflection area, and the like are a small number of areas.

FIG. 17 shows the result of processing the image of FIG. 14 with the Sobel filter for comparison. Here, as in the case of FIG. 10, it is indicated that the darker part has a larger difference result. By observing the infrequent portion in FIG. 16, that is, the dark portion of the screen, it was found that only the edge information of the in-focus portion in the image can be selectively extracted. That is, FIG. 18A is an image obtained by binarizing the image of FIG. 16 with a threshold value of 1.5n, and only the part of the bird doll in focus is detected. For comparison, FIG. 18B shows an image obtained by performing binarization processing on the image of FIG. 14 at a level at which the doll in the center can be detected. Here, the unfocused portion behind the doll was also detected, and it was impossible to clearly detect which portion of the doll was in focus.

Therefore, according to the present invention, by changing the detection parameters n and m in the above equation, it is possible to extract the macro information of the scene and to detect the in-focus portion finely. It was That is, when an unknown scene was photographed by a camera, it was possible to recognize a region having a high lightness and a large area as a plane in FIG. 15 and a region having a low lightness to a certain extent as a curved surface in FIG. Further, when the low lightness portion is examined, it is possible to detect edge information of a focused portion as shown in FIG. This was based on the fact that a blurred edge has a wider width, while an in-focus edge has a minimum frequency.

It has been found that when the value of m is increased, a gently curved surface, unevenness of shadow and illumination can be detected, and conversely, when the value of m is decreased, a surface or an edge having a large curvature can be detected. It turns out that this is just like the action of the human eye, and it can selectively acquire the necessary information of the scene. It was necessary to set the value of the number of times of image acquisition n according to the noise of the imaging system and the lighting environment. For an image with a low color change frequency, it is sufficient to take in a few times, but conversely, for an image with a large number of colors and a high color change frequency or an image with high contrast, it was necessary to increase the value of n.

[0036]

As described above, according to the present invention, there is an advantage that the amount of calculation is small and processing can be performed by a general-purpose computer without requiring dedicated hardware. Moreover, since it is possible to detect an edge of a portion matching the edges or focus only quickly and stably simple calculation,
The features of the scene can be acquired at high speed, and thus it can be applied to robot vision that requires real-time processing.

Further, since the present invention focuses on a small number of areas on the color histogram, which is an edge portion, FIG. 9 and FIG.
As can be seen from 0, the effect of the edge gradient (differential intensity) is smaller than that of the conventional method using spatial differentiation, and therefore, the boundary portion having a small difference between the two colors can be reliably detected. There is. Moreover, even if there is noise such as illumination fluctuation during image capturing, the frequency of a small number of parts is not affected.

Further, in an actual environment, it is suitable for detecting a character boundary of a color printed matter in which distinction between a small number region and a large number region as shown in FIG. 11 is clear, and has an advantage of high detection resolution. Further, according to the present invention, n color images are acquired for the same scene, and each color image is plotted on the color histogram space, so that n color images each of which has a gray screen showing the frequency of the color of each pixel are displayed. A frequency image is created, the frequency information is detected after adding the n frequency images, and the detected image is obtained by dividing the addition result by a value m that is less than or equal to the number of added images n. Therefore, by changing the detection parameters n and m, it is possible to detect information such as a flat surface or a curved surface other than the edge, or a focused point.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of an RBG color histogram having frequency information according to the present invention.

FIG. 2 is a diagram exemplifying a boundary portion of two colors of red and yellow.

FIG. 3 is a diagram showing an example of a color histogram of the image of FIG.

FIG. 4 is a diagram showing an example of a color image of an actual article.

FIG. 5 is a diagram showing an error due to random noise of a CCD camera or noise due to a change in illumination at a boundary portion in FIG. 2;

FIG. 6 is a diagram showing an error due to noise that occurs when data is transferred to the memory of the computer in the boundary portion of FIG. 2;

FIG. 7 is a diagram for explaining an edge detection procedure according to the present invention.

FIG. 8 is a diagram showing an example of a color image of an LSI product package.

9 is a diagram showing an example of a detected image based on the image of FIG.

10 is a diagram showing a result of processing the image of FIG. 8 by a conventional method.

FIG. 11 is a diagram showing an example of a color image on the cover of a magazine.

FIG. 12 is a diagram showing an example of a detected image based on the image of FIG.

13A and 13B are an enlarged view of a small character in FIG. 12 and an enlarged view showing a result of processing the same portion by a conventional method.

FIG. 14 is a diagram showing an example of a color image of a landscape in the laboratory during the day.

FIG. 15 is a diagram showing an example of a detected image based on the image of FIG.

16 is a diagram showing an example of a detected image when a detection parameter is changed as compared with FIG.

FIG. 17 is a diagram showing a result of processing the image of FIG. 14 by a conventional method.

18A and 18B are a diagram in which a part of the image in FIG. 16 is binarized with a constant threshold value, and a diagram in which the same part in FIG. 14 is binarized by a conventional method.

─────────────────────────────────────────────────── ─── Continuation of the front page (56) References Japanese Unexamined Patent Publication No. 10-31749 (JP, A) Sankar Krishnamur thy, S.M. Sithrama Iy engar, Histogram-Based Morphologic Edge Detector, IEEE Trans. on GEOSCIE NCE AND REMOTE SEN SING, USA, May 1994, Vol. 32 No. 4, p. 759-767 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T ⁷ /00-7/60 H04N 1/48 H04N 1/60 JISST file (JOIS)

Claims (6)

(57) [Claims] 1. Color of each pixel in a color image
The color histogram is plotted on the histogram space.
Extract frequency information for each color plotted in Lamb space
However, if the extracted frequency is lower than other colors,
The location of the color in the color image
Then, the specified position is detected as an edge of the color image or an edge of a focused portion of the color image. 2. The same scene is represented by n, where n is a natural number.
Acquire one color image, and
Then, n frequency images consisting of a gray screen representing the frequency of the color of each pixel are created, the n frequency images are added, and the frequency information is extracted from the addition result. 2. The method for processing a color image according to claim 1, wherein a portion lower than other portions in the image is detected as an edge or an edge of a focused portion in the color image. 3. The same scene is represented by n, where n is a natural number.
Acquire one color image, and
Then, n frequency images consisting of a gray screen showing the frequency of the color of each pixel are created, the n frequency images are added to obtain the addition result G, and m is set to be greater than 0 and equal to or less than n. When the addition result G is divided by m as a certain positive number, and the number of bits of the frequency image is s, g _m = 2 ^m −1 when g / m <2 ^s −1
g / m, and when G / m ≧ 2 ^s −1, g _m = 2 ^s
By setting −1, an image g _m for detection of an edge or in-focus portion is created, frequency information is extracted from this image, and the extracted frequency is higher than that of other portions in the image. 2. The color image processing method according to claim 1, wherein the low portion is detected as an edge or an edge of a focused portion. 4. The same scene is represented by n, where n is a natural number.
Acquires one color image and adds it to each color image.
Plot the color of each pixel on the color histogram space
Each plotted on the color histogram space
Color frequency information is extracted to create n frequency images consisting of a grayscale screen representing the color frequency of each pixel, and the n frequency images are added to obtain an addition result G. Let m be a positive number larger than 0 and not larger than n, and let the addition result G be m
And the bit number of the frequency image is s, G / m <
When 2 ^s −1, g _m = g / m, and G / m ≧ 2 ^s
When -1 is set to g _m = 2 ^s -1, an image for plane detection is created, and a region having a higher frequency and an area than other portions in this image is detected as a plane. Color image processing method. 5. The same scene is represented by n, where n is a natural number.
Acquires one color image and adds it to each color image.
Plot the color of each pixel on the color histogram space
Each plotted on the color histogram space
Color frequency information is extracted to create n frequency images consisting of a grayscale screen representing the color frequency of each pixel, and the n frequency images are added to obtain an addition result G. Let m be a positive number larger than 0 and not larger than n, and let the addition result G be m
And the bit number of the frequency image is s, G / m <
When 2 ^s −1, g _m = g / m, and G / m ≧ 2 ^s
When −1, g _m = 2 ^s −1 is used to create an image for detecting a curved surface, and a region having a lower frequency and an area in this image is detected as a curved surface. Color image processing method. 6. The color of each pixel in a color image is changed to a color.
The color histogram is plotted on the histogram space.
Extract frequency information for each color plotted in Lamb space
However, if the extracted frequency is lower than other colors,
The location of the color in the color image
Then, the specified position is detected as a color boundary of the surface of the in-focus portion of the color image.