JP5011268B2 - Image detection apparatus and image detection method - Google Patents

Description

  The present invention relates to an image detection device that detects a target image (for example, a circular road sign) having a predetermined shape from an image.

  As a technique for extracting a region expressed in a specific color from an image, there is one disclosed in Patent Document 1. In Patent Document 1, a color to be searched is plotted in a color space from a sample image, an outlier is determined from the distribution of the distribution, and a color extraction table is created using an area excluding the outlier.

There is also a technique called generalized Hough transform, which is a shape recognition technique used in the field of image recognition. For example, Non-Patent Document 1 obtains a relationship between an edge in a detection target pattern and its gradient direction, detects an edge of a search image, and if the edge is a part of the search target pattern, where is the search target A technique for voting to a voting space (position, rotation, size) to determine whether there is a possibility of detecting a pattern based on its peak value is disclosed.
Japanese Laid-Open Patent Publication No. 2006-313468 Dana H. Ballard, "Generalizing the Hough transform to detect arbitrary shapes"

  When edges are simply extracted from the input image and the shape is detected by the generalized Hough transform described in Non-Patent Document 1, many edges other than the search target are generated, so that there is a problem that false detection is likely to occur as a result of voting. Therefore, by extracting a region by paying attention to the color of the road sign before extracting the edge, it is possible to reduce edges other than the road sign and improve performance.

  However, since the color extraction of Patent Document 1 performs color conversion using only the color of one pixel of interest, if there is a pixel with a color similar to a road sign, the entire region including that pixel is extracted. In addition, when the background is complicated, there is a problem in that erroneous detection is likely to occur at a voting value concentration point that occurs by chance.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to perform image detection that realizes high-speed detection processing while reducing the occurrence of erroneous detection when detecting a predetermined target image from an image. To provide an apparatus.

An image detection apparatus according to the present invention is an image detection apparatus that detects a predetermined target image from an image, and includes image input means for inputting an image, and color similarity between each pixel of the input image and pixels in the vicinity thereof Candidate area extracting means for extracting a detection target area from the input image based on the degree and edge intensity of each pixel around the input image, and obtaining an edge direction and edge strength for each pixel included in the extracted detection target area In order to derive an edge extraction means, an area where the target image is highly likely to exist, voting means for voting based on edge direction and edge strength information, and referring to the voting result, a voting value equal to or higher than a threshold value is searched. Voting result evaluation means for detecting a target image included in the input image based on the searched result.

An image detection method according to the present invention is an image detection method for detecting a predetermined target image from an image, and includes a step of inputting an image, a color similarity between each pixel of the input image and pixels in the vicinity thereof, and Extracting a detection target region from the input image based on edge strength of pixels around each pixel of the input image, obtaining an edge direction and edge strength for each pixel included in the extracted detection target region, and the target image Voting based on the edge direction and edge strength information, and referring to the voting result, searching for a voting value that is equal to or greater than the threshold, and based on the searched result Detecting a target image included in the input image.

  According to the present invention, for example, 1) an image in which similar colors are distributed around a certain pixel, such as a road sign, and 2) a clear edge between the target image and its background As for the image, it is possible to reduce the occurrence of edges in regions other than the detection target image, so that it is possible to reduce erroneous detection and to speed up the calculation process. Therefore, when applied to an in-vehicle system, cost reduction effects such as a reduction in the specifications of the microcomputer to be mounted can be obtained. Further, since the processing is fast, the frame rate to be processed becomes fast, and as a result, reliable detection is possible even when the vehicle is moving at high speed.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

  FIG. 1 shows a functional block diagram of a road sign recognition apparatus according to an embodiment of the present invention. The road sign recognition apparatus 10 includes an image capturing unit 11 that captures an image in the vehicle forward direction. The image capturing unit 11 is a visible camera, a near infrared camera, or the like. When photographing an image, the image photographing unit 11 is installed in the direction of the road traveling direction. For example, when mounted on the vehicle, the image capturing unit 11 is installed in the forward direction around the rear mirror of the vehicle, around the grill of the front portion of the vehicle, and the like.

  Further, the road sign recognition device 10 includes an image input unit 12, a candidate marker region extraction unit 13, an edge extraction unit 13, a direction voting unit 15, a voting result evaluation unit 16, and a voting result peak shape evaluation unit 17. With. The functions of these function processing units 12 to 17 are realized by a processor such as a CPU executing a predetermined program.

  The image input unit 12 reads an image captured by the image capturing unit 11. The sign candidate area extraction unit 13 uses the color of each pixel of the input image and the texture information of the vicinity area of each pixel to identify an area that may contain a road sign (hereinafter referred to as “sign candidate area”). Extract.

  In general, road signs are made up of specific colors such as red, blue, and yellow, and the sign candidate area extraction unit 13 extracts these colors. As an extraction method, there is a method in which a large number of sample pixels of a predetermined color in an image are collected, plotted in a color space, an area where the distribution is dense is defined as a color model, and extraction is performed using the model. For example, in the case of a red road sign, a large number of red sample pixels in the image are collected and plotted in a color space, and a dense area of the distribution is defined as a red model of the road sign and extracted using this red model. To do.

  For modeling a densely distributed region, a method of applying a two-dimensional object color probability distribution region (for example, an elliptical region or a normal distribution) or a method of obtaining a region using One-Class SVM (Larry M. Manevitz, “One-Class SVMs for Document Classification”, Journal of Machine Learning Research 2, pp139-154, 2001). However, in these methods, there is a problem that all areas having colors similar to road signs are extracted as sign candidate areas.

  Therefore, in the present embodiment, attention is paid to the fact that the color of the road sign is very similar as long as it is within the same sign, and pixels having very similar colors are adjacent to each other. Then, a candidate marker area is extracted.

Specifically, as shown in FIG. 2, modeling is performed by newly adding the axis of the color difference between the target pixel and surrounding pixels when plotting the sample in the color space in the above-described method. It is carried out. In FIG. 2, for the sake of convenience, only the red (R) axis, the blue (B) axis, and the axis of the difference in color between the target pixel and the peripheral pixel are shown. Is plotted in a four-dimensional space with the green (G) axis added. As the color difference between the target pixel and the peripheral pixels, for example, the following can be considered.
a) A pixel in four directions adjacent to the target pixel in the upper, lower, left, and right directions is a peripheral pixel, the maximum value of the color difference between the target pixel and the peripheral pixel, the average value of the color difference between the target pixel and the peripheral pixel, Median difference in color between each peripheral pixel b) A pixel in a circular or rectangular area centered on the target pixel is a peripheral pixel, and the maximum color difference between the target pixel and the peripheral pixel, Average value of color difference, variance of color difference between target pixel and each peripheral pixel, median value of color difference between target pixel and each peripheral pixel c) Each in a circular or rectangular area centered on the target pixel Difference between the average value of the pixel color value and the color value of the target pixel

  A region where similar colors are distributed in the four-dimensional space as described above (that is, a region where the color difference value is smaller than a predetermined value) is searched, a road sign model is created from the region, and this model is used. To extract a marker candidate region. As a result, it is possible to extract only a region in which pixels of the same color exist together like a road sign and are close to the color of the road sign.

  In the case of a road sign, there is a clear edge between the road sign and the surrounding background. Therefore, paying attention to the presence of an edge between the road sign and the surrounding background, instead of or in addition to the color difference axis between the pixel of interest and the surrounding pixels, pixels around the pixel of interest The edge strength may be modeled by adding it to the color space as a new axis. In this case, a region inside the pixel group whose edge intensity is larger than a predetermined value is searched, and a road sign model is created from the region. As for the edge intensity values of the peripheral pixels, values such as an average value, a variance, and a median value may be obtained as in the case of the color difference from the peripheral pixels.

  Can the sign candidate region extraction unit 13 be a sign candidate for each pixel using the One-Class SVM from the road sign model obtained by plotting in a four-dimensional space including the color difference axis as described above? The marker candidate region is extracted while judging whether or not.

The edge extraction unit 14 obtains an edge direction (edge gradient direction) and an edge strength for each pixel of the marker candidate region extracted by the marker candidate region extraction unit 13. As a method for obtaining the edge direction, there is a method using a Sobel filter. FIG. 3A is a filter (x-direction Sobel filter) for determining the edge strength in the x direction of the image, and FIG. 3B is a filter (y-direction saw) for determining the edge strength in the y direction of the image. Bell filter). If the output of the x-direction Sobel filter is Sx (x, y) and the output of the y-direction Sobel filter is Sy (x, y) in image coordinates (x, y), the edge strength E (x, y) is It is obtained by the formula.

The edge directions θx (x, y) and θy (x, y) are obtained by the following equations.

  For each edge direction, the direction-specific voting unit 15 derives a place where there is a high possibility that a road sign exists using information on the edge strength E (x, y).

  Here, the generalized Hough transform used in the direction voting unit 15 will be described. In the generalized Hough transform, a voting space having the same size as the input image is prepared as a voting space, and the values of all coordinates of the voting space are initially set to zero. When the shape of the detection target is a circle having a radius r, the center of the circle is a point away from the circumferential edge by r in the gradient direction, as shown in FIG. Therefore, when the image coordinate (x, y) has an edge with intensity E (x, y), inclination θx (x, y), θy (x, y), the coordinate in the voting space (x + r x θx (x , Y), y + r × θy (x, y)) is repeated to add the value E (x, y) to increase the voting value of the coordinates of the center of the circle. Therefore, the center of the circle can be found by finding the peak of the vote value.

  When adding the value E (x, y) to the voting space, the coordinates around the coordinates (x + r × θx (x, y), y + r × θy (x, y)) You may make it add a value. As a result, even when the shape of the detection target is distorted or when the size is slightly different, it can be detected. Further, in this case, the voting operation may not be performed when the value E (x, y) is smaller than a certain threshold value. As a result, the processing speed can be increased. Further, at the time of voting, the value of log (E (x, y)) may be added instead of voting the value E (x, y) as it is.

In response to the generalized Hough transform as described above, the direction-specific voting unit 15 divides the edge direction into a plurality of ranges, and prepares a voting space for each range in the edge direction to perform voting. For example, as shown in FIG. 5, the range in the edge direction is divided (eight divisions) from −22.5 degrees to +22.5 degrees, 22.5 degrees to 67.5 degrees,..., 292.5 degrees to 337.5 degrees. The edge direction θ is obtained by the following equation.

  The direction-by-direction voting result evaluation unit 16 searches for a voting value that is equal to or greater than a threshold value in each range in the edge direction based on the voting result in each range in the edge direction, and determines whether there is a circular detection target. That is, a circular detection target is specified in the image.

  For example, if the center of a circular detection target exists at the image coordinates (x, y), the voting result for each direction should have a high voting value in any direction centered on the coordinates (x, y) . Therefore, when a voting value equal to or greater than a certain threshold is obtained for all directions of the voting results by direction, it is determined that there is a circular detection target having a center at that point. On the other hand, if the voting value is high only in some directions and the voting value is low in other directions, there is no circular object centered at that point, and the direction of the coincidence edge due to the influence of a complicated background etc. Can be determined to be a concentrated area. For example, in the example shown in FIG. 6A, since the voting value not less than a certain threshold value is not obtained in all directions, it can be determined that the input image is a combination of edge pieces. On the other hand, in the example shown in FIG. 6B, since the voting value equal to or greater than a certain threshold value is obtained in all directions, it can be determined that the input image is circular.

  Here, when it is desired to detect a circular target, if there is a quadrilateral figure 35 with strong edge strength in the image 30 as in the example shown in FIG. turn into. In the case of the graphic shown in FIG. 7, the voting value for each direction in the horizontal and vertical directions from each side of the quadrangle to the central part is larger than the voting value for each direction in the diagonal direction from each vertex of the quadrangle to the central part. That is, the voting value in each direction varies. Therefore, by detecting the variation between the voting values, it is possible to determine that the voting values are not circular if there is variability between the voting values even when the voting values exceeding a certain threshold value are obtained in all directions. . For example, if the sum of the voting results by direction corresponding to the horizontal and vertical directions is compared with the sum of the voting results by direction corresponding to the diagonal direction, and the ratio of these sums falls within a certain value, it is circular. It can be judged that there is.

  The voting result peak shape evaluation unit 17 determines the peak shape of the voting result even when a circular target is specified by the direction-specific voting result evaluation unit 16, and the peak shape is gentle. The identified circular object is judged to be a false detection and eliminated. In other words, the voting result peak shape evaluation unit 17 determines the peak shape of the voting result, and uses the circular shape image identified by the direction-specific voting result evaluation unit 16 as the final detection target image for the steep peak shape. Detect as. The above-described functions of the direction-specific voting result evaluation unit 16 and the voting result peak shape evaluation unit 17 can suppress erroneous detection due to concentration of voting values that occur accidentally in a complicated background. Hereinafter, this will be described more specifically.

  In the road sign detection device 10, when the background portion is complicated, erroneous detection may occur in the portion. For example, when the background portion is complicated as shown in FIG. 8A, erroneous detection occurs. The distribution of the sum of the voting results in all directions at this time becomes a distribution as shown in FIG. 8B, and the peak shape becomes relatively gentle. On the other hand, the peak shape in the portion of FIG. 9A has a distribution as shown in FIG. Thus, the correct detection target portion tends to exhibit a sharper peak shape than the erroneous detection portion. For this reason, in this embodiment, erroneous detection is reduced by discriminating the sharpness of such a peak shape. That is, when the peak shape sharpness (that is, the slope of the peak shape portion) is smaller than a predetermined value, it is determined that the circular target specified by the direction-specific voting result evaluation unit 16 is erroneously detected. To do. The sharpness of the peak shape is determined by, for example, determining whether the sum of voting values on the circumference separated by a certain distance from the peak portion is above or below the threshold value, or by a certain distance from the peak portion in four directions, up, down, left, and right It can be determined by determining whether the total of the four vote values isolated only above or below the threshold. Alternatively, it can be determined by determining whether the vote value at a point away from the peak portion by a certain distance is above or below the threshold value.

  In the above example, in order to detect a circular figure, it is determined that there is a circular object when a voting value equal to or greater than a certain threshold is obtained for all edge directions in the voting results by direction. The idea of the invention is not limited to searching for circular figures. The edge direction in which a voting value greater than a certain threshold is obtained and the edge direction in which a voting value greater than a certain threshold cannot be obtained are determined according to the shape of the figure. Therefore, it is possible to detect a figure having a specific shape by determining whether or not a vote value equal to or greater than a certain threshold value is obtained in a predetermined edge direction.

  In the above description, the road sign is described as the detection target, but the idea of the present invention is not limited to the detection of the road sign. The idea of the present invention can be applied to a technique for detecting an image having a circular shape or other predetermined shape from an image.

  With the above configuration, erroneous detection of the target image can be reduced, and the processing speed can be increased. As a result, when the system of the present embodiment is applied to an in-vehicle system having road sign detection and identification functions, in addition to improving detection performance, the processing can be reduced in weight, and therefore the cost reduction effect such as reduction in necessary microcomputer specifications can be achieved. Can also be obtained. In addition, since high-speed processing is possible, the frame rate to be processed becomes faster, and as a result, road signs can be reliably detected even when the vehicle is moving at high speed.

Block diagram of a road sign recognition apparatus according to an embodiment of the present invention The figure which shows the example of the plot result of the road sign data in the space which added the axis of the color difference with the circumference to the color space Illustration of Sobel filter Diagram for explaining generalized Hough transform Illustration to explain voting by direction Diagram explaining voting result evaluation by direction when circular target is not detected Diagram explaining voting result evaluation by direction when circular target is detected A diagram explaining an example of false detection The figure which showed the example of voting value distribution of the detection target and the detection target in case of erroneous detection The figure which showed the example of voting value distribution of the detection target and detection target when it can detect correctly

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Road sign recognition apparatus, 11 Image pick-up part, 12 Image input part, 13 Marking candidate area extraction part, 14 Edge extraction part, 15 Voting part according to direction, 16 Voting result evaluation part according to direction, 17 Voting result peak shape evaluation part

Claims (5)

An image detection device for detecting a predetermined target image from an image,
An image input means for inputting an image;
A candidate area extraction means for extracting a detection subject region from the input image based on the edge intensities of the pixels around each pixel of the color similarity and the input image among the pixels of each pixel and its neighboring region of the input image,
Edge extraction means for obtaining an edge direction and an edge strength for each pixel included in the extracted detection target region;
Voting means for voting based on edge direction and edge strength information in order to derive an area where the target image is likely to exist;
An image detection apparatus comprising: a voting result evaluation unit that refers to a voting result, searches for a voting value equal to or greater than a threshold value, and detects a target image included in the input image based on the searched result. The voting means determines a range in the edge direction, performs the voting for each range in the edge direction,
The voting result evaluation means searches for a voting value equal to or greater than a threshold value in each edge direction range from voting results for each edge direction range, and detects a target image included in the input image based on the search result. Item 2. The image detection apparatus according to Item 1.   The image detection apparatus according to claim 1, wherein the detection target image is a road sign.   The image detection apparatus according to claim 1, wherein the input image is an image obtained by capturing an image in front of the vehicle. An image detection method for detecting a predetermined target image from an image,
Inputting an image;
Extracting a detection target region from the input image based on the color similarity between each pixel of the input image and pixels in the vicinity thereof and the edge strength of pixels around each pixel of the input image;
Obtaining an edge direction and an edge strength for each pixel included in the extracted detection target region;
Voting based on edge direction and edge strength information to derive a region where the target image is likely to exist;
An image detection method comprising: referring to a vote result, searching for a vote value that is equal to or greater than a threshold value, and detecting a target image included in the input image based on the search result.