TW201106277A - Method of extracting digit image of road signs and vehicle having the same - Google Patents

Abstract

A method of extracting digit image of road signs, and it comprises the following steps: getting road sign images to provide an input signal. Performing road sign image binarization to find image point on the white background area. Detecting the edge of road sign images and separating the white background area of road sign and other patterns. Masking placement by a connected component analysis algorithm to decide the position and size that need to be repaired and to connect the space which produced by image binarization or the separating part between white background area. Fitting ellipse by outline of the road sign image binarization to get an ellipse that is similar to the outline. Extracting the digit of road sign image by the corrected scope of ellipse to binarize the road sign image again and then extracting the digit of road sign image.

Description

201106277, invention description: [Technical field of invention] The present invention relates to a digital image extraction method for road signs, and more particularly to a vehicle equipped with a digital image extraction method for road signs, thereby enabling driving safety assistance system It can automatically interpret the driving speed limit of the road surface and actively provide important information on the driver's road surface to prevent the driver from distracting due to the need to pay attention to too much information, so as to improve the driving safety of the driver. [Prior Art] When driving, drivers must always pay attention to all kinds of information on the road (such as distance, road direction, side and rear vehicles and road signs, etc.), so drivers are easily distracted and accidents occur. Although the vehicle is equipped with car seat belts and airbags to protect the safety of the driver and passengers of the car all the time, these devices do not completely guarantee the safety of the driver. Driving safety should be based on the attitude of “prevention is more important than treatment”. If the accident can be avoided, it will be the most effective protection for drivers and passengers. Therefore, when the driver is driving, if the driving safety assistance system of the vehicle actively provides important information on the road to the driver, the driver can be prevented from being distracted by paying attention to too much information, thereby greatly improving the driver's Driving safety. Since road signs indicate the conditions and specifications of the road, its function is to provide pre-alert and current restrictions to the driver, so when the vehicle can actively obtain road information, it also means that it can provide the driver's road. The conventional technology uses the color limit information of the infrared limit frame, the white background and the black character to reduce the search range of the road sign, and then confirms the position and size of the road sign by using the circular shape condition. When the road sign is recognized, the direct Identify the area of the road sign by taking a rectangular area or a roughly circular area that is detected as a road sign. However, these methods are processed by 201106277^, for example, (1) the camera is photographing the shirt image, (2) the road sign is a perfect circle image, and (3) the road sign image is an upright image without rotation of the angle. Wait. In fact, considering the dual factors of price and cost, general vehicles usually only install grayscale video cameras, and no color information can be used, or color information will change with ambient light, it is not easy to get fruit, or when light changes. In the dark, the color information is not obvious, even the fullness of the = so the prior art method cannot be used in such a case. In addition, as the vehicle progresses, the camera on the car and the road sign on the side of the road will have a large deflection • angle, which makes the road sign image not a perfect circle shape, but an elliptical shape, even A perspective projection produces a deformed elliptical shape. If the method of assuming that the road sign is a perfect circle is not practical, the processing result of f cannot be obtained. Finally, the roadmap images are not necessarily upright images. It may be that the road signs are set up or the camera frame is not fully upright. When the camera is close to the road sign, it will take a road sign image with a rotation angle. Therefore, the assumption that the road sign image is upright will also be inconsistent with the facts, so the processing method with this assumption is usually unable to get a good result. In the prior art, for example, U.S. Patent No. 7,068,844 discloses a correlation operation in the frequency domain to find out which type of road sign is present in that part of the entire image (including location and range). Because there are different types of road signs, and there are different image sizes, if there are κ road signs to be searched, and each road sign has different possible image sizes, the database should record the frequency feature template of one road sign. (Template^ When testing the input, these frequency feature templates are to be compared one by one for each position and size range of the entire image. If the jth size template of the i-th road sign is at the position (x, y When the correlation score is higher than the set threshold, it represents the jth type of road sign of the jth size at the position (x, y). Since the template comparison is quite time, the time is 201106277, and the comparison time is It will grow linearly with the type of road sign and the tolerable size, which is difficult to achieve the speed of immediate processing. Secondly, for example, U.S. Patent No. 7,466,841 discloses a calculation of color-related shape features, and then The Adaboosting algorithm is used to learn the road marker detector. When the road sign is detected, the multiple circular features designed to be integrated are used to determine the center point position of the road sign and Radius size. Then take a circular image of the center position and radius, and then normalize the operation to identify the road sign. This method assumes that the road sign is a perfect circular image, but if the camera and the road sign are not orthogonal angles, Then, the image of the road sign is often presented as an ellipse with an angle (even a small ellipse with a small head and a small end), and this method may not be applicable, resulting in many wrong results. [Invention] In order to solve the above problem, the present invention The main purpose is to provide a digital image extraction method for road signs, which is analyzed for a single channel (such as a combination of gray scale, brightness or color). The road map image can be processed only like an elliptical shape, and there is no limit on the angle rotation. Therefore, it can be applied to digital cameras with speed limit road signs with high practical performance in different weather, different cameras (color or gray scale), and positions where various road signs are actually placed. Another main object of the present invention is to provide A digital image extraction method for road signs, which is directed to a single channel (such as gray scale, brightness or Color combination) image analysis, road sign image can be processed only like an elliptical shape, and there is no limit on angular rotation, so it can be applied to different weather cameras, different cameras (color or gray scale), and various road signs. Position, so that the captured road sign image can be quickly processed to meet the requirements of rapid processing. A further main object of the present invention is to provide a digital image extraction method for road signs, which is directed to a single channel (such as a combination of gray scale, brightness or color) ) Image 201106277 analysis, road sign image can be processed only like an elliptical shape, and there is no limit on angular rotation, so it can be applied to different weather, different cameras (color or gray scale), and the actual placement of various road signs. Therefore, it can solve the problem that the background of the number and the road sign is not obvious or the road sign is too small, and the correct rate of the digital identification can be improved. Another main object of the present invention is to provide a vehicle equipped with a digital image extraction method for signposts, which can enable the driving safety assisting system to automatically interpret the driving speed limit of the road surface, and actively provide important information on the driver's road surface to avoid the driver. Distraction due to the need to pay attention to too much information to improve the driver's driving safety. • (4) The above purpose 'The present invention provides a digital image extraction method for road signs" a vehicle for configuring the method 'This method includes capturing - road sign images, and the road signs are displayed by contrasting moons (such as red box, white background and black characters) The pattern is composed and the road sign image located near the road is captured by the camera device disposed on the vehicle to provide an input road sign image; the binarized road sign image is subjected to image binarization operation to find out possible It is the image point of the white area in the road sign; the edge of the road sign image is detected, the original road sign image is detected by the edge detection to find all the edge points, and the edge point is removed in the binarized road sign image, so that the binary value The road sign image is separated from the bottom area 胄A other patterns, and the rounding operation is performed. The link element analysis is used to find all the links that may be road signs to determine the position and size of the rounds to be used for the road signs. The void formed by the digitally separated image or the disconnected portion between the white and white backgrounds can be connected; the rounding comparison is performed based on the binarized road sign image. The edge contour is used to find the closest circle to the veranda. Then, according to the size and position of the most approximate circle, the correct ellipse that is most likely to be the white background of the road sign is found; the number of the road sign image is cut, and the original road sign image is The broken _ is the range, and then a binarization operation is performed to cut out the digital image in the road sign. 201106277 [Embodiment] The present invention discloses a digital image extraction method for road signs, in particular, a vehicle equipped with a digital image extraction method for road signs, thereby enabling the driving safety assist system to automatically interpret the road speed. Limit, take the initiative to provide important information on the driver's road surface, to avoid the driver's distraction because of the need to pay attention to too much information to improve the driver's driving safety. The detailed fabrication or processing of some of the control wafer encoding methods used in the present invention is accomplished using the prior art and is not fully described in the following description. Moreover, the drawings in the following texts are only intended to represent schematic diagrams relating to the features of the invention. . Referring to Figure 1, there is shown a schematic diagram of one preferred embodiment of a digital image extraction method for landmarks in accordance with the present invention. As shown in Figure 1, the digital image extraction method of the road sign includes the following six steps: Capture a road sign image (100) 'The road sign can be composed of red frame, white background and black characters and configured by The camera on the vehicle captures the roadmap image located near the road to provide an input roadmap image; the digitized roadmap image (102) performs image binarization to identify possible road signs in white The image point of the area; the edge of the side road image (1G4), the original road sign image is found through the edge detection to find all the edge points, and the image value of the edge point relative to the position in the binarized image is set to 0 , so that the white background area after the binarization of the road sign image is separated from other patterns; 'execution of the round operation,' is a connected component analysis to find all the link elements that may be road signs to determine the position of the circle and The size, the hole formed by the number separating the road sign image, or the part between the white background and the bottom _ can be connected. Performing the ellipse comparison (:, based on the binary value The edge contour of the road sign image is obtained - the closest to the porch _, and then the closest ellipse size and 201106277 / the correct ellipse most likely to be the white background of the road sign; word image. Operation to cut out the road sign In the number x, the above six steps are further advanced. The car is placed in a thief, for example: near the road, the paste is placed on the vehicle on the vehicle, and the camera can be positioned and the image analysis target In addition, the present invention can be used; at the same time, the system can be obtained as long as it can obtain a roadmap image, and the position of the camera device is not limited to the position of the vehicle. The road sign image can be used. =, the image of the road sign picked up by the camera device is binarized (10)) The result of the road sign (for example, using the Adaboost algorithm) cannot be used to verify that the road sign must be located. In the center of the area, in order to avoid the background signal = his noise will affect the result of the identification, you need to find out the actual position of the P-marker in the area of the debt-tested area. In the ash 1^ white or child y image, the part of the white area of the road sign is usually the part that belongs to the gray level or the largest brightness value in the whole area, so the area image binarization operation can be used. Find out the image points that may be white areas in the road sign. However, in actual road conditions, 'the road sign image will be seriously affected by light and shadow and show uneven brightness change', so it cannot be binarized with a fixed threshold. . The invention considers the influence of light and shadow 'is to use the Ouzu (〇stu) algorithm to find a suitable threshold to complete the binarization operation, and when the image value is greater than the threshold value, it is judged as 丨, otherwise it is 〇, and because the road sign is usually It will be located near the center of the detected road sign area, and the shape of the road sign is circular. Considering the difference in angle, the shape of the road sign is deformed, so the ash used in the original image is used in the ou. The order or luminance histogram is obtained by giving an elliptical area to the system when detecting the binarization operation, and the central and short axes of the road sign area are proportional to the length and width of the road sign area ( Such as ^ get., and the formula of the long Ouzu algorithm of the circular region is: half length and half width). Above

Pi=fi!N C' = PDI, pt /ω person t, C2 = Pt+l ’ ω2(ί,,...,Pu / 份, t ®.(〇=Σα

i=D υ ω2(〇=Σα /=/+1 t

Mi(0 = Y, i*pi/〇jl(t)

i=D

U only 2 (0 = ^ * Pi /ύ) 2 (〇/=ί+1 μ, = μχ (〇♦ ωχ (〇+ μ2 (ή * ω2 (ί) σ» = ω,(〇(^(〇) -^)2 +(〇2(ίΧμ2(ί)~μγ tbest = argmax0^y(af) , where iV is a m-order color enamel containing pixels or even t/, grayscale gray image The order value 丨 's number, its gray scale value range probability M - threshold value, used to set the gray scale value f of the gray scale image f gt A ^ FI Γ η + - white value knife into two ranges¢ 7丨和:2 ςΜ·., phase grayscale value set, q is the range [set. Called (7) is the range CA probability value nano sum, j fire 1% value and secret (four), the flat material value \(^ = (4) (4) = the average gray scale value of the whole block, 4 relatives also select the benchmark value of the best threshold ', and then set the edge of the road sign image _ (1 () 4); The influence of conditions and factors, such as shadows and surface reflections, etc. = the effect of the image on the binarization, so that on the same road sign, the same = color = 201106277 = but there are different gray value performance, the most common standard The outer frame has a bright point road sign in the white area and the phenomenon of the formation, this - Zhongda # is caused by the phenomenon of reflection The frame part or the background is connected in the _ _ signpost white area and the signpost red signpost, and the 14 results will not be accurately determined

The result of good binarization will lead to errors in the results. In order to rectify the problem, the present invention specifically uses the edge information, and the original white background area and other patterns connected to the driver can be Wei noodle field. The secret "single (four) has the contrasting nature of the driving, _ outside 3 main =, road sign design It will present the material with strong words and bright periods «the case and =:=:::rr: two = r part (four) marked red frame part with C_y edge detection to find all the edge points of the wheel image, judge road signs The result of the binarization is not whether the image point of G is an edge point. If the result of the judgment is the edge point, the point is set to 〇, otherwise it remains unchanged, so that most of the way; ^ in the binarized image The white background area and other areas can be completely disconnected and no longer connected. The present invention then performs a rounding operation (106) which is used in image binarization (102) and binarization to remove edges. The white part of the target may still be split into several blocks, and because there are numbers in the middle of the speed limit roadmap, image binarization (102) may cause the numbers in the road sign to form a void, the above two Kinds of situations will cause the results of image morphology to be Let the part of the road sign be deformed or broken, instead of being shaped like a complete circle. In order to solve such a problem, the present invention uses the "rounding" method, the purpose of which is to achieve the result of binarization and edge removal, the digital The formed voids and the portions of the white and white background that are broken 201106277 can be connected. It should be emphasized here that when the present invention performs the rounding operation (1 〇 6), it uses the method of C〇nnected c〇mp〇nent Analysis to find all the link elements that may be white on the road sign to determine The position and size of the rounding, the invention utilizes the length and width of the connecting element, the position and the relative relationship with the original road marking area, and is designed to filter out the impossible roads that are impossible for the road sign white because the detected road signs are mainly located The original road "domain road sign area" (four) Λ γ λ set (four) money in the original pixel number must be greater than 001 times the original: two knots 'connected element turbulence; go = °. 6 times the total image of the original road sign area = total Γ Points are used to remove the number of element points that are too long and too large. 'The length and width of the other road sign area = the width and length must be less than the rectangular area formed by the G.9 times ^ element. The rounding may be the signpost white background. The length and width of the bottom rectangular area _ circle _ medium, long and short axis is G. 8 times slightly 襟 '

The center of the rectangular area. The center of the circle is the signpost on the white background, and then the circle is connected to the circle, and then the circle is compared (10)) to the heart, and then the closest to the circle is rounded up - the ellipse is the closest to the contour; since the real p position is most likely Road sign white copper:! Shape J learning results: == medium y suitable _ and " etc. ~ most

And in the situation of the solid=road (four) on the car, as the two parts of the car are not all ^: the angle will also change, so the road 4 is fine, to find the road sign "^?1 circle, so need Use the search; ^ = Ming in order to find the appropriate _ in the image morphology; the angle of view and the large morphological operation results, the first 1; 12 201106277 Finally 'to carry out road sign image cutting (10)), the correct ellipse is the range ' Perform a binarization operation again, the original image, ^ image: because the number of the speed limit road sign is cut out by binarization, the number of the road sign numbers may be around - some non-<*,= The result of binarization therefore precedes the result of binarization to find all the links _ ^ number, the noise, the step contains - filter unless the road sign ( word (1) 2) even =, after 'can enter -

It is shown that the filtering element is (4) the weight of the connecting element: as the center point of the white background in the h picture is the center of the circle, the width of the symbol is the square of the road sign / seat = '3, · (9) the number of pixels of the connecting element must be ^之Λ 〇.85 times the total number of pixels in the white background area, and less than #, ·01 times the purpose. Condition (because it belongs to the road sign number; ^ 1 total pixel number _ middle position. The link in the non-signature number = the result of the remaining (four) knot element and the outer _ shape of the entire road marking module. As a test of the actual performance of the method of the present invention, the speed limit road sign data is set up on the car to set up the vehicle _ position, (4) the actual road along the shooting department ^ these video screens use the manual frame selection method, the location of the speed limit road sign and The size is pivoted out. In the experiment of the present invention, there are a total of seven types of speed limit road signs, the speed limit of which is MUuwi, and the total number of samples is M97, wherein the smallest speed limit road sign sample size is 22x22 'The size of the largest speed limit roadmap sample is ΐ4〇·. The third picture shows some manual framed road image samples. It can be seen from the figure that these road signs have great variation in light, size and elevation angle' Moreover, the road signs do not deliberately make accurate frame selections. This is to simulate the error conditions of the road marking range during automatic detection. In terms of identification, the 1497 road signs are divided into two halves, of which 739 speed limit The standard is a sample of the series, as a domain vector machine (Supp (mVee such as heart) 13 201106277: 758 speed limit road signs for training one is the identification effect class used to generate test samples. To identify two (four) questions, in dealing with multiple 〇•, we use each type of support vector machine. Each gamma i 4 problem, you need to establish a support vector machine, and the mth can be The mth class is separated from the other classes, that is, it will distinguish whether the input sample belongs to the mth class. Therefore, for the input sample feature X, it is only necessary to determine which k is the largest output value of the support vector machine. Then, X is identified as category k. According to the ellipse range of the target background found in the road marking number cutting step, the image fire corresponding to the range in the original image or the (four) block 'renormalized into a fixed pixel size of 15χ15 is taken out. The grayscale (or brightness) value of the block size of the element is held to the input characteristic X. 1 is the recognition performance of the support vector machine, where "marked" is the category to which the test sample originally belongs, and the "result" is the test sample. Identified The category result, "all = the total number, the "identification accuracy rate" is positive for a certain category, the recognition rate 'last' is listed as the total number of all test samples, the correct number of identification: identification: rate. The support vector machine can be verified by the following table The identification of the combined feature comparison is good, with a recognition rate of 97.1% up to the south, which shows that the present invention can extract a digital image of the speed limit road sign that is effective for identification. In addition, it is equipped with C〇re2 Due E6400 2.13 GHz and i in terms of speed. 5GB modest system = test. For the 758 speed limit road test, the average digital extraction and recognition time is only about 5 milliseconds per road sign. In the system integration of the 72_ test film, each face The overall processing time for detection and identification also takes about 30 milliseconds to 40 milliseconds. With fast processing speed, it has been able to achieve the efficiency of real-time processing. 0...system 201106277 ~^果\标"^

---Sound milk ____ 辨识 This table supports the recognition performance of the support vector machine - this provides a digital image with a road sign: another - the preferred embodiment is a number of road signs configured: On the vehicle, the digital image recognition of the road sign is set to include the image recognition device including - camera - and the number of road signs === the number of the standard; _; "2 = 3 function" and the number of road signs = image cutting For the sake of detailed description, please refer to the above, and the method of extracting sub-images will not be repeated here. The above is only the preferred scope of the present invention. At the same time, the use of hlong is not limited to the invention. It is true to those who are familiar with the technical field of the technical field and the implementation of the invention. Therefore, the other is not in the spirit of the _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Figure 3 is a schematic diagram of the method. Figure 3 is a sample sample of the manual selection of the road sign. Figure 15 15 201106277 [Description of the main components] 100 Road sign image capture 102 Image binarization 104 Edge detection 106 Rounding operation 108 Ellipse alignment 110 road sign image cutting 112 filtering Number of road signs

Claims (1)

201106277 VII. Patent application scope: 1. A digital image extraction method for road signs, including: taking a road sign image. The road sign is composed of contrasting patterns and will be located near the road via a camera device disposed on the vehicle. Image manipulation to provide an input roadmap image; - Value the road image to perform image binarization to find an image point that may be white on the road sign. Detect the road; ^ the edge of the image, The binarized image of the road sign is used to find all the edge points of the image that are rounded into the image by the edge detection method, and the edge point is removed to make the white area after the binarization of the road sign image and other Pattern separation; "Connected Component ys S" is used to find all the connected elements that may be the road sign to determine the position and size of the rounding to form the number separated by the road sign image. The hollow or the broken portion between the white and white backgrounds is connected; performing the rounding comparison is based on the edge contour of the binarized road sign image - the ellipse closest to the rim, and then find the correct ellipse that is most likely to be the white background of the road sign according to the size and position of the most approximate circle; • a cut the number of the road sign image, according to the correct ellipse Then, perform the - human-valued operation to cut out the digital image of the road sign. 2. If the scope of the patent is claimed! The digital image extraction method of the road sign described in the item, the road sign image is a speed limit road sign. 3. For the digital image extraction method of the road sign described in the third paragraph of the patent (4), the comparison pattern is a pattern consisting of a red frame, a white background and a black character. 4. A method of digital image extraction of a road sign as described in claim 4, which causes the image binarization operation to cause a (four) ancestor (〇stu) algorithm to find a suitable inter-value. 5. If you apply for a patent scope! The digital image extraction method of the road sign described in the item is such that the edge extraction method uses the c_y edge prediction method to find the edge point of the input image. 6. The digital image extraction method of the road sign described in claim 5, wherein the method of removing the edge point is determined by whether the image point whose binarization result is not 0 is an edge point. 7. The digital image extraction method of the road sign described in claim 6 wherein when the image point of the road sign binarization result is not 0, the point is set to 0, otherwise the case is not maintained. change. 8. The digital image extraction method of the road sign as described in claim 1, wherein the number of pixels of the link element of the road sign found by the link element analysis is greater than 0.01 times the total number of pixels of the original road sign. 9. The digital image extraction method of the road sign as described in claim 8 wherein the number of pixels of the link element of the road sign found by the link element analysis is less than 0.6 times the total number of pixels of the original road sign. 10. A digital image extraction method for a road sign as described in claim 1, wherein the link element is found to have a width and a length of less than 0.9 times the length and width of the original road sign. 11. A digital image extraction method for a road sign as described in claim 1 of the patent application, further comprising the step of filtering the linked elements of the road sign number. 12. The digital image extraction method of the road sign according to claim 11 , wherein the condition of the filtering step comprises: the center of gravity of the connecting element is centered on the middle point of the road sign, and the length and width are 0.85 of the road marking area. Within the rectangular range of the multiple; the number of pixels of the connected element must be greater than 0.01 times the total number of pixels in the white area of the road sign, and less than 0.5 times the total number of pixels. 13. A vehicle equipped with a digital image recognition device with a road sign, wherein a digital image recognition device of a road sign is disposed on the vehicle, and the digital image recognition device 201106277 of the road sign includes a camera device and a camera The digital image of the road sign is configured to be connected to the control device, wherein the digital image of the road sign 4= is characterized by: a function for identification, and the road 2 has a method for cutting the digital image of the road sign. Including: ~ Digital image extraction like Detective Jun takes a road sign image, which is composed of a camera mounted on the vehicle and a contrasting pattern to provide an input roadmap image; Road sign image near the road, Binarizing the image of the image, the image point 像~image-valued operation of the white area in the road sign to find out the edge detection method that may detect the road sign image to find all edge points, and The original image of the road sign is removed by the white background 11 after the binarization of the edge image, so that the road sign is subjected to the rounding operation, and the image is separated by: 匕 pattern separation; Analysis) to find out all the possible/information analysis (c) °nnected C ah 0 (10) Set the size of the link element of the ^^ target to determine the fill circle is the space formed by the number separated by the broken image between the white background and the white background (6) The parts are connected; the ellipse is compared to the 'system' to find out - and the outline is the most important to the binarization of the edge of the image of the image of the edge of the image and the position of the ellipse® is most likely to be Recently, the correct rounding and -second binarization of the white background to cut the road sign image is to cut the tomb + 'according to the correct ellipse, and then execute β Μ · as claimed in the 13th The digital image in the road sign. The road sign is limited. The road sign image is a speed 15. The digital image extraction method of the road sign described in the comparatively significant pattern in the thirteenth patent application, the composition of the red frame, the white background and the black character. 201106277 The vehicle of claim 13 wherein the image binarization system uses an Ostu algorithm to find a suitable threshold. 17. The carrier of claim 13 wherein The edge method uses the Cannyit edge detection method to find all the edge points of the input image. A, as for the vehicle described in item 17 (4), wherein the edge point is removed and the road sign is binarized. If the image point of G is an edge point, the vehicle described in claim 18, wherein when the image point of the road sign binarization is not 0, the edge point is Set to 0, and vice versa. For example, the carrier described in claim 13 wherein the number of pixels of the element must be greater than two: 21. As described in the second paragraph of the patent application, the pixel The number of points, the original two or two = 22. As stated in the scope of claim 13 The width and length of the target link element shall be less than = 2 slightly standard number ^ \ = the slave is not allowed to pass - the step includes - unless the 24 = object, the 23rd item H, the towel axis except the step condition is the middle width All the connections it (four) pixels (four) must be greater than the number of G.Q1 times (four) white background area of the 201106277 total number of pixels, and less than 0.5 times the total number of pixels. twenty one