CN106709412B - Traffic sign detection method and device - Google Patents
Traffic sign detection method and device Download PDFInfo
- Publication number
- CN106709412B CN106709412B CN201510797278.0A CN201510797278A CN106709412B CN 106709412 B CN106709412 B CN 106709412B CN 201510797278 A CN201510797278 A CN 201510797278A CN 106709412 B CN106709412 B CN 106709412B
- Authority
- CN
- China
- Prior art keywords
- street view
- view image
- traffic sign
- color
- graph
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 claims abstract description 30
- 238000012216 screening Methods 0.000 claims abstract description 15
- 238000005259 measurement Methods 0.000 claims description 32
- 238000001914 filtration Methods 0.000 claims description 12
- 238000012706 support-vector machine Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 239000003086 colorant Substances 0.000 description 8
- 239000000470 constituent Substances 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 238000003064 k means clustering Methods 0.000 description 4
- 230000000717 retained effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to a traffic sign detection method and a device, wherein the method comprises the following steps: obtaining a street view image; acquiring a candidate area matched with the color value range of the symbolic graph in the traffic sign from the street view image; screening the obtained candidate regions, wherein the screened candidate regions accord with the preset region characteristics of the region where the symbolic graph is located; and after extracting the features of the screened candidate region, judging the extracted features through a classifier for judging whether the candidate region belongs to the landmark graph class or not so as to obtain a traffic sign detection result. The traffic sign detection method and the device provided by the invention can realize automatic traffic sign detection without manual acquisition, and the efficiency and the accuracy are improved.
Description
Technical Field
The invention relates to the technical field of image detection, in particular to a traffic sign detection method and device.
Background
The traffic sign refers to an object which is arranged beside a street and has a traffic warning or traffic prompting function, such as an interval speed measuring sign, a no-passing sign, a road condition prompting sign or a passing direction indicating sign and the like. The position of the traffic sign is marked in the electronic map, so that the method has important significance for the retrieval and navigation of road information.
However, the current sources of information acquisition of traffic signs mainly depend on manual acquisition, but manual acquisition requires a lot of manpower and consumes a lot of time, and is inefficient. And manual acquisition accuracy is poor.
Disclosure of Invention
Therefore, it is necessary to provide a method and an apparatus for detecting a traffic sign, aiming at the problem that the current traffic sign information acquisition source mainly depends on manual acquisition, which results in low efficiency and poor accuracy.
A traffic sign detection method, the method comprising:
obtaining a street view image;
acquiring a candidate area matched with the color value range of the symbolic graph in the traffic sign from the street view image;
screening the obtained candidate regions, wherein the screened candidate regions accord with the preset region characteristics of the region where the symbolic graph is located;
and after extracting the features of the screened candidate region, judging the extracted features through a classifier for judging whether the candidate region belongs to the landmark graph class or not so as to obtain a traffic sign detection result.
A traffic sign detection device, the device comprising:
the street view image acquisition module is used for acquiring a street view image;
the candidate region acquisition module is used for acquiring a candidate region matched with the color value range of the symbolic graph in the traffic sign from the street view image;
the candidate region screening module is used for screening the obtained candidate regions, and the screened candidate regions accord with the preset region characteristics of the region where the symbolic graph is located;
and the judging module is used for judging whether the extracted features belong to the symbolic graph class through a classifier used for judging to obtain a traffic sign detection result after the features of the screened candidate regions are extracted.
According to the traffic sign detection method and device, after the street view image is obtained, the candidate area is obtained according to the color value range of the symbolic graph, so that the area which obviously does not accord with the color characteristics of the traffic sign can be filtered from the street view image. And screening out the candidate region which accords with the preset region characteristics of the region where the symbolic graph is located from the candidate regions, so that the candidate regions which obviously do not accord with the preset region characteristics of the traffic sign can be further filtered. Through the judgment of the classifier, whether the street view image has the landmark graph or not can be finally detected, so that whether the corresponding traffic sign exists or not can be detected. Therefore, automatic traffic sign detection can be realized, manual acquisition is not needed, and the efficiency and the accuracy are improved.
Drawings
FIG. 1 is an internal block diagram of a computer used to implement a traffic sign detection method in one embodiment;
FIG. 2 is a schematic flow chart diagram illustrating a method for traffic sign detection in one embodiment;
FIG. 3 is a diagram illustrating an inter-zone velocity measurement flag according to an embodiment;
FIG. 4 is a schematic diagram of an interval velocity measurement flag according to another embodiment;
FIG. 5 is a schematic flow chart diagram of a traffic sign detection method in another embodiment;
FIG. 6 is a schematic diagram of a horizontal 360 degree panoramic street view image in one embodiment;
FIG. 7 is a schematic diagram illustrating an exemplary contour of an HSV color model interval velocity measurement flag;
FIG. 8 is a diagram illustrating an image at a range velocity marker in a binarized image after connected domain lookup in one embodiment;
FIG. 9 is a schematic illustration of a symbolic representation of a detected traffic sign in one embodiment;
FIG. 10 is a block diagram showing the construction of a traffic sign detecting apparatus according to an embodiment;
fig. 11 is a block diagram showing the structure of a traffic sign detecting apparatus according to another embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in FIG. 1, in one embodiment, a computer 100 is provided that includes a processor, a non-volatile storage medium, an internal memory, and an image collector connected by a system bus. Wherein the processor has a computing function and a function of controlling the operation of the computer 100. The processor is configured to perform a traffic sign detection method. The non-volatile storage medium comprises at least one of a magnetic storage medium, an optical storage medium and a flash memory type storage medium, the non-volatile storage medium stores an operating system and a traffic sign detection device, and the traffic sign detection device is used for realizing a traffic sign detection method. The image collector is used for collecting real-time images, and the image collector can be a 360-degree panoramic camera.
As shown in fig. 2, in one embodiment, a traffic sign detection method is provided, and this embodiment is exemplified by applying the method to the computer 100 in fig. 1. The method specifically comprises the following steps:
Specifically, the computer 100 may call the image collector to obtain the street view image, and the obtained street view image may be referred to as an original street view image. The image collector can be arranged on the movable device, so that street view images can be acquired in real time in the moving process through the movable device. The movable device includes at least one of an automobile, a drone, and a robot. The street view image is an image having an image of the roadside. Roads include highways, including expressways, and pedestrian streets.
In one embodiment, the street view image is a horizontal 360 degree panoramic street view image. Wherein the horizontal 360 degree panoramic street view image is a street view image including a field of view covering 360 degree directions of the horizontal plane. The computer 100 may control the image collector to collect street view images from multiple directions and synthesize the street view images into a 360 street view image, or may control the image collector to collect horizontal rotation and simultaneously acquire street view images and synthesize the street view images to obtain a 360 street view image.
And 204, acquiring a candidate area matched with the color value range of the symbolic graph in the traffic sign from the street view image.
A symbolic graphic refers to a graphic that distinguishes a traffic sign from other traffic signs in the traffic sign. For example, when the traffic sign is a zone speed measurement sign, the symbolic graph is an image that includes a camera image and does not include characters in the zone speed measurement sign, such as 3a1 in fig. 3.
The shapes of the traffic signs mainly comprise rectangles, circles and triangles, so that the shapes of the traffic signs have certain commonality and are not very good in distinction, and the traffic signs are detected by adopting the symbolic graphs in the traffic signs, so that the accuracy is high.
Traffic signs are distinguished from naturally occurring objects in that the color composition of a particular traffic sign is fixed. The specific interval speed measuring mark mainly comprises green and white, some interval speed measuring marks mainly comprise blue and white, and some interval speed measuring marks mainly comprise yellow and black. The interval speed measurement mark shown in fig. 4 includes a symbolic graph 400, where areas 401 and 402 of the symbolic graph 400 are green, and an area 403 is white.
Considering that the color presented by the symbolic graph changes due to the change of light in the natural environment, the color presented by the symbolic graph under various light conditions can be counted in advance, so that the value range of the color can be determined according to the counting result. The color range may be divided according to color channels under a specific color model, for example, if the street view image adopts an HSV color model, the color range may include a Value range of at least one of three color channels, i.e., H (Hue), S (Saturation), and V (brightness). The color model may also adopt an RGB (red green blue) color model or a YUV color model, etc.
And step 206, screening the obtained candidate regions, wherein the screened candidate regions accord with the preset region characteristics of the region where the symbolic graph is located.
Specifically, the candidate regions obtained after the color filtering step still include more candidate regions irrelevant to the traffic sign, and here, the candidate regions that do not meet the preset region feature are filtered out by filtering the obtained candidate regions, so as to obtain the candidate regions meeting the preset region feature. The preset regional characteristics are preset regional characteristics which the region where the symbolic graph is located should have.
And step 208, after extracting the features of the screened candidate regions, judging the extracted features through a classifier for judging whether the candidate regions belong to the landmark graph class or not so as to obtain a traffic sign detection result.
Specifically, the classifier is trained over a set of positive samples belonging to the signature graph class and a set of negative samples not belonging to the signature graph class. Wherein positive samples in the positive sample set are images that include a signature graphic; the negative examples in the negative example set are images that do not include the signature graphic, and the negative examples may include random images that are not related to the signature graphic or images that are similar to the signature graphic.
When the classifier is trained, after the features of the positive samples in the positive sample set and the features of the negative samples in the negative sample set are respectively extracted, the classifier is trained according to the extracted features. The trained classifier can be used for predicting whether a new image belongs to the symbolic graph class, and if the new image belongs to the symbolic graph class, the traffic sign is detected; if the detected traffic sign does not belong to the landmark graph class, the traffic sign is not detected. The traffic sign detection result includes whether the traffic sign is detected or not, and also includes the position of the detected traffic sign in the street view image.
The extracted Features may be HOG (Histogram of Oriented Gradient) Features, SIFT (Scale-Invariant Feature Transform) Features, SURF (Speeded Up robust Features) Features, or the like. The classifier may be an SVM (Support Vector Machine), a cascade classifier (e.g., Adaboost classifier), or an artificial neural network classifier.
According to the traffic sign detection method, after the street view image is obtained, the candidate area is obtained according to the color value range of the symbolic graph, so that the area obviously not conforming to the color characteristics of the traffic sign can be filtered from the street view image. And screening out the candidate region which accords with the preset region characteristics of the region where the symbolic graph is located from the candidate regions, so that the candidate regions which obviously do not accord with the preset region characteristics of the traffic sign can be further filtered. Through the judgment of the classifier, whether the street view image has the landmark graph or not can be finally detected, so that whether the corresponding traffic sign exists or not can be detected. Therefore, automatic traffic sign detection can be realized, manual acquisition is not needed, and the efficiency and the accuracy are improved.
In one embodiment, the computer 100 may further obtain geographic location information of the street view image, and mark a corresponding traffic sign on the electronic map according to the detected location of the traffic sign in the street view image and the geographic location information. Wherein the geographical location information of the street view image can be obtained when the street view image is obtained.
In one embodiment, the computer 100 may also perform warning prompt according to the detection result of the traffic sign. For example, a prompt of 'speed measurement ahead' is sent after the interval speed measurement mark is detected.
In one embodiment, before step 204, the method further includes: and cutting the street view image according to the statistical information of the appearance position of the traffic sign in the pre-acquired street view image.
Specifically, the pre-captured street view image refers to a pre-captured street view image, and corresponding statistical information is obtained by counting the occurrence positions of the traffic signs in the pre-captured street view image. The statistical information may reflect where the traffic sign does not appear in the street view image or may appear in the street view image.
For example, the street view image may include an image of the sky and an image of the road, the image of the sky is usually located in an upper region of the street view image, and the image of the road is located in a lower region of the street view image, and the regions may be cut out without traffic signs.
In this embodiment, the street view image is cut according to the statistical information of the appearance position of the traffic sign in the pre-acquired street view image, and the area where the traffic sign is unlikely to appear can be directly abandoned at the front end of the process, so that the efficiency of subsequent processing is improved.
In one embodiment, step 204 includes: setting a pixel value matched with a color value range of a symbolic graph in a traffic sign in a street view image as a foreground color value, and setting a pixel value not matched with the color value range as a background color value to obtain a binary image; and searching a connected domain formed by foreground color values in the binary image to acquire a candidate region.
And searching a connected domain of the binary image to obtain a connected domain formed by pixels of the foreground color values, and taking the connected domain as a candidate region or taking the pixels of the connected domain corresponding to the original background image as the candidate region.
Further, the foreground color value may be 0, visually appearing black; the background color value may be 255 visually appearing as white. The candidate area is matched with the color value range, which means that the pixel value corresponding to the candidate area in the street view image is in the color value range.
The connected domain refers to an image region which is formed by foreground pixel points with the same pixel value and adjacent positions in an image. Connected component domain finding is a process of finding and labeling connected components in an image, also referred to as connected component domain analysis. The connected component search can adopt Two-Pass scanning algorithm and Seed Filling algorithm. After the connected domain is found, the angle of the connected domain can be adjusted, so that the angle is normalized. For example, the angle is adjusted to the range of [ -90 °, 90 ° ].
In one embodiment, step 204 includes: and acquiring a candidate area matched with a preset color value range of one component color of the symbolic graph in the traffic sign from the street view image. The one component color may be the largest area percentage color of the markered graphic. For example, when the traffic sign is an interval speed measurement sign, the color value range can be that the color of the camera image is green. The candidate region is obtained by a color range of the constituent colors, which can improve efficiency and retain edge information.
In one embodiment, step 204 includes: and acquiring at least one group of matched candidate regions in a plurality of groups of preset color value ranges of one component color of the symbolic graph in the traffic sign from the street view image.
Specifically, considering that the outdoor lighting conditions have large differences and the color range has large changes, a plurality of groups of preset color range are preset for one color, a certain area can be retained as a candidate area as long as one group of preset color range is hit, and the area is filtered if any group of preset color range is not met.
For example, if the traffic sign is an interval speed measurement sign and one of the constituent colors of the symbolic graph is green, the value ranges of the multiple groups of preset colors of the one of the constituent colors of the symbolic graph in the traffic sign may be listed as follows:
a first group: 50< H <100, S >55, 75< V < 160.
Second group: 50< H <100, S >50, V < 70.
Third group: 60< H <90, S >35, 170< V < 220.
And a fourth group: 28< H <100, 110< V < 178.
And a fifth group: 20< H <110, 130< S <230, 30< V < 60.
Where H, S and V represent the pixel values in the hue color channel, the saturation color channel, and the brightness color channel, respectively.
In this embodiment, through setting up the multiunit that constitutes the colour and predetermine the colour value range, can ensure that the colour filters the step and has higher recall rate to the detection of traffic sign, prevents to miss examining. The recall rate is also called recall rate, which is the ratio of the number of detected traffic signs to the number of actually existing traffic signs.
In one embodiment, the preset region features include: the size range, the height-width ratio range, the color cluster category number and the gradient feature of the region where the symbolic graph is located.
The size range of the region where the landmark pattern is located refers to the range of the width and the range of the height of the circumscribed rectangle of the region where the landmark pattern is located. For example, if the traffic sign is an interval speed measurement sign and the size of the obtained street view image is 8192 pixels (width) × 4096 pixels (height), the size range of the area where the interval speed measurement sign is located may be: 14 pixels < height <231 pixels, 10 pixels < width <600 pixels; wherein height represents the height of the area where the interval speed measuring mark is located, and width represents the width of the area where the interval speed measuring mark is located.
The aspect ratio range refers to a range of the ratio of the height to the width of a circumscribed rectangle of the region where the landmark pattern is located. For example, if the traffic sign is an interval speed measurement sign, the aspect ratio range of the area where the interval speed measurement sign is located may be: 1.0< height/width < 5.2; wherein height represents the height of the area where the interval speed measuring mark is located, and width represents the width of the area where the interval speed measuring mark is located.
The color clustering class number refers to the number of classes obtained by clustering the pixel values of the region where the landmark graph is located. The clustering can adopt a K-means clustering algorithm, and can also adopt K-modes and other clustering algorithms, which are not listed one by one.
For example, if the traffic sign is an interval speed measurement sign, a K-means clustering algorithm can be adopted, and K pixels are selected from the acquired candidate area at will to serve as an initial clustering center; and for the rest other pixels, respectively allocating the other pixels to the cluster represented by the cluster center closest to the other pixels according to the pixel value distances between the other pixels and the cluster centers; then calculating the clustering center of each obtained new cluster; this process is repeated until the standard measure function begins to converge. If the two types of the candidate regions can be clustered finally, the corresponding candidate regions are reserved; and if the two types cannot be clustered finally, filtering out the corresponding candidate regions. Wherein the pixel value distance can be calculated by Euclidean distance. The standard measure function may take the mean square error.
If an image is regarded as a two-dimensional discrete function, the gradient of the image is the derivative of the two-dimensional discrete function, the gradient can reflect the edge of the image, and the gradient feature can reflect the edge feature in the image. For example, if the traffic sign is an interval speed measurement sign, the traffic sign is easily confused with green leaves because the traffic sign includes a large number of green areas. However, the edges of the landmark graphics of the traffic sign are regular and less in number, and the edges of the leaves are disordered and more in number, so that the landmark graphics and the leaves can be distinguished by the gradient feature capable of reflecting the change of the transformation.
In the embodiment, the obtained candidate regions can be filtered from multiple dimensions of the region size, the aspect ratio, the color category and the gradient feature, the candidate regions which do not obviously accord with the preset region feature are filtered, the candidate regions in which the symbolic graphs may exist are reserved, and the subsequent distinguishing efficiency through the classifier is improved conveniently. And through the combination of the preset region characteristics of multiple dimensions, the subsequent discrimination efficiency through the classifier can be further improved.
As shown in fig. 5, in an embodiment, a method for detecting a traffic sign is provided, where the traffic sign is an inter-zone speed measurement sign in this embodiment. The method specifically comprises the following steps:
And step 504, according to the statistical information of the appearance positions of the traffic signs in the pre-acquired horizontal 360-degree panoramic street view image, cutting the acquired horizontal 360-degree panoramic street view image. Referring to fig. 6, both region 601 and region 602 of the horizontal 360 degree panoramic street view image may be cropped.
a first group: 50< H <100, S >55, 75< V < 160.
Second group: 50< H <100, S >50, V < 70.
Third group: 60< H <90, S >35, 170< V < 220.
And a fourth group: 28< H <100, 110< V < 178.
And a fifth group: 20< H <110, 130< S <230, 30< V < 60.
And step 508, setting the pixel values detected as matched in the horizontal 360-degree panoramic street view image as foreground color values, and setting the pixel values detected as unmatched in the horizontal 360-degree panoramic street view image as background color values, so as to obtain a binary image.
And 510, searching a connected domain of the binary image to obtain a candidate region. Specifically, the image at the interval speed measurement mark in the binarized image after the connected domain search is shown in fig. 8. The steps 506 to 510 are steps of obtaining, from the street view image, at least one candidate region matching at least one of the multiple sets of preset color value ranges of green of the symbolic graphic in the traffic sign under the HSV color model.
And step 512, filtering out candidate regions which do not accord with any one of the size range, the height-width ratio range, the color cluster category number and the gradient feature of the region where the symbolic graph is located from the obtained candidate regions.
Where the size range may be 14 pixels < height <231 pixels, 10 pixels < width <600 pixels. The aspect ratio range may be 1.0< height/width < 5.2. Wherein height represents the height of the area where the interval speed measuring mark is located, and width represents the width of the area where the interval speed measuring mark is located. The number of color cluster categories may take 2 categories. The gradient feature here is mainly a gradient feature that is distinguished from the gradient feature of leaves.
And 514, after the HOG features are extracted from the candidate areas reserved after filtering, judging the extracted HOG features through a support vector machine for judging whether the extracted HOG features belong to the landmark graph class or not so as to obtain a traffic sign detection result. The final detected traffic sign has a signature pattern 9a and 9b as shown in fig. 9.
As shown in fig. 10, in one embodiment, a traffic sign detecting apparatus 1000 is provided, which has functional modules for implementing the traffic sign detecting methods of the above-described embodiments. The traffic sign detecting device 1000 includes: a street view image acquisition module 1001, a candidate region acquisition module 1002, a candidate region screening module 1003, and a discrimination module 1004.
A street view image obtaining module 1001 configured to obtain a street view image.
Specifically, the street view image capturing module 1001 may be configured to call an image collector to capture a street view image, and the captured street view image may be referred to as an original street view image. The image collector can be arranged on the movable device, so that street view images can be acquired in real time in the moving process through the movable device. The movable device includes at least one of an automobile, a drone, and a robot. The street view image is an image having an image of the roadside. Roads include highways, including expressways, and pedestrian streets.
In one embodiment, the street view image is a horizontal 360 degree panoramic street view image. Wherein the horizontal 360 degree panoramic street view image is a street view image including a field of view covering 360 degree directions of the horizontal plane. The street view image acquiring module 1001 may be configured to control the image collector to collect street view images in multiple directions and synthesize the street view images into a 360 street view image, and may also control the image collector to collect horizontal rotation and simultaneously acquire and synthesize the street view images to obtain the 360 street view image.
The candidate region acquiring module 1002 is configured to acquire a candidate region matching a color value range of a landmark graph in a traffic sign from a street view image.
The traffic sign refers to an artificial object which is arranged beside a street and has a traffic warning or traffic prompting function, such as an interval speed measuring sign, a no-passing sign, a road condition prompting sign or a passing direction indicating sign. A symbolic graphic refers to a graphic that distinguishes a traffic sign from other traffic signs in the traffic sign. For example, when the traffic sign is a zone speed measurement sign, the symbolic graph may be an image including a camera image and not including text in the zone speed measurement sign, such as 3a1 in fig. 3.
The shapes of the traffic signs mainly comprise rectangles, circles and triangles, so that the shapes of the traffic signs have certain commonality and are not very good in distinction, and the traffic signs are detected by adopting the symbolic graphs in the traffic signs, so that the accuracy is high.
Traffic signs are distinguished from naturally occurring objects in that the color composition of a particular traffic sign is fixed. The specific interval speed measuring mark mainly comprises green and white, and some interval speed measuring marks mainly comprise blue and white. As shown in fig. 4, the region 401 is green, and the region 402 is white.
Considering that the color presented by the symbolic graph changes due to the change of light in the natural environment, the color presented by the symbolic graph under various light conditions can be counted in advance, so that the value range of the color can be determined according to the counting result. The color range may be divided according to color channels under a specific color model, for example, if the street view image adopts an HSV color model, the color range may include a Value range of at least one of three color channels, i.e., H (Hue), S (Saturation), and V (brightness). The color model may also adopt an RGB (red green blue) color model or a YUV color model, etc.
And a candidate region screening module 1003, configured to screen the obtained candidate region, where the screened candidate region meets a preset region characteristic of a region where the symbolic graph is located.
Specifically, the candidate regions obtained after color filtering still include more candidate regions irrelevant to the traffic sign, and here, the candidate regions that do not meet the preset region feature are filtered out by filtering the obtained candidate regions, so that the candidate regions meeting the preset region feature are obtained. The preset regional characteristics are regional characteristics which the region where the preset symbolic graph is located should have.
And the judging module 1004 is configured to, after extracting features from the screened candidate regions, judge the extracted features through a classifier for judging whether the extracted features belong to a landmark graph class, so as to obtain a traffic sign detection result.
Specifically, the classifier is trained over a set of positive samples belonging to the signature graph class and a set of negative samples not belonging to the signature graph class. Wherein positive samples in the positive sample set are images that include a signature graphic; the negative examples in the negative example set are images that do not include the signature graphic, and the negative examples may include random images that are not related to the signature graphic or images that are similar to the signature graphic.
When the classifier is trained, after the features of the positive samples in the positive sample set and the features of the negative samples in the negative sample set are respectively extracted, the classifier is trained according to the extracted features. The trained classifier can be used for predicting whether a new image belongs to the symbolic graph class, and if the new image belongs to the symbolic graph class, the traffic sign is detected; if the detected traffic sign does not belong to the landmark graph class, the traffic sign is not detected. The traffic sign detection result includes whether the traffic sign is detected or not, and also includes the position of the detected traffic sign in the street view image. The extracted features may adopt HOG features, SIFT features or SURF features. The classifier can adopt a support vector machine, a cascade classifier or an artificial neural network classifier and the like.
After the street view image is obtained, the traffic sign detection device 1000 obtains the candidate region according to the color value range of the symbolic graph, so that the region obviously not conforming to the color feature of the traffic sign can be filtered from the street view image. And screening out the candidate region which accords with the preset region characteristics of the region where the symbolic graph is located from the candidate regions, so that the candidate regions which obviously do not accord with the preset region characteristics of the traffic sign can be further filtered. Through the judgment of the classifier, whether the street view image has the landmark graph or not can be finally detected, so that whether the corresponding traffic sign exists or not can be detected. Therefore, automatic traffic sign detection can be realized, manual acquisition is not needed, and the efficiency and the accuracy are improved.
As shown in fig. 11, in one embodiment, the traffic sign detecting device 1000 further includes: the cropping module 1005 is configured to crop the street view image according to the statistical information of the appearance position of the traffic sign in the pre-acquired street view image.
Specifically, the pre-captured street view image refers to a pre-captured street view image, and corresponding statistical information is obtained by counting the occurrence positions of the traffic signs in the pre-captured street view image. The statistical information may reflect where the traffic sign does not appear in the street view image or may appear in the street view image.
For example, the street view image may include an image of the sky and an image of the road, the image of the sky is usually located in an upper region of the street view image, and the image of the road is located in a lower region of the street view image, and the regions may be cut out without traffic signs.
In this embodiment, the street view image is cut according to the statistical information of the appearance position of the traffic sign in the pre-acquired street view image, and the area where the traffic sign is unlikely to appear can be directly abandoned at the front end of the process, so that the efficiency of subsequent processing is improved.
In an embodiment, the candidate region obtaining module 1002 is further configured to set a pixel value in the street view image, which is matched with a color value range of a landmark graph in the traffic sign, as a foreground color value, and set a pixel value, which is not matched with the color value range, as a background color value, so as to obtain a binary image. And then searching a connected domain of the binary image to obtain a connected domain formed by pixels of the foreground color values, wherein the pixels of the connected domain corresponding to the original background image can be used as candidate regions.
Wherein the foreground color value may be 0, visually appearing black; the background color value may be 255 visually appearing as white. The candidate area is matched with the color value range, which means that the pixel value corresponding to the candidate area in the street view image is in the color value range.
The connected domain refers to an image region which is formed by foreground pixel points with the same pixel value and adjacent positions in an image. Connected component domain finding is a process of finding and labeling connected components in an image, also referred to as connected component domain analysis. The connected component search can adopt Two-Pass scanning algorithm and Seed Filling algorithm.
In one embodiment, the candidate region obtaining module 1002 is further configured to obtain a candidate region from the street view image, where the candidate region matches a preset color value range of one component color of the landmark graph in the traffic sign. The one component color may be the largest area percentage color of the markered graphic. For example, when the traffic sign is an interval speed measurement sign, the color value range can be that the color of the camera image is green. The candidate region is obtained by a color range of the constituent colors, which can improve efficiency and retain edge information.
In one embodiment, the candidate region obtaining module 1002 is further configured to obtain, from the street view image, a candidate region that matches at least one of a plurality of groups of preset color value ranges of a component color of a landmark graph in the traffic sign.
Specifically, considering that the outdoor lighting conditions have large differences and the color range has large changes, a plurality of groups of preset color range are preset for one color, a certain area can be retained as a candidate area as long as one group of preset color range is hit, and the area is filtered if any group of preset color range is not met.
For example, if the traffic sign is an interval speed measurement sign and one of the constituent colors of the symbolic graph is green, the value ranges of the multiple groups of preset colors of the one of the constituent colors of the symbolic graph in the traffic sign may be listed as follows:
a first group: 50< H <100, S >55, 75< V < 160.
Second group: 50< H <100, S >50, V < 70.
Third group: 60< H <90, S >35, 170< V < 220.
And a fourth group: 28< H <100, 110< V < 178.
And a fifth group: 20< H <110, 130< S <230, 30< V < 60.
Where H, S and V represent the pixel values in the hue color channel, the saturation color channel, and the brightness color channel, respectively.
In this embodiment, through setting up the multiunit that constitutes the colour and predetermine the colour value range, can ensure that the colour filters the step and has higher recall rate to the detection of traffic sign, prevents to miss examining. The recall rate is also called recall rate, which is the ratio of the number of detected traffic signs to the number of actually existing traffic signs.
In one embodiment, the preset region features include: the size range, the height-width ratio range, the color cluster category number and the gradient feature of the region where the symbolic graph is located.
The size range of the region where the landmark pattern is located refers to the range of the width and the range of the height of the circumscribed rectangle of the region where the landmark pattern is located. For example, if the traffic sign is an interval speed measurement sign and the size of the obtained street view image is 8192 pixels (width) × 4096 pixels (height), the size range of the area where the interval speed measurement sign is located may be: 14 pixels < height <231 pixels, 10 pixels < width <600 pixels; wherein height represents the height of the area where the interval speed measuring mark is located, and width represents the width of the area where the interval speed measuring mark is located.
The aspect ratio range refers to a range of the ratio of the height to the width of a circumscribed rectangle of the region where the landmark pattern is located. For example, if the traffic sign is an interval speed measurement sign, the aspect ratio range of the area where the interval speed measurement sign is located may be: 1.0< height/width < 5.2; wherein height represents the height of the area where the interval speed measuring mark is located, and width represents the width of the area where the interval speed measuring mark is located.
The color clustering class number refers to the number of classes obtained by clustering the pixel values of the region where the landmark graph is located. The clustering can adopt a K-means clustering algorithm, and can also adopt K-modes and other clustering algorithms, which are not listed one by one.
For example, if the traffic sign is an interval speed measurement sign, a K-means clustering algorithm can be adopted, and K pixels are selected from the acquired candidate area at will to serve as an initial clustering center; and for the rest other pixels, respectively allocating the other pixels to the cluster represented by the cluster center closest to the other pixels according to the pixel value distances between the other pixels and the cluster centers; then calculating the clustering center of each obtained new cluster; this process is repeated until the standard measure function begins to converge. If the two types of the candidate regions can be clustered finally, the corresponding candidate regions are reserved; and if the two types cannot be clustered finally, filtering out the corresponding candidate regions. Wherein the pixel value distance can be calculated by Euclidean distance. The standard measure function may take the mean square error.
If an image is regarded as a two-dimensional discrete function, the gradient of the image is the derivative of the two-dimensional discrete function, the gradient can reflect the edge of the image, and the gradient feature can reflect the edge feature in the image. For example, if the traffic sign is an interval speed measurement sign, the traffic sign is easily confused with green leaves because the traffic sign includes a large number of green areas. However, the edges of the landmark graphics of the traffic sign are regular and less in number, and the edges of the leaves are disordered and more in number, so that the landmark graphics and the leaves can be distinguished by the gradient feature capable of reflecting the change of the transformation.
In the embodiment, the obtained candidate regions can be filtered from multiple dimensions of the region size, the aspect ratio, the color category and the gradient feature, the candidate regions which do not obviously accord with the preset region feature are filtered, the candidate regions in which the symbolic graphs may exist are reserved, and the subsequent distinguishing efficiency through the classifier is improved conveniently. And through the combination of the preset region characteristics of multiple dimensions, the subsequent discrimination efficiency through the classifier can be further improved.
In one embodiment, the street view image capture module 1001 is further configured to capture a horizontal 360 degree panoramic street view image.
The cropping module 1005 is further configured to crop the horizontal 360-degree panoramic street view image according to statistical information of the occurrence positions of the traffic signs in the pre-acquired horizontal 360-degree panoramic street view image.
The candidate region obtaining module 1002 is further configured to obtain, from the street view image, a candidate region that matches at least one group of multiple groups of preset color value ranges of green of a symbolic graphic in a traffic sign under the HSV color model. The candidate region obtaining module 1002 is specifically configured to detect whether each pixel value in each color channel of the horizontal 360-degree panoramic street view image matches at least one group of multiple groups of preset color value ranges of green of the symbolic graph in the traffic sign, under the HSV color model. And setting the pixel value detected as matched in the horizontal 360-degree panoramic street view image as a foreground color value, and setting the pixel value detected as unmatched in the horizontal 360-degree panoramic street view image as a background color value to obtain a binary image. And searching the connected domain of the binary image, and taking the searched connected domain as a candidate region.
The candidate region screening module 1003 is further configured to filter out candidate regions from the candidate regions, where the candidate regions do not conform to the size range, the aspect ratio range, the number of color cluster categories, and the gradient feature of the region where the landmark graph is located.
The judging module 1004 is further configured to, after extracting the HOG features from the candidate regions retained after filtering, judge the extracted HOG features by using a support vector machine for judging whether the extracted HOG features belong to a landmark graph class, so as to obtain a traffic sign detection result.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A traffic sign detection method, the method comprising:
obtaining a street view image and geographical position information of the street view image;
under an HSV color model, setting at least one group of matched pixel values in a plurality of groups of preset color value ranges of one color forming a symbolic graph in a traffic sign in the street view image as foreground color values, and setting pixel values not matched with the plurality of groups of preset color value ranges as background color values to obtain a binary image; the traffic sign is an interval speed measuring sign; the symbolic graph is a graph which includes a camera pattern and does not include characters in the interval speed measurement mark; one component color of the symbolic graph is the color of the camera pattern;
searching a connected domain formed by foreground color values in the binary image, and adjusting the angle of the searched connected domain to standardize the angle so as to obtain a candidate region;
screening the obtained candidate regions, wherein the screened candidate regions accord with the preset region characteristics of the region where the symbolic graph is located; the preset area features include: at least one of the size range, the height-width ratio range, the color clustering category number and the gradient feature of the region where the symbolic graph is located; the color clustering category number is a category number obtained by clustering the pixel values of the area where the symbolic graph is located; the gradient feature is used for characterizing the edge feature of the symbolic graph;
after extracting the features of the screened candidate region, judging the extracted features through a classifier for judging whether the candidate region belongs to a landmark graph class or not so as to obtain a traffic sign detection result;
and when the traffic sign detection result is that the interval speed measuring sign is detected, prompting and early warning are carried out, the position of the interval speed measuring sign in the street view image is detected, and the corresponding interval speed measuring sign is marked on the electronic map according to the position and the geographic position information of the street view image.
2. The method according to claim 1, wherein before obtaining the binarized image, the method further comprises, under an HSV color model, setting, as foreground color values, pixel values in the street view image that match at least one of a plurality of preset color value ranges of one component color of a symbolic graphic in a traffic sign, and setting, as background color values, pixel values that do not match the plurality of preset color value ranges, before obtaining the binarized image:
and cutting the street view image according to the statistical information of the appearance position of the traffic sign in the pre-acquired street view image.
3. The method of claim 1, wherein the street view image is a horizontal 360 degree panoramic street view image;
after extracting the features from the screened candidate region, the extracted features are discriminated by a classifier for discriminating whether the extracted features belong to a landmark graph class or not to obtain a traffic sign detection result, which includes:
and after the HOG features are extracted from the screened candidate regions, judging the extracted HOG features through a support vector machine for judging whether the extracted HOG features belong to the landmark graph class or not so as to obtain a traffic sign detection result.
4. The method according to any one of claims 1 to 3, wherein the step of screening the obtained candidate regions, wherein the screened candidate regions conform to the preset region characteristics of the region where the symbolic graph is located, comprises:
and filtering out candidate regions which do not accord with any one of the size range, the aspect ratio range, the color clustering class number and the gradient characteristic of the region where the symbolic graph is located from the obtained candidate regions.
5. A traffic sign detection device, characterized in that said device comprises:
the street view image acquisition module is used for acquiring a street view image and the geographical position information of the street view image;
a candidate area obtaining module, configured to, in an HSV color model, set, as a foreground color value, at least one group of matched pixel values in a multiple group of preset color value ranges of a color that constitutes a landmark pattern in a traffic sign in the street view image, set, as a background color value, a binarized image, search a connected domain formed by the foreground color values in the binarized image, and adjust an angle of the searched connected domain, so that the angle is normalized, so as to obtain a candidate area; the traffic sign is an interval speed measuring sign; the symbolic graph is a graph which includes a camera pattern and does not include characters in the interval speed measurement mark; one component color of the symbolic graph is the color of the camera pattern;
the candidate region screening module is used for screening the obtained candidate regions, and the screened candidate regions accord with the preset region characteristics of the region where the symbolic graph is located; the preset area features include: at least one of the size range, the height-width ratio range, the color clustering category number and the gradient feature of the region where the symbolic graph is located; the color clustering category number is a category number obtained by clustering the pixel values of the area where the symbolic graph is located; the gradient feature is used for characterizing the edge feature of the symbolic graph;
the judging module is used for judging whether the extracted features belong to the symbolic graph class through a classifier used for judging whether the extracted features belong to the symbolic graph class or not so as to obtain a traffic sign detection result;
and the early warning module is used for prompting early warning when the traffic sign detection result is that the interval speed measuring sign is detected, detecting the position of the interval speed measuring sign in the street view image, and marking the corresponding interval speed measuring sign on the electronic map according to the position and the geographical position information of the street view image.
6. The apparatus of claim 5, further comprising:
and the cutting module is used for cutting the street view image according to the statistical information of the appearance position of the traffic sign in the pre-acquired street view image.
7. The apparatus of claim 5, wherein the street view image is a horizontal 360 degree panoramic street view image;
the judging module is specifically used for judging whether the extracted HOG features belong to the symbolic graph class through a support vector machine used for judging whether the extracted HOG features belong to the symbolic graph class or not after the HOG features are extracted from the screened candidate regions so as to obtain a traffic sign detection result.
8. The apparatus according to any one of claims 5 to 7, wherein the candidate region filtering module is specifically configured to filter out, from the obtained candidate regions, candidate regions that do not conform to any one of a size range, an aspect ratio range, a color cluster category number, and a gradient feature of a region in which the landmark graph is located.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 4.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510797278.0A CN106709412B (en) | 2015-11-17 | 2015-11-17 | Traffic sign detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510797278.0A CN106709412B (en) | 2015-11-17 | 2015-11-17 | Traffic sign detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106709412A CN106709412A (en) | 2017-05-24 |
| CN106709412B true CN106709412B (en) | 2021-05-11 |
Family
ID=58932413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510797278.0A Active CN106709412B (en) | 2015-11-17 | 2015-11-17 | Traffic sign detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106709412B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107392115B (en) * | 2017-06-30 | 2021-01-12 | 中原智慧城市设计研究院有限公司 | Traffic sign identification method based on hierarchical feature extraction |
| CN108021868A (en) * | 2017-11-06 | 2018-05-11 | 南京航空航天大学 | A kind of quick highly reliable circular target detection recognition method |
| CN109447139B (en) * | 2018-10-18 | 2021-03-05 | 中国农业大学 | Method and device for identifying scene point cloud of farm containing complete livestock |
| CN111325970B (en) * | 2018-12-14 | 2022-07-26 | 沈阳美行科技股份有限公司 | Road speed limit prompting method, device and related equipment |
| JP7289723B2 (en) * | 2019-05-23 | 2023-06-12 | 日立Astemo株式会社 | Object recognition device |
| CN112508989B (en) * | 2020-11-20 | 2024-03-01 | 腾讯科技（深圳）有限公司 | Image processing method, device, server and medium |
| CN113191272A (en) * | 2021-04-30 | 2021-07-30 | 杭州品茗安控信息技术股份有限公司 | Engineering image identification method, identification system and related device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799859A (en) * | 2012-06-20 | 2012-11-28 | 北京交通大学 | Method for identifying traffic sign |
| CN103366190A (en) * | 2013-07-26 | 2013-10-23 | 中国科学院自动化研究所 | Method for identifying traffic sign |
| CN103390167A (en) * | 2013-07-18 | 2013-11-13 | 奇瑞汽车股份有限公司 | Multi-characteristic layered traffic sign identification method |
| CN103544484A (en) * | 2013-10-30 | 2014-01-29 | 广东工业大学 | Traffic sign identification method and system based on SURF |
| CN104463105A (en) * | 2014-11-19 | 2015-03-25 | 深圳市腾讯计算机系统有限公司 | Guide board recognizing method and device |
| CN104616021A (en) * | 2014-12-24 | 2015-05-13 | 深圳市腾讯计算机系统有限公司 | Method and device for processing traffic sign symbols |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104867192A (en) * | 2015-04-30 | 2015-08-26 | 公安部交通管理科学研究所 | Automobile driving path identification system based on automotive electronic identification, and method thereof |
-
2015
- 2015-11-17 CN CN201510797278.0A patent/CN106709412B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799859A (en) * | 2012-06-20 | 2012-11-28 | 北京交通大学 | Method for identifying traffic sign |
| CN103390167A (en) * | 2013-07-18 | 2013-11-13 | 奇瑞汽车股份有限公司 | Multi-characteristic layered traffic sign identification method |
| CN103366190A (en) * | 2013-07-26 | 2013-10-23 | 中国科学院自动化研究所 | Method for identifying traffic sign |
| CN103544484A (en) * | 2013-10-30 | 2014-01-29 | 广东工业大学 | Traffic sign identification method and system based on SURF |
| CN104463105A (en) * | 2014-11-19 | 2015-03-25 | 深圳市腾讯计算机系统有限公司 | Guide board recognizing method and device |
| CN104616021A (en) * | 2014-12-24 | 2015-05-13 | 深圳市腾讯计算机系统有限公司 | Method and device for processing traffic sign symbols |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106709412A (en) | 2017-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106709412B (en) | Traffic sign detection method and device | |
| CN107516077B (en) | Traffic sign information extraction method based on fusion of laser point cloud and image data | |
| CN106651872B (en) | Pavement crack identification method and system based on Prewitt operator | |
| CN107506760B (en) | Traffic signal detection method and system based on GPS positioning and visual image processing | |
| CN108364010B (en) | License plate recognition method, device, equipment and computer readable storage medium | |
| CN109740478B (en) | Vehicle detection and identification method, device, computer equipment and readable storage medium | |
| CN110197589B (en) | Deep learning-based red light violation detection method | |
| CN105373794B (en) | A kind of licence plate recognition method | |
| CN105913041B (en) | It is a kind of based on the signal lamp recognition methods demarcated in advance | |
| CN103971126B (en) | A kind of traffic sign recognition method and device | |
| CN101334836B (en) | License plate positioning method incorporating color, size and texture characteristic | |
| CN109086687A (en) | The traffic sign recognition method of HOG-MBLBP fusion feature based on PCA dimensionality reduction | |
| CN107301405A (en) | Method for traffic sign detection under natural scene | |
| CN107016362B (en) | Vehicle weight recognition method and system based on vehicle front windshield pasted mark | |
| CN109215364B (en) | Traffic signal recognition method, system, device and storage medium | |
| CN103761529A (en) | Open fire detection method and system based on multicolor models and rectangular features | |
| KR20160128928A (en) | Method and apparatus for confirmation of relevant white inner circle in environment of circular traffic sign recognition | |
| CN109255279A (en) | A kind of method and system of road traffic sign detection identification | |
| Sallah et al. | Road sign detection and recognition system for real-time embedded applications | |
| CN104408431A (en) | Vehicle model identification method under traffic monitoring | |
| Chiang et al. | Road speed sign recognition using edge-voting principle and learning vector quantization network | |
| Chowdhury et al. | An adaptive technique for computer vision based vehicles license plate detection system | |
| CN113989771A (en) | Traffic signal lamp identification method based on digital image processing | |
| CN116503821B (en) | Road identification recognition method and system based on point cloud data and image recognition | |
| KR102204051B1 (en) | Mms image editting system having blur menu for image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |