CN110909674B - Traffic sign recognition method, device, equipment and storage medium - Google Patents
Traffic sign recognition method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN110909674B CN110909674B CN201911149014.9A CN201911149014A CN110909674B CN 110909674 B CN110909674 B CN 110909674B CN 201911149014 A CN201911149014 A CN 201911149014A CN 110909674 B CN110909674 B CN 110909674B
- Authority
- CN
- China
- Prior art keywords
- identification
- traffic sign
- picture
- target
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000012216 screening Methods 0.000 claims abstract description 18
- 230000002159 abnormal effect Effects 0.000 claims description 5
- 238000003062 neural network model Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 10
- 230000008901 benefit Effects 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013434 data augmentation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention discloses a traffic sign recognition method, a traffic sign recognition device, traffic sign recognition equipment and a traffic sign recognition storage medium. The method comprises the following steps: the method comprises the steps of obtaining pictures in an identification picture group to conduct traffic sign large-category identification, and obtaining target pictures with identification results meeting a first confidence coefficient condition, wherein the identification results comprise at least one identification area, traffic sign large categories identified in the identification area and identification confidence coefficients; screening out at least one target judgment area meeting traffic sign recognition conditions in each recognition area of the target picture; and carrying out traffic sign subdivision category identification in the at least one target judgment area, and acquiring the traffic sign subdivision category meeting the second confidence coefficient condition as a traffic sign identification result. By using the technical scheme of the embodiment of the invention, the classification of major categories, numerical values and the like of the traffic sign can be identified in real time, early warning can be carried out on a driver in time, and the driving safety is improved.
Description
Technical Field
The embodiment of the invention relates to an image recognition technology, in particular to a traffic sign recognition method, a traffic sign recognition device, traffic sign recognition equipment and a traffic sign recognition storage medium.
Background
When traffic control, road construction transformation, complex road conditions and the like occur, a driver easily enters a forbidden area by mistake and overspeed driving due to negligence or misjudgment of traffic signs, and even serious consequences of traffic accidents occur.
In the prior art, various recognition models are generally adopted to recognize traffic signs and early warn drivers. Existing traffic sign recognition models, such as SSD (Single Shot MultiBox Detector, single-stage multi-frame detector), whose actual operating results show that, although the type of traffic sign can be recognized normally, the numerical recognition error rate concerning specific values of the traffic sign, such as speed limit, is high. At this time, if the traffic sign recognition model is erroneously recognized, the driver is likely to make erroneous judgment, and problems such as overspeed and overweight driving occur.
Disclosure of Invention
The embodiment of the invention provides a traffic sign recognition method, a device, equipment and a storage medium, which are used for realizing real-time, real and accurate recognition of traffic signs and timely early warning to a driver in the driving process of the driver.
In a first aspect, an embodiment of the present invention provides a traffic sign recognition method, including:
the method comprises the steps of obtaining pictures in an identification picture group to conduct traffic sign large-category identification, and obtaining target pictures with identification results meeting a first confidence coefficient condition, wherein the identification results comprise at least one identification area, traffic sign large categories identified in the identification area and identification confidence coefficients;
screening out at least one target judgment area meeting traffic sign recognition conditions in each recognition area of the target picture;
and carrying out traffic sign subdivision category identification in the at least one target judgment area, and acquiring the traffic sign subdivision category meeting the second confidence coefficient condition as a traffic sign identification result.
Further, obtaining the pictures in the identification picture group to perform traffic sign large-class identification, and obtaining the target picture with the identification result meeting the first confidence coefficient condition includes:
acquiring a picture at the middle position from the identification picture group as a basic picture;
carrying out traffic sign large category identification on the basic picture, and acquiring traffic sign large categories identified in each identification area and identification confidence;
if the identification confidence coefficient of the basic picture exceeds a preset first confidence coefficient threshold value, respectively carrying out traffic sign large-class identification on the rest pictures in the identification picture group, and acquiring the picture with the highest identification confidence coefficient as the target picture.
Further, screening at least one target judgment area meeting the traffic sign recognition condition in each recognition area of the target picture, wherein the target judgment area comprises at least one of the following:
if the aspect ratio of the traffic sign contained in the target identification area is abnormal, deleting the target identification area;
if the sum of the length and the width of the traffic sign contained in the target identification area is smaller than the sum of the preset length and the width, deleting the target identification area; and
and deleting the target identification area if the traffic sign in the target identification area for identifying the forbidden traffic sign does not meet the forbidden traffic sign condition.
Further, determining that the traffic sign in the target recognition area for recognizing the forbidden traffic sign does not meet the forbidden traffic sign condition includes:
converting the target identification area into a hexagonal cone model format;
counting the number of red pixel points in the target identification area according to the tone parameters of the pixel points;
and if the number of the red pixel points is smaller than the preset pixel number, determining that the traffic sign contained in the target identification area does not meet the forbidden traffic sign condition.
Further, identifying the traffic sign subdivision category in the at least one target judgment area, and acquiring the traffic sign subdivision category meeting the second confidence condition as a traffic sign identification result, including:
carrying out traffic sign subdivision category identification on the at least one target judgment area to obtain an identification result, wherein the identification result comprises traffic sign subdivision categories identified in the target judgment area and identification confidence;
if the identification confidence of the target judgment area exceeds a preset second confidence threshold, the identified traffic sign subdivision category meets a second confidence condition, and the identified traffic sign subdivision category is used as a traffic sign identification result.
Further, before the obtaining of the pictures in the identification picture group for traffic sign large category identification, the method further comprises:
cutting road pictures shot in real time, and storing the cut road pictures into a picture queue;
acquiring road pictures with set quantity in the picture queue as the identification picture group;
after acquiring the recognition result satisfying the second confidence condition as the traffic sign recognition result, further comprising:
and returning to execute the operation of acquiring the road pictures with the set quantity in the picture queue as the identification picture group until the end identification condition is met.
Further, the cutting the road picture shot in real time includes:
cutting the shot road picture into a road picture with a preset size, wherein the cut road picture is the upper right part of the shot road picture.
In a second aspect, an embodiment of the present invention further provides a traffic sign recognition device, including:
the traffic sign large-category identification module is used for acquiring pictures in the identification picture group to carry out traffic sign large-category identification and acquiring target pictures with identification results meeting a first confidence coefficient condition, wherein the identification results comprise at least one identification area, the traffic sign large category identified in the identification area and the identification confidence coefficient;
the identification condition screening module is used for screening at least one target judgment area meeting the traffic sign identification condition in each identification area of the target picture;
the traffic sign subdivision category identification module is used for carrying out traffic sign subdivision category identification in the at least one target judgment area and acquiring the traffic sign subdivision category meeting the second confidence coefficient condition as a traffic sign identification result.
In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the traffic sign recognition method according to any embodiment of the present invention when executing the program.
In a fourth aspect, embodiments of the present invention also provide a computer readable storage medium having stored thereon a computer program which, when executed by a computer processor, implements a traffic sign recognition method as provided by any of the embodiments of the present invention.
According to the embodiment of the invention, the traffic sign major category identification is carried out on the target picture, the identification area is obtained, the identification area is screened, and the traffic sign subdivision category identification is carried out on the screened target judgment area, so that the traffic sign subdivision category is obtained. The method solves the problem of low recognition accuracy of the traffic sign subdivision categories after the large categories of the traffic signs are recognized in the prior art, and achieves the effect of accurately recognizing the large categories and the subdivision categories of the traffic signs in real time.
Drawings
FIG. 1 is a flow chart of a traffic sign recognition method according to a first embodiment of the present invention;
FIG. 2 is a flow chart of a traffic sign recognition method in a second embodiment of the present invention;
FIG. 3 is a flow chart of traffic sign identification suitable for use in an embodiment of the present invention;
fig. 4 is a schematic structural view of a traffic sign recognition device according to a third embodiment of the present invention;
fig. 5 is a schematic structural view of a traffic sign recognition device in a fourth embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Example 1
Fig. 1 is a flowchart of a traffic sign recognition method according to a first embodiment of the present invention, where the method may be applied to recognizing traffic signs in real time and early warning the driver during driving, and the method may be performed by a traffic sign recognition device, which may be implemented by software and/or hardware and may be generally integrated in a vehicle. The method specifically comprises the following steps:
step 110, obtaining pictures in the identification picture group to carry out traffic sign large-class identification, and obtaining target pictures with identification results meeting a first confidence coefficient condition;
the identification result comprises at least one identification area, a large traffic sign category identified in the identification area and identification confidence;
the traffic sign large categories mainly comprise speed limit, line limit, height limit, weight limit, speed measurement and the like, the traffic sign categories are accurately identified and early-warned in time, and driving safety of a driver can be guaranteed. Confidence, also referred to as reliability, confidence level, confidence coefficient, etc., refers to the probability that an overall parameter value falls within a certain region of a sample statistic. The recognition result of the target picture meets the confidence coefficient condition, and the large traffic sign category recognized by the recognition area of the target picture is basically reliable. The identification area is a part which is intercepted and possibly contains traffic signs in the process of identifying the target picture. In a specific embodiment, the pixel size of the target picture may be 300×300, and the size of the identification area may be defined as 32×32, but the size of the identification area is not limited in this embodiment.
Wherein the traffic sign large category is identified as being completed by the deep neural network model one. The skeleton network structure of the model one can be any one or a variant of the convolutional neural network. For example: VGG (Visual Geometry Group ) model, res net (Deep residual network, depth residual network), denseNet (Dense Convolutional Network ), etc., the framework network structure of model one is not limited by this embodiment. In a specific embodiment, the model-skeleton network employs MobileNet V2, and the training parameters are set as follows: the size of the picture input pixel is 300 x 300, the training times are 300000, random clipping and left-right overturning are adopted for data augmentation, the length-width ratio of an anchor frame is set to be 1:1, 2:1, 1:2, 1:3 and 3:1, the initial value of the learning rate is 0.004, and the learning rate is adjusted to be the front value of 0.95 after each training time is 1000 times. After 200000 iterations, using the graph rewriter to quantize, the activation bit number is set to 8, the weight bit number is set to 8, and the quantization can compress the model to occupy less memory, and the running speed is high. The trained model I is responsible for identifying the large types of traffic signs of the pictures in the identification picture group, and when the input is the picture, the output of the model I is the large types of the traffic signs possibly contained in each identification area in the picture and the corresponding confidence level. Step 120, screening out at least one target judgment area meeting traffic sign recognition conditions in each recognition area of the target picture;
if the aspect ratio of the traffic sign contained in the target identification area is abnormal, deleting the target identification area; the advantage of this arrangement is that, in real life, the aspect ratio of the road traffic sign is 1:1, so that when the aspect ratio of the traffic sign in the target recognition area is too large or too small, the traffic sign can be basically judged to be misrecognized, and the traffic sign can be removed from the recognition area, so that the workload of recognition of the subdivision category of the traffic sign in the later stage can be reduced. In a specific embodiment, the case where the length/width ratio is less than 0.7 or the width/length ratio is greater than 1.5 may be defined as an aspect ratio abnormality, but the specific threshold value of the aspect ratio abnormality is not limited thereto.
If the sum of the length and the width of the traffic sign contained in the target identification area is smaller than the sum of the preset length and the width, deleting the target identification area; the advantage of this arrangement is that when the sum of the length and the width of the traffic sign is smaller, the proportion of the number of pixels in the whole identification area is lower, erroneous judgment is easy to be generated on the major class and the subdivision class of the traffic sign, and the accuracy of the identification of the traffic sign can be ensured by eliminating the traffic sign from the identification area. In a specific embodiment, the size of the identification area may be defined as 32×32, where if the sum of the length and the width of the identified traffic sign is less than 28, which indicates that the proportion of the traffic sign in the identification area is too small, it is necessary to reject the traffic sign, but the specific threshold value of the abnormal aspect ratio is not limited in this embodiment.
And if the traffic sign in the target identification area for identifying the forbidden traffic sign does not meet the forbidden traffic sign condition, deleting the target identification area. The arrangement has the advantages that the base colors of the forbidden traffic signs such as speed limit, weight limit, height limit, line limit and the like are red, if the forbidden traffic signs are identified in the identification area, the number of red pixel points in the identification area is too small, which indicates that misjudgment is possible to happen at the moment, so that the forbidden traffic signs are removed from the identification area, and the accuracy of identifying the traffic signs is improved.
The method for determining the traffic sign in the target recognition area for recognizing the forbidden traffic sign does not meet the forbidden traffic sign condition comprises the following steps: converting the target identification area into a hexagonal cone model format; counting the number of red pixel points in the target identification area according to the tone parameters of the pixel points; and if the number of the red pixel points is smaller than the preset pixel number, determining that the traffic sign contained in the target identification area does not meet the forbidden traffic sign condition.
The hexagonal cone model format is also called an HSV (Hue, saturation, value, hue, saturation and brightness) color model, the HSV color model expresses a color image by three parameters of hue, saturation and brightness, the hue, vividness and brightness of the color can be intuitively expressed, and the color comparison is convenient. The target recognition area is originally in the space domain of expressing the color image by using an RGB (Red, green, blue, red, green and blue) model, and is converted into the space domain of expressing the color image by using an HSV model, so that the number of red pixels can be counted conveniently. In the HSV model, the value of the H parameter is 0-360, wherein the pixel points with the H value smaller than 15 and the H value larger than 345 are red pixel points. And counting the number of the pixel points with the value H in the range as the number of the red pixel points. In a specific embodiment, when the number of red pixels is less than 80 when the identification area is set to 32×32, it may be determined that erroneous determination is occurred, and the erroneous determination is removed from the identification area, but the threshold of the number of red pixels is not limited in this embodiment.
And 130, identifying the traffic sign subdivision category in the at least one target judgment area, and acquiring the traffic sign subdivision category meeting the second confidence level condition as a traffic sign identification result.
And (3) identifying the traffic sign subdivision category of the target judgment area screened in the step (120). The traffic sign subdivision category, i.e. the traffic sign major category and the numerical values or letters following the traffic sign major category, e.g. the speed limit 30 when the traffic sign major category is speed limit. If the recognition of the traffic sign subdivision class meets the second confidence condition, the traffic sign subdivision class recognized in the step is basically reliable, and the recognition result of the traffic sign subdivision class can be used as a traffic sign recognition result to prompt a driver.
The traffic sign subdivision type identification is completed through a classified neural network model II. In a specific embodiment, the two-model network adopts DenseNet, so that the network has the advantages of adopting a characteristic diagram to connect the network layers, having few model parameters and high accuracy. The model training parameters were set as: the network depth is set to 40, the dimension is reduced by 1X1 convolution before the input of each layer, and the dimension is reduced by 1X1 convolution between different DenseNet modules. The learning rate was set to 0.0001 and the conversion layer compression factor was set to 0.5. And the trained model II is responsible for identifying the traffic sign subdivision category, and when the traffic sign subdivision category is input into the target judgment area containing the traffic sign, the traffic sign subdivision category of each target judgment area and the corresponding confidence coefficient are output.
According to the technical scheme, the traffic sign classification identification is carried out on the target picture, the identification area is obtained, the identification area is screened, and the traffic sign classification identification is carried out on the screened target judgment area, so that the traffic sign classification is obtained. The method solves the problem of low recognition accuracy of the traffic sign subdivision categories after the large categories of the traffic signs are recognized in the prior art, and achieves the effect of accurately recognizing the large categories and the subdivision categories of the traffic signs in real time.
Example two
Fig. 2 is a flowchart of a traffic sign recognition method in a second embodiment of the present invention, where, based on the technical solution of the foregoing embodiment, a process for recognizing a large traffic sign category and a process for recognizing a sub-division category of a traffic sign are further refined, and specifically includes the following steps:
step 210, cutting the road picture shot in real time, and storing the cut road picture into a picture queue;
and cutting the shot road picture into a road picture with a preset size, wherein the cut road picture is the upper right part of the shot road picture.
The advantage of setting up like this lies in that road traffic sign general condition all sets up in the road right side to the vehicle is right-hand driving too, therefore, the probability that traffic sign appears in the road photo lower left side is extremely low, consequently cuts the road picture of taking in advance, keeps the upper right part of road picture, can reduce the work load of follow-up discernment process. In a specific embodiment, the picture size may be cut to 300×300, but the present embodiment does not provide a limiting definition of the picture size.
Step 220, obtaining road pictures with a set number in the picture queue as the identification picture group;
the road pictures with the set number are placed into the identification picture group for processing, and the setting has the advantages that the road pictures are shot in real time in the running process of the vehicle, so that the number of the road pictures is huge, and the set number of the road pictures is processed as a group, so that the workload can be reduced. Meanwhile, since the content of continuously taken photos is generally continuous, the influence of the grouping process on the recognition accuracy is negligible.
Fig. 3 is a flow chart of traffic sign recognition applicable to the embodiment of the present invention, and as shown in fig. 3, an odd number of pictures are taken as a recognition picture group. However, the present embodiment does not limit whether the set number is odd or even.
Step 230, obtaining a picture in the middle position from the identified picture group as a basic picture;
the advantage of this arrangement is that, because the road pictures taken continuously are generally continuous in content, the picture in the middle position most represents the identified picture group, and if no traffic sign is identified in the picture in the middle position, no traffic sign appears in other pictures in the identified picture group.
Step 240, identifying the traffic sign large category of the basic picture, and acquiring the traffic sign large category identified in each identification area and the identification confidence;
when the traffic sign large-class identification is performed on the picture in the middle position, namely the basic picture, one basic picture possibly comprises a plurality of areas where traffic signs possibly appear, and the areas where the traffic signs possibly appear are cut out to serve as identification areas. And respectively carrying out large-class recognition of the traffic sign on the recognition areas and calculating the confidence coefficient.
Step 250, if the recognition confidence coefficient of the basic picture exceeds a preset first confidence coefficient threshold value, respectively carrying out traffic sign large-class recognition on the rest pictures in the recognition picture group, and obtaining a picture with the highest recognition confidence coefficient as the target picture;
the recognition confidence coefficient of the basic picture exceeds a preset first confidence coefficient threshold value, which indicates that the reliability of the traffic sign in the basic picture is higher, so that the traffic sign large-class recognition is also carried out on other pictures in the picture recognition group, the confidence coefficient is calculated, and the picture with the highest confidence coefficient in the picture recognition group is obtained as the target picture.
As shown in fig. 3, the pictures in the middle position are taken, the calculation of the traffic sign large category and the confidence coefficient is performed on the pictures through a model pair, if the confidence coefficient is smaller than 0.6, the pictures are directly returned, and the odd-numbered pictures are taken as the identification picture group in the picture collection. If the confidence coefficient is greater than 0.6, the next operation can be performed, namely, carrying out traffic sign large-class recognition and confidence coefficient calculation on the rest of pictures in the recognition picture group through a model pair, selecting the picture with the highest confidence coefficient as a target picture, and acquiring at least one recognition area of the target picture.
Step 260, screening out at least one target judgment area meeting the traffic sign recognition condition in each recognition area of the target picture;
step 270, identifying the traffic sign subdivision category of the at least one target judgment area, and obtaining an identification result, wherein the identification result comprises the traffic sign subdivision category identified in the target judgment area and the identification confidence;
and identifying the traffic sign subdivision category of the screened target judgment area, and calculating the confidence coefficient.
And 280, if the identification confidence of the target judgment area exceeds a preset second confidence threshold, the identified traffic sign subdivision category meets a second confidence condition, and the identified traffic sign subdivision category is used as a traffic sign identification result.
If the recognition of the traffic sign subdivision category meets the second confidence condition, the traffic sign subdivision category recognized in the step is basically reliable, and the recognition result of the traffic sign subdivision category can be used as a traffic sign recognition result to prompt a driver.
As shown in fig. 3, after the model two performs traffic sign subdivision category recognition and confidence calculation on the screened target judgment area, if the confidence is greater than 0.96, the traffic sign subdivision category is output as a traffic sign recognition result.
Step 290, judging whether the end recognition condition is satisfied, if not, returning to the execution step 220; if so, step 2100 is performed.
If the identification process is not finished, after the identification picture group of the group is identified and the traffic sign identification result is obtained, the road pictures with the set number are re-acquired as new identification picture groups, and the identification process is continued.
Step 2100, end the identification process.
According to the technical scheme, through preprocessing the pictures in a cutting mode, taking the pictures with the preprocessed preset number as a picture group, judging whether the pictures in the middle position of the picture group can identify traffic sign major categories or not, calculating the confidence coefficient, if the confidence coefficient meets the requirement, identifying the rest pictures of the picture group, calculating the confidence coefficient, selecting the picture with the highest confidence coefficient, screening the identification area, carrying out recognition of traffic sign subdivision categories and calculation of the confidence coefficient again on the screened target judgment area, and if the confidence coefficient of the target judgment area meets the requirement, taking the identified traffic sign subdivision categories as a final result. The method solves the problem that in the prior art, although the major class of the traffic sign can be identified, the error rate is higher when the subdivision class of the traffic sign is identified. The effect of accurately identifying the major class and the subdivision class of the traffic sign is achieved.
Example III
Fig. 4 is a schematic structural diagram of a traffic sign recognition device according to a third embodiment of the present invention, the traffic sign recognition device including: a traffic sign large category identification module 310, an identification condition screening module 320, and a traffic sign sub-category identification module 330, wherein:
the traffic sign large category identification module 310 is configured to obtain a picture in the identification picture group to perform traffic sign large category identification, and obtain a target picture whose identification result meets a first confidence coefficient condition, where the identification result includes at least one identification area, a traffic sign large category identified in the identification area, and an identification confidence coefficient;
the identifying condition screening module 320 is configured to screen at least one target judgment area that meets the traffic sign identifying condition in each of the identifying areas of the target picture;
the traffic sign subdivision category identification module 330 is configured to identify a traffic sign subdivision category in the at least one target determination area, and obtain a traffic sign subdivision category that meets a second confidence condition as a traffic sign identification result.
On the basis of the above embodiment, the traffic sign large category identification module 310 includes:
a basic picture acquisition unit, configured to acquire, as a basic picture, a picture in a middle position in the identified picture group;
the basic picture large category identification unit is used for carrying out traffic sign large category identification on the basic picture, and acquiring traffic sign large categories identified in each identification area and identification confidence;
and the other picture large category identification unit is used for respectively carrying out traffic sign large category identification on the other pictures in the identification picture group if the identification confidence of the basic picture exceeds a preset first confidence threshold value, and acquiring the picture with the highest identification confidence as the target picture.
On the basis of the above embodiment, the identifying condition screening module 320 includes:
a first target recognition area deleting unit, configured to delete a target recognition area if it is determined that an aspect ratio of a traffic sign included in the target recognition area is abnormal;
a second target recognition area deleting unit, configured to delete a target recognition area if it is determined that a sum of length and width of traffic signs included in the target recognition area is smaller than a sum of preset length and width;
and the third target identification area deleting unit is used for deleting the target identification area if the traffic sign in the target identification area which identifies the forbidden traffic sign does not meet the forbidden traffic sign condition.
On the basis of the above embodiment, the third target recognition area deleting unit includes:
the format conversion subunit is used for converting the target identification area into a hexagonal cone model format;
the pixel point number counting subunit is used for counting the number of red pixel points in the target identification area according to the tone parameters of the pixel points;
and the forbidden traffic sign condition judging subunit is used for determining that the traffic sign contained in the target identification area does not meet the forbidden traffic sign condition if the number of the red pixel points is smaller than the preset pixel number.
On the basis of the above embodiment, the traffic sign subdivision category identification module 330 includes:
the traffic sign subdivision category identification unit is used for carrying out traffic sign subdivision category identification on the at least one target judgment area to obtain an identification result, wherein the identification result comprises the traffic sign subdivision category identified in the target judgment area and identification confidence;
and the second confidence condition judging unit is used for judging whether the recognized traffic sign subdivision category meets the second confidence condition or not if the recognition confidence of the target judging area exceeds a preset second confidence threshold value, and taking the recognized traffic sign subdivision category as a traffic sign recognition result.
On the basis of the above embodiment, the traffic sign recognition device further includes:
the image cutting module is used for cutting road images shot in real time and storing the cut road images into an image queue;
the identification picture group acquisition module is used for acquiring road pictures with the set quantity in the picture queue as the identification picture group;
and the return execution module is used for returning and executing the operation of acquiring the road pictures with the set quantity in the picture queue as the identification picture group until the end identification condition is met.
On the basis of the above embodiment, the picture cropping module includes:
and the road photo cutting unit is used for cutting the shot road photo into a road photo with a preset size, wherein the cut road photo is the upper right part of the shot road photo.
The traffic sign recognition device provided by the embodiment of the invention can execute the traffic sign recognition method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 5 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention, and as shown in fig. 5, the apparatus includes a processor 40, a memory 41, an input device 42 and an output device 43; the number of processors 40 in the device may be one or more, one processor 40 being taken as an example in fig. 5; the processor 40, the memory 41, the input means 42 and the output means 43 in the device may be connected by a bus or by other means, in fig. 5 by way of example.
The memory 41 is used as a computer readable storage medium for storing software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the traffic sign recognition method in the embodiment of the present invention (for example, the traffic sign large category recognition module 310, the recognition condition screening module 320, and the traffic sign subdivision category recognition module 330 in the traffic sign recognition device). The processor 40 performs various functional applications of the device and data processing, i.e., implements the traffic sign recognition method described above, by running software programs, instructions and modules stored in the memory 41. The method comprises the following steps:
the method comprises the steps of obtaining pictures in an identification picture group to conduct traffic sign large-category identification, and obtaining target pictures with identification results meeting a first confidence coefficient condition, wherein the identification results comprise at least one identification area, traffic sign large categories identified in the identification area and identification confidence coefficients;
screening out at least one target judgment area meeting traffic sign recognition conditions in each recognition area of the target picture;
and carrying out traffic sign subdivision category identification in the at least one target judgment area, and acquiring the traffic sign subdivision category meeting the second confidence coefficient condition as a traffic sign identification result.
The memory 41 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the terminal, etc. In addition, memory 41 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, memory 41 may further include memory located remotely from processor 40, which may be connected to the device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input means 42 may be used to receive entered numeric or character information and to generate key signal inputs related to user settings and function control of the device. The output means 43 may comprise a display device such as a display screen.
Example five
A fifth embodiment of the present invention also provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are for performing a traffic sign recognition method, the method comprising:
the method comprises the steps of obtaining pictures in an identification picture group to conduct traffic sign large-category identification, and obtaining target pictures with identification results meeting a first confidence coefficient condition, wherein the identification results comprise at least one identification area, traffic sign large categories identified in the identification area and identification confidence coefficients;
screening out at least one target judgment area meeting traffic sign recognition conditions in each recognition area of the target picture;
and carrying out traffic sign subdivision category identification in the at least one target judgment area, and acquiring the traffic sign subdivision category meeting the second confidence coefficient condition as a traffic sign identification result.
Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present invention is not limited to the method operations described above, and may also perform the related operations in the traffic sign recognition method provided in any embodiment of the present invention.
From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software and necessary general purpose hardware, but of course also by means of hardware, although in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, etc., and include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments of the present invention.
It should be noted that, in the embodiment of the traffic sign recognition device, each unit and module included are only divided according to the functional logic, but not limited to the above division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (9)
1. A traffic sign recognition method, comprising:
the method comprises the steps of obtaining pictures in an identification picture group to conduct traffic sign large-category identification, and obtaining target pictures with identification results meeting a first confidence coefficient condition, wherein the identification results comprise at least one identification area, traffic sign large categories identified in the identification area and identification confidence coefficients;
screening out at least one target judgment area meeting traffic sign recognition conditions in each recognition area of the target picture;
carrying out traffic sign subdivision category identification in the at least one target judgment area, and acquiring the traffic sign subdivision category meeting the second confidence coefficient condition as a traffic sign identification result;
the identifying traffic sign subdivision category in the at least one target judgment area comprises the following steps:
identifying the numerical value or the letter after the traffic sign major class through the classified neural network model;
the step of obtaining the pictures in the identification picture group for traffic sign large-class identification and obtaining the target pictures with the identification result meeting the first confidence coefficient condition comprises the following steps:
acquiring a picture at the middle position from the identification picture group as a basic picture;
carrying out traffic sign large category identification on the basic picture, and acquiring traffic sign large categories identified in each identification area and identification confidence;
if the identification confidence coefficient of the basic picture exceeds a preset first confidence coefficient threshold value, respectively carrying out traffic sign large-class identification on the rest pictures in the identification picture group, and acquiring the picture with the highest identification confidence coefficient as the target picture.
2. The method of claim 1, wherein screening at least one target judgment area satisfying traffic sign recognition conditions in each of the recognition areas of the target picture comprises at least one of:
if the aspect ratio of the traffic sign contained in the target identification area is abnormal, deleting the target identification area;
if the sum of the length and the width of the traffic sign contained in the target identification area is smaller than the sum of the preset length and the width, deleting the target identification area; and
and deleting the target identification area if the traffic sign in the target identification area for identifying the forbidden traffic sign does not meet the forbidden traffic sign condition.
3. The method of claim 2, wherein determining that the traffic sign within the target identification area where the forbidden traffic sign is identified does not satisfy a forbidden traffic sign condition comprises:
converting the target identification area into a hexagonal cone model format;
counting the number of red pixel points in the target identification area according to the tone parameters of the pixel points;
and if the number of the red pixel points is smaller than the preset pixel number, determining that the traffic sign contained in the target identification area does not meet the forbidden traffic sign condition.
4. The method of claim 1, wherein performing traffic sign subdivision category identification within the at least one target determination area and obtaining traffic sign subdivision categories that satisfy a second confidence condition as traffic sign identification results comprises:
carrying out traffic sign subdivision category identification on the at least one target judgment area to obtain an identification result, wherein the identification result comprises traffic sign subdivision categories identified in the target judgment area and identification confidence;
if the identification confidence of the target judgment area exceeds a preset second confidence threshold, the identified traffic sign subdivision category meets a second confidence condition, and the identified traffic sign subdivision category is used as a traffic sign identification result.
5. The method of any of claims 1-4, further comprising, prior to acquiring the pictures in the identified group of pictures for traffic sign large category identification:
cutting road pictures shot in real time, and storing the cut road pictures into a picture queue;
acquiring road pictures with set quantity in the picture queue as the identification picture group;
after acquiring the recognition result satisfying the second confidence condition as the traffic sign recognition result, further comprising:
and returning to execute the operation of acquiring the road pictures with the set quantity in the picture queue as the identification picture group until the end identification condition is met.
6. The method of claim 5, wherein cropping the road picture taken in real time comprises:
cutting the shot road picture into a road picture with a preset size, wherein the cut road picture is the upper right part of the shot road picture.
7. A traffic sign recognition device, comprising:
the traffic sign large-category identification module is used for acquiring pictures in the identification picture group to carry out traffic sign large-category identification and acquiring target pictures with identification results meeting a first confidence coefficient condition, wherein the identification results comprise at least one identification area, the traffic sign large category identified in the identification area and the identification confidence coefficient;
the identification condition screening module is used for screening at least one target judgment area meeting the traffic sign identification condition in each identification area of the target picture;
the traffic sign subdivision category identification module is used for carrying out traffic sign subdivision category identification in the at least one target judgment area and acquiring the traffic sign subdivision category meeting the second confidence coefficient condition as a traffic sign identification result;
the traffic sign subdivision category identification unit is used for identifying numerical values or letters after the traffic sign major categories through the classified neural network model;
the traffic sign large category identification module comprises:
a basic picture acquisition unit, configured to acquire, as a basic picture, a picture in a middle position in the identified picture group;
the basic picture large category identification unit is used for carrying out traffic sign large category identification on the basic picture, and acquiring traffic sign large categories identified in each identification area and identification confidence;
and the other picture large category identification unit is used for respectively carrying out traffic sign large category identification on the other pictures in the identification picture group if the identification confidence of the basic picture exceeds a preset first confidence threshold value, and acquiring the picture with the highest identification confidence as the target picture.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the traffic sign recognition method of any one of claims 1-6 when the program is executed by the processor.
9. A storage medium containing computer executable instructions which, when executed by a computer processor, are for performing the traffic sign recognition method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911149014.9A CN110909674B (en) | 2019-11-21 | 2019-11-21 | Traffic sign recognition method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911149014.9A CN110909674B (en) | 2019-11-21 | 2019-11-21 | Traffic sign recognition method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110909674A CN110909674A (en) | 2020-03-24 |
| CN110909674B true CN110909674B (en) | 2024-01-05 |
Family
ID=69818301
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911149014.9A Active CN110909674B (en) | 2019-11-21 | 2019-11-21 | Traffic sign recognition method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110909674B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111582029B (en) * | 2020-04-02 | 2022-08-12 | 天津大学 | Traffic sign identification method based on dense connection and attention mechanism |
| CN112699841A (en) * | 2021-01-13 | 2021-04-23 | 华南理工大学 | Traffic sign detection and identification method based on driving video |
| CN113963329B (en) * | 2021-10-11 | 2022-07-05 | 浙江大学 | Digital traffic sign detection and identification method based on double-stage convolutional neural network |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105809121A (en) * | 2016-03-03 | 2016-07-27 | 电子科技大学 | Multi-characteristic synergic traffic sign detection and identification method |
| CN107220646A (en) * | 2017-05-25 | 2017-09-29 | 杭州健培科技有限公司 | A kind of medical image Text region Enhancement Method for going ambient interferences |
| CN109255279A (en) * | 2017-07-13 | 2019-01-22 | 深圳市凯立德科技股份有限公司 | A kind of method and system of road traffic sign detection identification |
-
2019
- 2019-11-21 CN CN201911149014.9A patent/CN110909674B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105809121A (en) * | 2016-03-03 | 2016-07-27 | 电子科技大学 | Multi-characteristic synergic traffic sign detection and identification method |
| CN107220646A (en) * | 2017-05-25 | 2017-09-29 | 杭州健培科技有限公司 | A kind of medical image Text region Enhancement Method for going ambient interferences |
| CN109255279A (en) * | 2017-07-13 | 2019-01-22 | 深圳市凯立德科技股份有限公司 | A kind of method and system of road traffic sign detection identification |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110909674A (en) | 2020-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11417118B2 (en) | Lane line data processing method and apparatus, computer device, and storage medium | |
| CN110059694B (en) | Intelligent identification method for character data in complex scene of power industry | |
| CN110909674B (en) | Traffic sign recognition method, device, equipment and storage medium | |
| WO2020164282A1 (en) | Yolo-based image target recognition method and apparatus, electronic device, and storage medium | |
| CN109918969B (en) | Face detection method and device, computer device and computer readable storage medium | |
| CN111368600B (en) | Remote sensing image target detection and identification method and device, readable storage medium and equipment | |
| CN111784685A (en) | Power transmission line defect image identification method based on cloud edge cooperative detection | |
| CN107944450B (en) | License plate recognition method and device | |
| CN112819068B (en) | Ship operation violation behavior real-time detection method based on deep learning | |
| CN111767878B (en) | Deep learning-based traffic sign detection method and system in embedded device | |
| CN108509954A (en) | A kind of more car plate dynamic identifying methods of real-time traffic scene | |
| CN113642474A (en) | Hazardous area personnel monitoring method based on YOLOV5 | |
| CN110599453A (en) | Panel defect detection method and device based on image fusion and equipment terminal | |
| CN112949578B (en) | Vehicle lamp state identification method, device, equipment and storage medium | |
| CN112465031B (en) | Data classification method, device and computer readable storage medium | |
| CN108229473A (en) | Vehicle annual inspection label detection method and device | |
| CN108288388A (en) | A kind of intelligent traffic monitoring system | |
| CN114429640A (en) | Drawing segmentation method and device and electronic equipment | |
| CN113486856A (en) | Driver irregular behavior detection method based on semantic segmentation and convolutional neural network | |
| CN112784494A (en) | Training method of false positive recognition model, target recognition method and device | |
| CN115147642B (en) | Method, device, computer and storage medium for detecting muck truck based on vision | |
| CN112598664B (en) | Visual saliency-based insect grain grade determination method and device | |
| CN115690410A (en) | Semantic segmentation method and system based on feature clustering | |
| CN117291859A (en) | Page abnormality detection method and device, electronic equipment and storage medium | |
| CN114187270A (en) | Gluing quality detection method and system for mining intrinsic safety type controller based on CCD |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |