CN112380973A - Traffic signal lamp identification method and system - Google Patents
Traffic signal lamp identification method and system Download PDFInfo
- Publication number
- CN112380973A CN112380973A CN202011262208.2A CN202011262208A CN112380973A CN 112380973 A CN112380973 A CN 112380973A CN 202011262208 A CN202011262208 A CN 202011262208A CN 112380973 A CN112380973 A CN 112380973A
- Authority
- CN
- China
- Prior art keywords
- traffic signal
- signal lamp
- image
- target image
- color information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a system for identifying traffic signal lamps, wherein the method comprises the following steps: acquiring image data containing a traffic signal lamp; obtaining a target image of the traffic signal lamp according to the image data; obtaining shape information and color information of the traffic signal lamp and corresponding credibility according to the target image; judging whether the reliability is greater than a set threshold value; if the reliability is greater than the set threshold, determining the working state of the traffic signal lamp according to the shape information and the color information; if the reliability is less than or equal to the set threshold, performing image processing on the target image; and obtaining the color information and the shape information of the traffic signal lamp according to the processed target image to determine the working state of the traffic signal lamp. The invention can determine the working state of the traffic signal lamp through the color information and the shape information of the traffic signal lamp, thereby improving the detection rate, the accuracy and the anti-interference capability of identification.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a traffic signal lamp recognition method and a traffic signal lamp recognition system.
Background
The detection and identification of the traffic signal lamp are key technologies in the field of automatic driving perception, and have very important research values. At present, a plurality of scholars at home and abroad carry out extensive research on the detection and identification of the traffic signal lamp, and provide effective methods for detecting and identifying the traffic signal lamp.
Firstly, the purpose of detecting a circular signal lamp is realized by utilizing a Lab color space by using Mahipa.R.Yelal and the like, the color space of an image is converted, the RGB color space is converted into the Lab color space, then the image is divided, a candidate region of the signal lamp is obtained by clustering, the signal lamp is identified by utilizing edge feature extraction, the robustness of an algorithm is enhanced by the conversion of the color space, the algorithm can obtain a good identification effect in a simple background environment, but the algorithm cannot meet the practical requirement when being applied to a complex background environment; secondly, Masako Omachi and the like propose a method for detecting a signal lamp by using Hough transform, and simultaneously combine edge extraction and pixel clustering methods, color normalization is carried out on an RGB color space before processing, then color segmentation is carried out, an area meeting the color characteristics of the traffic signal lamp is taken as a candidate area, edge extraction is carried out on the candidate area, and a round signal lamp is detected by Hough transform. The method basically meets the requirement of real-time performance, but has poor anti-interference capability, cannot effectively eliminate the interference of impurities such as a traffic sign board, an automobile tail lamp and the like, and is easily influenced by light, and Masako and the like propose a signal lamp detection algorithm based on template matching, firstly, a signal lamp template is defined, and the whole image is matched with the template to detect and identify the traffic signal lamp, the method is less influenced by color factors, but when the scene is complex or an interference area similar to the shape of the signal lamp exists, errors are easily generated, and the algorithm calculation amount is large; fourthly, Kuo Hao Lu and the like provide a new method for detecting and identifying traffic lights, the method can identify not only round traffic lights, but also arrow-shaped traffic lights, the algorithm adopts HSI color space, image segmentation is carried out by using a fixed threshold value, the two types of traffic lights are respectively identified by using the geometric characteristics of the traffic lights, although the detection of the two types of traffic lights is realized, the algorithm only detects according to the geometric characteristics, so that the algorithm has higher false detection rate and omission ratio.
In summary, the identification of the traffic signal lamp can be roughly divided into two research directions, namely, the identification of the traffic signal lamp based on the color information and the identification of the traffic signal lamp based on the shape information, wherein the former can achieve a better identification effect on an image with the color of the traffic signal lamp obviously and strongly contrasted with the background environment, but is greatly influenced by light rays and cannot be applied to the detection of the traffic signal lamp in the complex background environment; although the latter can overcome the influence of factors such as color blurring and light unevenness, the detection may fail due to the fact that the signal lamp or the interferent with similar shapes in the environment is partially shielded, and accurate identification of the traffic signal lamp cannot be achieved only according to the shape characteristics.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the art described above. Therefore, an object of the present invention is to provide a traffic signal lamp recognition method, which can recognize a traffic signal lamp by using color information and shape information of the traffic signal lamp, thereby improving a detection rate, accuracy and interference resistance of recognition.
A second object of the present invention is to provide an identification system for traffic signal lamps.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides a method for identifying a traffic signal lamp, including the following steps: acquiring image data containing a traffic signal lamp; obtaining a target image of the traffic signal lamp according to the image data; obtaining shape information and color information of the traffic signal lamp and corresponding credibility according to the target image; judging whether the reliability is greater than a set threshold value; if the reliability is greater than the set threshold, determining the working state of the traffic signal lamp according to the shape information and the color information; if the reliability is less than or equal to the set threshold, performing image processing on the target image; and obtaining the color information and the shape information of the traffic signal lamp according to the processed target image to determine the working state of the traffic signal lamp.
According to the traffic signal lamp identification method provided by the embodiment of the invention, the image data including the traffic signal lamp is obtained, the target image of the traffic signal lamp is obtained according to the image data, the shape information and the color information of the traffic signal lamp are obtained according to the target image, the corresponding reliability is obtained, and whether the reliability is greater than the set threshold is judged, wherein if the reliability is greater than the set threshold, the working state of the traffic signal lamp is determined according to the shape information, if the reliability is less than or equal to the set threshold, the image processing is carried out on the target image, and finally the working state of the traffic signal lamp is determined according to the processed target image.
In addition, the identification method of the traffic signal lamp provided by the above embodiment of the invention may further have the following additional technical features:
according to an embodiment of the present invention, obtaining the target image of the traffic signal lamp according to the image data specifically includes: and positioning the position area of the traffic signal lamp in the image data by adopting a deep learning algorithm, and intercepting the position area of the traffic signal lamp to be used as a target image of the traffic signal lamp.
According to an embodiment of the present invention, obtaining shape information and color information of the traffic signal lamp according to the target image, and corresponding reliability specifically includes: and classifying the target image by adopting a deep learning model to obtain the shape information and the color information of the traffic signal lamp and the corresponding reliability.
According to one embodiment of the invention, the image processing is performed on the target image, and comprises the steps of performing binarization and denoising processing on the target image.
According to an embodiment of the present invention, obtaining the color information and the shape information of the traffic signal according to the processed target image to determine the operating state of the traffic signal specifically includes: obtaining the maximum outline of the traffic signal lamp according to the processed target image; calculating the outline centroid of the maximum outline of the traffic signal lamp to obtain the color information of the traffic signal lamp; acquiring a template image of the traffic signal lamp; carrying out image processing on the traffic signal lamp template image; obtaining the maximum outline of the traffic signal lamp template image according to the processed traffic signal lamp template image; calculating Hu moment values of the maximum contour of the traffic signal lamp and the maximum contour of the template image of the traffic signal lamp; judging the shape information of the traffic signal lamp according to the Hu moment value; and determining the working state of the traffic signal lamp according to the color information and the shape information of the traffic signal lamp.
According to one embodiment of the invention, the deep learning algorithm is yolov3 deep learning algorithm.
According to one embodiment of the invention, the deep learning model is the Resnet18 network model.
According to an embodiment of the invention, the threshold interval of the set threshold is 0.8-0.85.
In order to achieve the above object, a second embodiment of the present invention provides an identification system for a traffic signal lamp, including: the system comprises an image acquisition module, a traffic signal lamp acquisition module and a traffic signal lamp acquisition module, wherein the image acquisition module is used for acquiring image data containing the traffic signal lamp; the image detection module is used for obtaining a target image of the traffic signal lamp according to the image data; the first image processing module is used for obtaining shape information and color information of the traffic signal lamp and corresponding credibility according to the target image; the judging module is used for judging whether the reliability is greater than a set threshold value or not; the first output module is used for determining the working state of the traffic signal lamp according to the shape information and the color information if the reliability is greater than the set threshold value; the second image processing module is used for processing the target image through the second image processing module if the reliability is less than or equal to the set threshold; and the second output module is used for obtaining the color information and the shape information of the traffic signal lamp according to the processed target image so as to determine the working state of the traffic signal lamp.
According to the identification system of the traffic signal lamp provided by the embodiment of the invention, the image data containing the traffic signal lamp is acquired through the image acquisition module, the target image of the traffic signal lamp is acquired through the image detection module according to the image data, the shape information, the color information and the reliability of the traffic signal lamp are acquired through the first image processing module according to the target image, and whether the reliability is greater than the set threshold value is judged through the judgment module, wherein if the reliability is greater than the set threshold value, the working state of the traffic signal lamp is determined through the first output module according to the shape information and the color information, if the reliability is less than or equal to the set threshold value, the second image processing module is used for carrying out image processing on the target image, and finally, the working state of the traffic signal lamp is determined through the second output module according to the processed target image, so that the traffic signal lamp can be identified through the color information and the shape, therefore, the detection rate, accuracy and anti-interference capability of identification can be improved.
Drawings
FIG. 1 is a flow chart of a method of identifying a traffic signal according to an embodiment of the present invention;
FIG. 2 is a graph of image data including a traffic signal light according to one embodiment of the present invention;
FIG. 3 is a target image of a traffic signal in accordance with one embodiment of the present invention;
FIG. 4(a) is an image after binarization processing of a target image according to an embodiment of the invention;
FIG. 4(b) is a diagram illustrating an image after denoising of a target image according to an embodiment of the present invention;
FIG. 5 is a flow chart of a method of identifying traffic signal lights in accordance with one embodiment of the present invention;
fig. 6 is a block diagram illustrating an identification system of a traffic signal according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart of an identification method of a traffic signal lamp according to an embodiment of the present invention.
As shown in fig. 1, the method for identifying a traffic signal lamp according to the embodiment of the present invention includes the following steps:
s1, image data including traffic lights is acquired.
In one embodiment of the present invention, image data including a traffic signal, such as the image data including the traffic signal a shown in fig. 2, may be acquired through an onboard camera.
And S2, obtaining the target image of the traffic signal lamp according to the image data.
In one embodiment of the invention, a deep learning algorithm can be used to locate the position area of the traffic light in the image data, and the position area of the traffic light is intercepted as the target image of the traffic light. Specifically, the yolov3 deep learning algorithm may be used to locate the position region of traffic light a in the image data shown in fig. 2, and the position region of traffic light a in the image data shown in fig. 2 may be cut out as the target image of traffic light a, for example, the ROI image of traffic light a shown in fig. 3. The target image of the traffic signal lamp can be accurately extracted through the yolov3 deep learning algorithm, so that the influence of background factors in image data can be avoided.
And S3, obtaining the shape information and the color information of the traffic signal lamp and the corresponding credibility according to the target image.
In one embodiment of the invention, the deep learning model can be used for classifying the target images so as to obtain the shape information and the color information of the traffic signal lamp and the corresponding reliability. Specifically, the Resnet18 network model may be used to classify the target image to obtain the color information and shape information of the traffic signal in the target image, and the corresponding confidence level. The reliability is a judgment basis of the shape information and the color information accuracy of the traffic signal lamp, namely the color information of the traffic signal lamp, namely the red, green and yellow color information and the shape information, namely the driving direction pointing icon, for example, whether the information of the green light and the left turn pointing icon is accurate or not can be judged through the reliability.
S4, it is determined whether the reliability is greater than the set threshold.
In an embodiment of the present invention, the threshold interval of the set threshold may be 0.8-0.85, and the specific value of the set threshold may be selected within the threshold interval according to the actual requirement, for example, the set threshold may be selected to be 0.8, i.e. whether the confidence level is greater than 0.8 may be determined.
And S5, if the reliability is higher than the set threshold, determining the working state of the traffic signal lamp according to the shape information and the color information.
In one embodiment of the present invention, if the reliability is greater than the set threshold, for example, the reliability is greater than 0.8, it may be determined that the obtained traffic signal color information, i.e., the red, green, and yellow color information and the shape information, i.e., the driving direction pointing icon, e.g., the green light and the left turn pointing icon information, are accurately available, so that the operating state of the traffic signal may be determined according to the color information and the shape information of the traffic signal, i.e., the current traffic signal is the green light and the driving direction pointing icon is the left turn.
S6, if the reliability is equal to or less than the set threshold, the target image is subjected to image processing.
In one embodiment of the present invention, if the confidence level is less than or equal to the set threshold, for example, the confidence level is less than or equal to 0.8, the image processing may be performed on the target image, for example, the ROI image of the traffic signal a shown in fig. 3. Specifically, binarization and denoising processing may be performed on a target image, for example, an ROI image of a traffic signal a shown in fig. 3, so that a binarized image shown in fig. 4(a) and a denoised image shown in fig. 4(b) may be obtained.
And S7, obtaining the color information and the shape information of the traffic signal lamp according to the processed target image to determine the working state of the traffic signal lamp.
Specifically, the step S7 includes: obtaining the maximum outline of the traffic signal lamp according to the processed target image; calculating the outline centroid of the maximum outline of the traffic signal lamp to obtain the color information of the traffic signal lamp; acquiring a template image of a traffic signal lamp; carrying out image processing on the traffic signal lamp template image; obtaining the maximum outline of the traffic signal lamp template image according to the processed traffic signal lamp template image; calculating Hu moment values of the maximum outline of the traffic signal lamp and the maximum outline of the traffic signal lamp template image; judging the shape information of the traffic signal lamp according to the Hu moment value; and determining the working state of the traffic signal lamp according to the color information and the shape information of the traffic signal lamp.
The acquiring the template image of the traffic signal lamp and the image processing of the template image of the traffic signal lamp may further include: the template image related to the traffic signal is acquired, for example, the template image related to the traffic signal shown in fig. 3 may be acquired, and binarization and denoising processing may be performed on the acquired template image of the traffic signal.
In order to more clearly and completely illustrate the method for identifying a traffic signal according to the embodiment of the present invention, an overall implementation process of the method for identifying a traffic signal according to the embodiment of the present invention will be described with reference to fig. 5.
As shown in fig. 5, the method for identifying a traffic signal lamp according to the embodiment of the present invention includes the following steps:
s01, collecting images, specifically collecting RGB images containing traffic lights;
s02, detecting images, specifically adopting yolov3 deep learning algorithm to detect traffic lights;
s03, intercepting the target image, specifically intercepting an ROI image of the traffic signal lamp;
s04, processing the target image, specifically, classifying the target image by adopting a Resnet18 network model to obtain the shape information and the reliability of the traffic signal lamp;
s05, judging whether the reliability of the traffic light is larger than a set threshold value, if so, executing a step S13, and if not, executing steps S06 and S09;
s06, carrying out image processing on the target image, specifically carrying out binarization and denoising processing;
s07, obtaining the maximum outline of the traffic signal lamp according to the processed target image;
s08, judging the color information of the traffic signal lamp according to the maximum outline of the traffic signal lamp, and specifically calculating the outline centroid of the maximum outline of the traffic signal lamp to judge the color information of the traffic signal lamp, namely red, yellow and green color information;
s09, acquiring a traffic signal lamp template image;
s10, carrying out image processing on the traffic signal lamp template image, specifically carrying out binarization and denoising processing;
s11, obtaining the maximum outline of the traffic signal lamp template image according to the processed traffic signal lamp template image;
s12, judging the shape information of the traffic signal lamp according to the maximum outline of the traffic signal lamp template image and the maximum outline of the traffic signal lamp, specifically performing outline matching, namely calculating the Hu moment value of the maximum outline of the traffic signal lamp and the maximum outline of the traffic signal lamp template image to judge the shape information of the traffic signal lamp, namely, straight running or left turning;
and S13, outputting the color information and the shape information of the traffic signal lamp to determine the working state of the traffic signal lamp.
According to the traffic signal lamp identification method provided by the embodiment of the invention, the image data including the traffic signal lamp is obtained, the target image of the traffic signal lamp is obtained according to the image data, the shape information and the color information of the traffic signal lamp are obtained according to the target image, the corresponding reliability is obtained, and whether the reliability is greater than the set threshold is judged, wherein if the reliability is greater than the set threshold, the working state of the traffic signal lamp is determined according to the shape information, if the reliability is less than or equal to the set threshold, the image processing is carried out on the target image, and finally the working state of the traffic signal lamp is determined according to the processed target image.
Corresponding to the method for identifying the traffic signal lamp provided by the embodiment, the invention further provides a system for identifying the traffic signal lamp.
As shown in fig. 6, the traffic signal light recognition system according to the embodiment of the present invention includes an image acquisition module 10, an image detection module 20, a first image processing module 30, a determination module 40, a first output module 50, a second image processing module 60, and a second output module 70.
The image acquisition module 10 is configured to acquire image data including a traffic signal lamp; the image detection module 20 is configured to obtain a target image of the traffic signal lamp according to the image data; the first image processing module 30 is configured to obtain shape information and color information of the traffic signal lamp and corresponding reliability according to the target image; the judging module 40 is configured to judge whether the reliability is greater than a set threshold; if the reliability is greater than the set threshold, determining the working state of the traffic signal lamp through the first output module 50 according to the shape information and the color information; if the reliability is less than or equal to the set threshold, performing image processing on the target image through the second image processing module 60; the second output module 70 obtains the color information and the shape information of the traffic signal lamp according to the processed target image to determine the working state of the traffic signal lamp.
In one embodiment of the present invention, the image capturing module 10 may be an onboard camera, and the image capturing module 10, i.e., the onboard camera, may capture image data including a traffic signal, such as the image data including the traffic signal a shown in fig. 2.
In an embodiment of the present invention, the image detection module 20 may use a deep learning algorithm to locate the position region of the traffic light in the image data, and intercept the position region of the traffic light as the target image of the traffic light. Specifically, the image detection module 20 may use the yolov3 deep learning algorithm to locate the position region of the traffic signal a in the image data shown in fig. 2, and may intercept the position region of the traffic signal a in the image data shown in fig. 2 as a target image of the traffic signal a, for example, an ROI image of the traffic signal a shown in fig. 3. The target image of the traffic signal lamp can be accurately extracted through the yolov3 deep learning algorithm, so that the influence of background factors in image data can be avoided.
In one embodiment of the present invention, the first image processing module 30 may include a deep learning model, and the target image may be classified by the deep learning model to obtain shape information and color information of the traffic signal, and corresponding reliability. Specifically, the deep learning model may be a Resnet18 network model, and the target image may be classified by the Resnet18 network model to obtain color information and shape information of a traffic signal in the target image, and corresponding reliability. The reliability is a judgment basis of the shape information and the color information accuracy of the traffic signal lamp, namely the color information of the traffic signal lamp, namely the red, green and yellow color information and the shape information, namely the driving direction pointing icon, for example, whether the information of the green light and the left turn pointing icon is accurate or not can be judged through the reliability.
In an embodiment of the present invention, the threshold interval for setting the threshold in the determining module 40 may be 0.8-0.85, and the specific value of the threshold may be selected according to the actual requirement in the threshold interval, for example, the threshold may be selectively set to 0.8, that is, whether the confidence level is greater than 0.8 may be determined by the determining module 40.
In an embodiment of the present invention, if the reliability is greater than the set threshold, for example, the reliability is greater than 0.8, the color information of the traffic signal, i.e., the color information of red, green and yellow and the shape information, i.e., the information of the driving direction indicator icon, e.g., the information of the green light and the left turn indicator icon, are accurately available through the first output module 50, so that the operating state of the traffic signal can be determined according to the color information and the shape information of the traffic signal, i.e., the current traffic signal is the green light and the driving direction indicator icon is the left turn.
In an embodiment of the present invention, if the reliability is less than or equal to a set threshold, for example, the reliability is less than or equal to 0.8, the target image, for example, the ROI image of the traffic signal a shown in fig. 3, may be subjected to image processing by the second image processing module 60. Specifically, the second image processing module 60 may perform binarization and denoising processing on a target image, for example, the ROI image of the traffic signal a shown in fig. 3, so as to obtain a binarized image shown in fig. 4(a) and a denoised image shown in fig. 4 (b).
In one embodiment of the present invention, the second output module 70 may obtain color information and shape information of the traffic signal to determine an operation state of the traffic signal, specifically, the maximum outline of the traffic signal lamp can be obtained according to the processed target image, the outline centroid of the maximum outline of the traffic signal lamp can be calculated to obtain the color information of the traffic signal lamp, and can obtain the template image of the traffic signal lamp and process the image of the template image of the traffic signal lamp, and the maximum outline of the traffic signal lamp template image can be obtained according to the processed traffic signal lamp template image, the Hu moment values of the maximum contour of the traffic signal and the maximum contour of the traffic signal template image can then be calculated, and the shape information of the traffic signal lamp can be judged according to the Hu moment value, and finally the working state of the traffic signal lamp can be determined according to the color information and the shape information of the traffic signal lamp.
The acquiring the template image of the traffic signal lamp and the image processing of the template image of the traffic signal lamp may further include: the template image related to the traffic signal is acquired, for example, the template image related to the traffic signal shown in fig. 3 may be acquired, and binarization and denoising processing may be performed on the acquired template image of the traffic signal.
In order to more clearly and completely illustrate the operation of the identification system of the traffic signal lamp according to the embodiment of the present invention, the overall implementation process of the identification system of the traffic signal lamp according to the embodiment of the present invention will be described with reference to fig. 5.
As shown in fig. 5, the method comprises the following steps:
s01, collecting images, specifically collecting RGB images including traffic lights through the image acquisition module 10;
s02, detecting images, specifically, detecting traffic lights by the image detection module 20 by adopting yolov3 deep learning algorithm;
s03, intercepting the target image, specifically intercepting an ROI image of the traffic signal lamp through the image detection module 20;
s04, processing the target image, specifically, classifying the target image by the first image processing module 30 by using a Resnet18 network model to obtain the shape information and the reliability of the traffic signal lamp;
s05, determining whether the reliability of the traffic light is greater than a predetermined threshold, and particularly determining whether the reliability of the traffic light is greater than the predetermined threshold by the determining module 40, if so, performing the step S13 by the first output module 50, otherwise, performing the step S06 by the second image processing module 60, and performing the step S09 by the second output module 70;
s06, performing image processing on the target image, specifically performing binarization and denoising processing by the second image processing module 60;
s07, the maximum outline of the traffic signal lamp can be obtained through the second output module 70 according to the processed target image;
s08, the color information of the traffic signal lamp can be judged through the second output module 70 according to the maximum outline of the traffic signal lamp, and the outline centroid of the maximum outline of the traffic signal lamp can be specifically calculated to judge the color information of the traffic signal lamp, namely the red, yellow and green color information;
s09, the traffic signal lamp template image can be obtained through the second output module 70;
s10, the traffic signal lamp template image can be processed through the second output module 70, and specifically, binarization and denoising processing can be carried out;
s11, the maximum outline of the traffic signal lamp template image can be obtained through the second output module 70 according to the processed traffic signal lamp template image;
s12, the second output module 70 can judge the shape information of the traffic signal lamp according to the maximum outline of the traffic signal lamp template image and the maximum outline of the traffic signal lamp, and specifically can carry out outline matching, namely calculating the Hu moment value of the maximum outline of the traffic signal lamp and the maximum outline of the traffic signal lamp template image to judge the shape information of the traffic signal lamp, namely, straight going or left turning;
s13, the color information and the shape information of the traffic signal may be output through the second output module 70 to determine the operation state of the traffic signal.
According to the identification system of the traffic signal lamp provided by the embodiment of the invention, the image data containing the traffic signal lamp is acquired through the image acquisition module, the target image of the traffic signal lamp is acquired through the image detection module according to the image data, the shape information, the color information and the reliability of the traffic signal lamp are acquired through the first image processing module according to the target image, and whether the reliability is greater than the set threshold value is judged through the judgment module, wherein if the reliability is greater than the set threshold value, the working state of the traffic signal lamp is determined through the first output module according to the shape information and the color information, if the reliability is less than or equal to the set threshold value, the second image processing module is used for carrying out image processing on the target image, and finally, the working state of the traffic signal lamp is determined through the second output module according to the processed target image, so that the traffic signal lamp can be identified through the color information and the shape, thereby being capable of improving the detection rate and accuracy of identification.
In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The meaning of "plurality" is two or more unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (9)
1. A method for identifying a traffic signal lamp is characterized by comprising the following steps:
acquiring image data containing a traffic signal lamp;
obtaining a target image of the traffic signal lamp according to the image data;
obtaining shape information and color information of the traffic signal lamp and corresponding credibility according to the target image;
judging whether the reliability is greater than a set threshold value;
if the reliability is greater than the set threshold, determining the working state of the traffic signal lamp according to the shape information and the color information;
if the reliability is less than or equal to the set threshold, performing image processing on the target image;
and obtaining the color information and the shape information of the traffic signal lamp according to the processed target image to determine the working state of the traffic signal lamp.
2. The method for identifying a traffic signal according to claim 1, wherein obtaining the target image of the traffic signal according to the image data specifically comprises:
and positioning the position area of the traffic signal lamp in the image data by adopting a deep learning algorithm, and intercepting the position area of the traffic signal lamp to be used as a target image of the traffic signal lamp.
3. The method for identifying a traffic signal lamp as claimed in claim 2, wherein obtaining the shape information and the color information of the traffic signal lamp and the corresponding reliability according to the target image specifically comprises:
and classifying the target image by adopting a deep learning model to obtain the shape information and the color information of the traffic signal lamp and the corresponding reliability.
4. The method for identifying a traffic signal lamp as claimed in claim 3, wherein the image processing of the target image comprises binarization and de-noising of the target image.
5. The method for identifying a traffic signal lamp according to claim 4, wherein obtaining the color information and the shape information of the traffic signal lamp according to the processed target image to determine the operating state of the traffic signal lamp specifically comprises:
obtaining the maximum outline of the traffic signal lamp according to the processed target image;
calculating the outline centroid of the maximum outline of the traffic signal lamp to obtain the color information of the traffic signal lamp;
acquiring a template image of the traffic signal lamp;
carrying out image processing on the traffic signal lamp template image;
obtaining the maximum outline of the traffic signal lamp template image according to the processed traffic signal lamp template image;
calculating Hu moment values of the maximum contour of the traffic signal lamp and the maximum contour of the template image of the traffic signal lamp;
judging the shape information of the traffic signal lamp according to the Hu moment value;
and determining the working state of the traffic signal lamp according to the color information and the shape information of the traffic signal lamp.
6. The method for identifying a traffic signal lamp as claimed in claim 5, wherein the deep learning algorithm is yolov3 deep learning algorithm.
7. The method for identifying a traffic signal as recited in claim 5, wherein the deep learning model is a Resnet18 network model.
8. The method for identifying a traffic signal lamp as claimed in claim 5, wherein the threshold interval of the set threshold is 0.8-0.85.
9. An identification system for a traffic signal, comprising:
the system comprises an image acquisition module, a traffic signal lamp acquisition module and a traffic signal lamp acquisition module, wherein the image acquisition module is used for acquiring image data containing the traffic signal lamp;
the image detection module is used for obtaining a target image of the traffic signal lamp according to the image data;
the first image processing module is used for obtaining shape information and color information of the traffic signal lamp and corresponding credibility according to the target image;
the judging module is used for judging whether the reliability is greater than a set threshold value or not;
the first output module is used for determining the working state of the traffic signal lamp according to the shape information and the color information if the reliability is greater than the set threshold value;
the second image processing module is used for processing the target image through the second image processing module if the reliability is less than or equal to the set threshold;
and the second output module is used for obtaining the color information and the shape information of the traffic signal lamp according to the processed target image so as to determine the working state of the traffic signal lamp.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011262208.2A CN112380973B (en) | 2020-11-12 | 2020-11-12 | Traffic signal lamp identification method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011262208.2A CN112380973B (en) | 2020-11-12 | 2020-11-12 | Traffic signal lamp identification method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112380973A true CN112380973A (en) | 2021-02-19 |
| CN112380973B CN112380973B (en) | 2023-06-23 |
Family
ID=74583281
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011262208.2A Active CN112380973B (en) | 2020-11-12 | 2020-11-12 | Traffic signal lamp identification method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112380973B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113343873A (en) * | 2021-06-17 | 2021-09-03 | 湖北亿咖通科技有限公司 | Signal lamp identification method, device, equipment, medium and product |
| CN114863386A (en) * | 2022-03-30 | 2022-08-05 | 广州文远知行科技有限公司 | Traffic signal lamp detection method and device and electronic equipment |
| CN116152785A (en) * | 2023-04-24 | 2023-05-23 | 之江实验室 | Traffic signal lamp detection method and system based on CBAM combined with HSB mode |
| CN116152784A (en) * | 2023-04-21 | 2023-05-23 | 深圳市夜行人科技有限公司 | Signal lamp early warning method and system based on image processing |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103824091A (en) * | 2014-02-27 | 2014-05-28 | 中国石油大学(华东) | Vehicle license plate recognition method for intelligent transportation system |
| CN107463946A (en) * | 2017-07-12 | 2017-12-12 | 浙江大学 | A kind of type of merchandize detection method of combination template matches and deep learning |
| CN109063221A (en) * | 2018-11-02 | 2018-12-21 | 北京百度网讯科技有限公司 | Query intention recognition methods and device based on mixed strategy |
| CN109389079A (en) * | 2018-09-30 | 2019-02-26 | 无锡职业技术学院 | A kind of traffic lights recognition methods |
| US20200250487A1 (en) * | 2017-10-23 | 2020-08-06 | Hangzhou Hikvision Digital Technology Co., Ltd. | Target detection method and apparatus, and computer device |
| CN111680688A (en) * | 2020-06-10 | 2020-09-18 | 创新奇智(成都)科技有限公司 | Character recognition method and device, electronic equipment and storage medium |
-
2020
- 2020-11-12 CN CN202011262208.2A patent/CN112380973B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103824091A (en) * | 2014-02-27 | 2014-05-28 | 中国石油大学(华东) | Vehicle license plate recognition method for intelligent transportation system |
| CN107463946A (en) * | 2017-07-12 | 2017-12-12 | 浙江大学 | A kind of type of merchandize detection method of combination template matches and deep learning |
| US20200250487A1 (en) * | 2017-10-23 | 2020-08-06 | Hangzhou Hikvision Digital Technology Co., Ltd. | Target detection method and apparatus, and computer device |
| CN109389079A (en) * | 2018-09-30 | 2019-02-26 | 无锡职业技术学院 | A kind of traffic lights recognition methods |
| CN109063221A (en) * | 2018-11-02 | 2018-12-21 | 北京百度网讯科技有限公司 | Query intention recognition methods and device based on mixed strategy |
| CN111680688A (en) * | 2020-06-10 | 2020-09-18 | 创新奇智(成都)科技有限公司 | Character recognition method and device, electronic equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| M SWATHI等: "Automatic traffic sign detection and recognition: A review", 《 2017 INTERNATIONAL CONFERENCE ON ALGORITHMS, METHODOLOGY, MODELS AND APPLICATIONS IN EMERGING TECHNOLOGIES》 * |
| 张旭: "基于模板匹配和深度学习的港口舰船检测识别方法", 《信息技术与信息化》 * |
| 邢金金: "结合面向对象深度学习分类与模板匹配的道路网提取研究", 《中国硕士学位论文全文数据库》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113343873A (en) * | 2021-06-17 | 2021-09-03 | 湖北亿咖通科技有限公司 | Signal lamp identification method, device, equipment, medium and product |
| CN114863386A (en) * | 2022-03-30 | 2022-08-05 | 广州文远知行科技有限公司 | Traffic signal lamp detection method and device and electronic equipment |
| CN116152784A (en) * | 2023-04-21 | 2023-05-23 | 深圳市夜行人科技有限公司 | Signal lamp early warning method and system based on image processing |
| CN116152785A (en) * | 2023-04-24 | 2023-05-23 | 之江实验室 | Traffic signal lamp detection method and system based on CBAM combined with HSB mode |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112380973B (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112380973B (en) | Traffic signal lamp identification method and system | |
| CN113096130B (en) | Method and device for detecting object defects | |
| CN110889974B (en) | Intelligent parking space identification method and device and automobile | |
| US8686892B2 (en) | Synthetic aperture radar chip level cross-range streak detector | |
| US10339396B2 (en) | Vehicle accessibility determination device | |
| CN107798688B (en) | Moving target identification method, early warning method and automobile rear-end collision prevention early warning device | |
| US9261585B2 (en) | Radar apparatus using image change detector and method of operating the same | |
| CN108647664B (en) | Lane line detection method based on look-around image | |
| CN109300104B (en) | Angular point detection method and device | |
| CN111047615A (en) | Image-based line detection method and device and electronic equipment | |
| CN106296670A (en) | A kind of Edge detection of infrared image based on Retinex watershed Canny operator | |
| CN113820333B (en) | Battery pole piece abnormality detection method, device, upper computer and detection system | |
| CN109858438A (en) | A kind of method for detecting lane lines based on models fitting | |
| CN105787475A (en) | Traffic sign detection and identification method under complex environment | |
| CN112232246A (en) | Garbage detection and classification method and device based on deep learning | |
| CN114882470A (en) | Vehicle-mounted anti-collision early warning method and device, computer equipment and storage medium | |
| CN108520252B (en) | Road sign identification method based on generalized Hough transform and wavelet transform | |
| CN114620030A (en) | Parking performance testing method and device, server and product | |
| CN112885108B (en) | Vehicle change detection method and system on parking space based on deep learning algorithm | |
| CN108268866B (en) | Vehicle detection method and system | |
| CN112261402A (en) | Image detection method and system and camera shielding monitoring method and system | |
| US10621466B2 (en) | Method for extracting features of a thermal image | |
| CN113326749B (en) | Target detection method and device, storage medium and electronic equipment | |
| JP3505362B2 (en) | Vehicle detection method | |
| CN110853059B (en) | Image identification method for target ring number |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |