CN114120286A - Traffic signal lamp identification method and device - Google Patents
Traffic signal lamp identification method and device Download PDFInfo
- Publication number
- CN114120286A CN114120286A CN202111454157.8A CN202111454157A CN114120286A CN 114120286 A CN114120286 A CN 114120286A CN 202111454157 A CN202111454157 A CN 202111454157A CN 114120286 A CN114120286 A CN 114120286A
- Authority
- CN
- China
- Prior art keywords
- traffic signal
- signal lamp
- color
- shape
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4084—Transform-based scaling, e.g. FFT domain scaling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a traffic signal lamp identification method and a device, after an image to be identified in a traffic signal lamp target area is obtained, a characteristic extraction network in a traffic signal lamp identification model is used for extracting a characteristic diagram of the image to be identified, and the characteristic diagram is respectively input into a color identification branch network and a shape identification branch network in the traffic signal lamp identification model to realize accurate identification of the color and the shape of a luminous traffic signal lamp.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for identifying traffic signal lamps.
Background
Traffic signal lamp identification is a key technology in the field of intelligent driving, and at present, an identification method for a traffic signal lamp mainly comprises the following steps: feature-based methods and short-range communication techniques based on vehicles and traffic lights.
In the feature-based method, a method based on features such as a chromaticity space and the like, for example, an ellipsoid geometric threshold model is established in an HSV chromaticity space to extract an interested color region, and then two heterogeneous features for describing traffic light candidate regions are combined by using a kernel function, so that the method cannot adapt to actual scenes under different brightness conditions caused by different weather; the method based on the aspect ratio, the area, the position and the context of the traffic signal lamp (namely the surrounding environment information of the traffic signal lamp) has higher detection accuracy on a special scene because the method is the characteristic designed and used under one or more conditions, but the characteristic design methods are based on the prior knowledge of researchers, so that the design methods cannot cope with complex and various practical scenes, namely the robustness on the actual scene is poor.
Based on short-range communication technology of vehicles and traffic lights, infrastructure needs to be modified, such as adding V2I equipment to the traffic lights. This modification is costly and not possible in the short term.
Disclosure of Invention
In view of the above, the invention provides a traffic signal lamp identification method and device, which do not need to modify a traffic signal lamp, do not need to manually design traffic signal lamp characteristics by using prior knowledge, and have strong robustness for complex real scenes.
In order to achieve the above purpose, the invention provides the following specific technical scheme:
a method of identifying a traffic signal, comprising:
acquiring an image to be identified, wherein the image to be identified comprises a target area of a traffic signal lamp;
utilizing a feature extraction network in a traffic signal lamp recognition model to extract features of the image to be recognized, and obtaining a feature map corresponding to the image to be recognized;
and respectively inputting the characteristic diagrams into the color recognition branch network and the shape recognition branch network in the traffic signal lamp recognition model to obtain a color recognition result and a shape recognition result of the luminous traffic signal lamp.
Optionally, the acquiring the image to be recognized includes:
acquiring a target image including a detection frame obtained after the traffic signal lamp detection;
and cutting the target image according to the detection frame, and zooming the cut image to obtain the image to be recognized of the target area including the traffic signal lamp.
Optionally, the construction method of the traffic signal light recognition model is as follows:
acquiring sample images of traffic lights in different application scenes, wherein the application scenes comprise at least one of lane conditions, traffic light types, weather conditions and time conditions;
marking the color and the shape of the luminous traffic signal lamp in the sample image;
preprocessing a sample image to obtain a training sample and a test sample;
and training the self-learning feature extraction network, the color recognition branch network and the shape recognition branch network by using the training sample to obtain the traffic signal lamp recognition model by taking the recognition result approaching to the color and the shape marked in the training sample as a target.
Optionally, the preprocessing the sample image to obtain a training sample and a test sample includes:
carrying out traffic light detection on the sample image to obtain a sample image comprising a detection frame;
randomly expanding the detection frames in the sample image according to a preset proportion;
cutting the sample image according to the detection frame in the sample image, and carrying out scaling processing on the cut image to obtain a sample image of a target area including a traffic signal lamp;
carrying out data equalization processing on the sample image to obtain a sample image with balanced color type and balanced shape type of the luminous traffic signal lamp;
dividing a sample image into the training sample and the test sample.
Optionally, the method further includes:
testing the traffic signal lamp identification model by using the test sample to obtain the color identification accuracy and the shape identification accuracy of the traffic signal lamp identification model;
and setting a color confidence coefficient threshold value and a shape confidence coefficient threshold value according to the color recognition accuracy and the shape recognition accuracy of the traffic signal lamp recognition model.
Optionally, after obtaining the color recognition result and the shape recognition result of the luminous traffic signal lamp, the method further includes:
inputting the color recognition result comprising each color probability value into a first status register, wherein the color of the luminous traffic signal lamp output by the first status register is consistent with the color corresponding to the maximum probability value under the condition that the maximum probability value in the color recognition result is greater than the color confidence coefficient threshold value, and the color of the luminous traffic signal lamp output by the first status register is consistent with the last output result under the condition that the maximum probability in the color recognition result is not greater than the color confidence coefficient threshold value;
and inputting the shape recognition result comprising each shape probability value into a second state register, wherein the shape of the luminous traffic signal lamp output by the second state register is consistent with the shape corresponding to the maximum probability value under the condition that the maximum probability value in the shape recognition result is greater than the shape confidence coefficient threshold value, and the shape of the luminous traffic signal lamp output by the second state register is consistent with the last output result under the condition that the maximum probability in the shape recognition result is not greater than the shape confidence coefficient threshold value.
An identification device for a traffic signal, comprising:
the device comprises an image to be identified acquisition unit, a traffic signal lamp identification unit and a traffic signal lamp identification unit, wherein the image to be identified acquisition unit is used for acquiring an image to be identified, and the image to be identified comprises a target area of the traffic signal lamp;
the characteristic extraction unit is used for extracting the characteristics of the image to be recognized by utilizing a characteristic extraction network in a traffic signal lamp recognition model to obtain a characteristic diagram corresponding to the image to be recognized;
and the attribute identification unit is used for respectively inputting the characteristic diagram into the color identification branch network and the shape identification branch network in the traffic signal lamp identification model to obtain a color identification result and a shape identification result of the luminous traffic signal lamp.
Optionally, the image acquiring unit to be identified is specifically configured to:
acquiring a target image including a detection frame obtained after the traffic signal lamp detection;
and cutting the target image according to the detection frame, and zooming the cut image to obtain the image to be recognized of the target area including the traffic signal lamp.
Optionally, the apparatus further comprises:
the system comprises a sample image acquisition unit, a traffic signal acquisition unit and a traffic signal processing unit, wherein the sample image acquisition unit is used for acquiring sample images of traffic signal lamps in different application scenes, and the application scenes comprise at least one of lane conditions, traffic signal lamp types, weather conditions and time conditions;
the attribute marking unit is used for marking the color and the shape of the luminous traffic signal lamp in the sample image;
the sample preprocessing unit is used for preprocessing the sample image to obtain a training sample and a test sample;
and the model training unit is used for training the self-learning feature extraction network, the color recognition branch network and the shape recognition branch network by using the training sample to obtain the traffic signal lamp recognition model by taking the recognition result approaching to the color and the shape marked in the training sample as a target.
Optionally, the sample preprocessing unit is specifically configured to:
carrying out traffic light detection on the sample image to obtain a sample image comprising a detection frame;
randomly expanding the detection frames in the sample image according to a preset proportion;
cutting the sample image according to the detection frame in the sample image, and carrying out scaling processing on the cut image to obtain a sample image of a target area including a traffic signal lamp;
carrying out data equalization processing on the sample image to obtain a sample image with balanced color type and balanced shape type of the luminous traffic signal lamp;
dividing a sample image into the training sample and the test sample.
Optionally, the apparatus further comprises:
the model testing unit is used for testing the traffic signal lamp identification model by using the test sample to obtain the color identification accuracy and the shape identification accuracy of the traffic signal lamp identification model;
and the threshold setting unit is used for setting a color confidence threshold and a shape confidence threshold according to the color recognition accuracy and the shape recognition accuracy of the traffic signal lamp recognition model.
Optionally, the apparatus further comprises:
a color output unit, configured to input the color recognition result including each color probability value into a first status register, where, in a case where a maximum probability value in the color recognition result is greater than the color confidence threshold, a color of the luminous traffic signal lamp output by the first status register is consistent with a color corresponding to the maximum probability value, and, in a case where the maximum probability in the color recognition result is not greater than the color confidence threshold, the color of the luminous traffic signal lamp output by the first status register is consistent with a last output result;
a shape output unit, configured to input the shape recognition result including each shape probability value into a second status register, where, in a case where a maximum probability value in the shape recognition result is greater than the shape confidence threshold, a shape of the light-emitting traffic signal output by the second status register is consistent with a shape corresponding to the maximum probability value, and in a case where the maximum probability in the shape recognition result is not greater than the shape confidence threshold, the shape of the light-emitting traffic signal output by the second status register is consistent with a last output result.
Compared with the prior art, the invention has the following beneficial effects:
the invention discloses a traffic signal lamp identification method, which comprises the steps of extracting a characteristic diagram of an image to be identified by utilizing a characteristic extraction network in a traffic signal lamp identification model after acquiring the image to be identified in a traffic signal lamp target area, and accurately identifying the color and the shape of a luminous traffic signal lamp by respectively inputting the characteristic diagram into a multi-color identification branch network and a shape identification branch network in the traffic signal lamp identification model.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a schematic flow chart of an identification method for a traffic signal lamp according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a detection frame obtained by different traffic signal light detection methods according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a partial method flow of a traffic signal light recognition method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a partial method flow of a traffic signal light recognition method according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating a sample image after a random expansion of a detection frame according to an embodiment of the disclosure;
fig. 6 is a schematic structural diagram of an identification device of a traffic signal lamp according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a traffic signal lamp identification method based on computer vision, which can be applied to the field of automatic driving and used as a subsequent accurate identification scheme of any traffic signal lamp detection method, so that the color and the shape of a luminous traffic signal lamp are accurately identified, the transportability is strong, the traffic signal lamp is not required to be modified, the characteristics of the luminous traffic signal lamp are learned by self by taking the color and the shape of the luminous traffic signal lamp as targets, the characteristics of the traffic signal lamp are not required to be designed manually by utilizing priori knowledge, and the robustness is strong for complex real scenes.
Specifically, referring to fig. 1, the method for identifying a traffic signal lamp disclosed in this embodiment includes the following steps:
s101: and acquiring an image to be identified, wherein the image to be identified comprises a target area of the traffic signal lamp.
The image to be recognized is obtained after the traffic signal lamp detection, the traffic signal lamp detection is used for target positioning, namely, the image containing the traffic signal lamp is detected, but the target classification cannot be finely carried out, namely, the color and the shape of the traffic signal lamp cannot be recognized.
The traffic signal lamp detection can be any one of the existing detection algorithms, the method is not particularly limited, namely, the method is used as a subsequent accurate identification scheme of any one traffic signal lamp detection method, and the transportability is high.
Traffic lights are commonly called traffic lights, and the target area of the traffic lights is the area where the traffic lights are located.
Specifically, one optional implementation manner for acquiring the image to be recognized is as follows:
referring to fig. 2, fig. 2 is a schematic view of a detection frame obtained by using different traffic signal lamp detection methods, where an area in the detection frame is a target area of a traffic signal lamp, and the detection frames obtained by using different traffic signal lamp detection methods are different.
And cutting the target image according to the detection frame, zooming the cut image to obtain an image to be recognized in a target area including the traffic signal lamp, reading the image to be recognized through Opencv, and zooming (Resize) to zoom the images with different widths and heights to the size required by unified model input.
When the detection method is used, the index value of the currently accepted detection evaluation method mapp (mean Average Precision) is determined by both Confidence Score and IoU (Intersection over Intersection ratio). Confidence focuses on the accuracy of the class, and IoU focuses on the coincidence of the detection frame with the minimum bounding rectangle of the detection target. For the practical problem of identification of the attributes (i.e., color and shape) of the traffic signal, the accuracy requirement of the detection frame is not high, i.e., IoU does not need to reach more than 95%, and 90% or even lower is possible, and as long as the detection frame contains necessary information of the color, shape, etc. of the traffic signal, accurate attributes of the traffic signal can be obtained based on the method.
As shown in fig. 2, the detection frames obtained by different traffic signal lamp detection methods are different, wherein the minimum circumscribed rectangle is the detection frame in the expected ideal situation. And actually, other detection frames do not excessively comprise other non-traffic signal lamp information, and the accurate attribute information of the traffic signal lamp can be obtained by passing the detection result through the method.
From the above analysis, it is easy to know that the method has low precision requirement on the detection frame, and for practical problems, it is important to obtain the attribute information of the traffic signal lamp rather than accurately obtain the minimum circumscribed rectangle of the traffic signal lamp.
S102: and performing feature extraction on the image to be recognized by using a feature extraction network in the traffic signal lamp recognition model to obtain a feature map corresponding to the image to be recognized.
The feature extraction network may be an Xception-based feature extraction network, or may be another neural network.
Xception is a feature extraction network implemented with depthwise partial convolution. The deep separable convolution is an algorithm obtained by improving standard convolution in a convolutional neural network, and by splitting the correlation between the space dimensionality and the channel dimensionality, the number of parameters of convolution calculation is reduced, and the use efficiency of convolution kernel parameters is improved.
S103: and respectively inputting the characteristic diagrams into the color recognition branch network and the shape recognition branch network in the traffic signal lamp recognition model to obtain a color recognition result and a shape recognition result of the luminous traffic signal lamp.
The color identification branch network outputs probability values of each color of the luminous traffic signal lamp, and the colors of the luminous traffic signal lamp comprise: red, yellow, green; the shape recognition branch network outputs probability values of each shape of the luminous traffic signal lamp, and the shape of the luminous traffic signal lamp comprises: straight running, left turning, right turning, turning around, pedestrians, non-motor vehicles and the like.
It should be noted that, before the traffic signal identification model is used for performing the traffic signal identification, the traffic signal identification model needs to be constructed, please refer to fig. 3, and the construction method of the traffic signal identification model provided in this embodiment is as follows:
s201: and acquiring sample images of the traffic signal lamp in different application scenes.
The application scene comprises at least one of lane conditions, traffic light types, weather conditions and time conditions, namely sample images of the traffic light under different lane conditions, different traffic light types, different weather conditions and different time conditions need to be acquired, and the diversity of the samples is increased. The time condition here refers to sample images acquired at different times, and can be distinguished according to the time stamps of the sample images.
S202: and marking the color and the shape of the luminous traffic signal lamp in the sample image.
Note that only the color and shape of the traffic signal light that emits light in the sample image are labeled, and the traffic signal light that does not emit light is not labeled.
S203: and preprocessing the sample image to obtain a training sample and a test sample.
The pre-processing may include: sample expansion processing, data equalization processing, scaling processing, and the like.
S204: and training the self-learning feature extraction network, the color recognition branch network and the shape recognition branch network by using the training sample to obtain the traffic signal lamp recognition model by taking the recognition result approaching to the color and the shape marked in the training sample as a target.
Further, referring to fig. 4, the present embodiment provides an optional preprocessing method, including:
s301: carrying out traffic light detection on the sample image to obtain a sample image comprising a detection frame;
s302: randomly expanding the detection frames in the sample image according to a preset proportion;
ideally, the image desired to be input to the traffic signal recognition model is an image clipped according to the minimum bounding rectangle of the traffic signal, such as the image a in fig. 5. However, the traffic signal lamp detection method under the current technical conditions cannot accurately detect the traffic signal lamp according to the minimum circumscribed rectangle, so that in order to reduce the precision requirement on the detection frame and improve the transportability and the expansion capability of the traffic signal lamp identification model, the detection frame in the sample images is respectively expanded randomly according to a preset proportion so as to input various traffic signal lamp sample images, enhance the generalization performance of the model and reduce the high precision requirement on the detection model.
The traffic signal lamp identification model is trained in such a way, and the subsequent traffic signal lamp attribute accurate identification scheme serving as any detection method can be transplanted with great advantage. The pictures of the rectangular frame of the traffic signal lamp are randomly scaled up, and the b-e images in the figure 5 are shown in detail.
S303: cutting the sample image according to the detection frame in the sample image, and carrying out scaling processing on the cut image to obtain a sample image of a target area including a traffic signal lamp;
s304: carrying out data equalization processing on the sample image to obtain a sample image with balanced color type and balanced shape type of the luminous traffic signal lamp;
due to the fact that the ratio of each attribute in the traffic signal lamp sample image collected in the real scene is not uniform, namely, the attribute is not balanced, and data balance has significance for learning of model weight. Data equalization is achieved by performing data enhancement on a less sample image in an existing data equalization mode (all color attribute ratios are 1: 1: 1 and all shape attribute ratios are 1: 1: 1).
The data enhancement method comprises the following steps: photometric disorders, Random Brightness, Random Contrast, Random lighting noise, and the like.
For convenience of management, the sample images of the traffic lights can be stored according to the same folder result attribute.
S305: the sample image is divided into a training sample and a test sample.
The sample image can be divided into a training sample and a test sample according to a certain proportion according to actual needs.
Further, in some special situations, such as the traffic signal lamp has a block, it is impossible to identify the color and shape of the luminous traffic signal lamp according to the traffic signal lamp image having the block, and therefore, in order to ensure the validity of the output result of the traffic signal lamp identification model, after the self-learning feature extraction network, the color recognition branch network and the shape recognition branch network are trained by using the training sample to obtain the traffic signal lamp recognition model by taking the recognition result approaching to the color and the shape marked in the training sample as a target, the traffic signal lamp identification model is tested by utilizing the test sample to obtain the color identification accuracy and the shape identification accuracy of the traffic signal lamp identification model, and then setting a color confidence threshold and a shape confidence threshold according to the color recognition accuracy and the shape recognition accuracy of the traffic signal lamp recognition model.
Generally, the confidence threshold is adjusted to be low, so that the recall rate is improved, but too many false identifications are introduced, so that the identification precision is reduced. The mature model should have a high filtering threshold, such as 0.9, at which only attribute information with a prediction probability higher than 90% is output. All that is required for the threshold determination is to ensure that excessive false recognition is not introduced on the premise of ensuring high recognition recall rate by properly reducing or increasing the threshold. Specifically, an initial confidence threshold may be set first, and the initial confidence threshold may be dynamically adjusted according to the test result and the actual application result along with the test and the actual application of the model.
And under the condition that the maximum probability value in the color recognition result of the traffic signal lamp recognition model is greater than the color confidence coefficient threshold value, the recognition is effective recognition, otherwise, the recognition is invalid recognition, and the shape recognition is the same.
In order to ensure that the color and the shape of the current traffic signal lamp can be output regardless of whether the identification result is valid or not, the embodiment adopts two state registers to further process the identification result of the traffic signal lamp identification model, each code in the two registers has an independent register bit, and at any time, the two registers have one bit and only one bit to be valid respectively and are used for representing the valid identification result. The first state register is used for receiving a color recognition result of the traffic signal lamp recognition model, and the second state register is used for receiving a shape recognition result of the traffic signal lamp recognition model.
Specifically, the color recognition result including the probability value of each color is input into the first state register, the color recognition result is an effective color recognition result when the maximum probability value in the color recognition result is greater than the color confidence threshold value, the color of the luminous traffic signal lamp output by the first state register is consistent with the color corresponding to the maximum probability value, the color recognition result is an ineffective color recognition result when the maximum probability in the color recognition result is not greater than the color confidence threshold value, and the color of the luminous traffic signal lamp output by the first state register is consistent with the last output result.
And inputting the shape recognition result comprising each shape probability value into a second state register, wherein the shape recognition result is an effective shape recognition result when the maximum probability value in the shape recognition result is greater than a shape confidence coefficient threshold value, the shape of the luminous traffic signal lamp output by the second state register is consistent with the shape corresponding to the maximum probability value, the shape recognition result is an invalid shape recognition result when the maximum probability in the shape recognition result is not greater than the shape confidence coefficient threshold value, and the shape of the luminous traffic signal lamp output by the second state register is consistent with the last output result.
Therefore, the invention discloses a traffic signal lamp identification method, which comprises the steps of extracting a characteristic diagram of an image to be identified by using a characteristic extraction network in a traffic signal lamp identification model after acquiring the image to be identified in a traffic signal lamp target area, and accurately identifying the color and the shape of a luminous traffic signal lamp by respectively inputting the characteristic diagram into a color identification branch network and a shape identification branch network in the traffic signal lamp identification model.
Based on the above-mentioned identification method for traffic signal lamp disclosed in the embodiment, the embodiment correspondingly discloses an identification device for traffic signal lamp, please refer to fig. 6, the device includes:
an image to be recognized acquisition unit 100, configured to acquire an image to be recognized, where the image to be recognized includes a target area of a traffic signal lamp;
the feature extraction unit 200 is configured to perform feature extraction on the image to be identified by using a feature extraction network in a traffic signal lamp identification model to obtain a feature map corresponding to the image to be identified;
and the attribute identification unit 300 is configured to input the feature map into the color identification branch network and the shape identification branch network in the traffic signal lamp identification model respectively to obtain a color identification result and a shape identification result of the light-emitting traffic signal lamp.
Optionally, the image acquiring unit 100 to be identified is specifically configured to:
acquiring a target image including a detection frame obtained after the traffic signal lamp detection;
and cutting the target image according to the detection frame, and zooming the cut image to obtain the image to be recognized of the target area including the traffic signal lamp.
Optionally, the apparatus further comprises:
the system comprises a sample image acquisition unit, a traffic signal acquisition unit and a traffic signal processing unit, wherein the sample image acquisition unit is used for acquiring sample images of traffic signal lamps in different application scenes, and the application scenes comprise at least one of lane conditions, traffic signal lamp types, weather conditions and time conditions;
the attribute marking unit is used for marking the color and the shape of the luminous traffic signal lamp in the sample image;
the sample preprocessing unit is used for preprocessing the sample image to obtain a training sample and a test sample;
and the model training unit is used for training the self-learning feature extraction network, the color recognition branch network and the shape recognition branch network by using the training sample to obtain the traffic signal lamp recognition model by taking the recognition result approaching to the color and the shape marked in the training sample as a target.
Optionally, the sample preprocessing unit is specifically configured to:
carrying out traffic light detection on the sample image to obtain a sample image comprising a detection frame;
randomly expanding the detection frames in the sample image according to a preset proportion;
cutting the sample image according to the detection frame in the sample image, and carrying out scaling processing on the cut image to obtain a sample image of a target area including a traffic signal lamp;
carrying out data equalization processing on the sample image to obtain a sample image with balanced color type and balanced shape type of the luminous traffic signal lamp;
dividing a sample image into the training sample and the test sample.
Optionally, the apparatus further comprises:
the model testing unit is used for testing the traffic signal lamp identification model by using the test sample to obtain the color identification accuracy and the shape identification accuracy of the traffic signal lamp identification model;
and the threshold setting unit is used for setting a color confidence threshold and a shape confidence threshold according to the color recognition accuracy and the shape recognition accuracy of the traffic signal lamp recognition model.
Optionally, the apparatus further comprises:
a color output unit, configured to input the color recognition result including each color probability value into a first status register, where, in a case where a maximum probability value in the color recognition result is greater than the color confidence threshold, a color of the luminous traffic signal lamp output by the first status register is consistent with a color corresponding to the maximum probability value, and, in a case where the maximum probability in the color recognition result is not greater than the color confidence threshold, the color of the luminous traffic signal lamp output by the first status register is consistent with a last output result;
a shape output unit, configured to input the shape recognition result including each shape probability value into a second status register, where, in a case where a maximum probability value in the shape recognition result is greater than the shape confidence threshold, a shape of the light-emitting traffic signal output by the second status register is consistent with a shape corresponding to the maximum probability value, and in a case where the maximum probability in the shape recognition result is not greater than the shape confidence threshold, the shape of the light-emitting traffic signal output by the second status register is consistent with a last output result.
According to the traffic signal lamp identification device disclosed by the embodiment, after the image to be identified including the traffic signal lamp target area is obtained, the feature extraction network in the traffic signal lamp identification model is used for extracting the feature map of the image to be identified, and the feature map is respectively input into the multi-color identification branch network and the shape identification branch network in the traffic signal lamp identification model, so that the colors and the shapes of the luminous traffic signal lamps are accurately identified, and the robustness is strong for complex real scenes.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above embodiments can be combined arbitrarily, and the features described in the embodiments in the present specification can be replaced or combined with each other in the above description of the disclosed embodiments, so that those skilled in the art can implement or use the present application.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (12)
1. A method for identifying a traffic signal, comprising:
acquiring an image to be identified, wherein the image to be identified comprises a target area of a traffic signal lamp;
utilizing a feature extraction network in a traffic signal lamp recognition model to extract features of the image to be recognized, and obtaining a feature map corresponding to the image to be recognized;
and respectively inputting the characteristic diagrams into the color recognition branch network and the shape recognition branch network in the traffic signal lamp recognition model to obtain a color recognition result and a shape recognition result of the luminous traffic signal lamp.
2. The method of claim 1, wherein the obtaining the image to be identified comprises:
acquiring a target image including a detection frame obtained after the traffic signal lamp detection;
and cutting the target image according to the detection frame, and zooming the cut image to obtain the image to be recognized of the target area including the traffic signal lamp.
3. The method of claim 1, wherein the traffic signal recognition model is constructed by the following method:
acquiring sample images of traffic lights in different application scenes, wherein the application scenes comprise at least one of lane conditions, traffic light types, weather conditions and time conditions;
marking the color and the shape of the luminous traffic signal lamp in the sample image;
preprocessing a sample image to obtain a training sample and a test sample;
and training the self-learning feature extraction network, the color recognition branch network and the shape recognition branch network by using the training sample to obtain the traffic signal lamp recognition model by taking the recognition result approaching to the color and the shape marked in the training sample as a target.
4. The method of claim 3, wherein preprocessing the sample image to obtain a training sample and a test sample comprises:
carrying out traffic light detection on the sample image to obtain a sample image comprising a detection frame;
randomly expanding the detection frames in the sample image according to a preset proportion;
cutting the sample image according to the detection frame in the sample image, and carrying out scaling processing on the cut image to obtain a sample image of a target area including a traffic signal lamp;
carrying out data equalization processing on the sample image to obtain a sample image with balanced color type and balanced shape type of the luminous traffic signal lamp;
dividing a sample image into the training sample and the test sample.
5. The method of claim 3, further comprising:
testing the traffic signal lamp identification model by using the test sample to obtain the color identification accuracy and the shape identification accuracy of the traffic signal lamp identification model;
and setting a color confidence coefficient threshold value and a shape confidence coefficient threshold value according to the color recognition accuracy and the shape recognition accuracy of the traffic signal lamp recognition model.
6. The method of claim 5, wherein after obtaining the color recognition result and the shape recognition result of the illuminated traffic signal, the method further comprises:
inputting the color recognition result comprising each color probability value into a first status register, wherein the color of the luminous traffic signal lamp output by the first status register is consistent with the color corresponding to the maximum probability value under the condition that the maximum probability value in the color recognition result is greater than the color confidence coefficient threshold value, and the color of the luminous traffic signal lamp output by the first status register is consistent with the last output result under the condition that the maximum probability in the color recognition result is not greater than the color confidence coefficient threshold value;
and inputting the shape recognition result comprising each shape probability value into a second state register, wherein the shape of the luminous traffic signal lamp output by the second state register is consistent with the shape corresponding to the maximum probability value under the condition that the maximum probability value in the shape recognition result is greater than the shape confidence coefficient threshold value, and the shape of the luminous traffic signal lamp output by the second state register is consistent with the last output result under the condition that the maximum probability in the shape recognition result is not greater than the shape confidence coefficient threshold value.
7. An identification device for a traffic signal, comprising:
the device comprises an image to be identified acquisition unit, a traffic signal lamp identification unit and a traffic signal lamp identification unit, wherein the image to be identified acquisition unit is used for acquiring an image to be identified, and the image to be identified comprises a target area of the traffic signal lamp;
the characteristic extraction unit is used for extracting the characteristics of the image to be recognized by utilizing a characteristic extraction network in a traffic signal lamp recognition model to obtain a characteristic diagram corresponding to the image to be recognized;
and the attribute identification unit is used for respectively inputting the characteristic diagram into the color identification branch network and the shape identification branch network in the traffic signal lamp identification model to obtain a color identification result and a shape identification result of the luminous traffic signal lamp.
8. The apparatus according to claim 7, wherein the image capturing unit to be identified is specifically configured to:
acquiring a target image including a detection frame obtained after the traffic signal lamp detection;
and cutting the target image according to the detection frame, and zooming the cut image to obtain the image to be recognized of the target area including the traffic signal lamp.
9. The apparatus of claim 7, further comprising:
the system comprises a sample image acquisition unit, a traffic signal acquisition unit and a traffic signal processing unit, wherein the sample image acquisition unit is used for acquiring sample images of traffic signal lamps in different application scenes, and the application scenes comprise at least one of lane conditions, traffic signal lamp types, weather conditions and time conditions;
the attribute marking unit is used for marking the color and the shape of the luminous traffic signal lamp in the sample image;
the sample preprocessing unit is used for preprocessing the sample image to obtain a training sample and a test sample;
and the model training unit is used for training the self-learning feature extraction network, the color recognition branch network and the shape recognition branch network by using the training sample to obtain the traffic signal lamp recognition model by taking the recognition result approaching to the color and the shape marked in the training sample as a target.
10. The apparatus according to claim 9, wherein the sample pre-processing unit is specifically configured to:
carrying out traffic light detection on the sample image to obtain a sample image comprising a detection frame;
randomly expanding the detection frames in the sample image according to a preset proportion;
cutting the sample image according to the detection frame in the sample image, and carrying out scaling processing on the cut image to obtain a sample image of a target area including a traffic signal lamp;
carrying out data equalization processing on the sample image to obtain a sample image with balanced color type and balanced shape type of the luminous traffic signal lamp;
dividing a sample image into the training sample and the test sample.
11. The apparatus of claim 9, further comprising:
the model testing unit is used for testing the traffic signal lamp identification model by using the test sample to obtain the color identification accuracy and the shape identification accuracy of the traffic signal lamp identification model;
and the threshold setting unit is used for setting a color confidence threshold and a shape confidence threshold according to the color recognition accuracy and the shape recognition accuracy of the traffic signal lamp recognition model.
12. The apparatus of claim 11, further comprising:
a color output unit, configured to input the color recognition result including each color probability value into a first status register, where, in a case where a maximum probability value in the color recognition result is greater than the color confidence threshold, a color of the luminous traffic signal lamp output by the first status register is consistent with a color corresponding to the maximum probability value, and, in a case where the maximum probability in the color recognition result is not greater than the color confidence threshold, the color of the luminous traffic signal lamp output by the first status register is consistent with a last output result;
a shape output unit, configured to input the shape recognition result including each shape probability value into a second status register, where, in a case where a maximum probability value in the shape recognition result is greater than the shape confidence threshold, a shape of the light-emitting traffic signal output by the second status register is consistent with a shape corresponding to the maximum probability value, and in a case where the maximum probability in the shape recognition result is not greater than the shape confidence threshold, the shape of the light-emitting traffic signal output by the second status register is consistent with a last output result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111454157.8A CN114120286A (en) | 2021-12-01 | 2021-12-01 | Traffic signal lamp identification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111454157.8A CN114120286A (en) | 2021-12-01 | 2021-12-01 | Traffic signal lamp identification method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114120286A true CN114120286A (en) | 2022-03-01 |
Family
ID=80369286
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111454157.8A Pending CN114120286A (en) | 2021-12-01 | 2021-12-01 | Traffic signal lamp identification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114120286A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023197302A1 (en) * | 2022-04-15 | 2023-10-19 | 华为技术有限公司 | Signal light detection device, vehicle control method and system |
-
2021
- 2021-12-01 CN CN202111454157.8A patent/CN114120286A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023197302A1 (en) * | 2022-04-15 | 2023-10-19 | 华为技术有限公司 | Signal light detection device, vehicle control method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107346420B (en) | Character detection and positioning method in natural scene based on deep learning | |
| CN105373794B (en) | A kind of licence plate recognition method | |
| CN103345766B (en) | A kind of signal lamp recognition methods and device | |
| CN110619279B (en) | Road traffic sign instance segmentation method based on tracking | |
| CN102799879B (en) | Method for identifying multi-language multi-font characters from natural scene image | |
| EP2575077A2 (en) | Road sign detecting method and road sign detecting apparatus | |
| CN109215364B (en) | Traffic signal recognition method, system, device and storage medium | |
| CN110991221B (en) | Dynamic traffic red light running recognition method based on deep learning | |
| KR20070027768A (en) | Method for traffic sign detection | |
| CN114998852A (en) | Intelligent detection method for road pavement diseases based on deep learning | |
| CN114973207B (en) | Road sign identification method based on target detection | |
| CN106886757B (en) | A kind of multiclass traffic lights detection method and system based on prior probability image | |
| CN111814751A (en) | Vehicle attribute analysis method and system based on deep learning target detection and image recognition | |
| CN112651293B (en) | Video detection method for road illegal spreading event | |
| CN108734131A (en) | A kind of traffic sign symmetry detection methods in image | |
| CN110909598A (en) | Deep learning-based method for recognizing illegal traffic driving of non-motor vehicle lane | |
| CN106709412A (en) | Traffic sign detection method and apparatus | |
| CN112818853A (en) | Traffic element identification method, device, equipment and storage medium | |
| CN114120286A (en) | Traffic signal lamp identification method and device | |
| CN112101108A (en) | Left-right-to-pass sign identification method based on pole position characteristics of graph | |
| CN111401364A (en) | License plate positioning algorithm based on combination of color features and template matching | |
| CN109284678A (en) | Guideboard method for recognizing semantics and system | |
| CN108985197B (en) | Automatic detection method for taxi driver smoking behavior based on multi-algorithm fusion | |
| CN112071079B (en) | Machine vision vehicle high beam detection early warning system based on 5G transmission | |
| CN114495058A (en) | Traffic sign detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |