CN112949578A - Vehicle lamp state identification method, device, equipment and storage medium - Google Patents
Vehicle lamp state identification method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112949578A CN112949578A CN202110338964.7A CN202110338964A CN112949578A CN 112949578 A CN112949578 A CN 112949578A CN 202110338964 A CN202110338964 A CN 202110338964A CN 112949578 A CN112949578 A CN 112949578A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- lamp state
- network
- car
- vehicle lamp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000001514 detection method Methods 0.000 claims abstract description 56
- 230000007704 transition Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 14
- 230000007246 mechanism Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 8
- 238000005286 illumination Methods 0.000 abstract description 17
- 238000013136 deep learning model Methods 0.000 abstract 1
- 238000012545 processing Methods 0.000 description 25
- 238000010586 diagram Methods 0.000 description 15
- 238000012549 training Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 206010067484 Adverse reaction Diseases 0.000 description 1
- 201000004569 Blindness Diseases 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000006838 adverse reaction Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The application relates to a method, a device, equipment and a storage medium for recognizing the state of a vehicle lamp, wherein the method comprises the following steps: acquiring a vehicle video image; inputting the vehicle video image into a pre-trained vehicle detection model, and carrying out vehicle target detection to obtain a vehicle lamp state identification area of a corresponding vehicle; and inputting the car lamp state identification area into a pre-trained car lamp identification model, and identifying the car lamp state of the corresponding car. According to the vehicle lamp identification method, aiming at the illumination problem of the tunnel scene image, the two-stage deep learning model is adopted, the vehicle lamp state is identified, the accuracy and the precision of vehicle lamp state identification are improved, a driver can be urged to start the vehicle lamp according to the standard under the tunnel scene, and the travel safety is guaranteed. The problem of the low rate of accuracy of state identification of the vehicle lamp in the tunnel scene is solved.
Description
Technical Field
The application relates to a method, a device, equipment and a storage medium for recognizing the state of a car lamp, belonging to the technical field of image processing.
Background
Tunnel traffic is a major component of a transportation junction, and the safety of the tunnel traffic is increasingly important. The tunnel environment is relatively airtight, and the illumination mainly leans on the tunnel in-house lighting system, and simultaneously, the car light of the vehicle that drives into the tunnel can influence the illumination environment inside the tunnel.
At present, in order to relieve the inadaptability of a driver caused by sudden illumination change, a road traffic department adjusts the illumination brightness in a tunnel in stages according to the brightness outside the tunnel.
The use state of the car lights in the tunnel is an important influence factor related to the driving safety of the tunnel. In general, a vehicle enters a tunnel to drive, and the low beam lamp is mainly turned on. Daytime running lights and high beams are not suitable for tunnel scenes. When a vehicle enters a tunnel to drive, if a driver is stimulated by opposite high beams, adverse reactions such as dazzling and sudden blindness are easy to generate, and the driving safety is seriously damaged.
Disclosure of Invention
The application provides a vehicle lamp state identification method, a vehicle lamp state identification device and a storage medium, and aims to solve the problems that a tunnel illumination environment is complex, vehicle lamp state identification is difficult, and driving safety of a tunnel scene is affected.
In order to solve the technical problem, the application provides the following technical scheme:
in a first aspect of an embodiment of the present application, a method for identifying a vehicle lamp state is provided, where the method includes:
acquiring a vehicle video image;
inputting the vehicle video image into a pre-trained vehicle detection model, and carrying out vehicle target detection to obtain a vehicle lamp state identification area of a corresponding vehicle;
and inputting the car lamp state identification area into a pre-trained car lamp identification model, and identifying the car lamp state of the corresponding car.
The vehicle lamp identification method is used for positioning the position of a vehicle and intercepting a vehicle lamp state identification area by training the vehicle detection model aiming at the illumination problem of the tunnel scene image and according to the trained vehicle detection model. And recognizing the state of the car lamp by training the car lamp recognition model and taking the car lamp state recognition area as the input of the trained car lamp recognition model. This application realizes the discernment to the car light state through adopting above-mentioned two-stage network, has improved under the tunnel scene, and the precision and the degree of accuracy of car light state discernment can supervise the driver to open the car light according to the standard under the tunnel scene, guarantee trip safety.
In a first possible implementation of the first aspect of the present application, the vehicle detection model is a YOLOv3-SPP network;
the YOLOv3-SPP network is obtained by adding an SPP space pyramid structure between a first convolution layer and a second convolution layer of a large target detection branch of the YOLOv3 network.
In the scheme of the embodiment, the multi-scale local region features are pooled and combined through the SPP pyramid structure, so that the local features are aggregated in the output feature map, and then the global and local multi-scale features are combined to improve the target detection precision.
In a second possible implementation manner of the first aspect of the present application, a shallow feature is fused in a small target detection branch of the YOLOv3-SPP target detection network.
Because the shallow features contain the color features, texture features and the like of the image, the scheme of the embodiment is beneficial to building a tunnel illumination model by fusing the shallow features of the image in the small target detection branch of the target detection network, so that the network has self-adaptability to the illumination conditions of the tunnel scene. In the shallow layer features, for target frames with the same size, the image with large resolution is beneficial to extracting a small target feature map, so that the positioning of a small target at a distance is realized, and the accuracy of detecting the small target at the distance is improved.
In a third possible implementation manner of the first aspect of the present application, the performing, according to the vehicle video image, vehicle target detection by using a vehicle detection model trained in advance to determine a lamp state identification region of a corresponding vehicle includes:
inputting the vehicle video image into a pre-trained vehicle detection model to obtain a target frame with the height of H and indicating vehicle position information;
and intercepting the area with the height of the target frame between H/2 and H to obtain the vehicle lamp state identification area.
In a fourth possible implementation manner of the first aspect of the present application, the vehicle lamp identification model is a DenseNet classification network that is simplified by network channels and layers, and the DenseNet classification network includes three dense blocks and two transition layers, where one transition layer is disposed between every two dense blocks.
According to the scheme of the embodiment, the network channels and layers of the classification network are simplified, so that the training and predicting speed of the model is increased.
In a fifth possible implementation of the first aspect of the present application, an SE channel attention mechanism is added in the transition layer after the first dense block of the DenseNet network, constructed as an SE-DenseNet classification network.
According to the scheme of the embodiment, only some key information inputs are selected for processing by adding a channel attention mechanism, and the key areas needing attention are automatically acquired. The target to be identified increases expressiveness by using a channel attention mechanism, focuses on important features and suppresses unnecessary features, thereby improving the identification efficiency of the network.
In a sixth possible implementation manner of the first aspect of the present application, the loss function of the DenseNet classification network is:
Loss＝-αq(x)log(p(x))
wherein, alpha is a balance factor,
αNthe ratio of the number of the Nth vehicle lamp state type samples is obtained, and N is the number of the vehicle lamp state types; x is input characteristic data; p (x) is the predicted probability of the vehicle light state category; q (x) is the true probability of the vehicle light state category.
Of the present embodimentScheme, by introducing balance factors into the primary loss function
The loss function is corrected, prediction deviation caused by unbalanced quantity distribution of various types of samples in the automobile lamp state is solved, and classification accuracy is improved.
In a second aspect of the embodiments of the present application, there is provided a vehicle lamp state identification device, including:
the image acquisition module is used for acquiring a vehicle video image;
the vehicle detection module is used for inputting the vehicle video image into a vehicle detection model trained in advance, and performing vehicle target detection to obtain a vehicle lamp state identification area of a corresponding vehicle;
and the car lamp identification module is used for inputting the car lamp state identification area into a pre-trained car lamp identification model and identifying the car lamp state of the corresponding car.
In a third aspect of the embodiments of the present application, an electronic device is provided, which includes a processor and a memory, where the memory stores a computer program, and the computer program is used, when executed by the processor, to implement the steps of the method for identifying a vehicle lamp state in any one of the possible implementations of the first aspect of the embodiments of the present application.
In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, where a computer program is stored, and when the computer program is executed by a processor, the computer program is configured to implement the steps of the method for identifying a vehicle lamp state in any one of the possible implementations of the first aspect of the embodiments of the present application.
The foregoing description is only an overview of the technical solutions of the present application, and in order to make the technical solutions of the present application more clear and clear, and to implement the technical solutions according to the content of the description, the following detailed description is made with reference to the preferred embodiments of the present application and the accompanying drawings.
Drawings
Fig. 1 is a schematic diagram of a network architecture applied to a vehicle lamp state identification method and apparatus according to an embodiment of the present application;
FIG. 2 is a flow chart of a vehicle light identification method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a vehicle light status identification area intercepted according to an embodiment of the application;
FIG. 4 is a diagram of an improved YOLOv3-SPP network architecture provided by one embodiment of the present application;
fig. 5 is a schematic diagram of an improved DenseNet network structure provided by an embodiment of the present application;
FIG. 6 is a schematic diagram of a channel attention mechanism SE module provided in one embodiment of the present application;
fig. 7 is a block diagram of a vehicle lamp state identification device according to an embodiment of the present application;
fig. 8 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following examples are intended to illustrate the present application but are not intended to limit the scope of the present application.
In order to avoid the influence on the driving safety in the tunnel due to improper use of the high beam and the low beam when the vehicle runs in the tunnel scene, the following scheme can be adopted to identify the state of the vehicle lamp, and the safety of travel is guaranteed:
recognizing a vehicle light state based on the traffic monitoring image: the system has the advantages that the vehicle lamp state of the vehicle in the tunnel is monitored in real time, the vehicle information is automatically captured, and the vehicle lamp opening state is recorded, so that a driver is urged to open the vehicle lamp according to the standard under the tunnel scene, and the travel safety is guaranteed.
In practical application, the illumination intensity of the illumination equipment is different, so that the detection environment difference is large, and the illumination condition of traffic monitoring is complex due to sudden change of illumination brightness in the tunnel and change of running vehicle lamps. Meanwhile, the car lamp types are various, the appearance brightness difference is large, and the car lamp state cannot be accurately identified due to the reflection of other light sources and other influences, so that the identification difficulty of the car lamp state is increased.
In order to solve the above technical problem, embodiments of the present application provide a method and an apparatus for identifying a vehicle lamp state. Fig. 1 is a block diagram of a network architecture structure of a vehicle lamp state identification method and a system application provided in an embodiment of the present application, and as shown in fig. 1, the network architecture of the vehicle lamp state identification system includes: a vehicle 101, an image capturing apparatus 102, and an image processing apparatus 103. Optionally, a controller 104 and an alarm 105 may also be included, wherein,
the vehicle 101 runs in a tunnel, and is provided with a lamp including a headlight, a tail lamp, and a width lamp, wherein the headlight can switch a high beam and a low beam. When the vehicle runs in the tunnel, the front lamp, the width lamp and the tail lamp are turned on at the entrance of the tunnel, generally at about 50 meters, so as to recognize the condition of the front vehicle and attract the attention of the rear vehicle. Meanwhile, it is normally necessary to switch the headlamps to low-beam driving.
The vehicle lamp state recognized by the embodiment includes: low beam, high beam, and un-lighted lights.
Image capture device 102 may be a camera, video camera, or the like. For example, in a tunnel scene, the image capturing device may be mounted on the tunnel wall at a set distance for capturing video images of vehicles in the tunnel in real time.
The image capturing device 102 communicates with the image processing device 103, for example, the image capturing device 102 transmits captured video images of the vehicle in the tunnel to the image processing device 103. Alternatively, the image capturing device 102 may store the captured video images of the vehicles in the tunnel into the database, and the image processing device 103 may obtain the video images of the vehicles from the database according to a predetermined period.
The image processing device 103 may be, for example, an electronic device such as a PC, a server, or a smart mobile device (e.g., an IPAD), etc. The image processing device 103 of the present embodiment is equipped with a vehicle lamp recognition device for processing the acquired video images of the vehicles in the tunnel to recognize the vehicle lamp state of each vehicle in the video images of the tunnel.
Illustratively, the processing of the vehicle video image by the image processing device 103 includes:
and turning on a vehicle lamp recognition device, firstly, carrying out vehicle target detection through a pre-trained vehicle detection model according to the acquired vehicle video image by the vehicle lamp recognition device to obtain a vehicle target position, and outputting a vehicle lamp state recognition area according to the detected vehicle target position.
And then, according to the car lamp state identification area, adopting a pre-trained car lamp identification model to identify the car lamp state of the corresponding car.
The vehicle lamp state identification area is obtained by intercepting a rectangular target frame indicating a vehicle target position according to a preset rule.
Specific implementations of the vehicle detection model determining the vehicle light state identification region and the vehicle light identification model identifying the vehicle light state are described in detail below.
Optionally, the network architecture of the present embodiment may further include a controller 104 and an alarm 105 mounted on the vehicle.
Specifically, the controller 104 may be a lamp controller of the vehicle. In the process that the vehicle went in the tunnel, when image processing equipment 103 discerned that the car light that corresponds the vehicle was the high beam or did not open the car light, can send alarm information for the car light controller, after car light controller 104 received alarm information, send control signal for alarm 105, control alarm 105 sends alarm signal, reminds the driver in time to switch over the car light.
The alarm 105 may be a voice alarm, and the alarm signal may be, for example, "please turn on the vehicle lights," "please switch the dipped headlight," or the like.
Referring to fig. 2, a specific embodiment of the vehicle lamp state identification method of the present application will be described. Fig. 2 is a flowchart of a vehicle light state identification method according to an embodiment of the present application, and the vehicle light state identification method according to the embodiment of the present application may be applied to the image processing device 103 of the network architecture shown in fig. 1. The following describes a vehicle light state identification method according to an embodiment of the present application, with the image processing device 103 as an execution subject. In an embodiment of the present application, a method for recognizing a state of a vehicular lamp includes the following steps:
s201, obtaining a vehicle video image.
Specifically, in the embodiment of the present application, the acquired vehicle video image is acquired by an image acquisition device such as a camera and is sent to the image processing device, and the image processing device may be an electronic device, such as a PC, a server, an intelligent mobile device, and the like.
The image capturing device may also be a camera of the electronic device itself, and the obtained vehicle video image may also be directly captured by the camera of the electronic device itself, which is not limited herein.
For example, in a tunnel scene, in order to identify the state of a lamp of a running vehicle, image capturing devices (e.g., cameras) may be respectively mounted on a tunnel wall in the tunnel at set intervals for capturing video images of the running vehicle in the tunnel in real time, and the captured video images of the running vehicle may include a plurality of vehicles.
S202, inputting the vehicle video image into a pre-trained vehicle detection model, and carrying out vehicle target detection to obtain a vehicle lamp state identification area of a corresponding vehicle.
Specifically, the vehicle detection model of the present embodiment employs an object detection network based on YOLOv 3.
After the vehicle video image is acquired, the pre-trained vehicle detection model is input to perform vehicle target detection.
The vehicle detection model first determines a vehicle target position based on an input vehicle video image, and then outputs a vehicle lamp state recognition area determined by the vehicle target position based on the vehicle target position. The vehicle target position therein is indicated by a rectangular target frame.
The vehicle lamp state identification area in this embodiment is a sub-image captured according to a predetermined rule based on a rectangular target frame indicating a target position of a vehicle.
Illustratively, as shown in fig. 3, after the vehicle detection model detects a rectangular target frame indicating a target position of the vehicle, a subregion image with a height of H/2-H is captured as a vehicle light state identification region, where H is the height of the rectangular target frame.
When the vehicle detection model is pre-trained, if the obtained width-to-height ratio of the target frame in the output result of the vehicle target detection is larger than 2, the input sample is not used as a training sample.
And S203, adopting a pre-trained car lamp recognition model to recognize the car lamp state of the corresponding car according to the car lamp state recognition area.
Specifically, the vehicle lamp identification model of the present embodiment employs a classification network based on DenseNet (dense connection network).
According to the embodiment of the application, the vehicle lamp state identification area is used as the input of the vehicle lamp identification model, and the vehicle lamp state type of the vehicle target is predicted through the vehicle lamp identification model.
The following explains a specific embodiment related to step S202 and step S203 in conjunction with the structures of the vehicle detection model and the vehicle light identification model.
In step S202, the vehicle detection model adopts a modified YOLOv3 network, and the YOLOv3 network includes outputs of three branches, where the first branch yolo1 is a large target detection branch for detecting a large target in an image. The second branch yolo2 is a medium object detection branch for detecting medium objects in an image. The third branch yolo3 is a small object detection branch for detecting small objects in an image.
The structure and principle of the YOLOv3 network are well known in the art and will not be described herein.
The multi-scale prediction of the YOLOv3 network focuses on concatenating the global features of the multi-scale convolutional layers, and ignores the fusion of the multi-scale local features on the same convolutional layer, resulting in low detection accuracy.
In order to solve the above technical problem, in the embodiment of the present application, channel clipping is performed on the basis of the YOLOv3 network, and a Spatial pyramid (SPP) structure is added to construct a YOLOv3-SPP classification network as shown in fig. 4, and the technical solution of the present application is described in detail below.
As shown in fig. 4, the present embodiment uses YOLOv3 as a basic network, and improves a network model for the vehicle driving problem in the tunnel scene. Referring to fig. 4, the embodiment of the present application introduces an SPP spatial pyramid structure between the first convolutional layer and the second convolutional layer of the first branch yolo1 of the YOLOv3 network to form a YOLOv3-SPP classification network.
The SPP spatial pyramid structure of this embodiment includes a plurality of (e.g., three) parallel Max pooling layer Max pool layers with different kernel sizes. And inputting the multi-scale characteristic diagram of the SPP space pyramid structure, copying the multi-scale characteristic diagram through an S plit layer, and inputting the multi-scale characteristic diagram into three parallel Max pool layer branches. And respectively performing feature fusion on the feature graphs of the two branches through corresponding Concat layers, and inputting the feature graphs into corresponding Max pool layers to obtain the multi-scale local features. And inputting the feature map of the other branch to a corresponding Max pool layer directly to obtain the global feature. And the outputs of the three Max pool layers are fused by the Concat characteristic and then output.
In the SPP pyramid structure of this embodiment, the features of the multi-scale local regions are pooled and combined through the largest pooling layers with different kernel sizes, so that the output feature map is aggregated with the local features, and then the global and local multi-scale features are combined to improve the accuracy of target detection.
Because the illumination condition under the tunnel scene is relatively complicated, the target detection difficulty is high, and the accuracy is low. To solve the technical problem, optionally, the embodiment fuses the shallower feature map in the third branch yolo3 of the YOLOv3-SPP network model.
Referring to fig. 4, the feature extraction network in the YOLOv3-SPP network model constructed in this embodiment is composed of a large number of residual modules Res _ unit1, Res _ unit2, Res _ unit p.
The third branch yolo3 of the original YOLOv3 network fuses feature maps extracted by a deeper residual layer Res _ unit m, and in the embodiment of the present application, feature maps extracted by a shallower residual layer Res _ unit p are fused.
The shallow layer features contain color features, texture features and the like of the image, so that the tunnel illumination model is favorably established. In the embodiment, shallow features are fused to yolo3, and gradient feature extraction is performed on illumination conditions, so that the method is beneficial to network identification of targets and backgrounds.
Meanwhile, the third branch yolo3 uses larger Stride for uniform resolution due to the blending of shallower features. This makes the network adaptive to the lighting conditions of the tunnel scene.
In addition, in the shallow layer features, for target frames with the same size, the image with large resolution is beneficial to extracting a small target feature map, so that the positioning of a small target at a distance is realized, and the accuracy rate of detecting the small target at the distance is improved.
In step S203, the car light recognition model adopts an improved DenseNet network for classifying the car light states.
The DenseNet network itself belongs to the prior art and is common knowledge in the field, and the basic structure and principle of the DenseNet network are briefly introduced here and will not be described in detail. The DenseNet network is a convolutional neural network with dense connections. DenseNet consists of multiple denseblocks (dense blocks). Transitionlayers (transition layers) are arranged between different Densblock to realize Down sampling, and finally Classitionanlayer (classification layer).
In the DenseNet network, any two layers are directly connected, that is, the input of each layer of the network is the union of the outputs of all the previous layers, and the feature map learned by the layer is directly transmitted to all the next layers as input.
Fig. 5 shows a schematic structural diagram of a DenseNet network improved by the embodiment of the present application. The following describes in detail a vehicle lamp identification model obtained by improving the embodiment of the present application on the basis of a DenseNet network.
Optionally, in order to accelerate the training and predicting speed of the car light recognition model, network channel and network layer simplification is performed on the basis of a DenseNet network in the embodiment of the application.
Specifically, referring to fig. 5, the embodiment of the present application simplifies four Transition blocks and three Transition layers into three Transition blocks and two Transition layers, respectively, on the basis of the original Transition network.
Meanwhile, the iteration times of the three Dense blocks are respectively reduced to 6 times, 8 times and 8 times, so that the simplification of a network layer is realized. Meanwhile, the number of channels of each convolution layer is reduced. The training and predicting speed of the model is accelerated by channel and layer simplification.
Optionally, in order to make the network focus more on vehicle lamp state identification, an SE (channel attention mechanism) module is added to the DenseNet network in the embodiment of the present application, so as to construct an SE-DenseNet classification network.
Specifically, the SE-DenseNet classification network according to the embodiment of the present application uses a DenseNet network in which network channels and network layers are simplified as a basic framework, and an SE module is added to a Transition Layer after a first Dense Block.
The SE module is used for learning the weight of each channel of the input feature map, and the weight represents the relevance of the channel and the key information. The higher the weight, the higher the correlation, i.e. the more channels that need attention.
By adding the SE module, only some key information input is selected for processing, and key areas needing attention are automatically acquired. An object to be recognized (for example, a vehicular lamp) increases expressiveness by using a channel attention mechanism, focuses on an important feature and suppresses an unnecessary feature to improve the recognition efficiency of a network.
Fig. 6 is a schematic structural diagram of a channel attention mechanism SE module. In this embodiment, the SE module is added between the Convolution Layer and the Maxpool Layer of the Transition Layer to form a Transition Layer-SE structure.
The SE module comprises a split layer, an Average pool layer, two convolution layers and a Reshape layer which are sequentially cascaded, and a Scale (zoom translation) layer and an Eltwise (operation by element) layer.
Because channels of the Dense Block output feature map are interdependent, firstly, the Average pool is carried out globally to compress the global information of the split layer output feature map to a channel descriptor, so that the channel descriptor has a global receptive field.
Then, two constraint layers are adopted to model the correlation among the channels, wherein the first constraint layer realizes dimension reduction, and the second constraint layer realizes dimension lifting, so that the advantage of the method is that more nonlinear processing is added, the correlation among the channels can be better fitted, and meanwhile, the parameter quantity and the calculation quantity are greatly reduced.
And adjusting the output dimension of the second Convolition layer through the Reshape layer to obtain the normalized weights with the same number as the input features.
And the output of the Reshape layer and the output of the Split layer are used as the input of the Scale, the Scale layer carries out Scale operation on the output data of the Reshape layer and the Split layer, and the obtained normalized weights are weighted to the corresponding channel characteristics respectively.
The output of the Scale layer and the output of the Split layer are used as the input of the Eltwise layer, the Eltwise layer performs Eltwise operation on the output data of the Scale layer and the output data of the Split layer, the Scale layer and the output of the Split layer are combined and then output, and the output of the Eltwise layer is used as the input of the Max Pool layer.
In the embodiment, the channel attention mechanism is introduced into the SE-DenseNet classification network, so that the classification network takes global information as a starting point, valuable characteristic channels are amplified, characteristic channels with less information content are restrained, and the classification accuracy is improved.
In a tunnel scene, when the SE-DenseNet classification network constructed by the method is trained, in the collected vehicle video image samples, the number of the image samples for opening the high beam is far lower than the number of the two types of image samples for opening the low beam and not opening the vehicle lamps, so that the problem of unbalance of positive and negative samples is caused, and the classification effect is poor.
To solve this problem, the present application modifies the loss function of the SE-DenseNet classification network. Optionally, in the embodiment of the present application, a balance factor is introduced into the multi-class cross entropy loss function
A modified loss function is obtained, namely:
Loss＝-αq(x)log(p(x))
wherein x is the characteristic data of the input deep neural network model, p (x) is the probability of the predicted vehicle lamp state type, and q (x) is the probability of the real vehicle lamp state type.
The embodiment of the application introduces the balance factor
The loss function is corrected, prediction deviation caused by unbalanced quantity distribution of various types of samples in the automobile lamp state is solved, and classification accuracy is improved.
In summary, the YOLOv3 network is improved to solve the problem of tunnel scene image illumination, so that a vehicle detection model is obtained, and is used for positioning a vehicle position and intercepting a vehicle lamp state identification area. And improving the DenseNet classification network to obtain a car light identification model, and identifying the car light state by taking the car light state identification area as the input of the car light identification model. This application realizes the discernment to the car light state through adopting above-mentioned two-stage network, has improved the precision and the degree of accuracy of car light state discernment, can supervise the driver to open the car light according to the standard under the tunnel scene, guarantee trip safety.
Fig. 7 is a schematic structural diagram of a vehicle lamp state identification device according to an embodiment of the present application, and as shown in fig. 7, the vehicle lamp state identification device includes:
the image acquisition module is used for acquiring a vehicle video image;
the vehicle detection module is used for inputting the vehicle video image into a vehicle detection model trained in advance, and performing vehicle target detection to obtain a vehicle lamp state identification area of a corresponding vehicle;
and the car lamp identification module is used for inputting the car lamp state identification area into a pre-trained car lamp identification model and identifying the car lamp state of the corresponding car.
The embodiment of the vehicle lamp state identification device and the vehicle lamp state identification method provided by the embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment and is not described again.
It should be noted that: in the above embodiment, when the vehicle lamp state identification device identifies the vehicle lamp state, only the division of the functional modules is taken as an example, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the vehicle lamp state identification device is divided into different functional modules, so as to complete all or part of the functions described above.
Fig. 8 is a block diagram of a computer device according to an embodiment of the present disclosure, where the computer device may be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like, and the computer device may include, but is not limited to, a processor and a memory. Wherein,
the processor may include one or more processing cores, such as: 4 core processors, 6 core processors, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable gate array), PLA (Programmable logic array). The processor may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, the processor may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning. The processor is the control center of the computer equipment and is connected with all parts of the whole computer equipment by various interfaces and lines.
The memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a memory device, or other volatile solid state storage device. The memory stores thereon a computer program that is executable on the processor, and when the processor executes the computer program, all or part of the steps performed by the electronic device in the embodiments of the present application, such as those in the related embodiments of fig. 1-7, and/or other contents described in the text, are implemented.
Those skilled in the art will appreciate that fig. 8 is only one possible implementation manner of the embodiments of the present application, and other embodiments may include more or less components, or combine some components, or different components, and the present embodiment is not limited thereto.
Optionally, the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program is used for implementing the steps of the vehicle lamp state identification method shown in fig. 2 when being executed by a processor.
Optionally, the present application further provides a computer program product, which includes a computer readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the steps of implementing the vehicle lamp state identification method shown in fig. 2.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A vehicle lamp state identification method is characterized by comprising the following steps:
acquiring a vehicle video image;
inputting the vehicle video image into a pre-trained vehicle detection model, and carrying out vehicle target detection to obtain a vehicle lamp state identification area of a corresponding vehicle;
and inputting the car lamp state identification area into a pre-trained car lamp identification model, and identifying the car lamp state of the corresponding car.
2. The method of claim 1, wherein the vehicle detection model is a YOLOv3-SPP network;
the YOLOv3-SPP network is obtained by adding an SPP space pyramid structure between a first convolution layer and a second convolution layer of a large target detection branch of the YOLOv3 network.
3. The method of claim 2, wherein shallow features are fused in the small target detection branch of the YOLOv3-SPP network model.
4. The method according to claim 1, wherein the performing vehicle target detection by using a pre-trained vehicle detection model according to the vehicle video image to determine a car light state recognition region of a corresponding vehicle comprises:
inputting the vehicle video image into a pre-trained vehicle detection model to obtain a target frame with the height of H and indicating vehicle position information;
and intercepting the area with the height of the target frame between H/2 and H to obtain the vehicle lamp state identification area.
5. The method of claim 1 wherein the vehicle light identification model is a DenseNet classification network via network channel and layer refinement, the DenseNet classification network comprising three dense blocks and two transition layers, wherein one transition layer is disposed between each two dense blocks.
6. The method of claim 5, wherein a transition layer after the first dense block of the DenseNet network adds a channel attention mechanism, constructed as a SE-DenseNet classification network.
7. The method according to claim 5 or 6, characterized in that the loss function of the DenseNet classification network is:
Loss＝-αq(x)log(p(x))
wherein, alpha is a balance factor,
αNthe ratio of the number of the Nth vehicle lamp state type samples is obtained, and N is the number of the vehicle lamp state types; x is input characteristic data; p (x) is the predicted probability of the vehicle light state category; q (x) is the true probability of the vehicle light state category.
8. A vehicle lamp state identification device, characterized by comprising:
the image acquisition module is used for acquiring a vehicle video image;
the vehicle detection module is used for inputting the vehicle video image into a vehicle detection model trained in advance, and performing vehicle target detection to obtain a vehicle lamp state identification area of a corresponding vehicle;
and the car lamp identification module is used for inputting the car lamp state identification area into a pre-trained car lamp identification model and identifying the car lamp state of the corresponding car.
9. An electronic device comprising a processor and a memory, said memory storing a computer program, wherein said computer program, when executed by the processor, is adapted to carry out the steps of the method for identifying a status of a vehicle light according to any one of claims 1 to 7.
10. A computer-readable storage medium, which stores a computer program, wherein the computer program, when being executed by a processor, is adapted to implement the steps of the vehicle lamp state identification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110338964.7A CN112949578B (en) | 2021-03-30 | 2021-03-30 | Vehicle lamp state identification method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110338964.7A CN112949578B (en) | 2021-03-30 | 2021-03-30 | Vehicle lamp state identification method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112949578A true CN112949578A (en) | 2021-06-11 |
| CN112949578B CN112949578B (en) | 2023-04-07 |
Family
ID=76227463
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110338964.7A Active CN112949578B (en) | 2021-03-30 | 2021-03-30 | Vehicle lamp state identification method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112949578B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112101230A (en) * | 2020-09-16 | 2020-12-18 | 招商局重庆公路工程检测中心有限公司 | Method and system for detecting starting of headlamps of vehicles passing through highway tunnel |
| CN113903180A (en) * | 2021-11-17 | 2022-01-07 | 四川九通智路科技有限公司 | Method and system for detecting vehicle overspeed on expressway |
| CN115984828A (en) * | 2023-03-20 | 2023-04-18 | 江西省天轴通讯有限公司 | High beam light starting detection method, device and equipment based on geometric feature descriptor |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107563265A (en) * | 2016-06-30 | 2018-01-09 | 杭州海康威视数字技术股份有限公司 | A kind of high beam detection method and device |
| CN110232316A (en) * | 2019-05-05 | 2019-09-13 | 杭州电子科技大学 | A kind of vehicle detection and recognition method based on improved DSOD model |
| CN110619304A (en) * | 2019-09-17 | 2019-12-27 | 中控智慧科技股份有限公司 | Vehicle type recognition method, system, device and computer readable medium |
| US20200125095A1 (en) * | 2018-10-22 | 2020-04-23 | Robert Bosch Gmbh | Method and device for ascertaining a state of a vehicle light of a vehicle |
| CN111274942A (en) * | 2020-01-19 | 2020-06-12 | 国汽(北京)智能网联汽车研究院有限公司 | Traffic cone identification method and device based on cascade network |
| CN111695448A (en) * | 2020-05-27 | 2020-09-22 | 东南大学 | Roadside vehicle identification method based on visual sensor |
| US10789402B1 (en) * | 2019-05-01 | 2020-09-29 | Xilinx, Inc. | Compiler and hardware abstraction layer architecture for a neural network accelerator |
| CN112200189A (en) * | 2020-10-19 | 2021-01-08 | 平安国际智慧城市科技股份有限公司 | Vehicle type identification method and device based on SPP-YOLOv3 and computer readable storage medium |
-
2021
- 2021-03-30 CN CN202110338964.7A patent/CN112949578B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107563265A (en) * | 2016-06-30 | 2018-01-09 | 杭州海康威视数字技术股份有限公司 | A kind of high beam detection method and device |
| US20200125095A1 (en) * | 2018-10-22 | 2020-04-23 | Robert Bosch Gmbh | Method and device for ascertaining a state of a vehicle light of a vehicle |
| US10789402B1 (en) * | 2019-05-01 | 2020-09-29 | Xilinx, Inc. | Compiler and hardware abstraction layer architecture for a neural network accelerator |
| CN110232316A (en) * | 2019-05-05 | 2019-09-13 | 杭州电子科技大学 | A kind of vehicle detection and recognition method based on improved DSOD model |
| CN110619304A (en) * | 2019-09-17 | 2019-12-27 | 中控智慧科技股份有限公司 | Vehicle type recognition method, system, device and computer readable medium |
| CN111274942A (en) * | 2020-01-19 | 2020-06-12 | 国汽(北京)智能网联汽车研究院有限公司 | Traffic cone identification method and device based on cascade network |
| CN111695448A (en) * | 2020-05-27 | 2020-09-22 | 东南大学 | Roadside vehicle identification method based on visual sensor |
| CN112200189A (en) * | 2020-10-19 | 2021-01-08 | 平安国际智慧城市科技股份有限公司 | Vehicle type identification method and device based on SPP-YOLOv3 and computer readable storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112101230A (en) * | 2020-09-16 | 2020-12-18 | 招商局重庆公路工程检测中心有限公司 | Method and system for detecting starting of headlamps of vehicles passing through highway tunnel |
| CN113903180A (en) * | 2021-11-17 | 2022-01-07 | 四川九通智路科技有限公司 | Method and system for detecting vehicle overspeed on expressway |
| CN115984828A (en) * | 2023-03-20 | 2023-04-18 | 江西省天轴通讯有限公司 | High beam light starting detection method, device and equipment based on geometric feature descriptor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112949578B (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112949578B (en) | Vehicle lamp state identification method, device, equipment and storage medium | |
| CN109816024B (en) | Real-time vehicle logo detection method based on multi-scale feature fusion and DCNN | |
| CN113688652B (en) | Abnormal driving behavior processing method and device | |
| Lyu et al. | Small object recognition algorithm of grain pests based on SSD feature fusion | |
| Wan et al. | A novel neural network model for traffic sign detection and recognition under extreme conditions | |
| CN113361645B (en) | Target detection model construction method and system based on meta learning and knowledge memory | |
| CN115512251A (en) | Unmanned aerial vehicle low-illumination target tracking method based on double-branch progressive feature enhancement | |
| CN113095199B (en) | High-speed pedestrian identification method and device | |
| CN114550220B (en) | Training method of pedestrian re-recognition model and pedestrian re-recognition method | |
| CN113469933A (en) | Generative antagonistic network model for small road object detection | |
| Wang | Vehicle image detection method using deep learning in UAV video | |
| CN115620090A (en) | Model training method, low-illumination target re-recognition method and device and terminal equipment | |
| CN110909656B (en) | Pedestrian detection method and system integrating radar and camera | |
| Lashkov et al. | Edge-computing-facilitated nighttime vehicle detection investigations with CLAHE-enhanced images | |
| Nagaraj et al. | Edge-based street object detection | |
| CN113963333A (en) | Traffic sign board detection method based on improved YOLOF model | |
| CN116630904A (en) | Small target vehicle detection method integrating non-adjacent jump connection and multi-scale residual error structure | |
| CN116229443A (en) | Vehicle license plate detection method, device, computer equipment and storage medium | |
| Pan et al. | Traffic Light Detection for Self-Driving Vehicles Based on Deep Learning | |
| CN114639037B (en) | Method for determining vehicle saturation of high-speed service area and electronic equipment | |
| CN115762178B (en) | Intelligent electronic police violation detection system and method | |
| Wang et al. | Light Enhancement Algorithm Optimization for Autonomous Driving Vision in Night Scenes based on YOLACT++ | |
| Boby et al. | Enabling Vehicle Search Through Robust Licence Plate Detection | |
| CN115565152B (en) | Traffic sign extraction method integrating vehicle-mounted laser point cloud and panoramic image | |
| Ali et al. | Speed limit road sign detection and recognition system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |