CN111923915A - Traffic light intelligent reminding method, device and system - Google Patents
Traffic light intelligent reminding method, device and system Download PDFInfo
- Publication number
- CN111923915A CN111923915A CN201910394130.0A CN201910394130A CN111923915A CN 111923915 A CN111923915 A CN 111923915A CN 201910394130 A CN201910394130 A CN 201910394130A CN 111923915 A CN111923915 A CN 111923915A
- Authority
- CN
- China
- Prior art keywords
- traffic light
- state
- layer
- vehicle
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/10—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to vehicle motion
- B60W40/105—Speed
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W2050/0001—Details of the control system
- B60W2050/0043—Signal treatments, identification of variables or parameters, parameter estimation or state estimation
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
- B60W2050/143—Alarm means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2520/00—Input parameters relating to overall vehicle dynamics
- B60W2520/10—Longitudinal speed
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an intelligent traffic light reminding method, device and system, which can identify the state and countdown data of a traffic light in image data through a target detection network and a digital identification classification network, further predict the actual state of the traffic light when the vehicle is driven to the intersection to be passed through the traffic light by combining position data, vehicle speed data, the state of the traffic light and the countdown data when the vehicle approaches the intersection of the traffic light and the traffic light state is a red light state, and carry out brake reminding when the actual state of the traffic light is the red light state. Based on the invention, the driver can remind the driver of braking in time before passing through the traffic intersection, thereby reducing or even avoiding the condition of running the red light and improving the driving safety.
Description
Technical Field
The invention relates to the technical field of automobiles, in particular to an intelligent traffic light reminding method, device and system.
Background
With the increasing quantity of automobiles kept in China, the problems of urban traffic safety and intersection traffic efficiency become problems which need to be solved urgently.
At the present stage, a driver mainly makes a decision by subjective consciousness judgment when passing through a traffic intersection, and the situation of running a red light still occurs despite of strict traffic regulations, so that potential safety hazards are brought.
Disclosure of Invention
In view of the above, to solve the above problems, the present invention provides an intelligent traffic light reminding method, apparatus and system. The technical scheme is as follows:
a traffic light intelligent reminding method, the method comprising:
acquiring image data of a target in front of a vehicle, position data of the vehicle and an intersection of the vehicle and a traffic light to be passed through, and vehicle speed data of the vehicle;
inputting the image data into a pre-constructed target detection network, and determining the state of a traffic light and a detection frame of the traffic light in the image data through the target detection network; the target detection network is constructed based on a preset deep convolutional neural network, the preset deep convolutional neural network comprises a convolutional layer, a pooling layer and a full-connection layer, the convolutional layer comprises a large-scale shallow layer feature layer positioned at the output end of the deep convolutional layer, the large-scale shallow layer feature layer is used for performing regression operation on features extracted by the deep convolutional layer, the convolutional layer and the pooling layer are both connected with the full-connection layer, and the full-connection layer is used for fusing the features output by each layer;
inputting target image data marked by the detection frame of the traffic light in the image data into a pre-constructed digital recognition classification network, and determining countdown data in the target image data through the digital recognition classification network;
under the condition that the position data meet a first condition for representing the approaching of a traffic light intersection and the state of the traffic light is a red light state, predicting the actual state of the traffic light when the vehicle drives to the intersection to be passed through by using the position data, the vehicle speed data, the state of the traffic light and the countdown data;
and if the actual state of the traffic light is the red light state, performing brake reminding.
Preferably, before inputting the target image data labeled by the detection frame of the traffic light in the image data into a pre-constructed digital identification classification network, the method further comprises:
calculating the frame overlapping rate of the detection frame of the traffic light between two continuous frames of images;
and tracking the detection frame of the traffic light based on the frame overlapping rate.
Preferably, the brake reminding includes:
brake reminding is carried out in a voice playing mode.
Preferably, the method further comprises:
and when the position data meet a second condition which is used for representing that the vehicle runs to a traffic light intersection and the traffic light is in the red light state, performing brake reminding at least based on the vehicle speed data.
Preferably, the method further comprises:
under the condition that the position data meet a second condition which is used for representing the driving to a traffic light intersection and the state of the traffic light is a green light state, acquiring the state of a history traffic light which is identified by the traffic light identification model and is closest to the current distance;
and if the state of the historical traffic light is the red light state, acquiring new vehicle speed data of the vehicle within a specified time, and performing starting reminding when the new vehicle speed data is zero.
An intelligent warning device for traffic lights, the device comprising:
the system comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring image data of a target in front of a vehicle, position data of the vehicle and an intersection of the vehicle and a traffic light to be passed through and speed data of the vehicle;
the determining module is used for inputting the image data into a pre-constructed target detection network and determining the state of the traffic light and the detection frame of the traffic light in the image data through the target detection network; the target detection network is constructed based on a preset deep convolutional neural network, the preset deep convolutional neural network comprises a convolutional layer, a pooling layer and a full-connection layer, the convolutional layer comprises a large-scale shallow layer feature layer positioned at the output end of the deep convolutional layer, the large-scale shallow layer feature layer is used for performing regression operation on features extracted by the deep convolutional layer, the convolutional layer and the pooling layer are both connected with the full-connection layer, and the full-connection layer is used for fusing the features output by each layer; inputting target image data marked by the detection frame of the traffic light in the image data into a pre-constructed digital recognition classification network, and determining countdown data in the target image data through the digital recognition classification network;
the prediction module is used for predicting the actual state of the traffic light when the vehicle drives to the intersection to be passed through by utilizing the position data, the vehicle speed data, the state of the traffic light and the countdown data under the condition that the position data accords with a first condition which is used for representing the intersection approaching the traffic light and the state of the traffic light is the red light state;
and the reminding module is used for carrying out brake reminding if the actual state of the traffic light is the red light state.
Preferably, the apparatus further comprises:
the tracking module is used for calculating the frame overlapping rate of the detection frame of the traffic light between two continuous frames of images; and tracking the detection frame of the traffic light based on the frame overlapping rate.
Preferably, the reminding module is specifically configured to:
brake reminding is carried out in a voice playing mode.
A traffic light intelligent warning system, the system comprising:
the vehicle-mounted front-view camera is used for acquiring image data of a target in front of the vehicle;
the positioning device is used for acquiring position data of the vehicle and an intersection of the traffic light to be passed;
the velometer is used for acquiring the speed data of the vehicle;
the intelligent controller is respectively in communication connection with the vehicle-mounted forward-looking camera, the positioning device and the velometer and is used for acquiring the image data, the position data and the vehicle speed data; inputting the image data into a pre-constructed target detection network, and determining the state of a traffic light and a detection frame of the traffic light in the image data through the target detection network; the target detection network is constructed based on a preset deep convolutional neural network, the preset deep convolutional neural network comprises a convolutional layer, a pooling layer and a full-connection layer, the convolutional layer comprises a large-scale shallow layer feature layer positioned at the output end of the deep convolutional layer, the large-scale shallow layer feature layer is used for performing regression operation on features extracted by the deep convolutional layer, the convolutional layer and the pooling layer are both connected with the full-connection layer, and the full-connection layer is used for fusing the features output by each layer; inputting target image data marked by the detection frame of the traffic light in the image data into a pre-constructed digital recognition classification network, and determining countdown data in the target image data through the digital recognition classification network; under the condition that the position data meet a first condition for representing the approaching of a traffic light intersection and the state of the traffic light is a red light state, predicting the actual state of the traffic light when the vehicle drives to the intersection to be passed through by using the position data, the vehicle speed data, the state of the traffic light and the countdown data; and if the actual state of the traffic light is the red light state, performing brake reminding.
Preferably, the system further comprises:
and the voice player is in communication connection with the intelligent controller and is used for performing brake reminding in a voice playing mode.
The traffic light intelligent reminding method, the traffic light intelligent reminding device and the traffic light intelligent reminding system can identify the state and the countdown data of the traffic light in the image data through the target detection network and the digital identification classification network, further predict the actual state of the traffic light when the vehicle is driven to the intersection to be passed through the traffic light by combining the position data, the speed data, the state of the traffic light and the countdown data when the vehicle approaches the intersection of the traffic light and the traffic light state is the red light state, and carry out brake reminding when the actual state of the traffic light is the red light state. Based on the invention, the driver can remind the driver of braking in time before passing through the traffic intersection, thereby reducing or even avoiding the condition of running the red light and improving the driving safety.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a method of an intelligent traffic light reminding method according to an embodiment of the present invention;
FIG. 2 is a flowchart of another method of an intelligent traffic light reminding method according to an embodiment of the present invention;
fig. 3 is a flowchart of another method of an intelligent traffic light reminding method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an intelligent traffic light warning device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an intelligent traffic light reminding system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides an intelligent traffic light reminding method, the flow chart of which is shown in figure 1, and the method comprises the following steps:
s10, acquiring image data of the object in front of the vehicle, position data of the vehicle and the intersection to be passed through the traffic light, and vehicle speed data of the vehicle.
In the process of executing step S10, the vehicle-mounted front-view camera, such as a high-resolution, long-focus, high-dynamic-range vehicle-mounted camera capable of viewing a long distance, may be used to acquire image data of an object in front of the vehicle in real time, and the mounting position of the vehicle-mounted front-view camera is not limited in this embodiment, so as to ensure that the vehicle can shoot a traffic light when approaching a traffic intersection, such as can be mounted below a vehicle-mounted rear-view mirror.
In addition, for the position data of the vehicle and the intersection to be passed through the traffic light, for example, the distance to the intersection to be passed through the traffic light can be collected by a positioning device, such as a GPS. The speed data of the vehicle can be collected by a velometer of the vehicle.
S20, inputting the image data into a pre-constructed object detection network, and determining the state of the traffic light and the detection frame of the traffic light in the image data through the object detection network; the target detection network is constructed based on a preset deep convolutional neural network, the preset deep convolutional neural network comprises a convolutional layer, a pooling layer and a full-connection layer, the convolutional layer comprises a large-scale shallow layer feature layer located at the output end of the deep convolutional layer, the large-scale shallow layer feature layer is used for performing regression operation on features extracted by the deep convolutional layer, the convolutional layer and the pooling layer are both connected with the full-connection layer, and the full-connection layer is used for fusing the features output by each layer.
In this embodiment, the preset deep convolution neural network adopts a deep convolution mode, and the convolution layer thereof adopts depthwise convolution, so as to achieve the purposes of reducing the number of parameters and increasing the operation speed. In addition, in consideration of small targets of traffic signs and traffic lights, a large-scale shallow feature layer is added in the convolutional layer, and regression operation can be performed on features output by the deep convolutional layer so as to realize pixel amplification. In addition, the fully-connected layer can be fused with the features output by the convolutional layer, the pooling layer and the layer, so that multi-scale target classification is realized, and the classification accuracy is improved.
Of course, in the process of constructing the target detection network based on the preset deep convolutional neural network, the preset deep convolutional neural network may be trained by using an image labeled with a state of a traffic light in advance (for example, any one of a straight red light, a straight yellow light, a straight green light, a left-turn-prohibited red light, a left-turn yellow light, and a left-turn green light), and the target detection network with good performance is obtained after ten thousand times of iterative training.
And S30, inputting the target image data marked by the traffic light detection frame in the image data into a pre-constructed digital recognition classification network, and determining countdown data in the target image data through the digital recognition classification network.
In this embodiment, the number recognition classification network may be constructed based on a designated neural network, and since the requirement of number recognition is not high, the present embodiment does not limit the type of the designated neural network.
Of course, in the process of constructing the digital recognition classification network based on the designated neural network, the digital recognition classification network with good performance can be obtained after ten thousands of iterative trainings by training the designated neural network with the image labeled with countdown data in advance.
In addition, in order to avoid the interference of other numbers in the image, the digital identification classification network of the embodiment can identify the countdown data within a certain range of the traffic light detection frame.
Furthermore, the number of layers of the neural network is large, so that the neural network is suitable for a PC end and is not beneficial to being applied at an embedded end, and therefore, under the condition of not losing precision, the target detection network and/or the digital identification classification network can be compressed, and the requirement on real-time performance is met. Specifically, technologies based on TensorRT, pruning and the like can be adopted to compress the network model. Wherein the content of the first and second substances,
TensorRT is a C + + library based on GPU high-performance forward operation, can select high-efficiency network intermediate data types, and can evaluate and select based on layer parameters and performance, optimize the network and accelerate the forward reasoning time of the network.
Pruning is to prune redundant parameters in the neural network, and to omit neurons with low contribution, so that the model has higher running speed and smaller model files.
In other embodiments, in order to improve the robustness of the traffic light detection box, the method has less jitter, and before inputting the target image data marked by the traffic light detection box in the image data into the pre-constructed digital identification classification network, the method further comprises the following steps:
calculating the frame overlapping rate of a detection frame of the traffic light between two continuous frames of images; and tracking the detection frame of the traffic light based on the frame overlapping rate.
In this embodiment, a plurality of groups of traffic lights may appear in the same frame image, and accordingly, after passing through the target detection network, a plurality of detection frames of the traffic lights appear in the frame image. Considering that the moving tracks of the traffic lights are parallel in the process of vehicle advancing and the traffic lights have obvious intervals, a matching algorithm can be used for calculating the frame overlapping rate between any two continuous frames of images, and the detection frames of the two traffic lights with the overlapping rate larger than a certain value can be determined as the same detection frame, so that the detection frame tracking is realized.
And S40, under the condition that the position data meet the first condition for representing the approach of the traffic light intersection and the state of the traffic light is in the red light state, predicting the actual state of the traffic light when the vehicle drives to the intersection to be passed through by the traffic light intersection by using the position data, the vehicle speed data, the state of the traffic light and the countdown data.
In the process of executing step S40, a first condition indicative of an approach to the traffic light intersection may be preset, for example, the first condition is that "the distance between the vehicle and the traffic light intersection is less than a first specified threshold", and when the position data meets the first condition, it indicates that the vehicle is approaching the traffic light intersection.
In addition, in the process of predicting the actual state of the traffic light, the position data, the speed data, the state of the traffic light and the countdown data are combined, and the actual state of the traffic light when the vehicle drives to the intersection to be passed is calculated through the Kalman filtering algorithm. Wherein the content of the first and second substances,
the Kalman filtering is to try to remove the influence of noise by using the information of a moving target so as to obtain a position estimation with better robustness about the position of the target, and the essence of the Kalman filtering is to predict the state of the next step. In the invention, the target is a vehicle and is a dynamic moving target, a motion state equation is established by combining position data and vehicle speed data (such as speed and acceleration), and the output is the position of the vehicle at the next moment, so that the higher robustness of the output position is ensured. And outputting for multiple times to obtain the target time when the vehicle reaches the intersection of the traffic lights, so that the state of the traffic lights at the target time is determined by combining the state of the traffic lights and the countdown data, namely the actual state of the traffic lights when the vehicle drives to the intersection to be passed through.
And S50, if the actual state of the traffic light is the red light state, performing brake reminding.
In the process of executing step S50, a brake alert may be performed by a specific manner, such as blinking a colored light, and further, such as vibrating.
Of course, in order to improve the timeliness and reliability of the reminding, the brake reminding can be performed by adopting a voice playing mode, specifically, a specified playing message, such as 'please notice the red light ahead and make brake preparation', can be sent to the voice player, and the voice player periodically performs the brake reminding.
In other embodiments, to avoid waiting for a vehicle to roll in a red light, on the basis of the traffic light intelligent reminding method shown in fig. 1, the method further includes the following steps, and the flow chart of the method is shown in fig. 2:
and S60, when the position data meet the second condition which is used for representing the driving to the traffic light intersection and the state of the traffic light is the red light state, braking reminding is carried out at least based on the vehicle speed data.
In the process of executing step S60, a second condition indicative of the vehicle reaching the traffic light intersection may be preset, such as "the distance between the vehicle and the traffic light intersection is smaller than a second specified threshold, where the second specified threshold is smaller than the first specified threshold", and when the position data meets the second condition, the vehicle is indicated to reach the traffic light intersection.
In addition, if the current gear of the vehicle is neutral, the vehicle is stopped, and at the moment, once the vehicle speed data is greater than or equal to the specified speed and indicates that the vehicle rolls, the brake reminding can be carried out in a specified mode so as to remind the driver of engaging the R gear or pulling the hand brake. Certainly, in order to improve the timeliness and reliability of the reminding, the braking reminding can be performed by adopting a voice playing mode, specifically, a specified playing message, such as "please prepare for braking" can be sent to the voice player, and the voice player performs the braking reminding periodically.
In addition, the distance between the vehicle and the front and rear vehicles can be further monitored, braking reminding can be timely performed when the current gear of the vehicle is neutral and the distance between the vehicles is too small, and the specific content of the second condition is not limited in the embodiment.
In other embodiments, in order to avoid the delay of green light starting caused by distraction of the driver, on the basis of the traffic light intelligent reminding method shown in fig. 1, the method further includes the following steps, and a flow chart of the method is shown in fig. 3:
and S70, acquiring the state of the history traffic light which is identified by the traffic light identification model and is closest to the current distance under the condition that the position data meets the specified second condition for representing the traffic light crossing and the state of the traffic light is in a green light state.
In the process of executing step S70, a second condition indicative of the vehicle reaching the traffic light intersection may be preset, such as "the distance between the vehicle and the traffic light intersection is smaller than a second specified threshold, where the second specified threshold is smaller than the first specified threshold", and when the position data meets the second condition, the vehicle is indicated to reach the traffic light intersection.
In addition, the state of the history traffic light closest to the current time identified by the traffic light identification model is the state of the traffic light output by the traffic light identification model at the last moment, for example, the state of the traffic light identification model outputting one traffic light every 1S is the current 5S, and the state of the history traffic light closest to the current time is the state of the traffic light output by the traffic light identification model 4S.
And S80, if the historical traffic light state is the red light state, acquiring new vehicle speed data of the vehicle within a specified time, and performing starting reminding when the new vehicle speed data is zero.
In the process of executing step S80, if the status of the historical traffic light is red, it indicates that the traffic light is changed from red to green; and further, if the new vehicle speed data of the vehicle in the specified duration is zero, the vehicle is not started, and a specified mode can be adopted for starting reminding.
Certainly, in order to improve the timeliness and reliability of the reminding, the starting reminding can be performed by adopting a voice playing mode, specifically, a specified playing message, such as "please prepare for starting" can be sent to the voice player, and the voice player periodically performs the starting reminding.
The traffic light intelligent reminding method provided by the embodiment of the invention can remind a driver in different modes under different scenes of passing through a traffic intersection, thereby improving the driving safety and ensuring the passing efficiency of the traffic intersection.
Based on the traffic light intelligent reminding method provided by the above embodiment, the embodiment of the present invention correspondingly provides a device for executing the traffic light intelligent reminding method, and a schematic structural diagram of the device is shown in fig. 4, and the device includes:
the acquisition module 101 is used for acquiring image data of an object in front of a vehicle, position data of the vehicle and an intersection to be passed through a traffic light, and vehicle speed data of the vehicle.
The determining module 102 is configured to input the image data into a pre-constructed target detection network, and determine a state of a traffic light and a detection frame of the traffic light in the image data through the target detection network; the target detection network is constructed based on a preset deep convolutional neural network, the preset deep convolutional neural network comprises a convolutional layer, a pooling layer and a full-connection layer, the convolutional layer comprises a large-scale shallow layer feature layer located at the output end of the deep convolutional layer, the large-scale shallow layer feature layer is used for performing regression operation on features extracted by the deep convolutional layer, the convolutional layer and the pooling layer are both connected with the full-connection layer, and the full-connection layer is used for fusing the features output by each layer; target image data marked by a detection frame of the traffic light in the image data are input into a pre-constructed digital recognition classification network, and countdown data in the target image data are determined through the digital recognition classification network.
And the prediction module 103 is used for predicting the actual state of the traffic light when the vehicle drives to the intersection to be passed through the traffic light by using the position data, the vehicle speed data, the state of the traffic light and the countdown data under the condition that the position data accords with a first condition which is specified and used for representing the intersection approaching the traffic light and the state of the traffic light is the red light state.
And the reminding module 104 is used for carrying out brake reminding if the actual state of the traffic light is a red light state.
Optionally, the determining module 102 is further configured to:
calculating the frame overlapping rate of a detection frame of the traffic light between two continuous frames of images; and tracking the detection frame of the traffic light based on the frame overlapping rate.
Optionally, the reminding module 104 is specifically configured to:
brake reminding is carried out in a voice playing mode.
Optionally, the reminding module 104 is further configured to:
and when the position data meet a second condition which is specified and used for representing the driving to the intersection of the traffic lights and the state of the traffic lights is a red light state, performing braking reminding at least based on the vehicle speed data.
Optionally, the reminding module 104 is further configured to:
under the condition that the position data meet a second condition which is used for representing the fact that the vehicle drives to the intersection of the traffic lights and the state of the traffic lights is a green light state, acquiring the state of the historical traffic lights which are identified by the traffic light identification model and are closest to the current traffic lights; and if the historical traffic light state is the red light state, acquiring new vehicle speed data of the vehicle within a specified time, and reminding starting when the new vehicle speed data is zero.
The traffic light intelligent reminding device provided by the embodiment of the invention can remind a driver in different modes under different scenes of passing through a traffic intersection, so that the driving safety is improved, and the passing efficiency of the traffic intersection is ensured.
The traffic light intelligent reminding device provided by the embodiment of the invention can remind a driver in different modes under different scenes of passing through a traffic intersection, so that the driving safety is improved, and the passing efficiency of the traffic intersection is ensured.
Based on the traffic light intelligent reminding method and device provided by the above embodiment, the embodiment of the invention also provides a traffic light intelligent reminding system, a schematic structural diagram of the system is shown in fig. 5, and the traffic light intelligent reminding system comprises:
and the vehicle-mounted front camera 201 is used for acquiring image data of an object in front of the vehicle.
And the positioning device 202 is used for acquiring position data of the intersection between the vehicle and the traffic light to be passed.
And the velometer 203 is used for collecting the speed data of the vehicle.
An intelligent controller 204 in communication connection with the on-vehicle front-view camera 201, the positioning device 202 and the velometer 203, respectively, for acquiring image data, position data and vehicle speed data; inputting image data into a pre-constructed target detection network, and determining the state of a traffic light and a detection frame of the traffic light in the image data through the target detection network; the target detection network is constructed based on a preset deep convolutional neural network, the preset deep convolutional neural network comprises a convolutional layer, a pooling layer and a full-connection layer, the convolutional layer comprises a large-scale shallow layer feature layer located at the output end of the deep convolutional layer, the large-scale shallow layer feature layer is used for performing regression operation on features extracted by the deep convolutional layer, the convolutional layer and the pooling layer are both connected with the full-connection layer, and the full-connection layer is used for fusing the features output by each layer; inputting target image data marked by a detection frame of a traffic light in the image data into a pre-constructed digital identification classification network, and determining countdown data in the target image data through the digital identification classification network; under the condition that the position data meet a first condition for representing the approaching of the traffic light intersection and the state of the traffic light is a red light state, predicting the actual state of the traffic light when the vehicle drives to the intersection to be passed through by using the position data, the speed data, the state of the traffic light and the countdown data; and if the actual state of the traffic light is the red light state, performing brake reminding.
Optionally, the system further comprises:
and the voice player is in communication connection with the intelligent controller and is used for performing brake reminding in a voice playing mode.
The traffic light intelligent reminding system provided by the embodiment of the invention can remind a driver in different modes under different scenes of passing through a traffic intersection, thereby improving the driving safety and ensuring the passing efficiency of the traffic intersection.
The traffic light intelligent reminding method, the traffic light intelligent reminding device and the traffic light intelligent reminding system are described in detail, specific examples are applied in the method to explain the principle and the implementation mode of the traffic light intelligent reminding system, and the description of the examples is only used for helping to understand the method and the core idea of the traffic light intelligent reminding system; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include or include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A traffic light intelligent reminding method is characterized by comprising the following steps:
acquiring image data of a target in front of a vehicle, position data of the vehicle and an intersection of the vehicle and a traffic light to be passed through, and vehicle speed data of the vehicle;
inputting the image data into a pre-constructed target detection network, and determining the state of a traffic light and a detection frame of the traffic light in the image data through the target detection network; the target detection network is constructed based on a preset deep convolutional neural network, the preset deep convolutional neural network comprises a convolutional layer, a pooling layer and a full-connection layer, the convolutional layer comprises a large-scale shallow layer feature layer positioned at the output end of the deep convolutional layer, the large-scale shallow layer feature layer is used for performing regression operation on features extracted by the deep convolutional layer, the convolutional layer and the pooling layer are both connected with the full-connection layer, and the full-connection layer is used for fusing the features output by each layer;
inputting target image data marked by the detection frame of the traffic light in the image data into a pre-constructed digital recognition classification network, and determining countdown data in the target image data through the digital recognition classification network;
under the condition that the position data meet a first condition for representing the approaching of a traffic light intersection and the state of the traffic light is a red light state, predicting the actual state of the traffic light when the vehicle drives to the intersection to be passed through by using the position data, the vehicle speed data, the state of the traffic light and the countdown data;
and if the actual state of the traffic light is the red light state, performing brake reminding.
2. The method of claim 1, wherein before inputting the target image data labeled by the traffic light detection box in the image data into a pre-constructed digital identification classification network, the method further comprises:
calculating the frame overlapping rate of the detection frame of the traffic light between two continuous frames of images;
and tracking the detection frame of the traffic light based on the frame overlapping rate.
3. The method of claim 1, wherein said performing a brake alert comprises:
brake reminding is carried out in a voice playing mode.
4. The method of claim 1, further comprising:
and when the position data meet a second condition which is used for representing that the vehicle runs to a traffic light intersection and the traffic light is in the red light state, performing brake reminding at least based on the vehicle speed data.
5. The method of claim 1, further comprising:
under the condition that the position data meet a second condition which is used for representing the driving to a traffic light intersection and the state of the traffic light is a green light state, acquiring the state of a history traffic light which is identified by the traffic light identification model and is closest to the current distance;
and if the state of the historical traffic light is the red light state, acquiring new vehicle speed data of the vehicle within a specified time, and performing starting reminding when the new vehicle speed data is zero.
6. An intelligent warning device for traffic lights, the device comprising:
the system comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring image data of a target in front of a vehicle, position data of the vehicle and an intersection of the vehicle and a traffic light to be passed through and speed data of the vehicle;
the determining module is used for inputting the image data into a pre-constructed target detection network and determining the state of the traffic light and the detection frame of the traffic light in the image data through the target detection network; the target detection network is constructed based on a preset deep convolutional neural network, the preset deep convolutional neural network comprises a convolutional layer, a pooling layer and a full-connection layer, the convolutional layer comprises a large-scale shallow layer feature layer positioned at the output end of the deep convolutional layer, the large-scale shallow layer feature layer is used for performing regression operation on features extracted by the deep convolutional layer, the convolutional layer and the pooling layer are both connected with the full-connection layer, and the full-connection layer is used for fusing the features output by each layer; inputting target image data marked by the detection frame of the traffic light in the image data into a pre-constructed digital recognition classification network, and determining countdown data in the target image data through the digital recognition classification network;
the prediction module is used for predicting the actual state of the traffic light when the vehicle drives to the intersection to be passed through by utilizing the position data, the vehicle speed data, the state of the traffic light and the countdown data under the condition that the position data accords with a first condition which is used for representing the intersection approaching the traffic light and the state of the traffic light is the red light state;
and the reminding module is used for carrying out brake reminding if the actual state of the traffic light is the red light state.
7. The apparatus of claim 6, further comprising:
the tracking module is used for calculating the frame overlapping rate of the detection frame of the traffic light between two continuous frames of images; and tracking the detection frame of the traffic light based on the frame overlapping rate.
8. The device of claim 6, wherein the reminder module is specifically configured to:
brake reminding is carried out in a voice playing mode.
9. A traffic light intelligent warning system, characterized in that the system comprises:
the vehicle-mounted front-view camera is used for acquiring image data of a target in front of the vehicle;
the positioning device is used for acquiring position data of the vehicle and an intersection of the traffic light to be passed;
the velometer is used for acquiring the speed data of the vehicle;
the intelligent controller is respectively in communication connection with the vehicle-mounted forward-looking camera, the positioning device and the velometer and is used for acquiring the image data, the position data and the vehicle speed data; inputting the image data into a pre-constructed target detection network, and determining the state of a traffic light and a detection frame of the traffic light in the image data through the target detection network; the target detection network is constructed based on a preset deep convolutional neural network, the preset deep convolutional neural network comprises a convolutional layer, a pooling layer and a full-connection layer, the convolutional layer comprises a large-scale shallow layer feature layer positioned at the output end of the deep convolutional layer, the large-scale shallow layer feature layer is used for performing regression operation on features extracted by the deep convolutional layer, the convolutional layer and the pooling layer are both connected with the full-connection layer, and the full-connection layer is used for fusing the features output by each layer; inputting target image data marked by the detection frame of the traffic light in the image data into a pre-constructed digital recognition classification network, and determining countdown data in the target image data through the digital recognition classification network; under the condition that the position data meet a first condition for representing the approaching of a traffic light intersection and the state of the traffic light is a red light state, predicting the actual state of the traffic light when the vehicle drives to the intersection to be passed through by using the position data, the vehicle speed data, the state of the traffic light and the countdown data; and if the actual state of the traffic light is the red light state, performing brake reminding.
10. The system of claim 9, further comprising:
and the voice player is in communication connection with the intelligent controller and is used for performing brake reminding in a voice playing mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910394130.0A CN111923915B (en) | 2019-05-13 | 2019-05-13 | Traffic light intelligent reminding method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910394130.0A CN111923915B (en) | 2019-05-13 | 2019-05-13 | Traffic light intelligent reminding method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111923915A true CN111923915A (en) | 2020-11-13 |
| CN111923915B CN111923915B (en) | 2021-11-09 |
Family
ID=73282812
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910394130.0A Active CN111923915B (en) | 2019-05-13 | 2019-05-13 | Traffic light intelligent reminding method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111923915B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112908006A (en) * | 2021-04-12 | 2021-06-04 | 吉林大学 | Method for identifying state of road traffic signal lamp and counting down time of display |
| CN113593253A (en) * | 2021-07-29 | 2021-11-02 | 北京紫光展锐通信技术有限公司 | Method and device for monitoring red light running of vehicle |
| CN114120693A (en) * | 2021-12-10 | 2022-03-01 | 智己汽车科技有限公司 | Traffic light reminding system and method for vehicle and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825696A (en) * | 2016-04-18 | 2016-08-03 | 吉林大学 | Driving assistance system based on signal lamp information prompts |
| CN106096531A (en) * | 2016-05-31 | 2016-11-09 | 安徽省云力信息技术有限公司 | A kind of traffic image polymorphic type vehicle checking method based on degree of depth study |
| CN107622678A (en) * | 2017-10-18 | 2018-01-23 | 冯迎安 | A kind of intelligent traffic control system and its method based on image procossing |
| CN107886034A (en) * | 2016-09-30 | 2018-04-06 | 比亚迪股份有限公司 | Driving based reminding method, device and vehicle |
| US20180262739A1 (en) * | 2017-03-10 | 2018-09-13 | Denso International America, Inc. | Object detection system |
| CN108804983A (en) * | 2017-05-03 | 2018-11-13 | 腾讯科技（深圳）有限公司 | Traffic signal light condition recognition methods, device, vehicle-mounted control terminal and motor vehicle |
| CN108875608A (en) * | 2018-06-05 | 2018-11-23 | 合肥湛达智能科技有限公司 | A kind of automobile traffic signal recognition method based on deep learning |
| CN109711227A (en) * | 2017-10-25 | 2019-05-03 | 北京京东尚科信息技术有限公司 | Traffic light recognition method, traffic light identifier and computer readable storage medium |
| CN112597793A (en) * | 2020-10-13 | 2021-04-02 | 禾多科技（北京）有限公司 | Method, device and equipment for identifying traffic light state and timer state |
-
2019
- 2019-05-13 CN CN201910394130.0A patent/CN111923915B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825696A (en) * | 2016-04-18 | 2016-08-03 | 吉林大学 | Driving assistance system based on signal lamp information prompts |
| CN106096531A (en) * | 2016-05-31 | 2016-11-09 | 安徽省云力信息技术有限公司 | A kind of traffic image polymorphic type vehicle checking method based on degree of depth study |
| CN107886034A (en) * | 2016-09-30 | 2018-04-06 | 比亚迪股份有限公司 | Driving based reminding method, device and vehicle |
| US20180262739A1 (en) * | 2017-03-10 | 2018-09-13 | Denso International America, Inc. | Object detection system |
| CN108804983A (en) * | 2017-05-03 | 2018-11-13 | 腾讯科技（深圳）有限公司 | Traffic signal light condition recognition methods, device, vehicle-mounted control terminal and motor vehicle |
| CN107622678A (en) * | 2017-10-18 | 2018-01-23 | 冯迎安 | A kind of intelligent traffic control system and its method based on image procossing |
| CN109711227A (en) * | 2017-10-25 | 2019-05-03 | 北京京东尚科信息技术有限公司 | Traffic light recognition method, traffic light identifier and computer readable storage medium |
| CN108875608A (en) * | 2018-06-05 | 2018-11-23 | 合肥湛达智能科技有限公司 | A kind of automobile traffic signal recognition method based on deep learning |
| CN112597793A (en) * | 2020-10-13 | 2021-04-02 | 禾多科技（北京）有限公司 | Method, device and equipment for identifying traffic light state and timer state |
Non-Patent Citations (2)
| Title |
|---|
| 李子康等: "基于渐变上下文特征的交通灯识别方法", 《传感器与微系统》 * |
| 王若瑜: "基于Resnet_50的智能驾驶红绿灯分类研究", 《电子测试》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112908006A (en) * | 2021-04-12 | 2021-06-04 | 吉林大学 | Method for identifying state of road traffic signal lamp and counting down time of display |
| CN113593253A (en) * | 2021-07-29 | 2021-11-02 | 北京紫光展锐通信技术有限公司 | Method and device for monitoring red light running of vehicle |
| CN114120693A (en) * | 2021-12-10 | 2022-03-01 | 智己汽车科技有限公司 | Traffic light reminding system and method for vehicle and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111923915B (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111923915B (en) | Traffic light intelligent reminding method, device and system | |
| CN108725440B (en) | Forward collision control method and apparatus, electronic device, program, and medium | |
| US20220262239A1 (en) | Determining causation of traffic events and encouraging good driving behavior | |
| US20190171204A1 (en) | Control Method and Control Device of Automatic Driving Vehicle | |
| WO2017067187A1 (en) | Method, device and system for processing startup of front vehicle | |
| WO2017078072A1 (en) | Object detection method and object detection system | |
| CN110188482B (en) | Test scene creating method and device based on intelligent driving | |
| US20180211121A1 (en) | Detecting Vehicles In Low Light Conditions | |
| US11840261B2 (en) | Ground truth based metrics for evaluation of machine learning based models for predicting attributes of traffic entities for navigating autonomous vehicles | |
| US20220144303A1 (en) | Driver behavior risk assessment and pedestrian awareness | |
| CN113734203B (en) | Control method, device and system for intelligent driving and storage medium | |
| CN114418895A (en) | Driving assistance method and device, vehicle-mounted device and storage medium | |
| JP2013242615A (en) | Driving scene transition prediction device and recommended driving operation presentation device for vehicle | |
| CN108482367A (en) | A kind of method, apparatus and system driven based on intelligent back vision mirror auxiliary | |
| DE112013004876T5 (en) | Vehicle navigation system and image recording device for a vehicle | |
| JP2021099877A (en) | Method, device, apparatus and storage medium for reminding travel on exclusive driveway | |
| CN111145569A (en) | Road monitoring and vehicle running control method and device and vehicle-road cooperative system | |
| CN115635961A (en) | Sample data generation method and trajectory prediction method and device applying same | |
| CN111951548A (en) | Vehicle driving risk determination method, device, system and medium | |
| Dunna et al. | A Deep Learning based system for fast detection of obstacles using rear-view camera under parking scenarios | |
| DE112017008236T5 (en) | METHOD AND DEVICE TO ASSIST DRIVING | |
| WO2016157277A1 (en) | Method and device for generating travelling environment abstract image | |
| WO2022179124A1 (en) | Image restoration method and apparatus | |
| CN115762153A (en) | Method and device for detecting backing up | |
| CN115964364A (en) | Data closed-loop method based on shadow mode |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220112 Address after: No. 203, Shanghai Songjiang Road, No. 201563, Pudong New Area Patentee after: SAIC Motor Corp.,Ltd. Patentee after: Shanghai Automotive Industry (Group) Corporation Address before: 201203 Room 509, Building No. 1, 563 Songtao Road, Zhangjiang High-tech Park, Pudong New Area, Shanghai Patentee before: SAIC Motor Corp.,Ltd. |