CN115534994A - Man-machine driving sharing control right self-adaptive switching method based on cooperative sensing inside and outside vehicle - Google Patents
Man-machine driving sharing control right self-adaptive switching method based on cooperative sensing inside and outside vehicle Download PDFInfo
- Publication number
- CN115534994A CN115534994A CN202211209546.9A CN202211209546A CN115534994A CN 115534994 A CN115534994 A CN 115534994A CN 202211209546 A CN202211209546 A CN 202211209546A CN 115534994 A CN115534994 A CN 115534994A
- Authority
- CN
- China
- Prior art keywords
- driving
- vehicle
- risk
- driver
- automobile
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/005—Handover processes
- B60W60/0051—Handover processes from occupants to vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/005—Handover processes
- B60W60/0059—Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Human Computer Interaction (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a man-machine driving sharing control right self-adaptive switching method based on cooperative sensing inside and outside a vehicle. The method comprises the following steps: s11, controlling the man-machine to drive the automobile together by the driver; s12, judging the driving risk level of the external environment by using an external sensing module, and identifying the distraction risk level of a driver by using an internal sensing module; s13, judging the driving state of the vehicle by coupling the driving risk grade judgment result and the driver distraction risk grade identification result; s14, further analyzing whether the condition that an automatic driving system takes over the man-machine co-driving of the automobile is met or not according to the driving state judgment result of the automobile; s15, if yes, the control weight of the driver and the automatic driving system to the man-machine co-driving automobile is distributed in a self-adaptive mode, and otherwise, the driver continues to control the man-machine co-driving automobile; and S16, gradually switching the control weight to an automatic driving system according to the calculated control weight, and finally controlling the man-machine co-driving automobile by the automatic driving system. The invention adaptively adjusts the control right between the human-computer by fusing the sensed surrounding environment information and the driver state information in the vehicle, thereby improving the human-computer cooperation rate, reducing human-computer conflicts and simultaneously improving the driving safety.
Description
Technical Field
The invention relates to the technical field of automobile automatic driving and the technical field of man-machine driving, in particular to a man-machine driving-together automobile control right self-adaptive switching method based on inside and outside automobile cooperative sensing.
Background
The automatic driving technology is an effective way for reducing the traffic accident rate and the workload of drivers, and attracts a great deal of research of enterprises and scholars in recent years. Considering the complexity of urban road traffic environment, before the automatic driving technology is completely mature, a driver needs to participate in the control process of automatically driving the automobile for a long time, namely, a man-machine driving mode between manual driving and full automatic driving exists for a long time.
The man-machine co-driving technology is a new mode for controlling a vehicle by a human driver and an automatic driving system through a shared control system, and aims to solve the problem of driving control right conversion in an L3 (high automatic driving) level. When the vehicle is in a dangerous scene, the driving subjects (drivers and automatic driving systems) in the shared control can control the vehicle by utilizing respective advantages of the drivers and the automatic driving systems, and driving safety is improved. At present, the problem of allocation of control rights of a driver and an automatic driving system becomes one of focuses of attention in the technical field of automatic driving and the technical field of man-machine co-driving, for example, a chinese invention patent (CN 201910700881) discloses a driving right switching system considering a driver state under a man-machine co-driving environment, whether a vehicle and the driver are in an unsafe state is respectively judged by fusing vehicle state parameters such as a vehicle speed, a steering wheel angle, GPS positioning information and the like and physiological information of the driver, and the driving control rights are switched rigidly according to a safety state judgment result; the Chinese invention patent (CN 202110848303) establishes a man-machine common driving model based on method deviation by fitting data of driving states and own vehicle acceleration, and indirectly adjusts dangerous operation of a driver, but the model only carries out control right switching research according to response of the driver to the risk of the surrounding environment of a vehicle. The control right switching methods rarely consider the coupling relation between the vehicle running risk and the driver distraction state, and the cooperative consistency of the driver and the automation system is difficult to ensure.
Therefore, in a man-machine driving state, how to fuse the in-vehicle and out-vehicle perception information and adaptively distribute the man-machine driving vehicle control right is a key problem which needs to be solved urgently.
Disclosure of Invention
Aiming at the defects of the existing research, the invention provides a man-machine co-driving automobile control right self-adaptive switching method based on the cooperative sensing inside and outside the automobile, which is used for self-adaptively distributing the driving control right of a driver and an automatic driving system by detecting the surrounding driving environment in real time and identifying the distraction state of the driver, and enhancing the control capability of the automatic driving system to the automobile after conversion and neutralization.
The invention specifically adopts the following technical scheme for realizing the purpose:
a man-machine co-driving automobile control right self-adaptive switching method based on in-automobile and out-automobile cooperative sensing comprises the following steps:
s11, controlling the man-machine to drive the automobile together by the driver;
and S12, judging the driving risk level of the external environment by using the external sensing module, and identifying the distraction danger level of the driver by using the internal sensing module. Judging the driving risk level of the external environment by using an external sensing module, wherein the driving risk level comprises data acquisition, calculation of field force of a driving risk field, calculation of a risk index and judgment of the driving risk level; wherein the driving risk grades are no risk, low risk and high risk. Identifying the distraction risk level of a driver by using an in-vehicle sensing module, wherein the identification comprises classifying common distraction behaviors of the driver, dividing the risk level, and constructing a long-time and short-time memory neural network model with an attention mechanism for identifying the distraction behaviors of the driver; the corresponding relation between the distraction behavior of the driver and the danger level is as follows: the driver is not distracted when driving safely, the driver is low distracted when stretching to the back or chatting with passengers, the driver is high distracted when making up or adjusting the vehicle-mounted equipment, and the driver is high distracted when sending short messages or making calls by using a mobile phone.
And S13, judging the driving state of the vehicle. And coupling the driving risk level and the distraction risk level of the driver, and judging the driving state of the self vehicle at the current moment, including three driving states of no danger, danger and very danger.
And S14, further analyzing whether the condition that the automatic driving system takes over the man-machine co-driving automobile is met or not according to the driving state judgment result of the self automobile, namely executing the step S15 when the driving state of the self automobile is in a dangerous or very dangerous driving state, or executing the step S11.
And S15, adaptively distributing the control weight of the driver and the automatic driving system to the man-machine common driving automobile according to the driving risk index, the driving risk level and the driver distraction risk level, and smoothly converting the driving control weight to the automatic driving system when necessary.
And S16, controlling the man-machine driving of the automobile by the automatic driving system. And after the automatic driving system takes over the vehicle, the driving task is continuously executed until the vehicle passes through the dangerous road section, and when the man-machine co-driven vehicle smoothly passes through the dangerous road section, the driver can select to continuously control the vehicle and also select to continuously drive by the automatic driving system.
Further, utilize outer perception module of car to judge external environment driving risk level still include:
and (6) data acquisition. The CAN bus and the vehicle-mounted laser radar sensor are used for collecting vehicle data in real time, wherein the vehicle data comprise the speed, the acceleration, the distance between the head and other vehicles around the vehicle;
and calculating the field force of the risk field. According to the lane line information detected by the vehicle-mounted camera, a driving risk field model of a lane line potential energy field related to a lane is established;
and calculating the driving risk index. Calculating a driving risk index according to a risk field model for establishing a surrounding driving environment;
and judging the driving risk level. And comparing the risk index at each moment with different driving risk thresholds, and judging the driving risk level of the surrounding environment at the current moment.
Further, the step of judging the driving risk level of the external environment by using the external sensing module specifically comprises the following steps:
s211, data acquisition
The CAN bus and the vehicle-mounted laser radar sensor are used for collecting vehicle data in real time, wherein the vehicle data comprises the speed, the acceleration and the distance between the head of the vehicle and other surrounding vehicles;
s212, calculating the field force of the risk field
According to the lane line information detected by the vehicle-mounted camera, taking the direction of the lane line as the Y axis of a road coordinate system, taking the vertical direction of the lane line as the X axis of the road coordinate system, and establishing a driving risk field model of a potential energy field related to a lane:
E T ＝E L +E B +E V (1)
in the formula (1), E T Total field strength for driving risk field, E L Is the field strength of the potential energy field of the lane line, E B To represent the road boundary potential energy field strength, E V The field intensity of the potential energy field of the vehicle;
in the formula (1), the field intensity of the lane line potential energy field consisting of the lane line potential energy field and the double yellow line potential energy field in the road environment is calculated by adopting the following formula (2):
in the formula (2), A i (i =1, 2) field intensity coefficients of potential energy fields of different types of lane lines, x represents an abscissa value of a position of the vehicle under a road coordinate system, and x represents a distance between the vehicle and the road coordinate system l,j Indicating the position coordinates of the jth lane line in the X-axis direction,
representing the distance from the vehicle to the jth lane line, and rho representing the speed change rate of the potential energy field of the lane line along with the approaching or departing of the vehicle from the lane line;
in the formula (1), the road boundary potential energy field intensity generated at the left and right road boundaries is calculated by the following formula (3):
in formula (3), x b,z Indicating the position coordinates of the z-th road boundary line in the X-axis direction,
the distance from the vehicle to the z-th road boundary line is represented, and eta represents a road boundary field intensity coefficient;
in the formula (1), the vehicle potential energy field strength including the motion state information is calculated by using the following formula (4):
in the formula (4), M represents the equivalent mass of the vehicle, M represents the actual mass of the vehicle, v represents the traveling speed of the vehicle at the current time, and d' represents the spatial coordinate (x) from the center of mass of the vehicle around a certain periphery 0 ,y 0 ) The safety distance to the space coordinates (x, y) of the center of mass of the vehicle, tau represents a critical threshold value of the safety distance between the surrounding vehicles and the vehicle, theta represents an included angle formed by a connecting line of the center of mass of a certain surrounding vehicle and the center of mass of the vehicle and the motion direction of the vehicle, and a represents the acceleration of the vehicle at the current moment;
s213, calculating the running risk index
Establishing a risk field model of the surrounding driving environment according to the formula (1), and calculating the driving risk index RI at the moment t t ：
In the formula (5), F represents the field force of the risk field suffered by the vehicle, and F represents the standard risk index;
s214, judging the driving risk level
Setting omega 1 And omega 2 Respectively low risk driving threshold and high risk driving threshold, and calculating the obtained driving risk index RI t Comparing with different driving risk threshold values, and judging the driving risk grade R of the surrounding environment at the current moment t The specific process comprises the following steps:
(1) If RI t <ω 1 Then R is t ＝R 0 I.e. the current driving risk level is R 0 Indicating that the surrounding driving environment is risk-free, and the grade risk strength is assigned as
(2) If ω is 1 ＜RI t <ω 2 Then R is t ＝R 1 I.e. the current driving risk level is R 1 Indicating a low risk of the surrounding driving environment, the magnitude of the risk strength is assigned as
(3) If RI is present t >ω 2 Then R is t ＝R 2 I.e. the current driving risk level is R 2 Indicating a high risk of the surrounding driving environment, the magnitude of the risk is assigned to
Further, the identifying the distraction risk level of the driver by using the in-vehicle sensing module further comprises:
and classifying the common distraction behavior of the driver. Preprocessing videos and pictures of distracting behaviors of a Driver in a vehicle in a public data set State Farm displaced Driver Detection to obtain six types of common distracting driving behaviors, wherein the six types of common distracting driving behaviors comprise that the Driver stretches to the back, the Driver chats with passengers, the Driver makes up, the Driver adjusts vehicle-mounted equipment, and the Driver sends short messages and makes calls by using a mobile phone;
and (4) dividing the danger level of common distraction behaviors. Four danger levels are divided according to the danger degree of each type of distraction behavior, and the specific corresponding relation is as follows: the safe driving mark of the driver is no distraction D0, the driver stretches behind or chats with passengers to mark low distraction D1, the driver makes up or adjusts the vehicle-mounted equipment to be medium distraction D2, and the driver uses a mobile phone to send a short message or make a call to be high distraction D3. And carrying out distraction risk level marking on 22424 collected driving images, wherein the number of samples of each risk level behavior is about 5600. For the pre-labeled data set, the following 9:1 into a training set and a test set, wherein the training set is used for optimizing the recognition model of the distraction behavior of the driver, and the test set is used for testing the classification accuracy of the recognition model. The training set and the testing set respectively comprise behaviors of no distraction, low distraction, middle distraction and high distraction of the driver;
and constructing a long-time memory neural network model with an attention mechanism for identifying the distraction behavior of the driver. The long-time memory neural network model with the attention mechanism comprises: the first layer of the network is a characteristic data input layer, the second layer is a long-time memory neural network layer with an attention mechanism, and the third layer is an output recognition result layer. The long-time and short-time memory neural network layer with the attention mechanism consists of an input gate, a forgetting gate and an output gate, the added attention mechanism provides corresponding weight for each hidden state, the characteristic of distraction of a driver is better captured, the structure of the long-time and short-time memory neural network is enhanced, and finally the danger level of the distraction of the driver is output through the full-connection layer with the SoftMax classifier.
Further, the determining the driving state of the vehicle further includes:
and coupling the driving risk level and the distraction risk level of the driver, and judging the driving state of the self vehicle at the current moment, including three driving states of no danger, danger and very danger.
Further, the adaptively allocating the control weight of the driver and the automatic driving system to the man-machine co-driving automobile further comprises:
and adaptively calculating the driving control weights corresponding to the driver and the automatic driving system according to the driving risk index, the driving risk level and the driver distraction risk level, and smoothly converting the driving control weights to the automatic driving system when necessary.
Due to the adoption of the technical scheme, the invention has the following advantages:
1. the driving risk level is judged by using the driving risk field, so that the driving collision risk can be well quantified and predicted;
2. the long-time memory network with the attention mechanism is adopted to identify the distraction risk level of the driver, so that the time characteristics of distraction behaviors of the driver in a continuous frame sequence can be captured, and distraction behaviors with different risk levels can be better identified;
3. the current driving state of the self-vehicle is judged by fusing the driving risk grade judgment result and the driver distraction risk grade identification result, and the corresponding driving control weights of the driver and the automatic driving system are adaptively distributed, so that man-machine conflict can be effectively reduced, and the control capability of the automatic driving system on the vehicle is enhanced.
The invention provides a man-machine co-driving automobile control right self-adaptive switching method based on in-vehicle and out-vehicle cooperative sensing, which adaptively adjusts the control right between the man machines by fusing sensed surrounding environment information and in-vehicle driver state information so as to improve the man-machine cooperation rate and the driving safety.
Drawings
Fig. 1 is a schematic diagram of a work flow of a man-machine co-driving automobile control right adaptive switching method based on in-vehicle and out-vehicle cooperative sensing according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a process of determining a driving risk level of an external environment using a vehicle exterior sensing module according to an embodiment of the present invention;
FIG. 3 is a three-dimensional schematic view of a lane line potential energy field strength in a field force of a calculated risk field;
FIG. 4 is a three-dimensional perspective view of the road boundary potential energy field strength in calculating the field force of the risk field;
FIG. 5 is a three-dimensional perspective view of a vehicle potential energy field strength in calculating a field force of a risk field;
FIG. 6 is a flowchart illustrating a process for identifying a level of risk of distraction of a driver using an in-vehicle sensing module according to an embodiment of the present invention;
fig. 7 is a flowchart illustrating the operation of the adaptive control right switching module according to the embodiment of the present invention.
Detailed Description
In order to clearly show the technical solutions of the embodiments of the present invention, the present invention will be described below with reference to specific embodiments and accompanying drawings.
Fig. 1 is a work flow chart of a man-machine co-driving automobile control right adaptive switching method based on in-vehicle and out-vehicle cooperative sensing provided by the embodiment of the invention, and the method comprises the following specific steps:
and S11, controlling the man-machine to drive the automobile together by the driver. In the stage, a driver operates the man-machine to drive the automobile in the whole process, and the automatic driving system is in a waiting state.
And S12, judging the driving risk level of the external environment by using the external sensing module, and identifying the distraction danger level of the driver by using the internal sensing module. And judging the driving risk level of the external environment by using the external sensing module, wherein the driving risk level comprises data acquisition, calculation of the field force of a driving risk field, calculation of a risk index and judgment of the driving risk level. The driving risk grades are no risk, low risk and high risk. The method comprises the steps of utilizing an in-vehicle sensing module to identify the distraction danger level of a driver, classifying common distraction behaviors of the driver, dividing the danger level, and constructing a long-time and short-time memory neural network model with an attention mechanism for identifying the distraction behaviors of the driver. The corresponding relation between the distraction behavior of the driver and the danger level is as follows: the driver is not distracted when driving safely, the driver is low distracted when stretching behind or chatting with passengers, the driver makes up or adjusts the vehicle-mounted equipment to be in the middle distracted, and the driver is high distracted when sending short messages or making calls by using a mobile phone.
And S13, judging the driving state of the vehicle. And coupling the driving risk level obtained in the S12 with the distraction risk level of the driver, and judging the driving state of the vehicle at the current moment, including three driving states of no danger, danger and very danger.
And S14, further analyzing whether the condition that the automatic driving system takes over the man-machine co-driving automobile is met or not according to the driving state judgment result of the self automobile, namely executing the step S15 when the driving state of the self automobile is in a dangerous or very dangerous driving state, or executing the step S11.
And S15, adaptively distributing the control weight of the driver and the automatic driving system to the man-machine co-driving automobile according to the driving risk index, the driving risk level and the driver distraction risk level, and smoothly converting the driving control weight to the automatic driving system if necessary.
And S16, controlling the man-machine driving of the automobile by the automatic driving system. And after the automatic driving system takes over the vehicle, the driving task is continuously executed until the vehicle passes through the dangerous road section, and when the man-machine co-driven vehicle smoothly passes through the dangerous road section, the driver can select to continuously control the vehicle and also select to continuously drive by the automatic driving system.
Fig. 2 is a flowchart of a work flow for determining a driving risk level of an external environment by using a vehicle exterior sensing module according to an embodiment of the present invention, where the determination of the driving risk level of the external environment in fig. 1 can be implemented by using the vehicle exterior sensing module shown in fig. 2, and includes the following specific steps:
and S211, collecting data. The CAN bus and the vehicle-mounted laser radar sensor are used for collecting vehicle data in real time, wherein the vehicle data comprise the speed, the acceleration, the distance between heads and the like of a vehicle and other vehicles around the vehicle.
And S212, calculating the field force of the risk field. According to the lane line information detected by the vehicle-mounted camera, taking the direction of the lane line as the Y axis of a road coordinate system, taking the vertical direction of the lane line as the X axis of the road coordinate system, and establishing a driving risk field model of a potential energy field related to a lane:
E T ＝E L +E B +E V (1)
in the formula (1), E T Total field strength for driving risk field, E L Is the field strength of the potential energy field of the lane line, E B To represent the road boundary potential energy field strength, E V The field intensity of the potential energy field of the vehicle;
in the formula (1), the field intensity of the lane line potential energy field consisting of the lane line potential energy field and the double yellow line potential energy field in the road environment is calculated by adopting the following formula (2):
in the formula (2), A i (i =1,2) represents the field strength coefficients of the potential energy fields of the different types of lane lines, which determine the maximum value of the field strength of the lane lines, where A 1 Field strength coefficient, A, representing the potential energy field of the lane line 2 Field strength coefficient representing a double yellow line potential energy field, typically A 2 >A 1 . x represents the abscissa value of the position of the vehicle under the road coordinate system, x l J represents the position coordinate of the jth lane line along the X-axis direction,
represents the distance from the vehicle to the jth lane line, and rho represents the potential energy field of the lane line along with the vehicleRate of change of speed near or far from lane line;
the lane line potential energy field strengths generated by the surrounding vehicles and the self-vehicle are shown in fig. 3. In the figure, x =3.75m and x =7.5m are two lane boundaries, respectively, and x =11.25m is a double yellow line. The potential energy field strength of the lane boundary is far lower than that of a double yellow line, and the potential energy field strength in the middle of each lane is the lowest, so that the requirement that the vehicle can keep running in the middle of the lane as far as possible can be met;
in the formula (1), the road boundary potential energy field intensity generated at the left and right road boundaries is calculated by adopting the following formula (3):
in the formula (2), x b Z represents a position coordinate of the z-th road boundary line in the X-axis direction,
the distance from the vehicle to the z-th road boundary line is represented, and eta represents the road boundary field intensity coefficient;
the road boundary potential energy field strength generated by the surrounding vehicles and the self-vehicle is shown in fig. 4. The road boundary in the image is arranged at the leftmost side of the road, the smaller the distance between the road boundary and the image is, the larger the potential energy field of the road boundary is, and the infinity is achieved at the leftmost side of the road;
in the formula (1), the vehicle potential energy field strength including the motion state information is calculated by the following formula (4):
in the formula (4), M represents the equivalent mass of the vehicle, M represents the actual mass of the vehicle, v represents the traveling speed of the vehicle at the current time, and d' represents the spatial coordinate (x) from the center of mass of the vehicle around a certain periphery 0 ,y 0 ) Safe distance to space coordinates (x, y) of the center of mass of the vehicle, tau represents a critical threshold value of the safe distance between the surrounding vehicles and the vehicle, theta represents the center of mass of a certain surrounding vehicle and the vehicleAn included angle formed by a connecting line of the mass centers and the motion direction of the vehicle, wherein a represents the acceleration of the vehicle at the current moment;
the vehicle potential energy field strength generated by the surrounding vehicles and the own vehicle is shown in fig. 5. The vehicle potential field is distributed in an ellipse on the whole in the figure, which shows that the vehicle is allowed to be relatively close to the side vehicle or a lane line in the driving process, the strength of the vehicle potential field reaches the maximum value at the center of the vehicle and gradually decreases along with the increase of the safety distance d';
and S213, calculating the driving risk index. Establishing a risk field model of the surrounding driving environment according to the formula (1), and calculating the driving risk index RI at the moment t t ：
In formula (5), F represents the field force of the risk field to which the vehicle is subjected, and F denotes the standard risk index. Among them, the time difference to collision (TTC) and following distance (THW) have been widely considered in previous studies as measures of the potential driving risk of a vehicle in a dangerous scene. THW of 1s and TTC of 4s are suggested warning criteria, so the field force value F experienced by the vehicle at TTC of 4s is taken as the standard risk index;
and S214, judging the driving risk level. Setting omega 1 And omega 2 Respectively low risk driving threshold and high risk driving threshold, and calculating the obtained driving risk index RI t Comparing with different driving risk threshold values, and judging the driving risk level R of the surrounding environment at the current moment t The specific process is as follows:
(1) If RI t <ω 1 Then R is t ＝R 0 I.e. the current driving risk level is R 0 Indicating that the surrounding driving environment is risk-free, and the grade risk strength is assigned as
(2) If ω is 1 ＜RI t <ω 2 Then R is t ＝R 1 I.e. the current driving risk level is R 1 Indicate surroundingsThe driving environment is low risk, and the grade risk intensity is assigned as
(3) If RI t >ω 2 Then R is t ＝R 2 I.e. the current driving risk level is R 2 Indicating a high risk of the surrounding driving environment, and assigning a value of the magnitude of the risk to
Fig. 6 is a flowchart of a work flow for identifying a distraction risk level of a driver by using an in-vehicle sensing module according to an embodiment of the present invention, where the identification of the distraction risk level of the driver in fig. 1 is implemented by the in-vehicle sensing module shown in fig. 6, and includes the following specific steps:
and S221, classifying the common distraction behaviors of the driver. Preprocessing videos and pictures of distracting behaviors of a Driver in a vehicle in a public data set State Farm displaced Driver Detection to obtain six types of common distracting driving behaviors, wherein the six types of common distracting driving behaviors comprise that the Driver stretches to the back, the Driver chats with passengers, the Driver makes up, the Driver adjusts vehicle-mounted equipment, and the Driver sends short messages and makes calls by using a mobile phone;
and S222, dividing the danger level of the common distraction behaviors. Four danger levels are divided according to the danger degree of each type of distraction behavior, and the specific corresponding relation is as follows: the safe driving mark of the driver is no distraction D0, the driver stretches behind or chats with passengers to mark low distraction D1, the driver makes up or adjusts the vehicle-mounted equipment to be medium distraction D2, and the driver uses a mobile phone to send a short message or make a call to be high distraction D3. And carrying out distraction risk level marking on 22424 collected driving images, wherein the number of samples of each risk level behavior is about 5600. For the pre-labeled data set, the following steps are performed: 1, dividing the ratio into a training set and a test set, wherein the training set is used for optimizing the recognition model of the distraction behavior of the driver, and the test set is used for testing the classification accuracy of the recognition model. The training set and the testing set respectively comprise behaviors of no distraction, low distraction, middle distraction and high distraction of the driver;
s223, constructing a long-time memory neural network model with an attention mechanism for identifying the distraction behavior of the driver, and being characterized in that: the pre-tagged data image is transferred to a Long short-term memory (LSTM) neural network, which is a special recurrent neural network structure with internal cells that provide Long-term and short-term memory and help the network process Long data sequences. The long-time and short-time memory neural network layer with the attention mechanism consists of an input gate, a forgetting gate and an output gate. Wherein, h hidden units are provided, the batch size is n, the input number is d, and the input is X t ∈R n×d The hidden state of the previous time step t-1 is H t-1 ∈R n×h Then, the calculation process of LSTM internal cells with attention mechanism is as follows:
the first step is to use the forgetting door F t Determining information that the cell will forget, the formula is as follows:
F t ＝σ(X t W xf +H t-1 W hf +b f ) (6)
in the formula (6), σ represents sigmoid activation function, W xf And W hf Weight parameter representing forgetting gate, b f Indicating the biasing parameters of the forgetting gate.
The second step is that the decision unit will use the input gate I t Candidate memory element
And memory element C t The content of the stored information is expressed as follows:
I t ＝σ(X t W xi +H t-1 W hi +b i ) (7)
in formula (7) -formula (9), W xi And W hi Weight parameter, W, representing input gate xc And W hc Weight parameter representing candidate memory gate, b i And b c Offset parameters representing input gate and candidate memory cell, respectively, C t-1 Representing the memory cell of the previous time step t-1.
The third step is to use the output cell to determine the output gate O of the cell t And hidden state H t The formula is as follows:
O t ＝σ(X t W xo +H t-1 W ho +b o ) (10)
H t ＝C t ⊙tanh(O t ) (11)
in formula (10), W xo And W ho Weight parameter representing output gate, b o Indicating the bias parameter of the output gate.
In order to fully utilize all hidden states of the LSTM storage unit in the last layer, an attention mechanism is added to the LSTM layer, a trainable weight value is distributed to each hidden state, and the characteristics of distraction of a driver are captured better. The formula is as follows:
in equation (12), N denotes a feature vector of the LSTM layer output, epsilon denotes a trainable weight vector, alpha denotes a distribution coefficient matrix,
is the feature vector output by the attention mechanism model.
Finally, the danger level of the distraction behavior of the driver at the time t is output through a full connection layer with a SoftMax classifier, and the output result is D t ∈[0,1,2,3]. When Dt =0, the result of the distraction risk level identification is non-distraction D0, and the score of the level distraction strength is S0=0; when Dt =1, the distraction risk grade identification result is low distraction D1, and the grade distraction strength is assigned to S1=1.5; when Dt =2, the distraction risk level identification result is middleA distraction D2, the fractional distraction strength assigned S2=2; when Dt =3, the distraction risk level identification result is high distraction D3, and the fractional distraction strength is assigned S3=2.5.
The method for determining the driving state S13 of the vehicle in fig. 1 includes the following specific steps:
and coupling the driving risk level obtained by the S12 with the distraction risk level of the driver, and judging the driving state of the vehicle at the current moment, wherein the driving states comprise three driving states of no danger, danger and very danger, and the specific process is as follows:
when the surrounding driving risk level is no risk R0 and the driver distraction risk level is no distraction D0, it is indicated that no risk exists around the vehicle and the driver distraction does not exist, the driver should hold all driving control rights, and the driving state of the vehicle is in a no risk state;
when the surrounding driving risk level is no risk R0, and the driver distraction risk level is low distraction D1, middle distraction D2 and high distraction D3, it is indicated that no risk exists around the vehicle, but because the driver generates distraction behaviors with different levels and cannot control the vehicle well, a part of driving control right should be mastered by the automatic driving system, and the driving state of the vehicle is in a dangerous state;
when the surrounding driving risk level is low risk R1, and the driver distraction risk level is non-distraction D0, low distraction D1, middle distraction D2 and high distraction D3, the surrounding driving risk is low, but the driver generates partial distraction dangerous behaviors to cause unstable vehicle driving, and an automatic driving system needs to master a part of driving control rights, and the driving state of the vehicle is in a dangerous state;
when the surrounding driving risk level is high risk R2, and the driver distraction risk level is non-distraction D0, low distraction D1 and distraction D2, the surrounding driving risk is high, but most of the attention of the driver is observing the current driving environment change when the driver executes a driving task with low distraction, and an automatic driving system should master a part of driving control right, and the driving state of the vehicle is in a dangerous state;
when the surrounding driving risk level is high risk R2 and the driver distraction risk level is high distraction D3, it is indicated that the surrounding driving risk is very high and the driver is influenced by distraction, and the driving state of the vehicle is a very dangerous state at the moment.
Fig. 7 is a flowchart of a control right adaptive switching module according to an embodiment of the present invention, where the adaptive distribution of the control weights of the driver and the automatic driving system in fig. 1 for the man-machine co-driving vehicle may be implemented by the control right adaptive switching module shown in fig. 7, which includes the following specific steps:
s411, if the condition that the automatic driving system takes over the man-machine co-driving automobile is met, namely when the driving state of the self-driving automobile is in a dangerous or very dangerous driving state, the control weight of the driver and the automatic driving system to the man-machine co-driving automobile is adaptively distributed according to the driving risk index, the driving risk level and the driver distraction risk level, and the driving control weight is stably converted to the automatic driving system if necessary, and the specific process is as follows:
based on ten dangerous driving states of R0D1, R0D2, R0D3, R1D0, R1D1, R1D2, R1D3, R2D0, R2D1 and R2D2 in S13, under different driving risk levels, part of attention of drivers in a vehicle can be influenced by distraction, but most of the drivers still have driving right of a capacity control part, in order to ensure driving safety, the drivers are informed of that the driving control right is about to be converted by sound early warning, and the driving control right is gradually transferred to an automatic driving system, and at the moment, the driving control right of the drivers and the automatic driving system is as follows:
in the formula (13), μ (t) is the control weight of the automatic driving system, ν (t) is the control weight of the driver, RI m The RI is taken for the maximum driving risk index calculated by a large amount of dangerous driving data sets m ＝2；
Based on the dangerous driving state of R2D3 in S13, the surrounding driving risk is very high, the driver is influenced by the distraction behavior, the driver needs to be informed of the dangerous driving state by emergency sound early warning, the driving control right can be immediately switched, and the automatic driving system takes over the dangerous driving state in an emergency mode, wherein the formula is as follows:
finally, it should be noted that the above embodiments of the present invention are specifically explained and illustrated, and only the method and the core design concept of the present invention are embodied, but not limited to the above embodiments. Those skilled in the art can make equivalent changes or modifications without departing from the principle and spirit of the present specification, and such changes should be considered as within the scope of protection.
Claims (6)
1. A man-machine driving-sharing control right self-adaptive switching method based on vehicle interior and exterior cooperative sensing is characterized by comprising the following steps:
s11, controlling a man-machine to drive the automobile together by a driver;
s12, judging the driving risk level of the external environment by using an external sensing module, and identifying the distraction risk level of a driver by using an internal sensing module;
s13, judging the driving state of the vehicle by coupling the driving risk grade judgment result and the driver distraction risk grade identification result;
s14, further analyzing whether the condition that an automatic driving system takes over the man-machine co-driving of the automobile is met or not according to the driving state judgment result of the automobile;
s15, if yes, the control weight of the driver and the automatic driving system to the man-machine co-driving automobile is distributed in a self-adaptive mode, and otherwise, the driver continues to control the man-machine co-driving automobile;
and S16, gradually switching the control weight to an automatic driving system according to the calculated control weight, and finally controlling the man-machine co-driving automobile by the automatic driving system.
2. The human-computer co-driving automobile control right adaptive switching method based on in-vehicle and out-vehicle cooperative sensing is characterized in that an out-vehicle sensing module is used for judging the driving risk level of an external environment, and the method further comprises the following steps: data acquisition, field force calculation of a driving risk field, driving risk index calculation and driving risk grade judgment; wherein the driving risk grades are no risk, low risk and high risk.
3. The human-computer co-driving automobile control right adaptive switching method based on in-vehicle and out-vehicle cooperative sensing is characterized in that the method for judging the external environment driving risk level by using the out-vehicle sensing module specifically comprises the following steps:
s211, data acquisition
The CAN bus and the vehicle-mounted laser radar sensor are used for collecting vehicle data in real time, wherein the vehicle data comprises the speed, the acceleration and the distance between the head of the vehicle and other surrounding vehicles;
s212, calculating the field force of the risk field
According to the lane line information detected by the vehicle-mounted camera, taking the direction of the lane line as the Y axis of a road coordinate system, taking the vertical direction of the lane line as the X axis of the road coordinate system, and establishing a driving risk field model of a potential energy field related to a lane:
E T ＝E L +E B +E V (1)
in the formula (1), E T Total field strength for driving risk field, E L Is the field strength of the potential energy field of the lane line, E B To represent the road boundary potential energy field strength, E V The field intensity of the potential energy field of the vehicle;
in the formula (1), the field intensity of the lane line potential energy field consisting of the lane line potential energy field and the double yellow line potential energy field in the road environment is calculated by adopting the following formula (2):
in the formula (2), A i (i =1, 2) field intensity coefficients of potential energy fields of different types of lane lines, x represents an abscissa value of a position of a host vehicle in a road coordinate system, and x represents an abscissa value of a position of the host vehicle in the road coordinate system l,j Indicating the position coordinates of the jth lane line in the X-axis direction,
representing the distance from the vehicle to the jth lane line, and rho representing the speed change rate of the potential energy field of the lane line along with the approaching or departing of the vehicle from the lane line;
in the formula (1), the road boundary potential energy field intensity generated at the left and right road boundaries is calculated by adopting the following formula (3):
in formula (3), x b,z Indicating the position coordinates of the z-th road boundary line in the X-axis direction,
the distance from the vehicle to the z-th road boundary line is represented, and eta represents the road boundary field intensity coefficient;
in the formula (1), the vehicle potential energy field strength including the motion state information is calculated by the following formula (4):
in the formula (4), M represents the equivalent mass of the vehicle, M represents the actual mass of the vehicle, v represents the traveling speed of the vehicle at the current time, and d' represents the spatial coordinate (x) from the center of mass of the vehicle around a certain periphery 0 ,y 0 ) The safety distance to the space coordinates (x, y) of the center of mass of the vehicle, tau represents a critical threshold value of the safety distance between the surrounding vehicles and the vehicle, theta represents an included angle formed by a connecting line of the center of mass of a certain surrounding vehicle and the center of mass of the vehicle and the motion direction of the vehicle, and a represents the acceleration of the vehicle at the current moment;
s213, calculating the driving risk index
Establishing a risk field model of the surrounding driving environment according to the formula (1), and calculating the driving risk index RI at the moment t t ：
In the formula (5), F represents the field force of the risk field suffered by the self vehicle, and F represents the standard risk index;
s214, judging the driving risk level
Setting omega 1 And ω 2 Respectively low risk driving threshold and high risk driving threshold, and calculating the obtained driving risk index RI t Comparing with different driving risk threshold values, and judging the driving risk grade R of the surrounding environment at the current moment t The specific process is as follows:
(1) If RI t <ω 1 Then R is t ＝R 0 I.e. the current driving risk level is R 0 Indicating that the surrounding driving environment is risk-free, and the grade risk strength is assigned as
(2) If ω is 1 ＜RI t <ω 2 Then R is t ＝R 1 I.e. the current driving risk level is R 1 Indicating a low risk of the surrounding driving environment, the magnitude of the risk strength is assigned as
(3) If RI t >ω 2 Then R is t ＝R 2 I.e. the current driving risk level is R 2 Indicating a high risk of the surrounding driving environment, and assigning a value of the magnitude of the risk to
4. The self-adaptive switching method for human-computer co-driving automobile control right based on in-vehicle and outside-vehicle cooperative sensing is characterized in that an in-vehicle sensing module is used for identifying the distraction risk level of a driver, and the method further comprises the following steps: classifying common distracting behaviors of drivers, classifying danger levels, and constructing a long-time memory neural network model with an attention mechanism for recognizing the distracting behaviors of the drivers; the corresponding relation between the distraction behavior of the driver and the danger level is as follows: the driver is not distracted when driving safely, the driver is low distracted when stretching behind or chatting with passengers, the driver makes up or adjusts the vehicle-mounted equipment to be in the middle distracted, and the driver is high distracted when sending short messages or making calls by using a mobile phone.
5. The human-computer co-driving automobile control right adaptive switching method based on in-vehicle and out-vehicle cooperative sensing is characterized in that the method for judging the driving state of the automobile further comprises the following steps: and coupling the driving risk level and the distraction risk level of the driver, and judging the driving state of the vehicle at the current moment, including three driving states of no risk, danger and very danger.
6. The human-computer co-driving automobile control weight self-adaptive switching method based on in-vehicle and in-vehicle cooperative sensing is characterized in that the method for self-adaptively distributing the control weights of a driver and an automatic driving system to a human-computer co-driving automobile comprises the following steps: and if the driving state of the self-vehicle meets the condition that the automatic driving system takes over the man-machine co-driving of the vehicle, namely the driving state of the self-vehicle is in a dangerous or very dangerous driving state, adaptively distributing the driver and the corresponding driving control weight of the automatic driving system according to the driving risk index, the driving risk level and the driver distraction danger level, and stably converting the driving control weight to the automatic driving system if necessary.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211209546.9A CN115534994A (en) | 2022-09-30 | 2022-09-30 | Man-machine driving sharing control right self-adaptive switching method based on cooperative sensing inside and outside vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211209546.9A CN115534994A (en) | 2022-09-30 | 2022-09-30 | Man-machine driving sharing control right self-adaptive switching method based on cooperative sensing inside and outside vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115534994A true CN115534994A (en) | 2022-12-30 |
Family
ID=84731885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211209546.9A Pending CN115534994A (en) | 2022-09-30 | 2022-09-30 | Man-machine driving sharing control right self-adaptive switching method based on cooperative sensing inside and outside vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115534994A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116279572A (en) * | 2023-02-08 | 2023-06-23 | 中南大学 | Vehicle safety situation assessment and steady driving mode switching method and system |
| CN117037545A (en) * | 2023-10-09 | 2023-11-10 | 济南卓伦智能交通技术有限公司 | Multi-vehicle beyond-sight-distance collaborative sensing method |
-
2022
- 2022-09-30 CN CN202211209546.9A patent/CN115534994A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116279572A (en) * | 2023-02-08 | 2023-06-23 | 中南大学 | Vehicle safety situation assessment and steady driving mode switching method and system |
| CN116279572B (en) * | 2023-02-08 | 2024-01-30 | 中南大学 | Vehicle safety situation assessment and steady driving mode switching method and system |
| CN117037545A (en) * | 2023-10-09 | 2023-11-10 | 济南卓伦智能交通技术有限公司 | Multi-vehicle beyond-sight-distance collaborative sensing method |
| CN117037545B (en) * | 2023-10-09 | 2024-01-12 | 济南卓伦智能交通技术有限公司 | Multi-vehicle beyond-sight-distance collaborative sensing method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110488802B (en) | Decision-making method for dynamic behaviors of automatic driving vehicle in internet environment | |
| CN115534994A (en) | Man-machine driving sharing control right self-adaptive switching method based on cooperative sensing inside and outside vehicle | |
| CN108995655B (en) | Method and system for identifying driving intention of driver | |
| CN109789777B (en) | Unintended pulse-change collision detector | |
| CN109885040B (en) | Vehicle driving control right distribution system in man-machine driving | |
| CN107544518B (en) | ACC/AEB system based on anthropomorphic driving and vehicle | |
| Althoff et al. | Model-based probabilistic collision detection in autonomous driving | |
| CN111079834A (en) | Intelligent vehicle safety situation assessment method considering multi-vehicle interaction | |
| CN113056749A (en) | Future object trajectory prediction for autonomous machine applications | |
| CN110077398B (en) | Risk handling method for intelligent driving | |
| CN111452789A (en) | Automatic driving overtaking control method and system | |
| CN110182210A (en) | Controller of vehicle and control method for vehicle | |
| CN113392692A (en) | Driver-centric risk assessment: risk object identification via causal reasoning for intent-aware driving models | |
| CN107521501A (en) | Driver assistance system decision-making technique, system based on game theory and other | |
| CN113012469A (en) | Intelligent traffic early warning system based on target recognition | |
| CN106218612B (en) | A kind of method, apparatus and terminal of vehicle safety travel | |
| CN115027484A (en) | Human-computer fusion perception method for high-degree automatic driving | |
| CN113147752A (en) | Unmanned driving method and system | |
| CN112046494A (en) | Vehicle control method, device, equipment and storage medium | |
| CN110634296A (en) | Emergency processing system and method for intelligent driving | |
| CN117141519A (en) | Unmanned system and method based on image processing | |
| CN112874515B (en) | System and method for carrying out safety reminding on driving assistance system by using driving posture | |
| CN113159576B (en) | Driving risk calculation method and online evaluation system for automatically driving automobile | |
| CN114274954A (en) | Vehicle active pipe connecting system and method combining vehicle internal and external sensing | |
| CN111038521A (en) | Method for forming automatic driving consciousness decision model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |