WO2022062659A1

WO2022062659A1 - Intelligent driving control method and apparatus, vehicle, electronic device, and storage medium

Info

Publication number: WO2022062659A1
Application number: PCT/CN2021/109831
Authority: WO
Inventors: 伍俊; 范亦卿; 陶莹; 许亮; 祁凯悦
Original assignee: 上海商汤临港智能科技有限公司
Priority date: 2020-09-23
Filing date: 2021-07-30
Publication date: 2022-03-31
Also published as: CN112141119A; JP2023542992A; CN112141119B

Abstract

The present disclosure relates to an intelligent driving control method and apparatus, a vehicle, an electronic device, and a storage medium. The method comprises: acquiring travelling state information of a vehicle; acquiring image information of a driving area of the vehicle; determining a driving state of a driver of the vehicle according to the travelling state information of the vehicle and the image information of the driving area; and in response to detecting that the driver of the vehicle is in a preset dangerous driving state, performing intelligent driving control.

Description

Intelligent driving control method and device, vehicle, electronic device and storage medium

technical field

The present disclosure relates to the field of computer technology, and in particular, to an intelligent driving control method and device, a vehicle, an electronic device, and a storage medium.

Background technique

In the process of driving the vehicle, the driver is in a dangerous driving state, which poses a hidden danger to the driving safety and is likely to cause traffic accidents. Therefore, how to accurately detect the driver's dangerous driving state and take timely measures is of great significance for safe driving.

SUMMARY OF THE INVENTION

The present disclosure proposes a technical solution for intelligent driving control.

According to an aspect of the present disclosure, an intelligent driving control method is provided, including:

Acquiring driving state information of the vehicle; acquiring image information of the driving area of the vehicle; determining the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area; in response to detecting The driver of the vehicle is in a preset dangerous driving state and performs intelligent driving control.

In a possible implementation manner, the determining the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area includes:

Detecting the behavior of the driver of the vehicle according to the image information of the driving area;

The driving state of the driver is determined according to the driving state information of the vehicle and the detection result of the driver's behavior.

In a possible implementation manner, the performing behavior detection on the driver of the vehicle includes:

determining the danger level corresponding to the behavior detection result of the driver;

The determining of the driving state of the driver according to the driving state information of the vehicle and the behavior detection result of the driver includes:

In response to determining that the vehicle is in an abnormal driving state according to the driving state information of the vehicle, upgrading the danger level of the driver's behavior detection result to a first danger level;

In response to determining that the first dangerous level reaches a preset warning level, it is determined that the driving state of the driver is in a preset dangerous driving state.

In a possible implementation manner, the determining the driving state of the driver according to the driving state information of the vehicle and the behavior detection result of the driver includes:

In response to determining that the driving state of the vehicle and the behavior state of the driver are both abnormal states according to the driving state information of the vehicle and the behavior detection result of the driver, it is determined that the driver is in a preset state. Dangerous driving condition.

In response to detecting that the vehicle is in an abnormal driving state according to the driving state information of the vehicle, performing behavior detection on the driver according to the image information of the driving area;

In response to determining that the driver's behavior detection result is the driver's distracted driving or fatigued driving, it is determined that the driver is in a preset dangerous driving state.

In a possible implementation manner, the abnormal driving state of the vehicle includes at least one of the following:

The number of times of line pressing of the vehicle within the first set time period reaches the threshold of the number of times of line pressing; the left and right swing of the vehicle reaches the set amplitude threshold; the vehicle does not pass according to the instructions of traffic signs or traffic lights; the speed of the vehicle exceeds the set speed threshold.

In a possible implementation manner, the performing intelligent driving control includes generating alarm information based on at least one of the following:

the driving state information of the vehicle;

The result of detecting the behavior of the driver of the vehicle based on the image information of the driving area;

the driving state of the driver.

In a possible implementation manner, the performing intelligent driving control includes:

The warning device is controlled to continuously output warning information until both the driving state of the vehicle and the driving state of the driver return to the normal state.

In response to detecting that the driver is in a preset dangerous driving state for a duration exceeding a preset time period, the assisted driving/automatic driving function is controlled to be activated, and/or the vehicle is controlled to decelerate to a stop.

In a possible implementation, the method further includes:

obtain environmental information around the vehicle;

The determining of the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area includes:

The driving state of the driver of the vehicle is determined according to the environmental information around the vehicle, the driving state information of the vehicle, and the image information of the driving area.

In a possible implementation manner, the environmental information includes road condition information;

The acquiring environmental information around the vehicle includes: acquiring road condition information of the road on which the vehicle is traveling;

The determining of the driving state of the driver of the vehicle according to the environmental information around the vehicle, the driving state information of the vehicle and the image information of the driving area includes:

In response to that the driving state information of the vehicle and the road condition information do not satisfy a preset matching relationship, and the result of detecting the behavior of the driver of the vehicle based on the image information of the driving area indicates the behavior state of the driver In the abnormal state, it is determined that the driver is in a preset dangerous driving state.

In a possible implementation manner, the acquiring the driving state information of the vehicle includes:

Acquiring driving sensing data, the sensing data includes at least one of the following: image information collected by an advanced driving assistance system of the vehicle, image information collected by a driving recorder of the vehicle, and speed information sensed by a speed sensor of the vehicle;

According to the sensing data, the driving state information of the vehicle is determined.

The obtained driving state information of the vehicle and the image information of the driving area are input into a trained neural network, and the driving state of the driver of the vehicle is determined through the neural network.

According to an aspect of the present disclosure, an intelligent driving control device is provided, comprising:

a first acquisition module, used for acquiring the driving state information of the vehicle;

a second acquisition module, configured to acquire image information of the driving area of the vehicle;

a driving state determination module, configured to determine the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area;

The control module is configured to perform intelligent driving control in response to detecting that the driver of the vehicle is in a preset dangerous driving state.

In a possible implementation manner, the driving state determination module includes a first driving state determination submodule and a second driving state determination submodule, wherein:

The first driving state determination sub-module is configured to perform behavior detection on the driver of the vehicle according to the image information of the driving area;

The second driving state determination sub-module is configured to determine the driving state of the driver according to the driving state information of the vehicle and the behavior detection result of the driver.

In a possible implementation manner, the first driving state determination sub-module is configured to determine the danger level corresponding to the behavior detection result of the driver;

The second driving state determination submodule is used for:

In a possible implementation manner, the second driving state determination sub-module is configured to determine the driving state of the vehicle and the driving state of the vehicle in response to the driving state information of the vehicle and the detection result of the driver's behavior The behavior states of the driver are all abnormal states, and it is determined that the driver is in a preset dangerous driving state.

In a possible implementation manner, the driving state determination module includes a third driving state determination submodule and a fourth driving state determination submodule, wherein:

The third driving state determination sub-module is configured to perform behavior detection of the driver according to the image information of the driving area in response to detecting that the vehicle is in an abnormal driving state according to the driving state information of the vehicle;

The fourth driving state determination sub-module is configured to determine that the driver is in a preset dangerous driving state in response to determining that the driver's behavior detection result is the driver's distracted driving or fatigued driving.

In a possible implementation manner, the control module is configured to generate alarm information based on at least one of the following:

the driving state information of the vehicle;

the driving state of the driver.

In a possible implementation manner, the control module is configured to control the warning device to continuously output warning information until both the driving state of the vehicle and the driving state of the driver return to a normal state.

In a possible implementation manner, the control module is configured to, in response to detecting that the driver is in a preset dangerous driving state for a duration exceeding a preset duration, control the assisted driving/automatic driving function to be activated, and/ Or control the vehicle to slow down and stop.

In a possible implementation, the apparatus further includes:

a third acquiring module, configured to acquire environmental information around the vehicle;

The driving state determination module is configured to determine the driving state of the driver of the vehicle according to the environment information around the vehicle, the driving state information of the vehicle and the image information of the driving area.

The third obtaining module is configured to obtain road condition information of the road on which the vehicle is traveling;

The driving state determination module is configured to detect the behavior of the driver of the vehicle based on the image information of the driving area in response to the fact that the driving state information of the vehicle and the road condition information do not satisfy a preset matching relationship The result indicates that the behavior state of the driver is an abnormal state, and it is determined that the driver is in a preset dangerous driving state.

In a possible implementation manner, the first acquisition module includes a sensor data acquisition sub-module and a driving state information determination sub-module, wherein:

The sensing data acquisition sub-module is used for acquiring driving sensing data, and the sensing data includes at least one of the following: image information collected by the vehicle's advanced driving assistance system, image information collected by the vehicle's driving recorder, vehicle The speed information sensed by the speed sensor;

The driving state information determination sub-module is configured to determine the driving state information of the vehicle according to the sensing data.

In a possible implementation manner, the driving state determination module is configured to input the acquired driving state information of the vehicle and the image information of the driving area into a trained neural network, and determine through the neural network The driving state of the driver of the vehicle.

A vehicle, characterized in that it includes:

a first sensor, used for collecting image information of the driving area of the vehicle;

a controller, configured to acquire the driving state information of the vehicle, determine the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area, and respond to detecting the driving of the vehicle The driver is in a preset dangerous driving state and performs intelligent driving control.

In a possible implementation manner, the vehicle further includes: a second sensor for collecting driving sensing data;

The controller is configured to acquire the driving state information of the vehicle according to the driving sensing data.

According to an aspect of the present disclosure, there is provided an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions implementing the above method when executed by a processor.

According to an aspect of the present disclosure, there is provided a computer program comprising computer-readable code, which when the computer-readable code is executed in an electronic device, is executed by a processor in the electronic device for implementing the above method.

In the embodiment of the present disclosure, by acquiring the driving state information of the vehicle and the image information of the driving area of the vehicle, the driving state of the driver of the vehicle is determined according to the driving state information of the vehicle and the image information of the driving area, and in response to the detection The driver of the vehicle is in a preset dangerous driving state and performs intelligent driving control. Thereby, the driving state of the driver of the vehicle can be determined by combining the driving state of the vehicle and the image information of the driving area, which improves the accuracy of the determined driving state of the driver. In addition, in response to detecting that the driver of the vehicle is in a preset dangerous driving state, intelligent driving control is performed, which further improves the driving safety of the vehicle.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the technical solutions of the present disclosure.

FIG. 1 shows a flowchart of an intelligent driving control method provided by an embodiment of the present disclosure.

FIG. 2 shows a block diagram of an intelligent driving control device provided by an embodiment of the present disclosure.

FIG. 3 shows a block diagram of an electronic device 800 provided by an embodiment of the present disclosure.

FIG. 4 shows a block diagram of an electronic device 1900 provided by an embodiment of the present disclosure.

detailed description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures denote elements that have the same or similar functions. While various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, may mean including from A, Any one or more elements selected from the set of B and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are set forth in the following detailed description. It will be understood by those skilled in the art that the present disclosure may be practiced without certain specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present disclosure.

The present disclosure provides an intelligent driving control method, and the execution body of the method may be an intelligent driving control device installed on a vehicle. In a possible implementation, the method may be executed by a terminal device or a server or other processing device. Wherein, the terminal device may be a vehicle-mounted device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device or a wearable devices, etc. Among them, the in-vehicle device may be a vehicle or a domain controller in the cabin, or may be an ADAS (Advanced Driving Assistance System), an OMS (Occupant Monitoring System, an occupant monitoring system) or a DMS (Driver Monitoring System, A device host used in a driver monitoring system) for executing an intelligent driving control method, etc. In some possible implementations, the intelligent driving control method may be implemented by the processor calling computer-readable instructions stored in the memory.

FIG. 1 shows a flowchart of an intelligent driving control method according to an embodiment of the present disclosure. As shown in FIG. 1 , the intelligent driving control method includes:

In step S11, the driving state information of the vehicle is obtained;

The vehicle here may be at least one type of vehicle, such as a private car, a shared car, an online car-hailing, a taxi, a truck, and the like, and the specific type of the vehicle is not limited in the present disclosure.

The driving state information is used to characterize the driving state of the vehicle. In a possible implementation, the driving state can be divided into a normal driving state and an abnormal driving state. The normal driving state can be that the vehicle follows a predetermined speed, direction, route and traffic regulations. Wait for the rules to follow. The abnormal driving state corresponds to the normal driving state, which may be driving not in accordance with predetermined rules such as speed, direction, route, and traffic regulations.

The driving state information may include information that characterizes the normal driving state or abnormal driving state of the vehicle, such as the speed, direction, acceleration of the vehicle, information on the lane in which it is located, line pressure information, body swing information, lane change information, speed change information, and brake information , at least one of braking information, route information, speeding situation, consistency information of driving state and traffic signs, and the like.

In a possible implementation manner, the abnormal driving state of the vehicle includes at least one of the following: the number of times of line pressing of the vehicle within the first set period of time reaches a set threshold of the number of times of line pressing; Amplitude threshold; the vehicle is not passing as indicated by traffic signs or traffic lights; the speed of the vehicle exceeds the set speed threshold.

Among them, the number of times of line pressing of the vehicle within the first set period of time reaches the threshold of the set number of line pressing times, indicating that the vehicle continuously presses the line in a relatively short period of time. In this case, it can be determined that the vehicle is in an abnormal driving state. The first set duration can be set according to the actual situation, for example, it can be 1 minute or 15 seconds, and the threshold value of the number of times to set the line can also be set according to the actual situation, for example, it can be 2 times, which is not specifically limited in this disclosure. .

When the left and right swing amplitude of the vehicle reaches the set amplitude threshold, it indicates that the left and right swing amplitude of the vehicle body is relatively large, and in this case, it can be determined that the vehicle is in an abnormal driving state. The set amplitude threshold may be set according to actual conditions such as the type of the vehicle, which is not specifically limited in the present disclosure.

In addition, if the vehicle does not pass according to the instructions of traffic signs or traffic lights, and the speed of the vehicle exceeds the set speed threshold, it also indicates that the vehicle is in an abnormal driving state, which will not be repeated here.

The driving state information of the vehicle can be determined according to the driving speed, driving direction, position of the vehicle, and the surrounding environment of the vehicle, etc., which can be used for sensing the external environment information of the vehicle cabin. The sensor that senses the running state is obtained.

In an optional implementation manner, the acquiring the driving state information of the vehicle includes: acquiring driving sensing data, where the sensing data includes at least one of the following: image information collected by an ADAS of the vehicle; The image information collected by the driving recorder of the vehicle and the speed information sensed by the speed sensor of the vehicle; and the driving state information of the vehicle is determined according to the sensing data.

For example, through the image acquisition device, millimeter wave/lidar, sonar sensor, etc. on the body, to obtain the image, temperature, pressure and other information outside the car cabin for monitoring the state of the car, these sensors can be installed in the front and rear insurance of the vehicle bar, side mirrors, inside the steering column, or on the windshield to capture perception data outside the cabin. Through these perception data, the environment around the vehicle can be detected, including objects such as pedestrians and vehicles around the vehicle, as well as information such as lane lines, traffic lights, and traffic signs. In addition, information such as the speed, acceleration, position, and attitude of the vehicle can also be determined through a speed/acceleration sensor, an attitude sensor, a positioning device, and the like.

In step S12, acquiring image information of the driving area of the vehicle;

The driving area here may be the area where the driver is located in the cabin, or any area including the area where the driver is located, and the area is usually the area of the main driver's seat.

Then, the image information of the driving area may be the image information of the area where the driver is located in the cabin, and the image information may be collected by an in-vehicle image acquisition device disposed in the cabin of the vehicle or outside the cabin, and the in-vehicle image acquisition device may be Vehicle camera or image acquisition device equipped with camera. The camera may be a camera for collecting image information inside the vehicle, or a camera for collecting image information outside the vehicle.

For example, the camera may include a camera in a DMS and/or a camera in an OMS, etc., these cameras may be used to collect image information inside the vehicle; the camera may also include a camera in ADAS, which may be used to collect image information outside the vehicle . Of course, the in-vehicle image acquisition device may also be a camera in other systems, or may also be a separately configured camera, and the embodiment of the present disclosure does not limit the specific in-vehicle image acquisition device.

The carrier of the image information here can be a two-dimensional image or a video. For example, the image information can be a visible light image/video, or an infrared light image/video; it can also be a three-dimensional image formed by a point cloud scanned by a radar, and so on. It may be determined according to the actual application scenario, which is not limited in the present disclosure.

The image information collected by the vehicle can be obtained through the communication connection established with the vehicle image collection device. In one example, the vehicle-mounted image acquisition device can transmit the collected image information to the vehicle-mounted controller or remote server through the bus or wireless communication channel in real time, and the vehicle-mounted controller or the remote server can receive the real-time image information through the bus or wireless communication channel .

In step S13, the driving state of the driver of the vehicle is determined according to the driving state information of the vehicle and the image information of the driving area;

The driving state of the driver may be the state of the driver when driving the vehicle, and the driving state may be divided into a normal driving state and a dangerous driving state. The dangerous driving state may be a state in which the driver has preset irregular driving behaviors, for example, a state under behaviors such as making a phone call, looking at a mobile phone, taking a hand off the steering wheel, and fatigued driving.

In the embodiments of the present disclosure, it is considered that when the driver has irregular driving behavior, the driving state of the vehicle is often changed. For example, when the driver is in a dangerous driving state such as distracted or fatigued driving, the There are often abnormal driving conditions, such as unstable vehicle speed, continuous line pressure and other abnormal driving conditions. Therefore, in the embodiment of the present disclosure, the determination of the driving state of the driver may be based on the image information of the driving area and the driving state information of the vehicle, so as to improve the reliability of the determined driving state of the driver.

As mentioned above, the image information of the driving area is the image information of the area where the driver is located in the cabin. Then, by analyzing and processing the image information of the driving area through image processing technology, the characteristics of the driver can be detected, for example, body characteristics, Facial features, etc. Based on the detected characteristics of the driver, the behavior of the driver can be analyzed to determine whether the driver has made a phone call, drinking water, taking his hands off the steering wheel, closing his eyes, etc.

In the case where the running state of the vehicle is already abnormal, the probability that the driver is in a preset dangerous driving state is high. Therefore, the driving state of the driver can be determined by combining the driving state information of the vehicle with the image information of the driving area. For example, by analyzing the image information of the driving area, it is determined that the confidence level of the driver in the fatigue driving state is 0.5, and at this time the vehicle appears in the driving state of continuous line pressure, the confidence level of the driver in the fatigue driving state can be determined. It is increased to 0.8; for another example, by analyzing the image information of the driving area, it is determined that the driver is in a state of mild fatigue driving, and at this time the vehicle is in a driving state of continuous line pressure, the mild fatigue driving state can be upgraded to Moderate fatigue driving state or severe fatigue driving state.

In step S14, intelligent driving control is performed in response to detecting that the driver of the vehicle is in a preset dangerous driving state.

The preset dangerous driving state is a preset driving state with potential driving safety hazards, which can include one or more predefined driver states, such as moderate fatigue driving state, severe fatigue state, and severe distracted driving. status, etc. The preset dangerous driving state may also be a state defined by a preset driver behavior combined with a preset vehicle state, for example, it is defined that the state when the driver makes a phone call and the vehicle presses the line for more than two consecutive times is the preset dangerous driving state. According to the preset dangerous driving state, intelligent driving control can be performed. The intelligent driving control here can be the vehicle control center or the remote control terminal to intervene driving according to the road conditions the vehicle is driving on. The specific intelligent driving control method can be, for example, issuing an alarm. information, control the activation of assisted driving/autonomous driving functions, control vehicle deceleration and other measures. The details will be described in detail with reference to the possible implementation manners later in the present disclosure, which will not be repeated here.

In some optional implementation manners, the danger level may be determined according to the dangerous driving state of the driver and the driving state of the vehicle, and then the corresponding intelligent driving control mode may be selected according to the danger level. For example, when the driver is in a dangerous driving state such as making a phone call, looking at a mobile phone, or taking his hand off the steering wheel, and the vehicle continues to press the line for less than a preset period of time (for example, 5 seconds), the intelligent driving control is performed by issuing a warning message; when driving When the driver is in a dangerous driving state such as making a phone call, looking at a mobile phone, or taking his hand off the steering wheel, and the vehicle is in a state of overspeeding by 50%, the risk is higher than that when the vehicle is continuously pressing the line for no more than a preset time (for example, 5 seconds). Intelligent driving control is carried out by controlling the deceleration of the vehicle, so as to realize flexible and safe driving control.

In the embodiment of the present disclosure, by acquiring the driving state information of the vehicle and the image information of the driving area of the vehicle, the driving state of the driver of the vehicle is determined according to the driving state information of the vehicle and the image information of the driving area, and in response to the detection The driver of the vehicle is in a preset dangerous driving state and performs intelligent driving control. In this way, the driving state of the driver of the vehicle can be determined in combination with the driving state of the vehicle and the image information of the driving area, which improves the accuracy of the determined driving state of the driver. In addition, in response to detecting that the driver of the vehicle is in a preset dangerous driving state, intelligent driving control is performed, which further improves the driving safety of the vehicle.

In some possible implementations, the image information of the driving area may be image information in the vehicle cabin, and the driving state information of the vehicle may be information determined according to perception data outside the vehicle cabin. That is, in the present disclosure, it may be The driving state of the driver is determined in combination with the information inside and outside the vehicle cabin, which improves the accuracy of the determined driving state of the driver. Intelligent driving control is carried out to improve the driving safety of the vehicle.

In a possible implementation manner, determining the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area includes: determining the driving state of the vehicle according to the image information of the driving area. The driver performs behavior detection; the driving state of the driver is determined according to the driving state information of the vehicle and the behavior detection result of the driver.

In this implementation manner, the behavior of the driver of the vehicle is detected according to the image information of the driving area to obtain the behavior detection result, and then the driving state of the driver is determined in combination with the driving state information of the vehicle and the behavior detection result, which improves the performance. The accuracy of the driver's driving state.

The image information of the driving area can be analyzed by image processing technology to detect the driver's behavior and obtain the behavior detection result. In the embodiment of the present disclosure, the preset dangerous driving state may be that the driver has a preset irregular driving behavior. Then, the behavior detection of the driver of the vehicle according to the image information of the driving area may be the non-standard driving behavior of the driver. Standardize the detection of driving behavior. For example, when it is detected that the driver is holding a phone and the phone is near the ear, it can be determined that the behavior detection result is that the driver has the behavior of making a phone call while driving; it can be detected that the driver's hand is not on the steering wheel. When the duration reaches the first duration, it is determined that the result of the behavior detection is that the driver has the act of leaving the steering wheel with his hands; when it is detected that the duration of the driver's eyes closed reaches the second duration, the result of the behavior detection is determined to be the presence of the driver. Behavior of mildly fatigued driving; it may be determined that the result of the behavior detection is that the driver has a behavior of moderately fatigued driving when it is detected that the duration of the driver's eyes closed reaches the third duration.

In some optional implementation manners, if the detection result is that the unsafe driving behavior of the driver is not detected, if the driving status information of the vehicle indicates that the driving status of the vehicle is also a normal status, the driving status of the driver can be determined for normal driving.

In some optional implementation manners, if the detection result is that the driver's unsafe driving behavior is detected, if the driving state information of the vehicle indicates that the driving state of the vehicle is a normal state, the detected irregularity The driving behavior determines the driving state of the driver. For example, if the behavior detection result is the driver's eye-closing frequency or the number of consecutive eye-closing times within the interval corresponding to mild fatigue driving, if the driving state of the vehicle is normal, the driver's driving state is still determined to be mild Fatigue driving.

In some optional implementation manners, if the detection result is that the driver's unsafe driving behavior is detected, if the driving state information of the vehicle indicates that the driving state of the vehicle is an abnormal state, the driving state information of the vehicle can The driver's behavior detection result is further modified to obtain the driver's driving state, so as to improve the reliability of the obtained behavior detection result. For example, since the abnormal driving state of the vehicle is often caused by the driver's irregular driving behavior, the confidence level of the driver's behavior in the behavior detection result can be increased to be an unsafe driving behavior, so as to improve the reliability of the obtained dangerous driving state. ; In addition, since the vehicle has been in an abnormal driving state, the danger level of the detection result can be increased so that higher-level response measures can be taken.

In a possible implementation manner, the performing behavior detection on the driver of the vehicle includes: determining a danger level corresponding to the behavior detection result of the driver; Determining the driving state of the driver from the behavior detection result of the driver includes: in response to determining that the vehicle is in an abnormal driving state according to the driving state information of the vehicle, upgrading the danger level of the driver's behavior detection result is a first dangerous level; in response to determining that the first dangerous level reaches a preset warning level, it is determined that the driving state of the driver is in a preset dangerous driving state.

The driver's behavior detection result may correspond to the danger level, and the danger level represents the dangerous degree of the driver's dangerous driving state. For example, when it is detected that the driver's eyes are closed for a second period of time, the behavior detection result can be determined as The driver has a behavior of driving with mild fatigue; when it is detected that the driver's eyes are closed for a third period of time, it may be determined that the behavior detection result is that the driver has a behavior of moderately fatigued driving. Among them, the third duration is longer than the second duration. Obviously, the longer the driver's eyes are closed, the higher the degree of danger and the higher the probability of vehicle accidents.

Since the vehicle has been in an abnormal driving state, the danger level of the detection result can be upgraded to take higher-level response measures. For the convenience of description, the upgraded danger level is referred to as the first danger level here. For the first danger level, it can be determined whether the first danger level reaches a preset alarm level, and if so, an alarm or other intelligent control operations can be performed.

For example, for the behavior of mild fatigue driving, an alarm may not be issued, and for the behavior of severe fatigue driving, because it belongs to a preset dangerous driving state, an alarm is issued.

In this implementation, by determining the danger level corresponding to the driver's behavior detection result, when the vehicle is in an abnormal driving state, the danger level of the driver's behavior detection result is upgraded to the first danger level, and in the first When the danger level reaches the preset warning level, it is determined that the driving state of the driver is in the preset dangerous driving state. Therefore, in the case that the vehicle has been in an abnormal driving state, the danger level of the detection result is increased, so that response measures can be taken in a timely and accurate manner, thereby improving the driving safety of the vehicle.

In a possible implementation manner, the determining the driving state of the driver according to the driving state information of the vehicle and the behavior detection result of the driver includes: in response to the driving state information of the vehicle and The result of the behavior detection of the driver determines that both the driving state of the vehicle and the behavior state of the driver are abnormal states, and it is determined that the driver is in a preset dangerous driving state.

In this implementation, considering that the driving state of the vehicle is abnormal, a traffic accident may be caused. If the driver's behavioral state is also abnormal at this time, it indicates that the driver is not driving safely, which has a great potential safety hazard. Therefore, when both the driving state of the vehicle and the behavior state of the driver are abnormal states, it can be determined that the driver is in a preset dangerous driving state, so as to respond to detecting that the driver of the vehicle is in a preset dangerous driving state Driving state, carry out intelligent driving control, improve the driving safety of the vehicle.

In a possible implementation manner, the determining the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area includes: responding to the driving state information according to the vehicle It is detected that the vehicle is in an abnormal driving state, and the behavior of the driver is detected according to the image information of the driving area; in response to determining that the driver's behavior detection result is the driver's distracted driving or fatigued driving, determine the driver's behavior. The driver is in a preset dangerous driving state.

The driver's distracted driving here can be the driver's behavior that the driver has his hands off the steering wheel, talking on the phone, drinking water while driving, playing with the mobile phone while driving, etc. yawning, etc.

In this implementation, considering that the driving state of the vehicle is abnormal, it may lead to a traffic accident. At this time, if the driver's behavior detection result is that the driver is distracted or fatigued, it can be determined that the driver is distracted or fatigued. Fatigue driving makes the vehicle in an unsafe state. Therefore, when the driving state of the vehicle is abnormal and the result of the driver's behavior detection is that the driver is distracted or fatigued, it can be determined that the driver is in a preset state. the dangerous driving state, so that the intelligent driving control can intervene in time to improve safety.

In a possible implementation manner, the performing intelligent driving control includes generating alarm information based on at least one of the following: information on the driving state of the vehicle; Behavior detection result; the driving state of the driver.

In this implementation, in order to achieve a better warning effect, the driver can be informed of at least one of the vehicle's driving state information, the driver's behavior detection result, and the driver's driving state, so that the driver is aware of the current The severity of the dangerous driving condition. For example: if the driving state information of the vehicle is "the vehicle has been pressed three times in a row", and the result of the behavior detection of the driver of the vehicle based on the image information of the driving area is "the driver is on the phone", the generated warning information can be: "You have pressed the line three times in a row, please don't make a phone call while driving." The specific form of sending the alarm information may be in the form of voice or video, which is not limited in the present disclosure.

In this implementation manner, the warning information is generated based on at least one of the driving state information of the vehicle, the behavior detection result of the driver, and the driving state of the driver, so that the driver is informed of the unsafe state of the vehicle and the current dangerous driving state to achieve a better warning effect.

In a possible implementation manner, the performing intelligent driving control includes: controlling the warning device to continuously output warning information until both the driving state of the vehicle and the driving state of the driver return to a normal state. By continuously outputting warning information to continuously warn the driver, the warning effect can be further improved, the continuous monitoring and warning of the driver's state can be realized, and the driving safety can be further improved. After the driving state of the vehicle and the driving state of the driver both return to normal, the alarm is stopped, which can reduce the interference caused by unnecessary alarms to the driver.

In a possible implementation manner, the performing intelligent driving control includes: in response to detecting that the driver is in a preset dangerous driving state for a duration exceeding a preset duration, controlling the assisted driving/automatic driving function to be activated, and /or control the vehicle to slow down and stop.

The duration of the driver in the preset dangerous driving state exceeds the preset time, indicating that the driver cannot control the vehicle normally for a long time, which will seriously threaten the driving safety of the vehicle and bring about great safety hazards. Therefore, in this case Under the control, the assisted driving/automatic driving function can be controlled to start, and the vehicle can also be controlled to decelerate and stop.

The assisted driving/autonomous driving function can control the accelerator, braking and direction according to the driving sensor data collected by the vehicle's sensors to adapt the vehicle to changing traffic conditions, such as adaptive cruise control, keeping the vehicle in the lane Driving functions such as driving and controlling the following distance between the vehicle and the vehicle in front.

In this implementation, when the driver is in the preset dangerous driving state for a duration exceeding the preset period, the assisted driving/automatic driving function is controlled to start, and/or the vehicle is controlled to decelerate and stop, so that the driver can When the vehicle cannot be normally controlled for a long time, the driving state of the vehicle is controlled, thereby improving the driving safety of the vehicle.

In a possible implementation manner, the method further includes: acquiring environmental information around the vehicle;

The determining of the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area includes: according to the environment information around the vehicle, the driving state information of the vehicle and the The image information of the driving area determines the driving state of the driver of the vehicle.

In actual driving scenarios, the driving state of the vehicle may be affected by the surrounding environment of the vehicle. For example, the speed of the vehicle may be fast or slow due to road congestion, and the vehicle swaying left and right may be due to the vehicle driving on a curved road. Therefore, in order to further improve the accuracy of the determined driving state of the vehicle driver, the driving state of the vehicle driver may be determined according to the surrounding environment information of the vehicle, the driving state information of the vehicle, and the image information of the driving area.

The environmental information around the vehicle here may include road condition information of the road on which the vehicle travels, and the road condition information may represent the condition of the road the vehicle travels, and may include indicating the degree of road congestion, indicating the shape of the road, indicating the type of the road, indicating the speed limit of the road at least one of the information, etc. According to the degree of congestion of the road, the road conditions can be divided into multiple grades, such as very crowded, crowded, unobstructed, etc. The shape of the road can be divided into curves and straight lines, and the types of roads can be divided into urban roads, rural roads, etc. Types of roads, highways, etc.

In a possible implementation manner, the acquiring the environmental information around the vehicle includes: acquiring road condition information of the road on which the vehicle travels; and the image information of the driving area to determine the driving state of the driver of the vehicle, including: in response to the driving state information of the vehicle and the road condition information not satisfying a preset matching relationship, and based on the driving area The result of detecting the behavior of the driver of the vehicle indicates that the behavior state of the driver is an abnormal state, and it is determined that the driver is detected to be in a preset dangerous driving state.

The road condition information of the road can be obtained through a third-party platform according to the current geographic location of the vehicle, for example, through a navigation service platform.

It can be determined whether the driving status information of the vehicle and the road condition information satisfy the preset matching relationship according to the environmental information around the vehicle and the driving status information of the vehicle. in an abnormal driving state.

The preset matching relationship refers to the matching relationship between the road condition information and the driving state information of the vehicle that runs safely under the road conditions represented by the road condition information. For example, it may include at least one of the following: the matching relationship between “unblocked” road conditions and the vehicle running at a constant speed; the matching relationship between the straight road conditions and the vehicle’s left-right swing amplitude that does not exceed the set amplitude threshold; the road speed limit information and the speed limit within the range The matching relationship of driving.

Correspondingly, the driving state information of the vehicle and the road condition information do not meet the preset matching relationship, for example, it may include: the left and right swing amplitude of the vehicle on the straight road section reaches the set amplitude threshold, the vehicle speed is fast or slow on the road section with clear roads, the vehicle Speeding, etc., all indicate that the vehicle is in an abnormal driving state.

Then, when the vehicle is in an abnormal driving state and the driver's behavior state is abnormal, it can be determined that the driver is in a preset dangerous driving state, the accuracy of the determined driving state of the driver is improved, and the further The driving safety of the vehicle is improved.

In an optional implementation manner, the determining the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area includes: obtaining the driving state of the vehicle The information and the image information of the driving area are input into the trained neural network, and the driving state of the driver of the vehicle is determined via the neural network. In this implementation manner, the driving state of the vehicle driver is determined through the neural network, which can improve the accuracy and speed of determining the driving state. And the neural network can be pre-trained and deployed, and can quickly detect images or video streams with a large amount of data, so it can be applied to intelligent driving control in real-time driving scenarios.

An application scenario of the embodiment of the present disclosure will be described below. In this application scenario, the image information of the driving area can be collected by the DMS camera, the processor can obtain the image information of the driving area from the DMS camera, and based on the image information of the driving area, the behavior of the driver of the vehicle can be detected, and the behavior detection can be obtained. As a result, in addition, the ADAS camera collects image information outside the cabin, and the processor can obtain the image information outside the cabin from the ADAS camera to determine the driving state information of the vehicle. If the determined driving state information of the vehicle is "the vehicle is pressing the line three times in a row", and the behavior detection result of the driver of the vehicle is "the driver is on the phone", a voice warning message will be generated: "You have pressed the line three times in a row, please Do not drive on the phone" and make a voice broadcast.

It can be understood that the above-mentioned method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle and logic. Those skilled in the art can understand that, in the above method of the specific embodiment, the specific execution order of each step should be determined by its function and possible internal logic.

In addition, the present disclosure also provides intelligent driving control devices, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any intelligent driving control method provided by the present disclosure. Corresponding records will not be repeated.

FIG. 2 shows a block diagram of an intelligent driving control device according to an embodiment of the present disclosure. As shown in FIG. 2 , the device includes:

The first acquisition module 21 is used to acquire the driving state information of the vehicle;

a second acquisition module 22, configured to acquire image information of the driving area of the vehicle;

a driving state determination module 23, configured to determine the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area;

The control module 24 is configured to perform intelligent driving control in response to detecting that the driver of the vehicle is in a preset dangerous driving state.

In a possible implementation manner, the driving state determination module 23 includes a first driving state determination submodule and a second driving state determination submodule, wherein:

The second driving state determination submodule is used for:

In a possible implementation manner, the driving state determination module 23 includes a third driving state determination submodule and a fourth driving state determination submodule, wherein:

In a possible implementation manner, the control module 24 is configured to generate alarm information based on at least one of the following:

the driving state information of the vehicle;

the driving state of the driver.

In a possible implementation manner, the control module 24 is configured to control the warning device to continuously output warning information until both the driving state of the vehicle and the driving state of the driver return to the normal state.

In a possible implementation manner, the control module 24 is configured to control the assisted driving/automatic driving function to be activated in response to detecting that the driver is in a preset dangerous driving state for a duration exceeding a preset period of time, and /or control the vehicle to slow down and stop.

In a possible implementation, the apparatus further includes:

The driving state determination module 23 is configured to determine the driving state of the driver of the vehicle according to the environmental information around the vehicle, the driving state information of the vehicle and the image information of the driving area.

The driving state determination module 23 is configured to respond to the fact that the driving state information of the vehicle and the road condition information do not satisfy a preset matching relationship, and determine the behavior of the driver of the vehicle based on the image information of the driving area. The detection result indicates that the behavior state of the driver is an abnormal state, and it is determined that the driver is in a preset dangerous driving state.

In a possible implementation manner, the first acquisition module 21 includes a sensor data acquisition sub-module and a driving state information determination sub-module, wherein:

In a possible implementation manner, the driving state determination module 23 is configured to input the acquired driving state information of the vehicle and the image information of the driving area into a trained neural network, and through the neural network, The driving state of the driver of the vehicle is determined.

In some embodiments, the functions or modules included in the apparatuses provided in the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments. For specific implementation, reference may be made to the descriptions of the above method embodiments. For brevity, here No longer.

The embodiments of the present disclosure also provide a vehicle, including:

The vehicle of this embodiment determines the driving state of the driver of the vehicle by combining the driving state of the vehicle and the image information of the driving area, thereby improving the accuracy of determining the driving state of the driver.

In some possible implementations, the vehicle further includes a second sensor for collecting driving sensor data; the controller is used for acquiring driving state information of the vehicle according to the driving sensor data. Thus, accurate detection of the driving state is achieved through sensors for sensing different data of the vehicle.

Embodiments of the present disclosure further provide a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented. The computer-readable storage medium may be a volatile computer-readable storage medium or a non-volatile computer-readable storage medium.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

Embodiments of the present disclosure also provide a computer program product, including computer-readable codes. When the computer-readable codes are run on a device, a processor in the device executes the method for implementing the intelligent driving control provided by any of the above embodiments. instruction.

Embodiments of the present disclosure further provide another computer program product for storing computer-readable instructions, which, when executed, cause the computer to execute the operations of the intelligent driving control method provided by any of the foregoing embodiments.

The electronic device may be provided as a terminal, server or other form of device.

FIG. 3 shows a block diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, electronic device 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc. terminal.

3, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814 , and the communication component 816 .

The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to perform all or some of the steps of the methods described above. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

Memory 804 is configured to store various types of data to support operation at electronic device 800 . Examples of such data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. Memory 804 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power supply assembly 806 provides power to various components of electronic device 800 . Power supply components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800 .

Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front-facing camera and/or a rear-facing camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.

Audio component 810 is configured to output and/or input audio signals. For example, audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when electronic device 800 is in operating modes, such as calling mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of electronic device 800 . For example, the sensor assembly 814 can detect the on/off state of the electronic device 800, the relative positioning of the components, such as the display and the keypad of the electronic device 800, the sensor assembly 814 can also detect the electronic device 800 or one of the electronic device 800 Changes in the position of components, presence or absence of user contact with the electronic device 800 , orientation or acceleration/deceleration of the electronic device 800 and changes in the temperature of the electronic device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as wireless network (WiFi), second generation mobile communication technology (2G) or third generation mobile communication technology (3G), or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmed gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation is used to perform the above method.

In an exemplary embodiment, a non-volatile computer-readable storage medium, such as a memory 804 comprising computer program instructions executable by the processor 820 of the electronic device 800 to perform the above method is also provided.

FIG. 4 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. 4, electronic device 1900 includes processing component 1922, which further includes one or more processors, and a memory resource represented by memory 1932 for storing instructions executable by processing component 1922, such as applications. An application program stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Additionally, the processing component 1922 is configured to execute instructions to perform the above-described methods.

The electronic device 1900 may also include a power supply assembly 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input output (I/O) interface 1958 . The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as a Microsoft server operating system (Windows Server ^™ ), a graphical user interface based operating system (Mac OS X ^™ ) introduced by Apple, a multi-user multi-process computer operating system (Unix ^™ ), Free and Open Source Unix-like Operating System (Linux ^™ ), Open Source Unix-like Operating System (FreeBSD ^™ ) or the like.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as memory 1932 comprising computer program instructions executable by processing component 1922 of electronic device 1900 to perform the above-described method.

The present disclosure may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present disclosure.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above. Computer-readable storage media, as used herein, are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.

The computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for carrying out operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages. Source or object code, written in any combination, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect). In some embodiments, custom electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be personalized by utilizing state information of computer readable program instructions. Computer readable program instructions are executed to implement various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.

The computer program product can be specifically implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Various embodiments of the present disclosure have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims

An intelligent driving control method, comprising:

Obtain the driving status information of the vehicle;

acquiring image information of the driving area of the vehicle;

determining the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area;

Intelligent driving control is performed in response to detecting that the driver of the vehicle is in a preset dangerous driving state.
The method according to claim 1, wherein the determining the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area comprises:

Detecting the behavior of the driver of the vehicle according to the image information of the driving area;

The driving state of the driver is determined according to the driving state information of the vehicle and the detection result of the driver's behavior.
The method according to claim 2, wherein the performing behavior detection on the driver of the vehicle comprises:

determining the danger level corresponding to the behavior detection result of the driver;

The determining of the driving state of the driver according to the driving state information of the vehicle and the behavior detection result of the driver includes:

In response to determining that the vehicle is in an abnormal driving state according to the driving state information of the vehicle, upgrading the danger level of the driver's behavior detection result to a first danger level;

In response to determining that the first dangerous level reaches a preset warning level, it is determined that the driving state of the driver is in a preset dangerous driving state.
The method according to claim 2, wherein the determining the driving state of the driver according to the driving state information of the vehicle and the behavior detection result of the driver comprises:

In response to determining that the driving state of the vehicle and the behavior state of the driver are both abnormal states according to the driving state information of the vehicle and the behavior detection result of the driver, it is determined that the driver is in a preset state. Dangerous driving condition.
The method according to any one of claims 2-4, wherein the determining the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area comprises:

In response to detecting that the vehicle is in an abnormal driving state according to the driving state information of the vehicle, performing behavior detection on the driver according to the image information of the driving area;

In response to determining that the driver's behavior detection result is the driver's distracted driving or fatigued driving, it is determined that the driver is in a preset dangerous driving state.
The method according to any one of claims 1-5, wherein the abnormal driving state of the vehicle comprises at least one of the following:

The number of times of line pressing of the vehicle within the first set time period reaches the threshold of the number of times of line pressing; the left and right swing of the vehicle reaches the set amplitude threshold; the vehicle does not pass according to the instructions of traffic signs or traffic lights; the speed of the vehicle exceeds the set speed threshold.
The method according to any one of claims 1-6, wherein the performing intelligent driving control comprises generating alarm information based on at least one of the following:

the driving state information of the vehicle;

The result of detecting the behavior of the driver of the vehicle based on the image information of the driving area;

the driving state of the driver.
The method according to any one of claims 1-7, wherein the performing intelligent driving control comprises:

The warning device is controlled to continuously output warning information until both the driving state of the vehicle and the driving state of the driver return to normal states.
The method according to any one of claims 1-8, wherein the performing intelligent driving control comprises:

In response to detecting that the driver is in a preset dangerous driving state for a duration exceeding a preset time period, the assisted driving/automatic driving function is controlled to be activated, and/or the vehicle is controlled to decelerate to a stop.
The method according to any one of claims 1-9, wherein the method further comprises:

obtain environmental information around the vehicle;

The determining of the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area includes:

The driving state of the driver of the vehicle is determined according to the environment information around the vehicle, the driving state information of the vehicle, and the image information of the driving area.
The method according to claim 10, wherein the environmental information comprises road condition information;

The acquiring the environmental information around the vehicle includes: acquiring road condition information of the road on which the vehicle travels;

The determining of the driving state of the driver of the vehicle according to the environmental information around the vehicle, the driving state information of the vehicle and the image information of the driving area includes:

In response to that the driving state information of the vehicle and the road condition information do not satisfy a preset matching relationship, and the result of detecting the behavior of the driver of the vehicle based on the image information of the driving area indicates the behavior state of the driver In the abnormal state, it is determined that the driver is in a preset dangerous driving state.
The method according to any one of claims 1-11, wherein the acquiring the driving state information of the vehicle comprises:

Acquiring driving sensing data, the sensing data includes at least one of the following: image information collected by the vehicle's advanced driving assistance system, image information collected by the vehicle's driving recorder, and speed information sensed by the vehicle's speed sensor;

According to the sensing data, the driving state information of the vehicle is determined.
The method according to any one of claims 1-12, wherein the determining the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area comprises:

The acquired driving state information of the vehicle and the image information of the driving area are input into a trained neural network, and the driving state of the driver of the vehicle is determined through the neural network.
An intelligent driving control device, characterized in that it includes:

a first acquisition module, used for acquiring the driving state information of the vehicle;

a second acquisition module, configured to acquire image information of the driving area of the vehicle;

a driving state determination module, configured to determine the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area;

The control module is configured to perform intelligent driving control in response to detecting that the driver of the vehicle is in a preset dangerous driving state.
A vehicle, characterized in that it includes:

a first sensor, used for collecting image information of the driving area of the vehicle;

The controller is configured to acquire the driving state information of the vehicle, determine the driving state of the driver of the vehicle according to the driving state information of the vehicle and the image information of the driving area, and respond to detecting the driving of the vehicle The driver is in a preset dangerous driving state and performs intelligent driving control.
The vehicle of claim 15, further comprising:

The second sensor is used to collect driving sensor data;

The controller is configured to acquire the driving state information of the vehicle according to the driving sensing data.
An electronic device, comprising:

processor;

memory for storing processor-executable instructions;

wherein the processor is configured to invoke the memory-stored instructions to perform the method of any one of claims 1-13.
A computer-readable storage medium on which computer program instructions are stored, characterized in that, when the computer program instructions are executed by a processor, the method described in any one of claims 1 to 13 is implemented.
A computer program comprising computer readable code, when the computer readable code is run in an electronic device, a processor in the electronic device executes the code for implementing the method described in any of claims 1-13 Methods.