WO2022188362A1

WO2022188362A1 - Distraction reminding method and apparatus, electronic device, and storage medium

Info

Publication number: WO2022188362A1
Application number: PCT/CN2021/113576
Authority: WO
Inventors: 曾彬; 范亦卿; 吴阳平; 许亮
Original assignee: 上海商汤临港智能科技有限公司
Priority date: 2021-03-12
Filing date: 2021-08-19
Publication date: 2022-09-15
Also published as: CN113060144A

Abstract

A distraction reminding method, comprising: obtaining the traveling speed of a vehicle; when the starting of a distraction detection function is determined according to the traveling speed of the vehicle, obtaining a video frame acquired for a driver of the vehicle; determining a driving state of the driver on the basis of the video frame of the driver and the traveling speed; and when it is determined that the driving state of the driver is a distracted driving state, generating distraction reminding information. Further disclosed are a distraction reminding apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

Description

Distraction reminder method and device, electronic device and storage medium

This disclosure claims the priority of the Chinese patent application with the application number CN202110268977.1 and the application title "Distraction reminder method and device, electronic device and storage medium" filed with the Chinese Patent Office on March 12, 2021, the entire contents of which are Incorporated in this disclosure by reference.

technical field

The present disclosure relates to the technical field of intelligent driving, and in particular, to a distraction reminder method and device, an electronic device and a storage medium.

Background technique

With the rapid development of intelligent driving technology, more and more vehicles are equipped with intelligent driving systems. Intelligent driving technology mainly includes two parts: automatic driving and assisted driving. Among them, assisted driving can assist users to drive safely. For example, it can remind users when they are tired or distracted, thereby reducing the occurrence of traffic accidents.

The distraction detection system can identify the user's distracted behavior while driving. Distracted behaviors can be understood as behaviors that the user does not pay attention to while driving. For example, behaviors such as looking for things while driving, looking at the scenery outside the window, answering calls, drinking water, etc., can be considered distracting behaviors.

SUMMARY OF THE INVENTION

The present disclosure provides a technical solution for distraction reminder.

According to an aspect of the present disclosure, a distraction reminder method is provided, comprising:

Acquire the driving speed of the vehicle; in the case of determining to activate the distraction detection function according to the driving speed of the vehicle, acquire the video frames collected for the driver of the vehicle; based on the video frames of the driver and the driving speed , determine the driving state of the driver; in the case that the driving state of the driver is determined to be a distracted driving state, generate distraction reminder information.

In one or more possible implementation manners, the method further includes: in the case that the running speed of the vehicle is greater than or equal to a preset starting vehicle speed, activating a distraction detection function; when the running speed of the vehicle is less than or equal to In the case of the starting vehicle speed, the distraction detection function is turned off.

In one or more possible implementation manners, the determining the driving state of the driver based on the video frame of the driver and the driving speed includes: detecting the driving based on the video frame of the driver the head deflection angle and/or the sight deflection angle of the driver; and the driving state of the driver is determined based on the head deflection angle and/or the sight deflection angle and the driving speed.

In one or more possible implementation manners, the determining the driving state of the driver based on the head deflection angle and/or the line of sight deflection angle and the driving speed includes: acquiring the corresponding driving speed The head deflection angle safety threshold and the sight deflection angle safety threshold; in response to the head deflection angle reaching the head deflection angle safety threshold corresponding to the driving speed and/or the sight deflection angle reaching the sight deflection angle safety A threshold value to determine that the driver is in a distracted driving state.

In one or more possible implementations, the head yaw angle safety threshold and/or the line of sight yaw angle safety threshold is based on the driver's head when driving the vehicle at the travel speed in a safe driving state Deflection angle and line-of-sight deflection angle calibration.

In one or more possible implementation manners, the method further includes: performing face recognition on the video frame to determine the driver's identity information; acquiring the safety of the head deflection angle corresponding to the driving speed The threshold value and the safety threshold of the line of sight deflection angle include: obtaining the safety threshold of the head deflection angle and the safety threshold of the line of sight deflection angle and corresponding to the driving speed, which are associated with the driver's identity information.

In one or more possible implementation manners, the acquiring the video frames collected for the driver of the vehicle includes: determining a frame sampling rate based on the driving speed of the vehicle; Frame sampling is performed on the video of the driver of the vehicle to obtain video frames collected for the driver of the vehicle.

In one or more possible implementation manners, the detecting the head deflection angle and/or the sight deflection angle of the driver based on the video frame of the driver includes: keying the video frame of the driver Point detection, determining at least one key point included in the face area of the video frame; determining the driver's head deflection angle and/or line of sight deflection angle based on the at least one key point included in the face area.

In one or more possible implementation manners, the generating distraction reminder information when it is determined that the driving state of the driver is a distracted driving state includes: determining that the driving state of the driver is a distracted driving state. In the case of a distracted driving state, the duration of the driver in the distracted driving state is determined; when the duration reaches a preset time threshold, distraction reminder information is generated.

In one or more possible implementations, the method further comprises: determining the time threshold according to the traveling speed of the vehicle.

According to an aspect of the present disclosure, a distraction reminder device is provided, comprising:

a first obtaining module, used for obtaining the driving speed of the vehicle;

a second acquisition module, configured to acquire video frames collected for the driver of the vehicle when the distraction detection function is determined to be activated according to the driving speed of the vehicle;

a determining module for determining the driving state of the driver based on the video frame of the driver and the driving speed;

A generating module, configured to generate distraction reminder information when it is determined that the driving state of the driver is a distracted driving state.

In one or more possible implementation manners, the apparatus further includes: a start module, configured to start a distraction detection function when the driving speed of the vehicle is greater than or equal to a preset start speed; When the running speed of the vehicle is lower than the starting vehicle speed, the distraction detection function is turned off.

In one or more possible implementation manners, the determining module is configured to detect the head deflection angle and/or the line of sight deflection angle of the driver based on the video frame of the driver; based on the head deflection angle and/or the deflection angle of the line of sight, and the driving speed, to determine the driving state of the driver.

In one or more possible implementations, the determining module is configured to acquire a head deflection angle safety threshold and a line of sight deflection angle safety threshold corresponding to the traveling speed; in response to the head deflection angle reaching the travel speed When the head deflection angle safety threshold corresponding to the speed and/or the line of sight deflection angle reaches the line of sight deflection angle safety threshold, it is determined that the driver is in a distracted driving state.

In one or more possible implementation manners, the apparatus further includes: an identification module, configured to perform face recognition on the video frame, and determine the driver's identity information; the determination module, configured to obtain the The head deflection angle safety threshold and the sight deflection angle safety threshold corresponding to the driving speed are associated with the driver's identity information.

In one or more possible implementation manners, the second obtaining module is configured to determine a frame sampling rate based on the driving speed of the vehicle; and frame the video of the driver of the vehicle based on the frame sampling rate Sampling to obtain video frames collected for the driver of the vehicle.

In one or more possible implementations, the determining module is configured to perform key point detection on the video frame of the driver, and determine at least one key point included in the face area of the video frame; based on the At least one key point included in the face area determines the head deflection angle and/or the sight deflection angle of the driver.

In one or more possible implementation manners, the generating module is configured to determine, in the case of determining that the driving state of the driver is a distracted driving state, Duration; when the duration reaches a preset time threshold, generate distraction reminder information.

In one or more possible implementations, the generating module is further configured to determine the time threshold according to the traveling speed of the vehicle.

According to an aspect of the present disclosure, there is provided an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions implementing the above method when executed by a processor.

According to an aspect of the present disclosure, a computer program product is provided, comprising computer-readable codes or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are processed in an electronic device When running in the device, the processor in the electronic device executes the method for implementing the above method.

In the embodiment of the present disclosure, the current driving speed of the vehicle may be acquired, and in the case where the distraction detection function is determined to be activated according to the driving speed of the vehicle, video frames collected for the driver of the vehicle may be acquired, based on the driver's video frame and the vehicle's driving speed. The driving speed of the driver can further determine the driving state of the driver, and when it is determined that the driving state of the driver is a distracted driving state, a distraction reminder message is generated, so that the driving speed of the vehicle can be used as the driving state of the driver. Consider factors to improve the stability of distraction alerts and improve the driver's user experience.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the technical solutions of the present disclosure.

FIG. 1 shows a flowchart of a distraction reminder method according to an embodiment of the present disclosure.

2 illustrates a block diagram of an example of a distraction detection system according to an embodiment of the present disclosure.

FIG. 3 shows a flowchart of an example of a distraction reminder method according to an embodiment of the present disclosure.

FIG. 4 shows a schematic diagram of head deflection angle according to an embodiment of the present disclosure.

FIG. 5 shows a block diagram of a distraction reminder device according to an embodiment of the present disclosure.

FIG. 6 shows a block diagram of an example of an electronic device according to an embodiment of the present disclosure.

FIG. 7 shows a block diagram of an example of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures denote elements that have the same or similar functions. While various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, and may mean including from A, B, and C. Any one or more elements selected from the set of B and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are set forth in the following detailed description. It will be understood by those skilled in the art that the present disclosure may be practiced without certain specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present disclosure.

The distraction reminder solution provided by the embodiments of the present disclosure can be applied to scenarios such as automatic driving or assisted driving systems, urban traffic, and traffic travel. For example, in urban traffic with large traffic flow, the distraction reminder solution provided by the embodiment of the present disclosure can be used to determine the driving state of the driver based on the driver's video frame and the driving speed of the vehicle, so that the driving state can be determined when driving In the case of distracted driving, a reminder is issued to assist driving and improve the safety of the vehicle. By taking the driving speed of the vehicle as a consideration factor for initiating the distraction reminder, the stability of the distraction reminder can be improved, and the negative experience and interference to the driver caused by the reminder due to false triggering can be reduced.

The distraction reminder method provided by the embodiments of the present disclosure may be executed by a terminal device, a server, or other types of electronic devices, where the terminal device may be a user equipment (User Equipment, UE), a mobile device, a user terminal, a cellular phone, or a cordless phone. , Personal Digital Assistant (PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. In some possible implementations, the distraction reminder method may be implemented by the processor invoking computer-readable instructions stored in the memory. Alternatively, the method may be performed by a server.

FIG. 1 shows a flowchart of a distraction reminder method according to an embodiment of the present disclosure. As shown in FIG. 1 , the distraction reminder method includes:

In step S11, the running speed of the vehicle is obtained.

In the embodiment of the present disclosure, the vehicle may be configured with a distraction detection system, and the distraction detection system may acquire the driving speed of the vehicle in real time, or periodically acquire the driving speed of the vehicle at a certain time interval, for example, with Time intervals such as 1s and 2s are cycles, and the running speed of the vehicle is obtained periodically. Here, the current running speed of the vehicle can be obtained through a vehicle-mounted Global Positioning System (Global Positioning System, GPS) or an in-vehicle communication bus (such as a Controller Area Network (CAN) bus). In some implementation manners, the travel time of the vehicle for a certain distance may also be obtained, and then the travel speed of the vehicle is calculated according to the ratio of the travel distance and the travel time.

Step S12, in the case that the distraction detection function is activated according to the driving speed of the vehicle, acquire a video frame collected for the driver of the vehicle.

In the embodiment of the present disclosure, whether to activate the distraction detection function of the vehicle may be determined according to the driving speed of the vehicle. For example, when the driving speed of the vehicle continues to increase, such as the time when the vehicle accelerates reaches a certain time threshold, the distraction detection function of the vehicle can be turned on; when the driving speed of the vehicle continues to decrease, such as the time when the vehicle slows down reaches a certain time threshold , you can turn off distraction detection in your vehicle. In the case where it is determined to activate the distraction detection function according to the driving speed of the vehicle, a video frame collected for the face of the driver of the vehicle may be acquired.

In some implementations, the distraction detection function can be activated when the vehicle's driving speed is greater than or equal to a preset start-up speed. Trigger distraction detection. When the driving speed of the vehicle is lower than the starting speed, the distraction detection function can be turned off. For example, when the vehicle stops or the driving speed is less than 10km/h, the distraction detection function can be turned off, thereby reducing the phenomenon of falsely triggered distraction warnings. For example, when the user parks and chats, a distraction warning may be triggered based on pure visual detection, which will bring a bad user experience to the driver. Heart detection function.

Here, a camera device such as an in-vehicle camera or a camera may be installed on the vehicle, and video frames in the video collected by the camera device such as the in-vehicle camera or camera for the driver's face may be acquired. In some implementations, in the case of acquiring the video frames of the driver, the video frames in the video may be extracted at a certain time interval or a certain frame number interval. For example, when it is determined that the driving state of the driver is a safe driving state according to the previous video frame of the current video frame, the video frames in the video can be extracted at the first time interval or the first frame number interval to obtain the current video frame. When it is determined that the driver's driving state is a distracted driving state according to the previous video frame of the current video frame, video frames in the video may be extracted at a second time interval or a second frame interval to obtain the current video frame. The first time interval may be smaller than the second time interval, and the first frame interval may be smaller than the second frame interval, so that when the driver is distracted driving, the frequency of video frame extraction can be increased, thereby improving distraction reminders Frequency of.

In some implementations, the frame sampling rate may also be determined based on the driving speed of the vehicle, and further frame sampling is performed on the video of the driver of the vehicle based on the frame sampling rate to obtain video frames collected for the driver of the vehicle. For example, the correspondence between the driving speed of the vehicle and the frame sampling rate may be preset, and when the vehicle is driving, frame sampling of the video of the driver of the vehicle may be performed according to the frame sampling rate corresponding to the driving speed of the vehicle. For example, when the driving speed is less than a certain speed, the video frames in the video can be extracted at the first time interval or the first frame number interval (the first frame sampling rate). When the driving speed is greater than or equal to a certain speed, the second time interval can be used. The interval or the second frame interval (second frame sampling rate) extracts video frames in the video, and the second frame sampling rate is greater than the first frame sampling rate, so that the frequency of distraction detection can be increased when the vehicle speed is high. Further improve driving safety.

Step S13, determining the driving state of the driver based on the video frame of the driver and the driving speed.

In this embodiment of the present disclosure, face detection may be performed on the acquired video frame to obtain a detection result of face detection. Then, the driving state of the current driver can be determined according to the detection result and the driving speed of the vehicle. For example, it can be determined whether the deviation or rotation angle of the driver's head reaches the angle threshold corresponding to the driving speed according to the detection result and the driving speed. If it is judged that the driver's head is deviated or the angle of rotation reaches the angle threshold corresponding to the driving speed, it can be determined that the current driving state of the driver is a distracted driving state. The angle threshold corresponding to the driving speed can determine that the current driving state of the driver is a safe driving state.

Step S14, when it is determined that the driving state of the driver is a distracted driving state, generate distraction reminder information.

In an example of the present disclosure, when it is determined that the driver's driving state is a distracted driving state, distraction reminder information may be generated and issued, for example, distraction reminder information such as a prompt tone, warning, etc., may be issued, so as to remind the driver. In some implementations, the generated distraction reminder information can also be sent to other in-vehicle devices, for example, the distraction reminder information is sent to the warning device, so that the warning device can send the warning information, and for example, the distraction reminder information is sent to the vehicle speed control device, so that the vehicle speed control device can automatically control the deceleration of the vehicle.

2 illustrates an exemplary block diagram of a distraction detection system according to an embodiment of the present disclosure. The distraction detection system can obtain the video frame of the driver from the camera device, and obtain the current driving speed of the vehicle through GPS or the in-vehicle communication bus. When it is determined that the driver's driving state is a distracted driving state, distraction reminder information can be generated, and the distraction reminder information can be sent to other devices such as an alarm device or a vehicle speed control device.

The embodiments of the present disclosure can reduce the false triggering of the distraction alarm by taking the driving speed of the vehicle as a consideration factor of the distraction detection function, improve the accuracy of the distraction reminder, and improve the user experience.

In the above step S13, the driving state of the driver may be determined based on the video frame of the driver and the driving speed of the vehicle, and in some implementations, the head deflection angle and/or the head deflection angle of the driver may be detected according to the video frame of the driver and/or The gaze deflection angle is then determined based on the detected head deflection angle and/or gaze deflection angle, and the travel speed of the vehicle to determine the driving state of the driver. For example, head deflection angle detection and/or line-of-sight deflection angle detection may be performed on the video frame of the driver. For example, a detection algorithm, a detection neural network, etc. /or perform line-of-sight deflection angle detection to obtain the driver's head deflection angle and/or line-of-sight deflection angle. Then, the detected head deflection angle and/or the sight deflection angle can be compared with the safety threshold corresponding to the driving speed, and it can be judged whether the detected head deflection angle and/or the sight deflection angle reach the safety threshold corresponding to the driving speed, When it is determined that the detected head deflection angle and/or the sight deflection angle reach the safety threshold, it can be considered that the driver is in a state of distracted driving, and when the head deflection angle and/or the sight deflection angle do not reach the safe threshold , it can be considered that the driver is in a safe driving state. By detecting the head deflection angle and/or the sight deflection angle obtained by the video frames collected by the driver, the driving state of the driver at the current driving speed can be quickly determined.

In some implementation manners, the driving speed of the vehicle may correspond to a corresponding safety threshold, and the safety threshold corresponding to the driving speed of the vehicle may be pre-stored. The safety threshold can indicate the upper and lower limits of the driver's head deflection angle and/or line of sight deflection angle under safe driving conditions at the corresponding driving speed, if the driver's head deflection angle and/or line of sight deflection angle exceeds the safety threshold Threshold value, it can be considered that the current driving state is a distracted driving state. The safety threshold may include a head yaw angle safety threshold and a line of sight yaw angle safety threshold. When determining the driving state of the driver, the head deflection angle safety threshold and the sight deflection angle safety threshold corresponding to the current driving speed can be obtained, and the detected head deflection angle and the head deflection angle safety threshold corresponding to the driving speed can be obtained. Compare, and compare the detected sight deflection angle with the sight deflection angle safety threshold corresponding to the driving speed. In response to the detected head deflection angle reaching the head deflection angle safety threshold corresponding to the travel speed, and/or the sight deflection angle reaching the sight deflection angle safe threshold, it may be determined that the driver is in a distracted driving state. If the detected head deflection angle does not reach the head deflection angle safety threshold, and the detected sight deflection angle does not reach the sight deflection angle safety threshold, it may be determined that the driver is in a safe driving state. By comparing the head deflection angle and/or the sight deflection angle with the safety threshold, the driving state of the driver can be quickly determined, and then according to the driving state of the driver, the driving of the user can be assisted and the safety of the vehicle can be improved.

Here, the correspondence between the traveling speed and the head deflection angle safety threshold and the correspondence between the traveling speed and the sight deflection angle safety threshold may be preset. The safety threshold of head deflection angle and the safety threshold of sight deflection angle corresponding to different driving speeds can be different. The higher the driving speed of the vehicle, the smaller the head deflection angle and/or the sight deflection angle of the driver in the safe driving state, so that the head deflection angle of the driver can be adjusted according to the safety threshold of the head deflection angle corresponding to the driving speed of the vehicle. The deflection angle is appropriately constrained, and the line of sight deflection angle is appropriately constrained according to the safety threshold of the line of sight deflection angle corresponding to the driving speed of the vehicle to assist the driver to drive safely.

In some implementations, the head deflection angle safety threshold and/or the line of sight deflection angle safety threshold corresponding to the driving speed may be based on the head deflection angle and line of sight deflection of the driver when the driver drives the vehicle at the driving speed in a safe driving state Angled. For example, a historical video of a preset duration (eg, 3 minutes) collected for a driver may be acquired, and in the historical video, the driver drives the vehicle at a certain speed in a safe driving state. Detect the head deflection angle and sight deflection angle of the driver in the video frame in the historical video, and count the head deflection angle and sight deflection angle of the driver in a safe driving state at this driving speed. The maximum or average value of the head deflection angle is demarcated as the safety threshold of the head deflection angle corresponding to the driving speed, and the maximum or average value of the statistical sight deflection angle is demarcated as the safety threshold of the sight deflection angle corresponding to the driving speed. In this way, through the head deflection angle and the sight deflection angle when the driver drives the vehicle at a certain speed in a safe driving state, the head deflection angle safety threshold and the sight deflection angle safety threshold corresponding to the driving speed can be accurately calibrated, so that the safety threshold of the head deflection angle and the sight deflection angle can be accurately calibrated. Distraction checks are more accurate.

In some implementations, the head deflection angle safety threshold and the sight deflection angle safety threshold corresponding to the driving speed may also be associated with the driver's identity information, that is, the head deflection angle safety threshold and the sight deflection angle safety corresponding to different drivers Thresholds can be different. In the case of obtaining the head deflection angle safety threshold and the sight deflection angle safety threshold corresponding to the driving speed, face recognition can also be performed on the video frames collected for the driver to determine the driver's identity information. For example, some people can be used. Face recognition algorithm, neural network, etc. perform face recognition on video frames, determine the driver's identity document (ID), driver's license number, name and other identity information, and then obtain the driver's identity information associated with the driver's identity information. The head deflection angle safety threshold and the sight deflection angle safety threshold corresponding to the current speed of the vehicle. In this way, the corresponding head deflection angle safety threshold and sight deflection angle safety threshold can be matched for the driver through face recognition, so that the head deflection angle safety threshold and the sight deflection angle safety threshold can be automatically adjusted to suit the current driver. Safety threshold, realize the personalized setting of safety threshold to meet the different needs of different drivers due to individual differences, such as the height, gender, driving habits and other individual differences of different drivers may lead to slightly different safety thresholds associated with drivers , for more flexible distraction reminders.

The distraction reminder solution provided by the present disclosure will be described below through an example. 3 shows a flowchart of an example of a distraction reminder method according to an embodiment of the present disclosure, including:

Step S201 , in the case that the distraction detection function is activated according to the driving speed of the vehicle, acquire a video frame collected for the driver of the vehicle.

Step S202, performing head deflection angle detection on the video frame to obtain the driver's head deflection angle.

In step S203, the line of sight deflection angle is detected on the video frame to obtain the line of sight deflection angle of the driver.

Step S204, judging whether the head deflection angle of the driver reaches the safety threshold of the head deflection angle corresponding to the driving speed, and judging whether the sight deflection angle reaches the sight deflection angle safety threshold corresponding to the driving speed;

When the head deflection angle of the driver reaches the head deflection angle safety threshold or the sight deflection angle reaches the sight deflection angle safety threshold, step S205 is performed; otherwise, step S201 is performed.

Step S205, generating distraction reminder information.

Through the distraction reminder solution provided in this example, when the distraction detection function is activated, the driver's driving state can be determined through the driver's head deflection angle and/or line of sight deflection angle and the driving speed of the vehicle. When it is determined that the driver is in a distracted driving state, a distraction reminder message can be generated to remind the driver to return to a safe driving state.

In some implementations, when detecting the driver's head deflection angle and/or the line of sight deflection angle based on the driver's video frame, key point detection may be performed on the driver's video frame to determine the face area of the video frame. At least one key point, and based on the at least one key point included in the face area, the driver's head deflection angle and/or line of sight deflection angle may be determined. Here, some detection algorithms, neural networks, etc. can be used to detect key points in the video frame to determine the face area where the face in the video frame is located. After the face area is determined, one or more key points included in the face area can be identified. point. Here, the key points may be key points of the human face, for example, the eyebrows, eyes, nose, mouth, facial contours, etc. included in the face region may all be key points.

For the head deflection angle, the driver's head deflection angle can be determined according to the position information of at least one key point included in the face area. and the position information of at least one key point, to determine the three-dimensional coordinates of at least one key point in three-dimensional space, that is, it can be understood as mapping two-dimensional key points in the video frame into three-dimensional space to obtain the three-dimensional coordinates of the key points. The driver's head deflection angle is further calculated according to the three-dimensional coordinates of the at least one key point. For example, by comparing the three-dimensional coordinates of the at least one key point with the preset reference coordinates, the angle at which the at least one key point deviates from the reference coordinates can be determined. The head yaw angle can be represented by the following three parameters: the yaw angle (yaw), the pitch angle (pitch) and the roll angle (roll). FIG. 4 shows a schematic diagram of head deflection angle according to an embodiment of the present disclosure. Wherein, yaw can be the angle of rotation of the face relative to the horizontal direction of the camera, that is, the y-axis is rotated as the rotation axis, and a negative number represents rotation to the left; pitch can be the angle of the face relative to the vertical direction of the camera, that is, Rotate with the x-axis as the rotation axis, and a negative number represents rotation upward; roll can be the angle of rotation of the face relative to the vertical direction of the camera device, that is, rotation with the z-axis as the rotation axis, and a negative number represents rotation downward to the right.

For the sight deflection angle, the human eye area in the face area can be intercepted. For the human eye area, the eyeball key points included in the human eye area and the position information of the orbital edge keypoints can be determined. The position information of the edge key point is compared with the position information of the human eye key point of the standard sight deflection angle, and the driver's sight deflection angle can be determined according to the comparison result.

In this implementation manner, by performing key point detection on the video frame, the driver's head deflection angle and/or line of sight deflection angle can be determined, so that the driver's driving state can be quickly determined by performing face detection on the video frame, which is safe for the Driving provides the foundation.

In some examples, a neural network can also be used to detect the driver's head deflection angle and/or line of sight deflection angle. For example, the driver's video frame is input into the trained neural network, and the video frame is processed by the neural network. Face detection, determine the face area where the driver's face is located, and then use the neural network to perform head deflection angle detection and/or line of sight deflection angle detection based on the image features of the face area to obtain the driver's head deflection angle and/or line of sight deflection angle. When the driving information includes the head deflection angle and the sight deflection angle, different branches of a neural network can be used to detect the head deflection angle and the sight deflection angle. In some implementations, two neural networks can be used to perform Head deflection angle detection and sight deflection angle detection.

In the above step S14, when it is determined that the driving state of the driver is a distracted driving state, distraction reminder information may be generated, and in some implementations, it may also be determined that the driving state of the driver is a distracted driving state In this case, the duration of the driver's distracted driving state is determined, and when the duration reaches a preset time threshold, distraction reminder information is generated. Here, the time threshold can be set according to the actual application scenario, and in the case that the driving state of the driver is determined to be a distracted driving state, the duration of the driver in the distracted driving state can be compared with a preset time threshold, If the duration of the driver in the distracted driving state reaches the set time threshold, it can be considered that the driver may cause a traffic accident or danger due to the distracted driving state, so that distraction reminder information can be generated to The heart reminder message reminds the driver to return to a safe driving state.

In some implementations, there may be multiple time thresholds, for example, the time thresholds may include a first time threshold, a second time threshold, and a third time threshold. Wherein, the first time threshold is smaller than the second time threshold, and the second time threshold is smaller than the third time threshold. Each time threshold may correspond to a level of distraction. The larger the time threshold, the higher the level of distraction. For example, the level of distraction corresponding to the first time threshold may be mild distraction, and the level of distraction corresponding to the second time threshold The level of distraction may be moderate distraction, and the level of distraction corresponding to the third time threshold may be severe distraction. When the duration of the driver's distracted driving state reaches the first time threshold, the first distraction reminder information may be generated, and when the duration of the driver's distracted driving state reaches the second time threshold , the second distraction reminder information can be generated, and when the duration of the driver in the distracted driving state reaches the third time threshold, the third distraction reminder information can be generated, the first distraction reminder information, the second distraction reminder The information responsiveness of the distraction reminder information and the third distraction reminder information may be positively correlated with the distraction level. Here, the information response degree may be reflected by means of reminder duration, reminder volume, reminder frequency, and the like. Taking the reminder duration as an example, the reminder durations of the first distraction reminder information, the second distraction reminder information and the third distraction reminder information can be sequentially increased, that is, it can be understood that the longer the driver is in the distracted driving state, the longer the reminder duration. Large, the higher the distraction level, the longer the reminder duration of the distraction reminder message. In this way, the driver can be reminded to return to a safe driving state as soon as possible to reduce the occurrence of traffic accidents.

In some implementations, a method for setting the above time threshold is also provided, that is, the above time threshold can be determined according to the driving speed of the vehicle. For example, the higher the driving speed of the vehicle, the smaller the corresponding time threshold, that is, the driving speed of the vehicle is smaller. Can be negatively correlated with a preset time threshold. In this way, the time threshold for the duration of the distracted driving state can be dynamically adjusted according to the driving speed of the vehicle, thereby improving the flexibility and stability of the distraction reminder and bringing a better user experience to the user.

The distraction reminder solution provided by the embodiment of the present disclosure is described below through an example. In this example, different safety threshold intervals can be preset for different driving speeds through the control interface. The distraction detection system can obtain the current speed of the vehicle through GPS or CAN bus. When the vehicle is stopped or the driving speed is less than or equal to 10km/h, the distraction detection function is turned off. When the vehicle's driving speed reaches the preset starting speed (10km/h), the distraction detection function is automatically triggered. After the distraction detection function is activated, the safety threshold corresponding to the driving speed of the vehicle can be determined. When the driving speed of the vehicle is greater than 10 km/h and less than or equal to 20 km/h, the safety threshold of the head deflection angle corresponding to the driving speed may include: yaw[-30,30], Pitch[-20,20]; When the driving speed of the vehicle is greater than 20 km/h, the safety threshold of the head deflection angle corresponding to the driving speed may include: yaw[-15,15], Pitch[-15,15]. If the driver's head deflection angle is within the head deflection angle safety threshold corresponding to the driving speed, it can be considered that the driver's driving state is a safe driving state, and distraction detection can be continued. If the driver's head deflection angle reaches the safety threshold of the head deflection angle corresponding to the driving speed, that is, exceeds the above interval, the driver's driving state can be considered to be a distracted driving state, and distraction reminder information can be generated to remind the driver.

The distraction reminder solution provided by the example of the present disclosure can automatically start the distraction detection function when the driving speed of the vehicle reaches the starting speed, and disable the distraction detection function when the driving speed of the vehicle does not reach the starting speed, thereby When the user stops chatting or slows down to stop, the distraction detection function can be stopped to reduce unnecessary distraction alerts and improve the user experience. On this basis, the safety threshold of the distraction reminder can be dynamically adjusted according to the driving speed of the vehicle, so as to improve the accuracy and stability of the distraction reminder.

It can be understood that the above-mentioned method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle and logic. Those skilled in the art can understand that, in the above method of the specific embodiment, the specific execution order of each step should be determined by its function and possible internal logic.

In addition, the present disclosure also provides a distraction reminder device, electronic device, computer-readable storage medium, and program, all of which can be used to implement any distraction reminder method provided by the present disclosure, and the corresponding technical solutions and descriptions and refer to the method Some of the corresponding records will not be repeated.

FIG. 5 shows a block diagram of a distraction reminder device according to an embodiment of the present disclosure. As shown in FIG. 5 , the device includes:

The first obtaining module 31 is used to obtain the running speed of the vehicle;

The second obtaining module 32 is configured to obtain video frames collected for the driver of the vehicle when the distraction detection function is determined to be activated according to the driving speed of the vehicle;

A determination module 33, configured to determine the driving state of the driver based on the video frame of the driver and the driving speed;

The generating module 34 is configured to generate distraction reminder information when it is determined that the driving state of the driver is a distracted driving state.

In one or more possible implementations, the determining module 33 is configured to detect the head deflection angle and/or the line of sight deflection angle of the driver based on the video frame of the driver; based on the head deflection The angle and/or the deflection angle of the line of sight, and the driving speed, determine the driving state of the driver.

In one or more possible implementation manners, the determining module 33 is configured to acquire the head deflection angle safety threshold and the line of sight deflection angle safety threshold corresponding to the traveling speed; in response to the head deflection angle reaching the When the head deflection angle safety threshold corresponding to the driving speed and/or the line of sight deflection angle reaches the line of sight deflection angle safety threshold, it is determined that the driver is in a distracted driving state.

In one or more possible implementations, the apparatus further includes:

an identification module, for performing face recognition on the video frame to determine the identity information of the driver;

The determining module 33 is configured to acquire the head deflection angle safety threshold and the sight deflection angle safety threshold and corresponding to the driving speed, which are associated with the driver's identity information.

In one or more possible implementations, the second obtaining module 32 is configured to determine a frame sampling rate based on the driving speed of the vehicle; Frame sampling to obtain video frames collected for the driver of the vehicle.

In one or more possible implementations, the determining module 33 is configured to perform key point detection on the video frame of the driver, and determine at least one key point included in the face area of the video frame; At least one key point included in the face area is used to determine the head deflection angle and/or the sight deflection angle of the driver.

In one or more possible implementation manners, the generating module 34 is configured to determine that the driver is in the distracted driving state if it is determined that the driving state of the driver is the distracted driving state duration; when the duration reaches a preset time threshold, generate distraction reminder information.

In one or more possible implementations, the generating module 34 is further configured to determine the time threshold according to the traveling speed of the vehicle.

In some embodiments, the functions or modules included in the apparatuses provided in the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments. For specific implementation, reference may be made to the descriptions of the above method embodiments. For brevity, here No longer.

Embodiments of the present disclosure further provide a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented. The computer-readable storage medium may be a volatile computer-readable storage medium or a non-volatile computer-readable storage medium.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

Embodiments of the present disclosure also provide a computer program product, including computer-readable codes or a non-volatile computer-readable storage medium carrying the computer-readable codes. When the computer-readable codes are executed on an electronic device, the electronic device The processor in executes the instructions for implementing the distraction reminder method provided by any of the above embodiments.

Embodiments of the present disclosure further provide another computer program product for storing computer-readable instructions, which, when executed, cause the computer to perform the operations of the distraction reminder method provided by any of the foregoing embodiments.

The electronic device may be provided as a terminal, server or other form of device.

FIG. 6 shows a block diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, electronic device 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc. terminal.

6, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814 , and the communication component 816 .

The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to perform all or some of the steps of the methods described above. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

Memory 804 is configured to store various types of data to support operation at electronic device 800 . Examples of such data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. Memory 804 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power supply assembly 806 provides power to various components of electronic device 800 . Power supply components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800 .

Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front-facing camera and/or a rear-facing camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.

Audio component 810 is configured to output and/or input audio signals. For example, audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when electronic device 800 is in operating modes, such as calling mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of electronic device 800 . For example, the sensor assembly 814 can detect the on/off state of the electronic device 800, the relative positioning of the components, such as the display and the keypad of the electronic device 800, the sensor assembly 814 can also detect the electronic device 800 or one of the electronic device 800 Changes in the position of components, presence or absence of user contact with the electronic device 800 , orientation or acceleration/deceleration of the electronic device 800 and changes in the temperature of the electronic device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as wireless network (WiFi), second generation mobile communication technology (2G) or third generation mobile communication technology (3G), or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmed gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation is used to perform the above method.

In an exemplary embodiment, a non-volatile computer-readable storage medium, such as a memory 804 comprising computer program instructions executable by the processor 820 of the electronic device 800 to perform the above method is also provided.

FIG. 7 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. 7, electronic device 1900 includes processing component 1922, which further includes one or more processors, and a memory resource represented by memory 1932 for storing instructions executable by processing component 1922, such as applications. An application program stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Additionally, the processing component 1922 is configured to execute instructions to perform the above-described methods.

The electronic device 1900 may also include a power supply assembly 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input output (I/O) interface 1958 . The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as a Microsoft server operating system (Windows Server ^™ ), a graphical user interface based operating system (Mac OS X ^™ ) introduced by Apple, a multi-user multi-process computer operating system (Unix ^™ ), Free and Open Source Unix-like Operating System (Linux ^™ ), Open Source Unix-like Operating System (FreeBSD ^™ ) or the like.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as memory 1932 comprising computer program instructions executable by processing component 1922 of electronic device 1900 to perform the above-described method.

The present disclosure may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present disclosure.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above. Computer-readable storage media, as used herein, are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.

The computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for carrying out operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages. Source or object code, written in any combination, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect). In some embodiments, custom electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be personalized by utilizing state information of computer readable program instructions. Computer readable program instructions are executed to implement various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.

The computer program product can be specifically implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Various embodiments of the present disclosure have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims

A distraction reminder method, comprising:

Get the speed of the vehicle;

In the case that the distraction detection function is determined to be activated according to the driving speed of the vehicle, acquiring video frames collected for the driver of the vehicle;

determining the driving state of the driver based on the video frame of the driver and the driving speed;

When it is determined that the driving state of the driver is a distracted driving state, distraction reminder information is generated.
The method according to claim 1, wherein the method further comprises:

In the case that the running speed of the vehicle is greater than or equal to the preset starting speed, the distraction detection function is activated;

In the case that the driving speed of the vehicle is lower than the starting vehicle speed, the distraction detection function is turned off.
The method according to claim 1, wherein the determining the driving state of the driver based on the video frame of the driver and the driving speed comprises:

Detecting the driver's head deflection angle and/or line of sight deflection angle based on the driver's video frame;

The driving state of the driver is determined based on the head deflection angle and/or the sight deflection angle, and the driving speed.
The method according to claim 3, wherein the determining the driving state of the driver based on the head deflection angle and/or the sight deflection angle and the driving speed comprises:

obtaining the head deflection angle safety threshold and the sight deflection angle safety threshold corresponding to the travel speed;

In response to the head deflection angle reaching a head deflection angle safety threshold corresponding to the travel speed and/or the sight deflection angle reaching the sight deflection angle safe threshold, it is determined that the driver is in a distracted driving state.
5. The method of claim 4, wherein the head deflection angle safety threshold and/or the sight deflection angle safety threshold are based on the driver's experience when driving the vehicle at the travel speed in a safe driving state Head deflection angle and sight deflection angle calibration.
The method according to claim 4, wherein the method further comprises:

Perform face recognition on the video frame to determine the driver's identity information;

The obtaining of the head deflection angle safety threshold and the sight deflection angle safety threshold corresponding to the driving speed includes:

The head deflection angle safety threshold and the sight deflection angle safety threshold corresponding to the driving speed and associated with the driver's identity information are acquired.
The method according to any one of claims 1 to 6, wherein the acquiring the video frames collected for the driver of the vehicle comprises:

determining a frame sampling rate based on the speed of travel of the vehicle;

Frame sampling is performed on the video of the driver of the vehicle based on the frame sampling rate, and video frames collected for the driver of the vehicle are acquired.
The method according to claim 3, wherein the detecting the head deflection angle and/or the sight deflection angle of the driver based on the video frame of the driver comprises:

Perform key point detection on the video frame of the driver, and determine at least one key point included in the face area of the video frame;

Based on at least one key point included in the face area, a head deflection angle and/or a sight deflection angle of the driver is determined.
The method according to any one of claims 1 to 8, wherein the generating distraction reminder information when it is determined that the driving state of the driver is a distracted driving state comprises:

In the case of determining that the driving state of the driver is a distracted driving state, determining the duration of the driver in the distracted driving state;

When the duration reaches a preset time threshold, distraction reminder information is generated.
The method according to claim 9, wherein the method further comprises:

The time threshold is determined based on the travel speed of the vehicle.
A distraction reminder device, comprising:

a first obtaining module, used for obtaining the driving speed of the vehicle;

a second acquisition module, configured to acquire video frames collected for the driver of the vehicle when the distraction detection function is determined to be activated according to the driving speed of the vehicle;

a determining module for determining the driving state of the driver based on the video frame of the driver and the driving speed;

A generating module, configured to generate distraction reminder information when it is determined that the driving state of the driver is a distracted driving state.
The apparatus of claim 11, wherein the apparatus further comprises:

a startup module, configured to start the distraction detection function when the running speed of the vehicle is greater than or equal to the preset starting speed; when the driving speed of the vehicle is less than the starting speed, turn off the distraction detection Function.
The device according to claim 11, wherein the determining module is configured to detect the head deflection angle and/or the sight deflection angle of the driver based on the video frame of the driver; The deflection angle and/or the line of sight deflection angle, as well as the driving speed, determine the driving state of the driver.
The device according to claim 13, wherein the determining module is configured to obtain a head deflection angle safety threshold and a line of sight deflection angle safety threshold corresponding to the driving speed; The head deflection angle safety threshold corresponding to the driving speed and/or the line of sight deflection angle reaches the line of sight deflection angle safety threshold, and it is determined that the driver is in a distracted driving state.
15. The apparatus of claim 14, wherein the head yaw angle safety threshold and/or the line of sight yaw angle safety threshold are based on the driver's experience when driving the vehicle at the travel speed in a safe driving state Head yaw angle and line of sight yaw angle calibration; and/or

The device also includes:

an identification module, for performing face recognition on the video frame to determine the identity information of the driver;

The determining module is configured to acquire the head deflection angle safety threshold and the sight deflection angle safety threshold and corresponding to the driving speed, which are associated with the driver's identity information.
The device according to any one of claims 11 to 15, wherein the second obtaining module is configured to determine a frame sampling rate based on the traveling speed of the vehicle; Frame sampling is performed on the video of the driver of the vehicle to obtain video frames collected for the driver of the vehicle.
The device according to any one of claims 11 to 16, wherein the generating module is configured to determine that the driver is in any The duration of the distracted driving state; when the duration reaches a preset time threshold, a distraction reminder message is generated.
The device according to claim 17, wherein the generating module is further configured to determine the time threshold according to the traveling speed of the vehicle.
An electronic device, comprising:

processor;

memory for storing processor-executable instructions;

wherein the processor is configured to invoke the memory-stored instructions to perform the method of any one of claims 1-10.
A computer-readable storage medium on which computer program instructions are stored, characterized in that, when the computer program instructions are executed by a processor, the method described in any one of claims 1 to 10 is implemented.
A computer program product, characterized by comprising computer-readable codes or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are executed in a processor of an electronic device, A processor in the electronic device executes the method for implementing any one of claims 1 to 10.