CN114701503A - Method, device and equipment for adjusting driving behavior of vehicle driver and storage medium - Google Patents

Method, device and equipment for adjusting driving behavior of vehicle driver and storage medium Download PDF

Info

Publication number
CN114701503A
CN114701503A CN202111672793.8A CN202111672793A CN114701503A CN 114701503 A CN114701503 A CN 114701503A CN 202111672793 A CN202111672793 A CN 202111672793A CN 114701503 A CN114701503 A CN 114701503A
Authority
CN
China
Prior art keywords
driver
posture
driving
information
standard
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111672793.8A
Other languages
Chinese (zh)
Inventor
侯玉璐
史利民
张磊
任献东
宋玉强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin Wuba Driving Test Information Technology Co ltd
Original Assignee
Tianjin Wuba Driving Test Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin Wuba Driving Test Information Technology Co ltd filed Critical Tianjin Wuba Driving Test Information Technology Co ltd
Priority to CN202111672793.8A priority Critical patent/CN114701503A/en
Publication of CN114701503A publication Critical patent/CN114701503A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/0098—Details of control systems ensuring comfort, safety or stability not otherwise provided for
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/143—Alarm means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/223—Posture, e.g. hand, foot, or seat position, turned or inclined
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/229—Attention level, e.g. attentive to driving, reading or sleeping

Abstract

The embodiment of the application provides a method, a device, equipment and a storage medium for adjusting driving behaviors of vehicle drivers. In the embodiment of the application, a plurality of images of a driver are collected in the driving process of the driver, the integral driving posture information and/or the visual attention information of the driver are determined according to the collected images, the behavior state of the driver is determined according to the integral driving posture information and/or the visual attention information of the driver, and when the driver is in the unsafe driving behavior state, adjustment guide information matched with the historical driving behavior state of the driver is timely output to prompt and guide the driver to adjust the driving behavior of the driver, so that the accuracy and the effectiveness of outputting the adjustment guide information are improved, and the driving safety of the driver is guaranteed.

Description

Method, device and equipment for adjusting driving behavior of vehicle driver and storage medium
Technical Field
The present disclosure relates to the field of vehicle driving technologies, and in particular, to a method, an apparatus, a device, and a storage medium for adjusting driving behaviors of a vehicle driver.
Background
At present, with the development of vehicle technology, vehicles enable people to go out more and more efficiently and conveniently. More and more users start driving the vehicle by themselves to go out, and certainly, the users need to take the driving license before driving on the road. The driving safety needs to be paid attention to no matter in the process of learning the vehicle or in the actual driving process after learning the vehicle.
In the prior art, a technical scheme for acquiring heartbeat, breath and other physiological parameters of a vehicle driver to detect whether the driver is in fatigue driving exists, and when the driver is detected to be in a fatigue driving state, the vehicle driver is timely reminded to stop for a rest to ensure driving safety. However, fatigue driving is only one factor affecting driving safety, and the reminding technology related to driving safety needs to be further improved.
Disclosure of Invention
Aspects of the application provide a method, a device, equipment and a storage medium for adjusting driving behaviors of a driver of a vehicle, so as to guarantee driving safety of the driver.
The embodiment of the application provides a method for adjusting driving behaviors of vehicle drivers, which is applied to a target vehicle, wherein at least one visual sensor is installed in the target vehicle, and the method comprises the following steps: collecting a plurality of images of a driver in a driving process by using at least one vision sensor; respectively identifying the integral driving posture information and/or the visual attention information of the driver according to the plurality of images; determining the driving behavior state of a driver according to the integral driving posture information and/or the visual attention information; when a driver is in a non-safe driving behavior state, outputting first adjustment guide information to prompt and guide the driver to adjust the driving behavior; the first adjustment guide information is adapted to the historical driving behavior state of the driver.
The embodiment of the present application still provides a vehicle navigating mate driving action adjusting device, is applied to the target vehicle, installs at least one vision sensor in the target vehicle, and vehicle navigating mate driving action adjusting device includes: the device comprises an acquisition module, an identification module, a determination module and an output module; the acquisition module is used for acquiring a plurality of images of a driver in the driving process by utilizing at least one vision sensor; the recognition module is used for respectively recognizing the integral driving posture information and/or the visual attention information of the driver according to the plurality of images; the determining module is used for determining the driving behavior state of a driver according to the integral driving posture information and/or the visual attention information; the output module is used for outputting first adjustment guiding information when a driver is in a non-safe driving behavior state so as to prompt and guide the driver to adjust the driving behavior; the first adjustment guide information is adapted to the historical driving behavior state of the driver.
The embodiment of the present application still provides a vehicle navigating mate driving action adjusting device, is applied to the target vehicle, installs at least one vision sensor in the target vehicle, and equipment includes: a memory and a processor; a memory for storing a computer program; and the processor is coupled with the memory and is used for executing the computer program so as to realize the steps in the method for adjusting the driving behavior of the vehicle driver provided by the embodiment of the application.
The embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the method for adjusting driving behavior of a driver of a vehicle provided in the embodiment of the present application.
In the embodiment of the application, a plurality of images of a driver are collected in the driving process of the driver, the integral driving posture information and/or the visual attention information of the driver are determined according to the collected images, the behavior state of the driver is determined according to the integral driving posture information and/or the visual attention information of the driver, and when the driver is in the unsafe driving behavior state, the adjustment guide information matched with the historical driving behavior state of the driver is output in time to prompt and guide the driver to adjust the driving behavior of the driver, so that the accuracy and the effectiveness of outputting the adjustment guide information are improved, and the driving safety of the driver is guaranteed.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic flow chart illustrating a method for adjusting driving behavior of a driver of a vehicle according to an exemplary embodiment of the present disclosure;
FIG. 2 is a schematic flow chart illustrating a process for determining a driving behavior state of a driver according to an exemplary embodiment of the present application;
FIG. 3a is a schematic diagram of an output result of a pose detection model according to an exemplary embodiment of the present application;
FIG. 3b is a schematic diagram of an output result of another gesture detection model provided in an exemplary embodiment of the present application;
FIG. 4 is a schematic diagram illustrating the generation and use of a gesture detection model provided by an exemplary embodiment of the present application;
FIG. 5 is a schematic structural diagram of a driving behavior adjustment apparatus for a vehicle driver according to an exemplary embodiment of the present application;
fig. 6 is a schematic structural diagram of a driving behavior adjustment device for a vehicle driver according to an exemplary embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the embodiment of the application, a plurality of images of a driver are collected during the driving process of the driver, the integral driving posture information and/or the visual attention information of the driver are determined according to the collected images, the behavior state of the driver is determined according to the integral driving posture information and/or the visual attention information of the driver, and when the driver is in the unsafe driving behavior state, adjustment guide information matched with the historical driving behavior state of the driver is timely output to prompt and guide the driver to adjust the driving behavior of the driver, so that the accuracy and the effectiveness of the output adjustment guide information are improved, and the driving safety of the driver is guaranteed.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of a method for adjusting driving behavior of a vehicle driver according to an exemplary embodiment of the present disclosure. The method is applied to a target vehicle in which at least one vision sensor is installed, as shown in fig. 1, and includes:
101. acquiring a plurality of images of a driver in a driving process by utilizing at least one vision sensor;
102. respectively identifying the integral driving posture information and/or the visual attention information of the driver according to the plurality of images;
103. determining the driving behavior state of a driver according to the integral driving posture information and/or the visual attention information;
104. when a driver is in an unsafe driving behavior state, outputting first adjustment guide information to prompt and guide the driver to adjust the driving behavior; the first adjustment guide information is adapted to the driver's historical driving behavior state.
In this embodiment, the vision sensor may include, but is not limited to: a monocular camera, a binocular camera, or a Red Green Blue Depth map (RGBD) camera, etc. The number of the visual sensors is not limited, and the number of the visual sensors may be 1, or may be plural, for example, 3 or 5. Wherein, the installation position of the vision sensor is not limited. In the case where the number of the vision sensors is one, the vision sensors may be mounted above the roof of the vehicle interior, on the side wall of the left door, or on the instrument desk; in the case where the number of the vision sensors is plural, for example, the number of the vision sensors is 3, and the vision sensors may be mounted above the instrument panel, on the roof on the upper right side of the back of the main cab, and on the front right side of the position of the front passenger door.
In this embodiment, the overall driving posture information and/or the visual attention information of the driver may be respectively recognized according to a plurality of images collected by at least one visual sensor. The overall driving posture information mainly represents the relative position relationship between the body trunk or the head of the driver and the target vehicle, for example, the distance between the head and the top of the target vehicle, the position relationship between the head and the door, the angle of the back seat, and the like. The visual attention information is information detected from a face area of the driver, and may be attention-focused, attention-lost, normal attention, or the like.
In the present embodiment, the driving behavior state of the driver is determined in conjunction with the overall driving posture information and/or the visual attention information of the driver. For example, the driving behavior state of the driver may be determined based on the overall driving posture information of the driver, or the driving behavior state of the driver may be determined based on the visual attention information of the driver, or the driving behavior state of the driver may be determined based on the overall driving posture information and the visual attention information of the driver.
In this embodiment, the driving behavior state includes a safe driving behavior state and an unsafe driving behavior state, and when the driver is in the unsafe driving behavior state, the first adjustment guidance information may be output to prompt and guide the driver to adjust the driving behavior of the driver. The first adjustment guide information is not limited, and for example, if the overall driving behavior posture indicates that the driver is in the unsafe driving behavior state, a prompt message for prompting the driver to adjust the overall posture of the driver, such as "please correct the head", or "please adjust the seat back forward", may be output externally. For example, if the visual attention information indicates that the driver is in the unsafe driving behavior state, the visual attention information may be output to the outside to prompt the driver to change the attention, for example, if it is detected that the driver is not paying attention, the visual attention information may be output to the outside to prompt "please pay attention", and if it is detected that the driver is paying attention too much, the visual attention information may be output to the outside to prompt "please relax".
Further, when the driver is in the unsafe driving behavior state, the first adjustment guidance information adapted to the historical driving behavior state of the driver may be output, and for any unsafe driving behavior state, if the historical driving behavior state of the driver indicates that the driver is in the unsafe driving behavior state for a plurality of times in the historical period, the first adjustment guidance information may also be output individually by the coach voice or the voice recorded by the driver himself/herself, for example, "danger! Danger! When the driver turns, the driver forgets to turn on the turn light for many times, and then pays attention to the turn light, or the driver is reminded not to be in the unsafe driving behavior state again through a honey device or a warning sound effect. In the whole process, the first adjustment guiding information can be output in a personalized mode according to the historical driving behavior state of the driver, attention of the driver is attracted, the driver can adjust the driving behavior in a targeted mode, and driving safety of the driver is guaranteed.
In addition, in the scene of learning the driving by the trainee, when the driver is in the unsafe driving behavior state, the voice guidance information of the coach can be remotely output to assist the driver to adjust the driving behavior state or guide the driver to execute subsequent driving behaviors besides outputting the first adjustment guidance information matched with the historical driving behavior state of the driver, and in the whole process, personalized reminding or guidance can be performed on the driving trainee, so that the driver can accurately and quickly master the driving skill. Further, under the condition that the driver is in the unsafe driving behavior state, the target vehicle driven by the driver can be remotely controlled, for example, the angle of a steering wheel is adjusted, a brake, a clutch or an accelerator is controlled, the running state of the target vehicle is adjusted in time, the driver is prevented from being in danger, and the safety of the driver is guaranteed.
In the embodiment of the application, a plurality of images of a driver are collected in the driving process of the driver, the integral driving posture information and/or the visual attention information of the driver are determined according to the collected images, the behavior state of the driver is determined according to the integral driving posture information and/or the visual attention information of the driver, and when the driver is in the unsafe driving behavior state, the adjustment guide information matched with the historical driving behavior state of the driver is output in time to prompt and guide the driver to adjust the driving behavior of the driver, so that the accuracy and the effectiveness of outputting the adjustment guide information are improved, and the driving safety of the driver is guaranteed.
In an alternative embodiment, an embodiment in which the number of the vision sensors is 2 and the vision sensor is a camera is taken as an example, and a plurality of images are acquired by the vision sensors is described. The camera a1 is above the instrument desk and is used to detect the head attitude of the driver. The camera a2 is located on the front right side of the passenger door position and detects the body posture of the driver. Wherein the body posture may be, but is not limited to: the distance of the torso from the steering wheel or the angle of the seat back, etc. Based on the above, an embodiment of recognizing the overall driving posture information of the driver from a plurality of images includes: according to a plurality of images collected by at least one vision sensor, a first actual distance between the trunk of a driver and a steering wheel, a second actual distance between the head of the driver and the top of a vehicle and an actual angle of a seat back are identified, and the first actual distance, the second actual distance and the actual angle of the seat back are used as the whole driving posture information of the driver.
In an alternative embodiment, the camera a1 may also detect the facial aspect or eye state of the driver in the case of detecting the head posture of the driver, or the camera a1 may detect both the facial aspect and eye state of the driver. Among these, facial expression includes but is not limited to: status information of eyebrows, mouth, facial muscles, etc., eye state including but not limited to: pupil, eyelid, eye circumference, etc. Based on this, an embodiment of recognizing visual attention information of a driver from a plurality of images includes: selecting a target image including a face area of a driver from the plurality of images; for example, a target image with a driver's face region may be selected from the images captured by camera a 1; the method comprises the steps of recognizing the facial expression or the eye expression state of a driver from a target image, determining the visual attention information of the driver according to the recognized facial expression or eye expression state, or recognizing the facial expression and the eye expression state of the driver from the target image, and determining the visual attention information of the driver according to the recognized facial expression and eye expression state.
In an alternative embodiment, as shown in fig. 2, an implementation of determining the driving behavior state of the driver according to the overall driving posture information and/or the visual attention information includes: judging whether the overall driving posture information accords with a standard driving posture corresponding to safe driving or not, and judging whether the visual attention information accords with a standard attention condition corresponding to the safe driving or not; and if the overall driving posture information does not accord with the standard driving posture or the visual attention information does not accord with the standard attention condition, determining that the driver is in the unsafe driving behavior state.
The standard attention condition may be an attention condition set in advance according to a historical driving behavior analysis of the driver or a facial expression and/or an eye expression of the driver. For example, after analyzing the historical behavior of a driver, when it is determined that the duration of time that the driver gazes at a certain location exceeds a certain threshold, the driver is in an inattentive state, the standard attention condition may be that the duration of time that the driver gazes at a certain location does not exceed a set duration threshold, which may be 1s, 2s, or 5s, and the like, wherein the duration threshold is determined according to the analysis result of the historical behavior of the driver, and different drivers may correspond to different thresholds. For another example, if the size of the eye opening of the driver depends on the height of the eye socket, the height of the eye opening of the driver can be analyzed, and it is determined that the driver is in a state of attention when the height of the eye opening of the driver is L ± 3mm, and the standard attention condition may be that the height of the eye opening is L ± 3mm, where L may be 5mm, 6mm, or 8mm, and the value of L is determined according to the height of the eye socket of the driver, and different drivers correspond to different values of L. The standard driving posture is adapted to the driver, the standard driving posture is not limited, and the standard driving posture is different according to different body forms of the driver, for example, a head of a high driver is shorter than the top of the vehicle, and a head of a low driver is longer than the top of the vehicle; for a driver with a large weight, the distance from the trunk to the steering wheel is short, and for a driver with a small weight, the distance from the trunk to the steering wheel is long. In view of the above, the standard driving posture can be set individually for each driver, for example, before driving, the driver is in the driving position with the standard posture, and at this time, at least one vision sensor collects the posture information of the driver, and the driving posture in the collected image is taken as the standard driving posture for the driver.
In an alternative embodiment, the standard driving gestures include: a first standard distance of the driver's torso from the steering wheel, a second standard distance of the driver's head from the top of the vehicle, and a standard angle of the seat back. Wherein, judge whether whole driving posture information accords with the implementation mode of the standard driving posture that safe driving corresponds, include: and if the first actual distance and the first standard distance do not meet the set first distance condition, or the second actual distance and the second standard distance do not meet the set second distance condition, or the actual angle and the standard angle do not meet the set angle condition, determining that the whole driving posture information does not meet the standard driving posture. The set first distance condition may be that an absolute value of a difference between the first actual distance and the first standard distance is smaller than a set first difference threshold, where the first difference threshold may be 1cm, 2cm, or 5cm, and the like. The set second distance condition may be that an absolute value of a difference between the second actual distance and the second standard distance is smaller than a set second difference threshold, and the first difference threshold may be 2cm, 3cm, or 4cm, and the like. The set angle condition may be that an absolute value of a difference between the actual angle and the standard angle is smaller than a set third difference threshold, and the third difference threshold may be 5 degrees, 7 degrees, or 8 degrees, which is not limited herein.
Correspondingly, when the driver is in the unsafe driving behavior state, outputting first adjustment guiding information to prompt and guide the driver to adjust the driving behavior, wherein the method comprises the following steps: if the first actual distance and the first standard distance do not meet the set first distance condition, trunk adjustment guiding information is output, for example, "body trunk is moved forward"; if the second actual distance and the second standard distance do not meet the set second distance condition, outputting head adjustment guide information, for example, "please put the head posture right"; if the actual angle and the standard angle of the seat back do not satisfy the set angle condition, the seat back adjustment guidance information is output, for example, "please adjust the angle of the seat back larger".
In an alternative embodiment, the determining whether the visual attention information meets the standard attention condition corresponding to safe driving includes: if the eyes of the driver are distracted and the duration exceeds a set first time threshold value, or the eyes of the driver are concentrated and the duration exceeds a set second time threshold value, determining that the visual attention information does not meet the standard attention condition; correspondingly, when the driver is in the unsafe driving behavior state, the first adjustment guide information is output to prompt and guide the driver to adjust the driving behavior of the driver, and the method comprises the following steps: if the distraction of the eyes of the driver is detected and the duration exceeds a set first time threshold, outputting guide information for reminding the driver to improve the attention; or if the fact that the attention of the driver is concentrated and the duration time exceeds a set second time threshold value is detected, guiding information for reminding the driver of relaxing is output.
In this embodiment, in a driving scene of a vehicle, for example, in a scene of a student learning to walk, the student instructs a coach to point positions when the student learns to walk, for example, when the student needs to walk to the right rear corner of a garage in the direction of sight when the student parks on the side, the student returns to the right steering wheel, and when the student parks in the garage, the student needs to grasp the timing of rotating the steering wheel according to the marking, and when the student parks in the garage, the student needs to determine whether the parking position is proper according to the distance between the rearview mirror and the marking. However, the height and the size of each student are different, the driving posture and the backrest angle suitable for the student are different, and all the factors cause the deviation when the student grasps the point. In the embodiment, the driving postures of different drivers can be determined individually according to the overall driving posture information and the visual attention information, so that the drivers are helped to search for accurate point locations, the accuracy of point location searching in the driving process is improved, and the driving safety is improved.
In an optional embodiment, the method further comprises detecting and recognizing a local driving gesture of the driver, wherein the local driving gesture comprises at least one of a head gesture, a hand gesture and a trunk gesture. Specifically, a plurality of images acquired by at least one vision sensor can be input into a posture detection model to detect and identify the local driving posture of a driver; and if at least one of the head posture, the hand posture and the trunk posture of the driver is recognized to be matched with the wrong posture of the corresponding part, outputting second adjustment guide information to the outside so as to enable the driver to adjust the posture of the corresponding part. The gesture detection module may be any target detection model, for example, an algorithm model for mobile network-you only look at once (mobile net-you only look once, mobile-yolov 4) to deeply learn target detection, which is a lightweight model and is suitable for a mobile terminal.
In an optional embodiment, if it is recognized that at least one of the head posture, the hand posture and the trunk posture of the driver matches the wrong posture of the corresponding part, the second adjustment guidance information is output externally to provide the driver with an implementation manner of adjusting the posture of the corresponding part, which is described in three cases below.
Embodiment X1:aiming at head postureAnd if the fact that the twisting angle of the driver relative to the standard head posture exceeds a set first head torsion angle or the angle of downward head lowering exceeds a set second head torsion angle is detected, outputting a head posture adjustment guide message to the outside.
For example, a driver should look straight ahead directly when driving, and can slightly twist his head when observing the rearview mirror, that is, the twisting angle relative to the standard head posture does not exceed the set first head twisting angle; however, if the head twisting amplitude is too large in the process of observing the rearview mirror, namely, the twisting angle of the driver relative to the standard head posture is detected to exceed the set first head twisting angle, the driving safety problem can exist, and the head posture adjustment guide message can be output outwards to adjust the head posture of the driver. For example, the second adjustment guidance information may be "please look ahead in time" or "please twist the head by a small amount" or the like. Wherein the first head torsion angle may be 10 degrees, 20 degrees, 50 degrees, or the like.
For another example, during driving, the driver may look over the pedal or the gear of the vehicle by lowering his head, and the angle of lowering his head downward exceeds the second set head torsion angle, which may cause a driving hazard, and output a head posture adjustment guidance message to the outside for the driver to adjust the head posture. For example, the head pose adjustment guidance message may be "please raise the head" or "please do not lower the head", or the like. Wherein, the second head torsion angle may be 20 degrees, 30 degrees, 50 degrees, or the like.
Embodiment X2:and outputting hand posture adjustment guide information if the hand posture of the driver does not meet the set steering wheel use condition when the driver uses the steering wheel. For example, for the case of holding the steering wheel gesture by the driver in a straight line, the driver should hold the steering wheel with both hands and put the hands in the 9 o 'clock direction and the 3 o' clock direction respectively in the straight line, but the driver may not put the hands in the above-mentioned positions, and then the hand posture adjustment guidance information may be output externally, for example: please place both hands on both sides of the steering wheel. As shown in FIG. 3a, a view taken by a vision sensorThe hand region (hand) is marked in fig. 3a by using the gesture detection model, and the probability that the region is a hand is determined, for example, the probability that the hand (hand) is recognized in the first mark region is 1.00, the probability that the hand (hand) is recognized in the second mark region is 0.99, and the probability that the hand gesture is correct (right) in the third mark region is 1.00. For another example, in the case of a gesture of holding a steering wheel during turning, when the driver rotates the steering wheel to the rightmost or leftmost position during turning, the hands should be adjusted back to appropriate positions, and the hands cannot be kept crossed, and at this time, if it is detected that the hands of the driver are crossed and a set time is exceeded, for example, 2 seconds, hand posture adjustment guide information is output to the outside, for example, "please not to place the hands crossed". For example, in the case of a single-handed steering wheel, the driver operates the steering wheel with only one hand due to an improper operation during driving, and if it is detected that the driver holds the steering wheel with one hand for more than a set time, for example, 1 second, hand posture adjustment guidance information, for example, "please not hold the steering wheel with one hand", is output to the outside.
When the driver uses the shift lever, the hand posture of the driver does not satisfy the set shift lever use condition, and the hand posture adjustment guide information is output. For example, the gesture detection of the driver's gear engagement mainly determines whether the timing of the driver's gear engagement is accurate, and cannot place the hand on the shift lever for a long time, and if the time for detecting that the hand is placed on the shift lever exceeds a set time, for example, 2 seconds, hand posture adjustment guidance information, for example, "please do not place the hand on the shift lever", is output to the outside.
And under the condition that the driver uses the loudspeaker, if the hand posture of the driver does not meet the set loudspeaker using condition, outputting hand posture adjustment guide information. For example, a driver may press the horn for a long time while using the horn, and if it is detected that the driver presses the horn for more than a set time, for example, 2 seconds, hand posture adjustment guidance information, for example, "please stop pressing the horn", is output to the outside.
And under the condition that the driver uses the light or the windshield wiper to adjust the switch, the hand posture of the driver does not meet the set switch using condition, and then hand posture adjustment guide information is output. For example, the driver outputs the hand posture adjustment guide information to the outside when the driver does not use the turn signal, for example, "please turn on the turn signal", or outputs the hand posture adjustment guide information to the outside when the driver does not use the wiper during rain, for example, "please turn on the wiper".
Embodiment X3:and for the trunk posture, if the fact that the trunk of the driver and the vehicle steering wheel do not meet the set third distance condition or the trunk posture of the driver does not meet the set inclination or rotation condition is detected, trunk posture adjustment guide information is output. For example, the driver should keep sitting upright while driving, and if it is detected that the trunk of the driver and the vehicle steering wheel do not satisfy the set third distance condition, for example, the set third distance condition is a distance range, and if it is detected that the distance between the trunk of the driver and the vehicle steering wheel is not within the distance range, it is considered that the driving behavior state of the driver does not satisfy the set third distance condition, and trunk posture adjustment guidance information, for example, "please adjust the body trunk forward", is output. For example, if the set inclination or rotation condition is an inclination angle range or a rotation angle range, and if it is detected that the inclination angle of the trunk posture of the driver is not within the inclination angle range or the rotation angle of the trunk posture of the driver is not within the rotation angle range, it is determined that the trunk posture of the driver does not satisfy the set inclination or rotation condition, trunk posture adjustment guidance information, for example, "please not to incline the body" or "please not to rotate the body", is output. As shown in fig. 3b, the probability of marking the torso pose as a wrong posture (wrongposition) in the first marker region is 0.93.
In this embodiment, the adjustment of the driving behavior of the vehicle driver can be realized by adopting a deep learning network model, wherein the deep learning network model can be realized as the posture detection model in the present application. The following is divided into three parts for description, wherein the first part is non-safe driving behavior state classification, the second part is model training, and the third part is driving behavior detection.
(1) A first part: and determining the unsafe driving behavior state of the driver. For example, the driver may be in an unsafe driving behavior state such as a head posture, a hand posture, and a torso posture, or may be in an unsafe driving behavior state such as a facial expression and/or an eye expression of the driver. The model of deep learning training is adopted, more unsafe driving behavior states can be found in the driving process of a driver, if new unsafe driving behavior states are found, transfer learning can be carried out on the basis of the original model, the model is retrained, the model is continuously improved, and the accuracy of the model for detecting the driving behavior states is improved.
(2) A second part: the model was trained as shown in fig. 4.
Collecting an image: the method comprises the steps of utilizing a visual sensor to collect images, wherein the images can be collected by the visual sensor or videos, selecting different types of images by reading video frames under the condition that the video is collected by the visual sensor, wherein the ratio of correct images (safe driving behavior state) to correct images (safe driving behavior state) is 30%, the ratio of wrong images (non-safe driving behavior state) to wrong images (non-safe driving behavior state) is 70%, and storing all images after disordering the sequence.
Labeling data: the images are labeled one by using an image labeling (labelImg) tool, and the labeled images and Extensible Markup Language (xml) files are sorted into files which can be used for target detection training.
Training a model: and performing model training by using a target detection model, such as an algorithm model for deep learning target detection of mobilenet-yolov4, wherein the model is a lightweight model and is suitable for a mobile terminal. And training the model by using the marked data, and storing the trained model.
(3) And a third part: driving behavior detection
Collecting images in real time: capturing a video frame from a video acquired by a visual sensor to obtain a plurality of images, processing the plurality of images and transmitting the processed images to a model for detecting the driving behavior posture;
and (3) detecting the model: and deploying the trained target detection model to terminal equipment, vehicle-mounted terminal equipment or a server, inputting the image into the target detection model to detect the driving behavior gesture, judging whether the driver has an unsafe driving behavior state in the driving process, and feeding back.
And (3) giving adjustment guide information: if the behavior of the driver has no problem, no prompt is given, and if the unsafe driving behavior state exists, the adjustment guide information is immediately output to prompt and guide the driver to adjust the driving behavior.
It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subjects of steps 101 to 103 may be device a; for another example, the execution subject of steps 101 and 102 may be device a, and the execution subject of step 103 may be device B; and so on.
In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations, such as 101, 102, etc., are merely used for distinguishing different operations, and the sequence numbers do not represent any execution order per se. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
Fig. 5 is a schematic structural diagram of a driving behavior adjustment device for a vehicle driver, which is mountable on a target vehicle implemented as an on-board terminal of the target vehicle, as shown in fig. 5, where at least one visual sensor is mounted in the target vehicle, and the driving behavior adjustment device for the vehicle driver includes: an acquisition module 51, a recognition module 52, a determination module 53 and an output module 54.
The acquisition module 51 is used for acquiring a plurality of images of a driver in the driving process by utilizing at least one vision sensor;
the recognition module 52 is configured to recognize the overall driving posture information and/or the visual attention information of the driver according to the plurality of images;
a determining module 53, configured to determine a driving behavior state of the driver according to the overall driving posture information and/or the visual attention information;
the output module 54 is configured to output first adjustment guidance information to prompt and guide the driver to adjust the driving behavior of the driver when the driver is in the unsafe driving behavior state; the first adjustment guide information is adapted to the historical driving behavior state of the driver.
In an optional embodiment, the identification module 52 is specifically configured to: and identifying a first actual distance between the trunk of the driver and the steering wheel, a second actual distance between the head of the driver and the top of the vehicle and an actual angle of the seat back according to the plurality of images to serve as the overall driving posture information of the driver.
In an optional embodiment, the identification module 52 is specifically configured to: selecting a target image including a face area of a driver from the plurality of images; identifying the facial expression and/or eye expression of the driver from the target image; and determining the visual attention information of the driver according to the facial expression and/or the eye expression state.
In an optional embodiment, the determining module 53 is specifically configured to: judging whether the overall driving posture information accords with a standard driving posture corresponding to safe driving or not, and judging whether the visual attention information accords with a standard attention condition corresponding to the safe driving or not; and if the integral driving posture information does not accord with the standard driving posture or the visual attention information does not accord with the standard attention condition, determining that the driver is in a non-safe driving behavior state, wherein the standard driving posture and the standard attention condition are matched with the driver.
In an alternative embodiment, the standard driving gestures include: a first standard distance of a driver's torso from a steering wheel, a second standard distance of a driver's head from a vehicle roof, and a standard angle of a seat back; the determining module 53 is specifically configured to: if the first actual distance and the first standard distance do not meet the set first distance condition, or the second actual distance and the second standard distance do not meet the set second distance condition, or the actual angle and the standard angle do not meet the set angle condition, determining that the whole driving posture information does not meet the standard driving posture; the output module 54 is specifically configured to: if the first actual distance and the first standard distance do not meet the set first distance condition, outputting trunk adjustment guiding information; if the second actual distance and the second standard distance do not meet the set second distance condition, outputting head adjustment guide information; and if the actual angle and the standard angle of the seat back do not meet the set angle condition, outputting seat back adjustment guide information.
In an optional embodiment, the determining module 53 is specifically configured to: if the eyes of the driver are distracted and the duration exceeds a set first time threshold value, or the eyes of the driver are concentrated and the duration exceeds a set second time threshold value, determining that the visual attention information does not meet the standard attention condition; the output module 54 is specifically configured to: if the distraction of the eyes of the driver is detected and the duration exceeds a set first time threshold, outputting guide information for reminding the driver to improve the attention; or if the fact that the attention of the driver is concentrated and the duration time exceeds a set second time threshold value is detected, guiding information for reminding the driver of relaxing is output.
In an optional embodiment, further comprising: a processing module; the processing module is used for inputting the images into the gesture detection model and detecting and identifying local driving gestures of a driver, wherein the local driving gestures comprise at least one of head gestures, hand gestures and trunk gestures; the output module 54 is further configured to: and if at least one of the head posture, the hand posture and the trunk posture of the driver is recognized to be matched with the wrong posture of the corresponding part, outputting second adjustment guide information to the outside so as to enable the driver to adjust the posture of the corresponding part.
In an alternative embodiment, the output module 54 is specifically configured to: aiming at the head posture, if the fact that the twisting angle of a driver relative to the standard head posture exceeds a set first head torsion angle or the angle of downward head lowering exceeds a set second head torsion angle is detected, a head posture adjustment guide message is output outwards; for the hand gesture, if the hand gesture of the driver does not meet the set steering wheel using condition under the condition that the driver uses the steering wheel, or the hand gesture of the driver does not meet the set gear lever using condition under the condition that the driver uses the gear lever, or the hand gesture of the driver does not meet the set horn using condition under the condition that the driver uses the horn, or the hand gesture of the driver does not meet the set switch using condition under the condition that the driver uses the light or the windshield wiper adjusting switch, the hand gesture adjusting guide information is output; and for the trunk posture, if the fact that the trunk of the driver and the vehicle steering wheel do not meet the set third distance condition or the trunk posture of the driver does not meet the set inclination or rotation condition is detected, trunk posture adjustment guide information is output.
The vehicle driver driving behavior adjusting device provided by the embodiment of the application collects multiple images of a driver in the driving process of the driver, determines the whole driving posture information and/or the visual attention information of the driver according to the collected multiple images, determines the behavior state of the driver according to the whole driving posture information and/or the visual attention information of the driver, and outputs the adjustment guide information matched with the historical driving behavior state of the driver in time when the driver is in the unsafe driving behavior state so as to prompt and guide the driver to adjust the driving behavior of the driver, so that the accuracy and the effectiveness of the output adjustment guide information are improved, and the driving safety of the driver is guaranteed.
Fig. 6 is a schematic structural diagram of a driving behavior adjustment device for a vehicle driver according to an exemplary embodiment of the present application. The apparatus is applicable to a target vehicle in which at least one vision sensor is installed, as shown in fig. 6, and includes: a memory 64 and a processor 65.
A memory 64 for storing a computer program and may be configured to store other various data to support operations on the vehicle operator driving behavior adjustment apparatus. Examples of such data include instructions for any application or method operating on the vehicle operator driving behavior adjustment device.
The memory 64 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
A processor 65, coupled to the memory 64, for executing computer programs in the memory 64 for: collecting a plurality of images of a driver in a driving process by using at least one vision sensor; respectively identifying the integral driving posture information and/or the visual attention information of the driver according to the plurality of images; determining the driving behavior state of a driver according to the integral driving posture information and/or the visual attention information; when a driver is in a non-safe driving behavior state, outputting first adjustment guide information to prompt and guide the driver to adjust the driving behavior; the first adjustment guidance information is adapted to the driver's historical driving behavior state.
In an alternative embodiment, the processor 65, when recognizing the overall driving posture information of the driver from the plurality of images, is specifically configured to: and identifying a first actual distance between the trunk of the driver and the steering wheel, a second actual distance between the head of the driver and the top of the vehicle and an actual angle of the seat back according to the plurality of images to serve as the overall driving posture information of the driver.
In an alternative embodiment, the processor 65, when identifying the visual attention information of the driver from the plurality of images, is specifically configured to: selecting a target image including a face area of a driver from the plurality of images; identifying the facial expression and/or eye expression of the driver from the target image; and determining the visual attention information of the driver according to the facial expression and/or the eye expression state.
In an alternative embodiment, the processor 65, when determining the driving behavior state of the driver based on the overall driving posture information and/or the visual attention information, is specifically configured to: judging whether the overall driving posture information accords with a standard driving posture corresponding to safe driving or not, and judging whether the visual attention information accords with a standard attention condition corresponding to the safe driving or not; and if the overall driving posture information does not accord with the standard driving posture or the visual attention information does not accord with the standard attention condition, determining that the driver is in the unsafe driving behavior state, wherein the standard driving posture and the standard attention condition are matched with the driver.
In an alternative embodiment, the standard driving gestures include: a first standard distance of a driver's torso from a steering wheel, a second standard distance of a driver's head from a vehicle roof, and a standard angle of a seat back; when determining whether the overall driving posture information conforms to the standard driving posture corresponding to safe driving, the processor 65 is specifically configured to: if the first actual distance and the first standard distance do not meet the set first distance condition, or the second actual distance and the second standard distance do not meet the set second distance condition, or the actual angle and the standard angle do not meet the set angle condition, determining that the whole driving posture information does not meet the standard driving posture; accordingly, when the driver is in the unsafe driving behavior state, the processor 65 outputs first adjustment guidance information to prompt and guide the driver to adjust the driving behavior, specifically to: if the first actual distance and the first standard distance do not meet the set first distance condition, outputting trunk adjustment guiding information; if the second actual distance and the second standard distance do not meet the set second distance condition, outputting head adjustment guide information; and if the actual angle and the standard angle of the seat back do not meet the set angle condition, outputting seat back adjustment guide information.
In an alternative embodiment, the processor 65, when determining whether the visual attention information meets the standard attention condition corresponding to safe driving, is specifically configured to: if the eyes of the driver are distracted and the duration exceeds a set first time threshold value, or the eyes of the driver are concentrated and the duration exceeds a set second time threshold value, determining that the visual attention information does not meet the standard attention condition; accordingly, when the driver is in the unsafe driving behavior state, the processor 65 outputs the first adjustment guidance information to prompt and guide the driver to adjust the driving behavior, and is specifically configured to: if the distraction of the eyes of the driver is detected and the duration exceeds a set first time threshold, outputting guide information for reminding the driver to improve the attention; or if the attention of the driver is detected to be concentrated and the duration exceeds a set second time threshold, outputting guide information for reminding the driver to relax.
In an alternative embodiment, the processor 65 is further configured to: inputting the multiple images into a posture detection model, and detecting and identifying local driving postures of a driver, wherein the local driving postures comprise at least one of head postures, hand postures and trunk postures; and if at least one of the head posture, the hand posture and the trunk posture of the driver is recognized to be matched with the wrong posture of the corresponding part, outputting second adjustment guide information to the outside so as to enable the driver to adjust the posture of the corresponding part.
In an optional embodiment, if it is recognized that at least one of the head posture, the hand posture and the trunk posture of the driver matches the wrong posture of the corresponding part, the processor 65 is specifically configured to: aiming at the head posture, if the fact that the twisting angle of a driver relative to the standard head posture exceeds a set first head torsion angle or the angle of downward head lowering exceeds a set second head torsion angle is detected, a head posture adjustment guiding message is output outwards; for the hand gesture, if the hand gesture of the driver does not meet the set steering wheel using condition under the condition that the driver uses the steering wheel, or the hand gesture of the driver does not meet the set gear lever using condition under the condition that the driver uses the gear lever, or the hand gesture of the driver does not meet the set horn using condition under the condition that the driver uses the horn, or the hand gesture of the driver does not meet the set switch using condition under the condition that the driver uses the light or the windshield wiper adjusting switch, the hand gesture adjusting guide information is output; and for the trunk posture, if the fact that the trunk of the driver and the vehicle steering wheel do not meet the set third distance condition or the trunk posture of the driver does not meet the set inclination or rotation condition is detected, trunk posture adjustment guide information is output.
The vehicle driver driving behavior adjusting device provided by the embodiment of the application collects multiple images of a driver in the driving process of the driver, determines the whole driving posture information and/or the visual attention information of the driver according to the collected multiple images, determines the behavior state of the driver according to the whole driving posture information and/or the visual attention information of the driver, and outputs the adjustment guide information matched with the historical driving behavior state of the driver in time when the driver is in the unsafe driving behavior state so as to prompt and guide the driver to adjust the driving behavior of the driver, so that the accuracy and the effectiveness of the output adjustment guide information are improved, and the driving safety of the driver is guaranteed.
Further, as shown in fig. 6, the vehicle driver's driving behavior adjustment apparatus further includes: communication components 66, display 67, power components 68, audio components 69, and the like. Only some of the components are schematically shown in fig. 6, and it is not intended that the vehicle driver's driving behavior adjustment apparatus includes only the components shown in fig. 6. It should be noted that the components shown in the dashed line in fig. 6 are optional components, not necessary components, and may be determined according to the product form of the driving behavior adjustment device for the vehicle driver.
Accordingly, the present application also provides a computer readable storage medium storing a computer program, which when executed by a processor, causes the processor to implement the steps in fig. 1 provided by the present application.
The communication component of fig. 6 described above is configured to facilitate communication between the device in which the communication component is located and other devices in a wired or wireless manner. The device where the communication component is located can access a wireless network based on a communication standard, such as a WiFi, a 2G, 3G, 4G/LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
The display in fig. 6 described above includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
The power supply assembly of fig. 6 described above provides power to the various components of the device in which the power supply assembly is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.
The audio component of fig. 6 described above may be configured to output and/or input an audio signal. For example, the audio component includes a Microphone (MIC) configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (11)

1. A method for adjusting driving behavior of a driver of a vehicle, applied to a target vehicle, wherein at least one vision sensor is installed in the target vehicle, the method comprising:
acquiring a plurality of images of a driver in the driving process by utilizing the at least one vision sensor;
respectively identifying the integral driving posture information and/or the visual attention information of the driver according to the plurality of images;
determining the driving behavior state of the driver according to the overall driving posture information and/or the visual attention information;
when the driver is in a non-safe driving behavior state, outputting first adjustment guide information to prompt and guide the driver to adjust the driving behavior of the driver; the first adjustment guide information is adapted to the historical driving behavior state of the driver.
2. The method of claim 1, wherein identifying the overall driving posture information of the driver from the plurality of images comprises:
and recognizing a first actual distance between the trunk of the driver and a steering wheel, a second actual distance between the head of the driver and the top of the vehicle and an actual angle of a seat back according to the plurality of images, and taking the first actual distance, the second actual distance and the actual angle as the overall driving posture information of the driver.
3. The method of claim 2, wherein identifying visual attention information of the driver from the plurality of images comprises:
selecting a target image including a face area of the driver from the plurality of images;
identifying the facial and/or eye expression of the driver from the target image;
and determining the visual attention information of the driver according to the facial expression and/or the eye expression state.
4. The method of claim 3, wherein determining the driving behavior state of the driver from the overall driving posture information and/or the visual attention information comprises:
judging whether the integral driving posture information accords with a standard driving posture corresponding to safe driving or not, and judging whether the visual attention information accords with a standard attention condition corresponding to safe driving or not;
and if the overall driving posture information does not accord with the standard driving posture or the visual attention information does not accord with the standard attention condition, determining that the driver is in a non-safe driving behavior state, wherein the standard driving posture and the standard attention condition are adapted to the driver.
5. The method of claim 4, wherein the standard driving posture comprises: a first standard distance of a driver's torso from a steering wheel, a second standard distance of a driver's head from a vehicle roof, and a standard angle of a seat back;
judging whether the integral driving posture information accords with a standard driving posture corresponding to safe driving or not, and the judging method comprises the following steps:
if the first actual distance and the first standard distance do not meet a set first distance condition, or the second actual distance and the second standard distance do not meet a set second distance condition, or the actual angle and the standard angle do not meet a set angle condition, determining that the overall driving posture information does not meet the standard driving posture;
correspondingly, when the driver is in the unsafe driving behavior state, outputting first adjustment guiding information to prompt and guide the driver to adjust the driving behavior of the driver, wherein the method comprises the following steps:
if the first actual distance and the first standard distance do not meet the set first distance condition, trunk adjustment guiding information is output; if the second actual distance and the second standard distance do not meet the set second distance condition, outputting head adjustment guide information; and if the actual angle of the seat back and the standard angle do not meet the set angle condition, outputting seat back adjustment guide information.
6. The method of claim 4, wherein determining whether the visual attention information meets a standard attention condition for safe driving comprises:
if the eyes of the driver are not distracted and the duration exceeds a set first time threshold value, or the eyes of the driver are concentrated and the duration exceeds a set second time threshold value, determining that the visual attention information does not meet a standard attention condition;
correspondingly, when the driver is in the unsafe driving behavior state, outputting first adjustment guiding information to prompt and guide the driver to adjust the driving behavior of the driver, wherein the method comprises the following steps:
if the driver is detected to be distracted and the duration time of the distraction exceeds a set first time threshold, outputting guide information for reminding the driver to improve the attention; or if the attention of the driver is detected to be concentrated and the duration exceeds a set second time threshold, outputting guide information for reminding the driver to relax.
7. The method of claim 1, further comprising:
inputting the multiple images into a posture detection model, and detecting and identifying local driving postures of the driver, wherein the local driving postures comprise at least one of head postures, hand postures and trunk postures;
and if at least one of the head posture, the hand posture and the trunk posture of the driver is recognized to be matched with the wrong posture of the corresponding part, outputting second adjustment guide information to the outside so as to enable the driver to adjust the posture of the corresponding part.
8. The method of claim 7, wherein if it is recognized that at least one of the head posture, the hand posture and the trunk posture of the driver matches the wrong posture of the corresponding part, outputting second adjustment guidance information to the outside for the driver to adjust the posture of the corresponding part, comprising:
aiming at the head posture, if the fact that the twisting angle of the driver relative to the standard head posture exceeds a set first head torsion angle or the angle of downward head lowering exceeds a set second head torsion angle is detected, a head posture adjustment guide message is output outwards;
for the hand gesture, if the hand gesture of the driver does not meet the set steering wheel use condition under the condition that the driver uses the steering wheel, or the hand gesture of the driver does not meet the set gear lever use condition under the condition that the driver uses the gear lever, or the hand gesture of the driver does not meet the set horn use condition under the condition that the driver uses the horn, or the hand gesture of the driver does not meet the set switch use condition under the condition that the driver uses the light or the windshield wiper adjusting switch, then outputting hand gesture adjusting guide information;
and for the trunk posture, if the fact that the trunk of the driver and the vehicle steering wheel do not meet the set third distance condition or the trunk posture of the driver does not meet the set inclination or rotation condition is detected, trunk posture adjustment guide information is output.
9. A vehicle driver driving behavior adjustment device applied to a target vehicle, characterized in that at least one visual sensor is installed in the target vehicle, the vehicle driver driving behavior adjustment device comprising: the device comprises an acquisition module, an identification module, a determination module and an output module;
the acquisition module is used for acquiring a plurality of images of a driver in the driving process by utilizing the at least one vision sensor;
the recognition module is used for respectively recognizing the integral driving posture information and/or the visual attention information of the driver according to the images;
the determining module is used for determining the driving behavior state of the driver according to the integral driving posture information and/or the visual attention information;
the output module is used for outputting first adjustment guiding information when the driver is in a non-safe driving behavior state so as to prompt and guide the driver to adjust the driving behavior of the driver; the first adjustment guide information is adapted to the driver's historical driving behavior state.
10. A vehicle driver's driving behavior adjustment apparatus applied to a target vehicle, characterized in that at least one vision sensor is installed in the target vehicle, the apparatus comprising: a memory and a processor; the memory for storing a computer program; the processor, coupled to the memory, is configured to execute the computer program to implement the steps of the method of any of claims 1-8.
11. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 8.
CN202111672793.8A 2021-12-31 2021-12-31 Method, device and equipment for adjusting driving behavior of vehicle driver and storage medium Pending CN114701503A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111672793.8A CN114701503A (en) 2021-12-31 2021-12-31 Method, device and equipment for adjusting driving behavior of vehicle driver and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111672793.8A CN114701503A (en) 2021-12-31 2021-12-31 Method, device and equipment for adjusting driving behavior of vehicle driver and storage medium

Publications (1)

Publication Number Publication Date
CN114701503A true CN114701503A (en) 2022-07-05

Family

ID=82167563

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111672793.8A Pending CN114701503A (en) 2021-12-31 2021-12-31 Method, device and equipment for adjusting driving behavior of vehicle driver and storage medium

Country Status (1)

Country Link
CN (1) CN114701503A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115035502A (en) * 2022-07-08 2022-09-09 北京百度网讯科技有限公司 Driver behavior monitoring method and device, electronic equipment and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115035502A (en) * 2022-07-08 2022-09-09 北京百度网讯科技有限公司 Driver behavior monitoring method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
US11249544B2 (en) Methods and systems for using artificial intelligence to evaluate, correct, and monitor user attentiveness
US10908677B2 (en) Vehicle system for providing driver feedback in response to an occupant's emotion
CN110214107B (en) Autonomous vehicle providing driver education
US10525984B2 (en) Systems and methods for using an attention buffer to improve resource allocation management
US9558414B1 (en) Method for calculating a response time
CN105270292B (en) System and method for controlling access to human-machine interface in vehicle
JP6948559B2 (en) Driver monitoring device and driver monitoring method
US11386678B2 (en) Driver authentication for vehicle-sharing fleet
CN108725357A (en) Parameter control method, system based on recognition of face and cloud server
CN109664891A (en) Auxiliary driving method, device, equipment and storage medium
JP2015110417A (en) Driving assist device based on feeling of driver
CN103732480A (en) Method and device for assisting a driver in performing lateral guidance of a vehicle on a carriageway
US20160307056A1 (en) Arrangement for creating an image of a scene
JP6090340B2 (en) Driver emotion estimation device
US20180348751A1 (en) Partially Autonomous Vehicle Passenger Control in Difficult Scenario
CN112041201B (en) Method, system, and medium for controlling access to vehicle features
WO2014126304A1 (en) Vsdt rearview mirror having sleepiness prevention function
JPWO2020145161A1 (en) Information processing equipment, mobile devices, and methods, and programs
WO2021067380A1 (en) Methods and systems for using artificial intelligence to evaluate, correct, and monitor user attentiveness
CN112114671A (en) Human-vehicle interaction method and device based on human eye sight and storage medium
JP6834578B2 (en) Vehicle health examination device
CN114701503A (en) Method, device and equipment for adjusting driving behavior of vehicle driver and storage medium
JP6149842B2 (en) Driver emotion estimation apparatus and method
CN114987500A (en) Driver state monitoring method, terminal device and storage medium
US11511703B2 (en) Driver personalization for vehicle-sharing fleet

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination