CN115195770A - Remote driver wake-up method and device for automatically driving vehicle - Google Patents

Remote driver wake-up method and device for automatically driving vehicle Download PDF

Info

Publication number
CN115195770A
CN115195770A CN202210707304.6A CN202210707304A CN115195770A CN 115195770 A CN115195770 A CN 115195770A CN 202210707304 A CN202210707304 A CN 202210707304A CN 115195770 A CN115195770 A CN 115195770A
Authority
CN
China
Prior art keywords
driver
wake
control instruction
vehicle
execution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210707304.6A
Other languages
Chinese (zh)
Inventor
何特
郑聪海
王英东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Network Technology Shanghai Co Ltd
Original Assignee
International Network Technology Shanghai Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Network Technology Shanghai Co Ltd filed Critical International Network Technology Shanghai Co Ltd
Priority to CN202210707304.6A priority Critical patent/CN115195770A/en
Publication of CN115195770A publication Critical patent/CN115195770A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W50/16—Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/005—Handover processes
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0872—Driver physiology
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/143—Alarm means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146—Display means

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention provides a remote driver wake-up method and a remote driver wake-up device for an automatic driving vehicle, wherein the method comprises the following steps: acquiring heart rate data of a driver, dynamic video of the driver sent by a vehicle end and fatigue state information of the driver identified by a driver monitoring system; identifying and obtaining the sleeping state of the driver based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver; and under the condition that the driver is in a sleep state in the sleeping state, generating a waking control instruction based on the waking configuration of the operator, and sending the waking control instruction to the vehicle end to wake the driver. The invention can improve the automatic driving safety and improve the interaction experience of the driver.

Description

Remote driver wake-up method and device for automatically driving vehicle
Technical Field
The invention relates to the technical field of automatic driving, in particular to a remote driver wake-up method and device for an automatic driving vehicle.
Background
With the gradual popularization of automobile automatic driving technology, the automatic driving automobiles of L2+ and L3 levels enter the stage of mass production application. However, due to the existence of a plurality of long-tail scenes, in recent years and in the future, the role of a driver or a safety driver (hereinafter, referred to as a driver) is still required to be preserved for a long time so as to ensure that the driver can take over the vehicle to ensure the driving safety when a scene or a situation which cannot be covered by an automatic driving system or an auxiliary automatic driving system of the vehicle occurs.
However, as the autodrive technology matures, drivers may increasingly build confidence in the autodrive capabilities. In this case, particularly when the vehicle is operated in the automatic driving mode for a long time in a high-speed scene or the like, the driver is likely to fall out of the driving state and sleep due to fatigue or the like. Although this situation is clearly not allowed for L2+ and L3-level autonomous vehicles, it is still difficult to avoid in practice. Particularly, for a heavy truck for an automatic driver operating on a highway for a long time, it is found that the phenomenon that the driver is easy to fall off the driving state and sleep due to fatigue and the like is particularly obvious, and the driving safety of the automatic driver is greatly influenced.
Disclosure of Invention
The invention provides a method and a device for remotely waking up a driver of an automatic driving vehicle, which are used for solving the problems.
The invention provides a driver remote wake-up method of an automatic driving vehicle, which is applied to a cloud and comprises the following steps:
acquiring heart rate data of a driver, dynamic video of the driver sent by a vehicle end and fatigue state information of the driver identified by a driver monitoring system;
identifying and obtaining the sleeping state of the driver based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver;
and under the condition that the driver is in the sleep state, generating a wake-up control instruction based on the wake-up configuration of the operator, and sending the wake-up control instruction to the vehicle end to wake up the driver.
According to the invention, the method for remotely waking up the driver of the automatic driving vehicle, which is based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver, comprises the following steps:
determining the current sleep state of the driver according to the heart rate data of the driver and the fatigue state information of the driver and by utilizing the prestored heart rate of the driver and the corresponding relation between the fatigue state and the sleep state; the corresponding relation is obtained by carrying out data analysis on historical heart rate data of the driver and historical fatigue state information of the driver;
the method comprises the steps that a dynamic video of a driver is displayed under the condition that the current driver is in sleep for the driver, so that an operator is reminded of confirming the sleep state of the current driver, and after a confirmation instruction of the operator is received, a wakeup control instruction is generated based on wakeup configuration of the operator and is sent to a vehicle end to wake the driver.
According to the remote wake-up method for the driver of the automatic driving vehicle, provided by the invention, the wake-up configuration comprises a wake-up mode, a wake-up intensity and a wake-up duration; the waking modes comprise acoustic waking, optical waking, tactile waking and text waking;
the wake-up configuration is formed by:
setting the wake-up time;
determining at least one mode to be executed from the acoustic wakeup, the optical wakeup, the tactile wakeup and the text wakeup;
setting the wake-up intensity for each mode to be executed;
correspondingly, the wake-up control instruction is generated based on the wake-up configuration of the operator, and comprises the following steps:
and generating a wakeup control instruction based on the set wakeup time length, at least one to-be-executed mode and the corresponding wakeup strength.
According to the invention, after the driver is sent to the vehicle end to wake up the driver, the method for remotely waking up the driver of the automatic driving vehicle further comprises the following steps:
when receiving execution feedback generated by the vehicle end according to the received wake-up control instruction within preset feedback time, generating vehicle end execution wake-up reminder for reminding operators of the progress of the wake-up of the vehicle end;
and when the execution feedback generated by the vehicle end according to the received wake-up control instruction is not received within the preset feedback time, generating a network connection overtime prompt for prompting the operator that the wake-up task is not completed.
According to the invention, before the driver is sent to the vehicle end to wake up the driver, the method for remotely waking up the driver of the automatic driving vehicle further comprises the following steps:
judging whether the communication response delay between the cloud end and the vehicle end meets a preset threshold value or not;
under the condition that the communication response delay meets a preset threshold value, sending the wake-up control instruction to a vehicle end;
and under the condition that the communication response delay does not accord with a preset threshold value, the wake-up control instruction is not sent to a vehicle end, and the wake-up reminding is stopped.
The invention also provides a driver remote wake-up method of the automatic driving vehicle, which is applied to a vehicle end and comprises the following steps:
acquiring a dynamic video of a driver and fatigue state information of the driver, which is identified by a driver monitoring system, and sending the information to a cloud end;
receiving a wake-up control instruction generated by the cloud according to the sleeping state of a driver, and executing the wake-up control instruction; the sleep state of the driver is obtained by recognition based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver.
According to the invention, the method for remotely waking up the driver of the automatic driving vehicle comprises the following steps before executing the wake-up control command:
judging whether to execute a wake-up control instruction or not;
under the condition that the execution of the wake-up control instruction is judged, generating execution feedback for executing the wake-up control instruction according to the wake-up control instruction, executing the wake-up control instruction, and sending the execution feedback to a cloud end;
and under the condition that the wake-up control instruction is not executed, generating execution feedback of the non-execution wake-up control instruction and the reason thereof according to the wake-up control instruction, and sending the execution feedback to the cloud.
According to the invention, the method for remotely waking up the driver of the automatic driving vehicle, which executes the wake-up control command, comprises the following steps:
when an event with priority higher than that of the wake-up control instruction occurs in the execution process, interrupting execution of the wake-up control instruction, generating an execution process interruption feedback, and sending the execution process interruption feedback to the cloud end; wherein the event with priority higher than the wake-up control instruction is one of activating an autonomous driving mode, exiting an autonomous driving mode, or driving task support;
when an event with the priority not higher than that of the wake-up control instruction occurs or no event occurs in the execution process, the wake-up control instruction is executed, and after the execution is completed, execution completion feedback is generated and sent to the cloud end; wherein the event with the priority not higher than the wake-up control instruction is an automatic driving behavior prompt in an automatic driving mode.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the processor executes the program to implement any one of the above-mentioned methods for remotely waking up a driver of an autonomous vehicle.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements any of the above-described methods of remotely waking up a driver of an autonomous vehicle.
According to the remote driver awakening method and device for the automatic driving vehicle, the sleep state of the driver is comprehensively judged according to the information superposition of three angles of fatigue state identification of the driver, the heart rate of the driver and the dynamic video image of the driver by the DMS, the sleep state of the driver is accurately identified, and on the basis, awakening service is triggered, so that the safety of automatic driving is improved, and meanwhile the interactive experience of the driver can be improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a method for remotely waking up a driver of an autonomous vehicle in a cloud according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a method for remotely waking up a driver of an autonomous vehicle at a vehicle end according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a driver remote wake-up apparatus of an autonomous vehicle applied in a cloud according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a remote driver wake-up device for an autonomous vehicle applied to a vehicle end according to an embodiment of the present invention;
fig. 5 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flowchart of a method for remotely waking up a driver of an autonomous vehicle in a cloud according to an embodiment of the present invention; as shown in fig. 1, a method for remotely waking up a driver of an autonomous vehicle applied in a cloud includes the following steps:
s101, acquiring heart rate data of a driver, dynamic video of the driver sent by a vehicle end and fatigue state information of the driver identified by a driver monitoring system.
In this step, driver's rhythm of the heart data are the rhythm of the heart data of autopilot process, and it is gathered through equipment such as the intelligent bracelet that the driver wore and obtains, through the communication connection between intelligent bracelet and the high in the clouds, synchronizes in real time to the high in the clouds.
The dynamic driver video sent by the vehicle end is a real-time driver state video acquired by a vehicle-mounted TBOX (English full name Telematics BOX), and is sent to the cloud end in a low-delay manner through communication connection between the vehicle end and the cloud end.
The Driver fatigue state information recognized by the Driver Monitoring System is information automatically recognized by a Driver Monitoring System (DMS) provided in the automatic driving vehicle according to the state of the Driver in the cabin. Usually, the DMS identifies and determines whether the driver is tired according to the eye-closed state of the driver, and specifically, the DMS classifies the driver as mild, moderate, and severe tiredness according to the eye-closed duration, duty ratio, and the like, and uses the output fatigue level as driver fatigue state information. The vehicle end can send the fatigue information of the driver to the cloud end in real time through the communication connection between the vehicle end and the cloud end.
And S102, identifying and obtaining the sleeping state of the driver based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver.
In the step, the cloud determines whether the driver is in the sleep state at present according to the corresponding relation between the heart rate data and the fatigue state and whether the driver enters the sleep state, the current heart rate data of the driver and the current fatigue state information of the driver, and further confirms the driver in the sleep state through the dynamic video of the driver under the condition that the driver is in the sleep state, and finally obtains the result that the driver is in the sleep state. And if the current data and the corresponding relation are utilized to obtain no conclusion of being in the sleep state or the conclusion that the current data and the corresponding relation are not in the sleep state after the video confirmation is carried out, the subsequent wake-up task is not executed.
It should be noted that the correspondence between the heart rate data and the fatigue state and whether the driver is asleep is obtained by analyzing the historical heart rate data of the driver, the historical fatigue state analysis, and the corresponding sleep state. For example, the driver is in a sleep state when his heart rate is what is obtained by historical heart rate data analysis. In addition, due to individual differences, the corresponding relationships of different drivers are slightly different, for example, a driver is in a sleep state when the heart rate is a times/minute, and B driver is in a sleep state when the heart rate is B times/minute.
S103, under the condition that the driver is in a sleep state, a wake-up control instruction is generated based on the wake-up configuration of the operator and is sent to the vehicle end to wake up the driver.
In this step, under the condition that it is determined that the driver is in sleep, an operator of the cloud-end automatic driving vehicle management system starts a remote wake-up service, specifically, a wake-up control instruction is generated according to interactive contents designed by the operator for the wake-up service and sent to the vehicle end, and the vehicle-end automatic driving system executes the wake-up control instruction, so that interaction with the driver is realized, and a wake-up task is completed.
According to the remote driver awakening method for the automatic driving vehicle, provided by the embodiment of the invention, the sleep state of the driver is comprehensively judged according to the information superposition of three angles of fatigue state identification, the heart rate of the driver and a dynamic video image of the driver by the DMS, the sleep state of the driver is accurately identified, and on the basis, the awakening service is triggered, the automatic driving safety is improved, and meanwhile, the interactive experience of the driver can be improved.
Further, the identifying the sleep state of the driver based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver comprises:
determining the current sleep state of the driver according to the heart rate data of the driver and the fatigue state information of the driver and by utilizing the prestored heart rate of the driver and the corresponding relation between the fatigue state and the sleep state; and the corresponding relation is obtained by carrying out data analysis on the heart rate data of the historical driver and the fatigue state information of the historical driver.
In this embodiment, after the historical heart rate data of the driver and the historical fatigue state information of the driver are collected, the matching corresponding relationship between the actual sleep state of the driver and the fatigue state of the DMS, the fatigue duration of the DMS, the heart rate of the driver, and the heart rate change rate of the driver is found out by a big data screening analysis method, and the matching corresponding relationship is correspondingly stored in the driver. And after obtaining the current heart rate data and the current fatigue state information of the driver, obtaining the current sleep state of the driver based on the pre-stored matching corresponding relation, and finishing the identification.
The method comprises the steps that a dynamic video of a driver is displayed under the condition that the current driver is in sleep for the driver, so that an operator is reminded of confirming the sleep state of the current driver, and after a confirmation instruction of the operator is received, a wakeup control instruction is generated based on wakeup configuration of the operator and is sent to a vehicle end to wake the driver.
Specifically, under the condition that the current driver sleep state is that the driver is in sleep, the cloud can automatically pop up the real-time video of the driver, and after the operator confirms that the wake-up service needs to be provided, the remote start wake-up service stage can be started.
According to the remote driver wake-up method for the automatic driving vehicle, provided by the embodiment of the invention, whether the driver is in the sleep state at present is determined by combining the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver, so that the accuracy of sleep state identification is improved.
Further, the wake-up configuration comprises a wake-up mode, a wake-up intensity and a wake-up duration; the wake-up mode comprises acoustic wake-up, optical wake-up, tactile wake-up and text wake-up.
For example, the alarm sound and voice of acoustic wake-up, the atmosphere lamp, steering wheel alarm lamp and instrument panel alarm lamp of optical wake-up, the vibration seat and pre-tightening safety belt of tactile wake-up, the instrument text reminder of text wake-up, the large screen text reminder of central control, etc. The multiple wake-up modes are selected for combination, multi-angle human-computer interaction is achieved, and the fact that a driver can be awake under the wake-up service is guaranteed.
Illustratively, the wake-up configuration is as follows:
1) The continuous flashing time of the atmosphere lamp/the alarm lamp is long, and correspondingly, the flashing frequency of the atmosphere lamp/the alarm lamp has three modes of low/medium/high;
2) The continuous vibration duration of the vibration seat is long, and correspondingly, the vibration frequency of the vibration seat has three modes of low/medium/high;
3) The tightening times of the pre-tightening type safety belt are correspondingly low/medium/high;
4) The voice reporting times of the voice prompt correspond to the volume of the voice prompt in a low mode, a medium mode and a high mode;
5) The duration of the text prompt display.
The wake-up configuration is formed by: setting the wake-up time; determining at least one mode to be executed from the acoustic wake-up, the optical wake-up, the haptic wake-up and the text wake-up; the wake-up strength is set for each mode to be executed.
Taking the above wake-up configuration as an example, the duration of 1) 2) 5) is set to be T (generally 3 to 20 s). 3) And 4) calculating the times according to the set duration T, the single tightening and releasing duration and the single voice broadcasting duration. Specifically, 3) the single tightening release duration is fixed as T 1 (generally 3-5 s), and automatically calculating to obtain the tightening times of the safety belt, wherein the tightening times are approximately equal to T/T 1 Next (generally ≧ 1). 4) The time length of single voice broadcast is also fixed as T 2 (generally 3-8 s), automatically calculating to obtain the voice broadcast times ≈ T/T 2 Next (generally ≧ 1).
After the setting of the wake-up time length is completed, the wake-up mode is selected and each wake-up party is provided withFormula sets the wake-up intensity. For example, on the basis of the wake-up configuration content, 1) -5) related content are selected completely, if the reminding time length is set to t 1 Then the configuration generates the following wake-up control instructions:
1) Duration of flashing of atmosphere lamp/warning lamp t 1 The flicker frequency is middle;
2) Duration t of continuous vibration of vibration seat 1 The vibration frequency is middle;
3) Duration t of continuous display of text prompt 1 ；
4) The tightening frequency of the pre-tightening safety belt is approximately equal to t 1 /T 1 The tightening force is high;
5) The voice broadcasting times of voice prompt is approximately equal to t 1 /T 2 The volume is medium.
In another scenario, 1) -5) related contents can be selectively selected, if the reminding duration is set to t 2 Then the configuration generates the following wake-up control instructions:
1) The continuous flashing time of the atmosphere lamp/alarm lamp is 0, namely the atmosphere lamp/alarm lamp is not selected to be executed;
2) Duration t of continuous vibration of vibration seat 2 The vibration frequency is middle;
3) Duration t of continuous display of text prompt 2 ；
4) The tightening times of the pre-tightening type safety belt are 0, namely the pre-tightening type safety belt is not selected to be executed;
5) The voice broadcasting times of voice prompt is approximately equal to t 2 /T 2 The volume is medium.
Accordingly, the wake-up control instruction is generated based on the wake-up configuration of the operator, and comprises the following steps: and generating a wakeup control instruction based on the set wakeup time length, at least one to-be-executed mode and the corresponding wakeup strength.
Besides the execution time, the breathing flicker frequency of the corresponding light, the vibration frequency of the vibration frequency, the tightening force of the pre-tightening safety belt and the volume of the voice reminding can be remotely configured to provide the waking service with different intensity levels.
In the practical application process, in order to improve the implementation efficiency of the wake-up service, the execution modes and the wake-up intensity of 1) -5) are configured in advance to form a set of default wake-up control instructions, and the default wake-up control instructions can be selected and set again to form new wake-up control instructions when the operator needs to reconfigure the instructions.
In addition, the same ID number is issued for multiple times for the same wake-up service to prevent packet loss, and the vehicle end can identify whether the received signal is the same wake-up service through the ID number, and the wake-up service cannot be triggered repeatedly under the condition of the same ID number.
When the devices such as the atmosphere lamp/alarm lamp, the vibration seat, the text reminding, the pre-tightening safety belt, the voice reminding and the like have abnormal working conditions, the vehicle end can also upload corresponding device diagnosis to the background.
When a wake-up control instruction is generated at the cloud, based on the equipment diagnosis information, a single reminding mode corresponding to the equipment with a fault is displayed as an unconfigurable state (such as an grey option), and when a wake-up service is issued, a remote assistant is reminded that the single reminding mode cannot be executed.
The embodiment of the invention provides a remote driver waking method for an automatic driving vehicle, which realizes the waking of a driver from multiple angles of acoustic waking, optical waking, tactile waking and text waking.
Further, after the sending to the vehicle end to wake up the driver, the method further comprises:
and when receiving execution feedback generated by the vehicle end according to the received wake-up control instruction within preset feedback time, generating vehicle end execution wake-up reminder for reminding operators of the progress of the wake-up of the vehicle end.
In this embodiment, after the cloud issues the wake-up control command, if the cloud has a preset feedback time T timeout And if the feedback that the vehicle end has received and will execute the wake-up control instruction is received, the cloud end can remind the background remote assistant that the wake-up function is being executed.
After the cloud terminal issues the wake-up control instruction, if the time is T timeout Internally received by vehicle end but not executing wake-up controlAnd the cloud end can give a prompt of XXX to the background remote assistant for 'not executing the wake-up function'.
And when the execution feedback generated by the vehicle end according to the received wake-up control instruction is not received within the preset feedback time, generating a network connection overtime prompt for prompting the operator that the wake-up task is not completed.
In this embodiment, if the cloud issues the wake-up control command, the time T is timeout If the feedback that the vehicle end has received the wake-up control instruction is not received within the range, the cloud end can give a prompt of 'network connection overtime' to the background remote assistant.
It should be noted that, the time T after the cloud issues the command timeout Within and without receiving the circumstances of feedback, the remote wake-up service can not trigger the remote wake-up again (if trigger the button realization through grey operation platform) to the control that the high in the clouds was to the car end is issued and is piled up and queue up.
However, the cloud is at time T timeout After receiving the feedback of the vehicle end, the remote assistant can trigger the wake-up function again and issue an instruction. And the vehicle end receives the remote wake-up instruction again during the wake-up execution period, interrupts the wake-up function at the current time and immediately executes the next wake-up function.
According to the remote wakening method for the driver of the automatic driving vehicle, after the wakening control instruction is sent by the cloud, whether the wakening service is normally executed or not is determined according to information such as feedback of the vehicle end and abnormal reasons of the feedback, and whether the wakening control instruction needs to be sent again or not is determined according to the situation under the condition that the wakening service is not normally executed, so that the interactive experience of the driver is improved, and the driver is not repeatedly wakened all the time.
Further, before the sending to the vehicle end to wake up the driver, the method further comprises:
judging whether the communication response delay between the cloud end and the vehicle end meets a preset threshold value or not; under the condition that the communication response delay meets a preset threshold value, sending the wake-up control instruction to a vehicle end; and under the condition that the communication response delay does not accord with a preset threshold value, the wake-up control instruction is not sent to a vehicle end, and the wake-up reminding is stopped.
In this embodiment, the cloud end can not issue the control command of remote wakeup when detecting that the network corresponding to the vehicle end is lost, abnormal or the communication service is delayed excessively, and in addition, can remind the remote assistant in the backstage that the current network connection state is not good to reduce the issue of the remote wakeup command with excessive delay, cause to remind that the effect of waking up with excessive delay is not good.
Based on the above reasons, the vehicle end is waken at the high in the clouds and control the order and issue to and the vehicle end is to whether receiving wakening control order or when being to the feedback of wakening the progress of task execution, all have certain demand to the communication performance of high in the clouds and vehicle end, only with communication response delay control in certain extent, this functional design just makes sense. Otherwise, due to too large delay, the driver cannot be awakened effectively, adverse effects are brought, discontent of the driver is caused, and poor user experience is caused.
For example, the cloud sends a wake-up control instruction to the vehicle end, and the communication performance needs to meet the following requirements under the condition that the network connection is not interrupted: 1) More than 90% of the down signal delay is less than or equal to 1s, 2) more than 99% of the down signal delay is less than or equal to 5s.
Under the condition that the vehicle end receives and executes the wake-up control instruction and feeds back the wake-up control instruction to the cloud end and the network connection is not interrupted, the communication performance needs to meet the following requirements: 1) More than 90% of the down signal delay is less than or equal to 2s, 2) more than 99% of the down signal delay is less than or equal to 10s.
It should be noted that, the delay threshold corresponding to the communication performance requirement may be adjusted according to an actual situation.
According to the remote driver wake-up method for the automatic driving vehicle, whether the wake-up service is continuously executed or not is determined by considering the communication response delay, so that the user experience is improved.
FIG. 2 is a driver remote wake method for an autonomous vehicle at the vehicle end;
as shown in fig. 2, the method for remotely waking up a driver of an autonomous vehicle applied to a vehicle end includes:
s201, obtaining a driver dynamic video and driver fatigue state information identified by a driver monitoring system, and sending the driver fatigue state information to a cloud.
In the step, the vehicle end acquires a real-time video of the state of the driver through the vehicle-mounted TBOX, and the fatigue state information of the driver is obtained through automatic identification of a driver monitoring system DMS according to the state of the driver in the cabin.
Usually, the DMS identifies and determines whether the driver is tired according to the state of the closed eye of the driver, and specifically, the driver is classified into mild, moderate, and severe tiredness according to the duration and proportion of the closed eye. The vehicle end can send the fatigue information of the driver and the real-time video of the state of the driver to the cloud end through the communication connection between the vehicle end and the cloud end.
And S202, receiving a wake-up control instruction generated by the cloud according to the sleeping state of the driver, and executing the wake-up control instruction.
The sleep state of the driver is obtained by recognition based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver.
In this step, the sleep state of the driver is determined by the cloud according to the heart rate data, the fatigue state and the correspondence between whether the driver is sleeping or not, and the current heart rate data of the driver and the current fatigue state information of the driver. And under the condition of the sleeping state, further confirming through the driver dynamic video, and finally obtaining the result that the driver is in the sleeping state. If the current data and the corresponding relation are utilized to obtain the conclusion that the user is not in the sleep state or the user is not in the sleep state after the user is confirmed through the video, the subsequent wake-up task is not executed.
It should be noted that the correspondence between the heart rate data and the fatigue state and whether the driver is asleep is obtained by analyzing the historical heart rate data of the driver, the historical fatigue state analysis, and the corresponding sleep state.
And after the vehicle end receives the wake-up control instruction, the vehicle end executes the wake-up control instruction to realize the man-machine interaction with a driver.
According to the remote driver awakening method for the automatic driving vehicle, provided by the embodiment of the invention, the sleep state of the driver is comprehensively judged according to the information superposition of three angles of fatigue state identification, the heart rate of the driver and a dynamic video image of the driver by the DMS, the sleep state of the driver is accurately identified, and on the basis, the awakening service is triggered, the automatic driving safety is improved, and meanwhile, the interactive experience of the driver can be improved.
Further, prior to said executing said wake-up control instructions, the method comprises: judging whether to execute a wake-up control instruction or not; under the condition that the execution of the wake-up control instruction is judged, generating execution feedback for executing the wake-up control instruction according to the wake-up control instruction, executing the wake-up control instruction, and sending the execution feedback to a cloud end; and under the condition that the wake-up control instruction is not executed, generating execution feedback of the non-execution wake-up control instruction and the reason thereof according to the wake-up control instruction, and sending the execution feedback to the cloud.
In this embodiment, after receiving the feedback of the completion of the remote wake-up execution, the cloud reminds the remote assistant of the completion of the wake-up service by using the content of "the completion of the wake-up service execution with the delivery time xx: xx: xx".
When the cloud receives the feedback of the remote wake-up execution interruption, the cloud can use the wake-up service execution interruption of the issuing time xx: xx: xx, because the content of XXX reminds the remote assistant, and the wake-up service is interrupted. Interrupt causes include the triggering of the next wake instruction and interruption by a higher priority event.
It should be noted that, the remote assistant in the cloud may determine whether to issue the instruction again according to the circumstances after the instruction is issued last time and executed. Or issuing the instruction again to interrupt the previous wake-up service and execute the next wake-up service before the previous issued instruction is not fed back and the execution is completed.
The embodiment of the invention provides a remote driver wake-up method for an automatic driving vehicle, which can feed back the execution of a wake-up control instruction of the whole vehicle end and is convenient for operators to know the execution condition of the vehicle end.
Further, the executing the wake-up control instruction includes:
when an event with the priority higher than that of the wake-up control instruction occurs in the execution process, interrupting the execution of the wake-up control instruction, generating an execution process interruption feedback, and sending the execution process interruption feedback to the cloud; wherein the event with priority higher than the wake-up control instruction is one of activating an autonomous driving mode, exiting an autonomous driving mode, or driving task support;
when an event with the priority not higher than that of the wake-up control instruction occurs or no event occurs in the execution process, the wake-up control instruction is executed, and after the execution is completed, execution completion feedback is generated and sent to the cloud end; wherein the event with the priority not higher than the wake-up control instruction is an automatic driving behavior prompt in an automatic driving mode.
In this embodiment, the remote wakeup is one of the reminders of HMI interaction, which has priority present. Generally, the priority is as follows: activating an autonomous driving mode (i.e., activating the AD mode), exiting the autonomous driving mode (i.e., exiting the AD) mode, and driving task support (i.e., fallback) > remote wake > alert of autonomous driving behavior common within the AD mode.
The activation of the AD mode and the exit of the AD mode represent driver intervention, and when the driver intervenes, the vehicle end has strong HMI reminding, namely reminding with higher priority.
The driving task support (namely Fallback) means that a fault or a scene which cannot be processed by the system occurs, and the driver is requested to intervene, if the driver is not immediately involved, danger may exist, and at the moment, the vehicle end can also have stronger HMI reminding, namely reminding with higher priority.
These three events are accompanied by switching of driving modes or special driving behaviors, and it is necessary for the driver to know the first time after waking up from sleep and provide countermeasures, so that the three events are preferably reminded.
For the above reasons, if the vehicle-side is in the AD mode activation or the AD mode exit or the Fallback execution, the vehicle-side system does not perform the remote wake-up function. If an AD mode activation or an AD mode exit or a Fallback occurs while the remote wake-up function is performed, the remote wake-up function is also immediately interrupted.
In addition, if the driver has entered a sleep state, the automatic driving behavior reminder in the usual AD mode is lower priority than the remote wake-up function. Wherein, the common reminding comprises lane changing reminding, avoidance reminding, cone bucket reminding, solid line reminding and the like. The reason is that such HMI reminders are generally less intense and can be learned after waiting for the driver to be gradually awakened and adapted to the vehicle, and often there is no strong demand for the driver to respond for the first time, so these reminders are not high in priority of the driver wake-up function. Based on the above, if the AD mode is provided with the conventional automatic driving behavior reminding in the AD mode, the system is not influenced to execute the remote wake-up function.
After the description of the priority is carried out, under the normal condition of the network, the remote wake-up control instruction of the cloud end is normally issued to the vehicle end, and after the vehicle end normally receives the wake-up control instruction, if no event with higher priority is being executed, the received execution feedback is provided to the cloud end, and the wake-up function is executed.
If the vehicle end finds that a higher-quality event is being executed after normally receiving the wake-up requirement, the received but not executed feedback is provided to the cloud end, and the wake-up is not executed.
In addition, after the vehicle end normally receives the wake-up control instruction and starts to execute, if the wake-up function is completely executed, the executed feedback is provided to the cloud end; if the wake-up function is interrupted by a higher priority event, which causes an execution interrupt, the execution interrupt and the feedback of the interrupt reason are provided to the cloud.
It should be noted that the remote wake-up function is mainly applied to the AD mode, that is, in the case that the automatic driving mode is activated, the operation background can be implemented to wake up the driver. But may be used in a manual driving mode (i.e., MD mode) as needed to assist the driver in refreshing mind in the manual driving mode.
In addition, although the problem that the driver of the commercial vehicle enters an unexpected sleep state is obvious, the system is also applied to other vehicles, such as passenger vehicles, and exists as a functional point of an automatic driving or auxiliary automatic driving system for users to select.
The following describes the remote driver wake-up device for the cloud-based autonomous vehicle according to the present invention, and the remote driver wake-up device for the cloud-based autonomous vehicle described below and the remote driver wake-up method for the cloud-based autonomous vehicle described above may be referred to in correspondence.
Fig. 3 is a schematic structural diagram of a remote driver wake-up device of an autonomous vehicle applied to a cloud end according to an embodiment of the present invention, and as shown in fig. 3, the remote driver wake-up device of an autonomous vehicle applied to a cloud end includes:
the data acquisition module 301 is configured to acquire heart rate data of a driver, a dynamic video of the driver sent by a vehicle end, and driver fatigue state information identified by a driver monitoring system.
In this module, driver's rhythm of the heart data are the rhythm of the heart data of autopilot process, and it is gathered through equipment such as the intelligent bracelet that the driver wore, through the communication connection between intelligent bracelet and the high in the clouds, in real time synchronization to high in the clouds.
The dynamic driver video sent by the vehicle end is a real-time driver state video acquired by the vehicle-mounted TBOX, and is sent to the cloud end in a low-delay mode through communication connection between the vehicle end and the cloud end.
The driver fatigue state information recognized by the driver monitoring system refers to information that is automatically recognized by a driver monitoring system DMS equipped in the automatic driving vehicle according to the state of the driver in the cabin. Usually, the DMS identifies and determines whether the driver is tired according to the state of the closed eye of the driver, and specifically, the driver is classified into mild, moderate, and severe tiredness according to the duration and proportion of the closed eye. The vehicle end can be through with the high in the clouds communication connection between with driver's fatigue information real-time transmission to high in the clouds.
A sleep state identification module 302, configured to identify a sleep state of the driver based on the heart rate data of the driver, the dynamic video of the driver, and the fatigue state information of the driver.
In this module, the high in the clouds confirms whether the driver is in the sleep state at present according to the corresponding relation between heart rate data, fatigue state and whether getting into sleep to and current driver heart rate data, current driver fatigue state information, and under the condition that is in the sleep state, rethread driver dynamic video further confirms, obtains the result that the driver is in the sleep state at last. And if the current data and the corresponding relation are utilized to obtain no conclusion of being in the sleep state or the conclusion that the current data and the corresponding relation are not in the sleep state after the video confirmation is carried out, the subsequent wake-up task is not executed.
It should be noted that the correspondence between the heart rate data and the fatigue state and whether the driver falls asleep or not is obtained by analyzing the historical heart rate data and the historical fatigue state of the driver and the corresponding sleep state. For example, the driver is in a sleep state when his heart rate is what is obtained by historical heart rate data analysis. In addition, due to individual differences, the corresponding relationships of different drivers are slightly different, for example, a driver is in a sleep state when the heart rate is a times/minute, and B driver is in a sleep state when the heart rate is B times/minute.
And the waking control instruction generating module 303 is configured to generate a waking control instruction based on the waking configuration of the operator when the driver is in the sleep state, and send the waking control instruction to the vehicle end to wake the driver.
In this module, confirm that the driver is in the condition of sleeping, high in the clouds automatic driving vehicle management system's operation personnel can set out remote wake-up service, specifically according to operation personnel to this wake-up service design interactive content generate wake-up control instruction and send to the car end, car end automatic driving system carries out this wake-up control instruction, realizes the interaction between and the driver, accomplishes this task of waking up.
According to the remote driver awakening device for the automatic driving vehicle, provided by the embodiment of the invention, the sleep state of the driver is comprehensively judged according to the information superposition of three angles of fatigue state identification, the heart rate of the driver and a dynamic video image of the driver by the DMS, the sleep state of the driver is accurately identified, and on the basis, the awakening service is triggered, so that the automatic driving safety is improved, and meanwhile, the interactive experience of the driver can be improved.
The following describes the driver remote wake-up apparatus of an autonomous vehicle applied to a vehicle end according to the present invention, and the driver remote wake-up apparatus of an autonomous vehicle applied to a vehicle end described below and the driver remote wake-up method of an autonomous vehicle applied to a vehicle end described above may be referred to in correspondence with each other.
Fig. 4 is a schematic structural diagram of a driver remote wake-up device of an autonomous vehicle applied to a vehicle end according to an embodiment of the present invention, and as shown in fig. 4, a driver remote wake-up device of an autonomous vehicle applied to a vehicle end includes:
the data sending module 401 is configured to obtain a driver dynamic video and driver fatigue state information identified by the driver monitoring system, and send the driver fatigue state information to the cloud.
In the module, a vehicle end acquires a real-time video of the state of a driver through a vehicle-mounted TBOX, and the fatigue state information of the driver is obtained through automatic identification of a driver monitoring system DMS according to the state of the driver in a cabin.
Usually, the DMS identifies and determines whether the driver is tired according to the state of the closed eye of the driver, and specifically, the driver is classified into mild, moderate, and severe tiredness according to the duration and proportion of the closed eye. The vehicle end can send the fatigue information of the driver and the real-time video of the state of the driver to the cloud end through the communication connection between the vehicle end and the cloud end.
And the wakeup execution module 402 is configured to receive a wakeup control instruction generated by the cloud according to the sleep state of the driver, and execute the wakeup control instruction.
The sleep state of the driver is obtained by recognition based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver.
In this module, the driver sleep state is determined by the cloud according to the heart rate data, the fatigue state and the correspondence between whether to go to sleep or not, and the current driver heart rate data and the current driver fatigue state information. And under the condition of the sleeping state, further confirming through the driver dynamic video, and finally obtaining the result that the driver is in the sleeping state. If the current data and the corresponding relation are utilized to obtain the conclusion that the user is not in the sleep state or the user is not in the sleep state after the user is confirmed through the video, the subsequent wake-up task is not executed.
It should be noted that the correspondence between the heart rate data and the fatigue state and whether the driver is asleep is obtained by analyzing the historical heart rate data of the driver, the historical fatigue state analysis, and the corresponding sleep state.
And after the vehicle end receives the wake-up control instruction, the vehicle end executes the wake-up control instruction to realize the man-machine interaction with a driver.
According to the remote driver awakening device for the automatic driving vehicle, provided by the embodiment of the invention, the sleep state of the driver is comprehensively judged according to the information superposition of three angles of fatigue state identification, the heart rate of the driver and a dynamic video image of the driver by the DMS, the sleep state of the driver is accurately identified, and on the basis, the awakening service is triggered, so that the automatic driving safety is improved, and meanwhile, the interactive experience of the driver can be improved.
Fig. 5 is a schematic entity structure diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 5, the electronic device may include: a processor (processor) 510, a communication Interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication Interface 520 and the memory 530 communicate with each other via the communication bus 540. Processor 510 may invoke logic instructions in memory 530 to perform a driver remote wake-up method for an autonomous vehicle, which is applied in the cloud, including: acquiring heart rate data of a driver, dynamic video of the driver sent by a vehicle end and fatigue state information of the driver identified by a driver monitoring system; identifying and obtaining the sleeping state of the driver based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver; and under the condition that the driver is in the sleep state, generating a wake-up control instruction based on the wake-up configuration of the operator, and sending the wake-up control instruction to the vehicle end to wake up the driver.
Or a driver remote wake-up method for executing an autonomous vehicle applied to a vehicle end, comprising: acquiring a dynamic video of a driver and fatigue state information of the driver, which is identified by a driver monitoring system, and sending the information to a cloud end; receiving a wake-up control instruction generated by the cloud according to the sleeping state of a driver, and executing the wake-up control instruction; the sleep state of the driver is obtained by recognition based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver.
In addition, the logic instructions in the memory 530 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention further provides a computer program product, where the computer program product includes a computer program that can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can execute a method for remotely waking a driver of an autonomous vehicle provided by the above methods, where the method for remotely waking the driver of the autonomous vehicle is applied to a cloud, and the method includes: acquiring heart rate data of a driver, dynamic video of the driver sent by a vehicle end and fatigue state information of the driver identified by a driver monitoring system; identifying and obtaining the sleeping state of the driver based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver; and under the condition that the driver is in a sleep state, generating a wake-up control instruction based on the wake-up configuration of the operator, and sending the wake-up control instruction to the vehicle end to wake up the driver.
Or a driver remote wake-up method for executing an autonomous vehicle applied to a vehicle end, comprising: acquiring a dynamic video of a driver and fatigue state information of the driver, which is identified by a driver monitoring system, and sending the information to a cloud end; receiving a wake-up control instruction generated by the cloud according to the sleeping state of a driver, and executing the wake-up control instruction; the sleep state of the driver is obtained by recognition based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the method for remotely waking a driver of an autonomous vehicle provided by the above method, the method for remotely waking the driver of the autonomous vehicle being applied in a cloud, comprising: acquiring heart rate data of a driver, dynamic video of the driver sent by a vehicle end and fatigue state information of the driver identified by a driver monitoring system; identifying and obtaining the sleeping state of the driver based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver; and under the condition that the driver is in the sleep state, generating a wake-up control instruction based on the wake-up configuration of the operator, and sending the wake-up control instruction to the vehicle end to wake up the driver.
Or a driver remote wake-up method for executing an autonomous vehicle applied to a vehicle end, comprising: acquiring a dynamic video of a driver and fatigue state information of the driver, which is identified by a driver monitoring system, and sending the information to a cloud end; receiving a wake-up control instruction generated by the cloud according to the sleeping state of a driver, and executing the wake-up control instruction; the sleep state of the driver is obtained by recognition based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of the various embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. The utility model provides a driver remote wake-up method of automatic driving vehicle which characterized in that, this driver remote wake-up method of automatic driving vehicle uses in the high in the clouds, includes:
acquiring heart rate data of a driver, dynamic video of the driver sent by a vehicle end and fatigue state information of the driver identified by a driver monitoring system;
identifying and obtaining the sleeping state of the driver based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver;
and under the condition that the driver is in the sleep state, generating a wake-up control instruction based on the wake-up configuration of the operator, and sending the wake-up control instruction to the vehicle end to wake up the driver.
2. The method of claim 1, wherein identifying a driver sleep state based on the driver heart rate data, driver dynamics video, and driver fatigue state information comprises:
determining the current sleep state of the driver according to the heart rate data of the driver and the fatigue state information of the driver and by utilizing the prestored heart rate of the driver and the corresponding relation between the fatigue state and the sleep state; the corresponding relation is obtained by carrying out data analysis on the heart rate data of the historical driver and the fatigue state information of the historical driver;
the method comprises the steps that a dynamic video of a driver is displayed under the condition that the current driver is in sleep for the driver, so that an operator is reminded of confirming the sleep state of the current driver, and after a confirmation instruction of the operator is received, a wakeup control instruction is generated based on wakeup configuration of the operator and is sent to a vehicle end to wake the driver.
3. The method of claim 1, wherein the wake-up profile includes a wake-up pattern, a wake-up intensity, and a wake-up duration; the waking mode comprises acoustic waking, optical waking, tactile waking and text waking;
the wake-up configuration is formed by:
setting a wake-up time;
determining at least one mode to be executed from the acoustic wake-up, the optical wake-up, the haptic wake-up and the text wake-up;
setting wake-up intensity for each mode to be executed;
accordingly, the wake-up control instruction is generated based on the wake-up configuration of the operator, and comprises the following steps:
and generating a wakeup control instruction based on the set wakeup time length, at least one to-be-executed mode and the corresponding wakeup strength.
4. The method of claim 1, wherein after the sending to the end of the vehicle to effect the waking of the driver, the method further comprises:
when receiving execution feedback generated by the vehicle end according to the received wake-up control instruction within preset feedback time, generating vehicle end execution wake-up reminder for reminding operators of the progress of wake-up of the vehicle end;
and when the execution feedback generated by the vehicle end according to the received wake-up control instruction is not received within the preset feedback time, generating a network connection overtime prompt for prompting operators that the wake-up task is not completed.
5. The method of claim 1, wherein prior to the sending to the vehicle end to effect the waking of the driver, the method further comprises:
judging whether the communication response delay between the cloud end and the vehicle end meets a preset threshold value or not;
under the condition that the communication response delay meets a preset threshold value, sending the wake-up control instruction to a vehicle end;
and under the condition that the communication response delay does not accord with a preset threshold value, the wake-up control instruction is not sent to a vehicle end, and the wake-up reminding is stopped.
6. A driver remote wake-up method of an autonomous vehicle is applied to a vehicle end, and includes:
acquiring a dynamic video of a driver and fatigue state information of the driver, which is identified by a driver monitoring system, and sending the information to a cloud end;
receiving a wake-up control instruction generated by the cloud according to the sleeping state of a driver, and executing the wake-up control instruction; the sleep state of the driver is obtained by recognition based on the heart rate data of the driver, the dynamic video of the driver and the fatigue state information of the driver.
7. The method of remotely waking a driver of an autonomous vehicle as recited in claim 6, wherein prior to the executing of the wake-up control instructions, the method comprises:
judging whether to execute a wake-up control instruction or not;
under the condition that the execution of the wake-up control instruction is judged, generating execution feedback for executing the wake-up control instruction according to the wake-up control instruction, executing the wake-up control instruction, and sending the execution feedback to a cloud end;
and under the condition that the wake-up control instruction is not executed, generating execution feedback of the non-execution wake-up control instruction and the reason thereof according to the wake-up control instruction, and sending the execution feedback to the cloud.
8. The method of remotely waking a driver of an autonomous vehicle as recited in claim 6, wherein said executing the wake-up control command comprises:
when an event with the priority higher than that of the wake-up control instruction occurs in the execution process, interrupting the execution of the wake-up control instruction, generating an execution process interruption feedback, and sending the execution process interruption feedback to the cloud; wherein the event with priority higher than the wake-up control instruction is one of activating an autonomous driving mode, exiting an autonomous driving mode, or driving task support;
when an event with the priority not higher than that of the wake-up control instruction occurs or no event occurs in the execution process, the wake-up control instruction is executed, and after the execution is completed, execution completion feedback is generated and sent to the cloud end; wherein the event with the priority not higher than the wake-up control instruction is an automatic driving behavior reminder in an automatic driving mode.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements a method for remote driver wake up of an autonomous vehicle as claimed in any of claims 1 to 5 or 6 to 8.
10. A non-transitory computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements a method for remotely waking a driver of an autonomous vehicle as recited in any one of claims 1 to 5 or 6 to 8.
CN202210707304.6A 2022-06-21 2022-06-21 Remote driver wake-up method and device for automatically driving vehicle Pending CN115195770A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210707304.6A CN115195770A (en) 2022-06-21 2022-06-21 Remote driver wake-up method and device for automatically driving vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210707304.6A CN115195770A (en) 2022-06-21 2022-06-21 Remote driver wake-up method and device for automatically driving vehicle

Publications (1)

Publication Number Publication Date
CN115195770A true CN115195770A (en) 2022-10-18

Family

ID=83575416

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210707304.6A Pending CN115195770A (en) 2022-06-21 2022-06-21 Remote driver wake-up method and device for automatically driving vehicle

Country Status (1)

Country Link
CN (1) CN115195770A (en)

Similar Documents

Publication Publication Date Title
WO2017185712A1 (en) Vehicle control method and system
CN107901828B (en) Method and device for reminding overtaking through intelligent equipment cooperation
CN113345204A (en) Vehicle-mounted remote fatigue awakening method and awakening system thereof
CN111439271A (en) Auxiliary driving method and auxiliary driving equipment based on voice control
CN111469677B (en) Trailer control method and system
CN110063735A (en) Fatigue monitoring method based on driving behavior
CN110053630B (en) Vehicle control method and device
CN111469854B (en) Vehicle-mounted method for guaranteeing safety of driver, vehicle-mounted electronic equipment, server method and server
KR101890379B1 (en) Infrastructure system for a vehicle
CN114407735A (en) Control method and device for automobile cabin, vehicle and storage medium
CN114407912A (en) Vehicle rescue method and device, vehicle and storage medium
CN115195770A (en) Remote driver wake-up method and device for automatically driving vehicle
DE102021108485A1 (en) REAR PASSENGER WARNING SYSTEM
US11345362B2 (en) Adaptive warnings and emergency braking for distracted drivers
CN109017556B (en) Automatic reminding method and device for vehicle
CN207274540U (en) A kind of automobile driving safe auxiliary system
CN110509870A (en) A kind of vehicle-mounted CAN network and vehicle-mounted CAN network management
CN110077417A (en) A kind of method and system of automobile guideboard acquisition of information prompt
CN113311744B (en) Manual control method and device for automatic driving
CN113968241A (en) Control method and device for dangerous driving behaviors, vehicle and storage medium
CN113190121A (en) Key control method, system and device and computer readable storage medium
CN218750738U (en) Monitoring system and car that car went
CN111137290A (en) Cruise control method and device
CN114368393B (en) Lane line loss early warning method and system on straight road and man-machine co-driving method
CN211455291U (en) Bus video monitoring system based on GPS navigation system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination