WO2024046091A1 - 一种驾驶员提醒方法、装置、系统和移动载体 - Google Patents

一种驾驶员提醒方法、装置、系统和移动载体 Download PDF

Info

Publication number
WO2024046091A1
WO2024046091A1 PCT/CN2023/112557 CN2023112557W WO2024046091A1 WO 2024046091 A1 WO2024046091 A1 WO 2024046091A1 CN 2023112557 W CN2023112557 W CN 2023112557W WO 2024046091 A1 WO2024046091 A1 WO 2024046091A1
Authority
WO
WIPO (PCT)
Prior art keywords
driver
reminder
mobile terminal
prompt
cockpit
Prior art date
Application number
PCT/CN2023/112557
Other languages
English (en)
French (fr)
Inventor
卢远志
高良柱
吕自波
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2024046091A1 publication Critical patent/WO2024046091A1/zh

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08Interaction between the driver and the control system
    • B60W50/14Means for informing the driver, warning the driver or prompting a driver intervention
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00Drive control systems specially adapted for autonomous road vehicles

Definitions

  • the embodiments of the present application relate to the field of intelligent driving, and more specifically, to a driver reminder method, device, system and mobile carrier.
  • the driver monitoring system monitors the situation of the driver in the loop in real time and provides hierarchical reminders. The system can promptly remind the driver to take over the vehicle to ensure driving safety.
  • the current driver monitoring and hierarchical reminder system is mainly used for advanced driving assist systems (ADAS) below L3 level. It mainly displays the takeover reminder information through the on-board instrument to remind the driver to take over the vehicle.
  • ADAS advanced driving assist systems
  • the driver's attention is on a mobile terminal (for example, a mobile phone)
  • the driver may miss the takeover reminder information issued by the driver monitoring and hierarchical reminder system based on the on-board instrument due to the shift of attention, thus increasing traffic. Risk of accidents.
  • the embodiments of the present application provide a driver reminder method, device, system and mobile carrier, which can ensure that the driver can receive the reminder information, thereby helping the driver to take over the mobile carrier in a timely and effective manner, thus improving the safety of the mobile carrier. .
  • a driver reminder method includes, when the mobile carrier is in a driving state, acquiring first data collected by a sensor from a mobile terminal, and acquiring second data collected by a sensor in a cockpit of the mobile carrier. data; determine the driver status according to the first data and the second data; when the driver status indicates that the driver is not around, the control prompting device prompts the driver to take over the mobile carrier.
  • the first data obtained through the mobile terminal can make up for the shortcomings in the perspective of the second data collected by the cockpit, and the driving status determined by the multivariate data can improve the accuracy of judgment of the driver's status. Based on accurate driver status, it can not only prevent driver management reminders due to misdetection by DMS and improve user experience, but also help issue driver reminders through multiple devices, reducing safety risks caused by drivers missing reminders. Thereby improving driving safety.
  • the method further includes: sending first prompt information to the mobile terminal, where the first prompt information is used to instruct the driver to take over the mobile carrier.
  • the takeover prompting device in the cockpit cooperates with the mobile terminal to send takeover reminder information to the driver, reducing the possibility of the driver causing an accident to the mobile carrier due to missing the takeover reminder issued in the cockpit, and improving the delivery of takeover reminder information. effectiveness and timeliness, thus improving the security of mobile carriers.
  • sending the first prompt information to the mobile terminal includes: sending the first prompt information to the mobile terminal when it is detected that the driver's attention is on the mobile terminal.
  • the terminal sends a takeover reminder at the first time, which can prevent the driver from missing the takeover reminder displayed on the takeover reminder device in the cockpit when the driver's attention is on the mobile terminal, which can improve the effectiveness of the takeover reminder and the timeliness of the driver receiving the takeover reminder. properties, thus improving the safety of mobile carriers.
  • the first prompt information is sent to the mobile terminal through the first hierarchical reminder method of the mobile terminal.
  • the first hierarchical reminder method of the mobile terminal includes a first-level reminder of the mobile terminal, a second-level reminder of the mobile terminal, and a third-level reminder of the mobile terminal.
  • the first-level reminder of the mobile terminal may include a mobile terminal display reminder
  • the second-level reminder of the mobile terminal may include a mobile terminal display reminder and a sound reminder, or include a mobile terminal display reminder and a vibration reminder
  • the third-level reminder of the mobile terminal includes a mobile terminal
  • the terminal displays reminders, sound reminders and vibration reminders.
  • the first prompt is sent to the mobile terminal through the second hierarchical reminder method of the mobile terminal. information.
  • the second hierarchical reminder method of the mobile terminal includes a first-level reminder of the mobile terminal, a second-level reminder of the mobile terminal, and a third-level reminder of the mobile terminal.
  • the first-level reminder of the mobile terminal may include a vibration reminder of the mobile terminal or the sound reminder of the mobile terminal
  • the second-level reminder of the mobile terminal may include a vibration reminder and a sound reminder of the mobile terminal
  • the third-level reminder of the mobile terminal includes the mobile terminal. Display reminders, sound reminders and vibration reminders.
  • a takeover reminder is issued on the mobile terminal's takeover reminder device through vibration or sound, which can reduce the risk of the driver's attention not being on the mobile terminal.
  • the possibility of missing the takeover reminder when the terminal is on the terminal and the driver is not around further improves the effectiveness of the takeover reminder and the timeliness of the driver receiving the takeover reminder, thereby improving the safety of the mobile carrier.
  • the control prompt device prompts the driver to take over the mobile carrier including: based on the duration of the driver's absence from the ring, the control prompt device prompts the driver to take over the mobile carrier.
  • the cockpit hierarchical reminder method is used to control the prompt device to prompt the driver to take over the mobile carrier based on the duration of the driver's absence.
  • the display device in the cockpit is controlled to prompt the driver to take over the mobile carrier through the first-level reminder in the cockpit; if the driver is in the first-level cockpit If the mobile carrier is not taken over under the reminder, then when the driver's absence duration reaches the second threshold (for example, 30s), the display device and sound device in the cockpit will be controlled through the second level reminder in the cockpit to prompt the driver to take over the mobile carrier.
  • the first threshold for example, 15s
  • the display device in the cockpit is controlled to prompt the driver to take over the mobile carrier through the first-level reminder in the cockpit; if the driver is in the first-level cockpit If the mobile carrier is not taken over under the reminder, then when the driver's absence duration reaches the second threshold (for example, 30s), the display device and sound device in the cockpit will be controlled through the second level reminder in the cockpit to prompt the driver to take over the mobile carrier.
  • the third-level reminder of the cockpit is used to control the vehicle in the cockpit. Display devices, sound devices and seat belt vibration devices prompt the driver to take over the mobile carrier.
  • the control prompt device prompts the driver to take over the mobile carrier including: when the driver status indicates that the driver is not around and an emergency event is detected, the control prompt device prompts The driver takes over the mobile carrier.
  • the control display device after detecting the occurrence of an emergency event, prompts the driver to take over the mobile carrier through the first-level reminder from the cockpit. If the driver does not take over the mobile carrier under the first-level reminder from the cockpit, Based on the duration of the driver's absence, the prompt device is controlled through the cockpit hierarchical reminder method to prompt the driver to take over the mobile carrier.
  • the display in the cockpit is controlled through the second-level reminder of the cockpit.
  • the device and the sounding device prompt the driver to take over the mobile carrier; if the driver does not take over the mobile carrier under the second level reminder of being in the cockpit, then when the driver is not in the loop and the duration accumulates to the fifth threshold (for example, 35s), the driver will be informed through the cockpit.
  • the third level reminder controls the display device, sound device and seat belt vibration device in the cockpit to prompt the driver to take over the mobile carrier.
  • the control prompt device can prompt the driver to take over the mobile carrier based on the duration of the driver's absence in the loop, and the driver can be promptly and effectively transferred to the mobile carrier. Takeover reminder information is delivered to the driver, thereby improving driving safety.
  • the method further includes: before the control prompt device prompts the driver to take over the mobile carrier, determining that the traveling speed of the mobile carrier is greater than or equal to the speed threshold.
  • the driving speed of the mobile carrier is used as a consideration for triggering the takeover reminder, which can further improve driving safety and user experience.
  • the driver can take off his hands and eyes. At this time, the speed of the moving carrier is less than the speed threshold. Unless an emergency occurs, the driver will not receive a takeover reminder message. This can improve the use of user experience.
  • control prompting device prompts the driver to take over the mobile carrier and sends the first prompt information to the mobile terminal simultaneously.
  • the method further includes: recording the response parameters of the driver taking over the mobile carrier; sending the response parameters to the cloud server, where the response parameters are used to determine the driver's driving behavior Score, the driving behavior score is used to evaluate the safety of the driver's behavior; customized information is received from the cloud server, and the customized information is determined based on the driving behavior score.
  • the driver reminder issued based on this will be more accurate, allowing the driver to take over the control of the mobile carrier.
  • Response parameters are recorded more accurately to improve the accuracy of subsequent information push.
  • a driver reminder device which device includes an acquisition unit, a determination unit and a control unit.
  • the acquisition unit is configured to acquire first data collected by a sensor from a mobile terminal when the mobile carrier is in a driving state, and acquire second data collected by a sensor in a cockpit of the mobile carrier.
  • the determining unit is used to determine the driver's status based on the first data and the second data.
  • the control unit is used to control the prompt device to prompt the driver to take over the mobile carrier when the driver status indicates that the driver is not around.
  • the first data obtained through the mobile terminal can make up for the shortcomings in the perspective of the second data collected by the cockpit, and the driving status determined by the multivariate data can improve the accuracy of judgment of the driver's status. Based on accurate driver status, it can not only prevent driver reminders due to misdetection by DMS and improve user experience, but also help issue driver reminders through multiple devices, reducing safety risks caused by drivers missing takeover reminders. Thereby improving driving safety.
  • the device further includes a communication unit, the communication unit is used to send first prompt information to the mobile terminal, and the first prompt information is used to instruct the driver to take over the mobile carrier. .
  • the takeover prompting device in the cockpit cooperates with the mobile terminal to send takeover reminder information to the driver, reducing the possibility of the driver causing an accident to the mobile carrier due to missing the takeover reminder issued in the cockpit, and improving the delivery of takeover reminder information. effectiveness and timeliness, thus improving the security of mobile carriers.
  • the device further includes a first detection unit.
  • the communication unit is specifically configured to, Send the first prompt information to the mobile terminal.
  • the prompt device in the cockpit and the mobile terminal are used to issue a takeover reminder at the first time, which can avoid missing the driver's attention on the mobile terminal.
  • the takeover reminder displayed on the takeover reminder device in the cockpit can improve the effectiveness of the takeover reminder and the timeliness of the driver receiving the takeover reminder, thereby improving the safety of the mobile carrier.
  • the communication unit when the first detection unit detects that the driver's attention is on the mobile terminal, the communication unit is configured to send the first prompt information to the mobile terminal through the first hierarchical reminder method of the mobile terminal. .
  • the first hierarchical reminder method of the mobile terminal includes a first-level reminder of the mobile terminal, a second-level reminder of the mobile terminal, and a third-level reminder of the mobile terminal.
  • the first-level reminder of the mobile terminal may include a mobile terminal display reminder
  • the second-level reminder of the mobile terminal may include a mobile terminal display reminder and a sound reminder, or include a mobile terminal display reminder and a vibration reminder
  • the third-level reminder of the mobile terminal includes a mobile terminal
  • the terminal displays reminders, sound reminders and vibration reminders.
  • the communication unit when the driver status indicates that the driver is not in the loop and the first detection unit does not detect that the driver's attention is on the mobile terminal, the communication unit is configured to provide a second hierarchical reminder through the mobile terminal. method, sending the first prompt information to the mobile terminal.
  • the second hierarchical reminder method of the mobile terminal includes a first-level reminder of the mobile terminal, a second-level reminder of the mobile terminal, and a third-level reminder of the mobile terminal.
  • the first-level reminder of the mobile terminal may include a vibration reminder of the mobile terminal or the sound reminder of the mobile terminal
  • the second-level reminder of the mobile terminal may include a vibration reminder and a sound reminder of the mobile terminal
  • the third-level reminder of the mobile terminal includes the mobile terminal. Display reminders, sound reminders and vibration reminders.
  • a takeover reminder is issued on the mobile terminal's takeover reminder device through vibration or sound, which can reduce the risk of the driver's attention not being on the mobile terminal.
  • the possibility of missing the takeover reminder when the terminal is on the terminal and the driver is not around further improves the effectiveness of the takeover reminder and the timeliness of the driver receiving the takeover reminder, thereby improving the safety of the mobile carrier.
  • control unit is used to: according to the duration of the driver's absence from the ring,
  • the control prompting device prompts the driver to take over the mobile carrier.
  • control unit is configured to control the prompt device to prompt the driver to take over the mobile carrier through the cockpit hierarchical reminder method.
  • the display device in the cockpit is controlled to prompt the driver to take over the mobile carrier through the first-level reminder in the cockpit; if the driver is in the first-level cockpit If the mobile carrier is not taken over under the reminder, then when the driver's absence duration reaches the second threshold (for example, 30s), the display device and sound device in the cockpit will be controlled through the second level reminder in the cockpit to prompt the driver to take over the mobile carrier.
  • the first threshold for example, 15s
  • the display device in the cockpit is controlled to prompt the driver to take over the mobile carrier through the first-level reminder in the cockpit; if the driver is in the first-level cockpit If the mobile carrier is not taken over under the reminder, then when the driver's absence duration reaches the second threshold (for example, 30s), the display device and sound device in the cockpit will be controlled through the second level reminder in the cockpit to prompt the driver to take over the mobile carrier.
  • the third-level reminder of the cockpit is used to control the vehicle in the cockpit. Display devices, sound devices and seat belt vibration devices prompt the driver to take over the mobile carrier.
  • the device further includes a second detection unit.
  • the control unit uses At this time, the control prompt device prompts the driver to take over the mobile carrier.
  • control unit is specifically configured to, after detecting the occurrence of an emergency, control the display device to prompt the driver to take over the mobile carrier through the first-level reminder in the cockpit. If the driver is in the first-level reminder in the cockpit, If the mobile carrier is not taken over, the prompt device will be controlled to prompt the driver to take over the mobile carrier through the cockpit hierarchical reminder method based on the duration of the driver's absence.
  • the display in the cockpit is controlled through the second-level reminder of the cockpit.
  • the device and the sounding device prompt the driver to take over the mobile carrier; if the driver does not take over the mobile carrier under the second level reminder of being in the cockpit, then when the driver is not in the loop and the duration accumulates to the fifth threshold (for example, 35s), the driver will be informed through the cockpit.
  • the third level reminder controls the display device, sound device and seat belt vibration device in the cockpit to prompt the driver to take over the mobile carrier.
  • the control prompt device can prompt the driver to take over the mobile carrier based on the duration of the driver's absence in the loop, and the driver can be promptly and effectively transferred to the mobile carrier. Takeover reminder information is delivered to the driver, thereby improving driving safety.
  • the determination unit is further configured to: determine that the traveling speed of the mobile carrier is greater than or equal to the speed threshold before the control prompt device prompts the driver to take over the mobile carrier.
  • the driving speed of the mobile carrier is used as a consideration for triggering the takeover reminder, which can further improve driving safety and user experience.
  • the driver can take off his hands and eyes. At this time, the speed of the moving carrier is less than the speed threshold. Unless an emergency occurs, the driver will not receive a takeover reminder message. This improves user experience.
  • control unit controls the prompting device to prompt the driver to take over the mobile carrier and the communication unit sends the first prompting information to the mobile terminal simultaneously.
  • the device further includes a recording unit, the recording unit is used to record the response parameters of the driver taking over the mobile carrier.
  • the communication unit is also used to send response parameters to the cloud server, where the response parameters are used to determine the driver's driving behavior score, and the driving behavior score is used to evaluate the driver's behavioral safety.
  • the communication unit is also specifically configured to receive customized information from the cloud server, where the customized information is determined based on the driving behavior score.
  • the driver reminder issued based on this will be more accurate, allowing the driver to take over the control of the mobile carrier.
  • Response parameters are recorded more accurately to improve the accuracy of subsequent information push.
  • a driver reminder device in a third aspect, includes at least one processor and an interface circuit.
  • the processor communicates with the reminder device through the interface circuit; the at least one processor is used to execute any one of the above first aspects.
  • the method described controls the prompt device.
  • the prompt device may be at least one of a display device, a sound-generating device (eg, a speaker), a seat belt vibrating device, and an ambient light.
  • the display device may include a vehicle-mounted display screen or a projection display screen.
  • a driver reminder device in a fourth aspect, includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the device executes any of the instructions in the first aspect.
  • the storage unit is used to store instructions
  • the processing unit executes the instructions stored in the storage unit, so that the device executes any of the instructions in the first aspect.
  • the above-mentioned processing unit may include at least one processor, and the above-mentioned storage unit may be a memory, wherein the memory may It is a storage unit within the chip (eg, register, cache, etc.), or it may be a storage unit located outside the chip in the mobile carrier (eg, read-only memory, random access memory, etc.).
  • the above-mentioned storage unit may be a memory, wherein the memory may It is a storage unit within the chip (eg, register, cache, etc.), or it may be a storage unit located outside the chip in the mobile carrier (eg, read-only memory, random access memory, etc.).
  • a driver reminder system in a fifth aspect, includes a computing platform and a prompting device.
  • the computing platform includes the device described in any one of the above-mentioned second aspect, third aspect, or fourth aspect.
  • the prompt device may be at least one of a display device, a sound-generating device (eg, a speaker), a seat belt vibrating device, and an ambient light.
  • a sixth aspect provides a mobile carrier, which includes the device described in any one of the above second, third, or fourth aspects, or the mobile carrier includes the driving device described in the fifth aspect.
  • Staff reminder system includes the device described in any one of the above second, third, or fourth aspects, or the mobile carrier includes the driving device described in the fifth aspect.
  • the mobile carrier is a vehicle.
  • a computer program product includes: computer program code.
  • the computer program code When the computer program code is run on a computer, it causes the computer to perform any of the possible methods in the first aspect.
  • the above computer program code can be stored in whole or in part on the first storage medium, where the first storage medium can be packaged together with the processor, or can be packaged separately from the processor. This is not the case in the embodiments of this application. Specific limitations.
  • a computer-readable medium stores program code.
  • the computer program code When the computer program code is run on a computer, it causes the computer to perform any of the possible methods in the first aspect. .
  • inventions of the present application provide a chip system.
  • the chip system includes a processor for calling a computer program or computer instructions stored in a memory, so that the processor executes any of the possible methods of the first aspect. method.
  • the processor is coupled with the memory through an interface.
  • the chip system further includes a memory, and a computer program or computer instructions are stored in the memory.
  • Figure 1 is a functional block diagram of a mobile carrier provided by an embodiment of the present application.
  • Figure 2 is a schematic flow chart of a driver reminder method provided by an embodiment of the present application.
  • Figure 3 is a schematic diagram of an interaction process for a mobile terminal to access a mobile carrier provided by an embodiment of the present application
  • Figure 4 is a schematic diagram of the connection between a mobile terminal and a vehicle cockpit digital platform provided by an embodiment of the present application;
  • Figure 5 is a schematic diagram of the connection between another mobile terminal and a vehicle cockpit digital platform provided by an embodiment of the present application
  • Figure 6 is a schematic diagram of the connection between another mobile terminal and a vehicle cockpit digital platform provided by an embodiment of the present application
  • Figure 7 is an interactive schematic diagram of a driver reminder method provided by an embodiment of the present application.
  • Figure 8 is a schematic diagram of a cockpit classification reminder method provided by an embodiment of the present application.
  • Figure 9 is a schematic diagram of a hierarchical reminder method for a mobile terminal provided by an embodiment of the present application.
  • Figure 10 is an interactive schematic diagram of another driver reminder method provided by an embodiment of the present application.
  • Figure 11 is a schematic diagram of a driving behavior assessment method based on driver reminder records provided by an embodiment of the present application.
  • Figure 12 is a schematic diagram of a driver reminder information sending process provided by an embodiment of the present application.
  • Figure 13 is a schematic diagram of human-computer interaction between driver reminder information in the cockpit and driver reminder information on a mobile terminal provided by an embodiment of the present application;
  • Figure 14 is a schematic diagram of a driver reminder device provided by an embodiment of the present application.
  • Figure 15 is a schematic diagram of the hardware structure of a driver reminder device provided by the implementation of this application.
  • Prefixes such as “first” and “second” are used in the embodiments of this application only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects.
  • the use of ordinal words and other prefixes used to distinguish the described objects does not constitute a limitation on the described objects.
  • the statement of the described objects please refer to the description of the context in the claims or embodiments. description, the use of such prefixes should not constitute unnecessary restrictions.
  • “plurality” means two or more.
  • FIG. 1 is a functional block diagram of a mobile carrier 100 provided by an embodiment of the present application.
  • the mobile carrier 100 may include a sensing system 120 , a display device 130 and a computing platform 150 , where the sensing system 120 may include several types of sensors that sense information about the environment around the mobile carrier 100 .
  • the sensing system 120 may include a positioning system, which may be a global positioning system (GPS), Beidou system or other positioning systems, or an inertial measurement unit (IMU).
  • GPS global positioning system
  • Beidou system Beidou system or other positioning systems
  • IMU inertial measurement unit
  • the sensing system 120 may also include one or more of lidar, millimeter wave radar, ultrasonic radar, and camera devices.
  • the computing platform 150 may include one or more processors, for example, processors 151 to 15n (n is a positive integer).
  • the processor is a circuit with signal processing capabilities.
  • the processor may be a circuit with Circuits with the ability to read and execute instructions, such as central processing unit (CPU), microprocessor, graphics processing unit (GPU) (can be understood as a microprocessor), or digital signal processing (digital signal processor, DSP), etc.; in another implementation, the processor can achieve certain functions through the logical relationship of the hardware circuit. The logical relationship of the hardware circuit is fixed or can be reconstructed.
  • the processor is a dedicated Hardware circuits implemented by application-specific integrated circuits (ASICs) or programmable logic devices (PLDs), such as field programmable gate arrays (FPGAs).
  • ASICs application-specific integrated circuits
  • PLDs programmable logic devices
  • FPGAs field programmable gate arrays
  • the process of the processor loading the configuration file and realizing the hardware circuit configuration can be understood as the process of the processor loading instructions to realize the functions of some or all of the above units.
  • it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing Unit (deep learning processing unit, DPU), etc.
  • the computing platform 150 may also include a memory, which is used to store instructions. Some or all of the processors 151 to 15n may call instructions in the memory and execute the instructions to implement corresponding functions.
  • the mobile carrier in this application may include road vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment, etc.
  • the mobile carrier can be a vehicle, which is a vehicle in a broad sense, and can be a means of transportation (such as commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (such as forklifts, trailers, tractors, etc.) etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), agricultural equipment (such as lawn mowers, harvesters, etc.), amusement equipment, toy vehicles, etc.
  • the embodiments of this application do not specifically limit the types of vehicles.
  • the mobile carrier can be a vehicle such as an airplane or a ship. The following description takes the mobile carrier as a vehicle as an example.
  • the display device 130 in the cockpit of the mobile carrier is mainly divided into two categories.
  • the first category is a vehicle-mounted display screen;
  • the second category is a projection display screen, such as a head-up display (HUD).
  • the vehicle display screen is a physical display screen and an important part of the vehicle infotainment system.
  • There can be multiple displays in the cockpit such as digital instrument display, central control screen, passenger in the co-pilot seat (also known as The display in front of the front passenger), the display in front of the left rear passenger, the display in front of the right rear passenger, and even the car window can be used as a display.
  • Head-up display also known as head-up display system. It is mainly used to display driving information such as speed and navigation on the display device in front of the driver (such as the windshield).
  • HUD includes, for example, combined head-up display (combiner-HUD, C-HUD) system, windshield-type head-up display (windshield-HUD, W-HUD) system, and augmented reality head-up display system (augmented reality HUD, AR-HUD).
  • the vehicle display screen may include a human machine interface (HMI).
  • DMS Driver monitor system
  • the driver monitoring system is used to monitor the driver's status to prevent driver fatigue, distraction and other behaviors that may harm the driving process.
  • the operational design domain is the external conditions under which autonomous vehicles can drive normally and safely, such as road type, driving area, speed, environment (for example, weather, day/night, etc.), etc.
  • ODD is the operating constraint used to activate the autonomous driving system. If the external environment in which the vehicle is traveling does not meet the ODD, that is, it exceeds the setting of the automatic driving system. ODD, then the autonomous driving system can issue driver reminders so that the driver can take over the vehicle in time to prevent accidents.
  • Minimum risk strategy refers to the risk-minimizing solution path taken by the autonomous driving system when a systemic failure occurs (a fault that causes the system to not work) or exceeds the original ODD of the system to ensure that the autonomous driving vehicle operates in Safe during operation. This strategy can be executed automatically when the autonomous driving system requires manual takeover and no response is received, or it can be executed automatically when there is a serious risk of collision or vehicle failure.
  • L0 level is no automation
  • L1 level is driving support
  • L2 level is partial automation
  • L3 level is conditional automation
  • L4 level is high automation
  • L5 level is complete automation.
  • the tasks of monitoring and responding to road conditions from L1 to L3 are jointly completed by the driver and the system, and the driver is required to take over the dynamic driving task.
  • L1 to L3 autonomous driving belongs to the stage of human-machine co-driving.
  • L4 and L5 levels allow the driver to completely transform into a passenger role.
  • autonomous driving is divided into different levels, the driver needs to take over the vehicle in time if the following situations occur. To prevent accidents from happening.
  • the automatic driving system fails, or the vehicle driving conditions are outside ODD, or the vehicle is in an emergency situation, etc.
  • the autonomous driving system can actually be regarded as a driving assistance system, assisting the driver.
  • driving assistance systems below L3 level do not allow the driver to be hands-off or eyes-off, that is, the driver is in the loop during the entire driving process. Therefore, when the current driver monitoring system detects that the driver is not in the loop, the hierarchical reminder system issues a driver reminder.
  • Visually, acoustically, and tactilely, graded takeover reminders are realized through display devices, sound-generating devices (e.g., speakers), and safety belts respectively.
  • the driver is affected by physiological and psychological factors, and the driver cannot always pay attention to the display device in the cockpit.
  • the driver may be using a mobile terminal, such as a smartphone, Mobile terminals such as tablets, smart bracelets or smart watches. If an emergency occurs at this time and the driver monitoring system detects that the driver is not around, the hierarchical reminder system needs to send out a driver reminder message. Since the driver's attention is not on the driving task, the current method of displaying reminders based on the cockpit display device may allow the driver to receive the best time to take over the display reminder, thereby causing potential traffic accidents. .
  • embodiments of the present application propose a driver reminder method, device, system and vehicle to reduce the risk of the driver missing the best time to receive the takeover display reminder, thereby improving the driving safety of autonomous driving.
  • FIG. 2 is a schematic flowchart of a driver reminder method 200 provided by an embodiment of the present application.
  • the method 200 can be executed by a mobile carrier (for example, a vehicle), or the method 200 can be executed by a system composed of a computing platform and a prompting device, or the method 200 can be executed by a computing platform, or,
  • the method 200 may be executed by a system-on-a-chip (SOC) in the computing platform, or the method 200 may be executed by a processor in the computing platform.
  • SOC system-on-a-chip
  • the method 200 includes:
  • adaptive cruise control ACC
  • LCC lane centering control
  • pilot-assisted driving can be enabled for the mobile carrier in the driving scene.
  • NOA hover on autopilot
  • NOA auto parking assist
  • APA automatic parking assist
  • the senor of the mobile terminal can be a camera device of the mobile terminal or a screen touch sensing device, etc.
  • the sensor in the cockpit of the mobile carrier can be a camera device in the DMS system, and the camera device can be installed on the interior rearview mirror. , on the A-pillar on the driving position side, or on the steering wheel, etc., the embodiment of the present application does not limit this.
  • the mobile terminal may be a mobile terminal such as a smartphone, a tablet, a smart bracelet, or a smart watch.
  • S220 Determine the driver's status based on the first data and the second data.
  • the reliability of the first data and the second data is judged, and the driver's state is determined based on the reliable driving data, where the reliable driving data can be the first data or the second data.
  • the sensor in the cockpit of the mobile carrier tracks the driver's line of sight
  • only one of the driver's eyes can be tracked
  • both of the driver's eyes can be tracked. Then the sensor from the mobile terminal The first data is more reliable.
  • the second data from the sensors in the cockpit of the mobile carrier are more reliable.
  • the first data and the second data are fused to obtain the driver status.
  • the first data and the original data of the first data are fused to obtain the driver's status, or the data result of the first data and the data result of the second data are fused to obtain the driver's status.
  • the driver status may be the driver in the loop or the driver not in the loop, where the driver in the loop indicates that the driver's attention is on the driving task, and the driver not in the loop may indicate that the driver's attention is not on the driving task, for example , drivers exhibit abnormal driving behaviors such as distraction, fatigue, and use of mobile terminal devices.
  • control prompting device prompts the driver to take over the mobile carrier.
  • the second data comes from a camera device fixedly installed in the cockpit. Its installation position is fixed and the installation angle is fixed. Therefore, the camera device in the cockpit may have an insufficient field of view (FOV) or be blocked. This situation makes the data collected by the camera device in the cockpit inaccurate, resulting in inaccurate driver status obtained only based on the DMS. Therefore, in the embodiment of the present application, the first data obtained through the mobile terminal can make up for the shortcomings in the perspective of the second data collected by the cockpit, and the driving status determined by the multivariate data can improve the accuracy of judgment of the driver's status. Based on accurate driver status, it can not only prevent driver reminders due to misdetection by DMS and improve user experience, but also help issue driver reminders through multiple devices, reducing safety risks caused by drivers missing takeover reminders. Thereby improving driving safety.
  • FOV field of view
  • Figure 3 is a schematic diagram of an interaction process 300 for a mobile terminal to access a mobile carrier provided by an embodiment of the present application.
  • the mobile terminal sends an access request message to the vehicle cockpit digital platform.
  • the mobile terminal can send an access request message to the vehicle cockpit digital platform through wireless communication.
  • the wireless communication method can be near field communication (NFC), Bluetooth or Green Tooth, etc., and the embodiments of the present application are not limited to this.
  • the vehicle cockpit digital platform may be a cockpit domain controller (CDC).
  • CDC cockpit domain controller
  • the vehicle cockpit digital platform authorizes mobile terminal access.
  • the vehicle cockpit digital platform uses an information security mechanism to determine and authenticate whether the ID of the mobile terminal that requests access belongs to a registered whitelist user based on the identity document (ID) of the mobile terminal that requests access. Corresponding mobile terminal. If it is determined that the authentication is passed, then the vehicle cockpit digital platform receives the access request of the mobile terminal and authorizes the mobile terminal to access. If it is determined that the authentication fails, the vehicle cockpit digital platform rejects the access request of the mobile terminal.
  • ID identity document
  • the vehicle cockpit digital platform sends an access response message to the mobile terminal.
  • vehicle cockpit digital platform can also send the access response message to the mobile terminal through wireless communication.
  • the specific wireless communication method is similar to that in S301.
  • S304 The vehicle cockpit digital platform sends a mobile terminal access message to the vehicle's computing platform.
  • the vehicle cockpit digital platform sends the mobile terminal access message to the vehicle's computing platform through the gateway.
  • the computing platform of the vehicle may be the computing platform 130 in Figure 1 , for example, it may be a mobile data center (MDC).
  • MDC mobile data center
  • the interactive process of the mobile terminal accessing the vehicle can be directly implemented through wireless communication, allowing the mobile terminal to access the vehicle's computing platform.
  • Figure 4 is a schematic diagram of the connection between a mobile terminal and a vehicle cockpit digital platform provided by an embodiment of the present application.
  • the display device of the vehicle-mounted cockpit digital platform displays the pop-up window content:
  • the mobile terminal has been accessed.
  • the central control screen displays: Mobile terminal 1 has been accessed, where the mobile terminal 1 can be a smartphone as shown in Figure 4.
  • the mobile terminal displays the pop-up content "Local device is connected to the vehicle cockpit digital platform", as shown in Figure 4.
  • the driver-in-the-loop application is launched, and the mobile terminal displays the pop-up content: "Driver-in-the-loop application” Start, as shown in Figure 4.
  • the sensor in the cockpit may be a camera device in the cockpit.
  • the camera device in the cockpit is installed on the A-pillar on the driver's side.
  • the sensor in the mobile terminal may be the front camera of a smartphone.
  • the mobile terminal is connected to the vehicle cockpit digital platform
  • the camera device in the cockpit is connected to the vehicle's computing platform through the in-car communication network
  • the mobile terminal's camera device is connected to the vehicle's computing platform through the wireless communication network.
  • the in-vehicle communication network can be a car serial bus, such as a controller area network (CAN) bus, a local interconnect network (LIN) bus, etc., or it can also It is a vehicle-mounted Ethernet network, and the embodiment of this application does not limit this.
  • CAN controller area network
  • LIN local interconnect network
  • Figure 5 is a schematic diagram of the connection between another mobile terminal and a vehicle cockpit digital platform provided by an embodiment of the present application.
  • the mobile terminal in response to the first operation, opens a human-computer interaction application, for example, a driving application (application, APP).
  • a human-computer interaction application for example, a driving application (application, APP).
  • the information activated by the "Driver in the loop service” is a pop-up window information. This information will automatically disappear after being displayed on the mobile terminal for a period of time. At this time, the "Driver in the loop service” is running in the background.
  • Figure 6 is a schematic diagram of the connection between another mobile terminal and a vehicle cockpit digital platform provided by an embodiment of the present application.
  • the mobile terminal does not have a camera device, the driver's use of the mobile terminal can be monitored through the screen touch sensing device of the mobile terminal.
  • the display device of the vehicle-mounted cockpit digital platform displays the pop-up window content:
  • the mobile terminal has been accessed.
  • the central control screen displays: the mobile terminal 2 has been accessed, where the mobile terminal 2 can be a smart watch as shown in Figure 6.
  • the startup process of the mobile terminal's dynamic display driver-in-the-loop service is shown in Figure 6 (a) to Figure 6 (c).
  • the dynamic startup process of the driver-in-the-loop service is the same as (a) to (c) in Figure 5 , and will not be described in detail here for the sake of simplicity.
  • the ADS system is activated, that is, the ADS system is in running state, the DMS system in the cockpit is in running state and the "driver in the loop service" in the mobile terminal has been started.
  • Figure 7 is an interactive schematic diagram of a driver reminder method 700 provided by an embodiment of the present application. As shown in Figure 7, the method 700 includes:
  • the mobile terminal obtains the first data.
  • the first data is collected through the camera device of the mobile terminal.
  • the first data may be the driver's video stream data collected from the perspective of the location of the mobile terminal.
  • the first data is collected through a non-camera device (for example, a screen touch sensing device) of the mobile terminal.
  • the first data may be recorded data of the driver clicking on the mobile terminal.
  • the vehicle's computing platform obtains the second data and the vehicle's driving speed.
  • the vehicle's computing platform obtains the second data collected by the sensor of the mobile carrier.
  • the senor of the mobile carrier may be a camera device in the cockpit.
  • the second data may be data of the driver's line of sight tracked by the camera device in the cockpit.
  • the vehicle's computing platform obtains the traveling speed of the mobile carrier through in-vehicle communication.
  • S703 The vehicle's computing platform obtains the first data of the mobile terminal.
  • the vehicle's computing platform receives the first data from the mobile terminal through wireless communication.
  • the mobile terminal sends the first data to the vehicle cockpit digital platform through wireless communication, and the vehicle cockpit digital platform then sends the first data to the vehicle's computing platform through the in-vehicle communication network.
  • the first data may reflect whether the driver's attention is on the mobile terminal.
  • the vehicle's computing platform determines the driver's status based on the first data and/or the second data.
  • the driver status is used to indicate whether the driver is in the ring or the driver is not in the ring. If the driver is not in the ring, the driver status is also used to indicate whether it is detected that the driver's attention is on the mobile terminal.
  • driver status can be obtained through the second data, or through the first data, or through the second data and the first data.
  • Table 1 “1" represents the data used by the driver's status, and "0" represents the data not used by the driver's status.
  • Method 1 The vehicle's computing platform determines the driver's status based on the second data.
  • the second data is analyzed through an artificial intelligence (AI) algorithm to complete detection tasks such as driver presence detection, user ID account detection, head posture detection, and gaze tracking detection, thereby obtaining The driver status of whether the driver is in the loop.
  • AI artificial intelligence
  • the embodiments of the present application do not limit the specific AI algorithm.
  • it can be a convolutional neural network, a recurrent neural network, an adversarial neural network, etc.
  • the driver's abnormal driving behavior can be detected, for example, the driver's abnormal behavior such as fatigue, distraction, long-term use of mobile phones, etc., where the driver's attention is not on the driving task.
  • the driver's abnormal driving behavior such as fatigue, distraction, long-term use of mobile phones, etc.
  • the driver's status includes the driver's absence.
  • Method 2 The vehicle's computing platform determines the driver's status based on the first data.
  • the first data is obtained through the camera device of the mobile terminal, the first data is processed in a similar manner to the second data, and for the sake of simplicity, no further description is given here.
  • Method 3 The vehicle's computing platform determines the driver's status based on the second data and the first data.
  • the reliability of the first data and the second data is judged, and the driver's state is determined based on the reliable driving data, where the reliable driving data can be the first data or the second data.
  • the vehicle's computing platform fuses the second data and the first data to obtain the driver's status.
  • the second data and the first data are fused through the AI algorithm to obtain the driver's status.
  • both the second data and the first data are video stream data.
  • the AI algorithm analyzes the driver's line of sight in the second data and the first data to obtain whether the driver is in the circle. If the driver is not in the circle, further Analyze whether the driver's attention is on the mobile terminal.
  • the driver status There are several situations in the driver status. The first one is that the driver is not in the ring and it is detected that the driver's attention is on the mobile terminal. The second one is that the driver is not in the ring and the driver's attention is not detected on the mobile terminal. On, the third type of driver is in the loop.
  • processing process of the second data is similar to that in method 1, and will not be described again for the sake of simplicity.
  • the first data is obtained through the camera device of the mobile terminal, the first data is processed in a similar manner to the second data, and for the sake of brevity, no further description is given here.
  • the driver can be determined whether the driver is in the loop by recording the time of screen touch and the length of time the driver operates the screen. For example, if the length of time the driver operates the screen exceeds the threshold, it is determined that the driver is currently focusing on the mobile terminal and the driver is not in the loop.
  • the driver status is shown in Table 2.
  • the driving status data collected by the cockpit can reflect that the driver is distracted, and the driver status collected by the mobile terminal can also reflect the driving status. Members are not around.
  • the first data collected by the sensor of the mobile terminal can be a record of the driver operating the mobile terminal.
  • the record can include data type: whether the driver operates the mobile terminal; data return value: 0 or 1, where "0" can mean The driver did not operate the mobile terminal. "1" may indicate that the driver operated the mobile terminal.
  • the data return value is 1, the record also includes the length of time the driver operated the mobile terminal.
  • the first data collected by the sensor of the mobile terminal may be the driver's gaze tracking data.
  • the driver's gaze tracking data indicates that the driver's attention is not detected on the mobile terminal. superior.
  • the driver status is unknown whether the driver is in the environment. .
  • the scenario corresponding to case 2 can be that the driving status data collected through the cockpit can determine that the driver has abnormal behaviors such as fatigue and distraction, that is, the driver is not in the loop, but the driving status data collected by the mobile terminal can determine that the driver is in the loop, that is, The driver is not operating the mobile terminal.
  • the vehicle's computing platform controls the prompt device to prompt the driver to take over the mobile carrier according to the duration of the driver's absence.
  • the vehicle's computing platform control prompt device prompts the driver to take over the mobile carrier. For example, if the driver is not in the loop for more than 15 seconds, the vehicle's computing platform control prompt device prompts the driver to take over the mobile carrier.
  • the vehicle's computing platform controls the prompt device to prompt the driver to take over the mobile carrier.
  • the emergency event may be an emergency event such as ADS failure, vehicle driving conditions being outside ODD, or emergencies occurring in the driving vehicle.
  • control prompting device prompts the driver to take over the mobile carrier.
  • the prompt device may be located in the cabin, such as a display device, a sound device (eg, a speaker), an ambient light, a seat belt vibration device, etc. in the cabin, which are not limited in the embodiments of the present application.
  • the traveling speed of the mobile carrier is greater than or equal to the speed threshold.
  • the speed threshold may be 10 km/h, or other values, which are not limited in the embodiments of this application.
  • the driving speed of the mobile carrier is used as a consideration in triggering the takeover reminder, which can further improve driving safety and user experience.
  • the driver can take off his hands and eyes. At this time, the speed of the moving carrier is less than the speed threshold. Unless an emergency occurs, the driver will not receive a takeover reminder message. This improves user experience.
  • the vehicle's computing platform uses a cockpit hierarchical reminder method to control the prompt device to prompt the driver to take over the mobile carrier.
  • Figure 8 is a schematic diagram of a cockpit classification reminder method provided by an embodiment of the present application.
  • the cockpit hierarchical reminder method may include a first-level reminder for the cockpit, a second-level reminder for the cockpit, and a third-level reminder for the cockpit.
  • the first-level reminder in the cockpit includes display reminders in the cockpit
  • the second-level reminders in the cockpit include display reminders and sound reminders in the cockpit
  • the third-level reminders in the cockpit include display reminders, sound reminders and vibration reminders in the cockpit.
  • the takeover reminder message is sent to the driver through the cooperation of the takeover reminder device in the cockpit and the mobile terminal, which reduces the possibility of the vehicle having an accident due to the driver missing the takeover reminder issued in the cockpit, and improves the takeover reminder information. Passed on efficiency and timeliness, thereby improving vehicle safety.
  • the vehicle's computing platform sends the first prompt information to the mobile terminal according to the duration of the driver's absence.
  • the vehicle's computing platform when the driver status indicates that the driver is not around and an emergency event is detected, the vehicle's computing platform sends the first prompt information to the mobile terminal.
  • the first prompt information is sent to the mobile terminal.
  • the vehicle's computing platform sends the first prompt information to the mobile terminal through the mobile terminal's hierarchical reminder method.
  • the vehicle's computing platform when it is detected that the driver's attention is on the mobile terminal, the vehicle's computing platform sends the first prompt information to the mobile terminal through the mobile terminal's first hierarchical reminder method.
  • the prompt device in the cockpit and the mobile terminal issue a takeover reminder at the first time, which can avoid the situation when the driver's attention is on the mobile terminal. Missing the takeover reminder displayed on the takeover reminder device in the cockpit can improve the effectiveness of the takeover reminder and the timeliness of the driver receiving the takeover reminder, thereby improving the safety of the vehicle.
  • the vehicle's computing platform sends driver reminder information to the mobile terminal through the mobile terminal's second hierarchical reminder method.
  • a takeover reminder is issued on the mobile terminal takeover reminder device through vibration or sound, which can reduce the risk of the driver's attention being focused on the mobile terminal.
  • the possibility of missing the takeover reminder when the driver is not on the mobile terminal and is not in the loop further improves the effectiveness of the takeover reminder and the timeliness of the driver receiving the takeover reminder, thereby improving the safety of the vehicle. For example, if the driver status indicates that the driver is not around, or the driver exhibits abnormal behaviors such as fatigue or distraction, but the driver does not operate the mobile terminal, relying solely on the display reminder in the cockpit may not allow the driver to receive the takeover reminder in time. Therefore, the additional vibration reminder or sound reminder of the mobile terminal can improve the timeliness and effectiveness of the driver receiving the takeover reminder, thereby improving the safety of the vehicle.
  • Figure 9 is a schematic diagram of a hierarchical reminder method for a mobile terminal provided by an embodiment of the present application.
  • the first hierarchical reminder method of the mobile terminal includes a first-level reminder of the mobile terminal, a second-level reminder of the mobile terminal, and a third-level reminder of the mobile terminal.
  • the first-level reminder of the mobile terminal may include a mobile terminal display reminder
  • the second-level reminder of the mobile terminal may include a mobile terminal display reminder and a sound reminder, or include a mobile terminal display reminder and a vibration reminder
  • the third-level reminder of the mobile terminal includes a mobile terminal
  • the terminal displays reminders, sound reminders and vibration reminders.
  • the second hierarchical reminder method of the mobile terminal includes a first-level reminder of the mobile terminal, a second-level reminder of the mobile terminal, and a third-level reminder of the mobile terminal.
  • the first-level reminder of the mobile terminal may include a vibration reminder of the mobile terminal or the sound reminder of the mobile terminal
  • the second-level reminder of the mobile terminal may include a vibration reminder and a sound reminder of the mobile terminal
  • the third-level reminder of the mobile terminal includes the mobile terminal. Display reminders, sound reminders and vibration reminders.
  • the sound reminder may include a voice reminder with specific prompt content, for example, a voice reminder of "Please take over the vehicle", or it may be a voice reminder without specific prompt content, such as a sound reminder of "beep-beep".
  • the embodiment of the present application does not limit the type of sound reminder.
  • the sound reminder in the second level reminder can be stronger than the sound reminder in the first level reminder. For example, if it is a voice reminder that includes specific content, then the volume of the voice reminder can be increased. If it is a voice reminder that does not include specific content, the volume of the voice reminder can be increased. If you want to provide sound reminders for content, you can increase the frequency of voice reminders from the short sound of "beep-beep" to the rapid sound of "beep-beep-beep".
  • FIG 10 is an interactive schematic diagram of another driver reminder method 1000 provided by an embodiment of the present application.
  • the method 1000 includes:
  • the mobile terminal obtains the first data.
  • S1001 is similar to S701, please refer to S701 for details.
  • the vehicle's computing platform obtains the second data and the vehicle's driving speed.
  • S1002 is similar to S702, please refer to S702 for details.
  • the mobile terminal obtains the second data of the vehicle's computing platform and the vehicle's driving speed.
  • the mobile terminal obtains the second data from the vehicle's computing platform through wireless communication.
  • the mobile terminal determines the driver's status based on the first data and/or the second data.
  • the mobile terminal sends first prompt information to the computing platform of the vehicle, where the first prompt information is used to instruct the driver to take over the mobile carrier.
  • the process of determining the driver's status can be implemented by the cloud server.
  • the cloud server sends the driver's status to the vehicle's computing platform.
  • the vehicle's computing platform controls the prompt device to prompt the driver to take over the vehicle and sends it to the mobile terminal. First prompt information.
  • Figure 11 is a schematic diagram of a driving behavior assessment method 1100 based on driver reminder records provided by an embodiment of the present application.
  • the vehicle's computing platform records the response parameters of the driver taking over the vehicle.
  • response parameters include the number of takeover reminders in a driving task and the corresponding takeover reminder level when the driver takes over the vehicle in each takeover reminder.
  • a driving task can be a normal driving task from point A to point B, or it can also be an automatic parking task.
  • the number of takeover reminders indicates the number of times the driver is not in the loop during a driving task.
  • the higher the corresponding takeover reminder level when the driver takes over the vehicle the longer the driver's reaction time when taking over the vehicle is, that is, the more dangerous the driver's driving behavior is.
  • the vehicle's computing platform sends the response parameters to the cloud server.
  • the cloud server receives the response parameters and determines the driver's driving behavior score based on the response parameters.
  • the driver's driving behavior score is used to evaluate the safety of the driver's driving behavior.
  • the driving behavior score will be lower. If the corresponding takeover reminder level is higher when the driver takes over the vehicle in each takeover reminder, the driver behavior score will be lower.
  • the driver reminder issued based on this will be more accurate, allowing the driver to take over the vehicle.
  • the response parameters are recorded more accurately to improve the accuracy of subsequent information push.
  • the cloud server pushes customized information to the vehicle based on the driver's driving behavior score.
  • driver's driving behavior score is low, driving safety education information can be pushed to the vehicle, and if the driver's driving behavior score is high, new autonomous driving functions can be pushed to the vehicle.
  • the driver reminder information can be delivered to the driver through the devices in the cockpit and the mobile terminal through the hierarchical reminder method of the cockpit and the hierarchical reminder method of the mobile terminal respectively.
  • the embodiment of the present application does not limit the timing of issuing each level of reminder in the hierarchical reminder method of the cockpit and the timing of issuing each level of reminder in the hierarchical reminder method of the mobile terminal.
  • driver reminder information in conjunction with FIGS. 12 and 13 in which the timing of issuing each level of reminder in the cockpit in the cockpit hierarchical reminder method is the same as the timing of issuing each level of reminder in the mobile terminal in the hierarchical reminder method of the mobile terminal. sending process.
  • Figures 12 and 13 take the driver reminder method shown in Figure 7 as an example to illustrate.
  • the delivery of time-triggered driver reminder information is taken as an example for illustration.
  • Figure 12 is a schematic diagram of a driver reminder information sending process 1200 provided by an embodiment of the present application.
  • the vehicle's computing platform control prompt device prompts the driver to take over the mobile carrier and sends the first prompt information to the mobile terminal.
  • S1202 When the vehicle's computing platform controls the prompt device to issue a first-level reminder of the cockpit, it synchronously sends the first prompt information to the mobile terminal through the first-level reminder of the mobile terminal.
  • the vehicle's computing platform synchronously passes the third level of the mobile terminal when the control prompt device issues the first level reminder of the cockpit.
  • the first level reminder sends the first prompt information to the mobile terminal.
  • the vehicle's computing platform synchronizes the control prompt device to issue the first level reminder of the cockpit through the third level of the mobile terminal.
  • the first level reminder sends the first prompt information to the mobile terminal.
  • This scenario may be that the duration of the driver's absence from the loop has not accumulated to the first threshold, but at this time, an emergency event is detected and the driver needs to take over the vehicle.
  • the vehicle's computing platform controls the display device in the vehicle cockpit to display the takeover reminder information, and synchronously sends the display reminder to the mobile terminal.
  • the vehicle's computing platform controls the display device in the vehicle cockpit to display the takeover reminder information and synchronously sends a vibration reminder to the mobile terminal; or, the vehicle's computing platform The display device in the vehicle cockpit is controlled to display the takeover reminder information and synchronously send an audio reminder to the mobile terminal.
  • S1201 When the driver has taken over the vehicle, S1201 is executed, that is, the driver reminder enters a new reminder cycle.
  • the control prompt device When the driver status indicates that the driver is not in the loop, the control prompt device prompts the driver to take over the mobile carrier and sends the first prompt information to the mobile terminal. .
  • S1204 is executed.
  • the vehicle's computing platform sends a second-level reminder to the cockpit in the control prompt device, and synchronously sends the first prompt information to the mobile terminal through the second-level reminder of the mobile terminal.
  • the vehicle's computing platform issues a second-level reminder in the cockpit in the control prompt device, and synchronously sends the mobile terminal to the mobile terminal. second level reminder.
  • a second threshold for example, 30 seconds
  • the vehicle's computing platform issues a second-level reminder to the cockpit in the control prompt device.
  • the second level reminder of the mobile terminal is synchronously sent to the mobile terminal.
  • This scenario can be that when an emergency event is detected, the driver does not take over the vehicle under the first-level reminder from the cockpit and mobile terminal. At this time, the duration of the driver's absence from the loop accumulates to the fourth threshold, triggering the cockpit and mobile terminal. second level reminder.
  • the vehicle's computing platform controls the prompt device to issue a second-level reminder in the cockpit, and synchronously sends the first-level reminder to the mobile terminal through the second-level reminder of the mobile terminal.
  • the prompt information may be: first, the vehicle's computing platform controls the display device in the vehicle cockpit to display the takeover reminder information and the sound device in the vehicle cockpit to issue a sound reminder, and synchronously sends the display reminder and sound reminder to the mobile terminal. Or, secondly, the vehicle's computing platform controls the display device in the vehicle cockpit to display the takeover reminder information and the sound device in the vehicle cockpit to issue a sound reminder, and simultaneously sends the display reminder and vibration reminder to the mobile terminal.
  • the vehicle's computing platform controls the prompt device to issue a second-level reminder in the cockpit, and synchronously sends a first-level reminder to the mobile terminal through the first-level reminder of the mobile terminal.
  • the prompt information may be: first, the vehicle's computing platform controls the display device in the vehicle cockpit to display the takeover reminder information and the sound device in the vehicle cockpit to issue a sound reminder, and simultaneously sends a vibration reminder and a sound reminder to the mobile terminal.
  • S1201 is executed, which means the driver reminder enters a new reminder cycle.
  • S1206 is executed.
  • the vehicle's computing platform sends a third-level reminder of the cockpit to the control prompt device, and synchronously sends the first prompt information to the mobile terminal through the third-level reminder of the mobile terminal.
  • the vehicle's computing platform issues a third-level reminder in the cockpit through the control prompt device, and synchronously passes the third-level reminder of the mobile terminal.
  • the level reminder sends the first prompt information to the mobile terminal.
  • the vehicle's computing platform issues a third-level reminder in the cockpit through the control prompt device.
  • the third-level reminder of the mobile terminal is synchronously sent to the mobile terminal.
  • This scenario can be that when an emergency event is detected, the driver does not take over the vehicle under the first-level reminder and the second-level reminder from the cockpit and the mobile terminal. At this time, the duration of the driver's absence from the loop accumulates to the fifth threshold, Trigger third-level reminders in the cockpit and mobile terminal.
  • the vehicle's computing platform controls the display device in the vehicle cabin to display the takeover reminder information, and the sound in the vehicle cabin
  • the device emits a sound reminder and the seat belt in the vehicle cockpit emits a vibration reminder, and simultaneously sends the display reminder, sound reminder and vibration reminder of the mobile terminal to the mobile terminal.
  • S1201 is executed, which means the driver reminder enters a new reminder cycle.
  • S1208 is executed, and the automatic driving system enters a penalty state, for example, the automatic driving system forces braking.
  • the timeliness and effectiveness of sending driver reminder information can be enhanced, and driving safety can be improved.
  • driver reminder information in the cockpit and driver reminder information on the mobile terminal will be described in detail below with reference to Figure 13. Take the delivery of time-triggered driver reminder information as an example to illustrate
  • Figure 13 is a schematic diagram of human-computer interaction between driver reminder information in the cockpit and driver reminder information on a mobile terminal provided by an embodiment of the present application.
  • Figure 13 shows the driver's state including the driver's attention on the mobile terminal.
  • the display device of the cockpit and the screen of the mobile terminal synchronously display the first-level reminder, for example, displayed on the display device of the cockpit
  • the reminder of "Please take over the vehicle” is synchronized with the pop-up window of "Please take over the vehicle” displayed on the screen of the mobile terminal.
  • the computing platform of the vehicle can control the display device in the vehicle to display the first-level reminder in the cabin through in-vehicle communication (eg, automotive serial bus or vehicle Ethernet network).
  • the vehicle's computing platform sends the first-level reminder to the mobile terminal through the wireless network, or the vehicle's computing platform sends the first-level reminder to the vehicle cockpit digital platform through in-car communication, and the vehicle cockpit digital platform then sends the first-level reminder through wireless communication.
  • the first level reminder is sent to the mobile terminal.
  • the display device, sound device and vibration device of the cockpit (for example, Active seat belt vibration) issues a three-level reminder, and the screen, sound device and vibration device of the mobile terminal simultaneously issue a three-level reminder.
  • the reminder "Please take over the vehicle” is displayed on the display device of the cockpit
  • the reminder "Please take over the vehicle” is issued on the sound device of the cockpit
  • the driver's seat belt emits a vibration reminder
  • "Please take over the vehicle” is displayed on the screen of the mobile terminal simultaneously.
  • "Please take over the vehicle” pop-up window the sound device of the mobile terminal emits a voice reminder of "Please take over the vehicle", and the vibration device of the mobile terminal emits a vibration reminder.
  • the automatic driving system When the driver fails to take over the vehicle in time under the third-level reminder, the automatic driving system enters a penalty state, for example, the automatic driving system forces braking.
  • the timing of issuing each level of reminder in the cockpit in the cockpit hierarchical reminder method and the timing of issuing each level of reminder in the mobile terminal in the hierarchical reminder method of the mobile terminal may be different.
  • the prompting device is first controlled to issue a first-level reminder in the cockpit.
  • the first-level reminder from the mobile terminal is then sent to the mobile terminal.
  • the second-level reminder in the cockpit, the second-level reminder in the mobile terminal, the third-level reminder in the cockpit and the third-level reminder in the mobile terminal will be issued in sequence.
  • the autonomous driving system enters a penalty state.
  • the timing of issuing the cockpit takeover reminder information and the timing of issuing the mobile terminal takeover reminder information are staggered, which can avoid multiple devices sending takeover reminders at the same time. Instead, different devices send out reminder information at staggered times. While the total reminder duration remains unchanged, increase the frequency of taking over reminders. Compared with the current takeover reminder that is only based on the equipment in the cockpit, even if the current reminder frequency is increased, it may not be possible to effectively receive the takeover reminder information and take over the vehicle in a timely manner after missing the takeover reminder issued in the cockpit. , improve driving safety.
  • the timing of sending takeover reminder messages is staggered, which can avoid Drivers receive too many reminder messages at one time and become bored, which can improve driving safety while also improving user experience.
  • the first level reminder of the cockpit and the first level reminder of the mobile terminal are issued at the same time. If the driver does not take over the vehicle, the second level reminder of the cockpit is issued in sequence, and then the third level reminder of the mobile terminal is issued. Second-level reminder, if the driver does not take over the vehicle under the second-level reminder of the cockpit and mobile terminal, then the third-level reminder of the cockpit and the third-level reminder of the mobile terminal will be issued simultaneously. If the driver is in the multi-device multi-level When the driver fails to take over the vehicle despite being reminded, the autonomous driving system enters a penalty state.
  • the driver when the driver is not in the ring, the driving speed meets the speed threshold condition, and when an emergency occurs or the duration of the driver's absence reaches the first threshold, the driver can be reminded through multiple devices at the first time The driver takes over, thereby preventing the driver from missing the takeover reminder message at the first time, thereby improving driving safety.
  • the above content is the driver reminder method according to the embodiment of the present application.
  • the driver reminder device will be described in detail below with reference to Figures 14 and 15. It should be understood that the description of the device embodiments and the description of the method embodiments correspond to each other. Therefore, for content that is not described in detail, please refer to the above method embodiments, and for the sake of brevity, they will not be described again here.
  • Figure 14 is a schematic diagram of a driver reminder device provided by an embodiment of the present application. As shown in Figure 14, the device includes: an acquisition unit 1401, a determination unit 1402 and a control unit 1403.
  • the acquisition unit 1401 is configured to acquire the first data collected by the sensor from the mobile terminal when the mobile carrier is in the driving state, and acquire the second data collected by the sensor in the cockpit of the mobile carrier;
  • the determining unit 1402 is used to determine the driver's status according to the first data and the second data;
  • the control unit 1403 is configured to control the prompt device to prompt the driver to take over the mobile carrier when the driver status indicates that the driver is not in the loop.
  • the device further includes a communication unit 1404, which is configured to send first prompt information to the mobile terminal, where the first prompt information is used to instruct the driver to take over the mobile carrier.
  • a communication unit 1404 which is configured to send first prompt information to the mobile terminal, where the first prompt information is used to instruct the driver to take over the mobile carrier.
  • the device further includes a first detection unit 1405.
  • the communication unit 1404 is configured to send the first prompt information to the mobile terminal.
  • control unit 1403 is configured to control the prompting device to prompt the driver to take over the mobile carrier according to the duration of the driver's absence from the ring.
  • the device also includes a second detection unit 1406.
  • the control unit 1403 is used to control the prompting device to prompt the driver. Take over the mobile carrier.
  • the determining unit 1402 is also configured to determine that the traveling speed of the mobile carrier is greater than or equal to the speed threshold before the control prompt device prompts the driver to take over the mobile carrier.
  • the acquisition unit 1401 may be the computing platform in FIG. 1 or a processing circuit, processor or controller in the computing platform. Taking the acquisition unit 1401 as the processor 151 in the computing platform as an example, when the mobile carrier is in the driving state, the processor 151 can acquire the first data collected by the sensors of the mobile terminal and the second data collected by the sensors in the mobile cockpit.
  • the determining unit 1402 may be the computing platform in FIG. 1 or a processing circuit, processor or controller in the computing platform. Taking the determination unit 1402 as the processor 152 in the computing platform as an example, the processor 152 can determine the driver's status according to the first data and the second data.
  • control unit 1403 may be the computing platform in FIG. 1 or a processing circuit, processor or controller in the computing platform. Taking the control unit 1403 as the processor 153 in the computing platform as an example, when the driver status indicates that the driver is not around, the processor 152 can control the prompt device to prompt the driver to take over the mobile carrier.
  • the above functions implemented by the acquisition unit 1401, the determination unit 1402 and the control unit 1403 can be implemented by different processors, or can also be implemented by the same processor, which is not limited in the embodiment of the present application.
  • each unit in the above device is only a division of logical functions. In actual implementation, it can be fully or partially integrated into a physical entity, or it can also be physically separated.
  • the unit in the device can be implemented in the form of a processor calling software; for example, the device includes a processor, the processor is connected to a memory, instructions are stored in the memory, and the processor calls the instructions stored in the memory to implement any of the above methods.
  • the processor is, for example, a general-purpose processor, such as a CPU or a microprocessor
  • the memory is a memory within the device or a memory outside the device.
  • the units in the device can be implemented in the form of hardware circuits, and some or all of the functions of the units can be implemented through the design of the hardware circuits, which can be understood as one or more processors; for example, in one implementation,
  • the hardware circuit is an ASIC, which realizes the functions of some or all of the above units through the design of the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented through PLD, taking FPGA as an example. It can include a large number of logic gate circuits, and the connection relationships between the logic gate circuits can be configured through configuration files to realize the functions of some or all of the above units. All units of the above device may be fully realized by the processor calling software, or may be fully realized by hardware circuits, or part of the units may be realized by the processor calling software, and the remaining part may be realized by hardware circuits.
  • the processor is a circuit with signal processing capabilities.
  • the processor may be a circuit with instruction reading and execution capabilities, such as a CPU, a microprocessor, a GPU, or DSP, etc.; in another implementation, the processor can realize certain functions through the logical relationship of the hardware circuit. The logical relationship of the hardware circuit is fixed or can be reconstructed.
  • the processor is a hardware circuit implemented by ASIC or PLD. For example, FPGA.
  • the process of the processor loading the configuration file and realizing the hardware circuit configuration can be understood as the process of the processor loading instructions to realize the functions of some or all of the above units.
  • it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as NPU, TPU, DPU, etc.
  • each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA , or a combination of at least two of these processor forms.
  • processors or processing circuits
  • each unit in the above device may be integrated together in whole or in part, or may be implemented independently. In one implementation, these units are integrated together and implemented as a SOC.
  • the SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device.
  • the at least one processor may be of different types, such as a CPU and an FPGA, or a CPU and an artificial intelligence processor. CPU and GPU etc.
  • FIG. 15 is a schematic diagram of the hardware structure of a driver reminder device provided by the implementation of this application.
  • the driver reminder device 1500 shown in FIG. 15 may include: a memory 1510, a processor 1520, and a communication interface 1530.
  • the memory 1510, the processor 1520, and the communication interface 1530 are connected through an internal connection path.
  • the memory 1510 is used to store instructions
  • the processor 1520 is used to execute the instructions stored in the memory 1520 to control the input/output interface 1530 to receive/send. at least some parameters of the second channel model.
  • the memory 1510 can be coupled with the processor 1520 through an interface or integrated with the processor 1520 .
  • the driver reminder device 1500 may be the computing platform 150 in FIG. 1 .
  • the above-mentioned communication interface 1530 uses a transceiver device such as but not limited to a transceiver to implement communication between the communication device 1500 and other devices or communication networks.
  • the above-mentioned communication interface 1530 may also include an input/output interface.
  • each step of the above method can be completed by instructions in the form of hardware integrated logic circuits or software in the processor 1520 .
  • the method disclosed in conjunction with the embodiments of the present application can be directly implemented by a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.
  • the software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other mature storage media in this field.
  • the storage medium is located in the memory 1510.
  • the processor 1520 reads the information in the memory 1510 and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.
  • the memory may include read-only memory and random access memory, and provide instructions and data to the processor.
  • Part of the processor may also include non-volatile random access memory.
  • the processor may also store information about the device type.
  • the size of the sequence numbers of the above-mentioned processes does not mean the order of execution.
  • the execution order of each process should be determined by its functions and internal logic, and should not be used in the embodiments of the present application.
  • the implementation process constitutes any limitation.
  • the embodiment of the present application also provides a display system, which includes a computing platform and a prompting device.
  • the computing platform may include a driver reminding device 1500 .
  • the prompt device may include at least one of a display device, a sound device (for example, a speaker), a seat belt vibration device, and an ambient light.
  • An embodiment of the present application also provides a device, which includes a processing unit and a storage unit, where the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the device executes the above driver reminder method.
  • Embodiments of the present application also provide a computer-readable medium.
  • the computer-readable medium stores program code.
  • the computer program code When the computer program code is run on a computer, it causes the computer to execute the above method.
  • An embodiment of the present application also provides a chip, including: at least one processor and a memory, the at least one processor and the memory
  • the memory is coupled to read and execute instructions in the memory to perform the above method.
  • An embodiment of the present application also provides a vehicle, including: at least one processor and a memory, the at least one processor being coupled to the memory and configured to read and execute instructions in the memory to perform the above method.
  • a component may be, but is not limited to, a process, a processor, an object, an executable file, a thread of execution, a program and/or a computer running on a processor.
  • applications running on the computing device and the computing device may be components.
  • One or more components can reside in a process and/or thread of execution and a component can be localized on one computer and/or distributed between 2 or more computers. Additionally, these components can execute from various computer-readable media having various data structures stored thereon.
  • a component may, for example, be based on a signal having one or more data packets (eg, data from two components interacting with another component, a local system, a distributed system, and/or a network, such as the Internet, which interacts with other systems via signals) Communicate through local and/or remote processes.
  • data packets eg, data from two components interacting with another component, a local system, a distributed system, and/or a network, such as the Internet, which interacts with other systems via signals
  • the disclosed systems, methods and devices can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented.
  • the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the apparatus described in various embodiments of this application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code. .

Abstract

一种驾驶员提醒方法、装置、系统和移动载体(100)。该方法包括:在移动载体(100)处于驾驶状态时,获取来自移动终端的传感器采集的第一数据,且获取移动载体(100)的座舱内的传感器采集的第二数据。根据第一数据和第二数据,确定驾驶员状态。在驾驶员状态指示驾驶员不在环时,控制提示装置提示驾驶员接管移动载体(100)。

Description

一种驾驶员提醒方法、装置、系统和移动载体
本申请要求于2022年09月01日提交中国专利局、申请号为202211069559.0、申请名称为“一种驾驶员提醒方法、装置、系统和移动载体”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及智能驾驶领域,更具体地,涉及一种驾驶员提醒方法、装置、系统和移动载体。
背景技术
随着车辆的智能化发展,国际自动机工程师学会(society of automotive engineers international,SAE international)将自动驾驶技术分为L0至L5共六个等级。全自动驾驶是车辆的智能化发展的目标。然而,在这个发展过程中,从L3等级以下的自动驾驶仍然属于辅助驾驶,驾驶员仍然是事故发生的主要责任主体,驾驶员应当时刻关注驾驶任务及车辆周边交通场景。因此,驾驶员监测系统(driver monitor system,DMS)和分级提醒系统,是L3等级以下的自动驾驶强制准入功能系统。
当自动驾驶系统故障,或者车辆驾驶条件处于运行设计场景(operation design domain,ODD)之外,或者车辆处于紧急情况之下,通过驾驶员监测系统实时监测在环驾驶员的情况,并通过分级提醒系统可以及时提醒驾驶员接管车辆,以保证行车安全。
目前的驾驶员监测和分级提醒系统主要用于L3等级以下的高级驾驶辅助系统(advanced driving assistant system,ADAS),主要通过车载仪表显示接管提醒信息来提醒驾驶员接管车辆。然而,如果驾驶员注意力在移动终端(例如,手机)上,那么驾驶员可能会由于注意力的转移,错过基于车载仪表的驾驶员监测和分级提醒系统发出的接管提醒信息,从而增加了交通事故发生的风险。
因此,在自动驾驶系统(autonomous driving system,ADS)启动过程中,当需要驾驶员接管车辆时,如何确保驾驶员可以接收到提醒信息,有助于驾驶员及时有效地接管车辆,成为一个亟待解决的问题。
发明内容
本申请实施例提供一种驾驶员提醒方法、装置、系统和移动载体,可以保证驾驶员可以接收到提醒信息,从而有助于驾驶员及时有效地接管移动载体,进而提高了移动载体的安全性。
第一方面,提供了一种驾驶员提醒方法,该方法包括,在移动载体处于驾驶状态时,获取来自移动终端的传感器采集的第一数据,且获取移动载体的座舱内的传感器采集的第二数据;根据第一数据和第二数据,确定驾驶员状态;在驾驶员状态指示驾驶员不在环时,控制提示装置提示驾驶员接管移动载体。
在本申请中,通过移动终端获得的第一数据可以弥补座舱采集的第二数据在视角上存在的不足,多元数据确定的驾驶状态,可以提高对驾驶员状态判断的准确性。基于准确的驾驶员状态,既能防止由于DMS的误检发出的驾驶员管提醒,改善用户体验,又能有助于通过多设备发出驾驶员提醒,降低驾驶员错过提醒带来的安全隐患,从而提高行驶安全性。
结合第一方面,在第一方面的某些实现方式中,该方法还包括:向移动终端发送第一提示信息,第一提示信息用于指示驾驶员接管移动载体。
在本申请中,通过座舱内接管提示装置和移动终端协作向驾驶员发出接管提醒信息,降低驾驶员由于错过座舱内发出的接管提醒而使得移动载体发生事故的可能性,提高了接管提醒信息传递的有效性和及时性,从而提高了移动载体的安全性。
结合第一方面,在第一方面的某些实现方式中,向移动终端发送第一提示信息包括:在检测到驾驶员的注意力在移动终端上时,向移动终端发送第一提示信息。
在本申请中,当驾驶员不在环并且驾驶员注意力在移动终端上时,通过座舱内提示装置和移动终 端在第一时间发出接管提醒,可以避免驾驶员注意力在移动终端上时,错过座舱内接管提醒装置上显示的接管提醒,可以提高接管提醒发出的有效性和驾驶员接收到接管提醒的及时性,从而提高了移动载体的安全性。
在一些可能的实现方式中,在检测到驾驶员的注意力在移动终端上时,通过移动终端的第一分级提醒方式,向移动终端发送第一提示信息。
示例性地,移动终端的第一分级提醒方式包括移动终端的第一级提醒、移动终端的第二级提醒和移动终端的第三级提醒。其中,移动终端的第一级提醒可以包括移动终端显示提醒;移动终端的第二级提醒可以包括移动终端显示提醒和声音提醒,或者包括移动终端显示提醒和振动提醒;移动终端三级提醒包括移动终端显示提醒、声音提醒和振动提醒。
在一些可能的实现方式中,在驾驶员状态指示驾驶员不在环,并且未检测到驾驶员的注意力在移动终端上时,通过移动终端的第二分级提醒方式,向移动终端发送第一提示信息。
示例性地,移动终端的第二分级提醒方式包括移动终端的第一级提醒、移动终端的第二级提醒和移动终端的第三级提醒。其中,移动终端的第一级提醒可以包括移动终端的振动提醒,或者移动终端的声音提醒;移动终端的第二级提醒可以包括移动终端的振动提醒和声音提醒;移动终端三级提醒包括移动终端的显示提醒、声音提醒和振动提醒。
在本申请中,当驾驶员不在环并且未检测到驾驶员注意力在移动终端上时,通过振动方式或者声音方式在移动终端接管提醒装置上发出接管提醒,可以降低当驾驶员注意力不在移动终端上且驾驶员不在环时错过接管提醒的可能性,进一步提高接管提醒发出的有效性和驾驶员接收到接管提醒的及时性,从而提高了移动载体的安全性。
结合第一方面,在第一方面的某些实现方式中,控制提示装置提示驾驶员接管移动载体包括:根据驾驶员不在环的持续时长,控制提示装置提示驾驶员接管移动载体。
在一些可能的实现方式中,以驾驶员不在环持续时长的等级,通过座舱分级提醒方式控制提示装置提示驾驶员接管移动载体。
例如,当驾驶员不在环的持续时长达到第一阈值(例如,15s),通过座舱的第一级提醒,控制座舱内的显示装置提示驾驶员接管移动载体;如果驾驶员在座舱的第一级提醒下未接管移动载体,那么当驾驶员不在环持续时长累计到第二阈值(例如,30s)时,通过座舱的第二级提醒,控制座舱内的显示装置和发声装置提示驾驶员接管移动载体;如果驾驶员在在座舱的第二级提醒下未接管移动载体,那么当驾驶员不在环持续时长累计到第三阈值(例如,50s)时,通过座舱的第三级提醒,控制座舱内的显示装置、发声装置和安全带振动装置提示驾驶员接管移动载体。
结合第一方面,在第一方面的某些实现方式中,控制提示装置提示驾驶员接管移动载体包括:根在驾驶员状态指示驾驶员不在环,且检测到紧急事件发生时,控制提示装置提示驾驶员接管移动载体。
在一些可能的实现方式中,在检测到紧急事件发生后,通过座舱的第一级提醒,控制显示装置提示驾驶员接管移动载体,如果驾驶员在座舱的第一级提醒下未接管移动载体,以驾驶员不在环持续时长的等级,通过座舱分级提醒方式控制提示装置提示驾驶员接管移动载体。
示例性地,如果驾驶员在座舱的第一级提醒方式下未接管移动载体,驾驶员不在环持续时长达到第四阈值(例如,20s),通过座舱的第二级提醒,控制座舱内的显示装置和发声装置提示驾驶员接管移动载体;如果驾驶员在在座舱的第二级提醒下未接管移动载体,那么当驾驶员不在环持续时长累计到第五阈值(例如,35s)时,通过座舱的第三级提醒,控制座舱内的显示装置、发声装置和安全带振动装置提示驾驶员接管移动载体。
在本申请中,当驾驶员不在环时,如果通过紧急事件触发座舱的第一级提醒之后,可以以驾驶员不在环持续时长等级,控制提示装置提示驾驶员接管移动载体,可以及时有效地将接管提醒信息传递给驾驶员,从而提高行驶安全性。
结合第一方面,在第一方面的某些实现方式中,该方法还包括:在控制提示装置提示驾驶员接管移动载体之前,确定移动载体的行驶速度大于或等于速度阈值。
在本申请中,将移动载体的行驶速度作为接管提醒触发的考虑因素,可以进一步提高行驶安全性,以及用户体验。例如,当自动驾驶系统开启自动泊车功能时,驾驶员可以脱手脱眼,此时移动载体的速度是小于速度阈值的,除非有突发事件发生,不然驾驶员不会收到接管提醒信息,这样可以提高用 户体验。
结合第一方面,在第一方面的某些实现方式中,控制提示装置提示驾驶员接管移动载体和向移动终端发送第一提示信息同步进行。
结合第一方面,在第一方面的某些实现方式中,该方法还包括:记录驾驶员接管移动载体的响应参数;向云端服务器发送响应参数,其中,响应参数用于确定驾驶员的驾驶行为分数,驾驶行为分数用于评估驾驶员的行为安全性;接收来自云端服务器的定制信息,定制信息是根据驾驶行为分数确定的。
在本申请中,当驾驶员不在环的结果是通过座舱内驾驶员监测装置和移动终端采集的数据共同得到的时,那么基于此发出的驾驶员提醒越精准,从而使得驾驶员接管移动载体的响应参数记录地更加准确,以便于提高后续信息推送的准确性。
第二方面,提供了一种驾驶员提醒装置,该装置包括获取单元、确定单元和控制单元。该获取单元用于,在移动载体处于驾驶状态时,获取来自移动终端的传感器采集的第一数据,且获取移动载体的座舱内的传感器采集的第二数据。该确定单元用于,根据第一数据和第二数据,确定驾驶员状态。该控制单元用于,在驾驶员状态指示驾驶员不在环时,控制提示装置提示驾驶员接管移动载体。
在本申请中,通过移动终端获得的第一数据可以弥补座舱采集的第二数据在视角上存在的不足,多元数据确定的驾驶状态,可以提高对驾驶员状态判断的准确性。基于准确的驾驶员状态,既能防止由于DMS的误检发出的驾驶员提醒,改善用户体验,又能有助于通过多设备发出驾驶员提醒,降低驾驶员错过接管提醒带来的安全隐患,从而提高行驶安全性。
结合第二方面,在第二方面的某些实现方式中,该装置还包括通信单元,该通信单元用于,向移动终端发送第一提示信息,第一提示信息用于指示驾驶员接管移动载体。
在本申请中,通过座舱内接管提示装置和移动终端协作向驾驶员发出接管提醒信息,降低驾驶员由于错过座舱内发出的接管提醒而使得移动载体发生事故的可能性,提高了接管提醒信息传递的有效性和及时性,从而提高了移动载体的安全性。
结合第二方面,在第二方面的某些实现方式中,该装置还包括第一检测单元,在第一检测单元检测到驾驶员的注意力在移动终端上时,该通信单元具体用于,向移动终端发送第一提示信息。
在本申请中,当驾驶员不在环并且驾驶员注意力在移动终端上时,通过座舱内提示装置和移动终端在第一时间发出接管提醒,可以避免驾驶员注意力在移动终端上时,错过座舱内接管提醒装置上显示的接管提醒,可以提高接管提醒发出的有效性和驾驶员接收到接管提醒的及时性,从而提高了移动载体的安全性。
在一些可能的实现方式中,在第一检测单元检测到驾驶员的注意力在移动终端上时,该通信单元用于,通过移动终端的第一分级提醒方式,向移动终端发送第一提示信息。
示例性地,移动终端的第一分级提醒方式包括移动终端的第一级提醒、移动终端的第二级提醒和移动终端的第三级提醒。其中,移动终端的第一级提醒可以包括移动终端显示提醒;移动终端的第二级提醒可以包括移动终端显示提醒和声音提醒,或者包括移动终端显示提醒和振动提醒;移动终端三级提醒包括移动终端显示提醒、声音提醒和振动提醒。
在一些可能的实现方式中,在驾驶员状态指示驾驶员不在环,并且第一检测单元未检测到驾驶员的注意力在移动终端上时,通信单元用于,通过移动终端的第二分级提醒方式,向移动终端发送第一提示信息。
示例性地,移动终端的第二分级提醒方式包括移动终端的第一级提醒、移动终端的第二级提醒和移动终端的第三级提醒。其中,移动终端的第一级提醒可以包括移动终端的振动提醒,或者移动终端的声音提醒;移动终端的第二级提醒可以包括移动终端的振动提醒和声音提醒;移动终端三级提醒包括移动终端的显示提醒、声音提醒和振动提醒。
在本申请中,当驾驶员不在环并且未检测到驾驶员注意力在移动终端上时,通过振动方式或者声音方式在移动终端接管提醒装置上发出接管提醒,可以降低当驾驶员注意力不在移动终端上且驾驶员不在环时错过接管提醒的可能性,进一步提高接管提醒发出的有效性和驾驶员接收到接管提醒的及时性,从而提高了移动载体的安全性。
结合第二方面,在第二方面的某些实现方式中,该控制单元用于:根据驾驶员不在环的持续时长, 控制提示装置提示驾驶员接管移动载体。
在一些可能的实现方式中,以驾驶员不在环持续时长的等级,控制单元用于,通过座舱分级提醒方式控制提示装置提示驾驶员接管移动载体。
例如,当驾驶员不在环的持续时长达到第一阈值(例如,15s),通过座舱的第一级提醒,控制座舱内的显示装置提示驾驶员接管移动载体;如果驾驶员在座舱的第一级提醒下未接管移动载体,那么当驾驶员不在环持续时长累计到第二阈值(例如,30s)时,通过座舱的第二级提醒,控制座舱内的显示装置和发声装置提示驾驶员接管移动载体;如果驾驶员在在座舱的第二级提醒下未接管移动载体,那么当驾驶员不在环持续时长累计到第三阈值(例如,50s)时,通过座舱的第三级提醒,控制座舱内的显示装置、发声装置和安全带振动装置提示驾驶员接管移动载体。
结合第二方面,在第二方面的某些实现方式中,该装置还包括第二检测单元,在驾驶员状态指示驾驶员不在环,且第二检测单元检测到紧急事件发生时,控制单元用于,控制提示装置提示驾驶员接管移动载体。
在一些可能的实现方式中,控制单元具体用于,在检测到紧急事件发生后,通过座舱的第一级提醒,控制显示装置提示驾驶员接管移动载体,如果驾驶员在座舱的第一级提醒下未接管移动载体,以驾驶员不在环持续时长的等级,通过座舱分级提醒方式控制提示装置提示驾驶员接管移动载体。
示例性地,如果驾驶员在座舱的第一级提醒方式下未接管移动载体,驾驶员不在环持续时长达到第四阈值(例如,20s),通过座舱的第二级提醒,控制座舱内的显示装置和发声装置提示驾驶员接管移动载体;如果驾驶员在在座舱的第二级提醒下未接管移动载体,那么当驾驶员不在环持续时长累计到第五阈值(例如,35s)时,通过座舱的第三级提醒,控制座舱内的显示装置、发声装置和安全带振动装置提示驾驶员接管移动载体。
在本申请中,当驾驶员不在环时,如果通过紧急事件触发座舱的第一级提醒之后,可以以驾驶员不在环持续时长等级,控制提示装置提示驾驶员接管移动载体,可以及时有效地将接管提醒信息传递给驾驶员,从而提高行驶安全性。
结合第二方面,在第二方面的某些实现方式中,该确定单元还用于:在控制提示装置提示驾驶员接管移动载体之前,确定移动载体的行驶速度大于或等于速度阈值。
在本申请中,将移动载体的行驶速度作为接管提醒触发的考虑因素,可以进一步提高行驶安全性,以及用户体验。例如,当自动驾驶系统开启自动泊车功能时,驾驶员可以脱手脱眼,此时移动载体的速度是小于速度阈值的,除非有突发事件发生,不然驾驶员不会收到接管提醒信息,这样可以提高用户体验。
结合第二方面,在第二方面的某些实现方式中,控制单元控制提示装置提示驾驶员接管移动载体和通信单元向移动终端发送第一提示信息同步进行。
结合第二方面,在第二方面的某些实现方式中,该装置还包括记录单元,该记录单元用于,记录驾驶员接管移动载体的响应参数。通信单元还用于,向云端服务器发送响应参数,其中,响应参数用于确定驾驶员的驾驶行为分数,驾驶行为分数用于评估驾驶员的行为安全性。通信单元还具体用于,接收来自云端服务器的定制信息,定制信息是根据驾驶行为分数确定的。
在本申请中,当驾驶员不在环的结果是通过座舱内驾驶员监测装置和移动终端采集的数据共同得到的时,那么基于此发出的驾驶员提醒越精准,从而使得驾驶员接管移动载体的响应参数记录地更加准确,以便于提高后续信息推送的准确性。
第三方面,提供了一种驾驶员提醒装置,该装置包括至少一个处理器和接口电路,该处理器通过接口电路与提示装置通信;该至少一个处理器用于执行上述第一方面中任一项所述的方法,控制该提示装置。
在一些可能的实现方式中,该提示装置可以为显示装置、发声装置(例如,扬声器)、安全带振动装置、氛围灯中的至少一种。
在一些可能的实现方式中,该显示装置可以包括车载显示屏或者投影显示屏。
第四方面,提供了一种驾驶员提醒装置,该装置包括处理单元和存储单元,其中存储单元用于存储指令,处理单元执行存储单元所存储的指令,以使该装置执行第一方面中任一种可能的方法。
可选地,上述处理单元可以包括至少一个处理器,上述存储单元可以是存储器,其中存储器可以 是芯片内的存储单元(例如,寄存器、缓存等),也可以是移动载体内位于芯片外部的存储单元(例如,只读存储器、随机存取存储器等)。
第五方面,提供了一种驾驶员提醒系统,该驾驶员提醒系统包括计算平台和提示装置,该计算平台包括上述第二方面、第三方面或者第四方面任一方面所述的装置。
在一些可能的实现方式中,该提示装置可以为显示装置、发声装置(例如,扬声器)、安全带振动装置、氛围灯中的至少一种。
第六方面,提供了一种移动载体,该移动载体包括上述第二方面、第三方面或者第四方面中任一方面所述的装置,或者,该移动载体包括上述第五方面所述的驾驶员提醒系统。
结合第六方面,在第六方面的某些实现方式中,该移动载体为车辆。
第七方面,提供了一种计算机程序产品,所述计算机程序产品包括:计算机程序代码,当所述计算机程序代码在计算机上运行时,使得计算机执行上述第一方面中任一种可能的方法。
需要说明的是,上述计算机程序代码可以全部或者部分存储在第一存储介质上,其中第一存储介质可以与处理器封装在一起的,也可以与处理器单独封装,本申请实施例对此不作具体限定。
第八方面,提供了一种计算机可读介质,所述计算机可读介质存储有程序代码,当所述计算机程序代码在计算机上运行时,使得计算机执行上述第一方面中任一种可能的方法。
第九方面,本申请实施例提供了一种芯片系统,该芯片系统包括处理器,用于调用存储器中存储的计算机程序或计算机指令,以使得该处理器执行上述第一方面任一种可能的方法。
结合第九方面,在一种可能的实现方式中,该处理器通过接口与存储器耦合。
结合第九方面,在一种可能的实现方式中,该芯片系统还包括存储器,该存储器中存储有计算机程序或计算机指令。
附图说明
图1是本申请实施例提供的移动载体的一个功能框图示意;
图2是本申请实施例提供的一种驾驶员提醒方法的流程示意图;
图3是本申请实施例提供的一种移动终端接入移动载体的交互流程的示意图;
图4是本申请实施例提供的一种移动终端和车载座舱数字平台连接示意图;
图5本申请实施例提供的另一种移动终端和车载座舱数字平台连接示意图;
图6是本申请实施例提供的另一种移动终端和车载座舱数字平台连接示意图;
图7是本申请实施例提供的一种驾驶员提醒方法的交互示意图;
图8是本申请实施例提供的一种座舱分级提醒方式的示意图;
图9是本申请实施例提供的一种移动终端的分级提醒方式示意图;
图10是本申请实施例提供的另一种驾驶员提醒方法的交互示意图;
图11是本申请实施例提供的一种基于驾驶员提醒记录的驾驶行为评估方法的示意图;
图12是本申请实施例提供的一种驾驶员提醒信息发送流程的示意图;
图13是本申请实施例提供的一种座舱内驾驶员提醒信息和移动终端驾驶员提醒信息的人机交互示意图;
图14是本申请实施例提供的一种驾驶员提醒装置的示意图;
图15是本申请实施提供的一种驾驶员提醒装置的硬件结构示意图。
具体实施方式
下面将结合附图,对本申请实施例中的技术方案进行描述。其中,在本申请实施例的描述中,除非另有说明,“/”表示或的意思,例如,A/B可以表示A或B;本文中的“和/或”仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。
本申请实施例中采用诸如“第一”、“第二”的前缀词,仅仅为了区分不同的描述对象,对被描述对象的位置、顺序、优先级、数量或内容等没有限定作用。本申请实施例中对序数词等用于区分描述对象的前缀词的使用不对所描述对象构成限制,对所描述对象的陈述参见权利要求或实施例中上下文的描 述,不应因为使用这种前缀词而构成多余的限制。此外,在本实施例的描述中,除非另有说明,“多个”的含义是两个或两个以上。
图1是本申请实施例提供的移动载体100的一个功能框图示意。移动载体100可以包括感知系统120、显示装置130和计算平台150,其中,感知系统120可以包括感测关于移动载体100周边的环境的信息的若干种传感器。例如,感知系统120可以包括定位系统,定位系统可以是全球定位系统(global positioning system,GPS),也可以是北斗系统或者其他定位系统、惯性测量单元(inertial measurement unit,IMU)。感知系统120还可以包括激光雷达、毫米波雷达、超声雷达以及摄像装置中的一种或者多种。
移动载体100的部分或所有功能可以由计算平台150控制。计算平台150可以包括一个或者多个处理器,例如,处理器151至15n(n为正整数),处理器是一种具有信号的处理能力的电路,在一种实现中,处理器可以是具有指令读取与运行能力的电路,例如中央处理单元(central processing unit,CPU)、微处理器、图形处理器(graphics processing unit,GPU)(可以理解为一种微处理器)、或数字信号处理器(digital signal processor,DSP)等;在另一种实现中,处理器可以通过硬件电路的逻辑关系实现一定功能,该硬件电路的逻辑关系是固定的或可以重构的,例如处理器为专用集成电路(application-specific integrated circuit,ASIC)或可编程逻辑器件(programmable logic device,PLD)实现的硬件电路,例如现场可编程逻辑门阵列(field programmable gate array,FPGA)。在可重构的硬件电路中,处理器加载配置文档,实现硬件电路配置的过程,可以理解为处理器加载指令,以实现以上部分或全部单元的功能的过程。此外,还可以是针对人工智能设计的硬件电路,其可以理解为一种ASIC,例如神经网络处理单元(neural network processing unit,NPU)、张量处理单元(tensor processing unit,TPU)、深度学习处理单元(deep learning processing unit,DPU)等。此外,计算平台150还可以包括存储器,存储器用于存储指令,处理器151至15n中的部分或全部处理器可以调用存储器中的指令,执行指令,以实现相应的功能。
本申请中的移动载体可以包括路上交通工具、水上交通工具、空中交通工具、工业设备、农业设备、或娱乐设备等。例如移动载体可以为车辆,该车辆为广义概念上的车辆,可以是交通工具(如商用车、乘用车、摩托车、飞行车、火车等),工业车辆(如:叉车、挂车、牵引车等),工程车辆(如挖掘机、推土车、吊车等),农用设备(如割草机、收割机等),游乐设备,玩具车辆等,本申请实施例对车辆的类型不作具体限定。再如,移动载体可以为飞机、或轮船等交通工具。下文以移动载体是车辆为例进行说明。
移动载体的座舱内的显示装置130主要分为两类,第一类是车载显示屏;第二类是投影显示屏,例如抬头显示装置(head up display,HUD)。车载显示屏是一种物理显示屏,是车载信息娱乐系统的重要组成部分,座舱内可以设置有多块显示屏,如数字仪表显示屏,中控屏,副驾驶位上的乘客(也称为前排乘客)面前的显示屏,左侧后排乘客面前的显示屏以及右侧后排乘客面前的显示屏,甚至是车窗也可以作为显示屏进行显示。抬头显示,也称平视显示系统。主要用于在驾驶员前方的显示设备(例如挡风玻璃)上显示例如时速、导航等驾驶信息。以降低驾驶员视线转移时间,避免因驾驶员视线转移而导致的瞳孔变化,提升行驶安全性和舒适性。HUD例如包括组合型抬头显示(combiner-HUD,C-HUD)系统、风挡型抬头显示(windshield-HUD,W-HUD)系统、增强现实型抬头显示系统(augmented reality HUD,AR-HUD)。在一些可能的实现方式中,车载显示屏可以包括人机交互界面(human machine interface,HMI)。
在具体说明本申请实施例的具体方案之前,为了便于理解,首先对本申请实施例涉及的术语名词作如下解释。
1.驾驶员监测系统(driver monitor system,DMS)
驾驶员监测系统是用于监测驾驶员的状态,以防止驾驶员疲劳、分神等危害驾驶过程的行为发生。
2.运行设计域ODD
运行设计域,是自动驾驶汽车能够正常安全行驶的外部条件,如道路类型、行驶区域、速度、环境(例如,天气、白天/夜间等)等。
自动驾驶还处于不断发展的阶段,无法覆盖所有的驾驶场景。因此,ODD就是用于启动自动驾驶系统的运行限制条件。如果车辆行驶的外部环境不符合ODD,也就是超过自动驾驶系统的设定好的 ODD,那么自动驾驶系统可以发出驾驶员提醒,使得驾驶员可以及时接管车辆,以防事故的发生。
3.最小风险策略(minimum risk maneuver,MRM)
最小风险策略是指自动驾驶系统在出现系统性的失效(导致系统不工作的故障)或者出现超过系统原有的ODD的情况下,所采取的最小化风险的解决路径,以保障自动驾驶车辆在运行过程中安全。这套策略可在自动驾驶系统要求人工接管而未得到响应的情况下自动执行,也可在面临严重碰撞风险或车辆故障情况下自动执行。
自动驾驶分为不同的等级,L0级为无自动化;L1级为驾驶支援;L2级为部分自动化;L3级为有条件自动化;L4级为高度自动化;L5级为完全自动化。其中,L1级至L3级监测路况并做出反应的任务都由驾驶员和系统共同完成,并需要驾驶员接管动态驾驶任务。广义上来说,L1级至L3级的自动驾驶属于人机共驾的阶段。L4和L5级可以让驾驶员完全转变为乘客的角色。
虽然将自动驾驶分为不同的等级,但是如果发生以下情况,驾驶员都需要及时接管车辆。以防止事故的发生。例如,自动驾驶系统故障,或者车辆驾驶条件处于ODD之外,或者车辆处于紧急情况之下等。
目前,驾驶员监测系统和分级提醒系统中的显示提醒,大多数都是基于车载的显示装置130,在自动驾驶处于L3级以下时,自动驾驶系统实则可以看作驾驶辅助系统,辅助驾驶员的驾驶操作,驾驶员仍然为驾驶任务的主要操作者以及责任者。因此,处于L3级以下的驾驶辅助系统是不允许驾驶员处于脱手脱眼状态的,也就是驾驶员在整个驾驶过程中是在环状态。因此,当目前的驾驶员监测系统,监测到驾驶员处于不在环状态时,分级提醒系统发出驾驶员提醒。从视觉、声觉和触觉上,分别通过显示装置、发声装置(例如,扬声器)以及安全带来实现分级接管提醒。
随着自动驾驶技术的发展,当ADS系统被激活后,驾驶员处于生理和心理等因素的影响,驾驶员无法时刻关注座舱内的显示装置,驾驶员可能在使用移动终端,例如,智能手机、平板电脑、智能手环或智能手表等移动终端。如果此时出现紧急情况,驾驶员监测系统监测到驾驶员处于不在环状态时,分级提醒系统需要发出驾驶员提醒信息。由于驾驶员的注意力并不是在驾驶任务上,因此,基于目前的座舱显示装置进行显示提醒的方式,有可能使得驾驶员接收到接管显示提醒的最佳时机,从而引发潜在的交通事故的发生。
为了解决上述问题,本申请实施例提出了一种驾驶员提醒方法、装置、系统和车辆,降低驾驶员错过接收到接管显示提醒最佳时机的风险,从而提高自动驾驶的行车安全性。
图2是本申请实施例提供的一种驾驶员提醒方法200的流程示意图。如图2所述,该方法200可以由移动载体(例如,车辆)执行,或者,该方法200可以由计算平台和提示装置组成的系统执行,或者,该方法200可以由计算平台执行,或者,该方法200可以由计算平台中的片上系统(system-on-a-chip,SOC)执行,或者,该方法200可以由计算平台中的处理器执行。该方法200包括:
S210,在移动载体处于驾驶状态时,获取来自移动终端的传感器采集的第一数据,且获取移动载体的座舱内的传感器采集的第二数据。
应理解,在移动载体处于驾驶状态时,可以为在行车场景中移动载体开启了自适应巡航控制(adaptive cruise control,ACC)、移动载体开启了车道居中控制(lane centering control,LCC)、领航辅助驾驶(navigate on autopilot,NOA)等功能,还可以为在泊车场景中移动载体开启了自动泊车辅助功能(auto parking assist,APA)等功能。
需要说明的是,移动终端的传感器可以为移动终端的摄像装置或者屏幕触摸感应装置等,移动载体的座舱内的传感器可以为DMS系统中的摄像装置,该摄像装置可以安装于内后视镜上、驾驶位置侧的A柱上、或者方向盘上等位置,本申请实施例对此不作限制。
示例性地,移动终端可以是智能手机、平板电脑、智能手环或智能手表等移动终端。
S220,根据第一数据和第二数据,确定驾驶员状态。
作为一种可能的实现方式,判断第一数据和第二数据的可靠性,根据可靠的驾驶数据,确定驾驶员状态,其中,可靠的驾驶数据可以为第一数据或者第二数据。
示例性地,当移动载体的座舱内的传感器追踪驾驶员视线时,仅可以追踪到驾驶员的一只眼睛,而移动终端的传感器追踪驾驶员视线时,可以追踪到驾驶员的两只眼睛,那么来自移动终端的传感器 的第一数据更加可靠。反之,来自移动载体的座舱内的传感器的第二数据更加可靠。
作为一种可能的实现方式,融合第一数据和第二数据,得到驾驶员状态。
具体地,融合第一数据和第一数据的原始数据,得到驾驶员状态,或者融合第一数据的数据结果和第二数据的数据结果,得到驾驶员状态。
应理解,驾驶员状态可以为驾驶员在环或者驾驶员不在环,其中,驾驶员在环表示驾驶员注意力在驾驶任务上,驾驶员不在环可以表示驾驶员注意力不在驾驶任务上,例如,驾驶员出现分神、疲劳、使用移动终端设备等异常驾驶行为。
S230,在驾驶员状态指示驾驶员不在环时,控制提示装置提示驾驶员接管移动载体。
在本申请实施例中,第二数据来自固定安装在座舱内的摄像装置,其安装位置固定,安装角度固定,因此座舱内摄像装置可能存在视场角(filed of view,FOV)不足或者被遮挡的情况,使得座舱内摄像装置采集的数据不准确,从而导致仅根据DMS得到的驾驶员状态不准确的情况。因此,本申请实施例中,通过移动终端获得的第一数据可以弥补座舱采集的第二数据在视角上存在的不足,多元数据确定的驾驶状态,可以提高对驾驶员状态判断的准确性。基于准确的驾驶员状态,既能防止由于DMS的误检发出的驾驶员提醒,改善用户体验,又能有助于通过多设备发出驾驶员提醒,降低驾驶员错过接管提醒带来的安全隐患,从而提高行驶安全性。
图3是本申请实施例提供的一种移动终端接入移动载体的交互流程300的示意图。
S301,移动终端向车载座舱数字平台发送接入请求消息。
作为一种可能的实现方式,移动终端可以通过无线通信方式向车载座舱数字平台发送接入请求消息。其中,无线通信方式可以为近场通信(near field communication,NFC)、蓝牙(Bluetooth)或者绿牙等方式,本申请实施例对此不作限制。
示例性地,车载座舱数字平台可以为座舱域控制器(cockpit domain controller,CDC)。
S302,车载座舱数字平台授权移动终端接入。
作为一种可能的实现方式,车载座舱数字平台通过信息安全机制,根据请求接入的移动终端的身份标识码(identity document,ID),判断认证该移动终端的ID是否属于已注册的白名单用户对应的移动终端。如果判断认证通过,那么车载座舱数字平台接收移动终端的接入请求,并授权移动终端接入。如果判断认证不通过,那么车载座舱数字平台拒绝该移动终端的接入请求。
S303,车载座舱数字平台向移动终端发送接入应答消息。
应理解,车载座舱数字平台也可以通过无线通信方式将接入应答消息发送给移动终端,具体的无线通信方式和S301中类似。
S304,车载座舱数字平台向车辆的计算平台发送移动终端接入消息。
作为一种可能的实现方式,车载座舱数字平台通过网关将移动终端接入消息发送给车辆的计算平台。
应理解,车辆的计算平台可以为图1中的计算平台130,例如,可以为移动数据中心(mobile data center,MDC)。
作为一种可能的实现方式,如果车辆的计算平台具有无线通信功能,移动终端接入车辆的交互过程可以直接通过无线通讯的方式,实现移动终端接入车辆的计算平台中。
图4是本申请实施例提供的一种移动终端和车载座舱数字平台连接示意图。
当移动终端通过图3所示的消息交互流程接入车载座舱数字平台后,车载座舱数字平台的显示装置,显示弹窗内容:移动终端已接入。例如,如图4所示,中控屏幕显示:移动终端1已接入,其中,移动终端1可以为如图4所示的智能手机。
移动终端显示弹窗内容“本机设备接入车载座舱数字平台”,如图4所示,与此同时,驾驶员在环应用被启动,移动终端显示弹窗内容:“驾驶员在环应用”启动,如图4所示。
示例性地,如图4所示,座舱内的传感器可以是座舱内的摄像装置,座舱内的摄像装置安装于驾驶员侧的A柱上,移动终端的传感器可以为智能手机的前置摄像头。当移动终端和车载座舱数字平台连接后,座舱内的摄像装置通过车内通信网络和车辆的计算平台连接,移动终端的摄像装置通过无线通信网络和车辆的计算平台连接。其中,车内通信网络可以是汽车串行总线,例如,控制器域网(controller area network,CAN)总线、本地互联网络(local interconnect network,LIN)总线等,还可以 是车载以太网络,本申请实施例对此不作限制。
图5本申请实施例提供的另一种移动终端和车载座舱数字平台连接示意图。
如图5的所示,当移动终端通过图3所示的消息交互流程接入车载座舱数字平台后,车载座舱数字平台的显示装置上显示的内容和图4相同,为了简洁,在此不做赘述。移动终端的动态显示驾驶员在环服务的启动过程如图5的(a)至图5的(c)所示。
如图5的(a)所示,当移动终端通过图3所示的消息交互流程接入车载座舱数字平台后,移动终端显示弹窗内容“本机设备接入车载座舱数字平台”,如图5的(a)所示。
如图5的(b)所示,移动终端响应于第一操作,打开人机交互应用,例如,驾驶应用程序(application,APP)。
如图5的(c)所示的人机互联应用的操作界面,在人机交互应用启动之后,移动终端显示弹窗内容:“驾驶员在环服务”启动。
应理解,“驾驶员在环服务”启动的信息为弹窗信息,该信息在移动终端上显示一段时间之后,就会自动消失,此时,“驾驶员在环服务”在后台运行。
图6是本申请实施例提供的又一种移动终端和车载座舱数字平台连接示意图。当移动终端无摄像头装置时,可以通过移动终端的屏幕触摸感应装置监测驾驶员使用移动终端的情况。
当移动终端通过图3所示的消息交互流程接入车载座舱数字平台后,车载座舱数字平台的显示装置,显示弹窗内容:移动终端已接入。例如,如图6所示,中控屏幕显示:移动终端2已接入,其中,移动终端2可以为如图6所示的智能手表。移动终端的动态显示驾驶员在环服务的启动过程如图6的(a)至图6的(c)所示。驾驶员在环服务的动态启动过程和图5的(a)至图5的(c)相同,为了简洁,在此不做赘述。
ADS系统被激活,也就是ADS系统处于运行状态,座舱内的DMS系统处于运行状态和移动终端中的“驾驶员在环服务”均已启动。
下面将结合图7至图13详细说明本申请实施例提供驾驶员提醒方法。
图7是本申请实施例提供的一种驾驶员提醒方法700的交互示意图。如图7所示,该方法700包括:
S701,移动终端获取第一数据。
作为一种可能的实现方式,通过移动终端的摄像装置采集第一数据,第一数据可以是移动终端所处位置视角下采集到的驾驶员的视频流数据。
作为一种可能的实现方式,通过移动终端的非摄像装置(例如,屏幕触摸感应装置)采集第一数据,第一数据可以是驾驶员点击移动终端的记录数据。
S702,车辆的计算平台获取第二数据和车辆行驶速度。
作为一种可能的实现方式,车辆的计算平台获取移动载体的传感器采集的第二数据。
示例性地,移动载体的传感器可以为座舱内的摄像装置,此时第二数据可以为座舱内的摄像装置追踪驾驶员视线的数据。
作为一种可能的实现方式,车辆的计算平台通过车内通信获取移动载体的行驶速度。
应理解,本申请实施例对S701和S702两个步骤的执行顺序不作限制。
S703,车辆的计算平台获取移动终端的第一数据。
作为一种可能的实现方式,车辆的计算平台通过无线通信方式,接收来自移动终端的第一数据。
作为一种可能的实现方式,移动终端通过无线通信方式,将第一数据发送给车载座舱数字平台,车载座舱数字平台再通过车内通信网络将第一数据发送给车辆的计算平台。
应理解,第一数据可以反映驾驶员的注意力是否在移动终端上。
还应理解,本申请实施例对S702和S703两个步骤的执行顺序不作限制。
S704,车辆的计算平台根据第一数据和/或第二数据,确定驾驶员状态。
其中,驾驶员状态用于指示驾驶员在环或者驾驶员不在环,如果驾驶员不在环,那么驾驶员状态还用于指示是否检测到驾驶员注意力在移动终端上。
应理解,驾驶员状态可以通过第二数据获得,或者通过第一数据获得,或者通过第二数据和第一数据获得。如表1所示,“1”表示驾驶员状态所采用的数据,“0”表示驾驶员状态未采用的数据。
表1
方式1,车辆的计算平台根据第二数据,确定驾驶员状态。
示例性地,通过人工智能(artificial intelligent,AI)算法,对第二数据分析,完成对驾驶员活体在位检测、用户ID账号检测、头部位姿检测和视线追踪检测等检测任务,从而得到驾驶员是否在环的驾驶员状态。
其中,本申请实施例对具体的AI算法不作限定,例如,可以是卷积神经网络、循环神经网络、对抗神经网络等。
应理解,通过完成上述驾驶员检测任务,可以检测出驾驶员的驾驶异常行为,例如,驾驶员出现疲劳、分神、长时间使用手机等注意力不在驾驶任务上的异常行为。当检测出驾驶员出现驾驶异常行为,即可认为驾驶员状态中包括驾驶员不在环。
方式2,车辆的计算平台根据第一数据,确定驾驶员状态。
示例性地,当第一数据是通过移动终端的摄像装置获得的时,对第一数据的处理方式和第二数据类似,为了简洁,在此不做赘述。
方式3,车辆的计算平台根据第二数据和第一数据,确定驾驶员状态。
作为一种可能的实现方式,判断第一数据和第二数据的可靠性,根据可靠的驾驶数据,确定驾驶员状态,其中,可靠的驾驶数据可以为第一数据或者第二数据。
作为一种可能的实现方式,车辆的计算平台融合第二数据和第一数据,得到驾驶员状态。
通过AI算法融合第二数据和第一数据,得到驾驶员状态。
示例性地,第二数据和第一数据均为视频流数据,通过AI算法分析第二数据和第一数据中的驾驶员的视线,得到驾驶员是否在环,如果驾驶员不在环的话,进一步分析驾驶员的注意力是否在移动终端上。
驾驶员状态存在以下几种情况,第一种,驾驶员不在环,并且检测到驾驶员注意力在移动终端上,第二种,驾驶员不在环,并且未检测到驾驶员注意力在移动终端上,第三种驾驶员在环。
通过融合第二数据的数据结果和第一数据的数据结果,确定驾驶员状态
示例性地,第二数据的处理过程和方式1中类似,为了简洁,在此不做赘述。
示例性地,如果第一数据是通过移动终端的摄像装置获得的,那么对第一数据的处理方式和第二数据类似,为了简洁,在此不做赘述。
示例性地,如果第一数据是通过移动终端的屏幕触摸感应装置获得的,那么可以通过记录屏幕触摸的时间以及驾驶员操作屏幕的时间长度,来确定驾驶员是否在环。例如,如果驾驶员操作屏幕时间长度超过阈值,则确定驾驶员目前正在专注于移动终端,驾驶员不在环。
具体地,驾驶员状态如表2所示。
表2
情况1,如果根据第二数据,确定驾驶员状态为不在环,根据第一数据,也确定驾驶员状态为不在环,那么驾驶员状态包括驾驶员不在环,并且检测到驾驶员注意力在移动终端上。
例如,驾驶员正在查看移动终端的实时信息,开会等注意力在移动终端的情况,座舱采集的驾驶状态数据可以反映出驾驶员处于分神状态,移动终端采集的驾驶员状态也可以反映出驾驶员不在环。
情况2,如果根据第二数据,确定驾驶员状态为不在环,根据第一数据,确定驾驶员状态为未知是 否在环,那么驾驶员状态包括驾驶员不在环,并且未检测到驾驶员的注意力在移动终端上。
需要说明的是,通过第一数据得到驾驶员是否在环未知的结果,意味着驾驶员的注意力不在移动终端上,无法通过移动终端的传感器采集的数据判断驾驶员是否在环。
例如,移动终端的传感器采集的第一数据可以为驾驶员操作移动终端的记录,该记录可以包括数据类型:驾驶员是否操作移动终端;数据返回值:0或1,其中,“0”可以表示驾驶员未操作移动终端,“1”可以表示驾驶员操作移动终端,当数据返回值为1时,该记录还包括驾驶员操作移动终端的时长。
又如,移动终端的传感器采集的第一数据可以为驾驶员的视线追踪数据,当驾驶员的注意力不在移动终端上时,驾驶员的视线追踪数据指示未检测到驾驶员注意力在移动终端上。
示例性地,当第一数据驾驶员操作移动终端的记录的数据返回值为0时,或者当第一数据表示驾驶员的双眼不在注视移动终端时,确定驾驶员状态为未知驾驶员是否在环。
情况2对应的场景可以为,通过座舱采集的驾驶状态数据可以确定驾驶员出现疲劳、分神等异常行为,即驾驶员不在环,但通过移动终端采集的驾驶状态数据确定驾驶员在环,即驾驶员并未操作移动终端的情况。
情况3,如果根据第二数据,确定驾驶员状态为在环,根据第一数据,也确定驾驶员状态为在环,那么驾驶员状态为驾驶员在环。
S705,在驾驶员状态指示驾驶员不在环时,车辆的计算平台控制提示装置提示驾驶员接管移动载体。
作为一种可能的实现方式,根据驾驶员不在环的持续时长,车辆的计算平台控制提示装置提示驾驶员接管移动载体。
具体地,当驾驶员状态包括驾驶员不在环时,并且驾驶员不在环的持续时长大于或等于第一阈值时,车辆的计算平台控制提示装置提示驾驶员接管移动载体。示例如,驾驶员不在环的持续时间超过15秒,车辆的计算平台控制提示装置提示驾驶员接管移动载体。
作为一种可能的实现方式,在驾驶员状态指示驾驶员不在环,且检测到紧急事件发生时,车辆的计算平台控制提示装置提示驾驶员接管移动载体。
具体地,紧急事件可以是ADS出现故障、或者车辆驾驶条件处于ODD之外,或者行驶车辆出现突发事件等紧急事件。
示例性地,在驾驶员不在环的持续时长还未到第一阈值时,车辆的计算平台检测到紧急事件发生后,控制提示装置提示驾驶员接管移动载体。
应理解,提示装置可以位于座舱内,例如,座舱内的显示装置、发声装置(例如,扬声器)、氛围灯、安全带振动装置等,本申请实施例对此不作限制。
作为一种可能的实现方式,在车辆的计算平台控制提示装置提示驾驶员接管移动载体之前,确定移动载体的行驶速度大于或等于速度阈值。速度阈值可以为10km/h,或者是其他数值,本申请实施例对此不作限制。
在本申请实施例中,将移动载体的行驶速度作为接管提醒触发的考虑因素,可以进一步提高行驶安全性,以及用户体验。例如,当自动驾驶系统开启自动泊车功能时,驾驶员可以脱手脱眼,此时移动载体的速度是小于速度阈值的,除非有突发事件发生,不然驾驶员不会收到接管提醒信息,这样可以提高用户体验。
作为一种可能的实现方式,车辆的计算平台通过座舱分级提醒方式,来控制提示装置提示驾驶员接管移动载体。
图8是本申请实施例提供的一种座舱分级提醒方式的示意图。
如图8所示,座舱分级提醒方式可以包括座舱的第一级提醒、座舱的第二级提醒和座舱的三级提醒。其中,座舱的第一级提醒包括座舱内的显示提醒;座舱的第二级提醒包括座舱内的显示提醒和声音提醒;座舱的三级提醒包括座舱内的显示提醒、声音提醒和振动提醒。
S706,在驾驶员状态指示驾驶员不在环时,车辆的计算平台向移动终端发送第一提示信息,其中,第一提示信息用于指示驾驶员接管移动载体。
在本申请实施例中,通过座舱内接管提示装置和移动终端协作向驾驶员发出接管提醒信息,降低驾驶员由于错过座舱内发出的接管提醒而使得车辆发生事故的可能性,提高了接管提醒信息传递的有 效性和及时性,从而提高了车辆的安全性。
作为一种可能的实现方式,根据驾驶员不在环的持续时长,车辆的计算平台向移动终端发送第一提示信息。
作为一种可能的实现方式,在驾驶员状态指示驾驶员不在环,且检测到紧急事件发生时,车辆的计算平台向移动终端发送第一提示信息。
作为一种可能的实现方式,在驾驶员状态指示驾驶员不在环,且检测到驾驶员的注意力在移动终端上时,向移动终端发送第一提示信息。
作为一种可能的实现方式,车辆的计算平台通过移动终端的分级提醒方式,向移动终端发送第一提示信息。
具体地,在检测到驾驶员的注意力在移动终端上时,车辆的计算平台通过移动终端的第一分级提醒方式,向移动终端发送第一提示信息。
在本申请实施例中,当驾驶员不在环并且驾驶员注意力在移动终端上时,通过座舱内提示装置和移动终端在第一时间发出接管提醒,可以避免驾驶员注意力在移动终端上时,错过座舱内接管提醒装置上显示的接管提醒,可以提高接管提醒发出的有效性和驾驶员接收到接管提醒的及时性,从而提高了车辆的安全性。
可选地,在检测到驾驶员的注意力不在移动终端上时,车辆的计算平台通过移动终端的第二分级提醒方式,向移动终端发送驾驶员提醒信息。
在本申请实施例中,当驾驶员不在环并且未检测到驾驶员注意力在移动终端上时,通过振动方式或者声音方式在移动终端接管提醒装置上发出接管提醒,可以降低当驾驶员注意力不在移动终端上且驾驶员不在环时错过接管提醒的可能性,进一步提高接管提醒发出的有效性和驾驶员接收到接管提醒的及时性,从而提高了车辆的安全性。例如,驾驶员状态指示驾驶员不在环,驾驶员出现疲劳、分神等异常行为,但驾驶员并未操作移动终端的情况,仅仅依靠座舱内的显示提醒可能无法让驾驶员及时接收到接管提醒信息,因此,额外通过移动终端的振动提醒或者声音提醒,可以提高驾驶员接收到接管提醒的及时性和有效性,从而提高了车辆的安全性。
图9是本申请实施例提供的一种移动终端的分级提醒方式示意图。
如图9的(a)所示,移动终端的第一分级提醒方式包括移动终端的第一级提醒、移动终端的第二级提醒和移动终端三级提醒。其中,移动终端的第一级提醒可以包括移动终端显示提醒;移动终端的第二级提醒可以包括移动终端显示提醒和声音提醒,或者包括移动终端显示提醒和振动提醒;移动终端三级提醒包括移动终端显示提醒、声音提醒和振动提醒。
如图9的(b)所示,移动终端的第二分级提醒方式包括移动终端的第一级提醒、移动终端的第二级提醒和移动终端的第三级提醒。其中,移动终端的第一级提醒可以包括移动终端的振动提醒,或者移动终端的声音提醒;移动终端的第二级提醒可以包括移动终端的振动提醒和声音提醒;移动终端三级提醒包括移动终端的显示提醒、声音提醒和振动提醒。
应理解,声音提醒可以是包括具体提示内容的语音提醒,例如,“请接管车辆”的语音提醒,也可以是无具体提示内容的声音提醒,例如,“嘀-嘀”的声音提醒。本申请实施例对声音提醒的类型不作限制。
还应理解,第二级提醒中的声音提醒可以比第一级提醒中的声音提醒更强烈,例如,如果是包括具体内容的语音提醒,那么语音提醒的音量可以增大,如果是不包括具体内容的声音提醒,那么可以提高语音提醒的频率,从“嘀-嘀”短促的声音变成“嘀嘀-嘀嘀”的急促的声音。
图10是本申请实施例提供的另一种驾驶员提醒方法1000的交互示意图。该方法1000包括:
S1001,移动终端获取第一数据。
需要说明的是,S1001和S701类似,详情请参考S701。
S1002,车辆的计算平台获取第二数据和车辆行驶速度。
需要说明的是,S1002和S702类似,详情请参考S702。
S1003,移动终端获取车辆的计算平台的第二数据和车辆行驶速度。
作为一种可能的实现方式,移动终端通过无线通信方式,获取来自车辆的计算平台的第二数据。
S1004,移动终端根据第一数据和/或第二数据,确定驾驶员状态。
需要说明的是,S1004中移动终端确定驾驶员状态的过程和S704中车辆的计算平台确定驾驶员状态的过程类似,详情请参考S704。
S1005,在驾驶员状态指示驾驶员不在环时,移动终端控制移动终端相应的装置提示驾驶员接管移动载体。
需要说明的是,S1005中移动终端控制移动终端相应的装置提示驾驶员接管移动载体的过程和S705中车辆的计算平台控制提示装置提示驾驶员接管移动载体的过程类似,详情请参考S705。
S1006,如果驾驶员状态包括驾驶员不在环,移动终端向车辆的计算平台发送第一提示信息,其中,第一提示信息用于指示驾驶员接管移动载体。
需要说明的是,S1006中的具体步骤可以参考S706。
作为一种可能的实现方式,确定驾驶员状态的过程可以由云端服务器实现,云端服务器将驾驶员状态发送给车辆的计算平台,车辆的计算平台控制提示装置提示驾驶员接管车辆,向移动终端发送第一提示信息。
图11是本申请实施例提供的一种基于驾驶员提醒记录的驾驶行为评估方法1100的示意图。
S1101,车辆的计算平台记录驾驶员接管车辆的响应参数。
应理解,响应参数包括一个驾驶任务中,接管提醒的次数和每次接管提醒中驾驶员接管车辆时对应的接管提醒等级。
应理解,接管提醒的方式是通过本申请实施例中图7所示的方式实现的。
例如,一个驾驶任务可以是从A地到B地的正常行驶任务,或者还可以为自动泊车任务。
其中,接管提醒的次数表示在一个驾驶任务中驾驶员不在环的次数,接管提醒次数越多表示驾驶员处于异常驾驶行为状态的次数越多,即驾驶员的驾驶行为越危险。
其中,驾驶员接管车辆时对应的接管提醒等级越高意味着驾驶员接管车辆反应时间越长,即驾驶员的驾驶行为越危险。
S1102,车辆的计算平台将响应参数发送给云端服务器。
S1103,云端服务器接收响应参数,并根据响应参数,确定驾驶员的驾驶行为分数,驾驶员的驾驶行为分数用于评估驾驶员的驾驶行为的安全性。
应理解,在一个驾驶任务中,如果接管提醒次数越多,那么驾驶行为分数越低。如果每次接管提醒中驾驶员接管车辆时对应的接管提醒等级越高,那么驾驶员行为分数越低。
在本申请实施例中,当驾驶员不在环的结果是通过座舱内驾驶员监测装置和移动终端采集的数据共同得到的时,那么基于此发出的驾驶员提醒越精准,从而使得驾驶员接管车辆的响应参数记录地更加准确,以便于提高后续信息推送的准确性。
S1104,云端服务器根据驾驶员的驾驶行为分数,向车辆推送定制信息。
例如,如果驾驶员的驾驶行为分数低,那么可以向车辆推送驾驶安全教育信息,如果驾驶员的驾驶行为分数高,那么可以向车辆推送新的自动驾驶功能。
图7和图10中都分别通过座舱的分级提醒方式和移动终端的分级提醒方式,使得驾驶员提醒信息可以通过座舱内的装置和移动终端传递给驾驶员。本申请实施例对座舱的分级提醒方式中每一级提醒的发出时机和移动终端的分级提醒方式中每一级提醒的发出时机不作限定,例如,座舱分级提醒方式中座舱的第i级提醒的发出时机和移动终端分级提醒方式中移动终端的第i级提醒的发出时机可以相同或者不同,其中i=1,2,3。
示例性地,下面将结合图12和图13详细说明座舱分级提醒方式中座舱的每一级提醒的发出时机和移动终端分级提醒方式中移动终端的每一级提醒的发出时机相同的驾驶员提醒信息发送过程。图12和13以图7所示的驾驶员提醒方法为例进行说明。并且以时间触发驾驶员提醒信息的传递为例进行说明。
图12是本申请实施例提供的一种驾驶员提醒信息发送流程1200的示意图。
S1201,在驾驶员状态指示驾驶员不在环时,车辆的计算平台控制提示装置提示驾驶员接管移动载体和向移动终端发送第一提示信息。
S1202,车辆的计算平台在控制提示装置发出座舱的第一级提醒的,同步地通过移动终端的第一级提醒向移动终端发送第一提示信息。
作为一种可能的实现方式,驾驶员不在环持续时长累计到第一阈值(例如,15秒),车辆的计算平台在控制提示装置发出座舱的第一级提醒的,同步地通过移动终端的第一级提醒向移动终端发送第一提示信息。
作为一种可能的实现方式,在驾驶员状态指示驾驶员不在环,并且检测到紧急事件发生时,车辆的计算平台在控制提示装置发出座舱的第一级提醒的,同步地通过移动终端的第一级提醒向移动终端发送第一提示信息。
该场景可以是,驾驶员不在环持续时长尚未累计到第一阈值,但是此时,检测到紧急事件发生,需要驾驶员接管车辆。
示例性地,如果驾驶员状态包括驾驶员的注意力在移动终端上,车辆的计算平台控制车辆座舱内的显示装置显示接管提醒信息,同步地向移动终端发送该显示提醒。
示例性地,如果驾驶员状态包括驾驶员的注意力不在移动终端上,车辆的计算平台控制车辆座舱内的显示装置显示接管提醒信息,同步地向移动终端发送振动提醒;或者,车辆的计算平台控制车辆座舱内的显示装置显示接管提醒信息,同步地向移动终端发送声音提醒。
S1203,判断驾驶员是否已经接管车辆。
当驾驶员已经接管车辆,执行S1201,也就是驾驶员提醒进入新的提醒周期,在驾驶员状态指示驾驶员不在环时,控制提示装置提示驾驶员接管移动载体和向移动终端发送第一提示信息。
当驾驶员未接管车辆时,执行S1204,车辆的计算平台在控制提示装置发出座舱的第二级提醒,同步地通过移动终端的第二级提醒向移动终端发送第一提示信息。
作为一种可能的实现方式,驾驶员不在环持续时长累计到第二阈值(例如,30秒),车辆的计算平台在控制提示装置发出座舱的第二级提醒,同步地向移动终端发送移动终端的第二级提醒。
作为一种可能的实现方式,在检测到紧急事件发生后,驾驶员不在环持续时长累计到第四阈值(例如,20s)时,车辆的计算平台在控制提示装置发出座舱的第二级提醒,同步地向移动终端发送移动终端的第二级提醒。
该场景可以是,在检测到紧急事件发生时,在座舱和移动终端的第一级提醒下,驾驶员未接管车辆,此时驾驶员不在环持续时长累计到第四阈值,触发座舱和移动终端的第二级提醒。
示例性地,如果驾驶员状态包括驾驶员注意力在移动终端上,车辆的计算平台在控制提示装置发出座舱的第二级提醒,同步地通过移动终端的第二级提醒向移动终端发送第一提示信息可以为:第一,车辆的计算平台在控制车辆座舱内的显示装置显示接管提醒信息和车辆座舱内发声装置发出声音提醒,同步地向移动终端发送显示提醒和声音提醒。或者,第二,车辆的计算平台在控制车辆座舱内的显示装置显示接管提醒信息和车辆座舱内发声装置发出声音提醒,同步地向移动终端发送显示提醒和振动提醒。
示例性地,如果驾驶员状态包括驾驶员注意力不在移动终端上,车辆的计算平台在控制提示装置发出座舱的第二级提醒,同步地通过移动终端的第一级提醒向移动终端发送第一提示信息可以为第一,车辆的计算平台在控制车辆座舱内的显示装置显示接管提醒信息和车辆座舱内发声装置发出声音提醒,同步地向移动终端发送振动提醒和声音提醒。
S1205,判断驾驶员是否已经接管车辆,
当驾驶员已经接管车辆,执行S1201,也就是驾驶员提醒进入新的提醒周期。
当驾驶员未接管车辆时,执行S1206,车辆的计算平台在控制提示装置发出座舱的第三级提醒,同步地通过移动终端的第三级提醒向移动终端发送第一提示信息。
作为一种可能的实现方式,驾驶员不在环持续时长累计到第三阈值(例如,50秒),车辆的计算平台在控制提示装置发出座舱的第三级提醒,同步地通过移动终端的第三级提醒向移动终端发送第一提示信息。
作为一种可能的实现方式,在检测到紧急事件发生后,驾驶员不在环持续时长累计到第五阈值(例如,35s)时,车辆的计算平台在控制提示装置发出座舱的第三级提醒,同步地向移动终端发送移动终端的第三级提醒。
该场景可以是,在检测到紧急事件发生时,在座舱和移动终端的第一级提醒和第二级提醒下,驾驶员未接管车辆,此时驾驶员不在环持续时长累计到第五阈值,触发座舱和移动终端的第三级提醒。
示例性地,无论驾驶员状态包括驾驶员注意力在移动终端上还是包括驾驶员注意力不在移动终端上,车辆的计算平台在控制车辆座舱内的显示装置显示接管提醒信息,车辆座舱内的发声装置发出声音提醒和车辆座舱内的安全带发出振动提醒,同步地向移动终端发送移动终端的显示提醒、声音提醒和振动提醒。
S1207,判断驾驶员是否已经接管车辆,
当驾驶员已经接管车辆,执行S1201,也就是驾驶员提醒进入新的提醒周期。
当驾驶员未接管车辆时,执行S1208,自动驾驶系统进入惩罚状态,例如,自动驾驶系统强制制动。
在本申请实施例中,通过同步地使用座舱内接管提醒装置和移动终端接管提醒装置,可以加强驾驶员提醒信息发送的时效性和有效性,提高行驶安全性。
下面将结合图13详细说明座舱内驾驶员提醒信息和移动终端驾驶员提醒信息的人机交互示意图。以时间触发驾驶员提醒信息的传递为例进行说明
图13是本申请实施例提供的一种座舱内驾驶员提醒信息和移动终端驾驶员提醒信息的人机交互示意图。图13是驾驶员状态包括驾驶员注意力在移动终端上。
如图13的(a)所示,当驾驶员不在环持续时长累计到第一阈值时,座舱的显示装置和移动终端的屏幕同步地显示第一级提醒,例如,在座舱的显示装置中显示“请接管车辆”的提醒,同步地移动终端的屏幕上显示“请接管车辆”的弹窗。
应理解,车辆的计算平台可以通过车内通信(例如,汽车串行总线或者车载以太网络)控制车内的显示装置显示座舱的第一级提醒。车辆的计算平台通过无线网络将移动终端的第一级提醒发送给移动终端,或者车辆的计算平台通过车内通信将第一级提醒发送给车载座舱数字平台,车载座舱数字平台再通过无线通信将第一级提醒发给移动终端。
如图13的(b)所示,当驾驶员在第一级提醒下未及时接管车辆时,驾驶员不在环持续时长累计到第二阈值,座舱的显示装置和发声装置发出第二级提醒,同步地移动终端的屏幕和发声装置发出第二级提醒。例如,在座舱的显示装置中显示“请接管车辆”的提醒,在座舱的发声装置中发出“请接管车辆”的提醒,同步地移动终端的屏幕上显示“请接管车辆”的弹窗,移动终端的发声装置发出“请接管车辆”的语音提醒。
应理解,图13的(b)仅仅示出了如果驾驶员注意力在移动终端上的,座舱和移动终端发出的一种第二级提醒方式,其他第二级提醒方式请具体参考前文的描述。
如图13的(c)所示,当驾驶员在第二级提醒才未及时接管车辆时,驾驶员不在环持续时长累计到第三阈值,座舱的显示装置、发声装置和振动装置(例如,主动安全带振动)发出三级提醒,同步地移动终端的屏幕、发声装置和振动装置发出三级提醒。例如,在座舱的显示装置中显示“请接管车辆”的提醒,在座舱的发声装置中发出“请接管车辆”的提醒,驾驶员的安全带发出振动提醒,同步地移动终端的屏幕上显示“请接管车辆”的弹窗,移动终端的发声装置发出“请接管车辆”的语音提醒,移动终端的振动装置发出振动提醒。
当驾驶员在三级提醒下未及时接管车辆时,自动驾驶系统进入惩罚状态,例如,自动驾驶系统强制制动。
作为一种可能的实现方式,座舱分级提醒方式中座舱的每一级提醒的发出时机和移动终端分级提醒方式中移动终端的每一级提醒的发出时机可以不同。
示例性地,先控制提示装置发出座舱的第一级提醒,在驾驶员没有接管车辆的情况下,驾驶员注意力在移动终端上时,再将移动终端的第一级提醒发送给移动终端,如果驾驶员没有接管车辆,那么后续再依次发出座舱的第二级提醒、移动终端的第二级提醒、座舱的第三级提醒和移动终端的第三级提醒,如果驾驶员在多设备多级提醒下都未接管车辆时,自动驾驶系统进入惩罚状态。
在本申请实施例中,座舱接管提醒信息的发出时机和移动终端接管提醒信息发出的时机为交错的,可以避免同一时间多设备同时发出接管提醒,而是通过不同设备交错时间发出提醒信息,在总的提醒时长不变的情况下,提高接管提醒的频率。相比于,目前仅基于座舱内设备进行接管提醒而言,即使将目前的提醒频率提高,也不一定可以在错过座舱内发出的接管提醒之后,有效地接收到接管提醒信息,以及时接管车辆,提高行驶安全性。此外,接管提醒信息发出的时机为交错式的,可以避免在同 一时刻驾驶员接收过多的提醒信息产生厌烦情绪,以此在提高行驶安全性的同时也可以提高用户体验。
作为一种可能的实现方式,座舱的第一级提醒和移动终端的第一级提醒的发出时机相同,如果驾驶员没有接管车辆,那么依次发出座舱的第二级提醒,再发出移动终端的第二级提醒,如果驾驶员在座舱和移动终端的第二级提醒下都未接管车辆,那么同步地发出座舱的第三级提醒和移动终端的第三级提醒,如果驾驶员在多设备多级提醒下都未接管车辆时,自动驾驶系统进入惩罚状态。
在本申请实施例中,针对驾驶员不在环,行驶车速满足车速阈值条件,并且突发事件发生的情况或者驾驶员不在环持续时长达到第一阈值时,可以在第一时间通过多设备提醒驾驶员接管,从而防止驾驶员在第一时间错过接管提醒信息,从而提高行驶安全性。
上述内容为本申请实施例的驾驶员提醒方法,下面将结合图14和15对驾驶员提醒装置做详细说明。应理解,装置实施例的描述和方法实施例的描述相互对应。因此,未详细描述的内容可以参见上文方法实施例,为了简洁,在此不作赘述。
图14是本申请实施例提供的一种驾驶员提醒装置的示意图。如图14所示,该装置包括:获取单元1401、确定单元1402和控制单元1403。
获取单元1401用于,在移动载体处于驾驶状态时,获取来自移动终端的传感器采集的第一数据,且获取移动载体的座舱内的传感器采集的第二数据;
确定单元1402用于,根据第一数据和第二数据,确定驾驶员状态;
控制单元1403用于,在驾驶员状态指示驾驶员不在环时,控制提示装置提示驾驶员接管移动载体。
可选地,该装置还包括通信单元1404,该通信单元1404用于,向移动终端发送第一提示信息,第一提示信息用于指示驾驶员接管移动载体。
可选地,该装置还包括第一检测单元1405,在第一检测单元1405检测到驾驶员的注意力在移动终端上时,该通信单元1404用于,向移动终端发送第一提示信息。
可选地,该控制单元1403用于,根据驾驶员不在环的持续时长,控制提示装置提示驾驶员接管移动载体。
可选地,该装置还包括第二检测单元1406,在驾驶员状态指示驾驶员不在环,且第二检测单元1406检测到紧急事件发生时,该控制单元1403用于,控制提示装置提示驾驶员接管移动载体。
可选地,该确定单元1402还用于,在控制提示装置提示驾驶员接管移动载体之前,确定移动载体的行驶速度大于或等于速度阈值。
示例性地,该获取单元1401可以是图1中的计算平台或者计算平台中的处理电路、处理器或者控制器。以获取单元1401为计算平台中的处理器151为例,在移动载体处于驾驶状态时,处理器151可以获取来自移动终端的传感器采集的第一数据和移动座舱内的传感器采集的第二数据。
示例性地,该确定单元1402可以是图1中的计算平台或者计算平台中的处理电路、处理器或者控制器。以确定单元1402为计算平台中的处理器152为例,处理器152可以根据第一数据和第二数据,确定驾驶员状态。
示例性地,该控制单元1403可以是图1中的计算平台或者计算平台中的处理电路、处理器或者控制器。以控制单元1403为计算平台中的处理器153为例,在驾驶员状态指示驾驶员不在环时,处理器152可以控制提示装置提示驾驶员接管移动载体。
以上获取单元1401所实现的功能、确定单元1402和控制单元1403所实现的功能可以分别由不同的处理器实现,或者,也可以由相同的处理器实现,本申请实施例对此不作限定。
应理解以上装置中各单元的划分仅是一种逻辑功能的划分,实际实现时可以全部或部分集成到一个物理实体上,也可以物理上分开。此外,装置中的单元可以以处理器调用软件的形式实现;例如装置包括处理器,处理器与存储器连接,存储器中存储有指令,处理器调用存储器中存储的指令,以实现以上任一种方法或实现该装置各单元的功能,其中处理器例如为通用处理器,例如CPU或微处理器,存储器为装置内的存储器或装置外的存储器。或者,装置中的单元可以以硬件电路的形式实现,可以通过对硬件电路的设计实现部分或全部单元的功能,该硬件电路可以理解为一个或多个处理器;例如,在一种实现中,该硬件电路为ASIC,通过对电路内元件逻辑关系的设计,实现以上部分或全部单元的功能;再如,在另一种实现中,该硬件电路为可以通过PLD实现,以FPGA为例,其可以包括大量逻辑门电路,通过配置文件来配置逻辑门电路之间的连接关系,从而实现以上部分或全部单元的功能。 以上装置的所有单元可以全部通过处理器调用软件的形式实现,或全部通过硬件电路的形式实现,或部分通过处理器调用软件的形式实现,剩余部分通过硬件电路的形式实现。
在本申请实施例中,处理器是一种具有信号的处理能力的电路,在一种实现中,处理器可以是具有指令读取与运行能力的电路,例如CPU、微处理器、GPU、或DSP等;在另一种实现中,处理器可以通过硬件电路的逻辑关系实现一定功能,该硬件电路的逻辑关系是固定的或可以重构的,例如处理器为ASIC或PLD实现的硬件电路,例如FPGA。在可重构的硬件电路中,处理器加载配置文档,实现硬件电路配置的过程,可以理解为处理器加载指令,以实现以上部分或全部单元的功能的过程。此外,还可以是针对人工智能设计的硬件电路,其可以理解为一种ASIC,例如NPU、TPU、DPU等。
可见,以上装置中的各单元可以是被配置成实施以上方法的一个或多个处理器(或处理电路),例如:CPU、GPU、NPU、TPU、DPU、微处理器、DSP、ASIC、FPGA,或这些处理器形式中至少两种的组合。
此外,以上装置中的各单元可以全部或部分可以集成在一起,或者可以独立实现。在一种实现中,这些单元集成在一起,以SOC的形式实现。该SOC中可以包括至少一个处理器,用于实现以上任一种方法或实现该装置各单元的功能,该至少一个处理器的种类可以不同,例如包括CPU和FPGA,CPU和人工智能处理器,CPU和GPU等。
图15是本申请实施提供的一种驾驶员提醒装置的硬件结构示意图。图15所示的驾驶员提醒装置1500可以包括:存储器1510、处理器1520、以及通信接口1530。其中,存储器1510、处理器1520,通信接口1530通过内部连接通路相连,该存储器1510用于存储指令,该处理器1520用于执行该存储器1520存储的指令,以控制输入/输出接口1530接收/发送第二信道模型的至少部分参数。可选地,存储器1510既可以和处理器1520通过接口耦合,也可以和处理器1520集成在一起。
示例性地,驾驶员提醒装置1500可以是图1中的计算平台150。
需要说明的是,上述通信接口1530使用例如但不限于收发器一类的收发装置,来实现通信设备1500与其他设备或通信网络之间的通信。上述通信接口1530还可以包括输入/输出接口(input/output interface)。
在实现过程中,上述方法的各步骤可以通过处理器1520中的硬件的集成逻辑电路或者软件形式的指令完成。结合本申请实施例所公开的方法可以直接体现为硬件处理器执行完成,或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器,闪存、只读存储器,可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1510,处理器1520读取存储器1510中的信息,结合其硬件完成上述方法的步骤。为避免重复,这里不再详细描述。
还应理解,本申请实施例中,该存储器可以包括只读存储器和随机存取存储器,并向处理器提供指令和数据。处理器的一部分还可以包括非易失性随机存取存储器。例如,处理器还可以存储设备类型的信息。
应理解,本文中术语“和/或”,仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,本文中字符“/”,一般表示前后关联对象是一种“或”的关系。
应理解,在本申请的各种实施例中,上述各过程的序号的大小并不意味着执行顺序的先后,各过程的执行顺序应以其功能和内在逻辑确定,而不应对本申请实施例的实施过程构成任何限定。
本申请实施例还提供了一种显示系统,该显示系统包括计算平台和提示装置,该计算平台可以包括驾驶员提醒装置1500。
示例性的,该提示装置可以包括显示装置、发声装置(例如,扬声器)、安全带振动装置、氛围灯中的至少一种。
本申请实施例还提供了一种装置,该装置包括处理单元和存储单元,其中存储单元用于存储指令,处理单元执行存储单元所存储的指令,以使该装置执行上述驾驶员提醒方法。
本申请实施例还提供一种计算机可读介质,所述计算机可读介质存储有程序代码,当所述计算机程序代码在计算机上运行时,使得所述计算机执行上述方法。
本申请实施例还提供一种芯片,包括:至少一个处理器和存储器,所述至少一个处理器与所述存 储器耦合,用于读取并执行所述存储器中的指令,以执行上述方法。
本申请实施例还提供一种车辆,包括:至少一个处理器和存储器,所述至少一个处理器与所述存储器耦合,用于读取并执行所述存储器中的指令,以执行上述方法。
在本说明书中使用的术语“部件”、“模块”、“系统”等用于表示计算机相关的实体、硬件、固件、硬件和软件的组合、软件、或执行中的软件。例如,部件可以是但不限于,在处理器上运行的进程、处理器、对象、可执行文件、执行线程、程序和/或计算机。通过图示,在计算设备上运行的应用和计算设备都可以是部件。一个或多个部件可驻留在进程和/或执行线程中,部件可位于一个计算机上和/或分布在2个或更多个计算机之间。此外,这些部件可从在上面存储有各种数据结构的各种计算机可读介质执行。部件可例如根据具有一个或多个数据分组(例如来自与本地系统、分布式系统和/或网络间的另一部件交互的二个部件的数据,例如通过信号与其它系统交互的互联网)的信号通过本地和/或远程进程来通信。
本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同装置来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统、装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
在本申请所提供的几个实施例中,应该理解到,所揭露的系统、方法和装置,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本申请各个实施例所述装置的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以所述权利要求的保护范围为准。

Claims (18)

  1. 一种驾驶员提醒方法,其特征在于,包括:
    在移动载体处于驾驶状态时,获取来自移动终端的传感器采集的第一数据,且获取所述移动载体的座舱内的传感器采集的第二数据;
    根据所述第一数据和所述第二数据,确定驾驶员状态;
    在所述驾驶员状态指示所述驾驶员不在环时,控制提示装置提示所述驾驶员接管所述移动载体。
  2. 如权利要求1所述的方法,其特征在于,所述方法还包括:
    向所述移动终端发送第一提示信息,所述第一提示信息用于指示所述驾驶员接管所述移动载体。
  3. 如权利要求2所述的方法,其特征在于,所述向所述移动终端发送第一提示信息包括:
    在检测到所述驾驶员的注意力在所述移动终端上时,向所述移动终端发送第一提示信息。
  4. 如权利要求1至3任一项所述的方法,其特征在于,所述控制提示装置提示所述驾驶员接管所述移动载体包括:
    根据所述驾驶员不在环的持续时长,控制所述提示装置提示所述驾驶员接管所述移动载体。
  5. 如权利要求1至3任一项所述的方法,其特征在于,所述控制提示装置提示所述驾驶员接管所述移动载体包括:
    在所述驾驶员状态指示所述驾驶员不在环,且检测到紧急事件发生时,控制所述提示装置提示所述驾驶员接管所述移动载体。
  6. 如权利要求1至5任一项所述的方法,其特征在于,所述方法还包括:
    在控制所述提示装置提示所述驾驶员接管所述移动载体之前,确定所述移动载体的行驶速度大于或等于速度阈值。
  7. 一种驾驶员提醒装置,其特征在于,所述装置包括获取单元、确定单元和控制单元,
    所述获取单元用于,在移动载体处于驾驶状态时,获取来自移动终端的传感器采集的第一数据,且获取所述移动载体的座舱内的传感器采集的第二数据;
    所述确定单元用于,根据所述第一数据和所述第二数据,确定驾驶员状态;
    所述控制单元用于,在所述驾驶员状态指示所述驾驶员不在环时,控制提示装置提示所述驾驶员接管所述移动载体。
  8. 如权利要求7所述的装置,其特征在于,所述装置还包括通信单元:
    所述通信单元用于,向所述移动终端发送第一提示信息,所述第一提示信息用于指示所述驾驶员接管所述移动载体。
  9. 如权利要求8所述的装置,其特征在于,所述装置还包括第一检测单元,
    在所述第一检测单元检测到所述驾驶员的注意力在所述移动终端上时,所述通信单元用于:向所述移动终端发送第一提示信息。
  10. 如权利要求7至9任一项所述的装置,其特征在于,所述控制单元用于:
    根据所述驾驶员不在环的持续时长,控制所述提示装置提示所述驾驶员接管所述移动载体。
  11. 如权利要求7至9任一项所述的装置,其特征在于,所述装置还包括第二检测单元,所述控制单元用于:
    在所述驾驶员状态指示所述驾驶员不在环,且所述第二检测单元检测到紧急事件发生时,控制所述提示装置提示所述驾驶员接管所述移动载体。
  12. 如权利要求7至11任一项所述的装置,其特征在于,所述确定单元还用于:
    在所述控制单元控制所述提示装置提示所述驾驶员接管所述移动载体之前,确定所述移动载体的行驶速度大于或等于速度阈值。
  13. 一种驾驶员提醒装置,其特征在于,所述驾驶员提醒装置包括至少一个处理器和接口电路,所述处理器通过所述接口电路与提示装置通信;
    所述至少一个处理器用于执行如权利要求1至6中任一项所述的方法,控制所述提示装置。
  14. 一种驾驶员提醒系统,其特征在于,所述驾驶员提醒系统包括计算平台和提示装置,其中所 述计算平台包括如权利要求7至13中任一项所述的驾驶员提醒装置。
  15. 一种移动载体,其特征在于,包括如权利要求7至13中任一项所述的驾驶员提醒装置,或者包括如权利要求14所述的驾驶员提醒系统。
  16. 如权利要求15所述的移动载体,其特征在于,所述移动载体为车辆。
  17. 一种计算机可读存储介质,其特征在于,其上存储有计算机程序,所述计算机程序被计算机执行时,以使得实现如权利要求1至6所述的方法。
  18. 一种芯片,其特征在于,所述芯片包括处理器和数据接口,所述处理器通过所述数据接口读取存储器上存储的指令,以执行如权利要求1至6中任一项所述的方法。
PCT/CN2023/112557 2022-09-01 2023-08-11 一种驾驶员提醒方法、装置、系统和移动载体 WO2024046091A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211069559.0A CN115782914A (zh) 2022-09-01 2022-09-01 一种驾驶员提醒方法、装置、系统和移动载体
CN202211069559.0 2022-09-01

Publications (1)

Publication Number Publication Date
WO2024046091A1 true WO2024046091A1 (zh) 2024-03-07

Family

ID=85431684

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/112557 WO2024046091A1 (zh) 2022-09-01 2023-08-11 一种驾驶员提醒方法、装置、系统和移动载体

Country Status (2)

Country Link
CN (1) CN115782914A (zh)
WO (1) WO2024046091A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115782914A (zh) * 2022-09-01 2023-03-14 华为技术有限公司 一种驾驶员提醒方法、装置、系统和移动载体

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102014225680A1 (de) * 2014-12-12 2016-06-16 Volkswagen Aktiengesellschaft Verfahren und Vorrichtung zur Übernahmeaufforderung einer Fahraufgabe an den Fahrzeugführer eines Fahrzeugs im Rahmen des automatischen Fahrens des Fahrzeugs
CN111619479A (zh) * 2020-05-20 2020-09-04 重庆金康赛力斯新能源汽车设计院有限公司 驾驶接管提示方法、装置、系统、车载控制器和存储介质
CN113183968A (zh) * 2021-04-25 2021-07-30 前海七剑科技(深圳)有限公司 抗疲劳驾驶方法、装置、设备及存储介质
KR20220069177A (ko) * 2020-11-19 2022-05-27 전남대학교산학협력단 자율 주행 차량의 스마트 제어 시스템 및 방법
CN115027488A (zh) * 2022-07-08 2022-09-09 质子汽车科技有限公司 一种车辆控制方法、装置及智能车辆
CN115782914A (zh) * 2022-09-01 2023-03-14 华为技术有限公司 一种驾驶员提醒方法、装置、系统和移动载体

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102014225680A1 (de) * 2014-12-12 2016-06-16 Volkswagen Aktiengesellschaft Verfahren und Vorrichtung zur Übernahmeaufforderung einer Fahraufgabe an den Fahrzeugführer eines Fahrzeugs im Rahmen des automatischen Fahrens des Fahrzeugs
CN111619479A (zh) * 2020-05-20 2020-09-04 重庆金康赛力斯新能源汽车设计院有限公司 驾驶接管提示方法、装置、系统、车载控制器和存储介质
KR20220069177A (ko) * 2020-11-19 2022-05-27 전남대학교산학협력단 자율 주행 차량의 스마트 제어 시스템 및 방법
CN113183968A (zh) * 2021-04-25 2021-07-30 前海七剑科技(深圳)有限公司 抗疲劳驾驶方法、装置、设备及存储介质
CN115027488A (zh) * 2022-07-08 2022-09-09 质子汽车科技有限公司 一种车辆控制方法、装置及智能车辆
CN115782914A (zh) * 2022-09-01 2023-03-14 华为技术有限公司 一种驾驶员提醒方法、装置、系统和移动载体

Also Published As

Publication number Publication date
CN115782914A (zh) 2023-03-14

Similar Documents

Publication Publication Date Title
US10699569B2 (en) Information processing apparatus, information processing method, and program
US11295143B2 (en) Information processing apparatus, information processing method, and program
US10768617B2 (en) Drive assistance device and drive assistance method, and moving body
CA2897481C (en) Method and system for providing feedback based on driving behavior
US9493116B2 (en) Alert systems and methods for a vehicle
US9132774B2 (en) Alert systems and methods for a vehicle
US8970358B2 (en) Alert systems and methods for a vehicle
JP6491601B2 (ja) 車両内移動デバイス管理
US9701245B2 (en) Alert systems and methods for a vehicle
US11873007B2 (en) Information processing apparatus, information processing method, and program
US20130342365A1 (en) Alert systems and methods for a vehicle
US20130342364A1 (en) Alert systems and methods for a vehicle
US10666901B1 (en) System for soothing an occupant in a vehicle
WO2024046091A1 (zh) 一种驾驶员提醒方法、装置、系统和移动载体
US11285966B2 (en) Method and system for controlling an autonomous vehicle response to a fault condition
US20230037467A1 (en) Driving control device and hmi control device
US20140005886A1 (en) Controlling automotive functionality using internal- and external-facing sensors
US20220379727A1 (en) Hmi control device and non-transitory computer readable storage medium
US11254322B2 (en) System and method for monitoring and managing cognitive load of an occupant of a vehicle
CN114084159A (zh) 辅助驾驶功能提醒方法、装置、介质及车辆
WO2023035260A1 (zh) 提示车辆状态信息的方法及装置
WO2023001068A1 (zh) 一种车辆驾驶方法及装置
US20230401979A1 (en) Driving diagnostic device, driving diagnostic system, machine learning device and generation method of learned model
WO2023084985A1 (ja) 車載システム及び機能学習提示プログラム
US20200218263A1 (en) System and method for explaining actions of autonomous and semi-autonomous vehicles

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23859125

Country of ref document: EP

Kind code of ref document: A1