WO2022061702A1

WO2022061702A1 - Method, apparatus, and system for driving alerts

Info

Publication number: WO2022061702A1
Application number: PCT/CN2020/117687
Authority: WO
Inventors: 金鑫垚
Original assignee: 华为技术有限公司
Priority date: 2020-09-25
Filing date: 2020-09-25
Publication date: 2022-03-31
Also published as: CN112654547A

Abstract

Provided in the present application are a method, apparatus, and system for driving alerts, being applicable to smart cars in the autonomous driving field. The method comprises: acquiring first data from a first sensor of a vehicle and second data from a second sensor of the vehicle, the first data comprising data of traffic elements around the vehicle, and the second data comprising data of the driver of the vehicle; and sending alert information when a first distance is greater than or equal to a first threshold and a first line of sight direction does not focus on a traffic element, the first distance referring to the distance between the vehicle and the traffic element determined on the basis of the first data, and the first line of sight direction referring to the line of sight direction of the driver determined on the basis of the second data. The method in the embodiments of the present application helps to increase the accuracy of driving alerts.

Description

Method, device and system for driving reminder

technical field

The present application relates to the field of automatic driving, and more particularly, to a driving reminder method, device and system.

Background technique

Artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that responds in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and basic AI theory.

Autonomous driving is a mainstream application in the field of artificial intelligence. Autonomous driving technology relies on the cooperation of computer vision, radar, monitoring devices and global positioning systems, etc., so that motor vehicles can achieve autonomous driving without the need for human active operation. Autonomous vehicles use various computing systems to help transport passengers or cargo from one location to another. Some autonomous vehicles may require some initial or continuous input from an operator, such as a pilot, driver, or passenger. An autonomous vehicle allows the operator to switch from a manual mode of operation to an autonomous driving mode or a mode in between. Since automatic driving technology does not require humans to drive motor vehicles, it can theoretically effectively avoid human driving errors, reduce the occurrence of traffic accidents, and improve the efficiency of highway transportation. Therefore, autonomous driving technology is getting more and more attention.

With the increasing number of vehicles, the safety of autonomous driving has also received more and more attention. Driving reminders (for example, blind spot detection reminders, collision avoidance reminders, lane departure reminders, and fatigue driving reminders, etc.), as a driving assistance technology, can analyze the situation inside and outside the vehicle and send warnings to the driver in advance of possible dangerous situations. , to improve driving safety. However, the accuracy of driving reminders is currently not high.

SUMMARY OF THE INVENTION

The present application provides a driving reminder method, device and system, which are used to improve the accuracy of the driving reminder and the safety of driving.

In a first aspect, a driving reminder method is provided, the method comprising:

acquiring first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, the first data including data of traffic elements surrounding the vehicle, the second data including the vehicle The data of the driver of The distance between the vehicle and the traffic element, the first line of sight direction refers to the line of sight direction of the driver determined according to the second data.

In the embodiment of the present application, the distance between the vehicle and the traffic element and the direction of the driver's line of sight are used for judgment to determine whether to send reminder information, which can improve the accuracy of the driving reminder, thereby improving the user experience. experience.

Optionally, the traffic elements may include: pedestrians, animals, vehicles, street lights, guardrails and other objects around the vehicle.

Optionally, the first sensor may include at least one of a global positioning system, an inertial measurement unit, a radar, a laser rangefinder, and a camera, and the first sensor may be used to collect the current speed, Acceleration, current position and contour, etc.

Optionally, the second sensor may comprise a camera or other sensor, and the second sensor may be used to collect data on the driver of the vehicle.

For example, the second sensor may be used to collect the head posture of the driver, such as roll, pitch and yaw of the driver's head.

The line-of-sight direction of the driver may be determined according to the head posture of the driver.

With reference to the first aspect, in some implementations of the first aspect, when the distance ranges to which the first distance belongs are different, the reminder levels of the reminder information are different.

In the embodiment of the present application, when the distance between the vehicle and the traffic element belongs to different distance ranges, the reminder level of the reminder information is different. Therefore, the reminder information can be made more accurate, thereby helping Further improve the accuracy of driving reminders.

Among them, the reminder information of different reminder levels makes the driver feel different importance or urgency.

With reference to the first aspect, in some implementations of the first aspect, at least one of the number of reminders, the frequency of reminders, and the intensity of reminders corresponding to the reminder information of different reminder levels is different.

In the embodiment of the present application, the reminder information of different reminder levels can be distinguished by at least one of the number of reminders, the reminder frequency, and the reminder intensity, which can make the reminder information more accurate, thereby helping to further improve the user experience.

With reference to the first aspect, in some implementations of the first aspect, the acquiring first data from a first sensor of the vehicle and second data from a second sensor of the vehicle includes: When the speed is less than or equal to the preset speed, the first data and the second data are acquired.

In the embodiment of the present application, the data collected by the sensor is acquired only when the speed of the vehicle satisfies the preset condition, which can reduce the power consumption of the vehicle.

In a second aspect, a driving reminder method is provided, the method comprising:

acquiring first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, the first data including data of traffic elements surrounding the vehicle, the second data including the vehicle data of the driver; send reminder information to the driver according to the first data and the second data, the reminder level of the reminder information and the distance range to which the first distance belongs, and the distance determined according to the second data. The driver's fatigue driving level is related, wherein the first distance refers to the distance between the vehicle and the traffic element determined according to the first data.

In the embodiment of the present application, the reminder level of the reminder information is determined according to the distance between the vehicle and the traffic element and the driver's fatigue driving level, which can make the reminder information more accurate, thereby helping To further improve the accuracy of driving reminders.

For example, the second sensor may also be used to collect the proportion of the driver's eyes closed time within a certain time interval, for example, the driver's PERCLOS (percentage of eyeIid CIosure over the PupiI, over time) physical quantity.

The driver's fatigue driving level may be determined according to the proportion of the driver's eyes closed time within a certain time interval.

With reference to the second aspect, in some implementations of the second aspect, when the driver's fatigue driving levels determined according to the second data are different, and the distance ranges to which the first distance belongs are the same, the The reminder level of the reminder message is different.

In the embodiment of the present application, when the driver's fatigue driving level is different, and the distance between the traffic element and the vehicle belongs to the same distance range, the reminder level of the reminder information is different. Therefore, all The above reminder information is more accurate, which helps to further improve the accuracy of driving reminders.

With reference to the second aspect, in some implementations of the second aspect, when the driver's fatigue driving level determined according to the second data is the same, and the distance ranges to which the first distance belongs are different, the The reminder level of the reminder message is different.

In the embodiment of the present application, when the driver's fatigue driving level is the same, and the distance range of the traffic element and the vehicle to which the distance belongs is different, the reminder level of the reminder information is different. Therefore, all The above reminder information is more accurate, which helps to further improve the accuracy of driving reminders.

With reference to the second aspect, in some implementations of the second aspect, at least one of the number of reminders, the frequency of reminders, and the intensity of reminders corresponding to the reminder information of different reminder levels is different.

In a third aspect, a driving reminder device is provided, including:

an acquisition unit, configured to acquire first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, the first data includes data of traffic elements around the vehicle, the second data The data includes the data of the driver of the vehicle; the sending unit is configured to send reminder information when the first distance is greater than or equal to a first threshold and the first sight direction does not pay attention to the traffic element, wherein the first A distance refers to the distance between the vehicle and the traffic element determined according to the first data, and the first line of sight direction refers to the line of sight direction of the driver determined according to the second data.

With reference to the third aspect, in some implementations of the third aspect, when the distance ranges to which the first distance belongs are different, the reminder levels of the reminder information are different.

With reference to the third aspect, in some implementations of the third aspect, at least one of the number of reminders, the frequency of reminders, and the intensity of reminders corresponding to the reminder information of different reminder levels is different.

In the embodiment of the present application, the reminder information of different reminder levels can be distinguished by at least one of the number of reminders, the reminder frequency and the reminder intensity, which can make the reminder information more accurate, thereby helping to further improve the user experience.

With reference to the third aspect, in some implementations of the third aspect, the obtaining unit is specifically configured to: obtain the first data and the first data when the speed of the vehicle is less than or equal to a preset speed Two data.

In a fourth aspect, a driving reminder device is provided, including:

an acquisition unit, configured to acquire first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, the first data includes data of traffic elements around the vehicle, the second data The data includes the data of the driver of the vehicle; the sending unit is configured to send reminder information to the driver according to the first data and the second data, the reminder level of the reminder information and the distance range to which the first distance belongs , and the driver's fatigue driving level determined according to the second data, wherein the first distance refers to the distance between the vehicle and the traffic element determined according to the first data.

With reference to the fourth aspect, in some implementations of the fourth aspect, when the driver's fatigue driving levels determined according to the second data are different, and the distance ranges to which the first distance belongs are the same, the The reminder level of the reminder message is different.

With reference to the fourth aspect, in some implementations of the fourth aspect, when the driver's fatigue driving level determined according to the second data is the same, and the distance ranges to which the first distance belongs are different, the The reminder level of the reminder message is different.

With reference to the fourth aspect, in some implementations of the fourth aspect, at least one of the number of reminders, the frequency of reminders, and the intensity of reminders corresponding to the reminder information of different reminder levels is different.

In a fifth aspect, a driving reminder device is provided, the device includes a storage medium and a central processing unit, the storage medium may be a non-volatile storage medium, and a computer-executable program is stored in the storage medium, the The central processing unit is connected to the non-volatile storage medium, and executes the computer-executable program to implement the first aspect or the method in any possible implementation manner of the first aspect.

In a sixth aspect, a driving reminder device is provided, the device includes a storage medium and a central processing unit, the storage medium may be a non-volatile storage medium, and a computer-executable program is stored in the storage medium, and the The central processing unit is connected to the non-volatile storage medium, and executes the computer-executable program to implement the method in the second aspect or any possible implementation manner of the second aspect.

A seventh aspect provides a chip, the chip includes a processor and a data interface, the processor reads instructions stored in a memory through the data interface, and executes the first aspect or any possible implementation of the first aspect method in method.

Optionally, as an implementation manner, the chip may further include a memory, in which instructions are stored, the processor is configured to execute the instructions stored in the memory, and when the instructions are executed, the The processor is configured to perform the method in the first aspect or any possible implementation of the first aspect.

In an eighth aspect, a chip is provided, the chip includes a processor and a data interface, the processor reads an instruction stored in a memory through the data interface, and executes the second aspect or any possible implementation of the second aspect method in method.

Optionally, as an implementation manner, the chip may further include a memory, in which instructions are stored, the processor is configured to execute the instructions stored in the memory, and when the instructions are executed, the The processor is configured to perform the method of the second aspect or any possible implementation of the second aspect.

In a ninth aspect, a computer-readable storage medium is provided, where the computer-readable medium stores program codes for device execution, the program codes including the first aspect or any possible implementation manner of the first aspect. method in the directive.

In a tenth aspect, a computer-readable storage medium is provided, where the computer-readable medium stores program codes for device execution, the program codes including the second aspect or any possible implementation manner of the second aspect. method in the directive.

In an eleventh aspect, a driving reminder system is provided, the system comprising the driving reminder device according to the third aspect or the fourth aspect or the fifth aspect.

A twelfth aspect provides an automobile, which includes the driving reminder device according to the third aspect or the fourth aspect or the fifth aspect.

Description of drawings

FIG. 1 is a schematic structural diagram of an automatic driving vehicle according to an embodiment of the present application;

FIG. 2 is a schematic structural diagram of an automatic driving system according to an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a system architecture to which an embodiment of the present application is applicable;

4 is a schematic block diagram of a driving reminder method provided by an embodiment of the present application;

5 is a schematic block diagram of a driving reminder method provided by another embodiment of the present application;

6 is a schematic block diagram of a driving reminder method provided by another embodiment of the present application;

7 is a schematic block diagram of a driving reminder method provided by another embodiment of the present application;

8 is a schematic block diagram of a driving reminder device provided by an embodiment of the present application;

FIG. 9 is a schematic block diagram of a driving reminder device provided by another embodiment of the present application;

FIG. 10 is a schematic block diagram of a driving reminder device provided by another embodiment of the present application.

detailed description

The technical solutions in the present application will be described below with reference to the accompanying drawings.

The technical solutions of the embodiments of the present application can be applied to various vehicles, and the vehicle may be a diesel locomotive, an intelligent electric vehicle, or a hybrid vehicle, or the vehicle may also be a vehicle of other power types, or the like, or the embodiment of the present application The technical solution can also be applied to various other vehicles, such as airplanes and ships, which are not limited in the embodiments of the present application.

For convenience of description, the technical solutions of the embodiments of the present application are described below by taking a vehicle as an example.

The vehicle in the embodiment of the present application may be an automatic driving vehicle. For example, the automatic driving vehicle may be configured with an automatic driving mode, and the automatic driving mode may be a fully automatic driving mode, or may also be a partial automatic driving mode. The embodiment is not limited to this.

The vehicle in this embodiment of the present application may also be configured with other driving modes, and the other driving modes may include one or more of a variety of driving modes, such as a sport mode, an economy mode, a standard mode, a snow mode, and a hill climbing mode. . The automatic driving vehicle can switch between the automatic driving mode and the above-mentioned various driving models (the driver drives the vehicle), which is not limited in the embodiment of the present application.

FIG. 1 is a functional block diagram of a vehicle 100 provided by an embodiment of the present application.

In one embodiment, the vehicle 100 is configured in a fully or partially autonomous driving mode.

For example, the vehicle 100 may control itself while in an autonomous driving mode, and may determine the current state of the vehicle and its surrounding environment through human manipulation, determine the likely behavior of at least one other vehicle in the surrounding environment, and determine the other vehicle's The vehicle 100 is controlled based on the determined information with a confidence level corresponding to the likelihood of performing the possible behavior. When the vehicle 100 is in an autonomous driving mode, the vehicle 100 may be placed to operate without human interaction.

Vehicle 100 may include various subsystems, such as travel system 102 , sensor system 104 , control system 106 , one or more peripherals 108 and power supply 110 , computer system 112 and user interface 116 .

Alternatively, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple elements. Additionally, each of the subsystems and elements of the vehicle 100 may be interconnected by wire or wirelessly.

The travel system 102 may include components that provide powered motion for the vehicle 100 . In one embodiment, propulsion system 102 may include engine 118 , energy source 119 , transmission 120 , and wheels/tires 121 . The engine 118 may be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine of an air oil engine and an electric motor, a hybrid engine of an internal combustion engine and an air compression engine. Engine 118 converts energy source 119 into mechanical energy.

Examples of energy sources 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source 119 may also provide energy to other systems of the vehicle 100 .

Transmission 120 may transmit mechanical power from engine 118 to wheels 121 . Transmission 120 may include a gearbox, a differential, and a driveshaft.

In one embodiment, transmission 120 may also include other devices, such as clutches. Among other things, the drive shaft may include one or more axles that may be coupled to one or more wheels 121 .

The sensor system 104 may include several sensors that sense information about the environment surrounding the vehicle 100 .

For example, the sensor system 104 may include a positioning system 122 (which may be a GPS system, a Beidou system or other positioning system), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and camera 130. The sensor system 104 may also include sensors of the internal systems of the vehicle 100 being monitored (eg, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, velocity, etc.). This detection and identification is a critical function for the safe operation of the autonomous vehicle 100 .

The positioning system 122 may be used to estimate the geographic location of the vehicle 100 . The IMU 124 is used to sense position and orientation changes of the vehicle 100 based on inertial acceleration. In one embodiment, IMU 124 may be a combination of an accelerometer and a gyroscope.

Radar 126 may utilize radio signals to sense objects within the surrounding environment of vehicle 100 . In some embodiments, in addition to sensing objects, radar 126 may be used to sense the speed and/or heading of objects.

The laser rangefinder 128 may utilize laser light to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, laser scanners, and one or more detectors, among other system components.

Camera 130 may be used to capture multiple images of the surrounding environment of vehicle 100 . Camera 130 may be a still camera or a video camera.

The control system 106 controls the operation of the vehicle 100 and its components. Control system 106 may include various elements including steering system 132 , throttle 134 , braking unit 136 , sensor fusion algorithms 138 , computer vision system 140 , route control system 142 , and obstacle avoidance system 144 .

The steering system 132 is operable to adjust the heading of the vehicle 100 . For example, in one embodiment it may be a steering wheel system.

The throttle 134 is used to control the operating speed of the engine 118 and thus the speed of the vehicle 100 .

The braking unit 136 is used to control the deceleration of the vehicle 100 . The braking unit 136 may use friction to slow the wheels 121 . In other embodiments, the braking unit 136 may convert the kinetic energy of the wheels 121 into electrical current. The braking unit 136 may also take other forms to slow the wheels 121 to control the speed of the vehicle 100 .

Computer vision system 140 may be operable to process and analyze images captured by camera 130 in order to identify objects and/or features in the environment surrounding vehicle 100 . The objects and/or features may include traffic signals, road boundaries and obstacles. Computer vision system 140 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 may be used to map the environment, track objects, estimate the speed of objects, and the like.

The route control system 142 is used to determine the travel route of the vehicle 100 . In some embodiments, the route control system 142 may combine data from the sensors 138 , the GPS 122 , and one or more predetermined maps to determine a driving route for the vehicle 100 .

The obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise traverse potential obstacles in the environment of the vehicle 100 .

Of course, in one example, the control system 106 may additionally or alternatively include components other than those shown and described. Alternatively, some of the components shown above may be reduced.

Vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through peripheral devices 108 . Peripherals 108 may include a wireless communication system 146 , an onboard computer 148 , a microphone 150 and/or a speaker 152 .

In some embodiments, peripherals 108 provide a means for a user of vehicle 100 to interact with user interface 116 . For example, the onboard computer 148 may provide information to the user of the vehicle 100 . User interface 116 may also operate on-board computer 148 to receive user input. The onboard computer 148 can be operated via a touch screen. In other cases, peripheral devices 108 may provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 150 may receive audio (eg, voice commands or other audio input) from a user of vehicle 100 . Similarly, speakers 152 may output audio to a user of vehicle 100 .

Wireless communication system 146 may wirelessly communicate with one or more devices, either directly or via a communication network. For example, wireless communication system 146 may use 3G cellular communications, such as CDMA, EVDO, GSM/GPRS, or 4G cellular communications, such as LTE. Or 5G cellular communications. The wireless communication system 146 may communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the wireless communication system 146 may communicate directly with the device using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems, for example, wireless communication system 146 may include one or more dedicated short range communications (DSRC) devices, which may include communication between vehicles and/or roadside stations public and/or private data communications.

The power supply 110 may provide power to various components of the vehicle 100 . In one embodiment, the power source 110 may be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such a battery may be configured as a power source to provide power to various components of the vehicle 100 . In some embodiments, power source 110 and energy source 119 may be implemented together, such as in some all-electric vehicles.

Some or all of the functions of the vehicle 100 are controlled by the computer system 112 . Computer system 112 may include at least one processor 113 that executes instructions 115 stored in a non-transitory computer-readable medium such as data storage device 114 . Computer system 112 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed fashion.

The processor 113 may be any conventional processor, such as a commercially available CPU. Alternatively, the processor may be a dedicated device such as an ASIC or other hardware-based processor. Although FIG. 1 functionally illustrates the processor, memory, and other elements of the computer 110 in the same block, one of ordinary skill in the art will understand that the processor, computer, or memory may actually be included in the same physical Multiple processors, computers, or memories within a housing. For example, the memory may be a hard drive or other storage medium located within an enclosure other than computer 110 . Thus, reference to a processor or computer will be understood to include reference to a collection of processors or computers or memories that may or may not operate in parallel. Rather than using a single processor to perform the steps described herein, some components such as the steering and deceleration components may each have their own processor that only performs computations related to component-specific functions .

In various aspects described herein, a processor may be located remotely from the vehicle and in wireless communication with the vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle while others are performed by a remote processor, including taking steps necessary to perform a single maneuver.

In some embodiments, data storage 114 may include instructions 115 (eg, program logic) executable by processor 113 to perform various functions of vehicle 100 , including those described above. Data storage 114 may also contain additional instructions, including sending data to, receiving data from, interacting with, and/or performing data processing on one or more of propulsion system 102 , sensor system 104 , control system 106 , and peripherals 108 . control commands.

In addition to the instructions 115, the data storage device 114 may store data such as road maps, route information, the vehicle's position, direction, speed, and other such vehicle data, among other information. Such information may be used by the vehicle 100 and the computer system 112 during operation of the vehicle 100 in autonomous, semi-autonomous and/or manual modes.

A user interface 116 for providing information to or receiving information from a user of the vehicle 100 . Optionally, the user interface 116 may include one or more input/output devices within the set of peripheral devices 108 , such as a wireless communication system 146 , an onboard computer 148 , a microphone 150 and a speaker 152 .

Computer system 112 may control functions of vehicle 100 based on input received from various subsystems (e.g., travel system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 may utilize input from control system 106 in order to control steering unit 132 to avoid obstacles detected by sensor system 104 and obstacle avoidance system 144 . In some embodiments, computer system 112 is operable to provide control of various aspects of vehicle 100 and its subsystems.

Alternatively, one or more of these components described above may be installed or associated with the vehicle 100 separately. For example, data storage device 114 may exist partially or completely separate from vehicle 100 . The above-described components may be communicatively coupled together in a wired and/or wireless manner.

Optionally, the above component is just an example. In practical applications, components in each of the above modules may be added or deleted according to actual needs, and FIG. 1 should not be construed as a limitation on the embodiments of the present application.

An autonomous vehicle traveling on a road, such as vehicle 100 above, can recognize objects within its surroundings to determine adjustments to current speed. The objects may be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object may be considered independently, and based on the object's respective characteristics, such as its current speed, acceleration, distance from the vehicle, etc., may be used to determine the speed at which the autonomous vehicle is to adjust.

Alternatively, the vehicle 100 or a computing device associated with the vehicle 100 (eg, computer system 112, computer vision system 140, data storage device 114 of FIG. 1) may be based on the characteristics of the identified objects and the state of the surrounding environment (eg, traffic, rain, ice on the road, etc.) to predict the behavior of the identified object. Optionally, each identified object is dependent on the behavior of the other, so it is also possible to predict the behavior of a single identified object by considering all identified objects together. The vehicle 100 can adjust its speed based on the predicted behavior of the identified object. In other words, the autonomous vehicle can determine what steady state the vehicle will need to adjust to (eg, accelerate, decelerate, or stop) based on the predicted behavior of the object. In this process, other factors may also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 in the road being traveled, the curvature of the road, the proximity of static and dynamic objects, and the like.

In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device may also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and/or maintains contact with objects in the vicinity of the autonomous vehicle (eg, , cars in adjacent lanes on the road) safe lateral and longitudinal distances.

The above-mentioned vehicle 100 can be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a lawn mower, a recreational vehicle, a playground vehicle, construction equipment, a tram, a golf cart, a train, a cart, etc. The embodiment is not particularly limited.

FIG. 2 is a schematic diagram of an automatic driving system provided by an embodiment of the present application.

The automatic driving system shown in FIG. 2 includes a computer system 101 , wherein the computer system 101 includes a processor 103 , and the processor 103 is coupled with a system bus 105 . The processor 103 may be one or more processors, each of which may include one or more processor cores. A video adapter 107, which can drive a display 109, is coupled to the system bus 105. System bus 105 is coupled to input-output (I/O) bus 113 through bus bridge 111 . I/O interface 115 is coupled to the I/O bus. I/O interface 115 communicates with various I/O devices, such as input device 117 (eg, keyboard, mouse, touch screen, etc.), media tray 121, (eg, CD-ROM, multimedia interface, etc.). Transceiver 123 (which can send and/or receive radio communication signals), camera 155 (which can capture still and moving digital video images) and external USB interface 125 . Wherein, optionally, the interface connected to the I/O interface 115 may be a USB interface.

The processor 103 may be any conventional processor, including a reduced instruction set computing (reduced instruction set computer, RISC) processor, a complex instruction set computing (complex instruction set computer, CISC) processor or a combination of the above. Alternatively, the processor may be a dedicated device such as an application specific integrated circuit (ASIC). Optionally, the processor 103 may be a neural network processor or a combination of a neural network processor and the above-mentioned conventional processors.

Alternatively, in various embodiments described herein, computer system 101 may be located remotely from the autonomous vehicle (eg, computer system 101 may be located in a cloud or server) and may communicate wirelessly with the autonomous vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the autonomous vehicle, others are performed by a remote processor, including taking actions required to perform a single maneuver.

Computer 101 may communicate with software deployment server 149 through network interface 129 . Network interface 129 is a hardware network interface, such as a network card. The network 127 may be an external network, such as the Internet, or an internal network, such as an Ethernet network or a virtual private network (VPN). Optionally, the network 127 may also be a wireless network, such as a WiFi network, a cellular network, and the like.

The hard disk drive interface is coupled to the system bus 105 . The hard drive interface is connected to the hard drive. System memory 135 is coupled to system bus 105 . Data running in system memory 135 may include operating system 137 and application programs 143 of computer 101 .

The operating system includes a parser 139 (shell) and a kernel (kernel) 141 . The shell is an interface between the user and the kernel of the operating system. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: waiting for user input, interpreting user input to the operating system, and processing various operating system output.

Kernel 141 consists of those parts of the operating system that manage memory, files, peripherals, and system resources. Interacting directly with hardware, the operating system kernel usually runs processes and provides inter-process communication, providing CPU time slice management, interrupts, memory management, IO management, and more.

The application 143 includes programs related to driving reminders, such as acquiring data collected by sensors of the vehicle (for example, the collected data may include data of traffic elements around the vehicle and data of the driver of the vehicle), The collected data is processed to obtain processing results (for example, the processing results can be used to indicate the driving status of the vehicle and the driving status of the driver of the vehicle), and combined with the processing results The driver sends a reminder message.

Application 143 may also exist on the system of software deployment server 149 (deploying server). In one embodiment, the computer system 101 may download the application program 143 from a software deployment server 149 (deploying server) when the application program 143 needs to be executed.

Sensor 153 is associated with computer system 101 . The sensor 153 is used to detect the environment around the computer 101 . For example, the sensor 153 can detect animals, cars, obstacles and pedestrian crossings, etc. Further sensors can also detect the environment around the above-mentioned animals, cars, obstacles and pedestrian crossings, such as: the environment around animals, for example, animals appear around other animals, weather conditions, ambient light levels, etc. The sensors 153 may also be used to obtain status information of the vehicle. For example, the sensor 153 may detect vehicle status information such as the position of the vehicle, the speed of the vehicle, the acceleration of the vehicle, and the attitude of the vehicle. Alternatively, if the computer 101 is located on an autonomous vehicle, the sensors may be cameras, infrared sensors, chemical detectors, microphones, and the like.

For example, the application program 143 may process the data of the traffic elements around the vehicle and the data of the driver of the vehicle collected by the sensor 153, obtain the processing result, and send reminder information to the driver in combination with the processing result. At this time, the reminder information is sent to the driver in combination with the processing result, so that the safety of the automatic driving of the vehicle can be improved.

The processing result may be used to indicate the driving state of the vehicle and the driving state of the driver of the vehicle. For example, the driving state of the vehicle may include the distance between the vehicle and the traffic element. The driving state of the driver may include the line of sight direction of the driver and/or the driver's fatigue driving level.

FIG. 3 is a schematic structural diagram of a driving reminder system to which an embodiment of the present application is applied. It should be understood that the architecture 300 shown in FIG. 3 is only an example and not a limitation, and the architecture 300 may include more or less steps, which are not limited in this embodiment of the present application.

As shown in FIG. 3 , the architecture 300 may include: an internal camera, a camera control unit (control unit for the rear-view camera and side view, TRSVC), a central gateway (gateway), a head unit (HU), a combination instrument (combination instrument, KOMBI), camera-based driving assistance system (camera-based driving assistance system), exterior camera, central display, etc.

The inner camera can be connected to the gateway through TRSVC, the outer camera can be connected to the gateway through KAFAS, and the gateway is then connected to the HU. The HU can be used to execute the driving reminder method in the embodiment of this application, and the HU can process the images captured by the camera. , and send reminder information to the driver in combination with the processing results.

The HU may include an inner camera data processing unit, an outer camera data processing unit, and a fusion processing unit. The inner camera data processing unit may be used to process images captured by the inner camera, and the outer camera data processing unit may be used to process images captured by the outer camera. The fusion processing unit may be configured to combine the processing results of the inner camera data (images captured by the inner camera) and the processing results of the outer camera data (images captured by the outer cameras) to send reminder information to the driver.

The central display screen may be a display screen in a vehicle information terminal system (car informatic device, CID). The central display screen may be connected to the HU and used as the main display screen of the vehicle. , that is, send a reminder to the driver.

KOMBI can have an independent system, KOMBI can transmit data with HU through APIX (automotive pixel link), KOMBI can also transmit images with HU through LVDS (low voltage differential signal), KOMBI can also include a sound module, which can emit Therefore, KOMBI can output the reminder information in the embodiment of the present application through sound information, that is, send the reminder information to the driver.

In addition, the gateway can be used to receive data from various modules and then transmit it to the HU; TRSVC can include driving recorders and other automatic driving assistance functions; KAFAS can be used to detect the driver's state and behavior.

It should be noted that the architecture 300 and each module in the architecture 300 are only examples and not limitations, and are not limited in the embodiments of the present application.

For example, the inner camera and the outer camera can also be other sensors. Correspondingly, the inner camera data processing unit and the outer camera data processing unit can be the data processing units of other sensors, and the TRSVC can also be a control unit based on other sensors or based on other sensors. The processing unit, KAFAS can also be a driver assistance system based on other sensors (or, a control unit based on other sensors or a processing unit based on other sensors).

The driving reminder method in the embodiment of the present application will be described in detail below with reference to FIGS. 4 to 7 .

FIG. 4 is a schematic flowchart of a driving reminder method 400 provided by an embodiment of the present application.

The method 400 shown in FIG. 4 may include

steps

410 and 420. It should be understood that the method 400 shown in FIG. 4 is only an example and not a limitation, and the method 400 may include more or less steps. This is not limited, and the steps are described in detail below.

The method 400 shown in FIG. 4 may be performed by the processor 113 in the vehicle 100 in FIG. 1 , or the method 400 may also be performed by the processor 103 in the autonomous driving system in FIG. 2 , or the method 400 may also be performed by The head unit (HU) in Figure 3 executes (eg, by a fusion processing unit in the head unit).

S410: Acquire first data from a first sensor of the vehicle and second data from a second sensor of the vehicle.

Wherein, the first data may include data of traffic elements around the vehicle, and the second data may include data of a driver of the vehicle.

The first sensor may comprise a sensor in the sensor system 104 of FIG. 1, which may be used to collect data on traffic elements surrounding the vehicle.

For example, the first sensor may be used to collect the current speed, acceleration, current position, contour, and the like of the traffic element.

Optionally, the first sensor may include at least one of a global positioning system 122 , an inertial measurement unit 124 , a radar 126 , a laser rangefinder 128 and a camera 130 .

For example, the first sensor may include one or more laser rangefinders for collecting ranging data of the vehicle relative to its surrounding traffic elements.

Alternatively, the first sensor may include one or more cameras for capturing images of traffic elements surrounding the vehicle.

It should be understood that the traffic elements may include pedestrians, animals, vehicles, street lights, guardrails, and other objects around the vehicle.

The second sensor, which may include the camera 130 of FIG. 1 or other sensors, may be used to collect data on the driver of the vehicle.

Optionally, the line-of-sight direction of the driver may be determined according to the head posture of the driver.

Optionally, the acquiring the first data from the first sensor of the vehicle and the second data from the second sensor of the vehicle may include:

The first data and the second data are acquired when the speed of the vehicle is less than or equal to a preset speed.

In the embodiment of the present application, the first data collected by the first sensor and the data collected by the second sensor are acquired only when a preset condition (for example, the speed of the vehicle is less than or equal to the preset speed) is satisfied. That is, in other cases, the first sensor and the second sensor can temporarily stop data collection, and therefore, the power consumption of the vehicle can be reduced.

S420, in the case that the first distance is greater than or equal to the first threshold and the first line of sight direction does not pay attention to the traffic element, send reminder information.

Wherein, the first distance may refer to the distance between the vehicle and the traffic element determined according to the first data, and the first sight direction may refer to the driver determined according to the second data direction of sight.

Optionally, the reminder information may be sound information, image information, or other information.

For example, the reminder information may be sound information emitted by a speaker inside the vehicle; alternatively, the reminder information may also be a central display screen (or may also be referred to as a central control display or a near-close screen) in the vehicle. The image information displayed on the vehicle, the image information displayed on the instrument screen of the vehicle, or the image information displayed on other devices such as the heads up display of the vehicle; or, the reminder information can also be the vibration of the steering wheel or seat, etc. information, and the specific form of the reminder information is not limited in the embodiments of the present application.

For example, when the distance between the vehicle and the vehicle in front is greater than or equal to 5 meters, and the driver is not looking straight ahead (that is, the driver's line of sight does not pay attention to the vehicle in front), the driver should Send reminders.

Further, when the distance ranges between the traffic element and the vehicle determined according to the first data belong to different distance ranges, the reminder levels of the reminder information may be different.

Optionally, the reminder information of different reminder levels makes the driver feel different levels of importance or urgency.

Optionally, at least one of reminder times, reminder frequencies and reminder strengths corresponding to the reminder information of different reminder levels is different.

For example, the reminder information may be sound information from a speaker inside the vehicle, the distance between the vehicle and the vehicle in front is greater than or equal to 5 meters, and the driver is not looking straight ahead (that is, the driver If the distance between the vehicle and the vehicle in front is greater than or equal to 6 meters, and the driver has not looked straight ahead (that is, any If the driver does not pay attention to the preceding vehicle in the direction of the driver's line of sight), a voice message with an increased volume is sent to the driver.

FIG. 5 is a schematic flowchart of a driving reminder method 500 provided by an embodiment of the present application.

The method 500 shown in FIG. 5 may include

steps

510, 520, 530, 540, 550 and 560. It should be understood that the method 500 shown in FIG. 5 is only an example and not a limitation, and the method 500 may include more There are more or fewer steps, which are not limited in the embodiments of the present application, and these steps are described in detail below.

The method 500 shown in FIG. 5 may be performed by the processor 113 in the vehicle 100 in FIG. 1 , or the method 500 may also be performed by the processor 103 in the autonomous driving system in FIG. 2 , or the method 500 may also be performed by The head unit (HU) in Figure 3 executes (eg, by a fusion processing unit in the head unit).

In the following description, the method 500 is executed by the HU as an example in combination with the scenario of the start reminder.

S510, start the preceding vehicle identification.

The HU can obtain the speed information of the vehicle in real time, and activate the preceding vehicle identification function when the speed of the vehicle meets the preset conditions.

The preceding vehicle identification can identify the position, shape, direction, speed, image, etc. of the preceding vehicle.

For example, when the speed of the vehicle is lower than 30km/h, the HU can obtain the image of the preceding vehicle through the front camera, compare the image obtained by the front camera with the model library, and identify the preceding vehicle (that is, the other vehicles in front of the mentioned vehicle).

The size of the image obtained by the front camera can be 1028*578. After processing the image, the HU can obtain an image with a size of 640*300, and input the image with a size of 640*300 into the model library for comparison.

Alternatively, HU can also identify other vehicles, pedestrians, and non-motor vehicles.

For the specific identification method in the S510, reference may be made to the prior art, which will not be repeated here.

S520, the distance between the vehicle and the preceding vehicle is detected.

When both the vehicle in front and the vehicle remain stationary for 5 seconds, the distance L1 between the vehicle and the vehicle in front is identified, and the change in the distance between the vehicles (ie the distance between the vehicle and the vehicle in front) is monitored in real time .

For example, the distance L1 between the vehicle and the preceding vehicle may be identified based on a vision-based ACC with a single camera: bounds on range and range rate accuracy.

For the specific detection method in the S520, reference may be made to the prior art, which will not be repeated here.

S530, the head posture of the driver is detected.

Optionally, the head posture of the driver may be detected when the preceding vehicle enters the starting state.

For example, when the distance between the vehicle and the preceding vehicle is greater than L2 (L1+1 meter), it may be determined that the preceding vehicle enters the starting state.

Specifically, when detecting the driver's head posture, the roll, pitch, and yaw angles of the driver's head can be obtained, and the obtained roll, pitch, and yaw can be processed. , to detect the driver's head posture.

For the specific detection method in the S530, reference may be made to the prior art, which will not be repeated here.

S540, determine whether to send reminder information to the driver.

For example, in the case where the driver's head posture is not within the range of the mid-face attitude, it may be determined that the driver is not looking directly at the vehicle in front, and accordingly, S550 may be performed; if the driver's head attitude is within the range of the mid-face attitude In this case, it may be determined that the driver is facing the vehicle in front, and accordingly, S560 may be performed.

S550, when the driver does not face the vehicle in front, send a reminder message to the driver.

For example, when the driver does not face the vehicle in front, a reminder can be sent to the driver through the sound information and the image on the central control screen (that is, a reminder information is sent to the driver).

Optionally, when the distance between the vehicle and the preceding vehicle reaches L3 (L2+1 meter), the reminder information may be sent to the driver again.

Further, the reminder information can last for a certain period of time each time the reminder is performed, and if the driver immediately faces the vehicle in front within the duration of the reminder information, the reminder information can be stopped immediately.

S560, when the driver faces the vehicle in front, do not send reminder information to the driver.

FIG. 6 is a schematic flowchart of a driving reminder method 600 provided by an embodiment of the present application.

The method 600 shown in FIG. 6 may include

steps

610 and 620. It should be understood that the method 600 shown in FIG. 6 is only an example and not a limitation, and the method 600 may include more or less steps. This is not limited, and these steps are described in detail below.

The method 600 shown in FIG. 6 may be performed by the processor 113 in the vehicle 100 in FIG. 1 , or the method 600 may be performed by the processor 103 in the autonomous driving system in FIG. 2 , or the method 600 may also be performed by The head unit (HU) in Figure 3 executes (eg, by a fusion processing unit in the head unit).

S610: Acquire first data from a first sensor of the vehicle and second data from a second sensor of the vehicle.

For example, the second sensor may also be used to collect the percentage of the driver's eyes closed time within a certain time interval, for example, the driver's PERCLOS (percentage of eyeIid CIosure over the PupiI, over time) physical quantity.

Optionally, the driver's fatigue driving level may be determined according to the proportion of the driver's eyes closed time within a certain time interval.

S620. Send reminder information to the driver according to the first data and the second data.

The reminder level of the reminder information may be related to the distance range to which the first distance belongs and the driver's fatigue driving level determined according to the second data.

Optionally, the first distance may refer to the distance between the vehicle and the traffic element determined according to the first data.

That is, the reminder level of the reminder information may be determined in combination with the distance between the vehicle and the traffic element and the driver's fatigue driving level.

The reminder information may be sound information, image information or other information.

For example, the reminder information may be sound information emitted by a speaker inside the vehicle; or, the reminder information may also be a central display screen (or may also be referred to as a central control display or a near-close screen) in the vehicle. The image information displayed on the vehicle, the image information displayed on the instrument screen of the vehicle, or the image information displayed on other devices such as the heads up display of the vehicle; or, the reminder information can also be the vibration of the steering wheel or seat, etc. information, and the specific form of the reminder information is not limited in the embodiments of the present application.

Optionally, when the driver's fatigue driving levels determined according to the second data are different, and the distance ranges to which the first distance belongs are the same, the reminder levels of the reminder information may be different.

For example, when the driver is slightly fatigued and the distance between the vehicle and the vehicle in front is less than or equal to 100 meters, it can be determined that the reminder level of the reminder information is Level 1, that is, the reminder information ( That is, the volume of sound information) is 8, and a beep is heard.

When the driver is driving with moderate fatigue, and the distance between the vehicle and the vehicle in front is less than or equal to 100 meters (that is, the distance between the vehicle and the vehicle in front is the same), the reminder information may be determined The reminder level is level 2, that is, the volume of the reminder information (that is, the sound information) is 8, with two beeps.

Optionally, when the fatigue driving levels of the drivers determined according to the second data are the same, and the distance ranges to which the first distance belongs are different, the reminder levels of the reminder information may be different.

For example, when the driver is slightly fatigued and the distance between the vehicle and the vehicle in front is greater than 100 meters, it may be determined that the reminder level of the reminder information is level 1, that is, the reminder information (ie the sound information) at volume 8, beep once.

When the driver is driving with mild fatigue, and the distance between the vehicle and the vehicle in front is less than or equal to 100 meters, the reminder level of the reminder information can be adjusted to the second level, that is, the reminder information (that is, the sound information ) at volume 12, beep twice.

FIG. 7 is a schematic flowchart of a driving reminder method 700 provided by an embodiment of the present application.

The method 700 shown in FIG. 7 may include

steps

710, 720, 730 and 740. It should be understood that the method 700 shown in FIG. 7 is only an example and not a limitation, and the method 700 may include more or less steps , this is not limited in the embodiments of the present application, and the steps are described in detail below.

The method 700 shown in FIG. 7 may be performed by the processor 113 in the vehicle 100 in FIG. 1 , or the method 700 may also be performed by the processor 103 in the autonomous driving system in FIG. 2 , or the method 700 may also be performed by The head unit (HU) in Figure 3 executes (eg, by a fusion processing unit in the head unit).

The following description will be given by taking the method 700 being executed by the HU as an example in conjunction with the scenario of the fatigue driving reminder.

S710, determining the driver's fatigue driving level.

Optionally, whether the driver is fatigued driving and the fatigue driving level may be determined according to the proportion of the eyes closed time of the driver's eyes within a certain time interval.

For example, based on the physical quantity of PERCLOS (percentage of eyeIid CIosure over the PupiI, over time), the eye (surrounding image) can be collected at a frequency of 4fps, and the vertical integral projection of the eye (surrounding image) can be performed to judge the closed eye state, thereby Determine the driver's fatigue driving level.

Optionally, within a time window of 8 seconds (s), the fatigue driving level may be determined according to the proportion of eyes closed time, and fatigue driving may be defined to include three levels.

For example, the closed eye time can be taken as 20% of the total time window, that is, the eye closed time is 0.2*8s=1.6s, which is defined as mild fatigue; the eye closed time can be taken as 50% of the total time window, that is, the eye closed time It is 0.5*8s=4.0s, which is defined as moderate fatigue; the eye-closing time can be 80% of the total time window, that is, the eye-closing time is 0.8*8s=6.4s, which is defined as severe fatigue.

For the specific method for determining the fatigue driving level in the S710, reference may be made to the prior art, which will not be repeated here.

S720, the distance between the vehicle and the preceding vehicle is detected.

For example, the distance between the vehicle and the preceding vehicle may be identified based on a vision-based ACC with a single camera: bounds on range and range rate accuracy.

Optionally, the distance between the vehicle and the preceding vehicle may be classified into multiple levels.

For example, a distance between the vehicle and the preceding vehicle of 100 meters may be defined as the first level, 80 meters as the second level, and 50 meters as the third level.

For the specific detection method in the S720, reference may be made to the prior art, which will not be repeated here.

S730, adjust the level of the fatigue driving reminder information.

Optionally, the level of the fatigue driving reminder can be adjusted according to the distance between the vehicle and the preceding vehicle.

For example, in the case that the driver is slightly fatigued, when there is no vehicle in front of the vehicle or the distance between the vehicle and the vehicle in front is the first level, the fatigue driving reminder information is determined to be the first level; When the distance between the vehicle and the preceding vehicle is the second level, the fatigue driving reminder information is adjusted to the second level reminder; when the distance between the vehicle and the preceding vehicle is the third level, the fatigue driving reminder information is adjusted. for the third level.

Optionally, in the case where the fatigue driving reminder information is the third level, if the driver changes from mild fatigue to moderate fatigue after the fatigue driving reminder information is sent to the driver, the third level is sent to the driver again. Fatigue driving reminder information.

S740, sending fatigue driving reminder information to the driver.

Optionally, when the fatigue driving reminder information is sound information and the fatigue driving reminder information includes three levels, different levels of the fatigue driving reminder information may correspond to different reminder modes respectively.

For example, the volume of the sound information corresponding to the first-level fatigue driving reminder information may be 8, with one beep; the volume of the sound information corresponding to the second-level fatigue driving reminder information may be 12, with two continuous beeps; The volume of the sound information corresponding to the fatigue driving reminder information can be 16, with three consecutive beeps.

Optionally, the foregoing method 700 is merely an example and not a limitation, and the reminder information may also be image information, vibration information, or other information, etc., which is not limited in this embodiment of the present application.

FIG. 8 is a schematic block diagram of a driving reminder device 800 provided by an embodiment of the present application. It should be understood that the driving reminder device 800 shown in FIG. 8 is only an example, and the device 800 in this embodiment of the present application may further include other modules or units.

In some possible implementations, the apparatus 800 can perform various steps in the methods of FIGS. 4 and 5 .

For example, the acquiring unit 810 can be used to perform S410 in the method 400, and the sending unit 820 can be used to perform S420 in the method 400; It can be used to perform S540 , S550 and S560 in the method 500 .

The apparatus 800 may be specifically as follows:

The obtaining unit 810 is configured to obtain first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, where the first data includes data of traffic elements around the vehicle, and the first data includes data of traffic elements around the vehicle. 2. The data includes data of the driver of the vehicle;

The sending unit 820 is configured to send reminder information when the first distance is greater than or equal to the first threshold and the first line of sight direction does not pay attention to the traffic element.

Wherein, the first distance refers to the distance between the vehicle and the traffic element determined according to the first data, and the first line of sight direction refers to the line of sight of the driver determined according to the second data direction.

Optionally, when the distance ranges to which the first distance belongs are different, the reminder levels of the reminder information are different.

Optionally, the acquiring unit 810 is specifically configured to acquire the first data and the second data when the speed of the vehicle is less than or equal to a preset speed.

It should be understood that the driving reminder device 800 here is embodied in the form of functional modules. The term "module" here can be implemented in the form of software and/or hardware, which is not specifically limited. For example, a "module" may be a software program, a hardware circuit, or a combination of the two that implement the above-mentioned functions. The hardware circuits may include application specific integrated circuits (ASICs), electronic circuits, processors for executing one or more software or firmware programs (eg, shared processors, proprietary processors, or group processors) etc.) and memory, merge logic and/or other suitable components to support the described functions.

As an example, the driving reminder device 800 provided in this embodiment of the present application may be a processor in an automatic driving system, or may be a car machine in an automatic driving vehicle, or may also be a chip configured in the car machine , so as to execute the method described in the embodiments of the present application.

FIG. 9 is a schematic block diagram of a driving reminder device 900 provided by an embodiment of the present application. It should be understood that the driving reminder device 900 shown in FIG. 9 is only an example, and the device 900 in this embodiment of the present application may further include other modules or units.

In some possible implementations, the apparatus 900 can perform various steps in the methods of FIGS. 6 and 7 .

For example, the acquiring unit 910 can be used to perform S610 in the method 600, and the sending unit 920 can be used to perform S620 in the method 600; S730 and S740 in the method 700 are performed.

The apparatus 900 may be specifically as follows:

The obtaining unit 910 is configured to obtain first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, where the first data includes data of traffic elements around the vehicle, and the first data includes data of traffic elements around the vehicle. 2. The data includes data of the driver of the vehicle;

The sending unit 920 is configured to send reminder information to the driver according to the first data and the second data, the reminder level of the reminder information and the distance range to which the first distance belongs, and the range determined according to the second data. The driver's fatigue driving level is related.

The first distance refers to the distance between the vehicle and the traffic element determined according to the first data.

Optionally, when the driver's fatigue driving levels determined according to the second data are different, and the distance ranges to which the first distance belongs are the same, the reminder levels of the reminder information are different.

Optionally, when the fatigue driving levels of the drivers determined according to the second data are the same, and the distance ranges to which the first distance belongs are different, the reminder levels of the reminder information are different.

It should be understood that the driving reminder device 900 here is embodied in the form of functional modules. The term "module" here can be implemented in the form of software and/or hardware, which is not specifically limited. For example, a "module" may be a software program, a hardware circuit, or a combination of the two that implement the above-mentioned functions. The hardware circuits may include application specific integrated circuits (ASICs), electronic circuits, processors for executing one or more software or firmware programs (eg, shared processors, proprietary processors, or group processors) etc.) and memory, merge logic and/or other suitable components to support the described functions.

As an example, the driving reminder device 900 provided in this embodiment of the present application may be a processor in an automatic driving system, or may also be a car machine in an automatic driving vehicle, or may also be a chip configured in the car machine , so as to execute the method described in the embodiments of the present application.

FIG. 10 is a schematic block diagram of a driving reminder device 1000 according to an embodiment of the present application. The apparatus 1000 shown in FIG. 10 includes a memory 1001 , a processor 1002 , a communication interface 1003 and a bus 1004 . The memory 1001 , the processor 10102 , and the communication interface 1003 are connected to each other through the bus 1004 for communication.

In some examples, the apparatus 1000 may be an exemplary structure of the transmitting unit in FIG. 8 , or may be an exemplary structure of a chip that can be applied in the transmitting unit. In this example, the apparatus 1000 may be used to perform the steps or operations performed by the sending unit in the method shown in FIG. 8 .

In other examples, the apparatus 1000 may be an exemplary structure of the sending unit in FIG. 9 , or may be an exemplary structure of a chip that can be applied in the sending unit. In this example, the apparatus 1000 may be configured to perform the steps or operations performed by the sending unit in the method shown in FIG. 9 .

The memory 1001 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 101 may store a program, and when the program stored in the memory 1001 is executed by the processor 1002, the processor 1002 is used to execute each step of the driving reminder method of the embodiment of the present application. For example, FIG. 4, FIG. 5, FIG. 6 and each step of the embodiment shown in FIG. 7 .

The processor 1002 can use a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for executing related programs to The method for realizing the driving reminder of the method embodiment of the present application is implemented.

The processor 1002 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the driving reminder method in the embodiment of the present application may be completed by an integrated logic circuit of hardware in the processor 1002 or an instruction in the form of software.

The above-mentioned processor 1002 can also be a general-purpose processor, a digital signal processor (digital signal processing, DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, Discrete gate or transistor logic devices, discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory 1001, and the processor 1002 reads the information in the memory 1001 and, in combination with its hardware, completes the functions required to be performed by the units included in the driving reminder device in the embodiment of the present application, or executes the driving of the method embodiment of the present application. The reminding method, for example, can perform each step/function of the embodiment shown in FIG. 4 , FIG. 5 , FIG. 6 and FIG. 7 .

The communication interface 1003 can use, but is not limited to, a transceiver such as a transceiver to implement communication between the device 1000 and other devices or a communication network.

The bus 1004 may include a pathway for communicating information between the various components of the apparatus 1000 (eg, the memory 1001, the processor 1002, the communication interface 1003).

It should be understood that the apparatus 1000 shown in the embodiments of the present application may be a processor in an automatic driving system, or may also be a vehicle machine in an automatic driving vehicle, or may also be a chip configured in the vehicle machine.

It should be understood that the processor in the embodiment of the present application may be a central processing unit (central processing unit, CPU), and the processor may also be other general-purpose processors, digital signal processors (digital signal processors, DSP), application-specific integrated circuits (application specific integrated circuit, ASIC), off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

It should also be understood that the memory in the embodiments of the present application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically programmable Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory. Volatile memory may be random access memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (DRAM) Access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory Fetch memory (synchlink DRAM, SLDRAM) and direct memory bus random access memory (direct rambus RAM, DR RAM).

The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center by wire (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server, a data center, or the like containing one or more sets of available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media. The semiconductor medium may be a solid state drive.

It should be understood that the term "and/or" in this document is only an association relationship to describe associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, which can mean that A exists alone, and A and B exist at the same time , there are three cases of B alone, where A and B can be singular or plural. In addition, the character "/" in this document generally indicates that the related objects before and after are an "or" relationship, but may also indicate an "and/or" relationship, which can be understood with reference to the context.

In this application, "at least one" means one or more, and "plurality" means two or more. "At least one item(s) below" or similar expressions thereof refer to any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one item (a) of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple .

It should be understood that, in various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not be dealt with in the embodiments of the present application. implementation constitutes any limitation.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution, and the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

A driving reminder method, comprising:

acquiring first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, the first data including data of traffic elements surrounding the vehicle, the second data including the vehicle the driver's data;

In the case that the first distance is greater than or equal to the first threshold and the first line of sight direction does not pay attention to the traffic element, sending reminder information, wherein the first distance refers to the distance between the vehicle determined according to the first data and the traffic element. the distance between the traffic elements, and the first line of sight direction refers to the line of sight direction of the driver determined according to the second data.
The method according to claim 1, wherein when the distance ranges to which the first distance belongs are different, the reminder levels of the reminder information are different.
The method according to claim 2, wherein at least one of reminder times, reminder frequency and reminder intensity corresponding to the reminder information of different reminder levels is different.
The method according to any one of claims 1 to 3, wherein the acquiring the first data from the first sensor of the vehicle and the second data from the second sensor of the vehicle comprises:

The first data and the second data are acquired when the speed of the vehicle is less than or equal to a preset speed.
A driving reminder method, comprising:

acquiring first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, the first data including data of traffic elements surrounding the vehicle, the second data including the vehicle the driver's data;

Send reminder information to the driver according to the first data and the second data, the distance range to which the reminder level of the reminder information and the first distance belong, and the fatigue of the driver determined according to the second data The driving level is related, wherein the first distance refers to the distance between the vehicle and the traffic element determined according to the first data.
The method according to claim 5, wherein when the driver's fatigue driving level determined according to the second data is different, and the distance range to which the first distance belongs is the same, the reminder information Alert levels are different.
The method according to claim 5 or 6, characterized in that, when the driver's fatigue driving level determined according to the second data is the same, and the distance ranges to which the first distance belongs are different, the reminding The alert level of the message is different.
The method according to any one of claims 5 to 7, wherein at least one of the number of reminders, the frequency of reminders and the intensity of reminders corresponding to the reminder information of different reminder levels is different.
A driving reminder device, characterized in that it includes:

an acquisition unit, configured to acquire first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, the first data includes data of traffic elements around the vehicle, the second data the data includes data of the driver of the vehicle;

The sending unit is configured to send reminder information when the first distance is greater than or equal to the first threshold and the first line of sight direction does not pay attention to the traffic element, wherein the first distance refers to a value determined according to the first data The distance between the vehicle and the traffic element, and the first line of sight direction refers to the line of sight direction of the driver determined according to the second data.
The device according to claim 9, wherein when the distance ranges to which the first distance belongs are different, the reminder levels of the reminder information are different.
The device according to claim 10, wherein at least one of the number of reminders, the frequency of reminders and the intensity of reminders corresponding to the reminder information of different reminder levels is different.
The device according to any one of claims 9 to 11, wherein the acquiring unit is specifically configured to:

The first data and the second data are acquired when the speed of the vehicle is less than or equal to a preset speed.
A driving reminder device, characterized in that it includes:

an acquisition unit, configured to acquire first data from a first sensor of the vehicle and second data from a second sensor of the vehicle, the first data includes data of traffic elements around the vehicle, the second data the data includes data of the driver of the vehicle;

The sending unit is configured to send reminder information to the driver according to the first data and the second data, the reminder level of the reminder information and the distance range to which the first distance belongs, and the range determined according to the second data. is related to the driver's fatigue driving level, wherein the first distance refers to the distance between the vehicle and the traffic element determined according to the first data.
The device according to claim 13, wherein when the driver's fatigue driving level determined according to the second data is different, and the distance range to which the first distance belongs is the same, the reminder information Alert levels are different.
The device according to claim 13 or 14, wherein when the driver's fatigue driving level determined according to the second data is the same, and the distance ranges to which the first distance belongs are different, the reminder The alert level of the message is different.
The device according to any one of claims 13 to 15, wherein at least one of the number of reminders, the frequency of reminders and the intensity of reminders corresponding to the reminder information of different reminder levels is different.
A driving reminder device, characterized in that it comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions to execute any one of claims 1 to 8. method.
A driving reminder system, characterized by comprising the device according to any one of claims 9 to 17.
An automobile, characterized by comprising the device of any one of claims 9 to 17.
A computer-readable storage medium, characterized in that, program instructions are stored in the computer-readable storage medium, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented .
A chip, characterized in that the chip includes a processor and a data interface, and the processor reads an instruction stored in a memory through the data interface, so as to execute the method according to any one of claims 1 to 8 method.