CN111361566B - Takeover reminding method for automatic driving vehicle, vehicle-mounted equipment and storage medium - Google Patents
Takeover reminding method for automatic driving vehicle, vehicle-mounted equipment and storage medium Download PDFInfo
- Publication number
- CN111361566B CN111361566B CN201811486409.3A CN201811486409A CN111361566B CN 111361566 B CN111361566 B CN 111361566B CN 201811486409 A CN201811486409 A CN 201811486409A CN 111361566 B CN111361566 B CN 111361566B
- Authority
- CN
- China
- Prior art keywords
- reminding
- information
- vehicle
- takeover
- driver
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000008447 perception Effects 0.000 claims abstract description 44
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000001360 synchronised effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001953 sensory effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60N—SEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
- B60N2/00—Seats specially adapted for vehicles; Arrangement or mounting of seats in vehicles
- B60N2/002—Seats provided with an occupancy detection means mounted therein or thereon
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0818—Inactivity or incapacity of driver
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0818—Inactivity or incapacity of driver
- B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
Landscapes
- Engineering & Computer Science (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Aviation & Aerospace Engineering (AREA)
- Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Mathematical Physics (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention relates to a takeover reminding method of an automatic driving vehicle, a vehicle-mounted device and a storage medium, wherein the method comprises the following steps: acquiring image information of a vehicle driving position, perception information of the vehicle driving position and automatic vehicle driving information; determining driver state information based on the image information and the perception information; determining a takeover reminding type based on the driver state information and the vehicle automatic driving information; and performing takeover reminding based on the takeover reminding type. According to the embodiment of the invention, different states of the driver can be determined based on the image information and the perception information of the driving position of the vehicle, different takeover reminding types can be further determined based on different states of the driver, and different takeover reminding strategies can be adopted according to different states of the driver for the automatic driving vehicle at the level of L3, so that the driver can take over the vehicle within the preset time after the vehicle prompts takeover.
Description
Technical Field
The embodiment of the invention relates to the technical field of automatic driving of vehicles, in particular to a takeover reminding method of an automatic driving vehicle, vehicle-mounted equipment and a storage medium.
Background
In the process of driving a vehicle by a driver, the existing driver fatigue monitoring equipment monitors the face and eyes of the driver in real time, and then judges whether the driver is in a fatigue state.
For autonomous vehicles of the L3 class, where most driving operations are performed by the vehicle, the driver does not need to be in constant control of the vehicle, but rather the driver must ensure take-over capability for the vehicle to handle special situations. For example, when the vehicle is traveling on a highway, the driving operation is completed by the vehicle, and when the vehicle enters a low-speed highway from the highway, the vehicle immediately informs the driver to take over the driving; for another example, when the vehicle's autonomous driving system fails, the vehicle may immediately notify the driver to take over driving. Therefore, the driver needs to take over the vehicle for a preset time after the vehicle prompts take over.
Therefore, for the automatic driving vehicle at the level of L3, it is only necessary to judge whether the driver can take over the vehicle, and therefore, a method for reminding the take over of the automatic driving vehicle is urgently needed to meet the requirement of the automatic driving vehicle at the level of L3 on the detection of the state of the driver.
Disclosure of Invention
In order to solve the problems in the prior art, at least one embodiment of the invention provides a takeover reminding method for an automatic driving vehicle, a vehicle-mounted device and a storage medium.
In a first aspect, an embodiment of the present invention provides a takeover reminding method for an autonomous vehicle, where the method includes:
acquiring image information of a vehicle driving position, perception information of the vehicle driving position and automatic vehicle driving information;
determining driver state information based on the image information and the perception information;
determining a takeover reminding type based on the driver state information and the vehicle automatic driving information;
and performing takeover reminding based on the takeover reminding type.
In some embodiments, the perceptual information comprises: a pressure value detected by a seat sensor of the vehicle driving seat;
the determining driver state information based on the image information and the perception information includes:
determining the state information of the driver in the sleeping state as first state information based on the image information;
determining the state information of the driver with the pressure value lower than a preset pressure value as second state information based on the perception information;
and determining the driver state information of which the pressure value is not lower than a preset pressure value as third state information based on the perception information.
In some embodiments, the determining a takeover reminder type based on the driver state information and the vehicle autopilot information includes:
after determining the automatic driving information of the vehicle meeting the preset takeover condition, determining that the takeover reminding type corresponding to the first state information is a first type;
determining that the takeover reminding type corresponding to the second state information is a second type;
and after the automatic driving information of the vehicle meeting the preset takeover condition is determined, determining that the takeover reminding type corresponding to the third state information is a third type.
In some embodiments, the performing the takeover reminder based on the takeover reminder type includes:
performing takeover reminding with first reminding intensity based on the first type;
performing takeover reminding with second reminding intensity based on the second type;
performing takeover reminding with third reminding intensity based on the third type;
the first reminding strength is smaller than the second reminding strength, and the first reminding strength is larger than the third reminding strength.
In some embodiments, the preset takeover condition comprises: vehicle autopilot system failure.
In a second aspect, an embodiment of the present invention further provides a takeover reminding device for an autonomous vehicle, where the device includes:
the system comprises an acquisition unit, a display unit and a control unit, wherein the acquisition unit is used for acquiring image information of a vehicle driving position, perception information of the vehicle driving position and automatic vehicle driving information;
a first determination unit configured to determine driver state information based on the image information and the perception information;
the second determination unit is used for determining the type of taking over reminding based on the driver state information and the automatic vehicle driving information;
and the reminding unit is used for carrying out takeover reminding based on the takeover reminding type.
In some embodiments, the perceptual information comprises: a pressure value detected by a seat sensor of the vehicle driving seat;
the first determining unit is configured to:
determining the state information of the driver in the sleeping state as first state information based on the image information;
determining the state information of the driver with the pressure value lower than a preset pressure value as second state information based on the perception information;
and determining the driver state information of which the pressure value is not lower than a preset pressure value as third state information based on the perception information.
In some embodiments, the second determining unit is configured to:
after determining the automatic driving information of the vehicle meeting the preset takeover condition, determining that the takeover reminding type corresponding to the first state information is a first type;
determining that the takeover reminding type corresponding to the second state information is a second type;
and after the automatic driving information of the vehicle meeting the preset takeover condition is determined, determining that the takeover reminding type corresponding to the third state information is a third type.
In some embodiments, the reminding unit is configured to:
performing takeover reminding with first reminding intensity based on the first type;
performing takeover reminding with second reminding intensity based on the second type;
performing takeover reminding with third reminding intensity based on the third type;
the first reminding strength is smaller than the second reminding strength, and the first reminding strength is larger than the third reminding strength.
In some embodiments, the preset takeover condition comprises: vehicle autopilot system failure.
In a third aspect, an embodiment of the present invention further provides an on-board device, including:
a processor, memory, a network interface, and a user interface;
the processor, memory, network interface and user interface are coupled together by a bus system;
the processor is adapted to perform the steps of the method according to the first aspect by calling a program or instructions stored by the memory.
In a fourth aspect, an embodiment of the present invention also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the steps of the method according to the first aspect.
It can be seen that in at least one embodiment of the embodiments of the present invention, different states of a driver can be determined based on image information and perception information of a driving location of a vehicle, and then different takeover reminding types can be determined based on different states of the driver, and for an L3-class autonomous vehicle, different types of takeover reminding strategies can be adopted according to different states of the driver, so that the driver can take over the vehicle within a preset time period after the vehicle prompts takeover, and the driver does not need to control the vehicle all the time.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of an on-board device according to an embodiment of the present invention;
fig. 2 is a flowchart of a takeover reminding method for an autonomous vehicle according to an embodiment of the present invention;
fig. 3 is a block diagram of a take-over reminding device for an autonomous vehicle according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
Fig. 1 is a schematic structural diagram of an in-vehicle device according to an embodiment of the present invention.
The in-vehicle apparatus shown in fig. 1 includes: at least one processor 101, at least one memory 102, at least one network interface 104, and other user interfaces 103. The various components in the in-vehicle device are coupled together by a bus system 105. It is understood that the bus system 105 is used to enable communications among the components. The bus system 105 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 105 in FIG. 1.
The user interface 103 may include, among other things, a display, a keyboard, or a pointing device (e.g., a mouse, trackball, or touch pad, among others.
It will be appreciated that the memory 102 in this embodiment may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be a Read-only memory (ROM), a programmable Read-only memory (PROM), an erasable programmable Read-only memory (erasabprom, EPROM), an electrically erasable programmable Read-only memory (EEPROM), or a flash memory. The volatile memory may be a Random Access Memory (RAM) which functions as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (staticiram, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (syncronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM ), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and direct memory bus RAM (DRRAM). The memory 102 described herein is intended to comprise, without being limited to, these and any other suitable types of memory.
In some embodiments, memory 102 stores elements, executable units or data structures, or a subset thereof, or an expanded set thereof as follows: an operating system 1021 and application programs 1022.
The operating system 1021 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, and is used for implementing various basic services and processing hardware-based tasks. The application 1022 includes various applications, such as a media player (MediaPlayer), a Browser (Browser), and the like, for implementing various application services. Programs that implement methods in accordance with embodiments of the invention can be included in application 1022.
In the embodiment of the present invention, the processor 101 calls a program or an instruction stored in the memory 102, specifically, may be a program or an instruction stored in the application 1022, and the processor 101 is configured to execute the method steps provided by each embodiment of the takeover reminding method for an autonomous vehicle, for example, the method steps include the following steps one to four:
acquiring image information of a vehicle driving position, perception information of the vehicle driving position and automatic driving information of a vehicle;
secondly, determining the state information of the driver based on the image information and the perception information;
determining a takeover reminding type based on the driver state information and the vehicle automatic driving information;
and fourthly, performing takeover reminding based on the takeover reminding type.
The method disclosed by the above embodiment of the present invention can be applied to the processor 101, or implemented by the processor 101. The processor 101 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 101. The processor 101 may be a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software elements in the decoding processor. The software elements may be located in ram, flash, rom, prom, or eprom, registers, among other storage media that are well known in the art. The storage medium is located in the memory 102, and the processor 101 reads the information in the memory 102 and completes the steps of the method in combination with the hardware thereof.
It is to be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the processing units may be implemented within one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.
For a software implementation, the techniques described herein may be implemented by means of units performing the functions described herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present application, it should be understood that the execution sequence of the steps of the method embodiments can be arbitrarily adjusted unless there is an explicit precedence sequence. The disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or make a contribution to the prior art, or may be implemented in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
Fig. 2 is a flowchart of a take-over reminding method for an autonomous vehicle according to an embodiment of the present invention. The execution subject of the method is the vehicle-mounted equipment.
As shown in fig. 2, the method for reminding take-over of an autonomous vehicle disclosed in this embodiment may include the following steps 201 and 204:
201. the image information of the vehicle driving position, the perception information of the vehicle driving position and the automatic driving information of the vehicle are obtained.
202. Based on the image information and the perception information, driver state information is determined.
203. And determining the type of taking over reminding based on the driver state information and the automatic driving information of the vehicle.
204. And performing takeover reminding based on the takeover reminding type.
In this embodiment, an image sensor may be installed in the autonomous vehicle at a location that satisfies the need for the image sensor to capture the face of the driver without affecting the operation of the vehicle by the driver. The image sensor can collect image information of the driving position of the vehicle, and if a driver sits on the driving position, the image information contains information of the driver.
In this embodiment, the image sensor is, for example, a camera, and the image sensor may also be other devices capable of acquiring image information, which is not described herein again.
In this embodiment, the vehicle-mounted device is in communication connection with the image sensor, and can acquire image information of a driving seat of the vehicle.
In this embodiment, after the vehicle-mounted device obtains the image information of the driving seat of the vehicle, the vehicle-mounted device may process the image information to identify the face of the driver, and the processing manner may continue to use the prior art, which is not described herein again. The in-vehicle apparatus may determine whether the driver is in a sleep state after processing the image information.
In this embodiment, a perception sensor may be installed in the driving seat of the autonomous vehicle. The perception sensor can collect perception information of a vehicle driving seat. The vehicle-mounted equipment is in communication connection with the perception sensor and can acquire perception information of a vehicle driving position.
In this embodiment, the sensing sensor may be a pressure sensor, and the sensing information is pressure information. The pressure sensor is arranged below the driving position of the vehicle, and when a driver sits at the driving position of the vehicle, the pressure sensor can acquire a pressure value; when the driver does not sit at the driving position of the vehicle, the pressure sensor can output a constant pressure value or output a pressure value of zero. The on-board device may acquire pressure information of the pressure sensor.
In this embodiment, the in-vehicle apparatus may determine whether the driver is in the vehicle driving place based on the pressure information.
In this embodiment, the sensing sensor may also be a seat belt sensor, and information output by the sensing sensor may be analyzed as a seat belt being clamped or a seat belt being not clamped. The installation position of the sensing sensor and the information output by the sensing sensor can follow the prior art, and are not described in detail herein. The vehicle-mounted equipment can acquire and analyze the information output by the safety belt sensor, and the analyzing mode can continue to use the prior art and is not described herein any more.
In the present embodiment, the in-vehicle apparatus may determine whether the driver is in the driving position of the vehicle based on the information output by the seatbelt sensor.
It can be seen that the sensory information of the vehicle driving seat can assist the in-vehicle device to determine whether the driver is in the driving seat.
In this embodiment, the sensing sensor may also be other devices capable of acquiring sensing information of the driving position of the vehicle, and is not described herein again.
In this embodiment, the vehicle-mounted device may also acquire state information during vehicle operation, such as state information of vehicle speed, steering angle, and the like. The status information may assist the in-vehicle device in determining whether the driver is in the driving position.
In this embodiment, the vehicle automatic driving information may be obtained based on a vehicle automatic driving policy, which belongs to a common technology in the field of automatic driving and is not described herein again. In this embodiment, the vehicle-mounted device may obtain the vehicle automatic driving information based on the vehicle automatic driving policy.
In this embodiment, the vehicle-mounted device may determine the driver state information based on the image information of the driving position of the vehicle and the perception information of the driving position of the vehicle, where the driver state information includes three types of state information: the driver is in a sleeping state, the driver is not in the driving position, and the driver is in the driving position.
In this embodiment, different types of takeover reminding are required for different state information of the driver, so that the driver in different states can take over the vehicle within a preset time after the vehicle prompts takeover.
In this embodiment, the vehicle-mounted device may determine the takeover reminding type based on the driver state information and the vehicle automatic driving information, and then take over reminding is performed based on the takeover reminding type.
The method for reminding taking over of the automatically driven vehicle disclosed by the embodiment can determine different states of the driver based on the image information and the perception information of the driving position of the vehicle, further determine different taking over reminding types based on different states of the driver, and for the automatically driven vehicle at the L3 level, different types of taking over reminding strategies can be adopted according to different states of the driver, so that the driver can be ensured to be capable of taking over the vehicle within a preset time length after the vehicle is reminded to take over, and the driver is not required to control the vehicle all the time.
In some embodiments, the sensory information of the vehicle driver's seat includes: the pressure value detected by a seat sensor of the vehicle driver's seat. The seat sensor may be an existing seat sensor, and will not be described herein.
In this embodiment, the determining the driver state information based on the image information of the driving position of the vehicle and the perception information of the driving position of the vehicle may specifically include the following (1) to (3):
(1) based on the image information, it is determined that the driver state information in the sleep state is the first state information. The first state information may be understood as the driver being in a sleep state.
(2) And determining the state information of the driver with the pressure value lower than the preset pressure value as second state information based on the perception information. The second status information may be understood as the driver not being in the driving position.
(3) And determining the driver state information with the pressure value not lower than the preset pressure value as third state information based on the perception information. The third status information may be understood as the driver being in the driving position.
In this embodiment, the preset pressure value is determined based on a pressure value detected by the seat sensor when the driver is at the driving position. The specific value of the preset pressure value is not limited in this embodiment.
In some embodiments, the taking over reminding type is determined based on the driver state information and the vehicle automatic driving information, and specifically includes the following (1) to (3):
(1) after the automatic vehicle driving information meeting the preset takeover condition is determined, the corresponding takeover reminding type of the first state information (namely the driver is in the sleep state) is determined to be the first type.
(2) And determining that the takeover reminding type corresponding to the second state information (namely the driver is not in the driving position) is the second type.
(3) And after the automatic driving information of the vehicle meeting the preset takeover condition is determined, determining that the takeover reminding type corresponding to the third state information (namely the driver is in the driving position) is the third type.
In this embodiment, the preset takeover condition is, for example, a failure of an automatic driving system of the vehicle. The manner in which the failure of the vehicle autopilot system is determined may be continued using the prior art and will not be described further herein.
In some embodiments, a fault signal may be output when the vehicle automatic driving system fails, and the vehicle-mounted device may determine that the vehicle automatic driving system fails after receiving the fault signal.
In some embodiments, the preset take-over condition is, for example, a scenario in which it is determined that the driver is required to take over the vehicle based on an automatic driving strategy of the vehicle, for example, a scenario in which the vehicle enters a low-speed highway from a highway.
In some embodiments, the take-over condition may be preset based on actual conditions, for example, the driver may input the take-over condition into the vehicle-mounted device through an input device such as a keyboard, a touch screen, and the like. The in-vehicle apparatus may determine whether the take-over condition is satisfied during automatic driving of the vehicle.
In some embodiments, the taking over reminding is performed based on the taking over reminding type, which specifically includes the following (1) to (3):
(1) and performing the takeover reminding with the first reminding intensity based on the takeover reminding type being the first type (corresponding to the driver being in the sleep state).
In this embodiment, the takeover reminder of the first reminder intensity may be understood as an enhanced reminder, including but not limited to: the method comprises the following steps of reminding one or more of driving position vibration reminding, inching reminding and the like in a language with preset first volume.
(2) And performing takeover reminding with second reminding intensity based on the fact that the takeover reminding type is a second type (corresponding to the fact that the driver is not in the driving position).
In this embodiment, the takeover reminding example of the second reminding intensity may be understood as a system alarm.
(3) And performing takeover reminding with third reminding intensity based on the fact that the takeover reminding type is a third type (corresponding to the situation that the driver is in the driving position).
In this embodiment, the takeover reminder with the third reminder intensity can be understood as a general reminder, including but not limited to: and reminding by using a language with preset second volume. The preset second volume is smaller than the preset first volume.
In some embodiments, the takeover reminder of the third reminder intensity (i.e., the normal-type reminder) may follow the existing driving fatigue reminding strategy, and will not be described herein.
In this embodiment, different takeover reminding types correspond different reminding strengths, and the takeover reminding mode of different reminding strengths can be set based on actual conditions, and only the conditions need to be satisfied: the first reminder intensity (i.e., enhanced reminder) is less than the second reminder intensity (i.e., system alarm) and the first reminder intensity is greater than the third reminder intensity (normal reminder).
The taking-over reminding method based on the automatic driving vehicle is mainly different from the monitoring mode of the traditional driver fatigue monitoring equipment in the following steps (1) to (3):
(1) the method has the advantages that the purpose is different, the main purpose in the embodiment of the invention is to judge whether a driver is in a driving position or not and whether the driver is in a sleeping state or not; while the conventional driver fatigue monitoring apparatus mainly aims to monitor whether the driver is fatigued, it is not suitable for an autonomous driving vehicle of the L3 class.
(2) The embodiment of the invention is not limited to the camera, and the sensing sensor capable of acquiring the sensing information of the driving position of the vehicle is also used, so that different states of the driver can be conveniently determined.
(3) According to the embodiment of the invention, different takeover reminding types are determined based on different states of a driver, and for an automatic driving vehicle at the L3 level, different takeover reminding strategies can be adopted according to different states of the driver, so that the driver can take over the vehicle within a preset time after the vehicle prompts takeover; and the reminding mode of the traditional driver fatigue monitoring equipment is fixed.
As shown in fig. 3, the present embodiment discloses a take-over reminding device for an autonomous vehicle, which may include the following units: an acquisition unit 31, a first determination unit 32, a second determination unit 33 and a reminder unit 34. The concrete description is as follows:
an acquisition unit 31 for acquiring image information of a vehicle driving seat, perception information of the vehicle driving seat, and vehicle automatic driving information;
a first determination unit 32 for determining driver state information based on the image information and the perception information;
a second determination unit 33, configured to determine a takeover reminding type based on the driver state information and the vehicle automatic driving information;
and the reminding unit 34 is used for carrying out takeover reminding based on the takeover reminding type.
In some embodiments, the perceptual information comprises: a pressure value detected by a seat sensor of the vehicle driving seat;
the first determining unit 32 is configured to:
determining the state information of the driver in the sleeping state as first state information based on the image information;
determining the state information of the driver with the pressure value lower than a preset pressure value as second state information based on the perception information;
and determining the driver state information of which the pressure value is not lower than a preset pressure value as third state information based on the perception information.
In some embodiments, the second determining unit 33 is configured to:
after determining the automatic driving information of the vehicle meeting the preset takeover condition, determining that the takeover reminding type corresponding to the first state information is a first type;
determining that the takeover reminding type corresponding to the second state information is a second type;
and after the automatic driving information of the vehicle meeting the preset takeover condition is determined, determining that the takeover reminding type corresponding to the third state information is a third type.
In some embodiments, the reminding unit 34 is configured to:
performing takeover reminding with first reminding intensity based on the first type;
performing takeover reminding with second reminding intensity based on the second type;
performing takeover reminding with third reminding intensity based on the third type;
the first reminding strength is smaller than the second reminding strength, and the first reminding strength is larger than the third reminding strength.
In some embodiments, the preset takeover condition comprises: vehicle autopilot system failure.
The takeover reminding device for the autonomous driving vehicle disclosed in the above embodiments can implement the flow of the takeover reminding method for the autonomous driving vehicle disclosed in the above method embodiments, and is not described herein again to avoid repetition.
An embodiment of the present invention further provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the steps provided in the embodiments of the takeover reminding method for an autonomous vehicle, where the steps include, for example, the following first to fourth steps:
acquiring image information of a vehicle driving position, perception information of the vehicle driving position and automatic driving information of a vehicle;
secondly, determining the state information of the driver based on the image information and the perception information;
determining a takeover reminding type based on the driver state information and the vehicle automatic driving information;
and fourthly, performing takeover reminding based on the takeover reminding type.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Those skilled in the art will appreciate that although some embodiments described herein include some features included in other embodiments instead of others, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments.
Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.
Claims (6)
1. A takeover reminding method for an autonomous vehicle, the method comprising:
acquiring image information of a vehicle driving position, perception information of the vehicle driving position and automatic vehicle driving information;
determining driver state information based on the image information and the perception information;
determining a takeover reminding type based on the driver state information and the vehicle automatic driving information;
performing takeover reminding based on the takeover reminding type;
wherein the perception information comprises: a pressure value detected by a seat sensor of the vehicle driving seat;
the determining driver state information based on the image information and the perception information includes:
determining the state information of the driver in the sleeping state as first state information based on the image information;
determining the state information of the driver with the pressure value lower than a preset pressure value as second state information based on the perception information;
determining that the driver state information of which the pressure value is not lower than a preset pressure value is third state information based on the perception information;
the first state information is that the driver is in a sleep state;
the second state information is the state that the driver is not in the driving position;
the third state is that the driver is in a driving position state;
determining a take-over reminder type based on the driver state information and the vehicle autopilot information, comprising:
after determining the automatic driving information of the vehicle meeting the preset takeover condition, determining that the takeover reminding type corresponding to the first state information is a first type;
determining that the takeover reminding type corresponding to the second state information is a second type;
after determining the automatic driving information of the vehicle meeting the preset takeover condition, determining that the takeover reminding type corresponding to the third state information is a third type;
the taking over reminding based on the taking over reminding type comprises the following steps:
performing takeover reminding with first reminding intensity based on the first type;
performing takeover reminding with second reminding intensity based on the second type;
performing takeover reminding with third reminding intensity based on the third type;
the first reminding intensity is smaller than the second reminding intensity, and the first reminding intensity is larger than the third reminding intensity;
the taking over reminding with the first reminding intensity comprises at least one of reminding in a language with a preset first volume, driving position vibration reminding and stopping reminding, the taking over reminding with the second reminding intensity comprises system alarming, and the taking over reminding with the third reminding intensity comprises reminding in a language with a preset second volume; and the preset second volume is smaller than the preset first volume.
2. The method of claim 1, wherein the preset takeover condition comprises: vehicle autopilot system failure.
3. A takeover reminder for an autonomous vehicle, the device comprising:
the system comprises an acquisition unit, a display unit and a control unit, wherein the acquisition unit is used for acquiring image information of a vehicle driving position, perception information of the vehicle driving position and automatic vehicle driving information;
a first determination unit configured to determine driver state information based on the image information and the perception information;
the second determination unit is used for determining the type of taking over reminding based on the driver state information and the automatic vehicle driving information;
the reminding unit is used for carrying out takeover reminding based on the takeover reminding type;
wherein the perception information comprises: a pressure value detected by a seat sensor of the vehicle driving seat;
the first determining unit is configured to:
determining the state information of the driver in the sleeping state as first state information based on the image information;
determining the state information of the driver with the pressure value lower than a preset pressure value as second state information based on the perception information;
determining that the driver state information of which the pressure value is not lower than a preset pressure value is third state information based on the perception information;
the first state information is that the driver is in a sleep state;
the second state information is the state that the driver is not in the driving position;
the third state is that the driver is in a driving position state;
the second determination unit is configured to:
after determining the automatic driving information of the vehicle meeting the preset takeover condition, determining that the takeover reminding type corresponding to the first state information is a first type;
determining that the takeover reminding type corresponding to the second state information is a second type;
after determining the automatic driving information of the vehicle meeting the preset takeover condition, determining that the takeover reminding type corresponding to the third state information is a third type;
the reminding unit is used for:
performing takeover reminding with first reminding intensity based on the first type;
performing takeover reminding with second reminding intensity based on the second type;
performing takeover reminding with third reminding intensity based on the third type;
the first reminding intensity is smaller than the second reminding intensity, and the first reminding intensity is larger than the third reminding intensity;
the taking over reminding with the first reminding intensity comprises at least one of reminding in a language with a preset first volume, driving position vibration reminding and stopping reminding, the taking over reminding with the second reminding intensity comprises system alarming, and the taking over reminding with the third reminding intensity comprises reminding in a language with a preset second volume; and the preset second volume is smaller than the preset first volume.
4. The apparatus of claim 3, wherein the preset take-over condition comprises: vehicle autopilot system failure.
5. An in-vehicle apparatus, characterized by comprising:
a processor, memory, a network interface, and a user interface;
the processor, memory, network interface and user interface are coupled together by a bus system;
the processor is adapted to perform the steps of the method of any one of claims 1 to 2 by calling a program or instructions stored in the memory.
6. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the steps of the method of any one of claims 1 to 2.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811486409.3A CN111361566B (en) | 2018-12-06 | 2018-12-06 | Takeover reminding method for automatic driving vehicle, vehicle-mounted equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811486409.3A CN111361566B (en) | 2018-12-06 | 2018-12-06 | Takeover reminding method for automatic driving vehicle, vehicle-mounted equipment and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111361566A CN111361566A (en) | 2020-07-03 |
CN111361566B true CN111361566B (en) | 2022-04-29 |
Family
ID=71202028
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811486409.3A Active CN111361566B (en) | 2018-12-06 | 2018-12-06 | Takeover reminding method for automatic driving vehicle, vehicle-mounted equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111361566B (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112498364B (en) * | 2020-11-30 | 2022-04-08 | 奇瑞汽车股份有限公司 | Vehicle early warning method and device |
CN112455466B (en) * | 2020-12-07 | 2022-08-19 | 安徽江淮汽车集团股份有限公司 | Automatic driving control method, automatic driving control equipment, storage medium and device |
CN113968236A (en) * | 2021-10-29 | 2022-01-25 | 际络科技(上海)有限公司 | Vehicle fault processing method and device and vehicle |
CN114312839A (en) * | 2021-12-29 | 2022-04-12 | 阿波罗智联(北京)科技有限公司 | Information processing method, information processing apparatus, electronic device, and storage medium |
CN114771559A (en) * | 2022-06-16 | 2022-07-22 | 苏州魔视智能科技有限公司 | Vehicle human-computer interaction method, device and system |
CN115489536B (en) * | 2022-11-18 | 2023-01-20 | 中国科学院心理研究所 | Driving assistance method, system, equipment and readable storage medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN204196777U (en) * | 2014-10-30 | 2015-03-11 | 长城汽车股份有限公司 | Drive road conditions control system for identifying and vehicle |
CN106394319A (en) * | 2015-07-27 | 2017-02-15 | 丰田自动车株式会社 | Vehicle occupant information acquisition device and vehicle control system |
CN106976457A (en) * | 2016-01-18 | 2017-07-25 | 福特全球技术公司 | Method and apparatus for operating motor vehicles |
CN107463122A (en) * | 2016-05-02 | 2017-12-12 | 本田技研工业株式会社 | Vehicle control system, control method for vehicle and wagon control program product |
CN108609019A (en) * | 2018-05-09 | 2018-10-02 | 浙江金之路信息科技有限公司 | A kind of electric vehicle automatic Pilot method based on artificial intelligence platform |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102012102459A1 (en) * | 2012-03-22 | 2013-09-26 | Continental Automotive Gmbh | Method for adjusting control parameter of e.g. safety-relevant systems of passenger car, involves determining driving state based on information whether driver hand is present on steering wheel and/or whether vehicle exits in current lane |
CN104477167B (en) * | 2014-11-26 | 2018-04-10 | 浙江大学 | A kind of intelligent driving system and its control method |
JP6358212B2 (en) * | 2015-09-17 | 2018-07-18 | トヨタ自動車株式会社 | Awakening control system for vehicles |
CN106548596A (en) * | 2015-09-21 | 2017-03-29 | 天津思博科科技发展有限公司 | A kind of device for preventing driver's burst disease from causing traffic accident |
US10471963B2 (en) * | 2017-04-07 | 2019-11-12 | TuSimple | System and method for transitioning between an autonomous and manual driving mode based on detection of a drivers capacity to control a vehicle |
-
2018
- 2018-12-06 CN CN201811486409.3A patent/CN111361566B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN204196777U (en) * | 2014-10-30 | 2015-03-11 | 长城汽车股份有限公司 | Drive road conditions control system for identifying and vehicle |
CN106394319A (en) * | 2015-07-27 | 2017-02-15 | 丰田自动车株式会社 | Vehicle occupant information acquisition device and vehicle control system |
CN106976457A (en) * | 2016-01-18 | 2017-07-25 | 福特全球技术公司 | Method and apparatus for operating motor vehicles |
CN107463122A (en) * | 2016-05-02 | 2017-12-12 | 本田技研工业株式会社 | Vehicle control system, control method for vehicle and wagon control program product |
CN108609019A (en) * | 2018-05-09 | 2018-10-02 | 浙江金之路信息科技有限公司 | A kind of electric vehicle automatic Pilot method based on artificial intelligence platform |
Also Published As
Publication number | Publication date |
---|---|
CN111361566A (en) | 2020-07-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111361566B (en) | Takeover reminding method for automatic driving vehicle, vehicle-mounted equipment and storage medium | |
KR102459737B1 (en) | How to upgrade autonomous driving system, autonomous driving system and on-vehicle device | |
US9156352B2 (en) | Driving assistant system and method having warning function for risk level | |
US10198952B2 (en) | Operator alertness monitor | |
US9296395B2 (en) | Cognitive state supported automotive travel | |
CN109255005B (en) | Vehicle repositioning method and device, vehicle-mounted equipment, server and storage medium | |
US20170341512A1 (en) | Method for the automatic execution of at least one driving function of a motor vehicle | |
CN108549911B (en) | Driver steering intervention identification method based on neural network | |
US20130246866A1 (en) | System and method for verifying the integrity of a safety-critical vehicle control system | |
JP2018149941A5 (en) | ||
DE102015211126A1 (en) | Driver attention monitoring for automatic driving functions | |
CN110979180A (en) | Control method and device for preventing dangerous driving and vehicle | |
JP2018151907A5 (en) | ||
CN113911872A (en) | Fault processing method and system and early warning and rescue system | |
CN113129475A (en) | Monitoring method and device and vehicle | |
KR102213676B1 (en) | Terminal apparatus for autosar system with arithmetic operation supervision function and arithmetic operation supervision method of autosar system | |
Kinoshita et al. | Analysis of a driver and automated driving system interaction using a communicating sequential process | |
JP2018045450A (en) | Vehicle control apparatus | |
DE112017007230B4 (en) | FAULT DETERMINATOR, METHOD AND PROGRAM | |
CA2607435A1 (en) | Software process monitor | |
CN114760147A (en) | Security event processing method, security event processing device, equipment and medium | |
GB2558671A (en) | Communication control apparatus and method | |
US20210259604A1 (en) | Method, Device, Computer Program and Computer Program Product for Detecting the Attentiveness of the Driver of a Vehicle | |
CN114148337A (en) | Driver state information prompting method and device and computer readable storage medium | |
CN112549870A (en) | Monitoring method and device and vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |