CN113335300A - Man-vehicle takeover interaction method, device, equipment and storage medium - Google Patents
Man-vehicle takeover interaction method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113335300A CN113335300A CN202110814977.7A CN202110814977A CN113335300A CN 113335300 A CN113335300 A CN 113335300A CN 202110814977 A CN202110814977 A CN 202110814977A CN 113335300 A CN113335300 A CN 113335300A
- Authority
- CN
- China
- Prior art keywords
- driver
- takeover
- vehicle
- result
- interaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 76
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000000007 visual effect Effects 0.000 claims description 36
- 230000006870 function Effects 0.000 claims description 19
- 238000012790 confirmation Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000001960 triggered effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/082—Selecting or switching between different modes of propelling
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Traffic Control Systems (AREA)
- Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
Abstract
The embodiment of the invention discloses a method, a device, equipment and a storage medium for man-vehicle takeover interaction. The method comprises the following steps: acquiring a man-vehicle takeover interaction request; identifying whether the sight line of the driver is within a main observation range of the driving direction of the vehicle; determining a driver takeover result according to the sight line identification result; and sending out a takeover prompt based on an acousto-optic means according to the takeover result of the driver. By the technical scheme of the embodiment of the invention, the automatic driver and vehicle take-over task can be safely and reliably completed.
Description
Technical Field
The embodiment of the invention relates to a man-vehicle takeover technology, in particular to a man-vehicle takeover interaction method, a man-vehicle takeover interaction device, man-vehicle takeover interaction equipment and a storage medium.
Background
With the development of vehicle technology, the automatic driving of the L3/L4 gradually becomes a mainstream trend. In an automatic driving state, if a driving task is too light, a driver is easy to get into passive fatigue due to low cognitive burden, and the boring driving process also leads the driver to participate in non-driving main tasks such as social entertainment and the like more, so that driving distraction is brought; secondly, as the main attention resources are not concentrated in the vehicle state and the road situation in the automatic driving process, the situation awareness is reduced, and thus, when the automatic system exits and sends out a switching control request, a driver may be scared due to unexpected state change or prompt receipt; meanwhile, the lack of situational awareness level may cause confusion of driving modes; in addition, there is a possibility that an interactive misunderstanding may occur between the driver who thinks that the driving right has been delivered to the vehicle, and the vehicle judges the right to be in the driver's hand.
At present, a man-machine interaction system can be designed to comprise a master man-machine interaction module and a slave man-machine interaction module, alarm prompt is carried out through the master man-machine interaction module under a normal condition, and when the master man-machine interaction module fails, the slave man-machine interaction module takes over the system and carries out alarm prompt.
The human-computer interaction scheme between the driver and the automatic driving system is very important, and not only the usability of the automatic driving system is influenced, but also the safety of the system is influenced. Therefore, how to safely and reliably complete the vehicle taking over task of the automatic driver is a technical problem to be solved urgently by the technical personnel in the field.
Disclosure of Invention
The embodiment of the invention provides a method, a device, equipment and a storage medium for man-vehicle takeover interaction, so as to safely and reliably complete an automatic driver-vehicle takeover task.
In a first aspect, an embodiment of the present invention provides a method for human-vehicle takeover interaction, where the method includes:
acquiring a man-vehicle takeover interaction request;
identifying whether the sight of the driver is within a set visual field range;
wherein the set visual field range is a main observation range of the vehicle running direction;
determining a driver takeover result according to the sight line identification result;
and sending out a takeover prompt based on an acousto-optic means according to the takeover result of the driver.
In a second aspect, an embodiment of the present invention further provides a device for human-vehicle takeover interaction, including:
the request acquisition module is used for acquiring a man-vehicle takeover interaction request;
the visual field identification module is used for identifying whether the sight of the driver is in a set visual field range;
wherein the set visual field range is a main observation range of the vehicle running direction;
the takeover result confirmation module is used for determining a takeover result of the driver according to the sight line recognition result;
and the take-over prompting module is used for sending out take-over prompts based on an acousto-optic means according to the take-over results of the driver.
In a third aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the method for human-vehicle takeover interaction of any embodiment of the invention.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for human-vehicle takeover interaction according to any embodiment of the present invention.
The embodiment of the invention provides a method and a device for man-vehicle takeover interaction, electronic equipment and a storage medium, which can avoid misoperation or distracted operation to the greatest extent and safely and reliably complete an automatic driver-vehicle takeover task through the requirements of double-hand operation and sight range of a driver.
Drawings
Fig. 1 is a flowchart of a human-vehicle takeover interaction method according to an embodiment of the present invention;
fig. 2A is a flowchart of a human-vehicle takeover interaction method according to a second embodiment of the present invention;
fig. 2B is a logic diagram of a human-vehicle takeover interaction method according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a human-vehicle connection interaction device according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a human-vehicle takeover interaction method according to an embodiment of the present invention, which is applicable to a process of switching from an automatic driving control state to a state manually controlled by a human being when human-vehicle takeover is required in an automatic driving vehicle. The method of the embodiment may be performed by a human-vehicle takeover interaction device, which may be implemented in hardware and/or software. The device can be configured in a control device capable of taking over interaction for people and vehicles, such as a vehicle-mounted control device. The method specifically comprises the following steps:
and S110, acquiring a man-car takeover interaction request.
The man-vehicle taking over refers to a process of mutual acceptance and management between a driver and a computer. For example, mutual acceptance management between a driver and an autonomous vehicle, and when a danger is met, the autonomous driving is changed into the driver driving; when the road is safe, the driving of the driver is changed into automatic driving.
Human-vehicle interaction means that a conversation between a human and a computer is realized in an effective mode through computer input and output devices. For example, a driver requests the vehicle to achieve autonomous driving through an input device; the vehicle prompts the driver to take over the vehicle through an output or display device.
The human-vehicle takeover interaction request can be actively triggered by a driver, and can also be automatically triggered when a set road or driving condition is met according to a preset strategy to generate a human-vehicle takeover interaction request. For the case of active triggering by the driver, for example, in a journey with a flat road and a long way, the driver presses the multifunctional key with both hands simultaneously to trigger the man-car takeover interaction request.
S120, identifying whether the sight of the driver is in a set visual field range;
when a human-vehicle takeover interaction request is acquired, the vehicle-mounted control equipment does not immediately switch to a manual driving mode, but firstly identifies the sight line condition of a driver, specifically identifies whether the sight line of the driver is in a set visual field range. Wherein the set visual field range is a main observation range of the vehicle running direction. The driver's main observation range is a range that the driver can visually observe ahead.
The sight of the driver can be recognized by installing a front camera in a cab, acquiring a face image of the driver, and recognizing a sight focus of the driver in the face image so as to judge whether the sight of the driver is in a visual field range; the sight line of the driver can be tracked in a focus tracking mode, and whether the sight line of the driver is in the visual field range or not can be judged. The camera is arranged in the cab, so that the sight focus of the driver is tracked or recognized, the existing equipment in the vehicle is utilized, the cost is reduced, and the sight of the driver can be well judged.
S130, determining a driver taking over result according to the sight line identification result;
the sight line recognition result is that whether the sight line of the driver is in a preset sight range is judged by recognizing the focus of the sight line of the driver in the face image. For example, when the driver looks ahead, it is recognized that the driver's sight line focus is within a predetermined visual field range; when the driver does not visually recognize the front, it is recognized that the sight line focus of the driver is not within the predetermined visual field range.
Determining a driver takeover result according to the sight line recognition result, for example, if the sight line of the driver does not fall into a set visual field range, the driver is not allowed to take over the vehicle control; if the sight line of the driver falls into the set sight range, the sight line of the driver can be in an angle of 60 degrees right ahead, the driver is allowed to take over the vehicle directly; the driver may be allowed to take over the vehicle control after a reminder is given to the driver when the driver's line of sight is within a predetermined range of view from an angle of 60 ° directly in front to a maximum.
And S140, sending a takeover prompt based on an acousto-optic means according to the takeover result of the driver.
The sound and light means is to give a prompt to the vehicle taking-over result by means of sound and/or light, for example, the vehicle prompts a driver to take over the vehicle or the vehicle finishes taking over, the prompt can be given by flashing an indicator light, the prompt can be given by a caption image, and the prompt can be given by a dripping sound or a buzzing sound. The prompt of taking over is sent out by an acousto-optic means, so that the attention of a driver to the prompt of taking over can be improved, and the vehicle can be taken over in time.
The embodiment of the invention provides a man-vehicle takeover interaction method, which comprises the steps of identifying whether the sight of a driver is in the main observation range of the vehicle running direction or not by obtaining a man-vehicle takeover interaction request, determining a driver takeover result according to the sight identification result, and sending a takeover prompt based on an acousto-optic means according to the driver takeover result, so that man-vehicle takeover interaction is realized, and an automatic driver-vehicle takeover task is safely and reliably completed.
Example two
Fig. 2A is a flowchart of a method for human-vehicle takeover interaction according to a second embodiment of the present invention. In this embodiment, refinement is performed on the basis of the foregoing embodiment, and optionally, the method further includes: transmitting the determined driver takeover result to an infotainment system configured in the vehicle through an automatic driving controller; and sending a take-over prompt in a voice mode and/or a subtitle image mode according to the take-over result of the driver through the infotainment system.
Correspondingly, as shown in fig. 2A, the technical solution of the embodiment of the present invention specifically includes the following steps:
and S210, acquiring a man-vehicle takeover interaction request.
Optionally, the obtaining of the human-vehicle takeover interaction request includes: receiving a man-vehicle takeover interaction request input by a driver; or
And automatically determining to generate a human-vehicle takeover interaction request according to the current vehicle information and/or the current driver information.
Optionally, the receiving of the human-vehicle takeover interaction request input by the driver includes: and if the driver operates the set one or more function keys, determining that a human-vehicle takeover interaction request is received.
Optionally, the set multiple function keys include: and the function keys are symmetrically arranged on the steering wheel.
The current vehicle information includes, but is not limited to, the model and the running speed of the current vehicle; driver information includes, but is not limited to, driver status, driver mood, and driver gaze range.
The operation of the driver on one or more set function keys is received, wherein the operation of one function key can be received, and the operation of two or more function keys can also be received; when two or more than two function keys are received, the function keys can be the same function keys or different function keys; preferably, the driver triggers the central key of the left multifunctional key and the right multifunctional key simultaneously by both hands to send out a request for actively triggering the human-vehicle takeover interaction. Most vehicles are provided with multifunctional keys on a steering wheel, and are arranged in a left-right substantially symmetrical manner, and a center key among the arranged keys can be an operation object.
The driver needs to pay more attention to trigger the same task by both hands, misoperation can be avoided, and the driving behavior requires that both hands of the driver hold the steering wheel, which belongs to natural interaction.
And S220, identifying whether the sight of the driver is in the set visual field range.
And S230, determining a driver taking over result according to the sight line recognition result.
Optionally, the driver taking over result is determined according to the sight line identification result, the current vehicle information and the current road condition information.
Optionally, determining the driver takeover result according to the sight line recognition result includes:
if the sight of the driver falls into the set visual field range and the human-vehicle interaction takeover request is initiated manually by the driver, determining that the driver is allowed to take over the vehicle control;
if the driver's sight line does not fall within the set sight range, it is determined that the driver is not allowed to take over the vehicle control.
Wherein the current vehicle information includes, but is not limited to, the model of the driving vehicle. The current road condition information includes, but is not limited to, the speed limit of the road, whether the road is flat, and whether there are pedestrians and vehicles in front of the driving.
And S240, transmitting the determined driver taking over result to an infotainment system configured in the vehicle through an automatic driving controller.
The automatic driving controller transmits the taking-over result to the information entertainment system according to the requirement of the driver for triggering the taking-over request, the requirement of the sight line of the driver and the requirement of the road condition information.
And S250, sending a take-over prompt in a voice mode and/or a subtitle image mode through the infotainment system according to the take-over result of the driver.
The information entertainment system is a vehicle-mounted comprehensive information processing system formed by adopting a vehicle-mounted special central processing unit and based on a vehicle body bus system and internet services. For example, a take-over request sent by the autonomous driving controller is processed to prompt the driver of the take-over result.
Wherein, the taking over prompt is sent out in a voice and/or caption image mode, for example, a dripping sound can be sent out to prompt a driver to take over the vehicle; and the taking-over prompt can be sent out in modes of indicator light flashing, subtitle display, image and the like.
According to the technical scheme of the embodiment of the invention, whether the sight of the driver is in the set visual field range is identified through the acquired man-vehicle takeover interaction request, the determined driver takeover result is transmitted to the infotainment system configured in the vehicle through the automatic driving controller, and the infotainment system sends out a takeover prompt in a voice mode and/or a caption image mode, so that the misunderstanding of both man-machine parties during man-vehicle takeover is avoided, and the safety and reliability of the automatic driver-vehicle takeover task are ensured.
The above-mentioned operation process is described below by way of example, and as shown in fig. 2B, the operation process includes:
s201, a driver triggers a center key of a left multifunctional key and/or a center key of a right multifunctional key of the steering wheel.
S202, judging whether a driver simultaneously triggers center keys of the left and right multifunctional keys or not, and sending a take-over request if the center keys of the left and right multifunctional keys are simultaneously triggered; and if the central keys of the left multifunctional key and the right multifunctional key are not triggered at the same time, waiting for the next takeover request.
S203, the front camera identifies whether the sight line of the driver is in a specified visual field range.
S204, judging driver information, and if the driver sends a take-over request and the sight of the driver is in a specified visual field range, sending human take-over information to an automatic driving controller; if the driver sends the take-over request but the sight line of the driver is not in the specified visual field range, the next take-over request is waited.
And S205, identifying the road condition information.
S206, the comprehensive judgment strategy of the automatic driving controller is to comprehensively judge the road condition information and the human takeover information, and if the received human takeover information and the road condition information meet the requirements, the takeover result is sent to the information entertainment system; and if the received person takeover information does not meet the requirement of the road condition information, waiting for the next takeover request.
And S207, the information entertainment system receives the takeover result sent by the automatic driving controller and sends the takeover result prompt to the instrument.
S208, the instrument receives the takeover result prompting information sent by the infotainment system, and prompts the takeover result to the driver in a visual/sound mode.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a human-vehicle connection interaction device provided by a third embodiment of the present invention, where the device includes: a request acquisition module 310, a view identification module 320, a takeover result confirmation module 330, and a takeover prompt module 340. Wherein:
the request acquisition module is used for acquiring a man-vehicle takeover interaction request;
the visual field identification module is used for identifying whether the sight of the driver is in a set visual field range;
optionally, the set visual field range is a main observation range of the vehicle driving direction;
the takeover result confirmation module is used for determining a takeover result of the driver according to the sight line recognition result;
and the take-over prompting module is used for sending out take-over prompts based on an acousto-optic means according to the take-over results of the driver.
Optionally, the request obtaining module includes:
receiving a man-vehicle takeover interaction request input by a driver; or
And automatically determining to generate a human-vehicle takeover interaction request according to the current vehicle information and/or the current driver information.
Optionally, the request obtaining module further includes:
and if the driver operates the set one or more function keys, determining that a human-vehicle takeover interaction request is received.
Wherein, a plurality of function keys of said settlement include: and the function keys are symmetrically arranged on the steering wheel.
Optionally, the visual field recognition module includes:
acquiring a face image of a driver shot by a front camera in a vehicle;
and identifying the sight focus in the face image to determine whether the sight focus is in a set visual field range.
Optionally, the takeover prompting module includes:
transmitting the determined driver takeover result to an infotainment system configured in the vehicle through an automatic driving controller;
and sending a take-over prompt in a voice mode and/or a subtitle image mode according to the take-over result of the driver through the infotainment system.
Optionally, the takeover result confirmation module includes:
and determining a driver take-over result according to the sight line identification result, the current vehicle information and the current road condition information.
Optionally, the takeover result confirmation module further includes:
if the sight of the driver falls into the set visual field range and the human-vehicle interaction takeover request is initiated manually by the driver, determining that the driver is allowed to take over the vehicle control;
if the driver's sight line does not fall within the set sight range, it is determined that the driver is not allowed to take over the vehicle control.
The device can execute the man-vehicle takeover interaction method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the man-vehicle takeover interaction method.
Example four
Fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention, as shown in fig. 4, the electronic device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of the processors 410 in the device may be one or more, and one processor 410 is taken as an example in fig. 4; the processor 410, the memory 420, the input device 430 and the output device 440 in the apparatus may be connected by a bus or other means, for example, in fig. 4.
The memory 420 serves as a computer-readable storage medium, and may be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the method for human-vehicle takeover interaction in the embodiment of the present invention (for example, the request acquisition module 310, the visual field recognition module 320, the takeover result confirmation module 330, and the takeover prompt module 340 in the device for human-vehicle takeover interaction). The processor 410 executes various functional applications and data processing of the electronic device by executing software programs, instructions and modules stored in the memory 420, namely, the method for human-vehicle takeover interaction described above is realized.
The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 420 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 420 may further include memory located remotely from processor 410, which may be connected to an electronic device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input means 430 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the apparatus. The output device 440 may include a display device such as a display screen.
EXAMPLE five
An embodiment of the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method for human-vehicle takeover interaction, the method including:
acquiring a man-vehicle takeover interaction request;
identifying whether the sight of the driver is within a set visual field range;
wherein the set visual field range is a main observation range of the vehicle running direction;
determining a driver takeover result according to the sight line identification result;
and sending out a takeover prompt based on an acousto-optic means according to the takeover result of the driver.
Of course, the storage medium provided by the embodiments of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the operations of the method described above, and may also perform related operations in the method for human-vehicle takeover interaction provided by any embodiments of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the human-vehicle takeover interaction device, each included unit and module are only divided according to functional logic, but are not limited to the above division, as long as corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (11)
1. A method for human-vehicle takeover interaction is characterized by comprising the following steps:
acquiring a man-vehicle takeover interaction request;
identifying whether the sight of the driver is within a set visual field range;
wherein the set visual field range is a main observation range of the vehicle running direction;
determining a driver takeover result according to the sight line identification result;
and sending out a takeover prompt based on an acousto-optic means according to the takeover result of the driver.
2. The method of claim 1, wherein obtaining a human-vehicle takeover interaction request comprises:
receiving a man-vehicle takeover interaction request input by a driver; or
And automatically determining to generate a human-vehicle takeover interaction request according to the current vehicle information and/or the current driver information.
3. The method of claim 2, wherein receiving a driver-entered human-vehicle takeover interaction request comprises:
and if the driver operates the set one or more function keys, determining that a human-vehicle takeover interaction request is received.
4. The method of claim 3, wherein setting the plurality of function keys comprises: and the function keys are symmetrically arranged on the steering wheel.
5. The method of claim 1, wherein identifying whether the driver's gaze is within a set field of view comprises:
acquiring a face image of a driver shot by a front camera in a vehicle;
and identifying the sight focus in the face image to determine whether the sight focus is in a set visual field range.
6. The method of claim 1, wherein issuing a take-over prompt based on acousto-optic means in accordance with the driver take-over result comprises:
transmitting the determined driver takeover result to an infotainment system configured in the vehicle through an automatic driving controller;
and sending a take-over prompt in a voice mode and/or a subtitle image mode according to the take-over result of the driver through the infotainment system.
7. The method of claim 1, wherein determining a driver takeover result from the gaze recognition result comprises:
and determining a driver take-over result according to the sight line identification result, the current vehicle information and the current road condition information.
8. The method of claim 1, wherein determining a driver takeover result from the gaze recognition result comprises:
if the sight of the driver falls into the set visual field range and the human-vehicle interaction takeover request is initiated manually by the driver, determining that the driver is allowed to take over the vehicle control;
if the driver's sight line does not fall within the set sight range, it is determined that the driver is not allowed to take over the vehicle control.
9. A man-car takeover interaction device, comprising:
the request acquisition module is used for acquiring a man-vehicle takeover interaction request;
the visual field identification module is used for identifying whether the sight of the driver is in a set visual field range;
wherein the set visual field range is a main observation range of the vehicle running direction;
the takeover result confirmation module is used for determining a takeover result of the driver according to the sight line recognition result;
and the take-over prompting module is used for sending out take-over prompts based on an acousto-optic means according to the take-over results of the driver.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the human-vehicle takeover interaction method of any one of claims 1-8.
11. A storage medium containing computer executable instructions for performing the human-vehicle takeover interaction method of any one of claims 1-8 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110814977.7A CN113335300A (en) | 2021-07-19 | 2021-07-19 | Man-vehicle takeover interaction method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110814977.7A CN113335300A (en) | 2021-07-19 | 2021-07-19 | Man-vehicle takeover interaction method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113335300A true CN113335300A (en) | 2021-09-03 |
Family
ID=77479996
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110814977.7A Pending CN113335300A (en) | 2021-07-19 | 2021-07-19 | Man-vehicle takeover interaction method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113335300A (en) |
Citations (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2314489A1 (en) * | 2009-10-22 | 2011-04-27 | Audi AG | Method for controlling the operation of a fully automatic driver assistance system of a motor vehicle for independent vehicle guidance and motor vehicle |
| US20110148613A1 (en) * | 2008-07-03 | 2011-06-23 | Markus Koehler | Device and method for approving automatic guidance of a vehicle |
| US20140303827A1 (en) * | 2013-04-05 | 2014-10-09 | Google Inc. | Systems and Methods for Transitioning Control of an Autonomous Vehicle to a Driver |
| WO2015000621A1 (en) * | 2013-07-05 | 2015-01-08 | Robert Bosch Gmbh | Method for ascertaining a degree of awareness of a vehicle operator |
| EP3192695A1 (en) * | 2016-01-18 | 2017-07-19 | Ford Global Technologies, LLC | Method and device for operating a motor vehicle |
| DE102018002962A1 (en) * | 2017-05-09 | 2018-11-15 | Omron Corporation | DRIVER MONITORING APPARATUS AND DRIVER MONITORING PROCEDURE |
| US20180348758A1 (en) * | 2017-06-02 | 2018-12-06 | Honda Motor Co., Ltd. | Vehicle control system, vehicle control method, and storage medium |
| CN108944950A (en) * | 2017-05-19 | 2018-12-07 | 上海汽车集团股份有限公司 | A kind of method for handover control and device of vehicle drive mode |
| JP2018203180A (en) * | 2017-06-08 | 2018-12-27 | 株式会社デンソー | Transfer control device and control program |
| WO2019074288A1 (en) * | 2017-10-12 | 2019-04-18 | 르노삼성자동차 주식회사 | Conditions for switching to manual driving mode in autonomous vehicle |
| CN109664891A (en) * | 2018-12-27 | 2019-04-23 | 北京七鑫易维信息技术有限公司 | Auxiliary driving method, device, equipment and storage medium |
| GB201903421D0 (en) * | 2019-03-13 | 2019-04-24 | Jaguar Land Rover Ltd | Vehicle control system |
| WO2019087975A1 (en) * | 2017-11-01 | 2019-05-09 | 株式会社デンソー | Vehicle control device |
| WO2019097944A1 (en) * | 2017-11-17 | 2019-05-23 | ソニーセミコンダクタソリューションズ株式会社 | Information processing device and information processing method |
| CN110104051A (en) * | 2019-05-30 | 2019-08-09 | 爱驰汽车有限公司 | Automatic Pilot switching method, system, equipment and storage medium based on steering wheel |
| CN110178141A (en) * | 2017-01-12 | 2019-08-27 | 法雷奥开关和传感器有限责任公司 | Method for manipulating autonomous motor vehicles |
| EP3552909A1 (en) * | 2018-04-11 | 2019-10-16 | Hyundai Motor Company | Apparatus and method for managing control authority transition in vehicle |
| WO2019230020A1 (en) * | 2018-06-01 | 2019-12-05 | クラリオン株式会社 | State determination device, driving support device, state determination method, and driving support method |
| CN110660258A (en) * | 2019-08-23 | 2020-01-07 | 福瑞泰克智能系统有限公司 | Reminding method and device for automatically driving automobile |
| CN111516698A (en) * | 2020-04-08 | 2020-08-11 | 中国第一汽车股份有限公司 | Driving assistance system control method, driving assistance system, vehicle, and storage medium |
| US20200262441A1 (en) * | 2019-02-15 | 2020-08-20 | Subaru Corporation | Vehicle driving control system |
| EP3718848A1 (en) * | 2019-04-04 | 2020-10-07 | Visteon Global Technologies, Inc. | System and method for executing an automated driving maneuver in a takeover process of a vehicle from automated driving to manually driving |
| CN112114671A (en) * | 2020-09-22 | 2020-12-22 | 上海汽车集团股份有限公司 | Human-vehicle interaction method and device based on human eye sight and storage medium |
| CN112550310A (en) * | 2019-09-25 | 2021-03-26 | 株式会社斯巴鲁 | Vehicle control device |
| CN112758098A (en) * | 2019-11-01 | 2021-05-07 | 广州汽车集团股份有限公司 | Vehicle driving authority take-over control method and device based on driver state grade |
| CN112960001A (en) * | 2021-04-19 | 2021-06-15 | 北京七鑫易维信息技术有限公司 | Driving mode switching method and device, vehicle and storage medium |
| CN113120068A (en) * | 2021-05-17 | 2021-07-16 | 上海联适导航技术股份有限公司 | Automatic drive agricultural machinery and have steering wheel of button control function |
-
2021
- 2021-07-19 CN CN202110814977.7A patent/CN113335300A/en active Pending
Patent Citations (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110148613A1 (en) * | 2008-07-03 | 2011-06-23 | Markus Koehler | Device and method for approving automatic guidance of a vehicle |
| EP2314489A1 (en) * | 2009-10-22 | 2011-04-27 | Audi AG | Method for controlling the operation of a fully automatic driver assistance system of a motor vehicle for independent vehicle guidance and motor vehicle |
| US20140303827A1 (en) * | 2013-04-05 | 2014-10-09 | Google Inc. | Systems and Methods for Transitioning Control of an Autonomous Vehicle to a Driver |
| WO2015000621A1 (en) * | 2013-07-05 | 2015-01-08 | Robert Bosch Gmbh | Method for ascertaining a degree of awareness of a vehicle operator |
| EP3192695A1 (en) * | 2016-01-18 | 2017-07-19 | Ford Global Technologies, LLC | Method and device for operating a motor vehicle |
| CN110178141A (en) * | 2017-01-12 | 2019-08-27 | 法雷奥开关和传感器有限责任公司 | Method for manipulating autonomous motor vehicles |
| DE102018002962A1 (en) * | 2017-05-09 | 2018-11-15 | Omron Corporation | DRIVER MONITORING APPARATUS AND DRIVER MONITORING PROCEDURE |
| CN108944950A (en) * | 2017-05-19 | 2018-12-07 | 上海汽车集团股份有限公司 | A kind of method for handover control and device of vehicle drive mode |
| US20180348758A1 (en) * | 2017-06-02 | 2018-12-06 | Honda Motor Co., Ltd. | Vehicle control system, vehicle control method, and storage medium |
| JP2018203180A (en) * | 2017-06-08 | 2018-12-27 | 株式会社デンソー | Transfer control device and control program |
| WO2019074288A1 (en) * | 2017-10-12 | 2019-04-18 | 르노삼성자동차 주식회사 | Conditions for switching to manual driving mode in autonomous vehicle |
| WO2019087975A1 (en) * | 2017-11-01 | 2019-05-09 | 株式会社デンソー | Vehicle control device |
| WO2019097944A1 (en) * | 2017-11-17 | 2019-05-23 | ソニーセミコンダクタソリューションズ株式会社 | Information processing device and information processing method |
| EP3552909A1 (en) * | 2018-04-11 | 2019-10-16 | Hyundai Motor Company | Apparatus and method for managing control authority transition in vehicle |
| WO2019230020A1 (en) * | 2018-06-01 | 2019-12-05 | クラリオン株式会社 | State determination device, driving support device, state determination method, and driving support method |
| CN109664891A (en) * | 2018-12-27 | 2019-04-23 | 北京七鑫易维信息技术有限公司 | Auxiliary driving method, device, equipment and storage medium |
| US20200262441A1 (en) * | 2019-02-15 | 2020-08-20 | Subaru Corporation | Vehicle driving control system |
| GB201903421D0 (en) * | 2019-03-13 | 2019-04-24 | Jaguar Land Rover Ltd | Vehicle control system |
| EP3718848A1 (en) * | 2019-04-04 | 2020-10-07 | Visteon Global Technologies, Inc. | System and method for executing an automated driving maneuver in a takeover process of a vehicle from automated driving to manually driving |
| CN110104051A (en) * | 2019-05-30 | 2019-08-09 | 爱驰汽车有限公司 | Automatic Pilot switching method, system, equipment and storage medium based on steering wheel |
| CN110660258A (en) * | 2019-08-23 | 2020-01-07 | 福瑞泰克智能系统有限公司 | Reminding method and device for automatically driving automobile |
| CN112550310A (en) * | 2019-09-25 | 2021-03-26 | 株式会社斯巴鲁 | Vehicle control device |
| CN112758098A (en) * | 2019-11-01 | 2021-05-07 | 广州汽车集团股份有限公司 | Vehicle driving authority take-over control method and device based on driver state grade |
| CN111516698A (en) * | 2020-04-08 | 2020-08-11 | 中国第一汽车股份有限公司 | Driving assistance system control method, driving assistance system, vehicle, and storage medium |
| CN112114671A (en) * | 2020-09-22 | 2020-12-22 | 上海汽车集团股份有限公司 | Human-vehicle interaction method and device based on human eye sight and storage medium |
| CN112960001A (en) * | 2021-04-19 | 2021-06-15 | 北京七鑫易维信息技术有限公司 | Driving mode switching method and device, vehicle and storage medium |
| CN113120068A (en) * | 2021-05-17 | 2021-07-16 | 上海联适导航技术股份有限公司 | Automatic drive agricultural machinery and have steering wheel of button control function |
Non-Patent Citations (1)
| Title |
|---|
| 张大权等: "关于L3自动驾驶的人机交互设计研究", 《汽车文摘》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109353347B (en) | Vehicle and driving takeover reminding method and system thereof, electronic equipment and storage medium | |
| CN110727346B (en) | Man-machine interaction method and device, vehicle and storage medium | |
| CN108891413B (en) | Vehicle-mounted fatigue driving prevention system, method, equipment and storage medium | |
| JP6076595B2 (en) | Reporting system | |
| CN109624985B (en) | Early warning method and device for preventing fatigue driving | |
| CN110103989A (en) | Automatic Pilot active interactive onboard system, method, equipment and storage medium | |
| CN110659013A (en) | Message processing method and device and storage medium | |
| JP2022122981A (en) | Method and apparatus for connecting through on-vehicle bluetooth, electronic device, and storage medium | |
| CN114296943A (en) | Resource allocation method, device and equipment based on virtualization technology | |
| CN110648494A (en) | Multi-device reminding method and device | |
| CN112937601B (en) | Information prompting method and device and vehicle | |
| CN113335300A (en) | Man-vehicle takeover interaction method, device, equipment and storage medium | |
| CN114906086A (en) | Automobile alarm method, electronic equipment, automobile, storage medium and server | |
| CN112437246B (en) | Video conference method based on intelligent cabin and intelligent cabin | |
| CN111885559A (en) | Intelligent device searching method, vehicle-mounted device system and searching device | |
| CN207481769U (en) | A kind of vehicle switch function system for prompting | |
| CN114132329B (en) | Driver attention maintaining method and system | |
| CN112758028B (en) | Number display method, system, vehicle and computer readable storage medium | |
| CN114056283B (en) | Method for automatically releasing brake during dragging of vehicle, intelligent vehicle and storage medium | |
| CN116080565A (en) | Control method and device of vehicle-mounted intelligent equipment, vehicle and storage medium | |
| CN113183971B (en) | Driver state monitoring method and device, vehicle-mounted terminal and readable storage medium | |
| CN112970060A (en) | Auxiliary method and device for assisting the use of a voice assistant in a vehicle | |
| CN117726488A (en) | Silence alarm method, silence alarm device, vehicle and storage medium | |
| CN115734206A (en) | Vehicle-mounted machine Bluetooth connection method and device, electronic equipment, storage medium and vehicle | |
| CN115865789A (en) | Cross-domain communication method, communication domain, electronic device, medium, and autonomous vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210903 |