CN106354259A - Automobile HUD gesture-interaction-eye-movement-assisting system and device based on Soli and Tobii - Google Patents
Automobile HUD gesture-interaction-eye-movement-assisting system and device based on Soli and Tobii Download PDFInfo
- Publication number
- CN106354259A CN106354259A CN201610784282.8A CN201610784282A CN106354259A CN 106354259 A CN106354259 A CN 106354259A CN 201610784282 A CN201610784282 A CN 201610784282A CN 106354259 A CN106354259 A CN 106354259A
- Authority
- CN
- China
- Prior art keywords
- driver
- soli
- hud
- tobii
- gesture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
Abstract
The invention discloses an automobile HUD gesture-interaction-eye-movement-assisting system and device based on Soli and Tobii. The system comprises a Soli gesture recognition radar, a Tobii eye movement tracker, an image-processing and miniature projection unit and a processor. The Tobii eye movement tracker is used for collecting the movement of eyeballs of a driver to obtain an object clicking control instruction; the Soli gesture recognition radar is used for collecting the movement of fingers of the driver to obtain an input instruction; the data of the Soli gesture recognition radar and the data of the Tobii eye movement tracker are transmitted into the processor for integrating the interaction data and processed with the processor, and an operation instruction is generated and sent to vehicle-mounted devices. The whole device is designed to be of an integrated box shell structure, is installed on an instrument panel and comprises a support, a collecting and emitting face, a projection reflection face and a mainboard, the support is used for fixing a whole box, and the collecting and emitting face is located outside the box and faces the driver. The automobile HUD gesture-interaction-eye-movement-assisting system and device have the advantages that driving is safer, the sight line cannot be disturbed, and distraction of the driver is reduced.
Description
Technical field
The present invention relates to automobile field of human-computer interaction.
Background technology
The Consumer's Experience of traditional vehicle manned machine interaction schemes is bad.With vehicle intellectualized and car networking service development,
In-car equipment and information were all significantly increased more in the past, becoming increasingly complex of onboard system, simultaneously the demand of people's car mutual also with
Increase, the cognitive load of therefore driver is higher, and traditional human-computer interaction interface (physical button, knob, touch screen) cannot
Solve the problems, such as under the new situation, to lead to vehicle-mounted man-machine interaction Consumer's Experience bad.
Emerging vehicle-mounted man-machine interaction scheme Shortcomings:
" emerging vehicle-mounted man-machine interaction scheme " refers to related to the present invention two technology: gesture interaction and automobile are looked squarely now
Real (hud).
1st, gesture interaction: there is part manufacturer (as BMW 7 is) to release the onboard system supporting gesture interaction in the market,
But its scheme has the shortcomings that three aspects:
(1) a kind of gesture can only support single operational order, and in order to not make interactive mode excessively complicated, gesture interaction is only
Very limited amount of operation can be carried out；
(2) technology of existing gesture identification employs the scheme of computer vision, catches hand motion by photographic head and holds
It is vulnerable to and blocks, the position therefore gesture being occurred has compared with strict requirements (for example in central control panel dead ahead), handss during operation
Portion has to departure direction disk.The cognitive load lifting of the driver so making, causes driving to divert one's attention；
(3) the identifying schemes reliability of computer vision is poor, easily brings maloperation, identification mistake, operating delay etc.
Problem, and these problems extremely affect driving and carry out safety.
2nd, automobile hud: automobile hud be a kind of by onboard system information projection device on the windshield, traditional hud
Normally only there is the very limited function of the information such as presentation speed, minority emerging hud product possesses gesture or voice operating
Function (as navdy), but do not account for the application characteristic of hud, still continued to use the interaction logic of screen display, easily caused
Driver diverts one's attention, and leads to vehicle accident.
Content of the invention
It is an object of the invention to overcoming the deficiencies in the prior art, a kind of automobile hud eye based on soli and tobii is disclosed
Dynamic auxiliary gesture interaction system and its apparatus, present invention mainly solves problems with:
1. set up natural man-machine interaction mode, reduce cognitive load when driving the person's of sailing operation onboard system；
2. lift the availability of gesture interaction, make that gesture interaction mechanism is easy to learn, operation promptly and accurately, in interaction
Hand need not leave the travel visual field without departure direction disk, sight line；
3. interactive mode can respond the special demands under user's concrete scene, driver is not driven and interfere；
4. interactive mode complete function, can complete multiclass operation task using simple interaction logic.
For this reason, the technical scheme one that the present invention is given:
A kind of automobile hud eye dynamic auxiliary gesture interaction system based on soli and tobii is it is characterised in that include soli
Gesture identification radar, tobii eye move tracker, image procossing and micro projection unit, processor,
Described tobii eye moves tracker and is used for gathering driver's eyeball action acquisition click manipulation subject instructions；
Described soli gesture identification radar is used for gathering driver's finger movement acquisition input instruction；
Described soli gesture identification radar and tobii eye move the data transfer of tracker to integrated for interaction data
In processor, treated device generates operational order after processing, and operational order is sent to each mobile unit；Separately by mobile unit
Status data returns to processor, and is sent to hud image procossing and micro projection unit；Described hud image procossing and miniature
Video data is passed through in-car windshield and shows by projecting cell, is presented on driver at the moment, instructs further for driver
Operation.
1. utilize micro projector (5) by hud interface image through one piece of minute surface (7) reflective projection to windshield
On, the virtual image is shown in outside windshield about 2 meters of positions.Hud interface comprises to represent the icon of in-car controllable equipment, has unselected
In, choose, operate in 2 kinds of states, different conditions hypograph size, transparency are different.In operating process, hud plays operation in interface
The effect of staining effect.
2. utilize tobii is4 (1) to obtain the position data of the visual fixations of driver, system is noted according to pilot's line of vision
Apparent place judges driver's demand now and intention, and so that hud graphical interfaces is responded therewith.When driver watches road surface attentively,
Hud interface image size is less, and transparency is higher, does not affect the normal driving of driver.When driver watches position entrance circle attentively
During the region of face, hud interface becomes apparent from, and becomes large-sized.When the focus vision of driver is in an icon, watched attentively icon
Become large-sized, other icon size diminish, to driver to feed back, represent that driver now have selected the corresponding work(of this icon
Energy.
3. utilize soli (8) to obtain driver's hand motion, driver is interacted with onboard system by gesture, to control car
Interior equipment.System is provided with three groups of basic interaction gestures, respectively " upwards/lower swing palm ", " referring to left/right slip is double ", " receipts
Hold together/open the five fingers ", disclosure satisfy that the major part mobile unit such as media, air-conditioning, vehicle window, seat heating, phone, short message, navigation
And the operational requirements (table 1 as shown in Figure 6) of application, vehicular applications developer can call gesture interaction function for new opplication,
Can also be based on the new gesture of the system exploitation, to expand its function.Basic gesture and its corresponding interactive function, such as Fig. 6.
4. gesture and eye movement data generate operational order in interaction data integrated processor (12), send out through can bus
Deliver to each mobile unit electronic control unit, execute interactive instruction, meanwhile, the status data of mobile unit returns through can bus
To processor.After system gets data, hud interface produces response.
The technical scheme two that the present invention is given:
A kind of automobile hud eye dynamic auxiliary gesture interaction device based on soli and tobii is it is characterised in that whole device
Design integrated box shell structure, be installed on and be positioned over instrumental panel upper area, including stake body, collection and the surface of emission,
Projection reflecting surface, mainboard, described stake body is used for the whole box body of set；Described collection and the surface of emission are located at outside box body, towards driving
The person of sailing；Described projection emission face is towards windshield；Described mainboard is installed on tray interior, box inner part and being connected with mainboard
The peripheral functional modules connecing have tobii eye to move tracker 1, radar antenna 4, soli micro radar chip 8, minute surface reflector 7, figure
As processing and micro projection unit 5, interaction data integrated processor 12；
Described tobii eye moves tracker 1 and is located on collection and the surface of emission, and the eyes of be aligned driver catch driver's eye
The motion of ball；
Described radar antenna 4 is located on collection and the surface of emission, for launching and accepting driver's hand motion signal；
Described soli micro radar chip 8 is connected with radar antenna 4, for identifying driver's hand motion；
Described minute surface reflector 7 is installed on the projection reflecting surface of box body, and hud figure is refracted on windshield；
Described image is processed and micro projection unit 5 is installed on mainboard, generates and render hud interface image, and through mirror
Face reflector 7 projects on windshield, increases virtual image imaging distance by the refraction of minute surface reflector 7；
Described interaction data integrated processor 12 passes through mainboard and moves tracker 1, soli micro radar core with tobii eye respectively
Piece 8, image procossing and micro projection unit 5 connect, for data processing and generation interactive instruction.
Beneficial effects of the present invention:
1. operate mobile unit can reduce time and the distance that pilot's line of vision leaves the road by hud, make driving more
Safety；
2. move to follow the trail of by eye and make the demand of hud interface active response user: by pilot's line of vision, system may determine that
The wish of its interaction.When not interacting wish, outside functional area, hud interface assumes higher clarity to the sight line of driver
And reduced size, without interference with sight line, decrease driver distraction.
3. the rules of interaction that tracking establishes " the regarded target of driver is manipulated object " is moved by eye, with naturally regarding
Line focus instead of chooses target interactive object manually, makes driver in gesture interaction without the sub- level of entered function, you can straight
Connect and using general gestures, mobile unit is manipulated；
4. the present invention passes through eye dynamic tracking determination interactive object, compares the gesture interaction of anophthalmia dynamic auxiliary, gesture does not need
With the characteristic information distinguishing interactive object, the quantity of gesture motion is made to be reduced, same set of gesture can be to multiple differences
Equipment is operated, and ensure that the simple of interactive mode while assurance function is comprehensive.
5. the present invention adopts micro radar chip soli to detect gesture motion, makes driver during gesture interaction,
Without departure direction disk, position that need not be strict and movement range require hand, and gesture identification will not be because of the screening of steering wheel
Gear and angle problem are affected；
6. pass through the dynamic cooperation with gesture interaction of eye, imitated the mode of mankind's nature manual communication, made the man-machine friendship of automobile
Mutually more intuitive, the cognitive load of driver therefore reduces, and Consumer's Experience is better.
Brief description
Fig. 1. present invention display system Organization Chart
Fig. 2. the flow process interacting with windshield display system is moved by gesture and eye
Fig. 3. the perspective view of apparatus of the present invention and explosive view
Fig. 4. the present invention uses scene graph
Fig. 5 .hud interface response mechanism
Fig. 6 basic gesture and its corresponding interactive function
Specific embodiment
(1) technical solution of the present invention is produced by following technical support:
1.tobii is a visual movement tracker (eye tracker) that can be embedded into interiors of products, can obtain
The data of people's eyes fixation positions, and it is converted into digital signal, move interactive system for eye and use.
2.soli is a product identifying trickle finger movement using microchip radar, is researched and developed by Google.
Compare the method identifying gesture using computer vision technique, identification maneuver is more accurate, timely, to the position of hand motion, width
Degree has bigger pardon.
3. automobile hud (heads-up display) refers to for onboard system interface to pass through projector equipment, through minute surface folding
Penetrate the equipment that the virtual image is formed on the glass plate project to windshield or peripheral hardware.Operator onboard system can be made need not will to regard
Line moves to outside the travel visual field.
4. a dynamic auxiliary gesture interaction refers to move tracing equipment using eye, using human eye movement's information as the one of interactive system
Part input data, the gesture information getting in conjunction with gesture identification equipment, the accuracy of lifting gesture interaction, reduction are complicated
Degree, sets up more preferable Consumer's Experience.
(2), below in conjunction with accompanying drawing, technical solution of the present invention is described further.
Present system by tobi eye move tracker, soli gesture identification radar input interaction data, by can bus with
Vehicle equipment communication, and hud interface is exported by image procossing and micro projection unit.
As shown in figure 1, present system: soli gesture identification radar and tobii eye move the data transfer of tracker to friendship
Mutually in data integration processor, treated device generates operational order after processing, and operational order is sent to each vehicle-mounted through can bus
The electronic control unit of equipment.Separately, the status data of mobile unit is returned to processor through can bus, be sent to hud image
Process and micro projection unit.
1. utilize micro projector (5) by hud interface image through one piece of minute surface (7) reflective projection to windshield
On, the virtual image is shown in outside windshield about 2 meters of positions.Hud interface comprises to represent the icon of in-car controllable equipment, has unselected
In, choose, operate in 2 kinds of states, different conditions hypograph size, transparency are different.In operating process, hud plays operation in interface
The effect of staining effect.
2. utilize tobii is4 (1) to obtain the position data of the visual fixations of driver, system is noted according to pilot's line of vision
Apparent place judges driver's demand now and intention, and so that hud graphical interfaces is responded therewith.When driver watches road surface attentively,
Hud interface image size is less, and transparency is higher, does not affect the normal driving of driver.When driver watches position entrance circle attentively
During the region of face, hud interface becomes apparent from, and becomes large-sized.When the focus vision of driver is in an icon, watched attentively icon
Become large-sized, other icon size diminish, to driver to feed back, represent that driver now have selected the corresponding work(of this icon
Energy.
3. utilize soli (8) to obtain driver's hand motion, driver is interacted with onboard system by gesture, to control car
Interior equipment.System is provided with three groups of basic interaction gestures, respectively " upwards/lower swing palm ", " referring to left/right slip is double ", " receipts
Hold together/open the five fingers ", disclosure satisfy that the major part mobile unit such as media, air-conditioning, vehicle window, seat heating, phone, short message, navigation
And the operational requirements (table 1 as shown in Figure 6) of application, vehicular applications developer can call gesture interaction function for new opplication,
Can also be based on the system new gesture newly developed, to expand its function.Basic gesture and its corresponding interactive function, such as Fig. 6.
4. gesture and eye movement data generate operational order in interaction data integrated processor (12), send out through can bus
Deliver to each mobile unit electronic control unit, execute interactive instruction, meanwhile, the status data of mobile unit returns through can bus
To processor.After system gets data, hud interface produces response.
Based on above system schema, as shown in figure 3, the present invention discloses device embodiment, whole device design
Integrated box shell structure, is installed on and is positioned over instrumental panel upper area, including stake body, collection and the surface of emission, projection
Reflecting surface, mainboard, described stake body is used for the whole box body of set；Described collection and the surface of emission are located at outside box body, towards driver；
Described projection emission face is towards windshield；Described mainboard is installed on tray interior, box inner part and being connected with mainboard
Peripheral functional modules have:
Tobii eye moves tracker 1: sensor is located on collection and the surface of emission, and the eyes of be aligned driver catch driver
The motion of eyeball.Need to calibrate according to hud interface after start.
Radar antenna 4: the ingredient of the radar for soli, the radar antenna of this soli is located on collection and the surface of emission,
For launching and accepting driver's hand motion signal.
Soli micro radar chip 8: the ingredient of the radar for soli, it is connected with radar antenna 4, drive for identification
Member's hand motion.
Image procossing and micro projection unit 5: it is installed on mainboard, generate and render hud interface image, and anti-through minute surface
Tabula rasa 7 projects on windshield.
Minute surface reflector 7: be installed on the projection reflecting surface of box body, hud figure is refracted on windshield, by folding
Penetrate increase virtual image imaging distance.
Interaction data integrated processor 12: tracker 1, soli micro radar chip are moved with tobii eye respectively by mainboard
8th, image procossing and micro projection unit 5 connect, for data processing and generation interactive instruction.
Shift knob 2: be connected on mainboard.Button lip has annular display lamp, when this equipment is closed, short by once pressing
This equipment opened by button, and display lamp lights 3 seconds；When the device is opened, the short display lamp that touches the button lights three seconds, and length was pressed by 3 seconds
Button equipment is closed.
Usb jack 9: be connected on mainboard, for the exploitation of equipment, maintenance and expansion.
Data wire 10: limit and be arranged on mainboard, for realizing data transfer and the power supply of whole system.
Radiating electric fan 11: be connected on mainboard, obtain power supply, for equipment cooling.
Optimize further, be additionally provided with optical filter 3: outside collection and the surface of emission, for filtering, protecting and fix that eye is dynamic to be chased after
Track device and radar antenna.
Described support 13: be designed specifically to: in arch form, radian can be adjusted by bending, for product is fixed on instrument
Above dial plate.
Specifically, cover top portion 6 can also be set: for protection and fixing internal part.
As shown in Figure 2:
1. when this equipment begins to use, it is first turned on switching, system initialization, correction eye moves tracing system.
2., when tobii eye moves tracker and recognizes driver eye movement behavior, system can be sentenced according to sight line position
Disconnected driver intention, when sight line enters hud interface function region, interface icon becomes big, and transparency reduces, the icon watched attentively
Become big.
3. when driver is look at icon 5s and makes interaction gesture, soli recognizes hand motion, and system obtains drives
The interactive instruction of member, hud interface gives to feed back, and instruction simultaneously is sent to mobile unit, sight line function by can bus.
If 4. driver watches system in icon 5s attentively and do not recognize gesture, system can judge driver's interaction meaning
It is willing to, interface can be returned to original state.
As shown in figure 4, apparatus of the present invention are positioned over above instrumental panel when using, can be adjusted by adjusting bracket during arrangement
Device height, makes unobstructed between equipment and pilot's line of vision, and hud projection is in windshield Production Zones Suitable.
(3) it is, to further illustrate the occupation mode of the present invention, to adjust air conditioning for automobiles and music as an example:
1. driver feels that vehicle interior temperature is low, air-conditioner temperature to be adjusted.
2. pilot's line of vision shifts to hud interface, and interface becomes clear therewith, and icon size becomes big.
3. pilot's line of vision falls on the icon of corresponding air conditioning function, and this icon size becomes big simultaneously, other icon size
Reduce.
4. driver becomes with digital figure to swing palm, simultaneously air-conditioning icon, and numeral is with driver's hand
Often upper swing once increases by one.Driver's handss swing three times 3 times palms, and the numeral at interface increases and increases to 25 from 22.
5. driver slides palm to the left, and air-conditioning interface is changed into wind direction figure, swings to the left with driver's palm, air-conditioning
Wind direction turns left.
6. driver completes that choosing of air-conditioning is whole, and sight line returns to road, and hud interface diminishes, and returns to the transparency of acquiescence.
7. driver adjusts the volume of music it is only necessary to sight line is concentrated on music with the gesture same with adjustment air-conditioner temperature
On icon.
Claims (7)
1. a kind of automobile hud eye dynamic auxiliary gesture interaction system based on soli and tobii is it is characterised in that include soli handss
Gesture Discrimination Radar, tobii eye move tracker, image procossing and micro projection unit, processor,
Described tobii eye moves tracker and is used for gathering driver's eyeball action acquisition click manipulation subject instructions；
Described soli gesture identification radar is used for gathering driver's finger movement acquisition input instruction；
Described soli gesture identification radar and tobii eye move the data transfer of tracker to the process integrated for interaction data
In device, treated device generates operational order after processing, and operational order is sent to each mobile unit；Separately by the state of mobile unit
Data returns to processor, and is sent to hud image procossing and micro projection unit；Described hud image procossing and micro projection
Video data is passed through in-car windshield and shows by unit, is presented on driver at the moment, for the further command operating of driver.
2. the system as claimed in claim 1 is it is characterised in that utilize micro projector (5) by hud interface image through one piece of mirror
To on windshield, the virtual image is shown in the position outside windshield to face (7) reflective projection.
3. the system as claimed in claim 1 is it is characterised in that described hud interface comprises to represent the figure of in-car controllable equipment
Mark, have unselected, choose, operate in 2 kinds of states, different conditions hypograph size, transparency are different.
4. the system as claimed in claim 1 it is characterised in that obtain the visual fixations of driver using tobii is4 (1)
Position data, system watches position judgment driver demand now and intention attentively according to pilot's line of vision, and makes hud graphical interfaces
Respond therewith.
5. the system as claimed in claim 1 is it is characterised in that when driver watches road surface attentively, hud interface image size is relatively
Little, transparency is higher, does not affect the normal driving of driver.When driver watches position attentively and enters interface zone, hud interface becomes
Obtain apparent, become large-sized.When the focus vision of driver is in an icon, is watched attentively icon size and become big, other icons
Size diminishes, and to driver to feed back, represents that driver now have selected the corresponding function of this icon.
6. the system as claimed in claim 1 it is characterised in that obtaining driver's hand motion using soli (8), lead to by driver
Cross gesture to interact with onboard system, to control in-vehicle device.System is provided with three groups of basic interaction gestures, respectively " upwards/bottom
Start to slap ", " sliding double finger to left/right ", " drawing/open the five fingers in ", disclosure satisfy that media, air-conditioning, vehicle window, seat heating, electricity
The major part mobile unit such as words, short message, navigation and the operational requirements of application.
7. a kind of automobile hud eye dynamic auxiliary gesture interaction device based on soli and tobii is it is characterised in that whole device sets
Count integrated box shell structure, be installed on and be positioned over instrumental panel upper area, including stake body, collection and the surface of emission, throwing
Shadow reflecting surface, mainboard, described stake body is used for the whole box body of set；Described collection and the surface of emission are located at outside box body, towards driving
Member；Described projection emission face is towards windshield；Described mainboard is installed on tray interior, box inner part and being connected with mainboard
Peripheral functional modules have tobii eye to move tracker 1, radar antenna 4, soli micro radar chip 8, minute surface reflector 7, image
Process and micro projection unit 5, interaction data integrated processor 12,
Described tobii eye moves tracker 1 and is located on collection and the surface of emission, and the eyes of be aligned driver catch driver's eyeball
Motion；
Described radar antenna 4 is located on collection and the surface of emission, for launching and receiving radar signal；
Described soli micro radar chip 8 is connected with radar antenna 4, for identifying driver's hand motion；
Described minute surface reflector 7 is installed on the projection reflecting surface of box body, and hud figure is refracted on windshield；
Described image is processed and micro projection unit 5 is installed on mainboard, generates and render hud interface image, and anti-through minute surface
Tabula rasa 7 projects on windshield, increases virtual image imaging distance by the refraction of minute surface reflector 7；
Described interaction data integrated processor 12 pass through mainboard respectively with tobii eye move tracker 1, soli micro radar chip 8,
Image procossing and micro projection unit 5 connect, for data processing and generation interactive instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610784282.8A CN106354259A (en) | 2016-08-30 | 2016-08-30 | Automobile HUD gesture-interaction-eye-movement-assisting system and device based on Soli and Tobii |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610784282.8A CN106354259A (en) | 2016-08-30 | 2016-08-30 | Automobile HUD gesture-interaction-eye-movement-assisting system and device based on Soli and Tobii |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106354259A true CN106354259A (en) | 2017-01-25 |
Family
ID=57857016
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610784282.8A Pending CN106354259A (en) | 2016-08-30 | 2016-08-30 | Automobile HUD gesture-interaction-eye-movement-assisting system and device based on Soli and Tobii |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106354259A (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107310476A (en) * | 2017-06-09 | 2017-11-03 | 武汉理工大学 | Eye dynamic auxiliary voice interactive method and system based on vehicle-mounted HUD |
| CN108196758A (en) * | 2018-01-31 | 2018-06-22 | 努比亚技术有限公司 | Icon amplification method, mobile terminal and computer readable storage medium |
| CN108681688A (en) * | 2017-03-31 | 2018-10-19 | 斑马网络技术有限公司 | Gesture identification component and its recognition methods |
| CN109109666A (en) * | 2018-09-03 | 2019-01-01 | 王宣武 | A kind of car front windshield windscreen vision control system |
| CN109144235A (en) * | 2017-06-19 | 2019-01-04 | 天津锋时互动科技有限公司深圳分公司 | Man-machine interaction method and system based on head hand co-operating |
| CN109542217A (en) * | 2018-10-12 | 2019-03-29 | 深圳市元征科技股份有限公司 | A kind of eyeball tracking display methods and device |
| CN109987036A (en) * | 2017-12-29 | 2019-07-09 | 周秦娜 | A kind of control method improving interaction accuracy based on driver's body posture |
| CN110045904A (en) * | 2018-11-14 | 2019-07-23 | 蔚来汽车有限公司 | Man-machine interactive system, method and the vehicle including the system |
| WO2019149061A1 (en) * | 2018-02-02 | 2019-08-08 | Huawei Technologies Co., Ltd. | Gesture-and gaze-based visual data acquisition system |
| CN110488978A (en) * | 2019-08-22 | 2019-11-22 | 大陆投资(中国)有限公司 | Vehicle drive cabin system and operation instruction interactive device and method |
| CN110481419A (en) * | 2019-08-16 | 2019-11-22 | 广州小鹏汽车科技有限公司 | A kind of people-car interaction method, system, vehicle and storage medium |
| CN110825216A (en) * | 2018-08-10 | 2020-02-21 | 北京魔门塔科技有限公司 | Method and system for man-machine interaction of driver during driving |
| CN111267757A (en) * | 2020-02-24 | 2020-06-12 | 南京航空航天大学 | Eyeball tracking-based automobile speed projection display system and working method thereof |
| CN111324202A (en) * | 2020-02-19 | 2020-06-23 | 中国第一汽车股份有限公司 | Interaction method, device, equipment and storage medium |
| CN111931579A (en) * | 2020-07-09 | 2020-11-13 | 上海交通大学 | Automatic driving assistance system and method using eye tracking and gesture recognition technology |
| CN112109730A (en) * | 2020-06-10 | 2020-12-22 | 上汽通用五菱汽车股份有限公司 | Reminding method based on interactive data, vehicle and readable storage medium |
| CN112799515A (en) * | 2021-02-01 | 2021-05-14 | 重庆金康赛力斯新能源汽车设计院有限公司 | Visual interaction method and system |
| WO2021136495A1 (en) * | 2020-01-03 | 2021-07-08 | 恒大新能源汽车投资控股集团有限公司 | Method and apparatus for controlling page layout of on-board unit display interface |
| WO2021218900A1 (en) * | 2020-04-30 | 2021-11-04 | 长城汽车股份有限公司 | Vehicle control method, apparatus and system |
| CN114523886A (en) * | 2020-10-30 | 2022-05-24 | 上海汽车集团股份有限公司 | In-vehicle interaction control method and system |
| CN114876312A (en) * | 2022-05-25 | 2022-08-09 | 重庆长安汽车股份有限公司 | Vehicle window lifting control system and method based on eye movement tracking |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103303224A (en) * | 2013-06-18 | 2013-09-18 | 桂林电子科技大学 | Vehicle-mounted equipment gesture control system and usage method thereof |
| CN103793059A (en) * | 2014-02-14 | 2014-05-14 | 浙江大学 | Gesture recovery and recognition method based on time domain Doppler effect |
| CN104875680A (en) * | 2015-06-03 | 2015-09-02 | 深圳市光晕网络科技有限公司 | HUD (head up display) device combining voice and video recognition |
| CN105278674A (en) * | 2014-06-03 | 2016-01-27 | 谷歌公司 | Radar-Based Gesture-Recognition through a Wearable Device |
| WO2016022770A1 (en) * | 2014-08-07 | 2016-02-11 | Google Inc. | Radar-based gesture sensing and data transmission |
| CN105882521A (en) * | 2014-11-21 | 2016-08-24 | 戴姆勒大中华区投资有限公司 | Control device and control method for activating corresponding zone of vehicle-mounted display screen according to watched zone of driver |
-
2016
- 2016-08-30 CN CN201610784282.8A patent/CN106354259A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103303224A (en) * | 2013-06-18 | 2013-09-18 | 桂林电子科技大学 | Vehicle-mounted equipment gesture control system and usage method thereof |
| CN103793059A (en) * | 2014-02-14 | 2014-05-14 | 浙江大学 | Gesture recovery and recognition method based on time domain Doppler effect |
| CN105278674A (en) * | 2014-06-03 | 2016-01-27 | 谷歌公司 | Radar-Based Gesture-Recognition through a Wearable Device |
| WO2016022770A1 (en) * | 2014-08-07 | 2016-02-11 | Google Inc. | Radar-based gesture sensing and data transmission |
| CN105882521A (en) * | 2014-11-21 | 2016-08-24 | 戴姆勒大中华区投资有限公司 | Control device and control method for activating corresponding zone of vehicle-mounted display screen according to watched zone of driver |
| CN104875680A (en) * | 2015-06-03 | 2015-09-02 | 深圳市光晕网络科技有限公司 | HUD (head up display) device combining voice and video recognition |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108681688A (en) * | 2017-03-31 | 2018-10-19 | 斑马网络技术有限公司 | Gesture identification component and its recognition methods |
| CN107310476A (en) * | 2017-06-09 | 2017-11-03 | 武汉理工大学 | Eye dynamic auxiliary voice interactive method and system based on vehicle-mounted HUD |
| CN109144235A (en) * | 2017-06-19 | 2019-01-04 | 天津锋时互动科技有限公司深圳分公司 | Man-machine interaction method and system based on head hand co-operating |
| CN109144235B (en) * | 2017-06-19 | 2024-01-23 | 深圳巧牛科技有限公司 | Man-machine interaction method and system based on head-hand cooperative action |
| CN109987036A (en) * | 2017-12-29 | 2019-07-09 | 周秦娜 | A kind of control method improving interaction accuracy based on driver's body posture |
| CN108196758A (en) * | 2018-01-31 | 2018-06-22 | 努比亚技术有限公司 | Icon amplification method, mobile terminal and computer readable storage medium |
| WO2019149061A1 (en) * | 2018-02-02 | 2019-08-08 | Huawei Technologies Co., Ltd. | Gesture-and gaze-based visual data acquisition system |
| CN110825216A (en) * | 2018-08-10 | 2020-02-21 | 北京魔门塔科技有限公司 | Method and system for man-machine interaction of driver during driving |
| CN109109666A (en) * | 2018-09-03 | 2019-01-01 | 王宣武 | A kind of car front windshield windscreen vision control system |
| CN109542217A (en) * | 2018-10-12 | 2019-03-29 | 深圳市元征科技股份有限公司 | A kind of eyeball tracking display methods and device |
| CN109542217B (en) * | 2018-10-12 | 2022-05-17 | 深圳市元征科技股份有限公司 | Eyeball tracking display method and device |
| CN110045904A (en) * | 2018-11-14 | 2019-07-23 | 蔚来汽车有限公司 | Man-machine interactive system, method and the vehicle including the system |
| CN110481419A (en) * | 2019-08-16 | 2019-11-22 | 广州小鹏汽车科技有限公司 | A kind of people-car interaction method, system, vehicle and storage medium |
| CN110481419B (en) * | 2019-08-16 | 2021-12-07 | 广州小鹏汽车科技有限公司 | Human-vehicle interaction method, system, vehicle and storage medium |
| CN110488978A (en) * | 2019-08-22 | 2019-11-22 | 大陆投资(中国)有限公司 | Vehicle drive cabin system and operation instruction interactive device and method |
| WO2021136495A1 (en) * | 2020-01-03 | 2021-07-08 | 恒大新能源汽车投资控股集团有限公司 | Method and apparatus for controlling page layout of on-board unit display interface |
| CN111324202A (en) * | 2020-02-19 | 2020-06-23 | 中国第一汽车股份有限公司 | Interaction method, device, equipment and storage medium |
| CN111267757A (en) * | 2020-02-24 | 2020-06-12 | 南京航空航天大学 | Eyeball tracking-based automobile speed projection display system and working method thereof |
| WO2021218900A1 (en) * | 2020-04-30 | 2021-11-04 | 长城汽车股份有限公司 | Vehicle control method, apparatus and system |
| CN112109730A (en) * | 2020-06-10 | 2020-12-22 | 上汽通用五菱汽车股份有限公司 | Reminding method based on interactive data, vehicle and readable storage medium |
| CN111931579B (en) * | 2020-07-09 | 2023-10-31 | 上海交通大学 | Automatic driving assistance system and method using eye tracking and gesture recognition techniques |
| CN111931579A (en) * | 2020-07-09 | 2020-11-13 | 上海交通大学 | Automatic driving assistance system and method using eye tracking and gesture recognition technology |
| CN114523886A (en) * | 2020-10-30 | 2022-05-24 | 上海汽车集团股份有限公司 | In-vehicle interaction control method and system |
| CN112799515A (en) * | 2021-02-01 | 2021-05-14 | 重庆金康赛力斯新能源汽车设计院有限公司 | Visual interaction method and system |
| CN114876312A (en) * | 2022-05-25 | 2022-08-09 | 重庆长安汽车股份有限公司 | Vehicle window lifting control system and method based on eye movement tracking |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106354259A (en) | Automobile HUD gesture-interaction-eye-movement-assisting system and device based on Soli and Tobii | |
| KR101730315B1 (en) | Electronic device and method for image sharing | |
| CN110045825B (en) | Gesture recognition system for vehicle interaction control | |
| US20160132126A1 (en) | System for information transmission in a motor vehicle | |
| CN103303224B (en) | Vehicle-mounted equipment gesture control system and usage method thereof | |
| US9656690B2 (en) | System and method for using gestures in autonomous parking | |
| Pickering et al. | A research study of hand gesture recognition technologies and applications for human vehicle interaction | |
| US20190001987A1 (en) | Vehicle and control method thereof | |
| KR102029842B1 (en) | System and control method for gesture recognition of vehicle | |
| US20160188190A1 (en) | Configurable dash display | |
| US20150352953A1 (en) | Vehicle control system with mobile device interface | |
| EP3508370B1 (en) | Input output device and vehicle comprising the same | |
| WO2019137959A2 (en) | Autonomous vehicles and methods of using same | |
| US20110169750A1 (en) | Multi-touchpad multi-touch user interface | |
| CN106938612B (en) | Display apparatus and vehicle | |
| US20170277323A1 (en) | Display device and vehicle having the same | |
| US10209832B2 (en) | Detecting user interactions with a computing system of a vehicle | |
| CN105786172A (en) | System and method of tracking with associated sensory feedback | |
| US20190288916A1 (en) | System and method for a vehicle zone-determined reconfigurable display | |
| CN107310476A (en) | Eye dynamic auxiliary voice interactive method and system based on vehicle-mounted HUD | |
| JP2016027474A (en) | Display control device and display control program | |
| CN105991942A (en) | Vehicle and method for controlling the vehicle | |
| WO2020195625A1 (en) | Information processing device, information processing method, and information processing program | |
| EP3659848A1 (en) | Operating module, operating method, operating system and storage medium for vehicles | |
| EP4137380A1 (en) | Human-computer interaction method and apparatus, and electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170125 |
|
| WD01 | Invention patent application deemed withdrawn after publication |