CN110341617A - Eyeball tracking method, apparatus, vehicle and storage medium - Google Patents
Eyeball tracking method, apparatus, vehicle and storage medium Download PDFInfo
- Publication number
- CN110341617A CN110341617A CN201910611395.1A CN201910611395A CN110341617A CN 110341617 A CN110341617 A CN 110341617A CN 201910611395 A CN201910611395 A CN 201910611395A CN 110341617 A CN110341617 A CN 110341617A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- information
- eyeball tracking
- eye
- parameter information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 210000005252 bulbus oculi Anatomy 0.000 title claims abstract description 179
- 238000000034 method Methods 0.000 title claims abstract description 75
- 210000001508 eye Anatomy 0.000 claims abstract description 257
- 230000003287 optical effect Effects 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 21
- 238000004891 communication Methods 0.000 claims description 19
- 230000000007 visual effect Effects 0.000 claims description 16
- 230000007704 transition Effects 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 7
- 210000001742 aqueous humor Anatomy 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 13
- 230000009466 transformation Effects 0.000 description 9
- 230000015654 memory Effects 0.000 description 8
- 230000004424 eye movement Effects 0.000 description 6
- 230000003993 interaction Effects 0.000 description 6
- 210000004087 cornea Anatomy 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 210000001747 pupil Anatomy 0.000 description 4
- 241000208340 Araliaceae Species 0.000 description 3
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 3
- 235000003140 Panax quinquefolius Nutrition 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 235000008434 ginseng Nutrition 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000005291 magnetic effect Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000001028 reflection method Methods 0.000 description 2
- 244000283207 Indigofera tinctoria Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 210000000744 eyelid Anatomy 0.000 description 1
- 206010016256 fatigue Diseases 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 210000002189 macula lutea Anatomy 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Arrangement of adaptations of instruments
-
- B60K35/10—
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- B60K2360/149—
Abstract
The embodiment of the invention discloses a kind of eyeball tracking method, apparatus, vehicle and storage mediums, and wherein this method is applied to eyeball tracking equipment, and eyeball tracking device configuration is on vehicle, comprising: obtains the eyes image of user；The eye parameter information of user is determined according to eyes image and target alignment coefficient；Eye parameter information is sent to vehicle so that vehicle according to eye parameter information and vehicle configuration parameter determine user inside the vehicle watch information attentively.The embodiment of the present invention, which is realized by the way that a kind of portable eyeball tracking equipment is arranged on vehicle, watches the determination of information attentively, compared with the prior art, it occupies little space, with bigger convenience, scalability and robustness, and there is no limit to vehicle itself, the utilization rate of eyeball tracking equipment in the car is greatly improved, the safety of vehicle is further improved.
Description
Technical field
The present embodiments relate to vehicle technology field more particularly to a kind of eyeball tracking method, apparatus, vehicle and storage
Medium.
Background technique
With the development of automatic Pilot technology, the variation of automobile interior design is constantly pushed, and vehicle interior occurs respectively
Kind variety classes equipment is to support human-computer interaction abundant, and wherein eye movement interaction is one of important interactive mode.
In the prior art, the mode that eye movement interaction is realized in vehicle usually configures expensive eye movement mould inside the vehicle
The equipment of block or special eye movement identifying system.But existing eyeball tracking method in the car usually require in vehicle
Equipment be used cooperatively, different vehicles need to configure it is special realization eye movement interaction equipment or system, it is at high cost, use
Limitation is strong, is not able to satisfy the demand of people.
Summary of the invention
The embodiment of the invention provides a kind of eyeball tracking method, apparatus, vehicle and storage mediums, can solve existing skill
The at high cost and strong technical problem of limitation in art.
In a first aspect, being applied to eyeball tracking equipment, the eye the embodiment of the invention provides a kind of eyeball tracking method
Ball tracing equipment is configured on vehicle, comprising:
Obtain the eyes image of user；
The eye parameter information of user is determined according to the eyes image and target alignment coefficient；
The eye parameter information is sent to the vehicle, so that the vehicle is according to the eye parameter information and vehicle
Configuration parameter determine user inside the vehicle watch information attentively.
Further, before the eyes image for obtaining user, further includes:
The calibration factor for calibrating initialization, obtains the target alignment coefficient.
Further, the calibration factor for calibrating initialization obtains the target alignment coefficient, comprising:
By projection module according to preset target position information projection calibrating light, calibration mark is formed；
The eyes image calibrated when identifying and the target position information are watched attentively to the initialization according to user
Calibration factor calibrated, obtain the target alignment coefficient.
Further, the eyes image of user is obtained, comprising:
User images are acquired by image capture module；
The eyes image in the user images is extracted by eye detection algorithm.
Further, the eye parameter information of user is determined according to the eyes image and target alignment coefficient, comprising:
Extract the eye feature information in the eyes image；
According to the eye feature information and the target alignment coefficient, the eye parameter information is determined, wherein institute
Stating eye parameter information includes eyeball center and direction of visual lines vector.
Further, the target alignment coefficient includes corneal curvature radius, cornea aqueous humor refractive index, eye optical axis and view
At least one of the distance of the angle of axis and corneal curvature center to pupil center location.
Further, the vehicle configuration parameter includes the position ginseng positioned at least one vehicle device in the compartment
Number.
Second aspect, the embodiment of the invention also provides a kind of eyeball tracking methods, are applied to vehicle, match in the vehicle
It is equipped with eyeball tracking equipment, comprising:
Receive the eye parameter information that the eyeball tracking equipment is sent
Determine user in the information of watching attentively of the vehicle interior, the eye parameter information according to the eye parameter information
It is determined by eyeball tracking method as described in relation to the first aspect.
Further, determine that user watches information attentively in the vehicle interior according to the eye parameter information, comprising:
Obtain the first location information of the eyeball tracking equipment and the second location information of the vehicle；
According to the first location information and the second location information, vehicle coordinate transition matrix is determined；
According to the vehicle coordinate transition matrix and the eye parameter information, the vehicle eye parameter letter that obtains that treated
Breath；
Determine that user watches information attentively in the vehicle interior based on the vehicle eye parameter information.
Further, according to the eye parameter information determine user the vehicle interior watch information attentively after, also
Include:
Information execution predetermined registration operation is watched attentively according to described.
The third aspect, the embodiment of the invention also provides a kind of eyeball tracking devices, are set to eyeball tracking equipment, described
Eyeball tracking device configuration is on vehicle, comprising:
Image collection module, for obtaining the eyes image of user；
Information determination module, for determining that the eye parameter of user is believed according to the eyes image and target alignment coefficient
Breath；
Information sending module, for sending the eye parameter information to the vehicle, so that the vehicle is according to
Eye parameter information and vehicle configuration parameter determine user inside the vehicle watch information attentively.
Further, the eyeball tracking device further include:
Calibration module before the eyes image for obtaining user, calibrates the calibration factor of initialization, obtains the target
Calibration factor.
Further, the calibration module is specifically used for:
By projection module according to preset target position information projection calibrating light, calibration mark is formed；
The eyes image calibrated when identifying and the target position information are watched attentively to the initialization according to user
Calibration factor calibrated, obtain the target alignment coefficient.
Further, described image obtains module and is specifically used for:
User images are acquired by image capture module；
The eyes image in the user images is extracted by eye detection algorithm.
Further, the information determination module includes:
Feature extraction unit, for extracting the eye feature information in the eyes image；
Eye parameter information unit, for determining institute according to the eye feature information and the target alignment coefficient
Eye parameter information is stated, wherein the eye parameter information includes eyeball center and direction of visual lines vector.
Further, the target alignment coefficient includes corneal curvature radius, cornea aqueous humor refractive index, eye optical axis and view
At least one of the distance of the angle of axis and corneal curvature center to pupil center location.
Fourth aspect, the embodiment of the invention also provides a kind of eyeball tracking devices, are applied to vehicle, match in the vehicle
It is equipped with eyeball tracking equipment, comprising:
Information receiving module, the eye parameter information sent for receiving the eyeball tracking equipment
Information module is watched attentively, for determining that user watches letter attentively in the vehicle interior according to the eye parameter information
Breath, the eye parameter information are determined by eyeball tracking method as described in relation to the first aspect.
Further, described to watch information module attentively and be specifically used for:
Obtain the first location information of the eyeball tracking equipment and the second location information of the vehicle；
According to the first location information and the second location information, vehicle coordinate transition matrix is determined；
According to the vehicle coordinate transition matrix and the eye parameter information, the vehicle eye parameter letter that obtains that treated
Breath；
Determine that user watches information attentively in the vehicle interior based on the vehicle eye parameter information.
Further, the eyeball tracking device further includes；
Operation module, for according to the eye parameter information determine user the vehicle interior watch attentively information it
Afterwards, information execution predetermined registration operation is watched attentively according to described.
5th aspect is configured with eyeball tracking equipment, packet the embodiment of the invention also provides a kind of vehicle in the vehicle
It includes:
One or more first processors；
First storage device, for storing one or more first programs；
When one or more of first programs are executed by one or more of first processors so that it is one or
Multiple first processors realize the eyeball tracking method as described in second aspect；
Wherein, the eyeball tracking equipment includes:
One or more second processors；
Projection module is used for projection light；
Communication module, for sending eye parameter information to vehicle；
Image capture module, for acquiring user images；
Light source module, for providing visible light source for the projection device；
Second storage device, for storing one or more second programs；
When one or more of second programs are executed by one or more of second processors so that it is one or
Multiple second processors realize eyeball tracking method as described in relation to the first aspect.
6th aspect, the embodiment of the invention also provides a kind of computer readable storage mediums, are stored thereon with computer
Program realizes eyeball tracking method as described in relation to the first aspect and/or as described in second aspect when the program is executed by processor
Eyeball tracking method.
The embodiment of the present invention obtains the eyes image of user by the eyeball tracking equipment being set on vehicle, according to described
Eyes image and target alignment coefficient determine the eye parameter information of user；The eye parameter information is sent to the vehicle,
So that the vehicle according to the eye parameter information and vehicle configuration parameter determine user inside the vehicle watch information attentively.
The embodiment of the present invention, which is realized by the way that a kind of portable eyeball tracking equipment is arranged on vehicle, watches the determination of information attentively, compared to
The prior art occupies little space, and has bigger convenience, scalability and robustness, and there is no limit to vehicle itself,
The utilization rate of eyeball tracking equipment in the car is greatly improved, the safety of vehicle is further improved.
Detailed description of the invention
Fig. 1 is the flow chart of the eyeball tracking method in the embodiment of the present invention one；
Fig. 2 is the schematic diagram of the eyeball tracking method in the embodiment of the present invention one；
Fig. 3 is the outside schematic diagram of the eyeball tracking equipment in the embodiment of the present invention one；
Fig. 4 is the flow chart of the eyeball tracking method in the embodiment of the present invention two；
Fig. 5 is the flow chart of the eyeball tracking method in the embodiment of the present invention three；
Fig. 6 is the schematic diagram of the spatial alternation in the embodiment of the present invention three；
Fig. 7 is the structural schematic diagram of the eyeball tracking device in the embodiment of the present invention four；
Fig. 8 is the structural schematic diagram of the eyeball tracking device in the embodiment of the present invention five；
Fig. 9 is the structural schematic diagram of the vehicle in the embodiment of the present invention six.
Specific embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention rather than limiting the invention.It also should be noted that in order to just
Only the parts related to the present invention are shown in description, attached drawing rather than entire infrastructure.
Embodiment one
Fig. 1 is the flow chart of the eyeball tracking method in the embodiment of the present invention one, and the present embodiment is applicable to realize vehicle
The case where eyeball tracking is carried in driver's cabin, this method can be held by the eyeball tracking device being set in eyeball tracking equipment
Row, the device can be realized by the way of software and/or hardware.The eyeball tracking device configuration, can be with vehicle in vehicle
It interacts.
Fig. 2 is the schematic diagram of the eyeball tracking method in the embodiment of the present invention one, the eyeball tracking method in the present embodiment
It can be applied to be configured at the eyeball tracking equipment 11 on vehicle, the specific location of the eyeball tracking equipment 11 can be according to reality
Situation is set, such as eyeball tracking equipment 11 can be set in the reserved slot in compartment, and eyeball tracking equipment 11 is pre-
Stay the position in slot that can carry out adjusting up and down within a preset range according to the demand of user.It can be understood that should
Position the present embodiment of reserved slot is also not construed as limiting, such as can be set in front of gear controller or in front of windshield
Etc..Eyeball tracking equipment 11 can be connect by the communication protocol in communication module with vehicle, realize the interaction of information.
Eyeball tracking equipment 11 in figure is illustrated for being set to 16 front of gear controller, eyeball tracking in figure
Equipment 11 obtains the eyes image of user eyeball 14, and according to the eyes image of user and the preparatory target alignment coefficient calibrated
Eye parameter information is obtained, and the eye parameter information is sent to vehicle, vehicle can be according to eye parameter information and vehicle
Configuration parameter determine user inside the vehicle watch information attentively, i.e. blinkpunkt in figure is located at the A point in display screen 12.In figure
Box 12 is the image acquisition region of eyeball tracking equipment 11, and the front of user eyeball 14 is the steering wheel 15 in vehicle in figure.
Fig. 3 is the outside schematic diagram of the eyeball tracking equipment in the embodiment of the present invention one, the eyeball tracking in the present embodiment
Equipment can be a kind of portable equipment, can integrate in a cuboid electronic equipment of shape, eyeball tracking in Fig. 3
It can show projection module 120, communication module 130 and image capture module 140 outside equipment 11, other modules (such as
Control module, sound module and light source module etc.) it can be set in inside.
As shown in Figure 1, this method can specifically include:
S110, the eyes image for obtaining user.
Wherein, eyes image is the image for including user eyeball.
Specifically, eyeball tracking device can acquire user images by image capture module, and is detected and calculated by eye
Method extracts the eyes image in user images.Wherein, user images can be to include the image of face and do not include face
Image.After collecting user images, face lookup is carried out in user images, if not finding face in user images,
Reacquisition user images are then returned to, until face can be found in the user images obtained.Face is looked into the present embodiment
The method looked for is not construed as limiting, for example, by haar-like human-face detector algorithm detect face or by face characteristic (such as
The colour of skin or nose etc.) Face datection is carried out to user images, according to of face characteristic and standard feature in user images
Determine in user images whether include face with being worth.
Further, the eyes image in user images is extracted by eye detection algorithm, wherein to eye detection algorithm
Also it is not construed as limiting, such as eye detection algorithm may include method or Statistics-Based Method etc. based on template matching.This
By being illustrated for Statistics-Based Method in embodiment, by a large amount of samples including eyes image and do not include eye
The sample of eyeball image is trained study and obtains a group model parameter, is then based on model construction classifier or filter to examine
Survey the eyes image in user images.After detecting eyes image in user images, the eyes image is extracted.
S120, the eye parameter information that user is determined according to eyes image and target alignment coefficient.
Wherein, target alignment coefficient is used to estimate to watch information attentively by measurement eye motion situation, and watching information attentively can be with
Including sight or blinkpunkt etc..The specific method for determining that target alignment coefficient uses can be set according to the actual situation, this
Embodiment is illustrated by taking pupil-corneal reflection method in optical recording as an example, and the method for Eye-controlling focus may be implemented in other
It is also suitable.
It may include: to extract specifically, determining the eye parameter information of user according to eyes image and target alignment coefficient
Eye feature information in eyes image；According to eye feature information and target alignment coefficient, eye parameter information is determined,
Middle eye parameter information includes eyeball center and direction of visual lines vector.Wherein, eye feature information may include in pupil
Heart position, pupil shape, iris position, iris shape, eyelid position, canthus position, hot spot (also referred to as Purkinje image) position
Deng being illustrated by taking pupil center location in image and facula position as an example in the present embodiment.
This step can handle eyes image to obtain the gradient value of the gray scale on direction initialization, can be by the ash
The maximum of gradients present position of degree is determined as the pupil edge feature of the eyes image, then can to pupil edge feature into
Row fitting, determines the center of fitting image, and the center can be determined as to pupil center location in image.It can
With understanding, the means of determining pupil center location are not defined herein.Further, can to eyes image into
It is Purkinje image that gray level image after row processing can obtain the higher region of gray value by Spot recognition algorithm, and obtaining should
The position of Purkinje image is facula position.
Further, eye parameter information may include eyeball center and direction of visual lines vector, eye feature information
It may include that pupil center location and facula position according to eye feature information and target alignment coefficient determine eye parameter
Information may include: to obtain corneal curvature center according to the corneal curvature radius in facula position and target alignment coefficient, will
Corneal curvature center is determined as eyeball center；According in pupil center location, corneal curvature center and target alignment coefficient
Eye optical axis and the optical axis angle, obtain direction of visual lines vector.Wherein, optical axis is that pupil center location reaches foveal region of retina
Line, the optical axis is by blinkpunkt outside eye by the line of node and macula lutea, and 4 ° to 5 ° are formed between eye optical axis and the optical axis
Angle.Target alignment coefficient in the present embodiment may include corneal curvature radius, cornea aqueous humor refractive index, eye optical axis and view
The distance etc. of the angle of axis and corneal center to pupil center location.
S130, eye parameter information is sent to vehicle, so that vehicle is according to eye parameter information and vehicle configuration parameter
Determine user inside the vehicle watch information attentively.
Wherein, eye parameter information is the coordinate system based on eyeball tracking device.Vehicle configuration parameter includes being located at vehicle
The location parameter of at least one vehicle device in compartment, referring to fig. 2, vehicle device may include steering wheel 15 or windshield, HUD
(being not shown in Fig. 2) etc., location parameter are a coordinate of the vehicle device based on vehicle axis system.
Specifically, the eye parameter information can be sent to after eyeball tracking device gets eye parameter information
Vehicle, so that eye parameter information is converted to the ginseng of the eye under vehicle axis system according to first determining transformation matrix of coordinates by vehicle
Number information, and according to eye parameter information and vehicle configuration parameter determine user inside the vehicle watch information attentively.
Technical solution provided in this embodiment obtains the eye figure of user by the eyeball tracking equipment being set on vehicle
Picture determines the eye parameter information of user according to eyes image and target alignment coefficient；Eye parameter information is sent to vehicle, with
Make vehicle according to eye parameter information and vehicle configuration parameter determine user inside the vehicle watch information attentively.The present embodiment is logical
It crosses and a kind of portable eyeball tracking equipment is set on vehicle realizes the determination for watching information attentively, compared with the prior art, occupy
Space is small, has bigger convenience, scalability and robustness, and there is no limit greatly improve to vehicle itself
The utilization rate of eyeball tracking equipment in the car, further improves the safety of vehicle.
Embodiment two
Fig. 4 is the flow chart of the eyeball tracking method in the embodiment of the present invention two, base of the present embodiment in above-described embodiment
On plinth, above-mentioned eyeball tracking method has been advanced optimized.Correspondingly, the method for the present embodiment specifically includes:
The calibration factor that S210, calibration initialize, obtains target alignment coefficient.
Before executing the eyeball tracking in vehicle in the present embodiment, the calibration factor in eyeball tracking device is carried out in advance
Calibration.Determine that the specific method of target alignment coefficient is illustrated by taking pupil-corneal reflection method in optical recording as an example.
Specifically, calibration initialization calibration factor, obtain target alignment coefficient, may include: by projection module by
According to preset target position information projection calibrating light, calibration mark is formed；Watch eyes image when calibration mark attentively according to user
And target position information calibrates the calibration factor of initialization, obtains target alignment coefficient.
Wherein, target position information is set as the location information on objective plane, which can be arbitrary plane
The surface of object, such as horizontal table top, since the geometrical relationship in plane is easy to determine, not needing manpower intervention can determine sky
Between relationship, therefore, the target position in the present embodiment is set in plane.If the calibrating installation of eyeball tracking and target position
When spatial relationship predefines, which is not limited in plane.The quantity for calibrating mark can be multiple, calibration mark
It is not limited to an identification point, is also possible to measurement image and calibration picture that one is played mark action.For example, in the present embodiment
The shape of calibration mark can be for round, rectangular, linear and other irregular shapes etc., concrete shape can be according to reality
Situation is set.
The eye feature information that user images when eyeball tracking device can watch calibration mark attentively to user are extracted,
And according to eye feature information, target position information and the calibration factor of initialization, error on line of sight is obtained, according to error on line of sight school
Quasi- target alignment coefficient is calibrated according to error on line of sight and determines target alignment coefficient.Wherein, calibration factor can be for for being regarded
In the eye tracking model of line estimation it needs to be determined that parameter, the calibration factor of initialization is the calibration determined according to priori knowledge
Coefficient, the practical calibration factor are different for different users, it is therefore desirable to be calibrated.
Wherein, error on line of sight can be defined as eye boresight direction vector and estimate obtained eye to the side of calibration mark
To the differential seat angle between vector.Eyeball tracking device can be based on eye feature information, target position and the target of initialization school
Quasi- coefficient, direction vector of the available eye boresight direction vector sum eye to calibration mark；Based on angular error formula, eye
The direction vector that portion's boresight direction vector sum eye is identified to calibration, the error of available eye boresight direction regard eye
The error of axis direction is determined as error on line of sight.Then calibration unit can determine error on line of sight based on given error optimization algorithm
Minimal error, by the corresponding calibration factor of minimal error be determined as calibration after target alignment coefficient.
S220, the eyes image for obtaining user.
Eye feature information in S230, extraction eyes image.
S240, the eye parameter information that user is determined according to eye feature information and target alignment coefficient.
Specifically, eye parameter information may include eyeball center and direction of visual lines vector, eye feature information can
To include pupil center location and facula position, according to the target alignment system in eye feature information and target alignment coefficient
Number, determines eye parameter information, may include: to be obtained according to the corneal curvature radius in facula position and target alignment coefficient
Corneal curvature center is determined as eyeball center by corneal curvature center；According to pupil center location, corneal curvature center and
The angle of eye optical axis and the optical axis in target alignment coefficient, obtains direction of visual lines vector.
Wherein, corneal curvature center can be indicated using C, and pupil center location can be indicated using P, eye boresight direction
Vector can use VSIt indicates.Corneal curvature center can pass through C=f1(gi,Li, R) and it determines, wherein giIndicate facula position, Li
Indicate that the coordinate of light source in eyeball tracking equipment, R indicate corneal curvature radius in target alignment coefficient.Pupil center location can be with
Pass through P=f2(p, C, K, n) is determined, wherein p indicates that pupil center location coordinate in eyes image, K indicate target alignment coefficient
For middle corneal center to pupil center location distance, n indicates cornea aqueous humor refractive index in target alignment coefficient.Eye optical axis direction to
Amount can pass throughIt determines, whereinIndicate that the spherical surface of eye optical axis direction unit vector is sat
Mark indicates.Direction of visual lines vector (also known as eye boresight direction vector) can pass throughReally
It is fixed, wherein (α, β) indicates eye optical axis and optical axis angle.
S250, eye parameter information is sent to vehicle, so that vehicle is according to eye parameter information and vehicle configuration parameter
Determine user inside the vehicle watch information attentively.
Wherein, eye parameter information is the coordinate system based on eyeball tracking device.Vehicle configuration parameter includes being located at vehicle
The location parameter of at least one vehicle device in compartment, referring to fig. 2, vehicle device may include steering wheel 15 or windshield, HUD
(being not shown in Fig. 2) etc., location parameter are a coordinate of the vehicle device based on vehicle axis system.
Specifically, the eye parameter information can be sent to after eyeball tracking device gets eye parameter information
Vehicle, so that eye parameter information is converted to the ginseng of the eye under vehicle axis system according to first determining transformation matrix of coordinates by vehicle
Number information, and according to eye parameter information and vehicle configuration parameter determine user inside the vehicle watch information attentively.
Technical solution provided in this embodiment obtains the eye figure of user by the eyeball tracking equipment being set on vehicle
Picture determines the eye parameter information of user according to the target alignment coefficient after eyes image and calibration；Send eye parameter information
To vehicle so that vehicle according to eye parameter information and vehicle configuration parameter determine user inside the vehicle watch information attentively.
The present embodiment, which is realized by the way that a kind of portable eyeball tracking equipment is arranged on vehicle, watches the determination of information attentively, compared to existing
Technology occupies little space, and has bigger convenience, scalability and robustness, and there is no limit greatly to vehicle itself
Ground improves the utilization rate of eyeball tracking equipment in the car, further improves the safety of vehicle；And compared to existing
It carries out requiring to recalibrate when eyeball tracking every time in technology, due to the target alignment system of eyeball tracking equipment in the present embodiment
Number can directly be interacted by calibration in advance with vehicle, improve the efficiency of user's eye movement interaction.
Embodiment three
Fig. 5 is the flow chart of the eyeball tracking method in the embodiment of the present invention three.The present embodiment is applicable to realize vehicle
The case where middle eyeball tracking, this method can be executed by being set to eyeball tracking device in vehicle, which can use software
And/or the mode of hardware is realized.Eyeball tracking equipment is configured in the vehicle.This method can specifically include:
S310, the eye parameter information that eyeball tracking equipment is sent is received.
Wherein, eye parameter information can be determined in the eyeball tracking method in through the foregoing embodiment one and embodiment two,
The eye parameter information is to be indicated based on eyeball tracking device coordinate system.It can be by being set to eyeball in vehicle in the present embodiment
Communication module in follow-up mechanism receives the eye parameter information that eyeball tracking equipment is sent.
S320, determined according to eye parameter information user inside the vehicle watch information attentively.
Specifically, according to eyeball tracking equipment send eye parameter information determine user inside the vehicle watch letter attentively
Breath may include: the second location information for the first location information and vehicle for obtaining eyeball tracking equipment；According to first position
Information and second location information determine vehicle coordinate transition matrix；According to vehicle coordinate transition matrix and eye parameter information, obtain
Treated vehicle eye parameter information；Based on vehicle eye parameter information determine user inside the vehicle watch information attentively.
Wherein, the second location information of vehicle can be the origin of vehicle axis system, in setting the present embodiment of the origin
It is not construed as limiting, such as the origin can be set at center of windshield etc..The first location information of eyeball tracking equipment exists
Eyeball tracking equipment can determine after connecting with vehicle.The first location information of eyeball tracking equipment and the second position of vehicle
Information can be indicated by a unified virtual coordinate system, can be determined according to the first location information and second location information
Vehicle coordinate transformation matrix.Predetermined vehicle coordinate transformation matrix can be stored in vehicle configuration parameter in the present embodiment
In, it directly acquires to obtain.Vehicle coordinate transformation matrix may include spin matrix D and translation matrix T, be this referring to Fig. 6, Fig. 6
The schematic diagram of spatial alternation in inventive embodiments three, wherein coordinate system (x1, y1, z1) is the coordinate system of eyeball tracking equipment,
Coordinate system (x2, y2, z2) is vehicle axis system, and the eye parameter information of the eyeball 14 received may include eyeball center
P and direction of visual lines vector V, the eye parameter information received are that the coordinate system based on eyeball tracking equipment indicates, coordinate
Transformation matrix is [D, the T] in figure.It is available based on vehicle by transformation matrix of coordinates and the eye parameter information received
The vehicle eye parameter information of coordinate system, especially by Transform (p, v)=[D | T] * (p, v)=p1, v1, wherein p1 is
Eyeball center based on vehicle axis system, v1 are the direction of visual lines vector based on vehicle axis system.
After obtaining the vehicle eye parameter information based on vehicle axis system, by solving available vehicle eye parameter
Direction of visual lines vector v 1 falls in the intersection point in compartment, i.e. blinkpunkt in information.Illustratively, it is assumed that direction of visual lines is fallen in compartment
Interactive flat normal vector be n1, be based on space geometry formula, available starting point is p1 and direction vector is the straight of v1
The intersection point of line and the interactive flat.If the intersection point is located on the display screen of vehicle, it can further obtain the intersection point and show
Pixel coordinate on screen.
It is understood that if the position that eyeball tracking equipment is reserved in the car in slot can be according to the demand of user
When within a preset range adjust up and down, multiple adjusting positions can be corresponded at this time, and each adjusting position is corresponding
Transformation matrix of coordinates is different, therefore each adjusting position stores corresponding vehicle coordinate transformation matrix, in order to it is subsequent more
Efficiently carry out eyeball tracking.
S330, basis watch information attentively and execute predetermined registration operation.
Wherein, predetermined registration operation can be set according to the actual situation.
Eyeball tracking method in the present embodiment does not require vehicle itself, is suitable for any required progress eyeball tracking
Vehicle, such as in vehicle whether include display screen there is no limit.When in vehicle including display screen, referring to fig. 2, however, it is determined that
Blinkpunkt is the A point in display screen 12, then A point can be marked and be shown by vehicle, see user.When including in vehicle
When display screen, predetermined registration operation can be realization click or selection etc. on a display screen in the prior art usually by being physically contacted in fact
Existing operation, specifically, the corresponding predetermined registration operation of the operating condition can be executed when watching information attentively and meeting operating condition.Behaviour
Making condition and predetermined registration operation can correspond, and specific operating condition, which can according to need, to be configured, such as operating condition
Preset time threshold can be greater than for fixation time, corresponding predetermined registration operation is to realize to correspond to current fixation point on a display screen
Position click.
When not including display screen in vehicle, due to vehicle configuration parameter it is known that when vehicle is sent out according to eyeball tracking equipment
The eye parameter information sent determines user's watching attentively after information inside the vehicle, that is, can determine that the blinkpunkt of user is particularly located at
Vehicle device in compartment.If the vehicle device where the blinkpunkt of user is identical as preset vehicle device, can be to this
Preset vehicle device executes predetermined registration operation.Wherein, predetermined registration operation can be set according to the actual situation, such as when preset
When vehicle device is the radio in compartment, then predetermined registration operation can be set to the operation such as start or close；For another example, when default
Vehicle device when being front windshield, then predetermined registration operation can be set to the behaviour that starting rain brush etc. is built in front windshield
Make.
In addition, the eye parameter information sent according to eyeball tracking equipment when vehicle determines user's watching attentively inside the vehicle
After information, if the direction of visual lines of user not in preset direction, can determine that user is in frazzle or spirit not
The state of concentration, predetermined registration operation can be that prompt tired out or warning note, or starting are carried out by the acoustic device in vehicle
The emergency braking measure of vehicle, to guarantee the traffic safety of user.Wherein preset direction can be road ahead direction or a left side
Right road direction.
Technical solution provided in this embodiment, vehicle receive the eye parameter that the eyeball tracking equipment being disposed therein is sent
Information, and according to eye parameter information determine user inside the vehicle watch information attentively, according to watch attentively information execute predetermined registration operation.
There is no limit greatly improve eyeball tracking equipment in the car to vehicle itself for eyeball tracking method in the present embodiment
Utilization rate further improves the safety of vehicle.
Example IV
Fig. 7 is the structural schematic diagram of the eyeball tracking device in the embodiment of the present invention four, and the present embodiment is applicable to realize
In vehicle the case where eyeball tracking.The embodiment of the present invention one and reality can be performed in eyeball tracking device provided by the embodiment of the present invention
Eyeball tracking method provided by example two is applied, has the corresponding functional module of execution method and beneficial effect.Eyeball tracking dress
Setting can be set in eyeball tracking equipment, which can be configured in vehicle.
The device specifically includes image collection module 410, information determination module 420 and information sending module 430, in which:
Image collection module 410, for obtaining the eyes image of user；
Information determination module 420, for determining the eye parameter information of user according to eyes image and target alignment coefficient；
Information sending module 430, for sending eye parameter information to vehicle so that vehicle according to eye parameter information with
And vehicle configuration parameter determine user inside the vehicle watch information attentively.
The embodiment of the present invention obtains the eyes image of user by the eyeball tracking equipment being set on vehicle, according to eye
Image and target alignment coefficient determine the eye parameter information of user；Eye parameter information is sent to vehicle so that vehicle according to
Eye parameter information and vehicle configuration parameter determine user inside the vehicle watch information attentively.The embodiment of the present invention passes through in vehicle
The portable eyeball tracking equipment of one kind is arranged on realizes the determination for watching information attentively, compared with the prior art, occupies little space,
With bigger convenience, scalability and robustness, and to vehicle itself, there is no limit greatly improve eyeball tracking
The utilization rate of equipment in the car, further improves the safety of vehicle.
Further, eyeball tracking device further include:
Calibration module before the eyes image for obtaining user, calibrates the calibration factor of initialization, obtains target alignment
Coefficient.
Further, calibration module is specifically used for:
By projection module according to preset target position information projection calibrating light, calibration mark is formed；
Eyes image and target position information when watching calibration mark attentively according to user to the calibration factor of initialization into
Row calibration, obtains target alignment coefficient.
Further, image collection module 410 is specifically used for:
User images are acquired by image capture module；
The eyes image in user images is extracted by eye detection algorithm.
Further, information determination module 420 includes:
Feature extraction unit, for extracting the eye feature information in eyes image；
Eye parameter information unit, for determining that eye parameter is believed according to eye feature information and target alignment coefficient
Breath, wherein eye parameter information includes eyeball center and direction of visual lines vector.
Further, target alignment coefficient includes corneal curvature radius, cornea aqueous humor refractive index, eye optical axis and the optical axis
At least one of the distance of angle and corneal curvature center to pupil center location.
The embodiment of the present invention one can be performed in eyeball tracking device provided by the embodiment of the present invention and embodiment two is provided
Eyeball tracking method, have the corresponding functional module of execution method and beneficial effect.
Embodiment five
Fig. 8 is the structural schematic diagram of the eyeball tracking device in the embodiment of the present invention five, and the present embodiment is applicable to realize
In vehicle the case where eyeball tracking.Eyeball tracking device provided by the embodiment of the present invention can be performed the embodiment of the present invention three and be mentioned
The eyeball tracking method of confession has the corresponding functional module of execution method and beneficial effect.The eyeball tracking device is set to vehicle
In, eyeball tracking equipment is configured in vehicle.
The device specifically includes information receiving module 510 and watches information module 520 attentively, in which:
Information receiving module 510, for receiving the eye parameter information of eyeball tracking equipment transmission；
Watch information module 520 attentively, the eye parameter information for sending according to eyeball tracking equipment determines user in vehicle
It is internal to watch information attentively, eye parameter information through the foregoing embodiment one and embodiment two in eyeball tracking method determine.
In the embodiment of the present invention, vehicle receives the eye parameter information that the eyeball tracking equipment being disposed therein is sent, and
According to eye parameter information determine user inside the vehicle watch information attentively.Vehicle itself is not limited in the embodiment of the present invention
System, greatly improves the utilization rate of eyeball tracking equipment in the car, further improves the safety of vehicle.
Further, watch information module 510 attentively to be specifically used for:
Obtain the first location information of eyeball tracking equipment and the second location information of vehicle；
According to first location information and vehicle configuration parameter, vehicle coordinate transition matrix is determined；
According to vehicle coordinate transition matrix and eye parameter information, the vehicle eye parameter information that obtains that treated；
Based on vehicle eye parameter information determine user inside the vehicle watch information attentively.
Further, which further includes；
Operation module, for according to eye parameter information determine user inside the vehicle watch information attentively after, according to note
Visual information executes predetermined registration operation.
Eyeball tracking provided by the embodiment of the present invention three can be performed in eyeball tracking device provided by the embodiment of the present invention
Method has the corresponding functional module of execution method and beneficial effect.
Embodiment six
Fig. 9 is the structural schematic diagram of the vehicle in the embodiment of the present invention six, and Fig. 9, which is shown, to be suitable for being used to realizing that the present invention is real
Apply the block diagram of the example vehicle of mode.The vehicle that Fig. 9 is shown is only an example, should not be to the function of the embodiment of the present invention
Any restrictions are brought with use scope.
As shown in figure 9, being configured with eyeball tracking equipment in the vehicle, it can include but is not limited to: one or more first
Processor 60, first storage device 61, first communication device 62, the first input unit 63 and the first output device 64.In figure with
For one first processor 60.The quantity of first storage device 61 can be one or more in the vehicle, with one in Fig. 9
For a first storage device 61.First processor 60, first storage device 61, the first communication device 62, first of the vehicle
Input unit 63 can be connected with the first output device 64 by bus or other modes, to be connected as by bus in Fig. 9
Example.
Eyeball tracking equipment in Fig. 9 can include but is not limited to: the one or more storage of second processor 70, second dresses
Set 71, secondary communication device 72, secondary input device 73 and the second output device 74.It is with a second processor 70 in figure
Example.The quantity of the second storage device 71 can be one or more in the vehicle, be with second storage device 71 in Fig. 9
Example.The second processor 70 of the vehicle, the second storage device 71, secondary communication device 72, secondary input device 73 and second are defeated
Device 74 can be connected by bus or other modes out, in Fig. 9 for being connected by bus.Furthermore the eyeball in figure chases after
Track equipment can also include projection module, communication module, image capture module and light source module, wherein projection module, for throwing
Penetrate calibration light；Communication module, for sending eye parameter information to vehicle；Image capture module, for acquiring user images；Light
Source module, for providing visible light source for projection device.
Vehicle can pass through the secondary communication device 72 of first communication device 62 and eyeball tracking equipment therein therein
Connection.The specific implementation composition of first communication device 62 and secondary communication device 72 can be set according to the actual situation, then
It is not construed as limiting in specific connection type the present embodiment.Such as first communication device 62 and secondary communication device 72 can pass through indigo plant
Tooth realizes connection.
First storage device 61 and the second storage device 71 are used as a kind of computer readable storage medium, can be used for storing soft
Part program, computer executable program and module, first storage device 61, for storing one or more first programs, the
Two storage devices 71 are for storing one or more second programs.First storage device 61 and the second storage device 71 can be wrapped mainly
Include storing program area and storage data area, wherein storing program area can application needed for storage program area, at least one function
Program；Storage data area, which can be stored, uses created data etc. according to equipment.In addition, first storage device 61 and second is deposited
Storage device 71 may include high-speed random access memory, can also include nonvolatile memory, for example, at least a disk
Memory device, flush memory device or other non-volatile solid state memory parts.In some instances, first storage device 61 and
Two storage devices 71 can further comprise being respectively relative to first processor 60 and the remotely located memory of second processor 70,
These remote memories can pass through network connection to equipment.The example of above-mentioned network includes but is not limited to internet, in enterprise
Portion's net, local area network, mobile radio communication and combinations thereof.
First input unit 63 and secondary input device 73 can be used for receiving the number or character information of input, and produce
Raw key signals input related with user setting and function control, can also be the camera for obtaining image.First is defeated
Device 64 and the second output device 74 may include the audio frequency apparatuses such as loudspeaker out.It should be noted that the first input unit 63,
The concrete composition of secondary input device 73, the first output device 64 and the second output device 7 may be set according to actual conditions.
The first program that first processor 60 is stored in first storage device 61 by operation, thereby executing various functions
Using and data processing, such as realize eyeball tracking method provided by the embodiment of the present invention three, be applied to vehicle, in vehicle
Configured with eyeball tracking equipment, comprising:
Receive the eye parameter information that eyeball tracking equipment is sent；
According to eye parameter information determine user inside the vehicle watch information attentively, eye parameter information is real through the invention
The eyeball tracking method for applying example one and embodiment two determines.
The second program that second processor 70 is stored in the second storage device 71 by operation, thereby executing various functions
Using and data processing, such as realize eyeball tracking method provided by the embodiment of the present invention one and embodiment two, be applied to
Eyeball tracking equipment, eyeball tracking device configuration is on vehicle, comprising:
Obtain the eyes image of user；
The eye parameter information of user is determined according to eyes image and target alignment coefficient；
Eye parameter information is sent to vehicle, is used so that vehicle is determined according to eye parameter information and vehicle configuration parameter
Family inside the vehicle watch information attentively.
Embodiment seven
The embodiment of the present invention seven additionally provides a kind of computer readable storage medium, is stored thereon with computer program, should
The eyeball tracking method as provided by the embodiment of the present invention one and embodiment two is realized when program is executed by processor, and/or, this
Eyeball tracking method provided by inventive embodiments three.
Wherein, eyeball tracking method provided by the embodiment of the present invention one and embodiment two is applied to eyeball tracking equipment,
Eyeball tracking device configuration is on vehicle, comprising:
Obtain the eyes image of user；
The eye parameter information of user is determined according to eyes image and target alignment coefficient；
Eye parameter information is sent to vehicle, is used so that vehicle is determined according to eye parameter information and vehicle configuration parameter
Family inside the vehicle watch information attentively.
Eyeball tracking method provided by the embodiment of the present invention three is applied to vehicle, sets in vehicle configured with eyeball tracking
It is standby, comprising:
Receive the eye parameter information that eyeball tracking equipment is sent；
According to eye parameter information determine user inside the vehicle watch information attentively, eye parameter information is real through the invention
The eyeball tracking method for applying example one and embodiment two determines.
The computer storage medium of the embodiment of the present invention, can be using any of one or more computer-readable media
Combination.Computer-readable medium can be computer-readable signal media or computer readable storage medium.It is computer-readable
Storage medium for example may be-but not limited to-the system of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, device or
Device, or any above combination.The more specific example (non exhaustive list) of computer readable storage medium includes: tool
There are electrical connection, the portable computer diskette, hard disk, random access memory (RAM), read-only memory of one or more conducting wires
(ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-
ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.In this document, computer-readable storage
Medium can be any tangible medium for including or store program, which can be commanded execution system, device or device
Using or it is in connection.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including --- but it is unlimited
In wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
It further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion
Divide and partially executes or executed on remote computer or terminal completely on the remote computer on the user computer.It is relating to
And in the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or extensively
Domain net (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as provided using Internet service
Quotient is connected by internet).
Note that the above is only a better embodiment of the present invention and the applied technical principle.It will be appreciated by those skilled in the art that
The invention is not limited to the specific embodiments described herein, be able to carry out for a person skilled in the art it is various it is apparent variation,
It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out by above embodiments to the present invention
It is described in further detail, but the present invention is not limited to the above embodiments only, without departing from the inventive concept, also
It may include more other equivalent embodiments, and the scope of the invention is determined by the scope of the appended claims.
Claims (14)
1. a kind of eyeball tracking method, which is characterized in that be applied to eyeball tracking equipment, the eyeball tracking device configuration is in vehicle
On, comprising:
Obtain the eyes image of user；
The eye parameter information of user is determined according to the eyes image and target alignment coefficient；
The eye parameter information is sent to the vehicle, so that the vehicle is matched according to the eye parameter information and vehicle
It sets parameter and determines that user watches information attentively in the vehicle interior.
2. the method according to claim 1, wherein before the eyes image of acquisition user, further includes:
The calibration factor for calibrating initialization, obtains target alignment coefficient.
3. according to the method described in claim 2, it is characterized in that, the calibration factor of calibration initialization, obtains target alignment system
Number, comprising:
By projection module according to preset target position information projection calibrating light, calibration mark is formed；
The eyes image calibrated when identifying and the target position information are watched attentively to the school of the initialization according to user
Quasi- coefficient is calibrated, and target alignment coefficient is obtained.
4. the method according to claim 1, wherein obtaining the eyes image of user, comprising:
User images are acquired by image capture module；
The eyes image in the user images is extracted by eye detection algorithm.
5. being used the method according to claim 1, wherein being determined according to the eyes image and target alignment coefficient
The eye parameter information at family, comprising:
Extract the eye feature information in the eyes image；
According to the eye feature information and the target alignment coefficient, the eye parameter information is determined, wherein the eye
Portion's parameter information includes eyeball center and direction of visual lines vector.
6. the method according to claim 1, wherein the target alignment coefficient includes corneal curvature radius, angle
Film aqueous humor refractive index, eye optical axis at a distance from the angle of the optical axis and corneal curvature center to pupil center location at least it
One.
7. the method according to claim 1, wherein the vehicle configuration parameter includes being located in the compartment extremely
The location parameter of a few vehicle device.
8. a kind of eyeball tracking method, which is characterized in that be applied to vehicle, eyeball tracking equipment, packet are configured in the vehicle
It includes:
Receive the eye parameter information that the eyeball tracking equipment is sent；
User is determined in the information of watching attentively of the vehicle interior according to the eye parameter information, and the eye parameter information passes through
Any eyeball tracking method determines in claim 1-7.
9. according to the method described in claim 8, it is characterized in that, determining user in the vehicle according to the eye parameter information
Inside watches information attentively, comprising:
Obtain the first location information of the eyeball tracking equipment and the second location information of the vehicle；
According to the first location information and the second location information, vehicle coordinate transition matrix is determined；
According to the vehicle coordinate transition matrix and the eye parameter information, the vehicle eye parameter information that obtains that treated；
Determine that user watches information attentively in the vehicle interior based on the vehicle eye parameter information.
10. according to the method described in claim 8, it is characterized in that, determining user described according to the eye parameter information
Vehicle interior is watched attentively after information, further includes:
Information execution predetermined registration operation is watched attentively according to described.
11. a kind of eyeball tracking device, which is characterized in that be set to eyeball tracking equipment, the eyeball tracking device configuration in
On vehicle, comprising:
Image collection module, for obtaining the eyes image of user；
Information determination module, for determining the eye parameter information of user according to the eyes image and target alignment coefficient；
Information sending module, for sending the eye parameter information to the vehicle, so that the vehicle is according to the eye
Parameter information and vehicle configuration parameter determine user inside the vehicle watch information attentively.
12. a kind of eyeball tracking device, which is characterized in that be set to vehicle, eyeball tracking equipment, packet are configured in the vehicle
It includes:
Information receiving module, the eye parameter information sent for receiving the eyeball tracking equipment；
Information module is watched attentively, for determining user in the information of watching attentively of the vehicle interior, institute according to the eye parameter information
Eye parameter information is stated to determine by the eyeball tracking method any in claim 1-7.
13. a kind of vehicle, which is characterized in that be configured with eyeball tracking equipment in the vehicle, comprising:
One or more first processors；
First storage device, for storing one or more first programs；
When one or more of first programs are executed by one or more of first processors, so that one or more of
First processor realizes the eyeball tracking method as described in any in claim 8-10；
Wherein, the eyeball tracking equipment includes:
One or more second processors；
Projection module is used for projection calibrating light；
Communication module, for sending eye parameter information to vehicle；
Image capture module, for acquiring user images；
Light source module, for providing visible light source for the projection device；
Second storage device, for storing one or more second programs；
When one or more of second programs are executed by one or more of second processors, so that one or more of
Second processor realizes the eyeball tracking method as described in any in claim 1-7.
14. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
Realized when execution it is any in eyeball tracking method as described in any in claim 1-7 and/or claim 8-10 as described in
Eyeball tracking method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910611395.1A CN110341617B (en) | 2019-07-08 | 2019-07-08 | Eyeball tracking method, device, vehicle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910611395.1A CN110341617B (en) | 2019-07-08 | 2019-07-08 | Eyeball tracking method, device, vehicle and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110341617A true CN110341617A (en) | 2019-10-18 |
| CN110341617B CN110341617B (en) | 2021-05-28 |
Family
ID=68178450
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910611395.1A Active CN110341617B (en) | 2019-07-08 | 2019-07-08 | Eyeball tracking method, device, vehicle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110341617B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111159678A (en) * | 2019-12-26 | 2020-05-15 | 联想(北京)有限公司 | Identity recognition method, device and storage medium |
| CN111857333A (en) * | 2020-06-29 | 2020-10-30 | 维沃移动通信有限公司 | Eye movement tracking method and device and electronic equipment |
| CN112232255A (en) * | 2020-10-26 | 2021-01-15 | 上海鹰瞳医疗科技有限公司 | Identity authentication method and equipment based on multi-mode data |
| WO2021140069A1 (en) * | 2020-01-09 | 2021-07-15 | Volkswagen Aktiengesellschaft | Method and device for estimating an eye position of a driver of a vehicle |
| WO2023272453A1 (en) * | 2021-06-28 | 2023-01-05 | 华为技术有限公司 | Gaze calibration method and apparatus, device, computer-readable storage medium, system, and vehicle |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106203297A (en) * | 2016-06-30 | 2016-12-07 | 北京七鑫易维信息技术有限公司 | A kind of personal identification method and device |
| WO2018006019A1 (en) * | 2016-07-01 | 2018-01-04 | Itay Katz | System and method for driver monitoring |
| CN109542217A (en) * | 2018-10-12 | 2019-03-29 | 深圳市元征科技股份有限公司 | A kind of eyeball tracking display methods and device |
| CN109558012A (en) * | 2018-12-26 | 2019-04-02 | 北京七鑫易维信息技术有限公司 | Eyeball tracking method and device |
| CN109933205A (en) * | 2019-03-25 | 2019-06-25 | 一汽轿车股份有限公司 | A kind of vehicle-mounted expression in the eyes interactive device |
-
2019
- 2019-07-08 CN CN201910611395.1A patent/CN110341617B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106203297A (en) * | 2016-06-30 | 2016-12-07 | 北京七鑫易维信息技术有限公司 | A kind of personal identification method and device |
| WO2018006019A1 (en) * | 2016-07-01 | 2018-01-04 | Itay Katz | System and method for driver monitoring |
| CN109542217A (en) * | 2018-10-12 | 2019-03-29 | 深圳市元征科技股份有限公司 | A kind of eyeball tracking display methods and device |
| CN109558012A (en) * | 2018-12-26 | 2019-04-02 | 北京七鑫易维信息技术有限公司 | Eyeball tracking method and device |
| CN109933205A (en) * | 2019-03-25 | 2019-06-25 | 一汽轿车股份有限公司 | A kind of vehicle-mounted expression in the eyes interactive device |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111159678A (en) * | 2019-12-26 | 2020-05-15 | 联想(北京)有限公司 | Identity recognition method, device and storage medium |
| CN111159678B (en) * | 2019-12-26 | 2023-08-18 | 联想(北京)有限公司 | Identity recognition method, device and storage medium |
| WO2021140069A1 (en) * | 2020-01-09 | 2021-07-15 | Volkswagen Aktiengesellschaft | Method and device for estimating an eye position of a driver of a vehicle |
| CN111857333A (en) * | 2020-06-29 | 2020-10-30 | 维沃移动通信有限公司 | Eye movement tracking method and device and electronic equipment |
| CN111857333B (en) * | 2020-06-29 | 2022-09-13 | 维沃移动通信有限公司 | Eye movement tracking method and device and electronic equipment |
| CN112232255A (en) * | 2020-10-26 | 2021-01-15 | 上海鹰瞳医疗科技有限公司 | Identity authentication method and equipment based on multi-mode data |
| CN112232255B (en) * | 2020-10-26 | 2022-11-01 | 北京鹰瞳科技发展股份有限公司 | Identity authentication method and equipment based on multi-mode data |
| WO2023272453A1 (en) * | 2021-06-28 | 2023-01-05 | 华为技术有限公司 | Gaze calibration method and apparatus, device, computer-readable storage medium, system, and vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110341617B (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110341617A (en) | Eyeball tracking method, apparatus, vehicle and storage medium | |
| CN106056092B (en) | The gaze estimation method for headset equipment based on iris and pupil | |
| CN110032278B (en) | Pose identification method, device and system for human eye interested object | |
| CN109917920A (en) | Vehicle-mounted projection processing method, device, mobile unit and storage medium | |
| CN110703904B (en) | Visual line tracking-based augmented virtual reality projection method and system | |
| CN110009720B (en) | Image processing method and device in AR scene, electronic equipment and storage medium | |
| CN109889807A (en) | Vehicle-mounted projection adjusting method, device, equipment and storage medium | |
| CN108171673A (en) | Image processing method, device, vehicle-mounted head-up-display system and vehicle | |
| CN112102389A (en) | Method and system for determining spatial coordinates of a 3D reconstruction of at least a part of a physical object | |
| CN113808160B (en) | Sight direction tracking method and device | |
| CN109177922A (en) | Vehicle starting method, device, equipment and storage medium | |
| WO2023272453A1 (en) | Gaze calibration method and apparatus, device, computer-readable storage medium, system, and vehicle | |
| CN103366157A (en) | Method for judging line-of-sight distance of human eye | |
| CN110427101A (en) | Calibration method, device, equipment and the storage medium of eyeball tracking | |
| CN111854620B (en) | Monocular camera-based actual pupil distance measuring method, device and equipment | |
| CN109968979A (en) | Vehicle-mounted projection processing method, device, mobile unit and storage medium | |
| CN109670456A (en) | A kind of content delivery method, device, terminal and storage medium | |
| CN109635761A (en) | A kind of iris recognition image determines method, apparatus, terminal device and storage medium | |
| CN110956068B (en) | Fatigue detection method and device based on human eye state recognition | |
| CN108510542B (en) | Method and device for matching light source and light spot | |
| CN113597616A (en) | Pupil position determination method, device and system | |
| CN115223231A (en) | Sight direction detection method and device | |
| CN113325947A (en) | Display method, display device, terminal equipment and storage medium | |
| CN112651270A (en) | Gaze information determination method and apparatus, terminal device and display object | |
| CN114290998B (en) | Skylight display control device, method and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |