CN107644443B - Parameter setting method and device in sight tracking equipment - Google Patents
Parameter setting method and device in sight tracking equipment Download PDFInfo
- Publication number
- CN107644443B CN107644443B CN201710784884.8A CN201710784884A CN107644443B CN 107644443 B CN107644443 B CN 107644443B CN 201710784884 A CN201710784884 A CN 201710784884A CN 107644443 B CN107644443 B CN 107644443B
- Authority
- CN
- China
- Prior art keywords
- camera
- cornea
- light source
- light
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a parameter setting method and device for sight tracking equipment. Wherein the gaze tracking device comprises a first camera and a second camera, the method comprising: presetting the position of a first camera as a first position, and constructing an optimization objective equation by taking the curvature center coordinate of the cornea and the position of a second camera as variables; and determining the position of the corresponding second camera when the optimization target equation value is maximum to obtain a second position. The invention solves the technical problem that the matching of light source spots cannot be finished due to improper camera position of the sight tracking equipment in the prior art.
Description
Technical Field
The invention relates to the field of sight tracking equipment, in particular to a parameter setting method and device in the sight tracking equipment.
Background
In the prior art, when a sight tracking device is used, a VR device carries out sight estimation on a remote device of a fixation point according to pupil center coordinates and corneal reflection based on a 3D approximate spherical ball model of eyeballs, when the VR device uses a plurality of cameras and a plurality of light sources, sight can be estimated only through a single-point correction process, however, the method does not provide a scheme for designing the position of a camera, and in practical use of the VR device, because the relative positions of the light sources and the cameras are different, some position cameras cannot capture images or captured images are not good, and the light sources often do not have specificity, so that matching of light source spots cannot be completed when the cameras cannot capture images or the captured images are not good, and further sight direction cannot be determined.
Aiming at the problem that the camera position of the sight tracking equipment in the prior art is improper, so that the matching of light source light spots cannot be completed, an effective solution is not provided at present.
Disclosure of Invention
The embodiment of the invention provides a parameter setting method and device for sight tracking equipment, which at least solve the technical problem that the matching of light source spots cannot be finished due to the fact that the camera position of the sight tracking equipment is not appropriate in the prior art.
According to an aspect of an embodiment of the present invention, there is provided a parameter setting method in a gaze tracking apparatus including a first camera and a second camera, the method including: presetting the position of a first camera as a first position, and constructing an optimization objective equation by taking the curvature center coordinate of a cornea and the position of a second camera as variables, wherein a plurality of light sources are projected on the first camera and the second camera after being reflected by the cornea to obtain light spots, the curvature center coordinate of the cornea is changed in a preset empirical range, and the optimization objective equation comprises the following steps: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; and determining the position of the corresponding second camera when the optimization target equation value is maximum to obtain a second position.
Further, after obtaining the second position, the method further includes: determining the projection shapes of all the light spots on the second camera according to the second position to obtain a first projection shape; carrying out position difference matching on the first projection shape and a preset second projection shape to obtain a position difference matching result; and matching the light spot and the light source according to the position difference matching result to obtain a light spot and light source matching result.
Further, before the position difference matching is performed on the first projection shape and the preset second projection shape, the method further includes: determining a second projection shape; wherein determining the second projection shape comprises: presetting the position of a second camera as a third position; and determining the projection shapes of the reflection points of the plurality of light sources on the cornea on a plane parallel to the image plane of the second camera according to the third position to obtain a second projection shape.
Further, after obtaining the matching result of the light spot light source, the method further includes: determining the curvature center estimation coordinate of the cornea according to the light spot light source matching result; determining the position of the pupil center through the refraction process of the pupil center; determining the direction of the optical axis according to the estimated coordinates of the curvature center and the position of the pupil center; and determining the sight line direction according to the optical axis direction.
Further, the plurality of light sources are positioned such that: the plurality of light sources form a ring.
According to another aspect of the embodiments of the present invention, there is also provided a parameter setting apparatus in a gaze tracking device, the gaze tracking device including a first camera and a second camera, the apparatus including: the system comprises a construction module and an optimization objective equation, wherein the construction module is used for presetting the position of a first camera as a first position, and constructing the optimization objective equation by taking the curvature center coordinate of a cornea and the position of a second camera as variables, wherein a plurality of light sources are projected on the first camera and the second camera after being reflected by the cornea to obtain light spots, the curvature center coordinate of the cornea is changed in a preset empirical range, and the optimization objective equation comprises the following steps: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; and the first determining module is used for determining the position of the corresponding second camera when the optimization objective equation value is maximum to obtain a second position.
According to another aspect of the embodiments of the present invention, there is also provided a storage medium including a stored program, wherein the apparatus in which the storage medium is located is controlled to execute the parameter setting method in the gaze tracking apparatus described above when the program runs.
According to another aspect of the embodiments of the present invention, there is also provided a processor for executing a program, where the program executes the parameter setting method in the gaze tracking apparatus.
According to another aspect of the embodiments of the present invention, there is also provided a terminal including a gaze tracking device including a first camera and a second camera, the terminal further including: the system comprises a construction module and an optimization objective equation, wherein the construction module is used for presetting the position of a first camera as a first position, and constructing the optimization objective equation by taking the curvature center coordinate of a cornea and the position of a second camera as variables, wherein a plurality of light sources are projected on the first camera and the second camera after being reflected by the cornea to obtain light spots, the curvature center coordinate of the cornea is changed in a preset empirical range, and the optimization objective equation comprises the following steps: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; the first determining module is used for determining the position of the corresponding second camera when the optimization target equation value is maximum to obtain a second position; and a processor that runs a program, wherein the program runs to execute the parameter setting method in the gaze tracking apparatus described above with respect to the data output from the construction module and the first determination module.
According to another aspect of the embodiments of the present invention, there is also provided a terminal including a gaze tracking device including a first camera and a second camera, the terminal further including: the system comprises a construction module and an optimization objective equation, wherein the construction module is used for presetting the position of a first camera as a first position, and constructing the optimization objective equation by taking the curvature center coordinate of a cornea and the position of a second camera as variables, wherein a plurality of light sources are projected on the first camera and the second camera after being reflected by the cornea to obtain light spots, the curvature center coordinate of the cornea is changed in a preset empirical range, and the optimization objective equation comprises the following steps: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; the first determining module is used for determining the position of the corresponding second camera when the optimization target equation value is maximum to obtain a second position; a storage medium for storing a program, wherein the program executes the above-described parameter setting method in the gaze tracking apparatus on data output from the construction module and the first determination module when running.
In an embodiment of the present invention, for a device including a first camera and a second camera for tracking a line of sight, an optimization objective equation is constructed by presetting a position of the first camera as a first position and using a center of curvature coordinate of a cornea and a position of the second camera as variables, wherein a plurality of light sources are projected on the first camera and the second camera after being reflected by the cornea to obtain a light spot, the center of curvature coordinate of the cornea changes within a preset empirical range, and the optimization objective equation includes: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; the position of the corresponding second camera when the optimization target equation value is maximum is determined, the second position is obtained, and the purpose of determining the optimal position of the second camera is achieved, so that the technical effect that the light spot distribution with the maximum specificity is convenient for matching the light source light spots is achieved, and the technical problem that the matching of the light source light spots cannot be completed due to the fact that the camera position of the sight tracking equipment is not appropriate in the prior art is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a schematic diagram illustrating a parameter setting method in a gaze tracking apparatus according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an alternative method for setting parameters in a gaze tracking device, according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating an alternative method for setting parameters in a gaze tracking device, according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an alternative method for setting parameters in a gaze tracking device, according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating an alternative method for setting parameters in a gaze tracking device, according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating an alternative method for setting parameters in a gaze tracking device, in accordance with an embodiment of the present invention; and
fig. 7 is a schematic diagram of a parameter setting device in a gaze tracking apparatus according to an embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
In accordance with an embodiment of the present invention, there is provided a method embodiment of a parameter setting method in a gaze tracking device, it being noted that the steps illustrated in the flowchart of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Fig. 1 is a parameter setting method in a gaze tracking apparatus including a first camera and a second camera according to an embodiment of the present invention, as shown in fig. 1, the method including the steps of:
step S102, presetting the position of a first camera as a first position, and constructing an optimization objective equation by using the curvature center coordinate of the cornea and the position of a second camera as variables, wherein a plurality of light sources are projected on the first camera and the second camera after being reflected by the cornea to obtain light spots, the curvature center coordinate of the cornea is changed in a preset empirical range, and the optimization objective equation comprises: the entropy of the light spot distribution corresponding to the first camera and the second camera is the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source and the change of the curvature center coordinate of the cornea.
Optionally, the gaze tracking device in all embodiments of the present invention includes, but is not limited to, a virtual reality device, and a smart terminal capable of performing gaze tracking, for example, a mobile phone, a computer, a wearable device, and the like.
Specifically, as shown in fig. 2, fig. 2 is a schematic diagram of an eyeball, a light source and a camera, wherein, the left camera or the right camera in fig. 2 may be a first camera, correspondingly, the right camera or the left camera may be a second camera, O1 represents the position of the left camera, O2 represents the position of the right camera, the first camera may be first given a certain position, that is, the position of the first camera is preset as a first position, so that the position of the first camera is known, optionally, for convenience of calculation, the first position may be an origin coordinate, that is, the coordinate of the first camera is [0,0,0], I1 and I2 in fig. 2 represent the coordinates of the light source 1 and the light source 2, respectively, only two light sources are shown in fig. 2, it should be noted that, the number of the light sources is not limited in the present invention, after the light sources are reflected by the cornea of the eyeball, a light spot is projected on the image plane of the first camera, and projecting and obtaining a light spot on an image plane of the second camera, and under the condition that the specificity of the light spot in the two cameras is maximum, matching between the light spot and the light source can be realized in a mode of performing difference matching between the shape of the light spot in the cameras and the preset shape of a preset known light source, and the like, so that under the condition that the position of the first camera is known, the position which can enable the specificity of the light spot in the second camera to be maximum can be selected as the position of the second camera, and when the specificity of the light spot in the second camera is measured, measurement is performed simultaneously according to the distribution of the light spot in the cameras and the difference of the positions of the light spot in the two cameras, so that the constructed optimization objective equation comprises: specifically, in combination with fig. 2, in the case where the left camera is the first camera and the right camera is the second camera, the optimization objective equation may be as follows:
wherein, H (q)ij)＝-p(qij)·logp(qij),j∈{1,2}；
Where C denotes a curvature center coordinate of the cornea, C denotes a preset empirical range of C, i denotes a reference number of the light sources, N is the total number of the light sources, i is 1,2, …, N, λ denotes an empirical coefficient, j denotes a camera reference number, j denotes a first camera when j is 1, j denotes a second camera when j is 2, q is a predetermined empirical range of C, and j denotes a predetermined number of the light sourcesijIndicating the reflection point of the light source, referenced i, on the cornea for the jth camera,
the centre, i.e. the average position, of the spot in the j-th camera is indicated.
And step S104, determining the position of the corresponding second camera when the optimization target equation value is maximum, and obtaining a second position.
Specifically, the larger the entropy of the distribution of the light spots in the two cameras is, that is, the more uneven the distribution of the light spots is, the larger the difference in the positions of the light spots in the two cameras is, that is, the larger the difference in the positions of the light spots in the two cameras is, so that in order to satisfy the specificity of the light spots in the second camera, the entropy of the distribution of the light spots in the two cameras can be maximized and the difference in the positions of the light spots in the two cameras can be maximized, specifically, in the optimization objective equation given above, when the curvature center c of the cornea is changed in an empirical range, that is, under the condition of different positions of the curvature center c of the cornea, by searching all possible second camera positions O2, the corresponding position of the second camera when the value of the optimization objective equation is maximized can be obtained, that is, when the objective function value in formula (1) is maximized, the position of the second camera at this time.
In an embodiment of the present invention, for a device including a first camera and a second camera for tracking a line of sight, an optimization objective equation is constructed by presetting a position of the first camera as a first position and using a center of curvature coordinate of a cornea and a position of the second camera as variables, wherein a plurality of light sources are projected on the first camera and the second camera after being reflected by the cornea to obtain a light spot, the center of curvature coordinate of the cornea changes within a preset empirical range, and the optimization objective equation includes: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; the position of the corresponding second camera when the optimization target equation value is maximum is determined, the second position is obtained, and the purpose of determining the optimal position of the second camera is achieved, so that the technical effect that the light spot distribution with the maximum specificity is convenient for matching the light source light spots is achieved, and the technical problem that the matching of the light source light spots cannot be completed due to the fact that the camera position of the sight tracking equipment is not appropriate in the prior art is solved.
In an optional embodiment, after obtaining the second position in step S104, the method further includes:
step S202, determining the projection shapes of all the light spots on the second camera according to the second position to obtain a first projection shape;
step S204, carrying out position difference matching on the first projection shape and a preset second projection shape to obtain a position difference matching result;
and S206, matching the light spot and the light source according to the position difference matching result to obtain a light spot and light source matching result.
Specifically, after the second position of the second camera is determined, the projection shapes of all the light spots on the second camera, that is, the first projection shape, may be determined, the preset second projection shape, that is, the projection shapes of all the light spots on the second camera, that is, the pre-estimated projection shapes of all the light spots, and the matching relationship between the light spots on the second projection shape or other points representing the cursor and the light source is known, and by performing position difference matching on the first projection shape and the second projection shape, the light source corresponding to the light spots in the first projection shape may be matched according to the position difference matching result, and finally, the light spot light source matching result is obtained.
In an optional embodiment, before the performing the position difference matching on the first projection shape and the preset second projection shape in step S204, the method further includes:
step S302, determining a second projection shape; wherein the determining the second projection shape in step S302 includes:
step S402, presetting the position of a second camera as a third position;
step S404, determining the projection shapes of the reflection points of the plurality of light sources on the cornea on the plane parallel to the image plane of the second camera according to the third position, and obtaining a second projection shape.
Specifically, before performing the position difference matching on the first projection shape and the second projection shape, the second projection shape may be determined through the above steps S402 to S404, specifically, the positions of the first camera and the second camera may be given, after the positions of the two cameras are determined, the projection shape of the reflection point on the cornea based on the light sources of the two cameras on the plane parallel to the image plane of the cameras, that is, the second projection shape may be obtained, the light source corresponding to the point of the projection of the reflection point on the second projection shape is determined, by performing the position difference matching on the first projection shape and the second projection shape, the projection point of the reflection point on the second projection shape corresponding to the projection point of the spot on the first projection shape may be matched, and since the light source corresponding to the projection point of the reflection point is known, the light source corresponding to the projection point of the spot may be determined, thereby obtaining the light spot light source matching result.
Specifically, the coordinates of the reflection point can be obtained by the following formula:
||qij-c||＝R；
(li-oj)×(qij-oj)·(c-oj)＝0；
(li-qij)·(qij-c)·||oj-qij||＝(oj-qij)·(qij-c)·||li-qij||； (2)
where R denotes the radius of curvature of the cornea, Ii denotes the coordinates of the light source denoted by i, and Oj denotes the coordinates of the camera denoted by j.
Specifically, when the position of the second camera corresponding to the maximum optimization objective equation is determined in step S104, and the second position is obtained, under the condition that the curvature center c, the position O1 of the first camera, and the position O2 of the second camera are determined, the corresponding q can be obtained through the above formula (2)ijThen, the values of the optimization objective equation corresponding to all the second camera positions are obtained by substituting the values into the above equation (1), and O2 when the optimization objective equation is the maximum value is the final second camera position.
In a specific embodiment, in the case where a plurality of light sources constitute a ring light source, if the ring light source initial position coordinate I0 is [ -7.8180, 3.4020, 44.3486], c is [ (0.4, 0, 8), and R is 8, assuming that the left camera coordinate in fig. 2 is [0,0,0], the right camera coordinate is [ -16.0348, 0.0347, 0.2610], and the number of light sources is 180, the projection pattern formed by the reflection point of the light source on the cornea on the plane parallel to the image plane of the left camera is shown in fig. 3, the projection pattern formed by the reflection point of the light source on the cornea on the plane parallel to the image plane of the right camera is shown in fig. 4, when the number of the light sources is 8, the projection pattern of the reflection point of the light source on the cornea on the plane parallel to the image plane of the left camera is shown in fig. 5, and the projection pattern of the reflection point of the light source on the cornea on the plane parallel to the image plane of the right camera is shown in fig. 6.
In an optional embodiment, after obtaining the spot light source matching result in step S206, the method further includes:
step S502, determining the curvature center estimation coordinate of the cornea according to the light spot light source matching result;
step S504, determining the position of the pupil center through the refraction process of the pupil center;
step S506, determining the direction of the optical axis according to the estimated coordinates of the curvature center and the position of the pupil center;
step S508, determining the direction of the line of sight according to the direction of the optical axis.
Specifically, after the light spot light source matching result is obtained, the distribution condition of the light spots under the condition of different cornea curvature center coordinates c can be reversely verified, the light spot distribution condition under the second position of the second camera is compared, the cornea curvature center coordinate corresponding to the light spot distribution condition with the highest similarity is selected, namely the cornea curvature center estimated coordinate, then the pupil center position p is obtained through the refraction process of the pupil center, so that the cp direction, namely the primary optical axis direction is obtained, and the sight line direction can be obtained through correcting the kappa angle of the optical axis direction.
In an alternative embodiment, the plurality of light sources are positioned such that: the plurality of light sources form a ring.
Specifically, in order to facilitate the matching of the first projection shape and the second projection shape, the plurality of light sources may form a closed pattern, preferably a ring shape.
Example 2
According to an embodiment of the present invention, there is provided a product embodiment of a parameter setting device in a gaze tracking apparatus, fig. 7 is the parameter setting device in the gaze tracking apparatus according to the embodiment of the present invention, the gaze tracking apparatus includes a first camera and a second camera, as shown in fig. 7, the device includes a construction module and a first determination module, wherein the construction module is configured to preset a position of the first camera as a first position, and construct an optimization objective equation using a curvature center coordinate of a cornea and a position of the second camera as variables, wherein a plurality of light sources are reflected by the cornea and then projected on the first camera and the second camera to obtain a light spot, the curvature center coordinate of the cornea changes within a preset empirical range, and the optimization objective equation includes: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; and the first determining module is used for determining the position of the corresponding second camera when the optimization objective equation value is maximum to obtain a second position.
In an embodiment of the present invention, for a device including a first camera and a second camera for tracking a line of sight, a position of the first camera is preset as a first position by a construction module, and an optimization objective equation is constructed by taking a curvature center coordinate of a cornea and a position of the second camera as variables, wherein a plurality of light sources are reflected by the cornea and then projected on the first camera and the second camera to obtain a light spot, the curvature center coordinate of the cornea changes within a preset empirical range, and the optimization objective equation includes: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; the first determining module determines the position of the corresponding second camera when the optimization target equation value is maximum to obtain the second position, and the purpose of determining the optimal position of the second camera is achieved, so that the technical effect that the light spot distribution with the maximum specificity is convenient for light source light spot matching is achieved, and the technical problem that the matching of the light source light spots cannot be completed due to the fact that the camera position of the sight tracking equipment is not appropriate in the prior art is solved.
It should be noted here that the above building module, the first determining module and the calculating module correspond to steps S102 to S104 in embodiment 1, and the above modules are the same as the examples and application scenarios realized by the corresponding steps, but are not limited to the disclosure of embodiment 1. It should be noted that the modules described above as part of an apparatus may be implemented in a computer system such as a set of computer-executable instructions.
In an optional embodiment, the apparatus further includes a second determining module, a first matching module, and a second matching module, where the second determining module is configured to determine projection shapes of all the light spots on the second camera according to the second position after the first determining module obtains the second position, so as to obtain the first projection shape; the first matching module is used for carrying out position difference matching on the first projection shape and a preset second projection shape to obtain a position difference matching result; and the second matching module is used for matching the light spot and the light source according to the position difference matching result to obtain a light spot and light source matching result.
It should be noted here that the second determining module, the first matching module and the second matching module correspond to steps S202 to S206 in embodiment 1, and the modules are the same as the corresponding steps in the implementation example and application scenarios, but are not limited to the disclosure in embodiment 1. It should be noted that the modules described above as part of an apparatus may be implemented in a computer system such as a set of computer-executable instructions.
In an optional embodiment, the apparatus further includes a third determining module, configured to determine the second projection shape before the first matching module performs position difference matching on the first projection shape and a preset second projection shape; the third determining module comprises a presetting module and a fourth determining module, wherein the presetting module is used for presetting the position of the second camera as a third position; and the fourth determining module is used for determining the projection shapes of the reflection points of the plurality of light sources on the cornea on a plane parallel to the image plane of the second camera according to the third position to obtain a second projection shape.
It should be noted here that the third determining module, the preset module and the fourth determining module correspond to step S302 and step S402 to step S404 in embodiment 1, and the modules are the same as the examples and application scenarios realized by the corresponding steps, but are not limited to the disclosure in embodiment 1. It should be noted that the modules described above as part of an apparatus may be implemented in a computer system such as a set of computer-executable instructions.
In an optional embodiment, the apparatus further comprises a fifth determining module, a sixth determining module, a seventh determining module and an eighth determining module, wherein the fifth determining module is configured to determine the estimated curvature center coordinates of the cornea according to the matching result of the light spot light source after the matching result of the light spot light source is obtained by the second matching module; a sixth determining module, configured to determine a position of a pupil center through a refraction process of the pupil center; a seventh determining module, configured to determine the optical axis direction according to the curvature center estimation coordinate and the pupil center position; and the eighth determining module is used for determining the sight line direction according to the optical axis direction.
It should be noted here that the fifth determining module, the sixth determining module, the seventh determining module and the eighth determining module correspond to steps S502 to S508 in embodiment 1, and the modules are the same as the corresponding steps in the implementation example and application scenarios, but are not limited to the disclosure in embodiment 1. It should be noted that the modules described above as part of an apparatus may be implemented in a computer system such as a set of computer-executable instructions.
In an alternative embodiment, the plurality of light sources are positioned such that: the plurality of light sources form a ring.
Example 3
According to an embodiment of the present invention, there is provided an article of manufacture of a storage medium including a stored program, wherein the program, when executed, controls an apparatus in which the storage medium is located to execute the parameter setting method in the gaze tracking apparatus.
Example 4
According to an embodiment of the present invention, an article of manufacture of a processor for executing a program is provided, where the program executes the parameter setting method in the gaze tracking apparatus.
Example 5
According to an embodiment of the present invention, there is provided a product embodiment of a terminal, where the terminal includes a line-of-sight tracking device, the line-of-sight tracking device includes a first camera and a second camera, and the terminal further includes a construction module, a first determination module, and a processor, where the construction module is configured to preset a position of the first camera as a first position, and to construct an optimization objective equation using a center of curvature coordinate of a cornea and a position of the second camera as variables, where a plurality of light sources are reflected by the cornea and then projected on the first camera and the second camera to obtain a light spot, the center of curvature coordinate of the cornea changes within a preset empirical range, and the optimization objective equation includes: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; the first determining module is used for determining the position of the corresponding second camera when the optimization target equation value is maximum to obtain a second position; and a processor that runs a program, wherein the program runs to execute the parameter setting method in the gaze tracking apparatus described above with respect to the data output from the construction module and the first determination module.
Example 6
According to an embodiment of the present invention, there is provided a product embodiment of a terminal, where the terminal includes a line-of-sight tracking device, the line-of-sight tracking device includes a first camera and a second camera, and the terminal further includes a construction module, a first determination module, and a storage medium, where the construction module is configured to preset a position of the first camera as a first position, and to construct an optimization objective equation using a center of curvature coordinate of a cornea and a position of the second camera as variables, where a plurality of light sources are reflected by the cornea and then projected on the first camera and the second camera to obtain a light spot, the center of curvature coordinate of the cornea changes within a preset empirical range, and the optimization objective equation includes: the entropy of the light spot distribution corresponding to the first camera and the second camera and the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source change along with the curvature center coordinate of the cornea; the first determining module is used for determining the position of the corresponding second camera when the optimization target equation value is maximum to obtain a second position; a storage medium for storing a program, wherein the program executes the above-described parameter setting method in the gaze tracking apparatus on data output from the construction module and the first determination module when running.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method of parameter setting in a gaze tracking apparatus, the gaze tracking apparatus including a first camera and a second camera, the method comprising:
presetting the position of the first camera as a first position, and constructing an optimization objective equation by taking the curvature center coordinate of a cornea and the position of the second camera as variables, wherein a plurality of light sources are projected on the first camera and the second camera after being reflected by the cornea to obtain light spots, the curvature center coordinate of the cornea is changed in a preset empirical range, and the optimization objective equation comprises the following steps: the entropy of the light spot distribution corresponding to the first camera and the second camera is the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source and the change of the curvature center coordinate of the cornea;
determining the position of the second camera corresponding to the maximum optimized objective equation value to obtain a second position;
the optimization objective equation is as follows:
wherein, H (q)ij)＝-p(qij)·logp(qij)，j∈{1，2}；
C is the center of curvature coordinates of the cornea, C isc, i is the label of the light source, N is the total number of the light sources, i is 1,2, …, N, λ is the empirical coefficient, j is the camera label, q is the empirical rangeijFor the j camera, the reflection point of the light source, labeled i, on the cornea,
is the center of the spot in the j-th camera;
obtaining the coordinates of the reflection point by the following formula:
||qij-c||＝R，
(Ii-oj)×(qij-oj)·(c-oj)＝0，
(Ii-qij)·(qij-c)·||oj-qij||＝(oj-qij)·(qij-c)·||Ii-qij||，
wherein R is the radius of curvature of the cornea, IiCoordinates of a light source denoted by i, ojThe coordinates of the camera, referenced j.
2. The method of claim 1, wherein after obtaining the second location, the method further comprises:
determining the projection shapes of all the light spots on the second camera according to the second positions to obtain a first projection shape;
carrying out position difference matching on the first projection shape and a preset second projection shape to obtain a position difference matching result;
and matching the light spot and the light source according to the position difference matching result to obtain a light spot and light source matching result.
3. The method of claim 2, wherein prior to the location difference matching the first projected shape with a preset second projected shape, the method further comprises:
determining the second projection shape; wherein determining the second projection shape comprises:
presetting the position of the second camera as a third position;
and determining the projection shapes of the reflection points of the plurality of light sources on the cornea on a plane parallel to the image plane of the second camera according to the third position to obtain the second projection shape.
4. The method according to claim 2 or 3, wherein after obtaining the spot light source matching result, the method further comprises:
determining the curvature center estimation coordinate of the cornea according to the light spot light source matching result;
determining the position of the pupil center through a refraction process of the pupil center;
determining the direction of an optical axis according to the estimated coordinates of the curvature center and the position of the pupil center;
and determining the sight line direction according to the optical axis direction.
5. The method of any one of claims 1-3, wherein the plurality of light sources are positioned to: the plurality of light sources form a ring.
6. An apparatus for setting parameters in a gaze tracking device, the gaze tracking device comprising a first camera and a second camera, the apparatus comprising:
a building module, configured to preset a position of the first camera as a first position, and construct an optimization objective equation using a curvature center coordinate of a cornea and a position of the second camera as variables, where a plurality of light sources are reflected by the cornea and then projected on the first camera and the second camera to obtain a light spot, the curvature center coordinate of the cornea changes within a preset empirical range, and the optimization objective equation includes: the entropy of the light spot distribution corresponding to the first camera and the second camera is the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source and the change of the curvature center coordinate of the cornea;
the first determining module is used for determining the position of the second camera corresponding to the maximum optimized objective equation value to obtain a second position;
the optimization objective equation is as follows:
wherein, H (q)ij)＝-p(qij)·logp(qij)，j∈{1，2}；
C is the curvature center coordinate of the cornea, C is a preset empirical range of C, i is the label of the light source, N is the total number of the light sources, i is 1,2, …, N, λ is an empirical coefficient, j is a camera label, q is a linear coordinate of the light source, and the light source is a linear coordinate of the light sourceijFor the j camera, the reflection point of the light source, labeled i, on the cornea,
is the center of the spot in the j-th camera;
obtaining the coordinates of the reflection point by the following formula:
||qij-c||＝R，
(Ii-oj)×(qij-oj)·(c-oj)＝0，
(Ii-qij)·(qij-c)·||oj-qij||＝(oj-qij)·(qij-c)·||Ii-qij||，
wherein R is the radius of curvature of the cornea, IiCoordinates of a light source denoted by i, ojThe coordinates of the camera, referenced j.
7. A storage medium characterized by comprising a stored program, wherein an apparatus in which the storage medium is located is controlled to execute the parameter setting method in the gaze tracking apparatus according to any one of claims 1 to 5 when the program is executed.
8. A processor for executing a program, wherein the program executes to execute the parameter setting method in the eye-gaze tracking apparatus according to any one of claims 1 to 5.
9. A terminal comprising a gaze tracking device comprising a first camera and a second camera, the terminal further comprising:
a building module, configured to preset a position of the first camera as a first position, and construct an optimization objective equation using a curvature center coordinate of a cornea and a position of the second camera as variables, where a plurality of light sources are reflected by the cornea and then projected on the first camera and the second camera to obtain a light spot, the curvature center coordinate of the cornea changes within a preset empirical range, and the optimization objective equation includes: the entropy of the light spot distribution corresponding to the first camera and the second camera is the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source and the change of the curvature center coordinate of the cornea;
the first determining module is used for determining the position of the second camera corresponding to the maximum optimized objective equation value to obtain a second position;
a processor that runs a program, wherein the program runs to execute the parameter setting method in the gaze tracking apparatus according to any one of claims 1 to 5 on data output from the building module and the first determining module.
10. A terminal comprising a gaze tracking device comprising a first camera and a second camera, the terminal further comprising:
a building module, configured to preset a position of the first camera as a first position, and construct an optimization objective equation using a curvature center coordinate of a cornea and a position of the second camera as variables, where a plurality of light sources are reflected by the cornea and then projected on the first camera and the second camera to obtain a light spot, the curvature center coordinate of the cornea changes within a preset empirical range, and the optimization objective equation includes: the entropy of the light spot distribution corresponding to the first camera and the second camera is the sum of the position difference between the light spot on the first camera and the light spot on the second camera corresponding to the same light source and the change of the curvature center coordinate of the cornea;
the first determining module is used for determining the position of the second camera corresponding to the maximum optimized objective equation value to obtain a second position;
a storage medium storing a program that executes the parameter setting method in the gaze tracking apparatus according to any one of claims 1 to 5 on data output from the building module and the first determining module at runtime.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710784884.8A CN107644443B (en) | 2017-09-01 | 2017-09-01 | Parameter setting method and device in sight tracking equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710784884.8A CN107644443B (en) | 2017-09-01 | 2017-09-01 | Parameter setting method and device in sight tracking equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107644443A CN107644443A (en) | 2018-01-30 |
| CN107644443B true CN107644443B (en) | 2020-07-28 |
Family
ID=61110251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710784884.8A Active CN107644443B (en) | 2017-09-01 | 2017-09-01 | Parameter setting method and device in sight tracking equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107644443B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7657079B2 (en) * | 2002-06-28 | 2010-02-02 | Intel Corporation | Single constraint at a time (SCAAT) tracking of a virtual reality (VR) display |
| CN104089606A (en) * | 2014-06-30 | 2014-10-08 | 天津大学 | Free space eye tracking measurement method |
| CN104834381A (en) * | 2015-05-15 | 2015-08-12 | 中国科学院深圳先进技术研究院 | Wearable device for sight focus positioning and sight focus positioning method |
| CN105934902A (en) * | 2013-11-27 | 2016-09-07 | 奇跃公司 | Virtual and augmented reality systems and methods |
| CN106780617A (en) * | 2016-11-24 | 2017-05-31 | 北京小鸟看看科技有限公司 | A kind of virtual reality system and its localization method |
-
2017
- 2017-09-01 CN CN201710784884.8A patent/CN107644443B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7657079B2 (en) * | 2002-06-28 | 2010-02-02 | Intel Corporation | Single constraint at a time (SCAAT) tracking of a virtual reality (VR) display |
| CN105934902A (en) * | 2013-11-27 | 2016-09-07 | 奇跃公司 | Virtual and augmented reality systems and methods |
| CN104089606A (en) * | 2014-06-30 | 2014-10-08 | 天津大学 | Free space eye tracking measurement method |
| CN104834381A (en) * | 2015-05-15 | 2015-08-12 | 中国科学院深圳先进技术研究院 | Wearable device for sight focus positioning and sight focus positioning method |
| CN106780617A (en) * | 2016-11-24 | 2017-05-31 | 北京小鸟看看科技有限公司 | A kind of virtual reality system and its localization method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107644443A (en) | 2018-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10854012B1 (en) | Concealing loss of distributed simultaneous localization and mapping (SLAM) data in edge cloud architectures | |
| CN107705333B (en) | Space positioning method and device based on binocular camera | |
| CN102662476B (en) | Gaze estimation method | |
| CN107358217B (en) | Sight estimation method and device | |
| Baek et al. | Eyeball model-based iris center localization for visible image-based eye-gaze tracking systems | |
| EP4224424A1 (en) | Method and system for determining spatial coordinates of a 3d reconstruction of at least part of a real object at absolute spatial scale | |
| CN108334191B (en) | Method and device for determining fixation point based on eye movement analysis equipment | |
| CN106133649B (en) | It is tracked using the eye gaze that binocular gaze constrains | |
| TWI577327B (en) | Method, apparatus and computer program product for positioning pupil | |
| CN108681399B (en) | Equipment control method, device, control equipment and storage medium | |
| CN106778641B (en) | Sight estimation method and device | |
| CN105678686A (en) | Picture processing method and device | |
| EP3698275A1 (en) | Data processing method, apparatus, system and storage media | |
| WO2016078296A1 (en) | Garment try-on method and apparatus | |
| CN107767421B (en) | Light spot light source matching method and device in sight tracking equipment | |
| CN110807427A (en) | Sight tracking method and device, computer equipment and storage medium | |
| CN115205470B (en) | Continuous scanning repositioning method, device, equipment, storage medium and three-dimensional continuous scanning method | |
| CN110245671A (en) | A kind of endoscopic images characteristic point matching method and system | |
| CN108235778B (en) | Calibration method and device based on cloud computing, electronic equipment and computer program product | |
| CN113495613B (en) | Eyeball tracking calibration method and device | |
| CN108510542B (en) | Method and device for matching light source and light spot | |
| Shin et al. | A novel single IR light based gaze estimation method using virtual glints | |
| CN107644443B (en) | Parameter setting method and device in sight tracking equipment | |
| Choi et al. | Improving the usability of remote eye gaze tracking for human-device interaction | |
| CN109712230B (en) | Three-dimensional model supplementing method and device, storage medium and processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |