WO2018098772A1 - 视点确定方法、装置、电子设备和计算机程序产品 - Google Patents

视点确定方法、装置、电子设备和计算机程序产品 Download PDF

Info

Publication number
WO2018098772A1
WO2018098772A1 PCT/CN2016/108222 CN2016108222W WO2018098772A1 WO 2018098772 A1 WO2018098772 A1 WO 2018098772A1 CN 2016108222 W CN2016108222 W CN 2016108222W WO 2018098772 A1 WO2018098772 A1 WO 2018098772A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
eyeball
infrared camera
viewpoint
image sensor
Prior art date
Application number
PCT/CN2016/108222
Other languages
English (en)
French (fr)
Inventor
骆磊
Original Assignee
深圳前海达闼云端智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳前海达闼云端智能科技有限公司 filed Critical 深圳前海达闼云端智能科技有限公司
Priority to PCT/CN2016/108222 priority Critical patent/WO2018098772A1/zh
Priority to CN201680002751.4A priority patent/CN107003744B/zh
Publication of WO2018098772A1 publication Critical patent/WO2018098772A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements

Definitions

  • the present invention relates to the field of eyeball technology, and in particular, to a method, device, electronic device and computer program product for determining a viewpoint.
  • the current eye movement control is a relatively advanced technology.
  • the user no longer needs the mouse, the arrow keys, the touchpad and other devices to control the cursor, but by directly tracking the eyeball, tracking to the eyeball where to look where the cursor is.
  • the movement control of the cursor is realized, thereby greatly improving the operation efficiency.
  • the eye movement control is based on the determination of the user's eyeball viewpoint.
  • the existing viewpoint determination technology requires a tedious calibration work. The user must sit in a relatively fixed position, and the height of the line of sight also needs to enter a specific range, and then the eyeballs are sequentially gaze. Multiple calibration points on the screen can be used to determine the position of the user's eyeball point of view. If the relative position between the user and the system equipment changes, recalibration is required to determine the position of the viewpoint again. In the process of determining the user's viewpoint, in order to avoid multiple redundant calibrations, the spatial position of the user's eyeball needs to be kept as constant as possible, which affects the customer's user experience.
  • the embodiment of the invention provides a method, a device, an electronic device and a computer program product for determining a viewpoint, which are mainly used to improve the degree of convenience of the viewpoint determining process, thereby reducing passive participation of the user in the viewpoint determining process.
  • an embodiment of the present invention provides a method for determining a viewpoint, including the following steps:
  • the viewpoint position of the eyeball is determined according to the position of the user's eyeball and the rotation angle.
  • an embodiment of the present invention provides a viewpoint determining apparatus, including:
  • Obtaining a module configured to acquire an imaging position of an image sensor of the user's eyeball in the infrared camera and a rotation angle of the user's eyeball;
  • the viewpoint position determining module is configured to determine a distance between the eyeball of the user and the lens of the infrared camera, determine a reference position of the eyeball of the user according to the imaging position and the distance, and determine a viewpoint position of the eyeball according to the reference position and the rotation angle.
  • an embodiment of the present invention provides an electronic device, including: an infrared camera, a memory, one or more processors; and one or more modules, the one or more modules being stored In the memory, and configured to be executed by the one or more processors, the one or more modules include instructions for performing the various steps of any of the above methods.
  • embodiments of the present invention provide a computer program product for use with an electronic device including a camera, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, The computer program mechanism includes instructions for performing the various steps of any of the above methods.
  • the technical solution provided by the embodiment of the invention captures the user's eye through the infrared camera, obtains the imaging position of the image sensor of the user's eyeball in the infrared camera, and the distance between the user's eyeball and the lens of the infrared camera, thereby determining the position of the user's eyeball, and combining the user
  • the rotation angle of the eyeball determines the position of the eyeball.
  • the multi-point correction is not needed during the determination of the viewpoint.
  • the position of the user's viewpoint can be determined directly according to the shooting of the infrared camera, which reduces the passive participation of the user and shortens the operation execution time of the system to determine the user's viewpoint. Make the viewpoint determination process simpler and more efficient, and the user experience is better.
  • FIG. 1 is a flow chart of steps of a method for determining a viewpoint according to an embodiment of the present invention
  • FIG. 2 is a schematic structural diagram of a viewpoint determining apparatus according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a process of determining a user's eyeball viewpoint in an embodiment of the present invention
  • FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention.
  • the process is complex, time consuming, and the user experience is poor.
  • the embodiment of the present invention provides a method for determining a viewpoint, which will be described below.
  • FIG. 1 is a flow chart of steps of a method for determining a viewpoint according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps:
  • S101 acquiring an imaging position of an image sensor of the user's eyeball in the infrared camera and a rotation angle of the user's eyeball;
  • S102 Determine a distance between a user's eyeball and a lens of the infrared camera
  • S103 Determine a position of a user's eyeball according to the imaging position and the distance;
  • S104 Determine a viewpoint position of the eyeball according to the position of the user's eyeball and the rotation angle.
  • the image of the user's eyeball is captured by the infrared camera
  • the angle of rotation of the user's eyeball is obtained by analyzing the image of the user's eyeball
  • the distance between the user's eyeball and the lens of the infrared camera is determined according to the size of the eyeball in the image. This distance can also be determined based on other measurement tools such as infrared laser range finder, AF range finder, or 3D model range finder like RealSense.
  • the position of the user's eyeball is determined by acquiring the imaging position of the image sensor of the user's eyeball in the infrared camera, and determining the position of the user's eyeball in combination with the determined distance between the user's eyeball and the lens of the infrared camera, and according to the determined position of the user's eyeball and the angle of rotation of the user's eyeball.
  • the method for determining the viewpoint does not require the user to perform multiple times of calibration with the system device in the viewpoint determination process, and can directly determine the position of the user's viewpoint according to the shooting of the infrared camera, thereby reducing the passive participation of the user.
  • the system shortens the operation execution time of the user's viewpoint, and makes the viewpoint determination process simpler and more efficient, and the user experience is better.
  • determining that the distance between the user's eyeball and the lens of the infrared camera may be the distance from the center position of the user's two eyeballs to the lens, and the user's eyeball imaging position is the center position of the user's two eyeballs in the image sensor; or The distance from the eyeball to the lens, the user's eyeball imaging position is the user's eyeball imaged in the image sensor.
  • obtaining the imaging position of the image sensor of the user's eyeball at the infrared camera and the rotation angle of the user's eyeball and determining the distance between the user's eyeball and the lens of the infrared camera may be performed simultaneously, or may be sequentially performed in a certain order.
  • Determining the viewpoint position of the eyeball according to the position of the eyeball of the user and the rotation angle may include: determining a reference position according to a position of the eyeball of the user, where the reference position is located on the screen and the position of the eyeball of the user a point at which the line is perpendicular to the screen on which the user's eye is looking; determining the view of the eyeball based on the reference position and the angle of rotation Point location.
  • determining the viewpoint position of the eyeball according to the position of the user's eyeball and the rotation angle, the user's eyeball line of sight can be obtained by determining the position of the user's eyeball and the rotation angle of the user's eyeball, and the determined eyeball line of sight of the user is determined.
  • the intersection point intersecting the screen plane viewed by the user is the user viewpoint position; and the reference position perpendicular to the screen on which the user's eyeball is viewed is determined by the position of the user's eyeball according to the position of the user's eyeball, according to the reference position
  • the rotation angle determines a viewpoint position of the eyeball.
  • determining the reference position according to the position of the user's eyeball may include: establishing a two-dimensional coordinate space in the image sensor, where a center point of the image sensor is an intersection of an x-axis and a y-axis of the coordinate space The origin; obtaining the coordinates of the imaging position of the image sensor of the user's eyeball at the infrared camera, and determining the distance L2 between the incident origin of the user's eye and the angle of the image sensor ⁇ , the intersection of the x-axis and the y-axis, and the reference position, according to the following formula, and The coordinates of the reference position:
  • the coordinates of the reference position are (-L2*cos ⁇ , L2*sin ⁇ );
  • f is the focal length of the infrared camera
  • b is the distance between the imaging position and the origin
  • L1 is the distance between the user's eyeball and the lens of the infrared camera
  • ⁇ is the angle between the imaging position and the x-axis.
  • the two-dimensional image is built in the image sensor. After the coordinate space, since the two are in the same two-dimensional coordinate space, the coordinate position of each pixel on the video screen can be known, so that the user viewpoint position coordinates can be determined for subsequent eye movement control.
  • determining a viewpoint position of the eyeball according to the reference position and the rotation angle includes: determining a viewpoint position coordinate of the eyeball according to the reference position and the horizontal direction angle component j of the rotation angle and the vertical direction angle component k (-L1*cos(arctan(f/b))*cos ⁇ +L1*sin(arctan(f/b))*tan(j), L1*cos(arctan(f/b))*sin ⁇ +L1*sin(arctan(f/b))*tan(k)).
  • the infrared camera captures an image of the user's eyeball, and can obtain a rotation angle for the user's eyeball, and the horizontal direction angle component j and the vertical direction angle by splitting the rotation angle of the eyeball
  • the component k is combined with the reference position to move in the x-axis and y-axis directions to obtain the corresponding eyeball viewpoint position coordinates of the user.
  • determining a viewpoint position of the eyeball according to the position of the user's eyeball and the rotation angle includes: a variation amount of an imaging position of the image sensor of the user's eyeball at the infrared camera is greater than a first threshold, and/or a user When the amount of change in the imaging size of the image sensor of the infrared camera is greater than the second threshold, the user's eyeball is determined according to the image position of the image sensor of the infrared camera and the changed distance between the user's eyeball and the lens of the infrared camera.
  • the imaging position and the distance determine the position of the user's eye.
  • the determination of the user's viewpoint is performed in real time.
  • the viewpoint of the eye movement control may be a continuous multi-point control operation, and the relative position of the user's eyeball and the viewed screen may be due to The user's walking or head rotation changes. This change may be caused by the user's subjective action, or may be caused by the user's unconscious or slight action, and may be changed due to the user's subjective action.
  • the viewpoint determining method provided by the present invention can detect a change in position of an eyeball imaging of a user at the image sensor and/or a change in size of the image of the user in the image sensor, and detect an image of the eyeball of the user in the image.
  • the amount of change in position of the sensor is greater than a first threshold, and/or the user
  • the eyeball imaging re-determines the reference position according to the changed distance between the user's eyeball and the lens and the eyeball imaging when the amount of change in the size of the image sensor is greater than the second threshold, and further according to the re-determined quasi-position and the changed eyeball
  • the rotation angle determines an eyeball viewpoint position, or detects that a position change amount of the user's eyeball imaging at the image sensor is less than or equal to a first threshold, and an amount of change in size of the image of the user's eyeball in the image sensor is less than or equal to a second At the threshold value, it is not necessary to recalculate the new reference position, and the reference position calculated in the previous continuous viewpoint determination process can be continuously used, and the eyeball viewpoint position is determined based on the reference position and the changed eyeball rotation angle.
  • the viewpoint determining method provided in the embodiment of the present invention can support the user to continuously control the eye movement of the screen multiple times, and does not need to perform multiple corrections in the control process, and only needs to perform another check and discriminate to automatically calibrate and re-determine the user.
  • the precise location of the fixation point does not require the user to perform multiple coordination with the system device indication, shortening the viewpoint determination time, and the control operation is relatively smooth.
  • first threshold and the second threshold set in the method for determining a viewpoint provided by the present invention may be an allowable error value, and a reasonable threshold may be obtained according to a specific device specification, and the threshold may be different for different specifications devices. Planners can be freely set according to the specific implementation environment or their own experience.
  • the viewpoint determining method provided by the present invention may further prompt the user to view the eyeball perpendicular to the video screen to obtain the reference position.
  • the prompt makes the user's line of sight perpendicular to the screen can be any form, such as voice prompts, image prompts, and the like.
  • a viewpoint determining apparatus is further provided in the embodiment of the present invention. Since the principle of solving the problem is similar to the viewpoint determining method, the implementation of the apparatus may refer to the implementation of the method, and the repeated description is not repeated.
  • FIG. 2 is a schematic structural diagram of a viewpoint determining apparatus according to an embodiment of the present invention. As shown in FIG. 2, the method includes:
  • the acquiring module 201 is configured to acquire an imaging position of an image sensor of the user's eyeball in the infrared camera and a rotation angle of the eyeball of the user;
  • a viewpoint position determining module 202 configured to determine a distance between a user's eyeball and a lens of the infrared camera, determine a reference position of the user's eyeball according to the imaging position and the distance, and determine a viewpoint position of the eyeball according to the reference position and the rotation angle .
  • the viewpoint determining apparatus provided by the present invention is preferably fixed at the same plane as the screen to be viewed.
  • the viewpoint determining apparatus provided by the present invention can be disposed directly below the screen, directly above or obliquely. Below, etc., of course, can also be fixed at a position that is not in the same plane as the screen being viewed.
  • the viewpoint determining apparatus provided in the embodiment of the present invention can be integrated with the screen to be viewed, in addition to being separately installed and installed on the screen to be viewed, that is, the viewpoint determining unit is integrated into the screen to be viewed.
  • the viewpoint position determining module is configured to determine a viewpoint position of the eyeball according to the position of the eyeball of the user and the rotation angle, including: determining a reference position according to a position of the eyeball of the user, where the reference position is located on a screen, And a position point that is perpendicular to the screen of the user's eyeball and the screen that the user's eye is looking at; the viewpoint position of the eyeball is determined according to the reference position and the rotation angle.
  • the viewpoint position determining module is configured to determine a reference position according to the position of the user's eyeball, including: establishing a two-dimensional coordinate space in the image sensor, where a center point of the image sensor is the coordinate space The intersection origin of the x-axis and the y-axis; acquiring the coordinates of the imaging position of the image sensor of the user's eyeball in the infrared camera, and determining the angle of intersection of the incident light of the user's eye with the angle of the image sensor ⁇ , the x-axis and the y-axis according to the following formula to The distance L2 of the reference position and the coordinates of the reference position:
  • the coordinates of the reference position are (-L2*cos ⁇ , L2*sin ⁇ );
  • f is the focal length of the infrared camera
  • b is the distance between the imaging position and the origin
  • L1 is the distance between the user's eye and the lens of the infrared camera
  • ⁇ is the clip of the imaging position and the x-axis angle.
  • the viewpoint position determining module is configured to determine a viewpoint position of the eyeball according to the reference position and the rotation angle, including: a horizontal direction angle component j and a vertical direction angle component according to the reference position and the rotation angle k, determine the position coordinate of the eyeball as (-L1*cos(arctan(f/b))*cos ⁇ +L1*sin(arctan(f/b))*tan(j), L1*cos(arctan(f/ b)) *sin ⁇ +L1*sin(arctan(f/b))*tan(k)).
  • the viewpoint position determining module is configured to determine a viewpoint position of the eyeball according to the position of the user's eyeball and the rotation angle, including: the amount of change in the imaging position of the image sensor of the user's eyeball in the infrared camera is greater than the first Threshold, and/or when the amount of change in the imaging size of the image sensor of the infrared camera in the infrared camera is greater than a second threshold, according to the changed imaging position of the image sensor of the user's eyeball in the infrared camera and the changed user's eyeball and infrared camera The distance of the lens determines the position of the user's eyeball; the amount of change in the imaging position of the image sensor of the user's eyeball in the infrared camera is less than or equal to the first threshold, and the amount of change in the imaging size of the image sensor of the user's eyeball in the infrared camera is less than or equal to At the second threshold, the position of the user's eyeball is
  • the viewpoint determining device provided in the embodiment of the present invention may be disposed directly below the view screen, and may of course be disposed at other positions of the screen, for example, setting the viewpoint determining device not to be in the same plane as the viewed screen, or setting The viewpoint determining device is directly above the viewed screen, obliquely below, or left, and the like.
  • the viewpoint determining apparatus provided in the embodiment of the present invention can be integrated with the screen to be viewed, in addition to being separately installed and installed, that is, the viewpoint determining apparatus is integrated into the screen to be viewed.
  • the user's eyeball is imaged on the image sensor through the lens, and the two-dimensional coordinate space of the x-axis and the y-axis can be established in the image sensor, since the number of horizontal and vertical pixel points of the image sensor is known.
  • the image sensor is 1920x1080 pixels, and the center position is set to the (0,0) position of the xy axis.
  • the user's eyeball is imaged from left to right at the 1500th pixel, and from top to bottom at the 800th pixel.
  • the distance from the lens to the image sensor that is, the focal length
  • f the angle between the incident light at the center of the user's eye and the sensor plane
  • FIG. 3 is a schematic diagram of a process of determining a user's eyeball viewpoint in the embodiment of the present invention.
  • A is the user's eyeball position
  • B is the reference position
  • C is the user's viewpoint position
  • AB is the position of the user's eyeball and the user's eyeball is watching.
  • the screen is perpendicular to the vertical line
  • AO is the distance between the user's eyeball and the lens
  • ⁇ is the angle between the OA and the screen plane (image sensor plane). It is assumed that the distance between the distance measuring module and the human eye is L1, and the length of the OB is L2.
  • the infrared camera acquires the user's eyeball to watch the C point, the user's eyeball rotation angle, and splits the eyeball's rotation angle into two parts of the lateral rotation and the longitudinal rotation, respectively, the angle j and the angle k.
  • the position of the fixation point moves to B2 on the X-axis component, and moves to B1 on the Y-axis component.
  • L1 is the distance measurement value
  • f is the lens focal length
  • b and ⁇ are directly derived from the imaging point position of the human eye center position on the sensor
  • j and k are the lateral component and longitudinal direction of the eye rotation obtained by the eye movement detection system. The component, the coordinates of the point C, is obtained, and the position of the human eye is obtained.
  • the user's eye movement control may be a continuous process
  • the user after determining that the user's viewpoint position is C point and completing the control, the user also marks another position in the video screen, as shown in FIG. 3, assuming D point.
  • the amount of change of the imaging position of the image sensor of the user's eyeball in the infrared camera is less than or equal to the first threshold, and the amount of change in the imaging size of the image sensor of the user's eyeball in the infrared camera is less than or equal to the second threshold, and the point B may continue.
  • the reference position the angle at which the eyeball rotates when the user's eyeball is at the D point is split, the lateral component and the longitudinal component are acquired, and the position coordinate of the point D of the user viewpoint is acquired in conjunction with the reference position B.
  • the user after determining that the user's viewpoint position is C point and completing the control, the user also marks another position in the video screen, as shown in FIG. 3, which is assumed to be an E point, and at this time, the user's eyeball is determined to be in the infrared camera.
  • the amount of change of the imaging position of the image sensor is greater than the first threshold, and the amount of change in the imaging size of the image sensor of the infrared camera in the infrared camera is greater than the second threshold, and the imaging position of the image sensor of the infrared camera according to the changed user's eyeball is required.
  • the distance between the changed user's eyeball and the lens of the infrared camera re-determines the reference position (the determination of the reference position is the same as in the above embodiment, and will not be described again), thereby determining the coordinate position of the point E of the user's viewpoint.
  • an electronic device is also provided in the embodiment of the present invention. Since the principle is similar to the method for determining the viewpoint, the implementation of the method may refer to the implementation of the method, and the repeated description is not repeated.
  • FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention.
  • an electronic device includes: an infrared camera 403, a memory 401, and one or more processors 402; And one or more modules, the one or more modules being stored
  • the memory is configured to be executed by the one or more processors, the one or more modules comprising instructions for performing the various steps of any of the above methods.
  • a computer program product used in combination with an electronic device including a camera is also provided in the embodiment of the present invention. Since the principle is similar to the viewpoint determining method, the implementation may refer to the implementation of the method, and the repetition is not Let me repeat.
  • a computer program product for use in conjunction with an electronic device including a camera comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising Instructions for performing the various steps in any of the foregoing methods.
  • the viewpoint determining scheme provided by the invention avoids complicated calibration process, and no longer requires a fixed station position or a fixed sitting posture, and the user can use the screen very casually, has high intelligence and flexibility, and is shortened. With the use of viewpoint determination, the user experience can be greatly improved, and many new application scenarios can be expanded, and the practicality is greatly enhanced.
  • embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)
  • Position Input By Displaying (AREA)

Abstract

视点确定方法、装置、电子设备和计算机程序产品，所述方法包括：获取用户眼球在红外摄像头的图像传感器的成像位置以及用户眼球的转动角度；确定用户眼球与红外摄像头的镜头的距离；根据所述成像位置以及所述距离确定用户眼球的位置；根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置。提供的技术方案使用户在视点确定过程中无需配合系统设备进行多次多点校正，可直接根据红外摄像头的拍摄确定用户视点位置，减少了用户的被动参与，缩短了系统确定用户视点的操作执行时间，使视点确定过程更为简单高效，用户体验更好。

Description

视点确定方法、装置、电子设备和计算机程序产品 技术领域
本发明涉及眼球技术领域，具体涉及一种视点确定方法、装置、电子设备和计算机程序产品。
背景技术
当前眼动控制是一项比较前沿的技术，用户不再需要鼠标，方向键、触摸板等设备对光标进行控制，而是通过直接对眼球进行追踪，追踪到眼球看向哪里光标就在哪里，实现对光标的移动控制，从而在很大程度上提升了操作效率。
眼动控制基于用户眼球视点的确定，现有视点确定技术需进行繁冗的校准工作，用户必须坐在一个相对比较固定的位置，且视线的高矮也需要进入一个特定的范围内，然后眼球依次注视屏幕上的多个校准点，完成校准后才可进行用户眼球视点位置的确定，如果用户与系统设备间的相对位置发生了变化，则需要重新校准，才能再次进行视点位置的确定，因此，在用户视点确定过程中，为避免多次的繁冗校准，用户眼球的空间位置需保持尽量不变，影响客户用户体验。
发明内容
本发明实施例提出了视点确定方法、装置、电子设备和计算机程序产品，主要用以提高视点确定过程的便利化程度，从而减少视点确定过程中用户的被动参与。
在一个方面，本发明实施例提供了一种视点确定方法，包括以下步骤：
获取用户眼球在红外摄像头的图像传感器的成像位置以及用户眼球的转动角度；
确定用户眼球与红外摄像头的镜头的距离；
根据所述成像位置以及所述距离确定用户眼球的位置；
根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置。
在另一个方面，本发明实施例提供了一种视点确定装置，包括：
获取模块，用于获取用户眼球在红外摄像头的图像传感器的成像位置以及用户眼球的转动角度；
视点位置确定模块，用于确定用户眼球与红外摄像头的镜头的距离，根据所述成像位置以及所述距离确定用户眼球的基准位置，根据所述基准位置以及所述转动角度确定眼球的视点位置。
在另一个方面，本发明实施例提供了一种电子设备，所述电子设备包括：红外摄像头，存储器，一个或多个处理器；以及一个或多个模块，所述一个或多个模块被存储在所述存储器中，并被配置成由所述一个或多个处理器执行，所述一个或多个模块包括用于执行任一上述方法中各个步骤的指令。在另一个方面，本发明实施例提供了一种与包括摄像头的电子设备结合使用的计算机程序产品，所述计算机程序产品包括计算机可读的存储介质和内嵌于其中的计算机程序机制，所述计算机程序机制包括用于执行任一上述方法中各个步骤的指令。
本发明的有益效果如下：
本发明实施例所提供的技术方案通过红外摄像头拍摄用户眼球，从中获取用户眼球在红外摄像头的图像传感器的成像位置以及用户眼球与红外摄像头的镜头的距离，从而确定用户眼球的位置，并结合用户眼球的转动角度确定眼球的视点位置，视点确定过程中无需进行多点校正，可直接根据红外摄像头的拍摄确定用户视点位置，减少了用户的被动参与，缩短了系统确定用户视点的操作执行时间，使视点确定过程更为简单高效，用户体验更好。
附图说明
下面将参照附图描述本发明的具体实施例，其中：
图1为本发明实施例中提供的一种视点确定方法的步骤流程图，；
图2为本发明实施例中提供的一种视点确定装置的结构示意图；
图3为本发明实施例中用户眼球视点确定过程的示意图；
图4为本发明实施例中提供的一种电子设备示意图。
具体实施方式
为了使本发明的技术方案及优点更加清楚明白，以下结合附图对本发明的示例性实施例进行进一步详细的说明，显然，所描述的实施例仅是本发明的一部分实施例，而不是所有实施例的穷举。并且在不冲突的情况下，本说明中的实施例及实施例中的特征可以互相结合。
发明人在发明过程中注意到：现有眼动控制技术进行眼动控制前，需对用户视点位置进行确定，而此过程中可能需要用户配合系统设备进行多次多点校准，才能确保所确定的用户视点位置的准确性，并且用户需保持与系统设备间的相对位置尽量固定，如果相对位置发生变化，就需要用户重新配合系统设备再次进行多次多点校准，因此，现有用户视点确定过程复杂，耗时较长，且用户体验差。
针对上述不足，本发明实施例提出了一种视点确定方法，下面进行说明。
图1为本发明实施例中提供的一种视点确定方法的步骤流程图，如图1所示，包括以下步骤：
S101：获取用户眼球在红外摄像头的图像传感器的成像位置以及用户眼球的转动角度；
S102：确定用户眼球与红外摄像头的镜头的距离；
S103：根据所述成像位置以及所述距离确定用户眼球的位置；
S104：根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置。
在一些实施方式中，通过红外摄像头捕获用户眼球的图像，通过分析该用户眼球的图像可获知用户眼球的转动角度，以及根据图像中眼球的大小确定用户眼球与红外摄像头的镜头的距离，当然，该距离也可根据其他测量工具测量确定，如红外激光测距器，AF测距器，或者类似RealSense的三维模型测距器等等。通过获取用户眼球在红外摄像头的图像传感器的成像位置，并结合上述确定的用户眼球与红外摄像头的镜头的距离确定用户眼球的位置，而根据确定的用户眼球的位置以及用户眼球的转动角度可获知用户眼球视线，进而可获知用户眼球视线与用户所视屏幕平面相交的用户视点位置。由此可见，本发明实施例中提供的视点确定方法无需用户在视点确定过程中配合系统设备进行多次多点的校准，可直接根据红外摄像头的拍摄确定用户视点位置，减少了用户的被动参与，缩短了系统确定用户视点的操作执行时间，使视点确定过程更为简单高效，用户体验更好。
在一些实施方式中，确定用户眼球与红外摄像头的镜头的距离可以为用户两眼球的中心位置到镜头的距离，用户眼球成像位置为用户两眼球在图像传感器中成像的中心位置；或者为用户任意一只眼球到镜头的距离，用户眼球成像位置为用户该只眼球在图像传感器中成像。
在一些实施方式中，获取用户眼球在红外摄像头的图像传感器的成像位置以及用户眼球的转动角度和确定用户眼球与红外摄像头的镜头的距离可同时进行，也可按照一定顺序设定进行先后操作。
具体实施中。根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置，可以包括：根据所述用户眼球的位置确定基准位置，所述基准位置为位于屏幕上，且与所述用户眼球的位置的连线与用户眼球所注视的屏幕相垂直的位置点；根据所述基准位置以及所述转动角度确定眼球的视 点位置。
在一些实施方式中，根据所述用户眼球的位置以及所述转动角度实现确定眼球的视点位置，可通过确定的用户眼球的位置以及用户眼球的转动角度可获知用户眼球视线，确定的用户眼球视线与用户所视屏幕平面相交的交点，即为用户视点位置；也可通过根据用户眼球的位置确定用户眼球的位置的连线与用户眼球所注视的屏幕相垂直的基准位置，根据所述基准位置以及所述转动角度确定眼球的视点位置。
具体实施中，根据所述用户眼球的位置确定基准位置，可以包括：在所述图像传感器中建立二维坐标空间，所述图像传感器的中心点为所述坐标空间的x轴和y轴的相交原点；获取用户眼球在红外摄像头的图像传感器的成像位置的坐标，根据如下公式确定用户眼球入射光线与所述图像传感器夹角β，x轴和y轴的相交原点到基准位置的距离L2，以及基准位置的坐标：
β＝arctan(f/b)；
L2＝L1*cosβ；
基准位置的坐标为(-L2*cosα，L2*sinα)；
其中，f为所述红外摄像头焦距，b为所述成像位置与所述原点的距离，L1为用户眼球与红外摄像头的镜头的距离，α为所述成像位置与x轴的夹角。
在一些实施方式中，在本发明实施例中提供的视点确定方法中，由于红外摄像头与用户所视频幕两者相对位置固定，且所视屏幕的分辨率已知，在图像传感器中建立二维坐标空间后，由于两者在同一二维坐标空间中，则可知所视频幕上的每个像素点的坐标位置，从而可确定用户视点位置坐标，以便后续眼动控制实现。
具体实施中，根据所述基准位置以及所述转动角度确定眼球的视点位 置，包括：根据所述基准位置以及所述转动角度的水平方向角度分量j与竖直方向角度分量k，确定眼球的视点位置坐标为(-L1*cos(arctan(f/b))*cosα+L1*sin(arctan(f/b))*tan(j)，L1*cos(arctan(f/b))*sinα+L1*sin(arctan(f/b))*tan(k))。
在一些实施方式中，本发明提供的视点确定方法中，红外摄像头捕获用户眼球图像，可获取用于用户眼球的转动角度，通过拆分眼球的转动角度的水平方向角度分量j与竖直方向角度分量k，并结合基准位置，在其x轴和y轴方向进行移动，获取对应的用户眼球视点位置坐标。
具体实施中，根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置，包括：在所述用户眼球在红外摄像头的图像传感器的成像位置的变化量大于第一阈值，和/或用户眼球在红外摄像头的图像传感器的成像大小的变化量大于第二阈值时，根据变化后的用户眼球在红外摄像头的图像传感器的成像位置以及变化后的用户眼球与红外摄像头的镜头的距离确定用户眼球的位置；在所述用户眼球在红外摄像头的图像传感器的成像位置的变化量小于等于第一阈值，和用户眼球在红外摄像头的图像传感器的成像大小的变化量小于等于第二阈值时，根据所述成像位置以及所述距离确定用户眼球的位置。
在一些实施方式中，用户视点的确定是可实时进行，用户进行眼动控制时，眼动控制的视点可能是连续的多点的控制操作，并且用户的眼球与所视屏幕的相对位置可能由于用户的走动或头部转动而进行变化，这种变化可能是用户主观进行动作而产生的变化，也可能是用户无意识的或者轻微动作而产生的变化，对于可能由于用户主观进行动作而产生的变化，本发明提供的视点确定方法可检测用户的眼球成像在所述图像传感器的位置变化和/或所述用户的眼球成像在所述图像传感器的大小变化，在检测用户的眼球成像在所述图像传感器的位置变化量大于第一阈值，和/或所述用户 的眼球成像在所述图像传感器的大小变化量大于第二阈值时，根据变化后的用户眼球与所述镜头的距离以及眼球成像重新确定基准位置，进而根据重新确定的准位置以及变化后的眼球转动角度确定眼球视点位置，或者在检测到用户的眼球成像在所述图像传感器的位置变化量小于等于第一阈值，和所述用户的眼球成像在所述图像传感器的大小变化量小于等于第二阈值时，既无需重新计算新的基准位置，可继续利用前续视点确定过程中计算出的基准位置，基于该基准位置以及变化后的眼球转动角度确定眼球视点位置。因此，本发明实施例中提供的视点确定方法可支持用户连续的多次对屏幕的眼动控制，并且控制过程中无需多次校正，仅需重新进行一次检查判别就可自动校准，重新确定用户的注视点的精确位置，不需要用户进行多次配合系统设备指示，缩短了视点确定时间，并且控制操作较为流畅。
此外，本发明提供的视点确定方法中设置的第一阈值与第二阈值可以是允许误差值，具体可根据具体设备规格实验得出合理的阈值，不同规格设备此阈值可能也会不同，本领域计划人员可根据具体实施环境或自身经验自由设定。
在一些实施方式中，本发明提供的视点确定方法中还可以提示用户眼球视线垂直于所视频幕以便获取基准位置。实施中，提示使用户视线垂直于屏幕可以为任意形式，如语音提示，图像提示等。
基于同一发明构思，本发明实施例中还提供了一种视点确定装置，由于这些装置解决问题的原理与视点确定方法相似，因此这些装置的实施可以参见方法的实施，重复之处不再赘述。
图2为本发明实施例中提供的一种视点确定装置的结构示意图，如图2所示，包括：
获取模块201，用于获取用户眼球在红外摄像头的图像传感器的成像位置以及用户眼球的转动角度；
视点位置确定模块202，用于确定用户眼球与红外摄像头的镜头的距离，根据所述成像位置以及所述距离确定用户眼球的基准位置，根据所述基准位置以及所述转动角度确定眼球的视点位置。
在一些实施方式中，，本发明提供的视点确定装置优选固定在与所视屏幕同一平面的位置，例如，可将本发明提供的视点确定装置设置在所视屏幕的正下方，正上方或者斜下方等，当然，也可固定在与所视屏幕非同一平面的位置。此外，本发明实施例中提供的视点确定装置除与所视屏幕分别设置安装，也可与所视屏幕设置于一体，即将视点确定置集成于所视屏幕中。
具体实施中，视点位置确定模块用于根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置，包括：根据所述用户眼球的位置确定基准位置，所述基准位置为位于屏幕上，且与所述用户眼球的位置的连线与用户眼球所注视的屏幕相垂直的位置点；根据所述基准位置以及所述转动角度确定眼球的视点位置。
具体实施中，所述视点位置确定模块用于根据所述用户眼球的位置确定基准位置，包括：在所述图像传感器中建立二维坐标空间，所述图像传感器的中心点为所述坐标空间的x轴和y轴的相交原点；获取用户眼球在红外摄像头的图像传感器的成像位置的坐标，根据如下公式确定用户眼球入射光线与所述图像传感器夹角β，x轴和y轴的相交原点到基准位置的距离L2，以及基准位置的坐标：
β＝arctan(f/b)；
L2＝L1*cosβ；
基准位置的坐标为(-L2*cosα，L2*sinα)；
其中，f为所述红外摄像头焦距，b为所述成像位置与所述原点的距离，L1为用户眼球与红外摄像头的镜头的距离，α为所述成像位置与x轴的夹 角。
具体实施中，视点位置确定模块用于根据所述基准位置以及所述转动角度确定眼球的视点位置，包括：根据所述基准位置以及所述转动角度的水平方向角度分量j与竖直方向角度分量k，确定眼球的视点位置坐标为(-L1*cos(arctan(f/b))*cosα+L1*sin(arctan(f/b))*tan(j)，L1*cos(arctan(f/b))*sinα+L1*sin(arctan(f/b))*tan(k))。
具体实施中，视点位置确定模块用于根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置，包括：在所述用户眼球在红外摄像头的图像传感器的成像位置的变化量大于第一阈值，和/或用户眼球在红外摄像头的图像传感器的成像大小的变化量大于第二阈值时，根据变化后的用户眼球在红外摄像头的图像传感器的成像位置以及变化后的用户眼球与红外摄像头的镜头的距离确定用户眼球的位置；在所述用户眼球在红外摄像头的图像传感器的成像位置的变化量小于等于第一阈值，和用户眼球在红外摄像头的图像传感器的成像大小的变化量小于等于第二阈值时，根据所述成像位置以及所述距离确定用户眼球的位置。
为了便于本发明的实施，下面以具体实例进一步说明。
本发明实施例中提供的视点确定装置可设置在所视屏幕的正下方，当然，还可设置在所述屏幕的其他位置，例如，设置视点确定装置不与所视屏幕的同一平面，或设置视点确定装置的在所视屏幕的正上方，斜下方，或者左侧等等。此外，本发明实施例中提供的视点确定装置除与所视屏幕分别设置安装，也可与所视屏幕设置于一体，即将视点确定装置集成于所视屏幕中。
根据凸透镜成像原理，用户眼球通过镜头在图像传感器上进行成像，可在图像传感器中建立x轴与y轴的二维坐标空间，由于图像传感器的横向和纵向像素点个数是已知的。例如，图像传感器为1920x1080个像素点， 中心位置设置为xy轴的(0,0)位置，假设用户眼球成像为从左到右第1500个像素点，从上到下第800个像素点，则可知tanα＝(800-540)/(1500-960),继而可得到α角的值，其中，α为用户眼球成像点与x轴的夹角，由于每个像素点一般为正方形结构，假设边长为a(已知数值)，像素间距为0，假设人眼在图像传感器上的成像中心点与传感器中心点的直线距离为b，则可知(a*(800-540))2+(a*(1500-960))2＝b2,继而可求得b，其中，b为用户眼球成像点到原点的距离。
而由于镜头到图像传感器的间距，即焦距，在对焦完成后就是已知值，可由摄像头内部实时得到，假定为f，此刻用户眼球中心位置光线入射与传感器平面的夹角假设为β，则可知tanβ＝f/b，进而可求得β＝arctan(f/b)。
图3本发明实施例中用户眼球视点确定过程的示意图，如图3所示，假设A为用户眼球位置，B为基准位置，C为用户视点位置，AB为用户眼球的位置与用户眼球所注视的屏幕相垂直的垂线，AO为用户眼球与所述镜头的距离，β为OA与屏幕平面(图像传感器平面)的夹角，假设测距模块到人眼距离为L1，OB长度为L2，则根据立体空间中的直角三角形AOB，可得到cosβ＝OB/OA，进而可得到L2＝L1*cosβ，假设AB长度为L3，则L3＝L1*sinβ，由于OB与x轴夹角前面得出为α，则可得到B点坐标为(-L2*cosα，L2*sinα)，正负号取决于所在象限。
用户眼球注视所视屏幕C点，红外摄像头获取用户眼球注视C点时，用户眼球转动角度，将眼球的转动角度拆分为横向转动和纵向转动两个分量，分别为角j和角k，则注视点位置在X轴分量上移动到B2，在Y轴分量上移动到B1，可获知BB1长度为L3*tan(k),BB2长度为L3*tan(j),则C点坐标为(-L2*cosα+L3*tan(j)，L2*sinα+L3*tan(k))，带入前面的等式，可得到C点坐标为：
x＝-L1*cos(arctan(f/b))*cosα+L1*sin(arctan(f/b))*tan(j)；
y＝L1*cos(arctan(f/b))*sinα+L1*sin(arctan(f/b))*tan(k)，正负号取决于象限和眼球转动方向。此处L1为测距数值，f为镜头焦距，b和α都由人眼中心位置在传感器上的成像点位置直接得出，j和k为眼动检测系统得到的眼球转动的横向分量和纵向分量，获得C点的坐标，也就得到了人眼视点位置。
实施中，考虑到用户进行眼动控制可能是一个连续的过程，在确定用户视点位置为C点并完成控制后，用户又注视频幕中又一位置，如图3所示，假设为D点，此时判别用户眼球在红外摄像头的图像传感器的成像位置的变化量小于等于第一阈值，并且用户眼球在红外摄像头的图像传感器的成像大小的变化量小于等于第二阈值，可继续将B点作为基准位置，将用户眼球注视D点时眼球转动的角度进行拆分，获取横向分量和纵向分量，并结合基准位置B点获取用户视点D点的位置坐标。
另一种实施方式中，在确定用户视点位置为C点并完成控制后，用户又注视频幕中又一位置，如图3所示，假设为E点，此时判别用户眼球在红外摄像头的图像传感器的成像位置的变化量大于第一阈值，并且用户眼球在红外摄像头的图像传感器的成像大小的变化量大于第二阈值，则需要根据变化后的用户眼球在红外摄像头的图像传感器的成像位置以及变化后的用户眼球与红外摄像头的镜头的距离重新确定基准位置(基准位置的确定与上述实施中相同，不再赘述)，进而确定用户视点E点的坐标位置。
基于同一发明构思，本发明实施例中还提供了一种电子设备，由于其原理与视点确定方法相似，因此其实施可以参见方法的实施，重复之处不再赘述。
图4为本发明实施例中提供的一种电子设备示意图，如图4所示，本发明实施例中提供的一种电子设备包括：红外摄像头403，存储器401，一个或多个处理器402；以及一个或多个模块，所述一个或多个模块被存储在 所述存储器中，并被配置成由所述一个或多个处理器执行，所述一个或多个模块包括用于执行任一上述方法中各个步骤的指令。基于同一发明构思，本发明实施例中还提供了一种与包括摄像头的电子设备结合使用的计算机程序产品，由于其原理与视点确定方法相似，因此其实施可以参见方法的实施，重复之处不再赘述。
本发明实施例中提供的一种与包括摄像头的电子设备结合使用的计算机程序产品，所述计算机程序产品包括计算机可读的存储介质和内嵌于其中的计算机程序机制，所述计算机程序机制包括用于执行任一前述方法中各个步骤的指令。
本发明提供的视点确定方案避免了繁复的校准过程，也不再需要用户固定的站位或者固定的坐姿，用户可以在屏幕前很随意的使用，有较高的智能性和灵活性，并且缩短了视点确定的用时，用户体验可得到了大幅提升，可拓展出很多新的应用场景，实用性得到了极大的增强。
为了描述的方便，以上所述装置的各部分以功能分为各种模块或单元分别描述。当然，在实施本发明时可以把各模块或单元的功能在同一个或多个软件或硬件中实现。
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中 的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本发明的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。

Claims (12)

  1. 一种视点确定方法，其特征在于，包括以下步骤：
    获取用户眼球在红外摄像头的图像传感器的成像位置以及用户眼球的转动角度；
    确定用户眼球与红外摄像头的镜头的距离；
    根据所述成像位置以及所述距离确定用户眼球的位置；
    根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置。
  2. 如权利要求1所述的方法，其特征在于，所述根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置，包括：
    根据所述用户眼球的位置确定基准位置，所述基准位置为位于屏幕上，且与所述用户眼球的位置的连线与用户眼球所注视的屏幕相垂直的位置点；
    根据所述基准位置以及所述转动角度确定眼球的视点位置。
  3. 如权利要求2所述的方法，其特征在于，所述根据所述用户眼球的位置确定基准位置，包括：
    在所述图像传感器中建立二维坐标空间，所述图像传感器的中心点为所述坐标空间的x轴和y轴的相交原点；获取用户眼球在红外摄像头的图像传感器的成像位置的坐标，根据如下公式确定用户眼球入射光线与所述图像传感器夹角β，x轴和y轴的相交原点到基准位置的距离L2，以及基准位置的坐标：
    β＝arctan(f/b)；
    L2＝L1*cosβ；
    基准位置的坐标为(-L2*cosα，L2*sinα)；
    其中，f为所述红外摄像头焦距，b为所述成像位置与所述原点的距离，L1为用户眼球与红外摄像头的镜头的距离，α为所述成像位置与x轴的夹角。
  4. 如权利要求2所述的方法，其特征在于，所述根据所述基准位置以及所述转动角度确定眼球的视点位置，包括：
    根据所述基准位置以及所述转动角度的水平方向角度分量j与竖直方向角度分量k，确定眼球的视点位置坐标为(-L1*cos(arctan(f/b))*cosα+L1*sin(arctan(f/b))*tan(j)，L1*cos(arctan(f/b))*sinα+L1*sin(arctan(f/b))*tan(k))。
  5. 如权利要求1所述的方法，其特征在于，所述根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置，包括：
    在所述用户眼球在红外摄像头的图像传感器的成像位置的变化量大于第一阈值，和/或用户眼球在红外摄像头的图像传感器的成像大小的变化量大于第二阈值时，根据变化后的用户眼球在红外摄像头的图像传感器的成像位置以及变化后的用户眼球与红外摄像头的镜头的距离确定用户眼球的位置。
  6. 一种视点确定装置，其特征在于，包括：
    获取模块，用于获取用户眼球在红外摄像头的图像传感器的成像位置以及用户眼球的转动角度；
    视点位置确定模块，用于确定用户眼球与红外摄像头的镜头的距离，根据所述成像位置以及所述距离确定用户眼球的基准位置，根据所述基准位置以及所述转动角度确定眼球的视点位置。
  7. 如权利要求6所述的装置，其特征在于，所述视点位置确定模块用于根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置，包括：
    根据所述用户眼球的位置确定基准位置，所述基准位置为位于屏幕上，且与所述用户眼球的位置的连线与用户眼球所注视的屏幕相垂直的位置点；
    根据所述基准位置以及所述转动角度确定眼球的视点位置。
  8. 如权利要求7所述的装置，其特征在于，所述视点位置确定模块用于根据所述用户眼球的位置确定基准位置，包括：
    在所述图像传感器中建立二维坐标空间，所述图像传感器的中心点为所述坐标空间的x轴和y轴的相交原点；获取用户眼球在红外摄像头的图像传感器的成像位置的坐标，根据如下公式确定用户眼球入射光线与所述图像传感器夹角β，x轴和y轴的相交原点到基准位置的距离L2，以及基准位置的坐标：
    β＝arctan(f/b)；
    L2＝L1*cosβ；
    基准位置的坐标为(-L2*cosα，L2*sinα)；
    其中，f为所述红外摄像头焦距，b为所述成像位置与所述原点的距离，L1为用户眼球与红外摄像头的镜头的距离，α为所述成像位置与x轴的夹角。
  9. 如权利要求7所述的装置，其特征在于，所述视点位置确定模块用于根据所述基准位置以及所述转动角度确定眼球的视点位置，包括：
    根据所述基准位置以及所述转动角度的水平方向角度分量j与竖直方向角度分量k，确定眼球的视点位置坐标为(-L1*cos(arctan(f/b))*cosα+L1*sin(arctan(f/b))*tan(j)，L1*cos(arctan(f/b))*sinα+L1*sin(arctan(f/b))*tan(k))。
  10. 如权利要求6所说的装置，其特征在于，所述视点位置确定模块用于根据所述用户眼球的位置以及所述转动角度确定眼球的视点位置，包括：
    在所述用户眼球在红外摄像头的图像传感器的成像位置的变化量大于第一阈值，和/或用户眼球在红外摄像头的图像传感器的成像大小的变化量大于第二阈值时，根据变化后的用户眼球在红外摄像头的图像传感器的成像位置以及变化后的用户眼球与红外摄像头的镜头的距离确定用户眼球的 位置；
    在所述用户眼球在红外摄像头的图像传感器的成像位置的变化量小于等于第一阈值，和用户眼球在红外摄像头的图像传感器的成像大小的变化量小于等于第二阈值时，根据所述成像位置以及所述距离确定用户眼球的位置。
  11. 一种电子设备，其特征在于，所述电子设备包括：红外摄像头，存储器，一个或多个处理器；以及一个或多个模块，所述一个或多个模块被存储在所述存储器中，并被配置成由所述一个或多个处理器执行，所述一个或多个模块包括用于执行权利要求1-5中任一所述方法中各个步骤的指令。
  12. 一种与包括摄像头的电子设备结合使用的计算机程序产品，所述计算机程序产品包括计算机可读的存储介质和内嵌于其中的计算机程序机制，所述计算机程序机制包括用于执行权利要求1-5中任一所述方法中各个步骤的指令。
PCT/CN2016/108222 2016-12-01 2016-12-01 视点确定方法、装置、电子设备和计算机程序产品 WO2018098772A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2016/108222 WO2018098772A1 (zh) 2016-12-01 2016-12-01 视点确定方法、装置、电子设备和计算机程序产品
CN201680002751.4A CN107003744B (zh) 2016-12-01 2016-12-01 视点确定方法、装置和电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/108222 WO2018098772A1 (zh) 2016-12-01 2016-12-01 视点确定方法、装置、电子设备和计算机程序产品

Publications (1)

Publication Number Publication Date
WO2018098772A1 true WO2018098772A1 (zh) 2018-06-07

Family

ID=59431128

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/108222 WO2018098772A1 (zh) 2016-12-01 2016-12-01 视点确定方法、装置、电子设备和计算机程序产品

Country Status (2)

Country Link
CN (1) CN107003744B (zh)
WO (1) WO2018098772A1 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107656619A (zh) * 2017-09-26 2018-02-02 广景视睿科技（深圳）有限公司 一种智能投影方法、系统及智能终端
CN109995986A (zh) * 2017-12-29 2019-07-09 北京亮亮视野科技有限公司 控制智能眼镜拍摄视角移动的方法
CN109960412B (zh) * 2019-03-22 2022-06-07 北京七鑫易维信息技术有限公司 一种基于触控调整注视区域的方法以及终端设备
CN110196640A (zh) * 2019-05-31 2019-09-03 维沃移动通信有限公司 一种操作控制方法及终端
CN110286753B (zh) * 2019-06-11 2022-06-07 福建天泉教育科技有限公司 视频关注度判断方法、存储介质
CN110244853A (zh) * 2019-06-21 2019-09-17 四川众信互联科技有限公司 手势控制方法、装置、智能显示终端及存储介质
CN110377158B (zh) * 2019-07-22 2023-03-31 北京七鑫易维信息技术有限公司 基于变化视场范围的眼球追踪的校准方法及电子设备
CN112286350A (zh) * 2020-10-27 2021-01-29 珠海格力电器股份有限公司 设备控制方法及装置、电子设备、电子装置、处理器

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050200806A1 (en) * 2004-03-12 2005-09-15 Honda Motor Co., Ltd. Line-of-sight detection method and apparatus therefor
CN102662476A (zh) * 2012-04-20 2012-09-12 天津大学 一种视线估计方法
CN103604412A (zh) * 2013-10-30 2014-02-26 北京智谷睿拓技术服务有限公司 定位方法及定位装置
CN103654709A (zh) * 2012-09-26 2014-03-26 瑞萨微系统有限公司 视线检测设备、视线检测方法及用于其的程序
CN103809737A (zh) * 2012-11-13 2014-05-21 华为技术有限公司 一种人机交互方法及装置
CN102473033B (zh) * 2009-09-29 2015-05-27 阿尔卡特朗讯 一种注视点检测方法及其装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201400084A (zh) * 2012-06-27 2014-01-01 Yomiko Advertising Inc 注視線測定方法、注視線測定裝置、眼球回旋點測定方法及著目點測定裝置
CN102981736B (zh) * 2012-10-29 2016-11-23 华为终端有限公司 屏幕解锁的方法及终端

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050200806A1 (en) * 2004-03-12 2005-09-15 Honda Motor Co., Ltd. Line-of-sight detection method and apparatus therefor
CN102473033B (zh) * 2009-09-29 2015-05-27 阿尔卡特朗讯 一种注视点检测方法及其装置
CN102662476A (zh) * 2012-04-20 2012-09-12 天津大学 一种视线估计方法
CN103654709A (zh) * 2012-09-26 2014-03-26 瑞萨微系统有限公司 视线检测设备、视线检测方法及用于其的程序
CN103809737A (zh) * 2012-11-13 2014-05-21 华为技术有限公司 一种人机交互方法及装置
CN103604412A (zh) * 2013-10-30 2014-02-26 北京智谷睿拓技术服务有限公司 定位方法及定位装置

Also Published As

Publication number Publication date
CN107003744B (zh) 2019-05-10
CN107003744A (zh) 2017-08-01

Similar Documents

Publication Publication Date Title
WO2018098772A1 (zh) 视点确定方法、装置、电子设备和计算机程序产品
US9774837B2 (en) System for performing distortion correction and calibration using pattern projection, and method using the same
US9759548B2 (en) Image processing apparatus, projector and projector system including image processing apparatus, image processing method
JP5580164B2 (ja) 光学情報処理装置、光学情報処理方法、光学情報処理システム、光学情報処理プログラム
US20110249117A1 (en) Imaging device, distance measuring method, and non-transitory computer-readable recording medium storing a program
JP2016173313A (ja) 視線方向推定システム、視線方向推定方法及び視線方向推定プログラム
JP2019194616A (ja) 画像に基づく位置検出方法、装置、機器及び記憶媒体
CN107392961B (zh) 基于增强现实的空间定位方法及装置
WO2013005265A1 (ja) 三次元座標計測装置及び三次元座標計測方法
CN106570907B (zh) 一种相机标定方法及装置
JP6452235B2 (ja) 顔検出方法、顔検出装置、及び顔検出プログラム
US9525817B2 (en) System and method of controlling imaging direction and angle of view of camera
KR101592405B1 (ko) 3차원 영상 획득 방법, 장치 및 컴퓨터 판독 가능한 기록 매체
KR101349347B1 (ko) 증강현실을 위한 스마트폰 자이로센서 기반의 정면객체 생성시스템 및 방법
JP6288770B2 (ja) 顔検出方法、顔検出システム、および顔検出プログラム
WO2019045089A1 (ja) 情報処理装置、長さ測定システム、長さ測定方法およびプログラム記憶媒体
US20180077405A1 (en) Method and system for scene scanning
KR101578891B1 (ko) 패턴 인식을 이용한 영상 정보의 차원 일치 장치 및 방법
KR101340555B1 (ko) 기준 시점 영상 생성 장치 및 방법
JP2010145219A (ja) 運動推定装置及びプログラム
JP2013015519A (ja) ３次元相対座標計測装置およびその方法
JP5901379B2 (ja) 撮像装置校正方法および画像合成装置
JP5727969B2 (ja) 位置推定装置、方法、及びプログラム
JP2005323905A (ja) 眼球運動測定装置、及び眼球運動測定プログラム
JP2011118767A (ja) 表情モニタリング方法および表情モニタリング装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16922787

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16922787

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 30/09/2019)

122 Ep: pct application non-entry in european phase

Ref document number: 16922787

Country of ref document: EP

Kind code of ref document: A1