WO2017152592A1 - 一种移动终端应用操作方法及移动终端 - Google Patents

一种移动终端应用操作方法及移动终端 Download PDF

Info

Publication number
WO2017152592A1
WO2017152592A1 PCT/CN2016/097032 CN2016097032W WO2017152592A1 WO 2017152592 A1 WO2017152592 A1 WO 2017152592A1 CN 2016097032 W CN2016097032 W CN 2016097032W WO 2017152592 A1 WO2017152592 A1 WO 2017152592A1
Authority
WO
WIPO (PCT)
Prior art keywords
sight
line
user
mobile terminal
gaze
Prior art date
Application number
PCT/CN2016/097032
Other languages
English (en)
French (fr)
Inventor
赵洪涛
Original Assignee
乐视控股（北京）有限公司
乐视移动智能信息技术（北京）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 乐视控股（北京）有限公司, 乐视移动智能信息技术（北京）有限公司 filed Critical 乐视控股（北京）有限公司
Publication of WO2017152592A1 publication Critical patent/WO2017152592A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements

Definitions

  • the embodiments of the present invention relate to the field of electronic technologies, and in particular, to a mobile terminal application operating method and a mobile terminal.
  • the inventor finds that the existing mobile terminal realizes the control and operation of the mobile terminal application through the interaction mode of the button or the touch screen, and in many cases, the user needs to operate accurately and conveniently. With both hands, it is easy to carry out wrong operation with one hand and increase the risk of falling the mobile phone. Especially the large-screen mobile phone is inconvenient for one-handed operation, which brings great inconvenience to the user operation and greatly reduces the user. Experience.
  • the embodiment of the invention provides a mobile terminal application operation method, including:
  • the application registration step includes: constructing a listener for monitoring a line of sight gaze event, and registering at least one mobile terminal application on the listener;
  • the pupil capturing step includes: acquiring a plurality of user pupil images respectively from a plurality of cameras disposed on the mobile terminal;
  • the line-of-sight recognizing step includes: performing line-of-sight recognition on the plurality of user pupil images to obtain a user's line of sight including a gaze range;
  • the gaze range sending step includes: when the user's line of sight satisfies the preset line of sight gaze event A condition is met, a line-of-sight gaze event is triggered, and the gaze range of the user's line of sight is sent to the mobile terminal application registered on the listener.
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by a preset number of disappearance thresholds.
  • the threshold of the number of disappearances is twice, and the condition for confirming the gaze gaze event is:
  • the time interval between the first disappearance of the user's line of sight and the second disappearance of the user's line of sight is less than or equal to a preset time interval threshold.
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by a preset number of disappearance thresholds, and the gaze range of the user's line of sight is within the screen of the mobile terminal.
  • the distance measuring step includes: performing distance recognition on the plurality of user pupil images to obtain a distance between the user pupil and the center of the screen of the mobile terminal;
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by the preset number of disappearance thresholds, and the user's line of sight is within the screen of the mobile terminal, and the distance between the user's pupil and the center of the screen of the mobile terminal is at a preset distance threshold.
  • the embodiment of the invention provides a mobile terminal, including:
  • An application registration module configured to: construct a listener that monitors a line of sight gaze event, and register at least one mobile terminal application on the listener;
  • a pupil capturing module configured to: acquire multiple user pupil images from multiple cameras installed in the mobile terminal;
  • a line-of-sight recognition module configured to: perform line-of-sight recognition on the plurality of user pupil images to obtain a user's line of sight including a gaze range;
  • the gaze range sending module is configured to: when the user's line of sight meets the preset gaze gaze event confirmation condition, trigger a gaze gaze event, and send the gaze range of the user's line of sight to the mobile terminal application registered on the listener.
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by a preset number of disappearance thresholds.
  • the threshold of the number of disappearances is twice, and the condition for confirming the gaze gaze event is:
  • the time interval between the first disappearance of the user's line of sight and the second disappearance of the user's line of sight is less than or equal to a preset time interval threshold.
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by a preset number of disappearance thresholds, and the gaze range of the user's line of sight is within the screen of the mobile terminal.
  • a distance measuring module is further included;
  • the distance measuring module is configured to: perform distance recognition on the plurality of user pupil images to obtain a distance between the user's pupil and the center of the screen of the mobile terminal;
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by the preset number of disappearance thresholds, and the user's line of sight is within the screen of the mobile terminal, and the distance between the user's pupil and the center of the screen of the mobile terminal is at a preset distance threshold.
  • An embodiment of the present invention further provides a mobile terminal application operating device, including:
  • the processor, the memory, and the communication interface complete communication with each other through the bus;
  • the communication interface is used for information transmission of the mobile terminal application operating device and the camera;
  • the processor is configured to invoke logic instructions in the memory to perform the following method
  • a line-of-sight gaze event is triggered, and the gaze range of the user's line of sight is transmitted to the mobile terminal application registered on the listener.
  • a line-of-sight gaze event is triggered, and the gaze range of the user's line of sight is transmitted to the mobile terminal application registered on the listener.
  • Embodiments of the present invention also provide a storage medium for storing the above computer program.
  • the embodiment of the present invention has the following beneficial effects: capturing the user's pupil image through the camera, and acquiring the user's line of sight.
  • the user's line of sight meets the preset sight gaze event confirmation condition, the user is sent to the relevant mobile terminal application without the user having to operate with one hand or both hands.
  • the mobile terminal improves user operation accuracy, makes operation more convenient, and enhances user experience.
  • FIG. 1 is a schematic flowchart of a method for operating a mobile terminal application according to the present invention
  • FIG. 2 is a schematic flow chart of a preferred embodiment of the present invention.
  • FIG. 3 is a structural block diagram of a mobile terminal according to the present invention.
  • FIG. 4 is a structural block diagram of a mobile terminal according to the present invention.
  • FIG. 5 is a logic block diagram of a mobile terminal application operating device according to an embodiment of the present invention.
  • FIG. 1 is a schematic flowchart of a mobile terminal application operation method according to the present invention, and the specific steps are as follows:
  • Step S101 comprising: constructing a listener for monitoring a sight gaze event, and in the listener Registering at least one mobile terminal application;
  • Step S102 comprising: acquiring a plurality of user pupil images respectively from a plurality of cameras disposed on the mobile terminal;
  • Step S103 comprising: performing line-of-sight recognition on the plurality of user pupil images to obtain a user's line of sight including a gaze range;
  • Step S104 comprising: when the user's line of sight meets the preset line-of-sight event confirmation condition, triggering a line-of-sight gaze event, and transmitting the gaze range of the user's line of sight to the mobile terminal application registered on the listener.
  • the mobile terminal is preferably a smartphone.
  • Step S101 monitors the construction listener of the gaze gaze event in advance, and registers at least one mobile terminal application. When the user starts the mobile terminal, the listener will always monitor whether a gaze gaze event occurs.
  • step S102 acquires a plurality of user pupil images from the plurality of cameras, respectively.
  • the camera has at least two, preferably four, which are respectively arranged on the left and right sides of the front side of the mobile terminal, and one on the left and the bottom of the front side.
  • the mobile terminal captures the front image of the user through the camera, and extracts the user's pupil image through 3D image recognition.
  • Step S103 identifies the user's line of sight, specifically using the eyeball center and the pupil line as the user's line of sight, and adopting the human eye tracking method to determine the line of sight range of the user's line of sight.
  • Step S104 detects that the user's line of sight satisfies the line-of-sight gaze event confirmation condition, and defines the event that the line of sight disappears the preset number of disappearance thresholds as a line-of-sight gaze event, which is similar to the one-touch time of the touch screen, and the listener that monitors the line-of-sight gaze event is Triggering, and then sending the gaze range to the mobile terminal application registered on the listener, and the mobile terminal application performs processing similar to responding to the finger click event, for example, the news application receives the gaze range and enlarges the news within the gaze range. Display, or display hyperlinks within the gaze range, etc.
  • the invention captures the user's pupil image through the camera, and obtains the user's line of sight.
  • the user sends the message to the relevant mobile terminal application, without the user having to operate the mobile terminal with one hand or both hands, thereby improving the user operation accuracy. Make the operation more convenient and enhance the user experience.
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by a preset number of disappearance thresholds.
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by the preset number of disappearance thresholds, for example, the user blinks, at which time the user's line of sight disappears.
  • the number of disappearance thresholds is two
  • the line-of-sight event confirmation condition is:
  • the time interval between the first disappearance of the user's line of sight and the second disappearance of the user's line of sight is less than or equal to a preset time interval threshold.
  • the number of disappearance thresholds is set to one time, the user is likely to be erroneously triggered by blinking eyes, and if it is three or more times, the number of times the user needs to blink is too high, resulting in inconvenience to the user.
  • the embodiment increases the time interval threshold, and the time interval threshold is preferably 0.5 seconds, so that the false triggering behavior can be avoided very well.
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by a preset number of disappearance thresholds, and the gaze range of the user's line of sight is within the screen of the mobile terminal.
  • the gaze range is sent to the mobile terminal application registered by the listener only after the user's line of sight is within the screen and the preset number of disappearance thresholds disappears. Thereby, false triggering can be further reduced.
  • a distance measuring step is further included;
  • the distance measuring step includes: performing distance recognition on the plurality of user pupil images to obtain a distance between the user pupil and the center of the screen of the mobile terminal;
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by the preset number of disappearance thresholds, and the user's line of sight is within the screen of the mobile terminal, and the distance between the user's pupil and the center of the screen of the mobile terminal is at a preset distance threshold.
  • This embodiment further limits the distance between the user's pupil and the center of the screen of the mobile terminal to avoid false triggering when the mobile terminal is too far away from the user.
  • the distance threshold ranges from 20-25 cm.
  • FIG. 2 is a schematic flowchart of a preferred embodiment of the present invention, and the specific steps are as follows:
  • Step S201 constructing a listener for monitoring a line-of-sight gaze event, and registering at least one mobile terminal application on the listener;
  • Step S202 capturing a pupil image of the user through the camera
  • the camera has at least two, preferably four, respectively arranged on the left and right sides of the front side of the mobile terminal, one on the left and the bottom of the front side, and the front image of the user is captured by multiple cameras, and then identified by 3D image recognition technology.
  • Step S203 Calculate a line of sight of the user including the gaze range according to the image of the plurality of user pupils through the eyeball and the pupil, and calculate the distance between the user's pupil and the center of the screen of the mobile terminal, if the distance between the user's pupil and the screen of the mobile terminal If it is within 20-25 cm and the gaze range is within the screen, step S204 is performed, otherwise step S202 is performed;
  • Step S204 acquiring a line of sight where the user's pupil disappears for the first time
  • Step S205 Acquire a line of sight that the user's pupil disappears for the second time
  • the mobile terminal determines whether the first blink time and the second blink time interval are within 0.5 seconds. If yes, go to step S206, otherwise go to step S202;
  • Step S206 the mobile terminal records a visual confirmation, and uses the visual confirmation as a line-of-sight gaze event.
  • the listener that listens to the gaze gaze event listens to the gaze gaze event, and then sends the gaze range to each application registered with the listener. ;
  • the mobile terminal distributes the event to each application through the OnStare (int[]area) method, and the int[]area is the gaze range, which is an array of four values, which are:
  • the Y-axis coordinate of the lower boundary of the screen viewed by the pupil is the Y-axis coordinate of the lower boundary of the screen viewed by the pupil.
  • FIG. 3 is a structural block diagram of a mobile terminal according to the present invention, including:
  • the application registration module 301 is configured to: construct a listener for monitoring a line-of-sight gaze event, and register at least one mobile terminal application on the listener;
  • the pupil capturing module 302 is configured to: acquire a plurality of user pupil images respectively from a plurality of cameras disposed on the mobile terminal;
  • the line-of-sight recognition module 303 is configured to perform line-of-sight recognition on the plurality of user pupil images to obtain a user's line of sight including a gaze range;
  • the gaze range sending module 304 is configured to: when the user's line of sight meets the preset gaze gaze event confirmation condition, trigger a gaze gaze event, and send the gaze range of the user's line of sight to the mobile terminal application registered on the listener .
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by a preset number of disappearance thresholds.
  • the number of disappearance thresholds is two
  • the line-of-sight event confirmation condition is:
  • the time interval between the first disappearance of the user's line of sight and the second disappearance of the user's line of sight is less than or equal to a preset time interval threshold.
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by a preset number of disappearance thresholds, and the gaze range of the user's line of sight is within the screen of the mobile terminal.
  • a distance measuring module is further included;
  • the distance measuring module is configured to: perform distance recognition on the plurality of user pupil images to obtain a distance between the user's pupil and the center of the screen of the mobile terminal;
  • the line-of-sight gaze event confirmation condition is that the user's line of sight disappears by the preset number of disappearance thresholds, and the user's line of sight is within the screen of the mobile terminal, and the distance between the user's pupil and the center of the screen of the mobile terminal is at a preset distance threshold.
  • FIG. 4 is a structural block diagram of a mobile terminal according to the present invention, which mainly includes a processor 401, a memory 402, a camera 403, a communication component 404, and the like.
  • the mobile terminal of the present invention is preferably a smartphone or the like.
  • the memory 402 stores the specific code of the foregoing method
  • the processor 401 acquires an image from the camera 403 and specifically executes the code stored in the memory 402, and the communication component 404 is used to communicate letter.
  • FIG. 5 is a logic block diagram showing a mobile terminal application operating device according to an embodiment of the present invention.
  • the mobile terminal applies an operation device, including:
  • processor 501 a processor 501, a memory 502, a communication interface 503, and a bus 504;
  • the processor 501, the memory 502, and the communication interface 503 complete communication with each other through the bus 504;
  • the communication interface 503 is used for information transmission of the mobile terminal application operating device and the camera;
  • the processor 501 is configured to invoke logic instructions in the memory 502 to perform the following methods;
  • a line-of-sight gaze event is triggered, and the gaze range of the user's line of sight is transmitted to the mobile terminal application registered on the listener.
  • the embodiment discloses a computer program, including program code, where the program code is used to perform the following operations:
  • a line-of-sight gaze event is triggered, and the gaze range of the user's line of sight is transmitted to the mobile terminal application registered on the listener.
  • This embodiment discloses a storage medium for storing a computer program as described in the foregoing embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Eye Examination Apparatus (AREA)
  • Telephone Function (AREA)

Abstract

本发明实施例公开一种移动终端应用操作方法及移动终端，方法包括：构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。本发明通过摄像头捕捉用户瞳孔图像，获取用户视线，当用户视线满足预设视线注视事件确认条件时，则发送至相关移动终端应用，无需用户单手或者双手操作移动终端，提高用户操作准确性，使操作更加便捷，增强用户体验。

Description

一种移动终端应用操作方法及移动终端
交叉引用
本申请引用于2016年03月07日提交的专利名称为“一种移动终端应用操作方法及移动终端”的第2016101274775号中国专利申请，其通过引用被全部并入本申请。
技术领域
本发明实施例涉及电子技术领域，尤其涉及一种移动终端应用操作方法及移动终端。
背景技术
目前，在实现本发明实施例的过程中，发明人发现现有的移动终端通过按键或者触摸屏幕的交互方式来实现对移动终端应用的控制与操作，很多时候为了操作准确与方便，用户都需要双手进行操作，单手很容易进行错误操作并且增加了手机掉落的风险，尤其是大屏幕手机更是不方便进行单手操作，这样一来给用户操作带来很大不便，大大降低了用户体验感。
发明内容
基于此，有必要针对现有技术对移动终端应用操作不便的技术问题，提供一种移动终端应用操作方法及移动终端。
本发明实施例提出了一种移动终端应用操作方法，包括：
应用注册步骤，包括：构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
瞳孔捕捉步骤，包括：从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
视线识别步骤，包括：对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
注视范围发送步骤，包括：当所述用户视线满足预设视线注视事件确 认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
进一步的，所述视线注视事件确认条件为所述用户视线消失预设消失次数阈值。
更进一步的，所述消失次数阈值为两次，所述视线注视事件确认条件为：
所述用户视线第一次消失与所述用户视线第二次消失的时间间隔小于或等于预设时间间隔阈值。
再进一步的，所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线的注视范围在所述移动终端屏幕内。
再进一步的，还包括距离测量步骤；
所述距离测量步骤，包括：对所述多张用户瞳孔图像进行距离识别得到用户瞳孔与移动终端屏幕中心的距离；
所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线在所述移动终端屏幕内，且所述用户瞳孔与移动终端屏幕中心的距离在预设距离阈值范围内。
本发明实施例提出了一种移动终端，包括：
应用注册模块，用于：构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
瞳孔捕捉模块，用于：从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
视线识别模块，用于：对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
注视范围发送模块，用于：当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
进一步的，所述视线注视事件确认条件为所述用户视线消失预设消失次数阈值。
更进一步的，所述消失次数阈值为两次，所述视线注视事件确认条件为：
所述用户视线第一次消失与所述用户视线第二次消失的时间间隔小于或等于预设时间间隔阈值。
再进一步的，所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线的注视范围在所述移动终端屏幕内。
再进一步的，还包括距离测量模块；
所述距离测量模块，用于：对所述多张用户瞳孔图像进行距离识别得到用户瞳孔与移动终端屏幕中心的距离；
所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线在所述移动终端屏幕内，且所述用户瞳孔与移动终端屏幕中心的距离在预设距离阈值范围内。
本发明的实施例还提供了一种移动终端应用操作设备，包括：
处理器、存储器、通信接口和总线；其中，
所述处理器、存储器、通信接口通过所述总线完成相互间的通信；
所述通信接口用于所述移动终端应用操作设备与摄像头的信息传输；
所述处理器用于调用所述存储器中的逻辑指令，以执行如下方法；
构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
本发明的实施例还提供了一种计算机程序，包括程序代码，所述程序代码用于执行如下操作：
构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
本发明的实施例还提供了一种存储介质，用于存储上述的计算机程序。
实施本发明实施例，具有如下有益效果：通过摄像头捕捉用户瞳孔图像，获取用户视线，当用户视线满足预设视线注视事件确认条件时，则发送至相关移动终端应用，无需用户单手或者双手操作移动终端，提高用户操作准确性，使操作更加便捷，增强用户体验。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明移动终端应用操作方法的流程示意图；
图2为本发明最佳实施例的流程示意图；
图3为本发明一种移动终端的结构模块图；
图4为本发明移动终端的结构框图。
图5为本发明一个实施例的一种移动终端应用操作设备的逻辑框图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
下面结合附图和具体实施例对本发明做进一步详细的说明。
如图1所示为本发明移动终端应用操作方法的流程示意图，具体步骤如下：
步骤S101，包括：构建监听视线注视事件的监听器，并在所述监听器 上注册至少一个移动终端应用；
步骤S102，包括：从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
步骤S103，包括：对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
步骤S104，包括：当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
移动终端，优选为智能手机。步骤S101预先监听视线注视事件的构建监听器，并注册至少一个移动终端应用，当用户启动移动终端后，该监听器会一直监听是否有视线注视事件发生。
当用户注视移动终端的屏幕时，步骤S102从多个摄像头分别获取多张用户瞳孔图像。具体的，摄像头至少为两个，最佳为四个，分别布置在移动终端正面上部左右各一个，正面下部左右各一个。移动终端通过摄像头拍摄到用户正面图像，并通过3D图像识别提取出用户瞳孔图像。
步骤S103识别用户视线，具体采用眼球球心与瞳孔连线作为用户视线，采用人眼跟踪方法确定用户视线的视线范围。
步骤S104检测到用户视线满足视线注视事件确认条件，将视线消失次数预设消失次数阈值的事件定义为视线注视事件，则类似于触摸屏的一次手指点击时间，此时监听视线注视事件的监听器被触发，然后将注视范围发送给注册在所述监听器上的移动终端应用，由移动终端应用进行类似响应手指点击事件的处理，例如：新闻类应用接收到注视范围则将注视范围内的新闻放大显示，或者将注视范围内的超链接跳转显示等。
本发明通过摄像头捕捉用户瞳孔图像，获取用户视线，当用户视线满足预设视线注视事件确认条件时，则发送至相关移动终端应用，无需用户单手或者双手操作移动终端，提高用户操作准确性，使操作更加便捷，增强用户体验。
在其中一个实施例中，所述视线注视事件确认条件为所述用户视线消失预设消失次数阈值。
视线注视事件确认条件为用户视线消失预设消失次数阈值，例如用户眨眼，此时用户视线消失。
在其中一个实施例中，所述消失次数阈值为两次，所述视线注视事件确认条件为：
所述用户视线第一次消失与所述用户视线第二次消失的时间间隔小于或等于预设时间间隔阈值。
消失次数阈值如果设置为一次，则用户容易因为眼睛疲劳眨眼而误触发，而如果是三次或以上，则用户需要眨眼的次数过高，导致用户操作不便。
同时，为了避免将用户长时间间隔的眨眼行为导致的误触发，本实施例增加了时间间隔阈值，时间间隔阈值优选为0.5秒，从而能非常好地避免误触发行为。
在其中一个实施例中，所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线的注视范围在所述移动终端屏幕内。
本实施例，使得只有用户视线在屏幕内且消失预设消失次数阈值后，才会将注视范围发送给监听器所注册的移动终端应用。从而能够进一步减少误触发。
在其中一个实施例中，还包括距离测量步骤；
所述距离测量步骤，包括：对所述多张用户瞳孔图像进行距离识别得到用户瞳孔与移动终端屏幕中心的距离；
所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线在所述移动终端屏幕内，且所述用户瞳孔与移动终端屏幕中心的距离在预设距离阈值范围内。
本实施例进一步限制用户瞳孔与移动终端屏幕中心的距离，以避免移动终端与用户距离过远时的误触发。
优选地，所述距离阈值范围为20-25厘米。
20-25里面是人眼观察屏幕的最佳距离，因此，如果瞳孔与屏幕的距离在这个范围内，且满足其他条件，则将注视范围发送给在监听器注册的 移动终端应用。
如图2所示为本发明最佳实施例的流程示意图，具体步骤如下：
步骤S201，构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
步骤S202，通过摄像头捕捉用户瞳孔图像；
具体的，摄像头至少为两个，最佳为四个，分别布置在移动终端正面上部左右各一个，正面下部左右各一个，通过多个摄像头拍摄到用户正面图像，然后通过3D图像识别技术，识别出多张用户瞳孔图像；
步骤S203，根据多张用户瞳孔图像，通过眼球球心与瞳孔连线，计算出包括注视范围的用户视线，并计算出用户瞳孔与移动终端屏幕中心的距离，如果用户瞳孔与移动终端屏幕的距离为20-25厘米内且注视范围在屏幕内，则执行步骤S204，否则执行步骤S202；
步骤S204，获取所述用户瞳孔第一次消失的视线；
具体的，当用户第一次眨眼时，用户视线会消失在摄像头的记录当中，将该事件记录下来；
步骤S205：获取所述用户瞳孔第二次消失的视线；
具体的，当用户第二次眨眼时，用户视线会再次消失在摄像头的记录当中，将该事件记录下来，然后移动终端判断第一次眨眼时间与第二次眨眼时间间隔是否在0.5秒内，如果是，执行步骤S206，否则执行步骤S202；
步骤S206，移动终端记录一次视觉确认，将此次视觉确认作为视线注视事件，此时监听该视线注视事件的监听器监听到视线注视事件，则将注视范围发送给注册该监听器的各个应用使用；
具体的，移动终端通过OnStare(int[]area)方法将此事件分发给各个应用，int[]area即注视范围，其为一个含有4个值的数组，依次是:
瞳孔看的屏幕左侧边界的X轴坐标；
瞳孔看的屏幕右侧边界的X轴坐标；
瞳孔看的屏幕上侧边界的Y轴坐标；
瞳孔看的屏幕下侧边界的Y轴坐标。
如图3所示为本发明一种移动终端的结构模块图，包括：
应用注册模块301，用于：构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
瞳孔捕捉模块302，用于：从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
视线识别模块303，用于：对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
注视范围发送模块304，用于：当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
在其中一个实施例中，所述视线注视事件确认条件为所述用户视线消失预设消失次数阈值。
在其中一个实施例中，所述消失次数阈值为两次，所述视线注视事件确认条件为：
所述用户视线第一次消失与所述用户视线第二次消失的时间间隔小于或等于预设时间间隔阈值。
在其中一个实施例中，所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线的注视范围在所述移动终端屏幕内。
在其中一个实施例中，还包括距离测量模块；
所述距离测量模块，用于：对所述多张用户瞳孔图像进行距离识别得到用户瞳孔与移动终端屏幕中心的距离；
所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线在所述移动终端屏幕内，且所述用户瞳孔与移动终端屏幕中心的距离在预设距离阈值范围内。
如图4所示为本发明移动终端的结构框图，其主要包括：处理器401、存储器402、摄像头403及通信组件404等。一般来说，本发明的移动终端优选为智能手机等。
其中存储器402中存储前述方法的具体代码，由处理器401从摄像头403获取图像并具体执行存储器402中存储的代码，通信组件404用于通 信。
图5是示出本发明一实施例的移动终端应用操作设备的逻辑框图。
参照图5，所述移动终端应用操作设备，包括：
处理器(processor)501、存储器(memory)502、通信接口(Communications Interface)503和总线504；其中，
所述处理器501、存储器502、通信接口503通过所述总线504完成相互间的通信；
所述通信接口503用于所述移动终端应用操作设备与摄像头的信息传输；
所述处理器501用于调用所述存储器502中的逻辑指令，以执行如下方法；
构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
参看图1，本实施例公开一种计算机程序，包括程序代码，所述程序代码用于执行如下操作：
构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
本实施例公开一种存储介质，用于存储如前述实施例所述的计算机程序。
以上所述实施例仅表达了本发明的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进，这些都属于本发明的保护范围。因此，本发明专利的保护范围应以所附权利要求为准。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本发明的实施例的技术方案，而非对其限制；尽管参照前述实施例对本发明的实施例进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (13)

  1. 一种移动终端应用操作方法，其特征在于，包括：
    应用注册步骤，包括：构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
    瞳孔捕捉步骤，包括：从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
    视线识别步骤，包括：对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
    注视范围发送步骤，包括：当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
  2. 如权利要求1所述的移动终端应用操作方法，其特征在于，所述视线注视事件确认条件为所述用户视线消失预设消失次数阈值。
  3. 如权利要求2所述的移动终端应用操作方法，其特征在于，所述消失次数阈值为两次，所述视线注视事件确认条件为：
    所述用户视线第一次消失与所述用户视线第二次消失的时间间隔小于或等于预设时间间隔阈值。
  4. 如权利要求1-3任一项所述的移动终端应用操作方法，其特征在于，所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线的注视范围在所述移动终端屏幕内。
  5. 如权利要求4所述的移动终端应用操作方法，其特征在于，还包括距离测量步骤；
    所述距离测量步骤，包括：对所述多张用户瞳孔图像进行距离识别得到用户瞳孔与移动终端屏幕中心的距离；
    所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线在所述移动终端屏幕内，且所述用户瞳孔与移动终端屏幕中心的距离在预设距离阈值范围内。
  6. 一种移动终端，其特征在于，包括：
    应用注册模块，用于：构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
    瞳孔捕捉模块，用于：从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
    视线识别模块，用于：对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
    注视范围发送模块，用于：当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
  7. 如权利要求6所述的移动终端，其特征在于，所述视线注视事件确认条件为所述用户视线消失预设消失次数阈值。
  8. 如权利要求7所述的移动终端，其特征在于，所述消失次数阈值为两次，所述视线注视事件确认条件为：
    所述用户视线第一次消失与所述用户视线第二次消失的时间间隔小于或等于预设时间间隔阈值。
  9. 如权利要求6-8任一项所述的移动终端，其特征在于，所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线的注视范围在所述移动终端屏幕内。
  10. 如权利要求9所述的移动终端，其特征在于，还包括距离测量模块；
    所述距离测量模块，用于：对所述多张用户瞳孔图像进行距离识别得到用户瞳孔与移动终端屏幕中心的距离；
    所述视线注视事件确认条件为：所述用户视线消失预设消失次数阈值，且所述用户视线在所述移动终端屏幕内，且所述用户瞳孔与移动终端屏幕中心的距离在预设距离阈值范围内。
  11. 一种移动终端应用操作设备，其特征在于，包括：
    处理器、存储器、通信接口和总线；其中，
    所述处理器、存储器、通信接口通过所述总线完成相互间的通信；
    所述通信接口用于所述移动终端应用操作设备与摄像头的信息传输；
    所述处理器用于调用所述存储器中的逻辑指令，以执行如下方法；
    构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
    从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
    对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
    当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
  12. 一种计算机程序，其特征在于，包括程序代码，所述程序代码用于执行如下操作：
    构建监听视线注视事件的监听器，并在所述监听器上注册至少一个移动终端应用；
    从设置在移动终端的多个摄像头分别获取多张用户瞳孔图像；
    对所述多张用户瞳孔图像进行视线识别，得到包括注视范围的用户视线；
    当所述用户视线满足预设视线注视事件确认条件，则触发视线注视事件，将所述用户视线的注视范围发送到在所述监听器上注册的移动终端应用。
  13. 一种存储介质，用于存储如权利要求12所述的计算机程序。
PCT/CN2016/097032 2016-03-07 2016-08-26 一种移动终端应用操作方法及移动终端 WO2017152592A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610127477.5A CN105843376A (zh) 2016-03-07 2016-03-07 一种移动终端应用操作方法及移动终端
CN2016101274775 2016-03-07

Publications (1)

Publication Number Publication Date
WO2017152592A1 true WO2017152592A1 (zh) 2017-09-14

Family

ID=56587013

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/097032 WO2017152592A1 (zh) 2016-03-07 2016-08-26 一种移动终端应用操作方法及移动终端

Country Status (2)

Country Link
CN (1) CN105843376A (zh)
WO (1) WO2017152592A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190345052A1 (en) * 2018-05-11 2019-11-14 Entegris, Inc. Molds that include a ceramic material surface, and related methods for making and using the molds
EP4064153A4 (en) * 2019-12-25 2023-01-18 NEC Platforms, Ltd. BUSINESS MANAGEMENT SYSTEM, BUSINESS MANAGEMENT PROCESS AND BUSINESS MANAGEMENT PROGRAM

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105843376A (zh) * 2016-03-07 2016-08-10 乐视移动智能信息技术（北京）有限公司 一种移动终端应用操作方法及移动终端
CN107845112A (zh) * 2017-10-13 2018-03-27 上海斐讯数据通信技术有限公司 一种引导用户使用电子产品姿势的方法及系统
CN107817899B (zh) * 2017-11-24 2018-06-26 南京同睿信息科技有限公司 一种用户观看内容实时分析方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101311882A (zh) * 2007-05-23 2008-11-26 华为技术有限公司 视线跟踪人机交互方法及装置
CN101344919A (zh) * 2008-08-05 2009-01-14 华南理工大学 视线跟踪方法及应用该方法的残疾人辅助系统
CN105843376A (zh) * 2016-03-07 2016-08-10 乐视移动智能信息技术（北京）有限公司 一种移动终端应用操作方法及移动终端

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101891786B1 (ko) * 2011-11-29 2018-08-27 삼성전자주식회사 아이 트래킹 기반의 사용자 기능 운용 방법 및 이를 지원하는 단말기
CN103324290A (zh) * 2013-07-04 2013-09-25 深圳市中兴移动通信有限公司 终端设备及其眼睛操控方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101311882A (zh) * 2007-05-23 2008-11-26 华为技术有限公司 视线跟踪人机交互方法及装置
CN101344919A (zh) * 2008-08-05 2009-01-14 华南理工大学 视线跟踪方法及应用该方法的残疾人辅助系统
CN105843376A (zh) * 2016-03-07 2016-08-10 乐视移动智能信息技术（北京）有限公司 一种移动终端应用操作方法及移动终端

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190345052A1 (en) * 2018-05-11 2019-11-14 Entegris, Inc. Molds that include a ceramic material surface, and related methods for making and using the molds
TWI737992B (zh) * 2018-05-11 2021-09-01 美商恩特葛瑞斯股份有限公司 包含陶瓷材料表面的模具，以及製造和使用該模具的相關方法
US11572298B2 (en) * 2018-05-11 2023-02-07 Entegris, Inc. Molds that include a ceramic material surface, and related methods for making and using the molds
EP4064153A4 (en) * 2019-12-25 2023-01-18 NEC Platforms, Ltd. BUSINESS MANAGEMENT SYSTEM, BUSINESS MANAGEMENT PROCESS AND BUSINESS MANAGEMENT PROGRAM

Also Published As

Publication number Publication date
CN105843376A (zh) 2016-08-10

Similar Documents

Publication Publication Date Title
WO2017152592A1 (zh) 一种移动终端应用操作方法及移动终端
WO2020015468A1 (zh) 一种图像传输方法、装置、终端设备及存储介质
CN107395877B (zh) 一种终端防误触方法及终端
EP3342160B1 (en) Display apparatus and control methods thereof
US11386698B2 (en) Method and device for sending alarm message
KR102317021B1 (ko) 디스플레이 장치 및 이의 영상 보정 방법
US10424116B2 (en) Display apparatus and controlling method thereof
EP2824541B1 (en) Method and apparatus for connecting devices using eye tracking
US20140354538A1 (en) Method and apparatus for operating notification function in user device
CN110334629B (zh) 可多方位检测距离的方法、装置及可读存储介质
CN110866230B (zh) 已认证设备辅助的用户认证
TW202009761A (zh) 身分識別方法、裝置和電腦可讀儲存媒體
CN108206892B (zh) 联系人隐私的保护方法、装置、移动终端及存储介质
WO2018094911A1 (zh) 一种多媒体文件的分享方法及终端设备
CN105320261A (zh) 移动终端的控制方法及移动终端
CN108243275B (zh) 一种来电提醒方法及虚拟现实设备
KR102293416B1 (ko) 통신 장치, 서버 및 그것의 통신 방법
WO2015165331A9 (en) Information processing method and apparatus
CN109688366B (zh) 一种在线会议转移方法、装置及移动终端
WO2024040861A1 (zh) 操作权限的控制方法、控制系统、电子设备及存储介质
US10904067B1 (en) Verifying inmate presence during a facility transaction
CN115883959A (zh) 用于隐私保护的画面内容控制方法及相关产品
US10635802B2 (en) Method and apparatus for accessing Wi-Fi network
CN108093101B (zh) 来电显示控制方法、柔性屏终端及计算机存储介质
CN112507798A (zh) 活体检测方法、电子设备与存储介质

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16893237

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 16893237

Country of ref document: EP

Kind code of ref document: A1