CN112416126A - Page rolling control method and device, storage medium and electronic equipment - Google Patents

Page rolling control method and device, storage medium and electronic equipment Download PDF

Info

Publication number
CN112416126A
CN112416126A CN202011296482.1A CN202011296482A CN112416126A CN 112416126 A CN112416126 A CN 112416126A CN 202011296482 A CN202011296482 A CN 202011296482A CN 112416126 A CN112416126 A CN 112416126A
Authority
CN
China
Prior art keywords
target object
target
head
processing result
page
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011296482.1A
Other languages
Chinese (zh)
Other versions
CN112416126B (en
Inventor
郭凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Haier Technology Co Ltd
Haier Smart Home Co Ltd
Original Assignee
Qingdao Haier Technology Co Ltd
Haier Smart Home Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Haier Technology Co Ltd, Haier Smart Home Co Ltd filed Critical Qingdao Haier Technology Co Ltd
Priority to CN202011296482.1A priority Critical patent/CN112416126B/en
Publication of CN112416126A publication Critical patent/CN112416126A/en
Application granted granted Critical
Publication of CN112416126B publication Critical patent/CN112416126B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0485—Scrolling or panning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Geometry (AREA)
  • Ophthalmology & Optometry (AREA)
  • User Interface Of Digital Computer (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a page scrolling control method and device, a storage medium and electronic equipment. Wherein, the method comprises the following steps: in the process of displaying a target page in terminal equipment, calling a camera in the terminal equipment to acquire a head image of a target object of the target page being browsed; processing the head image based on a head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture; processing the head image based on a gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil location for indicating a pupil of a target subject in an eye; inputting the first processing result and the second processing result into a decision tree model to determine a gazing area of the target object; and controlling the target page to automatically scroll according to the watching area. The invention solves the technical problem of complicated page scrolling operation in the related technology.

Description

Page rolling control method and device, storage medium and electronic equipment
Technical Field
The invention relates to the technical field of mobile terminal display, in particular to a page scrolling control method and device, a terminal and a storage medium.
Background
With the development of internet technology, the contents such as web pages, electronic books, documents or short messages displayed on mobile terminals are large at present, and the contents (such as web pages) cannot be completely displayed on a mobile terminal screen at one time, and the contents can be completely displayed only by scrolling the contents of the web pages on the screen up and down.
At present, when a user uses a mobile terminal, the user usually relies on the user to manually operate the scroll bar ScrollView control to slide up and down to move the screen content of the mobile terminal. That is, the screen of the mobile terminal is operated to control scrolling of the contents to be read by the user by scrolling up or down by touching the screen down or up with a finger. In some scenarios (for example, when the mobile terminal user is wearing cotton gloves), the mobile terminal user cannot operate the screen of the mobile terminal with a finger at this time, and cannot control the scroll up and down of the ScrollView control. In addition, in other scenes, the screen of the mobile terminal is controlled to be scrolled and read by manual operation, for example, when a long text is viewed, the user needs to frequently slide on the screen of the mobile terminal, finger fatigue is easily caused, and the user experience is also poor.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a page scrolling control method and device, a storage medium and electronic equipment, and at least solves the technical problem that page scrolling operation is complex in the page scrolling control method provided by the related technology.
According to an aspect of an embodiment of the present invention, there is provided a page scroll control method, including: in the process of displaying a target page in terminal equipment, calling a camera in the terminal equipment to acquire a head image of a target object browsing the target page; processing the head image based on a head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture; processing the head image based on a gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil position for indicating the pupil of the target object in the eye; inputting the first processing result and the second processing result into a decision tree model to determine a watching area of the target object; and controlling the target page to automatically scroll according to the watching area.
According to another aspect of the embodiments of the present invention, there is also provided a page scrolling control device, including: the first acquiring unit is used for calling a camera in the terminal equipment to acquire a head image of a target object browsing a target page in the process of displaying the target page in the terminal equipment; the first processing unit is used for processing the head image based on a head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture; the second processing unit is used for processing the head image based on a gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil position for indicating the pupil of the target object in the eye; a first determining unit configured to input the first processing result and the second processing result into a decision tree model to determine a region of interest of the target object; and the first control unit is used for controlling the target page to automatically scroll according to the watching area.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including: a communication bus, a memory, and a processor, wherein: the communication bus is used for realizing communication connection between the processor and the memory; the memory is used for storing executable instructions; the processor is configured to execute the page scrolling control program in the memory to implement the following steps: in the process of displaying a target page in terminal equipment, calling a camera in the terminal equipment to acquire a head image of a target object browsing the target page; processing the head image based on a head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture; processing the head image based on a gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil position for indicating the pupil of the target object in the eye; inputting the first processing result and the second processing result into a decision tree model to determine a watching area of the target object; and controlling the target page to automatically scroll according to the watching area.
According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to execute the above page scrolling control method when running.
In the embodiment of the invention, in the process of displaying the target page in the terminal equipment, a camera in the terminal equipment is called to obtain the head image of the target object of the target page being browsed; processing the head image based on a head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture; processing the head image based on a gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil location for indicating a pupil of a target subject in an eye; inputting the first processing result and the second processing result into a decision tree model to determine a gazing area of the target object; and controlling the target page to automatically scroll according to the watching area. The method comprises the steps of determining a watching area of a target object based on a rotation vector of the head of the target object in the current posture and a target pupil position of the pupil of the target object in the eye, controlling a target page to automatically roll through the watching area, and replacing a manual operation rolling control to flexibly control the target page to automatically roll.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a schematic diagram of an application environment of an alternative page scrolling control method according to an embodiment of the invention;
FIG. 2 is a flowchart illustrating an alternative page scrolling control method according to an embodiment of the invention;
FIG. 3 is a schematic diagram of head pose rotation in a three-dimensional space coordinate system of an alternative page scrolling control method according to an embodiment of the present invention;
FIG. 4 is a schematic view of a decision tree algorithm process flow of an alternative page scrolling control method according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating an alternative page scrolling control method according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of an alternative page scrolling control device according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an alternative electronic device according to an embodiment of the invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Human-computer interaction between the user 102 and the terminal device 104 in fig. 1 is possible. The terminal device 104 comprises a memory 106 for storing interaction data and a processor 108 for processing the interaction data. End device 104 may interact with backend server 114 via network 112. The background server 114 comprises a database 116 for storing interaction data and a processing engine 118 for processing the interaction data. The page scrolling control method can be executed by the terminal device 104, and can also be executed by the background server 114. For example, taking the terminal device 104 as an example, the terminal device 104 acquires the gazing area of the user 102 and controls the target page displayed on the display 110 in the terminal device 104 to automatically scroll according to the gazing area.
Alternatively, the terminal device 104 may be, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a PC, and the like, and the network 112 may include, but is not limited to, a wireless network or a wired network. Wherein, this wireless network includes: WIFI and other networks that enable wireless communication. Such wired networks may include, but are not limited to: wide area networks, metropolitan area networks, and local area networks. The background server 114 may include, but is not limited to, any hardware device capable of performing computations.
An embodiment of the present invention provides a page scrolling control method, as shown in fig. 2, the method includes, but is not limited to, the following steps:
s202: in the process of displaying a target page in terminal equipment, calling a camera in the terminal equipment to acquire a head image of a target object of the target page being browsed;
s204: processing the head image based on a head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture;
s206: processing the head image based on a gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil location for indicating a pupil of a target subject in an eye;
s208: and inputting the first processing result and the second processing result into a decision tree model to determine the gazing area of the target object.
S210: and controlling the target page to automatically scroll according to the watching area.
In step S202, in practical application, the terminal device may include, but is not limited to, at least one of the following: mobile phones (such as Android phones, IOS phones, etc.), notebook computers, tablet computers, palm computers, MID (Mobile Internet Devices), PAD, desktop computers, smart televisions, etc. The target page may be a web page in a screen in the terminal device, an electronic document, a mail, or the like, the target object is a user currently operating the terminal device, and the head image of the target object includes a two-dimensional image or a three-dimensional image.
In step S204, in practical application, the head pose estimation algorithm may include a deep learning method or a face key point projection method, which is not limited herein, and the head pose estimation mainly obtains angle information of the face orientation. Usually expressed in terms of rotation matrices, rotation vectors, quaternions or euler angles (these four quantities may also be mutually transformed). Generally, the euler angle readability is better and the use is more widespread. The face pose information is represented by three euler angles (Yaw, Pitch, Roll). Further, the head pose estimation is a pose orientation parameter for recognizing the head within one spatial coordinate system, that is, a head position parameter (x, y, z) and a direction angle parameter (Yaw, Pitch, Roll). Fig. 3 is a schematic diagram of head pose rotation in a three-dimensional space coordinate system of the page scrolling control method according to the embodiment of the invention, and the diagram describes the direction parameters of the head pose of the target object in three rotational degrees of freedom in the three-dimensional space coordinate system, namely, a horizontal rotation euler angle (Yaw)301, a vertical rotation euler angle (Pitch)302, and a left and right rotation euler angle (Roll) 303. In general, the range of head movements for an adult is: the left and right deviation angle ranges from minus 40.9 degrees to 36.3 degrees, the vertical rotation angle ranges from minus 60.4 degrees to 69.6 degrees, and the horizontal rotation angle ranges from minus 79.8 degrees to 75.3 degrees.
The face key point projection algorithm comprises a Head Pose Estimation (Head position Estimation) algorithm, and the algorithm is mainly used for solving according to the coordinates of a plurality of points of a target object in a three-dimensional space coordinate system and a transformation relation matrix between point sets projected to a two-dimensional space coordinate system correspondingly by the coordinates of the plurality of points, so that an Estimation result of the Head Pose is obtained.
The first processing result is used to indicate a rotation vector of the head of the target object in the current posture, and may be a rotation vector of a horizontal rotational euler angle (Yaw)301, a rotation vector of a vertical rotational euler angle (Pitch)302, and a left and right rotational euler angle (Roll)303 rotation vector.
In step S206, in actual application, the gaze tracking algorithm is also called gaze estimation or eye movement tracking, and is a technique of acquiring the current gaze direction or gaze point of the target object by various detection means such as electricity and optics, capturing an eye image by a camera, and analyzing the eye image to estimate the direction of sight. The embodiment of the invention can adopt but is not limited to a pupil positioning method, wherein the pupil positioning method is based on RGB images acquired by a Charge Coupled Device (CCD) camera or a Complementary Metal Oxide Semiconductor (CMOS) camera through human face geometric features, the positions of human face feature points (face outer contour, eyes, eyebrows, nose, mouth and the like) of the front face are firstly detected, then the geometric relation between the head pose and specific feature points is defined by utilizing position constraints among the feature points, and finally an inverse trigonometric function is solved to obtain the head pose angle. And obtaining the coordinates of all the characteristic points of the face of the human face, including the coordinates of the eyes. An RGB image of the eyes is obtained according to the coordinates of the eyes, and an eye image including the pupils is obtained through gray level processing and binarization processing. And calculating the position of the pupil relative to the eye by using a geometric method, and estimating the direction of the sight.
In steps S208 to S210, the decision tree model may be a Random Forest (RF) algorithm, a Concept extraction (CLS) algorithm, etc., and is not limited herein. The area watched by the target object is judged by analyzing the first result and the second result, for example, when the area watched by the target object is the lower part of the terminal device, that is, reading of the current concerned page by the target object is about to end, the page of the terminal screen automatically moves upwards, and the next page of content of the current page can be automatically switched to.
The method and the device for controlling the page scrolling of the mobile terminal determine the watching area of the target object based on the rotation vector of the head of the target object in the current posture and the target pupil position of the pupil of the target object in the eye, control the target page to automatically scroll through the watching area, can replace a manual operation scroll control to flexibly control the target page to automatically scroll, solve the technical problem that the page scrolling operation is complex in a page scrolling control method in the related technology, and further achieve the technical effects of flexibly and conveniently controlling the page scrolling and reducing the complexity of the page scrolling operation.
In an embodiment, the determining the gaze region of the target object using the input of the first processing result and the second processing result into the decision tree model comprises: analyzing the first processing result and the second processing result through the decision tree model to obtain an analysis result; and determining a gazing area of the target object according to the analysis result, wherein the gazing area is used for indicating the position of the current sight line of the target object in a screen of the terminal equipment. In this embodiment, the analysis result may be three output categories, namely, a middle screen, an upper screen and a lower screen, and when the output category is the middle screen, the terminal device detects that the user is located at a middle position above the screen of the terminal device, and the screen of the terminal device is not scrolled; when the output type is the upper part of the screen, the terminal device detects that the user is positioned at the upper part above the screen of the terminal device, and at the moment, the terminal device controls the screen page of the terminal device to scroll downwards; when the output category is the lower part of the screen, the terminal device detects that the user is at the lower part position above the screen of the terminal device, and at the moment, the terminal device controls the screen page of the terminal device to scroll upwards.
In an embodiment, controlling the target page for automatic scrolling according to the gaze region may include: under the condition that the watching area indicates that the position of the current sight line in the screen is a first position, controlling a scroll control in a target page to execute downward scroll operation; under the condition that the watching area indicates that the position of the current sight line in the screen is a second position, controlling a scroll control in the target page not to scroll; under the condition that the watching area indicates that the position of the current sight line in the screen is a third position, controlling a scroll control in the target page to execute upward scroll operation; wherein the first position is higher than the second position, and the second position is higher than the third position. In this embodiment, the first position may be an upper portion of a screen of the terminal device, the second position may be a middle portion of the screen of the terminal device, and the third position may be a lower portion of the screen of the terminal device. The page rolling direction can be flexibly controlled by setting different positions, so that a user can conveniently read the page.
In an embodiment, determining the gaze region of the target object from the analysis result comprises: determining the position of the current sight line of the target object in the screen as a second position under the condition that the analysis result indicates that the target object blinks; determining the position of the current sight line of the target object in the screen as a first position under the condition that the analysis result indicates that the target object does not blink and the vertical coordinate of the target pupil position of the target object is greater than a first threshold value; determining the position of the current sight line of the target object in the screen as a first position under the condition that the analysis result indicates that the target object does not blink, the vertical coordinate of the target pupil position of the target object is smaller than or equal to a first threshold value, and the vertical rotation vector of the head of the target object is larger than a second threshold value; determining the position of the current sight line of the target object in the screen to be a third position under the condition that the analysis result indicates that the target object does not blink, and the vertical coordinate of the target pupil position of the target object is smaller than a third threshold value; and under the condition that the analysis result indicates that the target object does not blink, the vertical coordinate of the target pupil position of the target object is greater than or equal to a third threshold value, and the vertical rotation vector of the head of the target object is smaller than a fourth threshold value, determining that the position of the current sight line of the target object in the screen is a third position.
In this embodiment, for example, the first threshold and the second threshold are positive numbers, and the third threshold and the fourth threshold are negative numbers, that is, when the position of the pupil is equal to or greater than the first threshold, the coordinate of the target pupil position is in the positive direction of the Y axis, and the line of sight of the target object moves upward; when the position of the pupil is less than or equal to the third threshold value, the coordinate of the target pupil position is in the negative direction of the Y axis, and the sight line of the target object moves downwards at the moment; when the vertical rotation vector of the head of the target object is larger than or equal to a second threshold value, the target object is in a head-up moving state; when the vertical rotation vector of the head of the target object is smaller than or equal to the fourth threshold, the target object is in a head-down moving state, where the target object may be a user of the current terminal device.
In this embodiment, a gazing area of a target object is determined through a decision tree model algorithm, for example, as shown in fig. 4, first, in step S402, it is determined whether the target object blinks, if so, the step S404 is performed, the position of the gazing area is output as the middle of a screen, that is, a current screen page does not roll, if not, the step S406 is performed, it is determined whether a vertical coordinate of a pupil of the target object is greater than a preset value n, if so, the step S408 is performed, and the position of the gazing area is output as the upper part of the screen; if not, executing step S410, determining whether the head Pitch parameter of the target object, that is, whether the vertical rotation vector of the target object is greater than a preset value k, if so, determining that the target object is in a head-up state, and then, outputting the position of the gazing area as the upper part of the screen in step S412; if not, turning to step S414, judging whether the vertical coordinate of the pupil of the target object is smaller than a preset value m, if so, turning to step S416, and outputting the position of the gazing area as the lower part of the screen; if not, executing the step S418, determining whether the head Pitch parameter of the target object, that is, the vertical rotation vector of the target object is smaller than the preset value j, if so, determining that the target object is in a head-down state, entering the step S420, and outputting the position of the gazing area as the lower part of the screen; if not, the process goes to step S422, and the position of the gazing area is output as the middle of the screen. By utilizing the decision tree model to track the head posture of the target object and the change of the pupil position, the screen rolling operation of the user can be simplified, and the user can read conveniently.
In one embodiment, step S204 may include the steps of: acquiring a face model containing a plurality of key points from a head image, and determining three-dimensional coordinates of the key points in a three-dimensional space coordinate system; and determining a rotation Euler angle of the head of the target object according to the three-dimensional coordinates based on a head posture estimation algorithm, wherein the rotation Euler angle is used for representing a rotation vector of the head of the target object. For example, a 3D face model with n keypoints is provided, n being defined according to the actual accuracy requirement. In general, a common value for n may be 68.
In one embodiment, determining the euler angle of rotation of the head of the target object according to the three-dimensional coordinates based on the head pose estimation algorithm comprises: converting the three-dimensional coordinates into three two-dimensional plane coordinates; and determining a vertical rotation Euler angle of the head of the target object according to the two-dimensional plane coordinates of the vertical dimension, wherein the vertical rotation Euler angle is used for indicating a vertical rotation vector of the head of the target object, and the rotation vector comprises the vertical rotation vector. As shown in fig. 3, the rotation vector of the euler angle (Pitch)302 is vertically rotated, and it is possible to determine whether the target object is head up or head down and head up by determining the vertical rotation vector.
In one embodiment, step S206 may include the steps of: acquiring an eye model containing a plurality of key points from a head image; determining the target pupil position of the pupil of the target object by using the eye model; judging the blink state of the target object; a second processing result is derived based on the target pupil position and the blink status. In the present embodiment, an eye model including a plurality of key points is acquired from the plurality of key points in the head image; for example, 20 key points are selected from 68 key points set in the head image as key points of the eye model, and then a target pupil position where the target object is located is obtained according to the key points of the eye model, for example, a two-dimensional plane coordinate system is established with an eye center point of the target object as an axis, and the target pupil position may be any point in the plane coordinate system. In this embodiment, the second processing result includes blink state information of the user and the position of the target pupil.
In one embodiment, determining the target pupil position of the pupil of the target object by using the eye model includes: acquiring an RGB image corresponding to the eye model; performing image processing on the RGB image to obtain an eye image containing the pupil of the target object; a target pupil location at which a pupil is located is determined based on the eye image. In this embodiment, the processing of the RGB image of the eye model object may be a binarization processing of the image, and the white part of the eye of the target object is removed, that is, the pupil position of the target object may be obtained.
The embodiment of the invention determines the watching area of the target object based on the rotation vector of the head of the target object in the current posture and the target pupil position of the pupil of the target object in the eye, controls the target page to automatically roll through the watching area, can replace a manual operation rolling control to flexibly control the target page to automatically roll, solves the technical problem that the page rolling operation is complex in a page rolling control method in the related technology, and further achieves the technical effects of flexibly and conveniently controlling the page rolling and reducing the complexity of the page rolling operation.
In an application embodiment, as shown in fig. 5, when a target object performs a page scroll control operation, step S502 is first executed, a front camera of a terminal device is started to capture a head image of a user, then step S504 is executed based on the head image to perform head pose estimation, step S506 is executed to perform gaze tracking algorithm estimation, step S508 is executed based on the execution results of step S504 and step S506, the head image is input into a decision tree algorithm model to be processed, step S510 is then executed to determine a gaze position of the target object, step S512 is executed when the gaze position is above a screen, a control page is scrolled downward, step S514 is executed when the gaze position is below the screen, and step S516 is executed when the gaze position is in the middle of the screen, and no scroll occurs in the control page.
In step S504, the head pose estimation algorithm may include the following steps:
1) first, a 3D face model with n key points is set, where n may define the accuracy class of the page scrolling operation with the target object. For example, n is set to 68 using a common value.
2) And determining 2D face key points corresponding to the 3D face model by using a face detection technology and face key point detection.
3) And analyzing the rotation vector of the 3D face model of the target object, namely the change vector of three dimensional directions such as a horizontal rotation Euler angle (Yaw), a vertical rotation Euler angle (Pitch), a left and right rotation Euler angle (Roll) and the like.
4) And converting the three-dimensional rotation vectors into respective Euler angles.
5) A vertical rotational euler angle (Pitch) of the target subject's head is obtained.
In step S506, the gaze tracking algorithm may include the steps of:
a) first, left and right eye coordinates of a 2D face key point corresponding to the target object in the two-dimensional coordinate system are calculated according to the key points in the 3D face model in step S504.
b) RGB images of left and right eyes of a target object are acquired.
c) And carrying out gray processing on the eye RGB image.
d) And performing binarization processing on the RGB image subjected to the graying processing.
e) And judging whether the target object blinks or not according to whether key points of upper and lower eyelids of the target object coincide or not and whether key points of pupils appear or not.
f) And calculating the position of the pupil key point of the target object in the two-dimensional coordinate system.
g) And f, calculating the vertical coordinate of the pupil of the target object relative to the eye of the target object according to the position in the step f.
According to the method and the device, the watching area of the target object is determined through the rotation vector of the head of the target object in the current posture and the target pupil position of the pupil of the target object in the eye, the target page is controlled to automatically roll through the watching area, the automatic rolling of the target page can be flexibly controlled by replacing a manual rolling control, the technical problem that the page rolling operation is complex in a page rolling control method in the related technology is solved, and the technical effects of flexibly and conveniently controlling the page rolling and reducing the complexity of the page rolling operation are achieved.
Based on the foregoing embodiment, the present invention further provides a page scrolling control device. As shown in fig. 6, the apparatus includes:
a first obtaining unit 602, configured to, in a process of displaying a target page in a terminal device, invoke a camera in the terminal device to obtain a head image of a target object of the target page being browsed;
a first processing unit 604, configured to process the head image based on a head pose estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture;
a second processing unit 606, configured to process the head image based on the gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil location for indicating a pupil of a target subject in an eye;
a first determining unit 608, configured to input the first processing result and the second processing result into the decision tree model to determine a gazing area of the target object;
a first control unit 610, configured to control the target page to automatically scroll according to the gazing area.
In an embodiment, the first determining unit 608 is specifically configured to analyze the first processing result and the second processing result through a decision tree model to obtain an analysis result; and determining a gazing area of the target object according to the analysis result, wherein the gazing area is used for indicating the position of the current sight line of the target object in a screen of the terminal equipment.
In an embodiment, the first determining unit 608 is specifically configured to, in a case that the gaze area indicates that the position of the current line of sight in the screen is the first position, control the scroll control in the target page to perform a downward scroll operation. Under the condition that the watching area indicates that the position of the current sight line in the screen is a second position, controlling a scroll control in the target page not to scroll; under the condition that the watching area indicates that the position of the current sight line in the screen is a third position, controlling a scroll control in the target page to execute upward scroll operation; wherein the first position is higher than the second position, and the second position is higher than the third position.
In an embodiment, the first determining unit 608 is specifically configured to determine, when the analysis result indicates that the target object blinks, that the position of the current line of sight of the target object in the screen is the second position;
determining the position of the current sight line of the target object in the screen as a first position under the condition that the analysis result indicates that the target object does not blink and the vertical coordinate of the target pupil position of the target object is greater than a first threshold value;
under the condition that the analysis result indicates that the vertical coordinate of the target pupil position of the target object is smaller than or equal to a first threshold value and the vertical rotation vector of the head of the target object is larger than a second threshold value, determining that the position of the current sight line of the target object in the screen is a first position;
determining the position of the current sight line of the target object in the screen as a third position under the condition that the analysis result indicates that the target object does not blink and the vertical coordinate of the target pupil position of the target object is smaller than a third threshold value;
determining the position of the current sight line of the target object in the screen to be a third position under the condition that the analysis result indicates that the vertical coordinate of the target pupil position of the target object is greater than or equal to a third threshold value and the vertical rotation vector of the head of the target object is smaller than a fourth threshold value;
and determining the position of the current sight line of the target object in the screen as a second position when the analysis result indicates that the vertical rotation vector of the head of the target object is smaller than or equal to a second threshold and is larger than or equal to a fourth threshold.
In an embodiment, the first processing unit 604 is specifically configured to obtain a face model including a plurality of key points from the head image, and determine three-dimensional coordinates of the plurality of key points in a three-dimensional space coordinate system; and determining a rotation Euler angle of the head of the target object according to the three-dimensional coordinates based on a head posture estimation algorithm, wherein the rotation Euler angle is used for representing a rotation vector of the head of the target object.
In an embodiment, the first processing unit 604 is further specifically configured to convert the three-dimensional coordinates into three two-dimensional plane coordinates; and determining a vertical rotation Euler angle of the head of the target object according to the two-dimensional plane coordinates of the vertical dimension, wherein the vertical rotation Euler angle is used for indicating a vertical rotation vector of the head of the target object, and the rotation vector comprises the vertical rotation vector.
In an embodiment, the second processing unit 606 is specifically configured to obtain an eye model including a plurality of key points from the head image; determining the target pupil position of the pupil of the target object by using the eye model; judging the blink state of the target object; a second processing result is derived based on the target pupil position and the blink status.
In an embodiment, the second processing unit 606 specifically obtains an RGB image corresponding to the eye model; performing image processing on the RGB image to obtain an eye image containing the pupil of the target object; a target pupil location at which a pupil is located is determined based on the eye image.
According to the method and the device, the watching area of the target object is determined through the rotation vector of the head of the target object in the current posture and the target pupil position of the pupil of the target object in the eye, the target page is controlled to automatically roll through the watching area, the automatic rolling of the target page can be flexibly controlled by replacing a manual rolling control, the technical problem that the page rolling operation is complex in a page rolling control method in the related technology is solved, and the technical effects of flexibly and conveniently controlling the page rolling and reducing the complexity of the page rolling operation are achieved.
According to another aspect of the embodiment of the present invention, there is also provided an electronic device for implementing the above page scrolling control method. As shown in fig. 7, the electronic device comprises a memory 702 and a processor 704, the memory 702 having stored therein a computer program, the processor 704 being arranged to perform the steps of any of the above-described method embodiments by means of the computer program.
Optionally, in this embodiment, the electronic device may be located in at least one network device of a plurality of network devices of a computer network.
Alternatively, in this embodiment, the processor 704 may be configured to execute the following steps by a computer program:
s1, in the process of displaying the target page in the terminal equipment, calling a camera in the terminal equipment to acquire a head image of a target object browsing the target page;
s2, processing the head image based on the head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture;
s3, processing the head image based on the gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil location for indicating a pupil of a target subject in an eye;
s4, inputting the first processing result and the second processing result into a decision tree model to determine the gazing area of the target object;
and S5, automatically scrolling the control target page according to the watching area.
Alternatively, it can be understood by those skilled in the art that the structure shown in fig. 7 is only an illustration, and the electronic device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 7 is a diagram illustrating a structure of the electronic device. For example, the electronics may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 7, or have a different configuration than shown in FIG. 7.
The memory 702 may be used to store software programs and modules, such as program instructions/modules corresponding to the page scrolling control method and apparatus in the embodiments of the present invention, and the processor 704 executes various functional applications and data processing by running the software programs and modules stored in the memory 702, that is, implements the page scrolling control method. The memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 702 can further include memory located remotely from the processor 704, which can be connected to the terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 702 may be, but not limited to, information such as a head image for the target object. As an example, as shown in fig. 7, the memory 702 may include, but is not limited to, the first obtaining unit 602, the first processing unit 604, the second processing unit 606, the first determining unit 608, and the first controlling unit 610 of the page scrolling control apparatus. In addition, other module units in the page scrolling control device may also be included, but are not limited to, and are not described in detail in this example.
Optionally, the transmitting device 706 is used for receiving or sending data via a network. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 706 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 706 is a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
In addition, the electronic device further includes: a display 708 for displaying a scrolling state of the page; and a connection bus 710 for connecting the respective module parts in the above-described electronic apparatus.
In other embodiments, the terminal may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting a plurality of nodes through a network communication. Nodes can form a Peer-To-Peer (P2P, Peer To Peer) network, and any type of computing device, such as a server, a terminal, and other electronic devices, can become a node in the blockchain system by joining the Peer-To-Peer network.
According to the method and the device, the watching area of the target object is determined through the rotation vector of the head of the target object in the current posture and the target pupil position of the pupil of the target object in the eye, the target page is controlled to automatically roll through the watching area, the automatic rolling of the target page can be flexibly controlled by replacing a manual rolling control, the technical problem that the page rolling operation is complex in a page rolling control method in the related technology is solved, and the technical effects of flexibly and conveniently controlling the page rolling and reducing the complexity of the page rolling operation are achieved.
Based on the above embodiments, the present invention further provides a computer program product or a computer program, where the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the page scrolling control method provided by one or more of the technical solutions.
Alternatively, in the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:
s1, in the process of displaying the target page in the terminal equipment, calling a camera in the terminal equipment to acquire a head image of a target object browsing the target page;
s2, processing the head image based on the head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture;
s3, processing the head image based on the gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil location for indicating a pupil of a target subject in an eye;
s4, inputting the first processing result and the second processing result into a decision tree model to determine the gazing area of the target object;
and S5, automatically scrolling the control target page according to the watching area.
Alternatively, in this embodiment, a person skilled in the art may understand that all or part of the steps in the methods of the foregoing embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing one or more computer devices (which may be personal computers, servers, network devices, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims (11)

1. A page scroll control method, the method comprising:
in the process of displaying a target page in terminal equipment, calling a camera in the terminal equipment to acquire a head image of a target object browsing the target page;
processing the head image based on a head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture;
processing the head image based on a gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil location for indicating a pupil of the target subject in the eye;
inputting the first processing result and the second processing result into a decision tree model to determine a gazing area of the target object;
and controlling the target page to automatically scroll according to the watching area.
2. The method of claim 1, wherein inputting the first processing result and the second processing result into a decision tree model to determine the target object's gaze region comprises:
analyzing the first processing result and the second processing result through the decision tree model to obtain an analysis result;
determining the gazing area of the target object according to the analysis result, wherein the gazing area is used for indicating the position of the current sight line of the target object in a screen of the terminal equipment.
3. The method of claim 2, wherein controlling the target page for automatic scrolling according to the gaze region comprises:
under the condition that the watching area indicates that the position of the current sight line in the screen is a first position, controlling a scroll control in the target page to execute a downward scroll operation;
under the condition that the watching area indicates that the position of the current sight line in the screen is a second position, controlling a scroll control in the target page not to scroll;
under the condition that the watching area indicates that the position of the current sight line in the screen is a third position, controlling a scroll control in the target page to execute an upward scroll operation;
wherein the first position is higher than the second position, which is higher than the third position.
4. The method of claim 3, wherein determining the gaze region of the target object from the analysis results comprises:
determining the position of the current sight line of the target object in the screen as the second position if the analysis result indicates that the target object is blinking;
determining the position of the current sight line of the target object in the screen as a first position under the condition that the analysis result indicates that the target object does not blink and the vertical coordinate of the target pupil position of the target object is larger than a first threshold value;
determining the position of the current sight line of the target object in the screen as the first position under the condition that the analysis result indicates that the target object does not blink, the vertical coordinate of the target pupil position of the target object is smaller than or equal to the first threshold value, and the vertical rotation vector of the head of the target object is larger than a second threshold value;
determining the position of the current sight line of the target object in the screen as a third position under the condition that the analysis result indicates that the target object does not blink and the vertical coordinate of the target pupil position of the target object is smaller than a third threshold value;
determining that the position of the current sight line of the target object in the screen is the third position under the condition that the analysis result indicates that the target object does not blink, the vertical coordinate of the target pupil position of the target object is greater than or equal to the third threshold, and the vertical rotation vector of the head of the target object is smaller than a fourth threshold;
determining the position of the current sight line of the target object in the screen as the second position if the analysis result indicates that the target object is not blinking and the vertical rotation vector of the head of the target object is less than or equal to the second threshold and greater than or equal to the fourth threshold.
5. The method of claim 1, wherein the processing the head image based on the head pose estimation algorithm to obtain a first processing result comprises:
acquiring a face model containing a plurality of key points from the head image, and determining three-dimensional coordinates of the key points in a three-dimensional space coordinate system;
determining a rotational Euler angle of the head of the target object according to the three-dimensional coordinates based on the head pose estimation algorithm, wherein the rotational Euler angle is used for representing the rotation vector of the head of the target object.
6. The method of claim 5, wherein determining a rotational Euler angle of the head of the target object from the three-dimensional coordinates based on the head pose estimation algorithm comprises:
converting the three-dimensional coordinates into three two-dimensional plane coordinates;
determining a vertical rotational Euler angle of the head of the target object according to a two-dimensional plane coordinate of a vertical dimension, wherein the vertical rotational Euler angle is used for indicating a vertical rotational vector of the head of the target object, and the rotational vector comprises the vertical rotational vector.
7. The method of claim 1, wherein: the processing the head image based on the gaze tracking algorithm to obtain a second processing result comprises:
obtaining an eye model containing a plurality of key points from the head image;
determining the target pupil position of the pupil of the target object by using the eye model;
judging the blink state of the target object;
and obtaining the second processing result based on the target pupil position and the blink state.
8. The method of claim 7, wherein the determining the target pupil location where the pupil of the target object is located using the eye model comprises:
acquiring an RGB image corresponding to the eye model;
performing image processing on the RGB image to obtain an eye image containing the pupil of the target object;
determining the target pupil position where the pupil is located based on the eye image.
9. A page scroll control apparatus, comprising:
the first acquisition unit is used for calling a camera in the terminal equipment to acquire a head image of a target object browsing a target page in the process of displaying the target page in the terminal equipment;
the first processing unit is used for processing the head image based on a head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture;
the second processing unit is used for processing the head image based on a gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil location for indicating a pupil of the target subject in the eye;
a first determining unit, configured to input the first processing result and the second processing result into a decision tree model to determine a gazing area of the target object;
and the first control unit is used for controlling the target page to automatically scroll according to the watching area.
10. An electronic device, characterized in that the electronic device comprises: a communication bus, a memory, and a processor, wherein:
the communication bus is used for realizing communication connection between the processor and the memory;
the memory to store executable instructions;
the processor is used for executing the page scrolling control program in the memory to realize the following steps:
in the process of displaying a target page in terminal equipment, calling a camera in the terminal equipment to acquire a head image of a target object browsing the target page;
processing the head image based on a head posture estimation algorithm to obtain a first processing result; wherein the first processing result is used for indicating a rotation vector of the head of the target object in the current posture;
processing the head image based on a gaze tracking algorithm to obtain a second processing result; wherein the second processing result comprises: a target pupil location for indicating a pupil of the target subject in the eye;
inputting the first processing result and the second processing result into a decision tree model to determine a gazing area of the target object;
and controlling the target page to automatically scroll according to the watching area.
11. A computer-readable storage medium storing one or more programs, the one or more programs being executable by one or more processors to perform the steps of the page scrolling control method according to any of claims 1 to 8.
CN202011296482.1A 2020-11-18 2020-11-18 Page scrolling control method and device, storage medium and electronic equipment Active CN112416126B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011296482.1A CN112416126B (en) 2020-11-18 2020-11-18 Page scrolling control method and device, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011296482.1A CN112416126B (en) 2020-11-18 2020-11-18 Page scrolling control method and device, storage medium and electronic equipment

Publications (2)

Publication Number Publication Date
CN112416126A true CN112416126A (en) 2021-02-26
CN112416126B CN112416126B (en) 2023-07-28

Family

ID=74773480

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011296482.1A Active CN112416126B (en) 2020-11-18 2020-11-18 Page scrolling control method and device, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN112416126B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113434043A (en) * 2021-06-30 2021-09-24 青岛海尔科技有限公司 Page turning control method and device, storage medium and electronic device
CN115953813A (en) * 2022-12-19 2023-04-11 北京字跳网络技术有限公司 Expression driving method, device, equipment and storage medium

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103064520A (en) * 2013-01-31 2013-04-24 东莞宇龙通信科技有限公司 Mobile terminal and method for controlling page to roll by same
JP2016173313A (en) * 2015-03-17 2016-09-29 国立大学法人鳥取大学 Visual line direction estimation system, visual line direction estimation method and visual line direction estimation program
CN106355147A (en) * 2016-08-26 2017-01-25 张艳 Acquiring method and detecting method of live face head pose detection regression apparatus
CN106598221A (en) * 2016-11-17 2017-04-26 电子科技大学 Eye key point detection-based 3D sight line direction estimation method
CN110231871A (en) * 2019-06-14 2019-09-13 腾讯科技（深圳）有限公司 Page reading method, device, storage medium and electronic equipment
CN110333779A (en) * 2019-06-04 2019-10-15 Oppo广东移动通信有限公司 Control method, terminal and storage medium
CN110532933A (en) * 2019-08-26 2019-12-03 淮北师范大学 A kind of living body faces detection head pose returns the acquisition methods and detection method of device
CN110909611A (en) * 2019-10-29 2020-03-24 深圳云天励飞技术有限公司 Method and device for detecting attention area, readable storage medium and terminal equipment
CN111046744A (en) * 2019-11-21 2020-04-21 深圳云天励飞技术有限公司 Method and device for detecting attention area, readable storage medium and terminal equipment
CN111586459A (en) * 2020-05-22 2020-08-25 北京百度网讯科技有限公司 Method and device for controlling video playing, electronic equipment and storage medium

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103064520A (en) * 2013-01-31 2013-04-24 东莞宇龙通信科技有限公司 Mobile terminal and method for controlling page to roll by same
JP2016173313A (en) * 2015-03-17 2016-09-29 国立大学法人鳥取大学 Visual line direction estimation system, visual line direction estimation method and visual line direction estimation program
CN106355147A (en) * 2016-08-26 2017-01-25 张艳 Acquiring method and detecting method of live face head pose detection regression apparatus
CN106598221A (en) * 2016-11-17 2017-04-26 电子科技大学 Eye key point detection-based 3D sight line direction estimation method
CN110333779A (en) * 2019-06-04 2019-10-15 Oppo广东移动通信有限公司 Control method, terminal and storage medium
CN110231871A (en) * 2019-06-14 2019-09-13 腾讯科技（深圳）有限公司 Page reading method, device, storage medium and electronic equipment
CN110532933A (en) * 2019-08-26 2019-12-03 淮北师范大学 A kind of living body faces detection head pose returns the acquisition methods and detection method of device
CN110909611A (en) * 2019-10-29 2020-03-24 深圳云天励飞技术有限公司 Method and device for detecting attention area, readable storage medium and terminal equipment
CN111046744A (en) * 2019-11-21 2020-04-21 深圳云天励飞技术有限公司 Method and device for detecting attention area, readable storage medium and terminal equipment
CN111586459A (en) * 2020-05-22 2020-08-25 北京百度网讯科技有限公司 Method and device for controlling video playing, electronic equipment and storage medium

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113434043A (en) * 2021-06-30 2021-09-24 青岛海尔科技有限公司 Page turning control method and device, storage medium and electronic device
CN115953813A (en) * 2022-12-19 2023-04-11 北京字跳网络技术有限公司 Expression driving method, device, equipment and storage medium
CN115953813B (en) * 2022-12-19 2024-01-30 北京字跳网络技术有限公司 Expression driving method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN112416126B (en) 2023-07-28

Similar Documents

Publication Publication Date Title
CN110766777A (en) Virtual image generation method and device, electronic equipment and storage medium
JP2019535055A (en) Perform gesture-based operations
WO2013118458A1 (en) Image processing device, and computer program product
CN112288665B (en) Image fusion method and device, storage medium and electronic equipment
KR102461232B1 (en) Image processing method and apparatus, electronic device, and storage medium
WO2021143282A1 (en) Three-dimensional facial model generation method and apparatus, computer device and storage medium
US20160371888A1 (en) Interactive information display
CN113227878A (en) Method and system for gaze estimation
CN109215131B (en) Virtual face driving method and device
CN112416126B (en) Page scrolling control method and device, storage medium and electronic equipment
CN112083801A (en) Gesture recognition system and method based on VR virtual office
CN103581423A (en) Method and device for turning over screen and mobile terminal
CN113470112A (en) Image processing method, image processing device, storage medium and terminal
CN112927259A (en) Multi-camera-based bare hand tracking display method, device and system
CN112101109B (en) Training method and device for face key point detection model, electronic equipment and medium
CN105022480A (en) Input method and terminal
GB2525304B (en) Interactive information display
CN113342157A (en) Eyeball tracking processing method and related device
CN106200911A (en) A kind of motion sensing control method based on dual camera, mobile terminal and system
CN110941327A (en) Virtual object display method and device
CN115147193A (en) Commodity purchasing recommendation method and device, electronic purchasing terminal and medium
CN112183155B (en) Method and device for establishing action posture library, generating action posture and identifying action posture
CN112560553A (en) Living body detection method, living body detection device, and storage medium
CN111103967A (en) Control method and device of virtual object
CN111524240A (en) Scene switching method and device and augmented reality equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant