CN106101376B

CN106101376B - Message pushing device and method and mobile terminal

Info

Publication number: CN106101376B
Application number: CN201610357224.7A
Authority: CN
Inventors: 张菱垚; 徐群
Original assignee: Nubia Technology Co Ltd
Current assignee: Nubia Technology Co Ltd
Priority date: 2016-05-26
Filing date: 2016-05-26
Publication date: 2021-01-05
Anticipated expiration: 2036-05-26
Also published as: CN106101376A

Abstract

The invention discloses a message pushing device, a message pushing method and a mobile terminal, and relates to the technical field of mobile terminals, wherein the message pushing device comprises the following steps: the acceleration acquisition module is used for acquiring the motion state of eyeballs of the user to obtain the acceleration of eyeball rotation; the acceleration judging module is used for judging whether the acceleration is greater than a preset threshold value or not; the focus acquisition module is used for acquiring focus positions of two eyeballs of the user when the acceleration is smaller than a preset threshold value; the focus judgment module is used for judging whether the focus position is in a preset area or not; if yes, the user is interested in the currently viewed content; the message pushing module is used for classifying the currently viewed content so as to facilitate message pushing to the user.

Description

Message pushing device and method and mobile terminal

Technical Field

The present invention relates to the field of mobile terminals, and in particular, to a message pushing device, a message pushing method, and a mobile terminal.

Background

The message pushing is a technology which is commonly used in the internet, namely, the message pushing on the smart phone terminal is that the server actively pushes the message to the equipment of the user and informs the user in a notification mode. The technology is important on the intelligent terminal, and has great advantages in the aspects of resource occupation, power consumption and the like of intelligent equipment.

The existing message pushing modes measure the pushing range according to three dimensions of content, time and a user, generally speaking, the interest degree of the user in a message is determined according to the use habit of the learning user, usually, the browsing record of the user is recorded, and the method can push correct messages to the user to a certain extent. However, it is not enough to infer the user's interest level based on the user's usage habits, for example, when the user uses a news or blog client, sometimes only by mechanical operations of wearing out time, a connection is opened and then closed, the user is actually not interested in the content, but the system may misunderstand that the user is interested in the content, and the user's unintended usage habits interfere with the decision making.

Disclosure of Invention

The invention mainly aims to provide a message pushing device, a message pushing method and a mobile terminal, which can determine whether a user is interested in currently watched content or not by acquiring two factors, namely the motion state and the focus of eyeballs when the user watches a screen of the mobile terminal, so that a decision of message pushing is made, and the message pushing accuracy is improved.

In order to achieve the above object, the present invention provides a message pushing apparatus, including:

the acceleration acquisition module is used for acquiring the motion state of eyeballs of a user and acquiring the rotation acceleration of the eyeballs according to the motion state of the eyeballs;

the acceleration judging module is used for judging whether the acceleration is greater than a preset threshold value or not, and if so, the acceleration shows that the user is not interested in the currently watched content;

the focus acquisition module is used for acquiring focus positions of two eyeballs of the user when the acceleration is smaller than a preset threshold value;

the focus judgment module is used for judging whether the focus position is in a preset area or not, and if not, the focus judgment module indicates that the user is not interested in the currently watched content; if yes, the user is interested in the currently viewed content;

and the message pushing module is used for classifying the currently viewed content so as to facilitate message pushing to the user.

Optionally, the acceleration acquisition module includes:

an extraction unit configured to extract an eyeball image of a user in time series;

and the acceleration calculating unit is used for comparing the difference value of the eyeball image at each time point with the eyeball image at the previous time point and/or the next time point and calculating the acceleration of eyeball rotation.

Optionally, the focus acquisition module includes:

the reference unit is used for taking two eyeballs as a reference when the screen of the mobile terminal is in head view, wherein the positions of the eye pupils are located at the midpoint of the eyeballs, and the center positions of the two eye pupils are selected to form an eyeball reference plane parallel to the screen;

the included angle acquisition unit is used for acquiring the central positions of two eye pupils according to the eyeball image of each time point, and respectively acquiring the included angles between the two eye pupils and the eyeball reference plane: a left eye pupil angle and a right eye pupil angle;

and the focus calculation unit is used for calculating the focus positions of the two eyeballs through the central positions of the two eye pupils, the left eye pupil included angle, the right eye pupil included angle and the distance between the screen and the eyeball reference plane.

Optionally, the calculation formula for obtaining the included angle θ between the two eye pupils and the eyeball reference plane is as follows:

θ＝arccos(L2/L1)

where L1 is the horizontal width of the eye pupil when the eye looks straight ahead, and L2 is the actually measured horizontal width of the eye pupil.

As another aspect of the present invention, there is provided a message pushing method, including the steps of:

acquiring the motion state of eyeballs of a user, and acquiring the rotation acceleration of the eyeballs according to the motion state of the eyeballs;

judging whether the acceleration is larger than a preset threshold value or not, if so, indicating that the user is not interested in the currently watched content;

otherwise, acquiring the focal positions of two eyeballs of the user;

judging whether the focus position is in a preset area, if not, indicating that the user is not interested in the currently watched content;

if yes, the user is interested in the currently viewed content;

and classifying the currently viewed content so as to facilitate message pushing to the user.

Optionally, the obtaining the motion state of the eyeballs of the user, and the obtaining the acceleration of the eyeball rotation according to the motion state of the eyeballs includes:

extracting eyeball images of a user according to a time sequence;

and comparing the difference value of the eyeball image of each time point with the eyeball image of the previous time point and/or the next time point, and calculating to obtain the acceleration of eyeball rotation.

Optionally, the acquiring the focal positions of the two eyeballs of the user includes:

taking two eyeballs as a reference when the screen of the mobile terminal is in head view, wherein the positions of the eye pupils are located at the midpoint of the eyeballs, and selecting the center positions of the two eye pupils to form an eyeball reference plane parallel to the screen;

according to the eyeball image of each time point, the center positions of two eye pupils are obtained, and the included angles between the two eye pupils and the eyeball reference plane are respectively obtained: a left eye pupil angle and a right eye pupil angle;

and calculating the focal positions of the two eyeballs through the central positions of the two eye pupils, the included angle of the left eye pupil and the included angle of the right eye pupil and the distance between the screen and the eyeball reference plane.

θ＝arccos(L2/L1)

Optionally, before the classifying the currently viewed content to facilitate the message pushing to the user, the method further includes:

and acquiring the eye print information of the user, and associating the eye print information as identity information of the user with the currently viewed content.

As a further aspect of the present invention, there is provided a mobile terminal including any one of the above devices.

The invention provides a message pushing device, a message pushing method and a mobile terminal, wherein the device comprises: the acceleration acquisition module is used for acquiring the motion state of eyeballs of a user and acquiring the rotation acceleration of the eyeballs according to the motion state of the eyeballs; the acceleration judging module is used for judging whether the acceleration is greater than a preset threshold value or not, and if so, the acceleration shows that the user is not interested in the currently watched content; the focus acquisition module is used for acquiring focus positions of two eyeballs of the user when the acceleration is smaller than a preset threshold value; the focus judgment module is used for judging whether the focus position is in a preset area or not, and if not, the focus judgment module indicates that the user is not interested in the currently watched content; if yes, the user is interested in the currently viewed content; the message pushing module is used for classifying the currently viewed content so as to facilitate message pushing to the user.

Drawings

Fig. 1 is a schematic diagram of a hardware structure of a mobile terminal implementing various embodiments of the present invention;

FIG. 2 is a diagram of a wireless communication system for the mobile terminal shown in FIG. 1;

fig. 3 is a block diagram illustrating an exemplary structure of a message pushing apparatus according to an embodiment of the present invention;

FIG. 4 is a schematic view of an eye looking straight ahead according to an embodiment of the present invention;

FIG. 5 is a schematic view of an eye of a user when the eye is not in a head-up view according to an embodiment of the present invention;

fig. 6 is a schematic view illustrating a focus position determination according to an embodiment of the present invention;

fig. 7 is a schematic view illustrating another focus position determination according to an embodiment of the present invention;

fig. 8 is a schematic view illustrating a focus position determination according to another embodiment of the present invention;

fig. 9 is a block diagram illustrating an exemplary structure of a message pushing apparatus according to a second embodiment of the present invention;

fig. 10 is a block diagram illustrating an exemplary structure of a message pushing apparatus according to a third embodiment of the present invention;

fig. 11 is a flowchart of a message pushing method according to a fourth embodiment of the present invention;

fig. 12 is a flowchart of a message pushing method according to a fifth embodiment of the present invention;

fig. 13 is a flowchart of a message pushing method according to a sixth embodiment of the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

A mobile terminal implementing various embodiments of the present invention will now be described with reference to the accompanying drawings. In the following description, suffixes such as "module", "part", or "unit" used to denote elements are used only for facilitating the description of the present invention, and have no specific meaning by themselves. Thus, "modules" and "components" may be used in a mixture.

The mobile terminal may be implemented in various forms. For example, the terminal described in the present invention may include a mobile terminal such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. In the following, it is assumed that the terminal is a mobile terminal. However, it will be understood by those skilled in the art that the configuration according to the embodiment of the present invention can be applied to a fixed type terminal in addition to elements particularly used for moving purposes.

Fig. 1 is a schematic diagram of a hardware structure of an optional mobile terminal for implementing various embodiments of the present invention.

The mobile terminal 100 may include a wireless communication unit 110, an a/V (audio/video) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, an interface unit 170, a controller 180, and a power supply unit 190, etc. Fig. 1 illustrates a mobile terminal having various components, but it is to be understood that not all illustrated components are required to be implemented. More or fewer components may alternatively be implemented. Elements of the mobile terminal will be described in detail below.

The wireless communication unit 110 typically includes one or more components that allow radio communication between the mobile terminal 100 and a wireless communication system or network. For example, the wireless communication unit may include at least one of a mobile communication module 112, a wireless internet module 113, and a short-range communication module 114.

The mobile communication module 112 transmits and/or receives radio signals to and/or from at least one of a base station (e.g., access point, node B, etc.), an external terminal, and a server. Such radio signals may include voice call signals, video call signals, or various types of data transmitted and/or received according to text and/or multimedia messages.

The wireless internet module 113 supports wireless internet access of the mobile terminal. The module may be internally or externally coupled to the terminal. The wireless internet access technology to which the module relates may include WLAN (wireless LAN) (Wi-Fi), Wibro (wireless broadband), Wimax (worldwide interoperability for microwave access), HSDPA (high speed downlink packet access), and the like.

The short-range communication module 114 is a module for supporting short-range communication. Some examples of short-range communication technologies include bluetooth (TM), Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), zigbee (TM), and the like.

The a/V input unit 120 is used to receive an audio or video signal. The a/V input unit 120 may include a camera 121 and a microphone 1220, and the camera 121 processes image data of still pictures or video obtained by an image capturing apparatus in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 151. The image frames processed by the camera 121 may be stored in the memory 160 (or other storage medium) or transmitted via the wireless communication unit 110, and two or more cameras 1210 may be provided according to the construction of the mobile terminal. The microphone 122 may receive sounds (audio data) via the microphone in a phone call mode, a recording mode, a voice recognition mode, or the like, and can process such sounds into audio data. The processed audio (voice) data may be converted into a format output transmittable to a mobile communication base station via the mobile communication module 112 in case of a phone call mode. The microphone 122 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting audio signals.

The user input unit 130 may generate key input data according to a command input by a user to control various operations of the mobile terminal. The user input unit 130 allows a user to input various types of information, and may include a keyboard, dome sheet, touch pad (e.g., a touch-sensitive member that detects changes in resistance, pressure, capacitance, and the like due to being touched), scroll wheel, joystick, and the like. In particular, when the touch pad is superimposed on the display unit 151 in the form of a layer, a touch screen may be formed.

The sensing unit 140 detects a current state of the mobile terminal 100 (e.g., an open or closed state of the mobile terminal 100), a position of the mobile terminal 100, presence or absence of contact (i.e., touch input) by a user with the mobile terminal 100, an orientation of the mobile terminal 100, acceleration or deceleration movement and direction of the mobile terminal 100, and the like, and generates a command or signal for controlling an operation of the mobile terminal 100. For example, when the mobile terminal 100 is implemented as a slide-type mobile phone, the sensing unit 140 may sense whether the slide-type phone is opened or closed. In addition, the sensing unit 140 can detect whether the power supply unit 190 supplies power or whether the interface unit 170 is coupled with an external device.

The interface unit 170 serves as an interface through which at least one external device is connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The identification module may store various information for authenticating a user using the mobile terminal 100 and may include a User Identity Module (UIM), a Subscriber Identity Module (SIM), a Universal Subscriber Identity Module (USIM), and the like. In addition, a device having an identification module (hereinafter, referred to as "identification device") may take the form of a smart card, and thus, the identification device may be connected with the mobile terminal 100 via a port or other connection means. The interface unit 170 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal and the external device.

In addition, when the mobile terminal 100 is connected with an external cradle, the interface unit 170 may serve as a path through which power is supplied from the cradle to the mobile terminal 100 or may serve as a path through which various command signals input from the cradle are transmitted to the mobile terminal. Various command signals or power input from the cradle may be used as signals for recognizing whether the mobile terminal is accurately mounted on the cradle. The output unit 150 is configured to provide output signals (e.g., audio signals, video signals, alarm signals, vibration signals, etc.) in a visual, audio, and/or tactile manner. The output unit 150 may include a display unit 151, an audio output module 152, and the like.

The display unit 151 may display information processed in the mobile terminal 100. For example, when the mobile terminal 100 is in a phone call mode, the display unit 151 may display a User Interface (UI) or a Graphical User Interface (GUI) related to a call or other communication (e.g., text messaging, multimedia file downloading, etc.). When the mobile terminal 100 is in a video call mode or an image capturing mode, the display unit 151 may display a captured image and/or a received image, a UI or GUI showing a video or an image and related functions, and the like.

Meanwhile, when the display unit 151 and the touch pad are overlapped with each other in the form of a layer to form a touch screen, the display unit 151 may serve as an input device and an output device. The display unit 151 may include at least one of a Liquid Crystal Display (LCD), a thin film transistor LCD (TFT-LCD), an Organic Light Emitting Diode (OLED) display, a flexible display, a three-dimensional (3D) display, and the like. Some of these displays may be configured to be transparent to allow a user to view from the outside, which may be referred to as transparent displays, and a typical transparent display may be, for example, a TOLED (transparent organic light emitting diode) display or the like. Depending on the particular desired implementation, the mobile terminal 100 may include two or more display units (or other display devices), for example, the mobile terminal may include an external display unit (not shown) and an internal display unit (not shown). The touch screen may be used to detect a touch input pressure as well as a touch input position and a touch input area.

The audio output module 152 may convert audio data received by the wireless communication unit 110 or stored in the memory 160 into an audio signal and output as sound when the mobile terminal is in a call signal reception mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output module 152 may provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output module 152 may include a speaker, a buzzer, and the like.

The memory 160 may store software programs and the like for processing and controlling operations performed by the controller 180, or may temporarily store data (e.g., a phonebook, messages, still images, videos, and the like) that has been or will be output. Also, the memory 160 may store data regarding various ways of vibration and audio signals output when a touch is applied to the touch screen.

The memory 160 may include at least one type of storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. Also, the mobile terminal 100 may cooperate with a network storage device that performs a storage function of the memory 160 through a network connection.

The controller 180 generally controls the overall operation of the mobile terminal. For example, the controller 180 performs control and processing related to voice calls, data communications, video calls, and the like. In addition, the controller 180 may include a multimedia module 1810 for reproducing (or playing back) multimedia data, and the multimedia module 1810 may be constructed within the controller 180 or may be constructed separately from the controller 180. The controller 180 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image.

The power supply unit 190 receives external power or internal power and provides appropriate power required to operate various elements and components under the control of the controller 180.

The various embodiments described herein may be implemented in a computer-readable medium using, for example, computer software, hardware, or any combination thereof. For a hardware implementation, the embodiments described herein may be implemented using at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a processor, a controller, a microcontroller, a microprocessor, an electronic unit designed to perform the functions described herein, and in some cases, such embodiments may be implemented in the controller 180. For a software implementation, the implementation such as a process or a function may be implemented with a separate software module that allows performing at least one function or operation. The software codes may be implemented by software applications (or programs) written in any suitable programming language, which may be stored in the memory 160 and executed by the controller 180.

Up to this point, mobile terminals have been described in terms of their functionality. Hereinafter, a slide-type mobile terminal among various types of mobile terminals, such as a folder-type, bar-type, swing-type, slide-type mobile terminal, and the like, will be described as an example for the sake of brevity. Accordingly, the present invention can be applied to any type of mobile terminal, and is not limited to a slide type mobile terminal.

Referring to fig. 2, the CDMA wireless communication system may include a plurality of mobile terminals 100, a plurality of Base Stations (BSs) 270, Base Station Controllers (BSCs) 275, and a Mobile Switching Center (MSC) 280. The MSC280 is configured to interface with a Public Switched Telephone Network (PSTN) 290. The MSC280 is also configured to interface with a BSC275, which may be coupled to the base station 270 via a backhaul. The backhaul may be constructed according to any of several known interfaces including, for example, E1/T1, ATM, IP, PPP, frame Relay, HDSL, ADSL, or xDSL. It will be understood that a system as shown in fig. 2 may include multiple BSCs 2750.

Each BS270 may serve one or more sectors (or regions), each sector covered by a multi-directional antenna or an antenna pointing in a particular direction being radially distant from the BS 270. Alternatively, each partition may be covered by two or more antennas for diversity reception. Each BS270 may be configured to support multiple frequency allocations, with each frequency allocation having a particular frequency spectrum (e.g., 1.25MHz,5MHz, etc.).

The intersection of partitions with frequency allocations may be referred to as a CDMA channel. The BS270 may also be referred to as a Base Transceiver Subsystem (BTS) or other equivalent terminology. In such a case, the term "base station" may be used to generically refer to a single BSC275 and at least one BS 270. The base stations may also be referred to as "cell sites". Alternatively, each sector of a particular BS270 may be referred to as a plurality of cell sites.

As shown in fig. 2, a Broadcast Transmitter (BT)295 transmits a broadcast signal to the mobile terminal 100 operating within the system. A broadcast receiving module 111 as shown in fig. 1 is provided at the mobile terminal 100 to receive a broadcast signal transmitted by the BT 295. In fig. 2, several Global Positioning System (GPS) satellites 300 are shown. The satellite 300 assists in locating at least one of the plurality of mobile terminals 100.

In fig. 2, a plurality of satellites 300 are depicted, but it is understood that useful positioning information may be obtained with any number of satellites.

As a typical operation of the wireless communication system, the BS270 receives reverse link signals from various mobile terminals 100. The mobile terminal 100 is generally engaged in conversations, messaging, and other types of communications. Each reverse link signal received by a particular base station 270 is processed within the particular BS 270. The obtained data is forwarded to the associated BSC 275. The BSC provides call resource allocation and mobility management functions including coordination of soft handoff procedures between BSs 270. The BSCs 275 also route the received data to the MSC280, which provides additional routing services for interfacing with the PSTN 290. Similarly, the PSTN290 interfaces with the MSC280, the MSC interfaces with the BSCs 275, and the BSCs 275 accordingly control the BS270 to transmit forward link signals to the mobile terminal 100.

Based on the above mobile terminal hardware structure and communication system, the present invention provides various embodiments of the method.

Example one

As shown in fig. 3, in this embodiment, a message pushing apparatus includes:

the acceleration acquisition module 10 is configured to acquire a motion state of an eyeball of a user, and obtain an acceleration of eyeball rotation according to the motion state of the eyeball;

an acceleration determining module 20, configured to determine whether the acceleration is greater than a preset threshold, and if so, indicate that the user is not interested in the currently viewed content;

a focus obtaining module 30, configured to obtain focus positions of two eyeballs of the user when the acceleration is smaller than a preset threshold;

a focus judgment module 40, configured to judge whether the focus position is within a preset area, and if not, indicate that the user is not interested in the currently viewed content; if yes, the user is interested in the currently viewed content;

and the message pushing module 50 is used for classifying the currently viewed content so as to facilitate message pushing to the user.

In the embodiment, whether the user is interested in the currently viewed content is determined by acquiring two factors, namely the motion state and the focus of the eyeball when the user watches the screen of the mobile terminal, so that the decision of message pushing is made, and the accuracy of message pushing is improved.

In this embodiment, the present invention is based on learning the behavior habits of the user, including the physiological response and the physical response of the user, and when the user uses a mobile terminal such as a mobile phone, eighty-ninety percent of the information of the user is obtained from the eyes, so the eyes are the human organs that can most represent the interest index of the user in the currently viewed content.

In this embodiment, taking a mobile phone as an example when a user uses the mobile phone, capturing a face image of the user through a front camera of the mobile phone, extracting eyeball images of the user in time sequence, comparing the eyeball image of each time point with an eyeball image of a previous time point and/or a next time point in a difference value manner, calculating acceleration of eyeball rotation of the time point, judging whether the user is focusing on watching a screen according to the acceleration, and when the acceleration is greater than a certain preset threshold, indicating that the eye movement of the user is too frequent, it may be considered that the user is not interested in the current content.

In this embodiment, since two eyeballs of a person generally move in one direction at the same time and move at substantially the same speed, in this embodiment, the acceleration of eyeball rotation refers to the rotational acceleration of either eyeball, and when the rotational acceleration of one of the left eyeball or the right eyeball is greater than a preset threshold, it indicates that the user is not interested in the currently viewed content.

As another embodiment, in consideration of the difference between the rotation directions or speeds of the two eyeballs of some people, the person with the larger rotation acceleration among the left eyeball and the right eyeball may be selected to be compared with a preset threshold; the average value of the rotation acceleration of the left eyeball and the right eyeball can be selected to be compared with a preset threshold value.

However, it is not accurate enough to determine whether the user is interested in the currently viewed content by using the acceleration alone, and if it is considered that the user is extremely careless, the eyeball may be in a state of being almost still, which cannot determine whether the user is interested in the currently viewed content, and an erroneous determination may be made only by using the above method. Therefore, for the case that the acceleration is smaller than the preset threshold, whether the user focuses on the screen is determined by judging whether the focus position is in a preset area.

In this embodiment, the message pushing apparatus further includes:

the eye pattern acquisition module is used for acquiring eye pattern information of the user and associating the eye pattern information with the currently viewed content as identity information of the user; because there may be a plurality of users of a mobile terminal, and the hobbies of each user may also be different, the identity authentication may be performed by obtaining the eye print information of the current user, and the eye print information is associated with the currently viewed content, so that more accurate push may be performed by referring to the eye print information when performing message push.

In this embodiment, the flow of the calculation method for obtaining the focal positions of two eyeballs of the user is as follows:

firstly, the observation directions of the left eye pupil and the right eye pupil are identified, under the normal condition, when the eyes look ahead, the positions of the eye pupils are positioned at the midpoint of the eyeballs, as shown by the dotted line positions in fig. 4, and at this time, the center positions of the two eye pupils are selected to form an eyeball reference plane parallel to the screen.

When the eyes are not in front of the eyes, the shape of the eye pupil detected by the front camera is not completely circular, and the left eye pupil and the right eye pupil are not in the center of the eyes, and have a certain included angle with the reference plane of the eyeball, as shown in fig. 5, the horizontal eye pupil width is L2, and assuming that the horizontal eye pupil width in front of the eyes is L1, the included angle θ between the two eye pupils and the reference plane of the eyeball is calculated as:

θ＝arccos(L2/L1)

respectively calculating an included angle between the right eye pupil and the eyeball reference plane according to the formula: the included angle between the right eye pupil and the eyeball reference plane is as follows: and calculating the focal positions of the two eyeballs through the central positions of the two eyepupils, the right eye pupil included angle alpha, the left eye pupil included angle beta, the plane where the mobile phone screen is located and the distance between the two eyeball reference planes.

As shown in fig. 6, at this time, the user looks at the mobile phone screen, point a in the drawing indicates the center position of the right eye pupil, point B indicates the center position of the left eye pupil, point C indicates the position of a certain point on the mobile phone screen, the distance between A, B and the distance from C to line AB are calculated by using a distance sensor, then a coordinate system is established according to geometric knowledge, the intersection point D of the dotted lines where the sight lines of the two eyes are located is found, namely the focus of the sight lines of the two eyes, whether the focus position coincides with the position of the certain point C on the mobile phone screen is judged, and if so, the user is attentive to watch the screen.

In this embodiment, considering the relationship of errors, the determination of whether the focus position is on the mobile phone screen may be expanded to determine whether the focus position is within a preset area, where the preset area is a range value obtained from empirical data, where the user may be considered to be focusing on watching the screen, for example, a distance from the mobile phone screen within 1cm may be considered to be focusing on watching the screen.

As shown in fig. 7, when the focal positions of the two eyeballs are free from the mobile phone screen; or as shown in fig. 8, the focal points of the two eyeballs are not focused on the screen of the mobile phone, and both eyeballs should consider that the user is not interested in the currently viewed content.

Example two

As shown in fig. 9, in the present embodiment, the acceleration acquisition module 10 includes:

an extraction unit 11 for extracting user eyeball images in time series;

and the acceleration calculating unit 12 is configured to compare the eyeball image at each time point with the eyeball image at the previous time point and/or the next time point, and calculate an acceleration of eyeball rotation.

In this embodiment, a front camera of a mobile phone captures a face image of a user, eye images of the user are extracted in time sequence, an eye image at each time point is compared with an eye image at a previous time point and/or an eye image at a next time point in a difference value manner, acceleration of eye rotation at the time point is calculated, whether the user is focusing on a screen or not is judged according to the acceleration, when the acceleration is greater than a preset threshold, it indicates that the eye movement of the user is too frequent, and the user can be considered to be uninteresting in the current content.

EXAMPLE III

As shown in fig. 10, in the present embodiment, the focus acquisition module 30 includes:

the reference unit 31 is configured to use two eyeballs as a reference when the screen of the mobile terminal is viewed horizontally, where the positions of the eye pupils are located at the midpoint of the eyeballs, and select center positions of the two eye pupils to form an eyeball reference plane parallel to the screen;

an included angle obtaining unit 32, configured to obtain center positions of two eye pupils according to the eyeball image at each time point, and obtain included angles between the two eye pupils and the eyeball reference plane respectively: a left eye pupil angle and a right eye pupil angle;

and the focal point calculating unit 33 is configured to calculate the focal point positions of the two eyeballs through the central positions of the two eye pupils, the left eye pupil included angle, the right eye pupil included angle, and the distance between the screen and the eyeball reference plane.

The calculation method for acquiring the focal positions of two eyeballs of the user comprises the following steps:

θ＝arccos(L2/L1)

Example four

As shown in fig. 11, this embodiment proposes a message pushing method, where the method includes:

s10, acquiring the movement state of the eyeball of the user, and acquiring the rotation acceleration of the eyeball according to the movement state of the eyeball;

s20, judging whether the acceleration is larger than a preset threshold value, if so, entering the step S21 to show that the user is not interested in the currently watched content;

otherwise, step S30 is performed to obtain the focal positions of the two eyeballs of the user;

s40, judging whether the focus position is in a preset area, if not, entering the step S21 to show that the user is not interested in the currently watched content;

if yes, the step S50 is carried out, and the user is shown to be interested in the currently viewed content;

and S60, classifying the currently viewed content so as to facilitate message pushing to the user.

In this embodiment, before the classifying the currently viewed content so as to facilitate the message pushing to the user, the method further includes:

acquiring the eye print information of the user, and associating the eye print information serving as identity information of the user with the currently viewed content; because there may be a plurality of users of a mobile terminal, and the hobbies of each user may also be different, the identity authentication may be performed by obtaining the eye print information of the current user, and the eye print information is associated with the currently viewed content, so that more accurate push may be performed by referring to the eye print information when performing message push.

EXAMPLE five

As shown in fig. 12, in the present embodiment, the step S10 includes:

s11, extracting eyeball images of the user according to the time sequence;

and S12, comparing the eyeball image of each time point with the eyeball image of the previous time point and/or the next time point in a difference manner, and calculating the acceleration of the eyeball rotation.

EXAMPLE six

As shown in fig. 13, in the present embodiment, the step S30 includes:

s31, taking the two eyeballs as a reference when the screen of the mobile terminal is in head view, wherein the positions of the eye pupils are located at the midpoint of the eyeballs, and selecting the center positions of the two eye pupils to form an eyeball reference plane parallel to the screen;

s32, acquiring the center positions of two eye pupils according to the eyeball image of each time point, and respectively acquiring the included angles between the two eye pupils and the eyeball reference plane: a left eye pupil angle and a right eye pupil angle;

and S33, calculating the focal positions of the two eyeballs through the central positions of the two eye pupils, the left eye pupil included angle, the right eye pupil included angle and the distance between the screen and the eyeball reference plane.

θ＝arccos(L2/L1)

EXAMPLE seven

In this embodiment, a mobile terminal includes the message pushing apparatus described in the above embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. A message push apparatus, comprising:

the focus judgment module is used for judging whether the focus position is on a mobile phone screen, and if not, the focus judgment module indicates that the user is not interested in the currently watched content; if yes, the user is interested in the currently viewed content;

2. The message pushing apparatus according to claim 1, wherein the acceleration obtaining module comprises:

3. The message pushing apparatus according to claim 2, wherein the focus obtaining module comprises:

the reference unit is used for taking the two eyeballs to look up the screen of the mobile terminal horizontally as a reference, wherein the positions of the eye pupils are positioned at the midpoint of the eyeballs, and the center positions of the two eye pupils are selected to form an eyeball reference plane parallel to the screen;

4. The message pushing apparatus according to claim 3, wherein the calculation formula for obtaining the included angle θ between the two eye pupils and the reference plane of the eyeball is:

θ＝arccos(L2/L1)

5. A message pushing method, characterized in that the method comprises the steps of:

otherwise, acquiring the focal positions of two eyeballs of the user;

judging whether the focus position is on a mobile phone screen, if not, indicating that the user is not interested in the currently watched content;

if yes, the user is interested in the currently viewed content;

6. The message pushing method according to claim 5, wherein before classifying the currently viewed content to facilitate message pushing to the user, the method further comprises:

7. The message pushing method according to claim 5, wherein the obtaining of the movement state of the eyeball of the user, and the obtaining of the acceleration of the eyeball rotation according to the movement state of the eyeball comprises:

extracting eyeball images of a user according to a time sequence;

8. The message pushing method according to claim 7, wherein the obtaining the focal positions of two eyeballs of the user comprises:

taking two eyeballs looking straight at a screen of the mobile terminal as a reference, wherein the positions of the eye pupils are located at the midpoint of the eyeballs, and selecting the center positions of the two eye pupils to form an eyeball reference plane parallel to the screen;

9. The message pushing method according to claim 8, wherein the calculation formula for obtaining the included angle θ between the two eye pupils and the reference plane of the eyeball is:

θ＝arccos(L2/L1)

10. A mobile terminal, characterized in that it comprises the apparatus of any of claims 1-4.