WO2021218764A1

WO2021218764A1 - Message processing method for terminal, terminal, medium, and electronic device

Info

Publication number: WO2021218764A1
Application number: PCT/CN2021/088922
Authority: WO
Inventors: 熊刘冬; 李春东; 白锦华
Original assignee: 华为技术有限公司
Priority date: 2020-04-27
Filing date: 2021-04-22
Publication date: 2021-11-04
Also published as: CN113645349A; CN113645349B

Abstract

The present application relates to a message processing method for a terminal, a terminal, a medium, and an electronic device. The method comprises: a terminal determining a position, on a terminal screen, at which a user of the terminal's gaze lands, wherein at least one message is displayed on the terminal screen; when it is determined that the position at which the user's gaze lands on the terminal screen falls within a message range of a message displayed on the screen, the terminal determining the progress of the user browsing the message; and changing a display mode, on the terminal screen, of at least some of the display elements within the message range of the message from a first display mode to a second display mode according to the progress of the user browsing the message, wherein a display mode of a message which has not been browsed by the user of the terminal is the first display mode. In the present application, the viewing of a message appearing on a terminal screen by a user is tracked by means of eyeball tracking technology, and a viewed message and a non-viewed message are distinguished and displayed on the screen according to the progress of the user viewing messages, such that the user can quickly distinguish between the viewed message and the non-viewed message, and is prevented from missing an important message.

Description

Terminal message processing method, terminal, medium and electronic equipment

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on April 27, 2020, the application number is 202010343422.4, and the application name is "terminal message processing method, terminal, medium and electronic equipment", the entire content of which is incorporated by reference Incorporated in this application.

Technical field

This application relates to the field of information processing, and in particular to a terminal message processing method, terminal, medium, and electronic equipment.

Background technique

With the development of communication technology and smart terminal technology, smart terminals (such as smart phones, tablet computers, etc.) play an increasingly important role in people’s daily work and life, and there are more and more terminal device applications. Notification messages of various applications (such as missed calls, unread messages, memos, and other reminders, etc.) are continuously pushed out to remind users to check.

When the user is too late to view the notification messages one by one, usually a bunch of messages are not processed, and new messages pop up constantly. Important messages in the unread messages are easily overwhelmed by the messages that appear constantly, making it easy for users to ignore important notifications. Messages or to-do items bring troubles to users' work and life.

Summary of the invention

The embodiments of the present application provide a message processing method, terminal, medium, and electronic equipment of a terminal. The technical solution of this application uses eye tracking technology to track the user's viewing of the messages that appear on the screen of the electronic device, and displays the viewed messages and unviewed messages on the screen according to the user's progress in viewing these notification messages, thereby facilitating the user Quickly distinguish which messages have been read and which messages have not been read and need to pay attention to avoid the problem of users missing important messages due to too many notification messages.

In the first aspect, an embodiment of the present application provides a message processing method for a terminal, including:

The terminal determines the position where the user of the terminal is gazing at the terminal screen, where the message displayed on the terminal screen is displayed on the terminal screen; the terminal determines that the position of the user gazing at the terminal screen falls within the message range of one of the messages displayed on the terminal screen In this case, the progress of the user browsing the message is determined; according to the progress of the user browsing the message, the display mode of at least a part of the display elements within the message range of the message on the terminal screen is changed from the first display mode to the second display mode. The display mode of the message browsed by the terminal user is the first display mode. This makes it easy for users to quickly distinguish which messages have been read, which messages have not been read and need to be focused on, so as to avoid the problem of users missing important messages due to too many notification messages, such as important notifications, calls, to-do or reminders, etc. Wait.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: the message range of the message is the area occupied by the message box of the message on the screen of the terminal.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: according to the progress of the user browsing the message, changing the display mode of at least a part of the display elements within the message range of the message on the terminal screen from the first display mode to The second display mode includes: changing the background color of the message within the message range from the first color to the second color when the user has finished browsing the message, wherein the color of the message that has not been browsed by the terminal user is the first color. The different display mode here means that the background color of the message is different. This allows the user to easily distinguish which messages have been viewed and which have not been viewed based on the background color.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: according to the progress of the user browsing the message, changing the display mode of at least a part of the display elements within the message range of the message on the terminal screen from the first display mode to The second display mode includes: according to the progress of the user browsing the message, changing the background color of the area corresponding to the part of the message within the message range of the message from the first color to the second color (for example, the message is browsed) When half of the time, the color of the background area of the half-viewed message is changed), wherein the background color of the part of the message that is not viewed by the terminal user is the first color. Alternatively, in some embodiments, the background color of the corresponding area within the message range of the message can be changed according to the percentage of the user's progress in browsing the message.

In a possible implementation of the above-mentioned first aspect, the above-mentioned method further includes: in the case that the terminal determines that the position where the user gazes at the terminal screen falls within the message range of one of the messages displayed on the terminal screen, determining that the user browses The progress of the message includes:

When the terminal determines that the user's gazing position on the terminal screen falls within the message box of one of the messages displayed on the terminal screen, the terminal calculates the time the user gazes at the terminal screen within the message box; according to the calculated time and settings The ratio of the time threshold is determined to determine the progress of the user browsing the message; or, the calculated time is compared with the set time threshold, and the progress of the user browsing the message is determined according to the comparison result.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: setting the value of the time threshold to be related to at least one of the number of words and the number of lines of the message, and the cumulative display length of the message. It can be understood that the number of lines of a message is the number of lines corresponding to the content displayed on the terminal screen of the message. For example, if all the content of the message occupies 5 lines on the screen of the terminal, but the terminal is locked, and the message only displays 2 lines of content on the terminal screen, it can be considered that the number of lines of the message is 2 lines . The cumulative display length of the message can be the cumulative number of lines corresponding to the case where a part of the message is displayed. For example, the message display has a total of 5 lines, and only displays the complete first line and half of the second line, then the cumulative display length of the message is 1.5 lines.

When the terminal determines that the user's gazing position on the terminal screen falls within the message box of one of the messages displayed on the terminal screen, the terminal calculates the cumulative distance of the user browsing the message box based on the position of the user gazing at the terminal screen; according to the calculated The ratio of the accumulated distance to the set distance threshold determines the user's progress in browsing the message; or, the calculated accumulated distance is compared with the set distance threshold, and the user's progress in browsing the message is determined according to the comparison result.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: setting the value of the distance threshold to be related to at least one of the number of words and lines of the message, and the cumulative display length of the message.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: the message range includes the area occupied by the message itself on the screen of the terminal.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: according to the progress of the user browsing the message, changing the display mode of at least a part of the display elements within the message range of the message on the terminal screen from the first display mode to The second display mode includes: according to the progress of the user browsing the message, the color of the part of the message that has been viewed on the terminal screen is changed from the first color to the second color (for example, half of the message is viewed, the message itself is The background color of the half of the browsed part is changed), wherein the color of the message that is not browsed by the terminal user among the messages displayed on the terminal screen is the first color. In some embodiments, the background color of the corresponding area within the message range of the message can be changed according to the percentage of the user's progress in browsing the message.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: according to the progress of the user browsing the message, changing the display mode of at least a part of the display elements within the message range of the message on the terminal screen from the first display mode to The second display mode includes: in the case of judging that the message has been viewed, folding and displaying the message on the terminal screen. In this way, by folding and displaying the messages that have been viewed (for example, only the number of messages recently viewed by the user is displayed, and the specific content of the messages that have been viewed is not displayed), the user can more easily notice the unviewed messages information.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: according to the progress of the user browsing the message, changing the classification of at least a part of the display elements within the message range of the message on the terminal screen from the first classification to The second classification, for example, classify browsed messages as read messages and display them on the top of the terminal screen, and classify messages that have not been browsed as unread messages and display them one by one in the middle of the terminal screen.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: according to the progress of the user browsing the message, changing the display mode of at least a part of the display elements within the message range of the message on the terminal screen from the first display mode to The second display mode includes:

According to the progress of the user browsing the message, the display mode of at least a part of the display elements within the message range of the message on the terminal screen is changed from the first display mode to the second display mode via an intermediate transition state, wherein the intermediate transition state includes at least one A display method different from the first display method and the second display method. For example, if the message is browsed by the user, the background color of the message is changed from red to orange to gray.

In a possible implementation of the foregoing first aspect, the foregoing method further includes: according to the progress of the user browsing the message, changing the display mode of at least a part of the display elements within the message range of the message on the terminal screen from the first display mode to The second display mode includes: in the case of judging that the message has been viewed, an icon representing that the message has been viewed is displayed within the message range of the message on the terminal screen, so that the user can determine whether the message has been viewed according to the icon.

In the second aspect, an embodiment of the present application provides a terminal, including:

The eye tracking module is used to determine the position where the user of the terminal gazes at the terminal screen, where at least one message is displayed on the terminal screen;

The progress calculation module is used to determine the progress of the user browsing messages when the terminal determines that the position where the user gazes at the terminal screen falls within the message range of one of the messages displayed on the terminal screen;

The message management module is used to change the display mode of at least part of the display elements within the message range of the message on the terminal screen from the first display mode to the second display mode according to the progress of the user browsing the message, wherein the display mode is not browsed by the terminal user The display mode of the message is the first display mode.

In the third aspect, the embodiments of the present application provide a computer-readable medium with instructions stored on the computer-readable medium. When the instructions are executed on a computer, the computer executes the first aspect and various possible implementations of the first aspect. Any kind of terminal message processing method.

In a fourth aspect, an embodiment of the present application provides an electronic device, including:

Memory, used to store instructions executed by one or more processors of the system, and

The processor is one of the processors of the system, and is used to execute the first aspect and any one of the terminal message processing methods in the various possible implementations of the first aspect.

Description of the drawings

Figure 1(a) shows a display interface diagram of a mobile phone receiving multiple notification messages in a locked screen state according to some embodiments of the present application;

Figure 1(b) shows a display interface diagram with different background colors of the notification boxes of read messages and unread messages displayed on the screen of the mobile phone according to some embodiments of the present application;

Figure 2 shows a block diagram of the hardware structure of a mobile phone according to some embodiments of the present application;

Fig. 3 shows a scene diagram of eye tracking using a mobile phone according to some embodiments of the present application;

Fig. 4 shows a hardware and software structure diagram related to the message processing technology of the present application in a mobile phone according to some embodiments of the present application;

Fig. 5 shows a message processing flowchart of the software system shown in Fig. 4 according to some embodiments of the present application;

Fig. 6(a) shows a display interface in which messages on the main screen of the mobile phone are not viewed when the mobile phone is in a locked state according to some embodiments of the present application;

Fig. 6(b) shows a display interface in which some messages on the main screen of the mobile phone have been viewed when the mobile phone is in a locked state according to some embodiments of the present application;

Fig. 7 shows a flow chart of a method for processing a message in a mobile phone according to some embodiments of the present application;

FIG. 8(a) shows a display interface with different background colors for the messages that have been viewed and that have not been viewed on the main screen of the mobile phone in a locked state according to some embodiments of the present application;

FIG. 8(b) shows a display interface in which messages that have been viewed and that have not been viewed on the main screen of the mobile phone are categorized and displayed according to some embodiments of the present application;

FIG. 8(c) shows a display interface with icons displayed on messages that have been viewed on the main screen of the mobile phone in a locked state according to some embodiments of the present application;

Fig. 9 shows a structural block diagram of a message processing apparatus according to some embodiments of the present application;

Fig. 10 shows a block diagram of a system according to some embodiments of the present application;

FIG. 11 shows a block diagram of a system on chip (SoC) according to some embodiments of the present application.

Detailed ways

The illustrative embodiments of the present application include, but are not limited to, a terminal message processing method, device, medium, and electronic equipment.

The embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The embodiment of the present application discloses a message processing method for an electronic device, which can track a user's viewing of messages that appear on the screen of the electronic device through eye tracking technology, and compare the viewed messages on the screen according to the progress of the user viewing these notification messages. Unviewed messages are displayed separately, so that users can quickly distinguish which messages have been read, which messages have not been read and need to be focused on, so as to avoid the problem of users missing important messages due to too many notification messages, such as important notifications, phone calls , To-do or reminders, etc. It can be understood that the electronic devices provided in this application can be various electronic devices whose display screens can display message notifications and can detect eye movement, including but not limited to tablet computers, smart phones, laptop computers, desktop computers, Wearable electronic devices, head-mounted displays, mobile email devices, portable game consoles, portable music players, reader devices, etc., where wearable electronic devices include but are not limited to smart watches, smart bracelets or smart glasses, smart helmets , Smart headband, etc. For ease of description, the following takes the mobile phone 10 as an example to illustrate the technical solution of the present application.

Specifically, FIG. 1(a) shows a display interface diagram of the mobile phone 10 receiving multiple notification messages when the screen is locked. As shown in Figure 1(a), the main screen of the mobile phone 10 is in a locked state, and the display interface of the main screen displays four notification messages, which are two SMS messages (one billing information from 10086 and one from customer Zhang). SMS), a missed call from Li Lei and a WeChat message. As shown in Figure 1(b), when the user views the home screen in the locked state, the mobile phone 10 tracks the position of the user's eyes (hereinafter referred to as human eyes) gazing at the home screen to determine whether the human eyes have seen the notification message. , And distinguish the background color of the notification box of the read and unread messages. For example, in Figure 1(b), it is detected that the user has viewed the short message from the customer Zhang and the missed call message from Li Lei. The background color of Zhang's SMS notification box and the missed call message notification box from Li Lei have changed, which is different from the background color of the notification box of unchecked messages. This solution is also applicable to other scenarios where the display interface of the mobile phone 10 displays notification messages, for example, a message list that appears after the user pulls down the screen. In the prior art, when the user does not manually unlock the home screen and view each notification message in the notification center one by one, or only view some of the notification messages, as new notification messages continue to appear, it is usually difficult to distinguish which messages have been viewed. , Which messages have not been read, it is possible to miss important notifications, calls, to-do or reminders, etc. If you omit to view these important information, it may have a great negative impact on the user's work and life. As described above, the technical solution of the present application solves the above-mentioned problems. The user can learn the status of the message being viewed through the background change of the message without special operation, and the above-mentioned problem can be solved, and the operation process is simple and fast.

It can be understood that the notification messages displayed on the screen of the mobile phone 10 may include call records, short messages, third-party application notifications, and so on. In addition, the message displayed on the screen of the mobile phone 10 is not necessarily the latest push notification message, and multiple notification messages may be accumulated in history. In addition, when the background color is used to distinguish between read and unread notification messages, the background color of the notification frame of the message can be distinguished, or the background color of the message itself can be distinguished. In addition, when distinguishing between read and unread notification messages, in addition to changing the background color of the message box, other methods can also be used, such as bolding the font of the read message, and dividing the read message with the unread message. Read messages in different fonts, add an icon that characterizes the message that has been read, and categorize and display the read messages and unread messages in groups (for example, fold the read messages and display them in groups). Unread messages are expanded and displayed one by one) and so on.

According to an embodiment of the present application, FIG. 2 shows a schematic structural diagram of a mobile phone 10. The mobile phone 10 can execute the message processing method disclosed in the embodiment of the present application. In Figure 2, similar parts have the same reference numerals. As shown in FIG. 2, the mobile phone 10 may include a processor 110, a power module 140, a memory 180, a mobile communication module 130, a wireless communication module 120, an infrared light emitting module 101, a sensor module 190, an audio module 150, an infrared camera 170, and an interface Module 160 and display screen 102 and so on.

It is understandable that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the mobile phone 10. In other embodiments of the present application, the mobile phone 10 may include more or fewer components than shown, or combine certain components, or disassemble certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, may include a central processing unit (CPU), an image processor GPU (Graphics Processing Unit), a digital signal processor DSP, and a microprocessor MCU (Micro-programmed Control Unit), AI (Artificial Intelligence) processor, or programmable logic device FPGA (Field Programmable Gate Array) and other processing modules or processing circuits. Among them, the different processing units may be independent devices or integrated in one or more processors. The processor 110 may be used to process the video or image containing human eyes collected by the infrared camera 170 to obtain the position where the human eye is looking at the screen of the mobile phone 10. A storage unit may be provided in the processor 110 for storing instructions and data. In some embodiments, the storage unit in the processor 110 is a cache memory 180. The memory 180 may store an operating system and at least one application program (such as an application program for shooting video) required by the function. The memory 180 may also store a video or image containing human eyes collected by the infrared camera 170, and the processor 110 can monitor the infrared The video or image collected by the camera 170 includes human eyes position information, etc., obtained by processing the human eyes looking at the screen of the mobile phone 10.

The power supply module 140 may include a power supply, a power management component, and the like. The power source can be a battery. The power management component is used to manage the charging of the power supply and the power supply to other modules. The charging management module is used to receive charging input from the charger; the power management module is used to connect to a power source, and the charging management module is connected to the processor 110.

The mobile communication module 130 may include, but is not limited to, an antenna, a power amplifier, a filter, a low noise amplifier (LNA), and the like. The mobile communication module 130 may provide a wireless communication solution including 2G/3G/4G/5G and the like applied on the mobile phone 10. The mobile communication module 130 may receive electromagnetic waves by an antenna, and perform processing such as filtering, amplifying and transmitting the received electromagnetic waves to the modem processor for demodulation. The mobile communication module 130 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic waves for radiation by the antenna. In some embodiments, at least part of the functional modules of the mobile communication module 130 may be provided in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 130 and at least part of the modules of the processor 110 may be provided in the same device. Wireless communication technologies can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), and broadband code division. Multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), Bluetooth (blue tooth, BT), global Navigation satellite system (global navigation satellite system, GNSS), wireless local area network (wireless local area networks, WLAN), near field communication (NFC), frequency modulation (FM) and/or field communication, NFC ), infrared technology (infrared, IR) technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system ( Quasi-zenith satellite system, QZSS) and/or satellite-based augmentation systems (SBAS).

The wireless communication module 120 may include an antenna, and transmit and receive electromagnetic waves via the antenna. The wireless communication module 120 can provide applications on the mobile phone 10, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellites. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The mobile phone 10 can communicate with the network and other devices through wireless communication technology.

In some embodiments, the mobile communication module 130 and the wireless communication module 120 of the mobile phone 10 may also be located in the same module.

The display screen 102 includes a display panel. The display panel can use liquid crystal display (LCD), organic light-emitting diode (OLED), active matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode). AMOLED, flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oled, quantum dot light-emitting diode (QLED), etc. For example, the display screen 102 is used to display notification messages of various applications of the mobile phone 10 (such as missed calls, unread messages, memos and reminders of other matters, etc.).

The infrared light emitting module 101 includes an infrared light emitter, which can emit infrared light to the human eye, so as to determine the position of the human eye gazing at the display screen 102 by the position of the infrared light on the human eye's cornea. Infrared light emitters include but are not limited to infrared light emitting diodes and so on.

The sensor module 190 may include a proximity light sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

The audio module 150 may convert digital audio information into an analog audio signal for output, or convert an analog audio input into a digital audio signal. The audio module 150 can also be used to encode and decode audio signals. In some embodiments, the audio module 150 may be disposed in the processor 110, or part of the functional modules of the audio module 150 may be disposed in the processor 110. In some embodiments, the audio module 150 may include a speaker, a receiver, a microphone, and a headphone interface.

In some embodiments, the infrared camera 170 may collect a video or image containing human eyes to obtain the position of the infrared light emitted by the infrared light emitting module 101 in the cornea of the user's eye by processing the video or image containing the human eye. , And then determine the position where the user's eyes are gazing at the display screen 102 of the mobile phone 10. It can be understood that the mobile phone 10 may also include a common camera (not shown) for shooting color images or videos.

The interface module 160 includes an external memory interface, a universal serial bus (USB) interface, a subscriber identification module (SIM) card interface, and the like. The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 10. The external memory card communicates with the processor 110 through an external memory interface to realize the data storage function. The universal serial bus interface is used for communication between the mobile phone 10 and other mobile phones. The user identification module card interface is used to communicate with the SIM card installed in the mobile phone 10, for example, to read the phone number stored in the SIM card, or to write the phone number into the SIM card.

In some embodiments, the mobile phone 10 further includes buttons, motors, indicators, and the like. Among them, the keys may include a volume key, an on/off key, and so on. The motor is used to cause the mobile phone 10 to generate a vibration effect, for example, when the user's mobile phone 10 is called, it generates vibration to prompt the user to answer the incoming call of the mobile phone 10. The indicator can include a laser indicator, a radio frequency indicator, an LED indicator, and so on.

FIG. 3 shows a scene diagram of eye tracking using the mobile phone 10 according to some embodiments of the present application. In the embodiment shown in FIG. 3, the infrared light emitting module 101 has three infrared light emitters for illustration. The infrared emitters can be used to emit infrared light to the human eye. Correspondingly, the human eye is receiving After the infrared light emitted by the three infrared light emitters, three infrared light reflection points (reflection point A, reflection point B, and reflection point C as shown in FIG. 3) are formed in the cornea of the human eye. The infrared camera 170 is used to take a video or image of the human eye, and obtain the position of the reflection point formed in the cornea of the human eye and the center position of the pupil of the human eye from the captured video or image to determine the position where the human eye looks at the screen of the mobile phone 10 (That is, the line of sight of the human eye on the screen of the mobile phone 10). Specifically, in some embodiments, the position where the human eye is gazing at the mobile phone 10 can be determined in the following manner:

Process the human eye image to obtain the gradient value of the gray in the set direction, determine the position of the maximum value of the gray gradient as the pupil edge feature of the human eye image, and then fit the pupil edge feature , Determine the center position of the fitted image, and determine the center position as the center position of the pupil in the human eye image, and the gray image obtained after processing the human eye image can obtain the area with higher gray value based on the spot recognition algorithm It is a Purkin spot, and the position of the Purkin spot is the spot (or called the "reflection point") position. According to the position of the reflection point and the radius of curvature of the cornea, the center of curvature of the cornea is obtained. According to the position of the center of the pupil, the center of corneal curvature and the preset angle between the optical axis of the eye and the visual axis, the position of the human eye looking at the screen of the mobile phone 10 (that is, the human Eyes on the screen of the mobile phone 10).

It can be understood that, in other embodiments, other methods may also be used to confirm the position where the human eye is looking at the screen of the mobile phone 10, which is not limited to the above method.

In addition, it can be understood that, in other embodiments, the infrared light emitting module 101 may also include more than three infrared light emitters arranged in a predetermined manner, such as a square shape, a line shape, and the like. Moreover, in other embodiments, the number of infrared cameras 170 may also be two or more, which is not limited here.

Fig. 4 shows hardware and software related to the message processing technology of the present application in the mobile phone 10 shown in Fig. 3 according to some embodiments of the present application. As shown in FIG. 4, the mobile phone 10 includes a hardware system 103 and a software system 104. Among them, the hardware system 103 includes an infrared camera 170, an infrared light emitting module 101, and a processor 110; the software system 104 includes an operating system 107, and the operating system 107 includes a message management module 105 and an eye tracking module 106. Among them, the infrared light emitting module 101 includes three infrared light emitters, which are used as infrared light sources to emit infrared light. The infrared camera 170 is used to capture a video or image of the human eye and send it to the processor 110 so that the processor 110 obtains the position of the reflection point formed in the cornea of the human eye and the center position of the pupil of the human eye from the captured video or image of the human eye Then, according to the position of the reflection point in the cornea and the center position of the pupil of the human eye, the position where the human eye looks at the screen of the mobile phone 10 is determined. For the specific process, refer to the description of FIG. 3. The eye tracking module 106 continuously obtains the movement of the human eye to feed back the position of the human eye gazing at the screen of the mobile phone 10 in real time. The message management module 105 is a module that is set in the mobile phone 10 and dynamically changes the status display characteristics of the notification message of the mobile phone 10. For example, the message management module 105 can change the color of the message notification bar and the font size of the text. In the embodiment provided in this application, the message management module 105 is also used to determine whether the human eye has seen the notification message according to the position where the human eye is gazing at the screen of the mobile phone 10 determined by the eye tracking module 106, and determine whether the human eye has seen the notification message, and compare the read and unread The background color of the messages is distinguished, which is convenient for users to quickly distinguish which messages have been read and which messages have not been read and need to be focused. The user only needs eye movements. The operation process is simple and fast, and the user experience is improved.

It is understandable that the message management module 105 and the eye tracking module 106 provided in the embodiments of the present application are modules divided according to functions. In other embodiments, the message management module 105 and the eye tracking module 106 can also be combined into one module. , Or be divided into more modules. The hardware and software system architecture of the mobile phone 10 illustrated in the embodiment of the present application does not constitute a specific limitation on the mobile phone 10. In other embodiments of the present application, the mobile phone 10 may include more or fewer components than shown, or combine certain components, or disassemble certain components, or arrange different components.

Fig. 5 shows a message processing flow of the mobile phone 10 shown in Fig. 4 according to some embodiments of the present application. Specifically, as shown in Figure 5, it includes:

1) The message management module 105 and the eye tracking module 106 establish a connection (500). In some embodiments, the message management module 105 triggers the eye tracking module 106 to activate the eye tracking function when a new notification message is received (for example, when the home screen of the mobile phone 10 is in a locked state at this time). In some embodiments, when the message management module 105 receives the user's operation of pulling down the notification bar (for example, the mobile phone screen has been unlocked at this time), the eye tracking module 106 is triggered to enable the eye tracking function. In some embodiments, when the message management module 105 receives a notification from an application of the mobile phone 10, etc. (for example, when the user browses a web page using a search engine, a tip from the chat software pops up on the display interface of the mobile phone 10) , Trigger the eye tracking module 106 to start the eye tracking function.

After the eye tracking module 106 receives the registration of the message management module 105, the message management module 105 and the eye tracking module 106 establish a connection.

2) The eye tracking module 106 sends to the message management module 105 the coordinates of the position where the human eye is looking at the screen of the mobile phone 10 every preset time (502). For example, in some embodiments, the eye tracking module 106 sends to the message management module 105 the coordinates of the position where the human eye is gazing at the screen of the mobile phone 10 every 10 ms.

In some embodiments, the position of the human eye looking at the screen of the mobile phone 10 can be determined by an optical recording method, where the optical recording method is to record the movement of the human eye with a camera or video camera, that is, to obtain an eye image reflecting the movement of the human eye. And extract eye features from the acquired eye images to build a model of gaze/gaze point estimation. Among them, the eye features may include: the position of the center of the pupil, the shape of the pupil, the position of the iris, the shape of the iris, the position of the eyelid, the position of the corner of the eye, the position of the light spot (also referred to as the Purkin spot), and the like.

Specifically, the principle of the eye tracking module 106 in the embodiment of the present application for determining the position where the human eye is gazing at the screen of the mobile phone 10 may be: the infrared light emitting module 101 emits infrared light to the human eye, and the infrared camera 170 captures a video of human eye movement, and at the same time can shoot The reflection point of the infrared light on the cornea is the light spot (also called the Purkin spot), through which the eye image with the light spot is obtained. The eye tracking module 106 analyzes the eye image with light spots to obtain the center position of the pupil of the human eye and the position of the reflection point on the cornea, and then determine the position where the human eye looks at the screen of the mobile phone 10. It can be understood that as the human eye rotates, the relative positional relationship between the center of the pupil and the corneal reflection point changes accordingly. Several eye images with light spots collected by the infrared camera 170 reflect this positional relationship, and the eye tracking module 106 According to the position change relationship, the position where the human eye looks at the screen of the mobile phone 10 as the human eye rotates is determined.

It can be understood that, in other embodiments, other methods may also be used to confirm the position where the human eye is looking at the mobile phone 10, which is not limited to the above method.

3) The message management module 105 determines whether the position of the human eye gazing at the screen of the mobile phone 10 falls into the message notification frame according to the received position coordinates of the human eye gazing at the screen of the mobile phone 10 and the layout information of the message notification frame of the mobile phone 10 (504). In some embodiments, when the mobile phone 10 receives a new message, the message management module 105 can obtain the layout information of the message box corresponding to the message on the screen of the mobile phone 10 (that is, the coordinate range of the message box on the screen of the mobile phone 10). ), by judging whether the position coordinates of the human eyes looking at the screen of the mobile phone 10 fall within the coordinate range of the message box on the screen of the mobile phone 10, and determining whether the position of the human eyes looking at the screen of the mobile phone 10 falls within the message notification box. It can be understood that when the human eye is browsing a notification message, the position where the human eye looks at the screen of the mobile phone 10 must be in the message box corresponding to the message.

4) If it falls into the message notification box, the message management module 105 calculates the progress of the human eye viewing the message according to the length of the human eye gazing at the message or the continuously received position information of the human eye gazing at the screen of the mobile phone 10 (506). In some embodiments, the progress of the human eye browsing the messages can be calculated by calculating the length of time that the position of the human eye gazing at the screen of the mobile phone 10 falls into the corresponding message notification frame. Specifically, the progress of the human eye browsing the message may be the calculated ratio of the time that the position of the human eye gazing at the screen of the mobile phone 10 falls into the corresponding message notification frame and the set time threshold. It can be understood that the timing should start from determining that the position where the human eye is gazing at the screen of the mobile phone 10 falls within the message notification frame. For example, starting from determining that the position where the human eye is looking at the screen of the mobile phone 10 falls within the message notification frame, the eye tracking module 106 sends the position of the eye looking at the screen of the mobile phone 10 to the message management module 105 every 10 ms, and the human eye is looking at the screen for a continuous period of 100 ms. The position of the screen of the mobile phone 10 is always in the message notification frame. It is not difficult to understand that it can be considered that the duration of the human eye browsing the message in the message notification frame is at least 100 ms. If it takes 200ms for the human eye to completely browse the message (that is, the set time threshold of the message is 200ms), the browsing progress when the human eye browses the message reaches 100ms is 50%; if the human eye completely browses the message, it takes 120ms (That is, the set time threshold of the message is 120ms), then when the human eye browses the message for 100ms, the browsing progress is 83.33%. It can be understood that the set time threshold of the message may be related to at least one of the number of words and lines of the message, and the cumulative display length of the message. That is, when the font size of the message is the same, the greater the number of words, the greater the number of lines, or the longer the cumulative display length, the greater the set time threshold corresponding to the message.

It can be understood that, in some embodiments, the progress of the human eye browsing message can also be determined according to the relationship between the time when the human eye browses the message and the set time threshold. For example, for one or more messages displayed on the screen of the mobile phone 10, set The set time threshold is 80ms. If the time for human eyes to browse the message is greater than or equal to 80ms by calculation, then the message can be considered to have been viewed; if the time for human eyes to browse the message is less than 80ms, then the message can be considered Haven't seen it.

In some embodiments, the progress of the human eye browsing the messages can be calculated by the human eye gazing at the position information of the screen of the mobile phone 10. In one embodiment, the progress can be calculated by calculating the cumulative distance of the human eye browsing the messages in the message notification box. Specifically, the progress of the human eye browsing the message may be a calculated ratio of the cumulative distance of the human eye browsing the message in the message notification frame to the set distance threshold. For example, the message in the message notification box has a total of two lines. The message length of the first line is 5cm, and the message length of the second line is 3cm. The progress of viewing the message is 80%. In one embodiment, the progress can be calculated by calculating the height of the message in the message notification box that has been viewed by the human eye. For example, the message in the message notification box has a total of five lines, and the average height of each line is 0.5 cm. If the human eye has browsed through the four lines, the progress of the human eye is 80%. If it is determined that the human eye has browsed the three lines, the progress of the human eye of the message is 60%. It can be understood that the set distance threshold of the message may be related to at least one of the number of words and lines of the message, and the cumulative display length of the message. That is, when the font size of the message is the same, the greater the number of words, the greater the number of lines, or the longer the cumulative display length, the greater the set distance threshold corresponding to the message.

It can be understood that, in some embodiments, the progress of the human eye to browse the message can also be determined according to the relationship between the distance of the human eye to browse the message and the set distance threshold. For example, for one or more messages displayed on the screen of the mobile phone 10, set The predetermined distance threshold is 3 lines. If the distance of the human eye browsing the message is greater than or equal to 3 lines by calculation, then the message can be considered to have been seen; if the human eye browsing the message is less than 3 lines, it can be considered The message has not been read.

It can be understood that, in some embodiments, the size of each message notification frame (for example, the length and height of the message notification frame) may be the same. In other embodiments, the size of each message notification box can also be different. For example, there are two lines of text in the message notification box 1 and ten lines of text in the message notification box 2. If each line of text is the same height, it is not It is difficult to understand that the more lines the message content occupies, the greater the height of the message notification box.

It can be understood that in some embodiments, in the process of calculating the progress of human eye browsing messages, if it is determined that the position where the human eye is looking at the screen of the mobile phone 10 is not in the message notification frame, the calculation of the human eye browsing message progress is interrupted, and the current progress is performed. Save, so that the next time it is judged that the position where the human eye is looking at the screen of the mobile phone 10 falls within the message notification frame, the progress will continue to be calculated.

In some embodiments, in addition to calculating the progress of human eyes browsing messages from the moment the human eye falls into the message notification box, the message displayed on the screen of the mobile phone 10 has no message notification box (that is, only the message content is displayed). The progress of the human eye browsing the message can be started by judging that the human eye falls within the area occupied by the message itself on the screen of the mobile phone 10.

5) The message management module 105 judges whether the human eye has read the message according to the progress of the human eye browsing the message (508). In some embodiments, it can be judged whether human eyes have seen the message according to a preset progress threshold. For example, in one embodiment, the preset progress threshold is 70%. If it takes 200ms for the human eye to completely browse the message, and the browsing progress when the human eye browses the message for 100ms reaches 50%, it is considered that the human eye has not seen it. After passing this message, and if the browsing progress of the human eye reaches 80% when it reaches 160ms, it is considered that the human eye has read the message. For another example, in one embodiment, the message in the message notification box has a total of two lines. The message length of the first line is 5cm, and the message length of the second line is 3cm. If it is determined that the total length of the message viewed by the human eye is 6.4cm, If the progress of the human eye viewing the message is 80%, it is considered that the human eye has seen the message. It can be understood that the value of the aforementioned progress threshold is only exemplary and not restrictive. In a specific application, the value of the threshold may be set as required.

It can be understood that, in some embodiments, for a part of the message that has not been read (that is, according to the progress of the eye browsing the message, it is judged that the human eye has not read the message), the current browsing progress can be recorded, so as to build on this basis. Continue to accumulate the progress of attention to the message, or recalculate the progress next time.

6) If people have seen the message, the message management module 105 changes the color of the message box of the message (510). In some embodiments, if it is determined that the human eye has seen the message in the message notification box, the message management module 105 distinguishes the background color of the message box (for example, the background color of the message box is white when the message box has not been seen. After judging that the message has been viewed, the message management module 105 changes the background color of the message box to rose red to indicate that the message has been viewed. In some embodiments, the message management module 105 can also change the color of the message box being viewed by the human eye according to the progress of the human eye browsing the message. For example, in the embodiment shown in Figure 6(a), the home screen of the mobile phone 10 is in a locked state, and four messages are displayed on the home screen of the mobile phone 10, two messages from "Guo" and two messages from Mr. Zhang. The missed call message and one WeChat message were not viewed by the user. In the embodiment shown in Figure 6(b), the missed call message from Mr. Zhang has been checked, and the background color of the corresponding message box is completely changed (that is, the background color of the message that has not been checked is different ); A message from "Guo" about a problem discussed in a meeting was viewed 70%, and the background color of the 70% part of the message was changed.

It can be understood that in some embodiments, other ways can be used to distinguish between messages that have been seen by the human eye and messages that have not been seen, such as bolding the font of read messages, and making read messages different from unread messages. Fonts for display and so on.

Fig. 7 shows a message processing method of an electronic device according to some embodiments of the present application. The following takes the mobile phone 10 shown in FIG. 3 as an example to introduce the message processing method provided in the embodiment of the present application in detail. Specifically, as shown in Figure 7, it includes:

1) Determine the position where the human eye is gazing at the screen of the mobile phone (702). For example, the mobile phone 10 uses the eye tracking module 106 to determine where the human eye is looking at the screen of the mobile phone 10. Specifically, the infrared light emitting module 101 emits infrared light to the human eye, and the infrared camera 170 captures a video of human eye movement, and at the same time, the reflection point of the infrared light on the cornea can be captured, thereby obtaining an eye image with the reflection point. The eye tracking module 106 analyzes the eye image with reflection points to obtain the center position of the pupil of the human eye and the position of the reflection point on the cornea, and determines that the human eye is looking at the mobile phone 10 according to the center position of the pupil of the human eye and the position of the reflection point on the cornea. The position of the screen. Moreover, as the human eye rotates, the relative positional relationship between the pupil center and the corneal reflection point changes accordingly. Several eye images with reflection points collected by the infrared camera 170 reflect this positional change relationship. The eye tracking module 106 According to the position change relationship, the position where the human eye looks at the screen of the mobile phone 10 as the human eye rotates is determined.

2) According to the position of the human eye gazing at the mobile phone screen and the layout information of the message box displayed on the mobile phone screen, it is judged whether the position of the human eye gazing at the mobile phone screen falls within the message box (704). The message management module 105 can obtain the layout information of the message box corresponding to the message on the screen of the mobile phone 10 (that is, the coordinate range of the message box on the screen of the mobile phone 10), and determine whether the position coordinates of the human eye watching the screen of the mobile phone 10 fall into the message The coordinate range of the frame on the screen of the mobile phone 10 is used to determine whether the position where the human eye is looking at the screen of the mobile phone 10 falls within the message notification frame. That is: if the coordinates of the position where the human eye is looking at the screen of the mobile phone 10 fall within the coordinate range of the message box on the screen of the mobile phone 10, it is determined that the position where the human eye is looking at the screen of the mobile phone 10 falls within the message notification frame; if the human eye is looking at the screen of the mobile phone 10 If the position coordinate exceeds the coordinate range of the message box on the screen of the mobile phone 10, it is determined that the position where the human eye is looking at the screen of the mobile phone 10 does not fall into the message notification box.

3) In the case where the position where the human eye is gazing at the screen of the mobile phone falls within the message box, calculate the progress of the human eye browsing the messages in the message box (706). In some embodiments, the progress of the human eye browsing the messages can be calculated by calculating the length of time that the position of the human eye gazing at the screen of the mobile phone 10 falls into the corresponding message notification frame. In some embodiments, the progress of the human eye browsing the messages can be calculated by the human eye gazing at the position information of the screen of the mobile phone 10. The specific calculation method is similar to the calculation method shown in FIG. 5, please refer to the above for detailed description, and will not be repeated here.

4) Judging whether the human eye has seen the message in the message box according to the progress of the human eye browsing the message in the message box (708).

In some embodiments, it can be judged whether human eyes have seen the message according to a preset progress threshold. For example, the preset progress threshold is 80%. If the progress of the human eye browsing the message in the message box is greater than or equal to 80%, it is judged that the human eye has seen the message in the message box, otherwise, it is judged that the human eye has not seen the message in the message box. News.

5) When human eyes have seen the message in the message box, change the color of the message box from the first color to the second color (710). For example, in the embodiment shown in Figure 8(a), four messages are displayed on the main screen of the mobile phone 10, including two short messages from the mobile phone, one from Mr. Zhang's missed call and one message from WeChat. After the mobile phone user has seen the missed call notification message from Mr. Zhang, the message management module 105 distinguishes the background color of the message box containing the incoming call notification message from the background color of the unread message box. Which messages have been watched and which messages have not been watched can be quickly distinguished. The user only needs eye movements. The operation process is simple and fast, and the user experience is improved.

In some embodiments, according to the progress of the human eye browsing the message, as the human eye continues to browse the message, the color of the part that has been seen and the part that has not been seen in the message can be distinguished, for example, the unseen part. Part of the content of the message is displayed in white, and part of the content of the message that has been read is displayed in green.

In some embodiments, it is also possible to categorize the messages that have been seen and the messages that have not been seen, the unseen messages are displayed on the top of the screen of the mobile phone 10, and the messages that have been seen are displayed on the screen of the mobile phone 10. Show below. You can also display the unseen messages on the screen of the mobile phone 10 one by one, fold the messages that have been read, and only display the number of read messages, or only display the unseen messages on the main screen of the mobile phone 10. To display. For example, in the embodiment shown in FIG. 8(b), the main screen of the mobile phone 10 has 1 total missed call and 1 WeChat notification that has not been viewed, and there are 20 read notifications that are superimposed and displayed. This makes it easier for users to pay attention to notification messages that have not been viewed, so as to reduce the possibility of missing important notification messages.

In other embodiments, for a message that has been viewed, an icon indicating that the message has been checked may be displayed on the screen. For example, in the embodiment shown in FIG. 8(c), the main screen of the mobile phone 10 displays four messages, including two short messages from the mobile phone, one from Mr. Zhang's missed call and one message from WeChat. After the user has seen the SMS message from "Guo" and the missed call notification message from Mr. Zhang, the message management module 105 displays an icon of an eye on the upper right of the message box of the two messages, which symbolizes this Both messages have been viewed. The user can distinguish which messages have been watched and which have not been watched by seeing whether the eye icon is displayed on the message box, which is convenient for the user to quickly distinguish the messages that have been watched from those that have not been watched.

It can be understood that the eye icon in FIG. 8(c) that indicates that the message has been viewed can be any icon, and this solution does not limit the shape, size, color, etc. of the icon that indicates that the message has been viewed. In addition, this solution does not limit the position of the icon that indicates that a certain message has been viewed in the message box.

It can be understood that, in some other embodiments, for an unseen message, an icon indicating that the message has not been seen may also be displayed on the screen.

Fig. 9 provides a message processing apparatus 900 according to some embodiments of the present application, specifically, including:

The eye tracking module 902 is used to determine where the user of the terminal is gazing at the terminal screen, and at least one message is displayed on the terminal screen;

The progress calculation module 904 is configured to determine the progress of the user browsing messages when the terminal determines that the position where the user gazes at the terminal screen falls within the message range of one of the messages displayed on the terminal screen;

The message management module 906 is configured to change the display mode of at least a part of the display elements within the message range of the message on the terminal screen from the first display mode to the second display mode according to the progress of the user browsing the message, wherein The display mode of the browsed message is the first display mode.

It can be understood that the message processing apparatus 900 shown in FIG. 9 corresponds to the message processing method provided in this application, and the technical details in the specific description of the message processing method provided in this application still apply to the message processing shown in FIG. 9 For the device 900, please refer to the above for a specific description, and will not be repeated here.

Referring now to FIG. 10, shown is a block diagram of a system 100 according to an embodiment of the present application. Figure 10 schematically illustrates an example system 100 according to various embodiments. In one embodiment, the system 100 may include one or more processors 1004, a system control logic 1008 connected to at least one of the processors 1004, a system memory 1012 connected to the system control logic 1008, and a system control logic 1008 connected to the system control logic 1008. Non-volatile memory (NVM) 1016, and a network interface 1020 connected to the system control logic 1008. In some embodiments, the system 100 further includes an infrared light emitting module (not shown), an infrared camera (not shown), and so on. The infrared light emitting module includes an infrared light emitter, which can emit infrared light to human eyes. Infrared light emitters include, but are not limited to, infrared light-emitting diodes, infrared lasers, and so on. Infrared cameras can capture video or images containing human eyes.

In some embodiments, the processor 1004 may include one or more single-core or multi-core processors. In some embodiments, the processor 1004 may include any combination of a general-purpose processor and a special-purpose processor (for example, a graphics processor, an application processor, a baseband processor, etc.). In an embodiment in which the system 100 adopts an eNB (Evolved Node B, enhanced base station) 101 or a RAN (Radio Access Network, radio access network) controller, the processor 1004 may be configured to execute various conforming embodiments, for example In the embodiment shown in FIGS. 5 and 7, the processor 1004 processes the video or image containing the human eye collected by the infrared camera to obtain the position of the infrared light emitted by the infrared light emitting module in the cornea of the user’s eye, Then determine the position of the user's eyes gazing at the display screen of the mobile phone.

In some embodiments, the system control logic 1008 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1004 and/or any suitable device or component in communication with the system control logic 1008.

In some embodiments, the system control logic 1008 may include one or more memory controllers to provide an interface to the system memory 1012. The system memory 1012 can be used to load and store data and/or instructions. For example, loading the application program corresponding to the camera function of the infrared camera, storing the video or image containing the human eye collected by the infrared camera, and the processor processing the video or image containing the human eye collected by the infrared camera. Screen location information, etc. In some embodiments, the memory 1012 of the system 100 may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

The NVM/memory 1016 may include one or more tangible, non-transitory computer-readable media for storing data and/or instructions. In some embodiments, the NVM/memory 1016 may include any suitable non-volatile memory such as flash memory and/or any suitable non-volatile storage device, such as HDD (Hard Disk Drive, hard disk drive), CD (Compact Disc) , At least one of an optical disc drive and a DVD (Digital Versatile Disc, Digital Versatile Disc) drive.

The NVM/memory 1016 may include a part of the storage resources on the device where the system 100 is installed, or it may be accessed by the device, but not necessarily a part of the device. For example, the NVM/storage 1016 can be accessed through the network via the network interface 1020.

In particular, the system memory 1012 and the NVM/memory 1016 may respectively include: a temporary copy and a permanent copy of the instruction 1024. The instructions 1024 may include instructions that, when executed by at least one of the processors 1004, cause the system 100 to implement the methods shown in FIGS. 5 and 7. In some embodiments, the instructions 1024, hardware, firmware, and/or software components thereof may additionally/alternatively be placed in the system control logic 1008, the network interface 1020, and/or the processor 1004.

The network interface 1020 may include a transceiver to provide a radio interface for the system 100, and then communicate with any other suitable devices (such as a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the network interface 1020 may be integrated with other components of the system 100. For example, the network interface 1020 may be integrated with at least one of the processor 1004, the system memory 1012, the NVM/storage 1016, and the firmware device with instructions (not shown), when at least one of the processor 1004 executes the When instructed, the system 100 implements the methods shown in FIGS. 5 and 7.

The network interface 1020 may further include any suitable hardware and/or firmware to provide a multiple input multiple output radio interface. For example, the network interface 1020 may be a network adapter, a wireless network adapter, a telephone modem and/or a wireless modem.

In one embodiment, at least one of the processors 1004 may be packaged with the logic of one or more controllers for the system control logic 1008 to form a system in package (SiP). In one embodiment, at least one of the processors 1004 may be integrated with the logic of one or more controllers used for the system control logic 1008 on the same die to form a system on chip (System on Chip, SoC).

The system 100 may further include: an input/output (I/O) device 1032. The I/O device 1032 may include a user interface to enable a user to interact with the system 100; the design of the peripheral component interface enables the peripheral component to also interact with the system 100. In some embodiments, the system 100 further includes a sensor for determining at least one of environmental conditions and location information related to the system 100.

In some embodiments, the user interface may include, but is not limited to, a display (e.g., liquid crystal display, touch screen display, etc.), speakers, microphones, one or more cameras (e.g., still image cameras and/or video cameras), flashlights (e.g., LED flash) and keyboard.

In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

In some embodiments, the sensor may include, but is not limited to, a gyroscope sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit may also be part of or interact with the network interface 1020 to communicate with components of the positioning network (eg, global positioning system (GPS) satellites).

According to an embodiment of the present application, FIG. 11 shows a block diagram of a system on chip (System on Chip, SoC) 110. In Fig. 11, similar parts have the same reference numerals. In addition, the dashed box is an optional feature of the more advanced SoC. In FIG. 11, the SoC110 includes: an interconnection unit 1150; a system agent unit 1180; a bus controller unit 1190; an integrated memory controller unit 1140; a group or one or more coprocessors 1120, which may include integrated graphics logic, Image processor, audio processor and video processor; static random access memory (SRAM) unit 1130; direct memory access (DMA) unit 1160. In one embodiment, the coprocessor 1120 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, a graphics processor General Purpose Computing (General Purpose Computing on GPU, GPGPU), a high-throughput MIC processor, or embedded Type processor and so on.

The various embodiments of the mechanism disclosed in this application may be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system. The programmable system includes at least one processor and a storage system (including volatile and non-volatile memory and/or storage elements) , At least one input device and at least one output device.

Program codes can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purpose of this application, the processing system includes any processor having a processor such as, for example, a digital signal processor (Digital Signal Processing, DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor. system.

The program code can be implemented in a high-level programming language or an object-oriented programming language to communicate with the processing system. When needed, assembly language or machine language can also be used to implement the program code. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments can also be implemented as instructions carried by or stored on one or more transient or non-transitory machine-readable (eg, computer-readable) storage media, which can be executed by one or more processors Read and execute. For example, the instructions can be distributed through a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (for example, a computer), including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magnetic CD, Read Only Memory (ROM), Random Access Memory (RAM), Erasable Programmable Read Only Memory (EPROM), Electrically Erasable and Programmable Electronically Erasable Programmable Read-Only Memory (EEPROM), magnetic or optical card, flash memory, or used to transmit information by using the Internet to transmit information in electrical, optical, acoustic or other forms (for example, carrier waves, infrared signals, digital signals) Etc.) tangible machine-readable memory. Therefore, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (for example, a computer).

In the drawings, some structural or method features may be shown in a specific arrangement and/or order. However, it should be understood that such a specific arrangement and/or ordering may not be required. Rather, in some embodiments, these features may be arranged in a different manner and/or order than shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

It should be noted that each unit/module mentioned in each device embodiment of this application is a logical unit/module. Physically, a logical unit/module can be a physical unit/module or a physical unit/ A part of the module can also be realized by a combination of multiple physical units/modules. The physical realization of these logical units/modules is not the most important. The combination of the functions implemented by these logical units/modules is the solution to this application. The key to the technical question raised. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units/modules that are not closely related to solving the technical problems proposed by this application. This does not mean that the above-mentioned device embodiments do not exist. Other units/modules.

It should be noted that in the examples and specification of this patent, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply There is any such actual relationship or sequence between these entities or operations. Moreover, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes those that are not explicitly listed Other elements of, or also include elements inherent to this process, method, article or equipment. If there are no more restrictions, the element defined by the phrase "including one" does not exclude the existence of other same elements in the process, method, article, or equipment that includes the element.

Although the present application has been illustrated and described by referring to certain preferred embodiments of the present application, those of ordinary skill in the art should understand that various changes can be made in form and details without departing from the present application. The spirit and scope of the application.

Claims

A message processing method for a terminal, which is characterized in that it comprises:

The terminal determines where the user of the terminal gazes at the terminal screen, where at least one message is displayed on the terminal screen;

Determining, by the terminal, the progress of the user browsing the message in a case where it is determined that the position where the user gazes at the terminal screen falls within the message range of one of the messages displayed on the terminal screen;

According to the progress of the user browsing the message, the display mode of at least part of the display elements within the message range of the message on the terminal screen is changed from the first display mode to the second display mode, wherein the display mode is not browsed by the terminal user The display mode of the message is the first display mode.
The method according to claim 1, wherein the message range of the message is the area occupied by the message box of the message on the screen of the terminal.
The method according to claim 2, wherein the display mode of at least a part of the display elements within the message range of the message on the terminal screen is changed from the first display according to the progress of the user browsing the message. The mode change to the second display mode includes:

When the user has finished browsing the message, change the background color of the message within the message range from the first color to the second color, wherein the color of the message that has not been browsed by the terminal user is the The first color.
The method according to claim 2, wherein the display mode of at least a part of the display elements within the message range of the message on the terminal screen is changed from the first display according to the progress of the user browsing the message. The mode change to the second display mode includes:

According to the progress of the user browsing the message, the background color of the area corresponding to the part of the message that has been browsed within the message range of the message is changed from the first color to the second color, wherein the message The background color of the part that is not browsed by the terminal user is the first color.
The method according to claim 2, wherein when the terminal determines that the position where the user gazes at the terminal screen falls within the message range of one of the messages displayed on the terminal screen, Determining the progress of the user browsing the message includes:

The terminal calculates that the position of the user's gaze on the terminal screen falls within the message box of one of the messages displayed on the terminal screen in the case of determining that the position where the user gazes on the terminal screen falls within the message The time in the box;

Determine the progress of the user browsing the message according to the calculated ratio of the time to the set time threshold; or,

The calculated time is compared with a set time threshold, and the progress of the user browsing the message is determined according to the comparison result.
The method according to claim 5, wherein the value of the set time threshold is related to at least one of the number of message words and the number of lines of the message, and the cumulative display length of the message.
The method according to claim 2, wherein when the terminal determines that the position where the user gazes at the terminal screen falls within the message range of one of the messages displayed on the terminal screen, Determining the progress of the user browsing the message includes:

In the case that the terminal determines that the position where the user gazes at the terminal screen falls within the message box of one of the messages displayed on the terminal screen, the terminal calculates the position based on the position where the user gazes at the terminal screen The cumulative distance of the user browsing the message box;

Determine the progress of the user browsing the message according to the calculated ratio of the cumulative distance to the set distance threshold; or,

The calculated cumulative distance is compared with a set distance threshold, and the progress of the user browsing the message is determined according to the comparison result.
8. The method according to claim 7, wherein the value of the set distance threshold is related to at least one of the number of message words and the number of lines of the message, and the cumulative display length of the message.
The method according to claim 1, wherein the message range includes an area occupied by the message itself on the screen of the terminal.
The method according to claim 9, wherein the display mode of at least a part of the display elements within the message range of the message on the terminal screen is changed from the first display according to the progress of the user browsing the message. The mode change to the second display mode includes:

According to the progress of the user browsing the message, the color displayed on the terminal screen of the part of the message that has been viewed is changed from the first color to the second color, wherein among the messages displayed on the terminal screen The color of the message that is not browsed by the terminal user is the first color.
The method according to claim 1, wherein the display mode of at least a part of the display elements within the message range of the message on the terminal screen is changed from the first display according to the progress of the user browsing the message. The mode change to the second display mode includes:

In the case of determining that the message has been viewed, the message is folded and displayed on the terminal screen.
The method according to claim 1, wherein the display mode of at least a part of the display elements within the message range of the message on the terminal screen is changed from the first display according to the progress of the user browsing the message. The mode change to the second display mode includes:

In the case of determining that the message has been viewed, an icon indicating that the message has been viewed is displayed within the message range of the message on the terminal screen.
The method according to claim 1, wherein the method further comprises: categorizing at least a part of the display elements within the message range of the message on the terminal screen according to the progress of the user browsing the message Change from the first classification to the second classification.
The method according to claim 1, wherein the display mode of at least a part of the display elements within the message range of the message on the terminal screen is changed from the first display according to the progress of the user browsing the message. The mode change to the second display mode includes:

According to the progress of the user browsing the message, the display mode of at least a part of the display elements within the message range of the message on the terminal screen is changed from the first display mode to the second display mode via an intermediate transition state, wherein The intermediate transition state includes at least one display mode different from the first display mode and the second display mode.
A terminal, characterized in that it comprises:

An eye tracking module, configured to determine the position where the user of the terminal gazes at the terminal screen, and at least one message is displayed on the terminal screen;

The progress calculation module is configured to determine that the user browses the message when the terminal determines that the position where the user gazes at the terminal screen falls within the message range of one of the messages displayed on the terminal screen Progress

The message management module is configured to change the display mode of at least a part of the display elements within the message range of the message on the terminal screen from the first display mode to the second display mode according to the progress of the user browsing the message, wherein , The display mode of the message that is not browsed by the terminal user is the first display mode.
A computer-readable medium, characterized in that instructions are stored on the computer-readable medium, and when the instructions are executed on a computer, the computer executes the method according to any one of claims 1-14.
An electronic device, characterized in that it comprises:

Memory, used to store instructions executed by one or more processors of the system, and

The processor is one of the processors of the system and is used to execute the method described in any one of claims 1-14.