WO2024037379A1 - Notification checking method and system, and related apparatus - Google Patents

Abstract

Disclosed in the present application are a notification checking method and system, and a related apparatus. When an electronic device displays a notification prompt window in a first area of a touch-control screen, the electronic device can acquire any one of a facial image, an eye image, eye movement information and electrooculogram information of a user, and according to same, determines whether a gaze area of the user is the first area. When the gaze area of the user is located in the first area, the electronic device can receive and respond to an operation of the user for an area on the touch-control screen other than the first area, and the electronic device checks the content of a notification or cancels the display of the notification prompt window. In this way, when a user holds an electronic device in a single hand, the user can also conveniently check the specific content of a notification.

Description

Notification viewing method, system and related devices

This application claims priority to the Chinese patent application submitted to the China Patent Office on August 18, 2022, with application number 202210995037.7 and the application title "A notification viewing method, system and related devices", the entire content of which is incorporated by reference. in this application.

Technical field

The present application relates to the field of terminal technology, and in particular, to a notification viewing method, system and related devices.

Background technique

With the continuous development of terminal technology, electronic devices such as smartphones and tablets have become more and more closely connected with daily life. People often send information to each other through electronic devices to achieve efficient communication.

When an electronic device receives information or notifications from other applications, a notification prompt window will be displayed at the top. Whether the user needs to view a notification or quickly close the notification prompt window, he or she needs to operate the notification prompt window on the top of the electronic device. After receiving the user's operation on the notification prompt window, the electronic device will display the specific content of the notification or close the notification prompt window according to the user's operation.

However, using the above method, when the user holds the electronic device with one hand, it is inconvenient for the user to operate the notification prompt window on the top with one hand.

Contents of the invention

This application provides a notification viewing method, system and related devices. When the electronic device displays a notification prompt window in the first area of the touch screen, the electronic device can obtain any of the user's facial image, eye image, eye movement information, and electrooculogram information, and determine the user accordingly. Whether the gaze area is the first area. When the user's gaze area is located in the first area, the electronic device can receive and respond to the user's operation on an area other than the first area on the touch screen, view the content of the notification or cancel the display of the notification prompt window. This enables users to conveniently view notification content when holding the electronic device with one hand.

In a first aspect, this application provides a notification viewing method, including: a first electronic device displays a first interface; the first electronic device obtains a first notification of a first application; in response to the first notification, the first electronic device touches The first area of the control screen displays a first notification prompt window, and the first notification prompt window includes part or all of the content of the first notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device Receive the user's first operation on the second area, which is an area on the touch screen other than the first area; after the first electronic device detects the first operation, the first electronic device overlays and displays the first operation on the first interface. A floating window, the first floating window includes an interface of the first application, and the interface of the first application includes the content of the first notification.

In this way, when the user holds the electronic device with one hand, the user can conveniently view the specific content of the notification.

In a possible implementation, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes: the first electronic device outputs gaze feedback, and the gaze feedback is used to prompt that the user's gaze area is located in the first region. .

In a possible implementation, outputting gaze feedback includes any one or more of the following: changing the display color of the first interface, changing the display color of the first notification prompt window, changing the color of the controls in the first notification prompt window, Light up the indicator light, vibrate and play a tone of the first electronic device.

In a possible implementation, the first electronic device outputs gaze feedback, which specifically includes: when the first electronic device detects that the user gazes at the first area for longer than the first duration, the first electronic device outputs gaze feedback.

In this way, the first electronic device can prompt the user's gaze area to be located in the first area through gaze feedback.

In a possible implementation, the first operation is a sliding operation; after the first electronic device receives the user's first operation on the second area, the method further includes: when the movement distance of the first operation on the touch screen is greater than When the first movement distance is reached, the first electronic device moves the display position of the first notification prompt window; when the movement distance of the first operation on the touch screen is greater than the second movement distance, the first electronic device expands the first notification prompt window. to form a first floating window; wherein the second movement distance is greater than the first movement distance.

In another possible implementation, the first operation is a sliding operation; after the first electronic device receives the user's first operation on the second area, the method further includes: when the movement distance of the first operation on the touch screen When the first movement distance is greater than the first movement distance, the first electronic device moves the display position of the first notification prompt window; when the movement distance of the first operation on the touch screen is greater than the second movement distance, the first electronic device displays the first floating window, And the display of the first notification prompt window is cancelled, and the second moving distance is greater than the first moving distance.

In this way, the first electronic device can control the first notification prompt window to perform different functions based on the movement distance of the first operation on the touch screen.

In a possible implementation, the first operation is a sliding operation; after the first electronic device receives the user's first operation for the second area, the method further includes: when the duration of the first operation is greater than the second duration, The first electronic device moves the display position of the first notification prompt window; when the duration of the first operation is greater than the third duration, the first electronic device expands the first notification prompt window into a first floating window, and the third duration is longer than the second duration. duration.

In another possible implementation, the first operation is a sliding operation; after the first electronic device receives the user's first operation on the second area, the method further includes: when the duration of the first operation is greater than the second duration , the first electronic device moves the display position of the first notification prompt window; when the duration of the first operation is greater than the third duration, the first electronic device displays the first floating window, and cancels the display of the first notification prompt window, and the third duration greater than the second duration.

In this way, the first electronic device can control the first notification prompt window to perform different functions based on the duration of the first operation.

In a possible implementation, the method further includes: after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives a second operation from the user for the second area; in response to the second operation, The first electronic device displays an interface of the first application.

In a possible implementation, the method further includes: the first electronic device obtains a second notification of the first application; in response to the second notification, the first electronic device displays a second notification prompt window in the first area of the touch screen , the second notification prompt window includes part or all of the content of the second notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's second operation for the second area; responds In the second operation, the first electronic device displays the interface of the first application, and the interface of the first application includes the content of the second notification.

In a possible implementation, the method further includes: the first electronic device obtains a third notification from the second application; in response to the third notification, the first electronic device displays a third notification prompt window in the first area of the touch screen , the third notification prompt window includes part or all of the content of the third notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's second operation for the second area; responds In the second operation, the first electronic device displays the interface of the second application, and the interface of the second application includes the content of the third notification.

In a possible implementation, the method further includes: after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives a third operation from the user for the second area; in response to the third operation, The first electronic device cancels display of the first notification prompt window.

In a possible implementation, the method further includes: the first electronic device obtains a second notification of the first application; in response to the second notification, the first electronic device displays a second notification prompt window in the first area of the touch screen , the second notification prompt window includes part or all of the content of the second notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's third operation for the second area; responds In the third operation, the first electronic device cancels display of the second notification prompt window.

In a possible implementation, the method further includes: the first electronic device obtains a third notification from the second application; in response to the third notification, the first electronic device displays a third notification prompt window in the first area of the touch screen , the third notification prompt window includes part or all of the content of the third notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's third operation for the second area; responds In the third operation, the first electronic device cancels display of the third notification prompt window.

In a possible implementation, after the first electronic device cancels displaying the first notification prompt window, the method further includes: the first electronic device redisplays the first notification prompt window after a fourth period of time.

In a possible implementation, after the first electronic device cancels display of the first notification prompt window, the method further includes: the first electronic device displays the floating ball; the first electronic device receives the user's input for the floating ball; and responds to the user In response to input from the floating ball, the first electronic device displays a first notification prompt window or a first floating window.

In a possible implementation, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes: when the first electronic device receives the user's fourth operation for the second area, the first electronic device The first function corresponding to the first interface is executed, and the fourth operation is different from the first operation.

In a possible implementation, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes: when the first electronic device determines that the user's gaze area changes to the second area, and the first notification prompts When the display duration of the window is longer than the fifth duration, the first electronic device cancels display of the first notification prompt window.

In a possible implementation, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes: the first electronic device displays an operation prompt, and the operation prompt is used to prompt the user to perform an operation on the first area in the second area. Notification prompt window to control.

In a possible implementation, the first area is a top area of the touch screen.

In a possible implementation, the second area is a bottom area of the touch screen. In another possible implementation, the second area may also be the middle area of the touch screen.

In a possible implementation, the method further includes: the first electronic device obtains gaze information, and the gaze information includes any one or more of the following: the user's eye image, the user's facial image, eye movement information, and electrooculogram information. ; The first electronic device determines that the user's gaze area is located in the first area based on the gaze information.

In a possible implementation, the first electronic device obtains the gaze information, which specifically includes: the first electronic device receives the gaze information sent by the second electronic device.

In a possible implementation, the method further includes: the first electronic device receives a line of sight estimation result sent by the second electronic device, the line of sight estimation result includes that the user's gaze area is located in the first area; the first electronic device determines based on the line of sight estimation result The user's gaze area is located in the first area.

In a possible implementation, the method further includes: the first electronic device receiving the user's gaze direction sent by the second electronic device; and the first electronic device determining that the user's gaze area is located in the first area based on the user's gaze direction.

In a second aspect, this application provides an electronic device, which is a first electronic device including: one or more processors and one or more memories. The one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program codes. The computer program codes include computer instructions. When the one or more processors execute the computer instructions, the first electronic The device performs the notification viewing method in any of the possible implementations of any of the above aspects.

In a third aspect, the present application provides a computer storage medium that includes computer instructions. When the computer instructions are run on an electronic device, the electronic device causes the electronic device to execute any of the above aspects in a possible implementation manner. How to view notifications.

In a fourth aspect, this application provides a computer program product. When the computer program product is run on a computer, it causes the computer to execute the notification viewing method in any of the possible implementations of any of the above aspects.

Description of drawings

Figure 1 is a schematic diagram of the one-hand operation range when an electronic device is held with one hand according to an embodiment of the present application;

FIG. 2A is a schematic diagram of an electronic device 100 provided by an embodiment of the present application;

Figure 2B is a schematic structural diagram of a system 10 provided by an embodiment of the present application;

Figure 3 is a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application;

Figure 4 is a schematic structural diagram of an electronic device 200 provided by an embodiment of the present application;

Figures 5A-5H are schematic interface diagrams of a notification viewing method provided by an embodiment of the present application;

Figures 6A-6B are schematic interface diagrams of another notification viewing method provided by an embodiment of the present application;

Figures 7A-7D are schematic interface diagrams of a method for canceling the display of a notification prompt window provided by an embodiment of the present application;

Figures 8A-8C are schematic interface diagrams of another method for canceling the display of a notification prompt window provided by an embodiment of the present application;

9A-9B are schematic interface diagrams of an electronic device responding to operations in a non-gaze operation set according to an embodiment of the present application;

Figure 10 is a schematic interface diagram of another electronic device outputting gaze feedback provided by an embodiment of the present application;

Figure 11A is a schematic flowchart of a notification viewing method provided by an embodiment of the present application;

Figures 11B to 11D show three different situations in which the gaze area is located in the first area provided by the embodiment of the present application;

Figure 12 is a schematic flowchart of a method for an electronic device according to an embodiment of the present application to execute a response corresponding to the operation based on the relationship between the received operation and the set of gaze operations;

Figure 13 is a schematic flowchart of another notification viewing method provided by an embodiment of the present application;

FIG. 14 is a schematic diagram of functional modules of an electronic device 100 provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and in detail below with reference to the accompanying drawings. Among them, in the description of the embodiments of this application, unless otherwise stated, "/" means or, for example, A/B can mean A or B; "and/or" in the text is only a way to describe related objects. The association relationship means that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiment of the present application , "plurality" means two or more than two.

Hereinafter, the terms “first” and “second” are used for descriptive purposes only and shall not be understood as implying or implying relative importance or implicitly specifying the quantity of indicated technical features. Therefore, the features defined as “first” and “second” may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, “plurality” The meaning is two or more.

The term "user interface (UI)" in the following embodiments of this application is a media interface for interaction and information exchange between an application or operating system and a user. It realizes the difference between the internal form of information and the form acceptable to the user. conversion between. The user interface is source code written in specific computer languages such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed graphically. It can be text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements displayed on the display screen of an electronic device.

Figure 1 shows a schematic diagram of the one-hand operation range when the electronic device is held with one hand.

As shown in Figure 1, when the user holds the electronic device with one hand, the touch screen of the electronic device can be divided into the following three areas according to the difficulty of the user's one-hand operation: area 1001, area 1002 and area 1003 . Among them, area 1001 is an area that is difficult to operate when the user holds it with one hand, area 1003 is an area that is inconvenient to operate when the user holds it with one hand, and area 1002 is an area that is easy to operate when the user holds it with one hand.

When the electronic device obtains a new notification, the electronic device displays a notification prompt window on the top of the touch screen. The display area of the notification prompt window is located in the area 1001 shown in Figure 1 . Therefore, when the user holds the electronic device with one hand, it is inconvenient for the user to operate the notification prompt window with one hand.

The execution subject of the notification viewing method provided by the embodiment of the present application may be the electronic device 100 .

FIG. 2A shows a schematic diagram of an electronic device 100 provided by an embodiment of the present application.

As shown in Figure 2A, the electronic device 100 includes a front camera 101. The front camera 101 can be used to obtain the user's face or eye image. The face or eye image can be used to estimate the user's line of sight and determine the relationship between the user's line of sight and the electronic device. 100 touch screen intersection. When the electronic device 100 displays the top notification prompt window, the electronic device 100 can view the notification content corresponding to the notification prompt window or close the notification prompt window based on the result of the line of sight estimation and the user's operation on the touch screen.

It should be noted that the electronic device 100 may be a mobile phone as shown in FIG. 2A or other electronic devices such as a tablet computer.

The embodiment of the present application also provides a system that can execute a notification viewing method. The following introduces the architecture of a system provided by embodiments of this application.

As shown in FIG. 2B , system 10 may include electronic device 100 and electronic device 200 .

The electronic device 100 may be a mobile phone as shown in FIG. 2B , a tablet computer, or the like. The electronic device 200 may be smart glasses as shown in Figure 2B, or it may be an augmented reality (Augmented Reality, AR) device, a Virtual Reality (VR) device, or the like.

Electronic device 200 may be used to obtain images of the user's face or eyes. The electronic device 200 may send the user's face or eye image to the electronic device 100 for the electronic device 100 to estimate the user's gaze. In some embodiments, the electronic device 200 can also obtain the positional relationship between the electronic device 100 and the electronic device 200, and estimate the user's line of sight based on the image of the user's face or eyes and the positional relationship between the electronic device 100 and the electronic device 200. The intersection point of the user's line of sight and the touch screen of the electronic device 100 is determined, and the result of the line of sight estimation is sent to the electronic device 100 . The electronic device 100 can view the notification content corresponding to the notification prompt window or close the notification prompt window based on the result of the line of sight estimation and the user's operation on the touch screen.

The electronic device 100 and the electronic device 200 may communicate through Bluetooth technology (including basic rate (BR)/enhanced data rate (EDR) Bluetooth and Bluetooth low energy (BLE)).

The following describes a notification viewing method based on line of sight estimation and inertial measurement unit (IMU) tracking provided by embodiments of the present application.

The electronic device 100 can obtain the user's facial image through the front camera, and determine whether the area the user is looking at is the area displayed by the notification prompt window based on the user's facial image. When it is determined that the user's gaze area is within the area displayed in the notification prompt window, the electronic device 100 may perform corresponding operations in conjunction with the motion of the electronic device 100 measured by the IMU. For example, when the electronic device 100 detects that the electronic device 100 is approaching When the user moves, the electronic device 100 may display a floating window, and display the specific content of the notification corresponding to the notification prompt window in the floating window. For another example, when the electronic device 100 determines that the electronic device 100 swings to the right with the hand, the electronic device 100 determines to ignore the notification and moves the notification prompt window to the right until it disappears completely. For another example, when the electronic device 100 determines that the electronic device 100 swings to the left with the hand, the electronic device 100 may determine to display the notification prompt window later, and move the notification prompt window to the left until it disappears completely, etc. wait.

Using the above method, users can view the specific content of the notification through one-handed operation, or quickly cancel the notification prompt window at the top of the display, etc. However, there may be misjudgments when the IMU recognizes actions. The electronic device 100 may not be accurately recognized as the user's hand swings or approaches, resulting in the electronic device 100 being unable to perform corresponding operations. In addition, in some special scenarios, for example, when the user carries heavy objects with both hands, the actions in the above method (such as swinging, approaching the user, etc.) are also inconvenient for the user to operate.

An embodiment of the present application provides a notification viewing method. When the electronic device 100 displays a notification prompt window in the first area of the touch screen, the electronic device 100 can obtain the gaze information, and determine whether the user's gaze area is located in the first area based on it. After determining that the user's gaze area is located in the first area, the electronic device 100 may receive and respond to the user's operation on the second area on the electronic device 100, view the content of the notification or cancel the display of the notification prompt window, etc. The second area is the electronic device 100 for areas outside the first area. In this way, when the user holds the electronic device 100 with one hand, the user can conveniently operate the notification prompt window (for example, view the specific content of the notification or quickly cancel the display of the notification prompt window, etc.).

FIG. 3 shows a schematic structural diagram of the electronic device 100.

The electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (personal digital assistant) digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, vehicle-mounted device, smart home device and/or Smart city equipment, the embodiment of this application does not place special restrictions on the specific type of electronic equipment.

The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, sensor module 180, button 190, camera 193, display screen 194, and subscriber identification module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a touch sensor 180K and the like.

It can be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors.

The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 110 . If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.

The charging management module 140 is used to receive charging input from the charger.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, resistance resistance) and other parameters. In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: Antenna 1 can be reused as a diversity antenna for a wireless LAN. In other embodiments, antennas may be used in conjunction with tuning switches.

The mobile communication module 150 can provide solutions for wireless communication including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be disposed in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.

A modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110 and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (bluetooth, BT), and global navigation satellites. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi) -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is an image processing microprocessor and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used to display images, videos, etc. Display 194 includes a display panel. The display panel can use a liquid crystal display (LCD). The display panel can also use an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (OLED). Manufacturing of active-matrix organic light emitting diodes (AMOLED), flexible light-emitting diodes (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera sensor through the lens, the optical signal is converted into an electrical signal, and the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise and brightness. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193.

Camera 193 is used to capture still images or video. The object passes through the lens to produce an optical image that is projected onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can continuously learn by itself. Intelligent cognitive applications of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, etc.

The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

Touch sensor 180K, also known as "touch device". The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation on or near the touch sensor 180K. The touch sensor can pass the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a location different from that of the display screen 194 .

The buttons 190 include a power button, a volume button, etc. Key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .

Optionally, the electronic device 100 may also include a motor 191, and the motor 191 may generate a vibration prompt. The motor 191 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. The motor 191 can also correspond to different vibration feedback effects for touch operations in different areas of the display screen 194 . Different application scenarios (such as time reminders, receiving information, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

Optionally, the electronic device 100 may also include an indicator 192. The indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to indicate messages, missed calls, notifications, etc.

Figure 4 shows a schematic structural diagram of an electronic device 200.

As shown in FIG. 4 , the electronic device 200 may include: a processor 201 , a memory 202 , a Bluetooth communication module 203 , a camera module 204 and a power module 205 . Optionally, the electronic device 200 may also include a button 206 . in:

Processor 201 may be used to read and execute computer readable instructions. In specific implementation, the processor 201 may mainly include a controller, arithmetic unit, and registers. Among them, the controller is mainly responsible for decoding instructions and issuing control signals for operations corresponding to the instructions. The arithmetic unit is mainly responsible for saving register operands and intermediate operation results temporarily stored during instruction execution. In specific implementation, the hardware architecture of the processor 201 may be an application specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, etc.

Memory 202 is coupled to processor 201 for storing various software programs and/or sets of instructions. In specific implementations, the memory 202 may include high-speed random access memory. In some embodiments, the memory 202 may also include non-volatile memory. Memory 202 may store communication programs that may be used to communicate with electronic device 100 or other devices. In other embodiments, the memory 202 can also store operating systems, such as uCOS, VxWorks, RTLinux and other embedded operating systems.

The Bluetooth communication module 203 can be used to establish a communication connection with the electronic device 100 . The Bluetooth communication module 203 may include a Bluetooth chip. The electronic device 200 can pair with the Bluetooth chip of the electronic device 100 and establish a Bluetooth connection through the Bluetooth chip, so as to realize wireless communication and business processing between the electronic device 200 and the electronic device 100 through the Bluetooth connection. Usually, the Bluetooth chip can support BR/EDR Bluetooth and BLE, for example, it can receive/method paging (page) information, receive/send BLE broadcast messages, etc. In some embodiments, the Bluetooth communication module 203 may also include an antenna. The Bluetooth communication module 203 receives electromagnetic waves via the antenna, frequency modulates and filters the electromagnetic wave signals, and processes the processed electromagnetic waves. The signal is sent to processor 201. The Bluetooth communication module 203 can also receive the signal to be sent from the processor 201, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna for radiation.

The camera module 204 may be used to obtain images of the user's eyes and/or face. In specific implementation, the camera module 204 may include 1 or N cameras, where N is a positive integer greater than 1. Cameras are used to capture still images or videos. The object passes through the lens to produce an optical image that is projected onto the photosensitive element. The photosensitive element may be a charge coupled device or a complementary metal oxide semiconductor phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals. In some embodiments, the camera module 204 may also be used to acquire images of the electronic device 100 .

The power module 205 can be used to provide system power for the electronic device 200, power each module of the electronic device 200, and support the electronic device 200 to receive charging input, etc. The power module 205 may include a power management unit (PMU) and a battery. Among them, the power management unit can receive external charging input; provide the electrical signal input by the charging circuit to charge the battery; and can also provide the electrical signal provided by the battery to other modules in the electronic device 200 to prevent the battery from overcharging and over-discharging. , short circuit or overcurrent, etc. In some embodiments, the power module 205 may also include a wireless charging coil for wirelessly charging the electronic device 200 . In addition, the power management unit can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters.

Optionally, in some embodiments, the electronic device 200 may also include buttons and/or sensors (not shown in FIG. 4 ) and the like. The buttons can be physical buttons or touch buttons (used in conjunction with touch sensors), etc., used to trigger operations such as power on, power off, pause, play, record, start pairing, reset, etc. The touch sensor can detect the user's touch operations such as single click, double click, multiple clicks, long press, heavy pressure, etc. It can also perform user fingerprint recognition to authenticate the user's identity in security-sensitive business scenarios.

It is understood that the electronic device 200 shown in FIG. 4 is only an example, and the electronic device 200 may have more or fewer components than shown in FIG. 4 , two or more components may be combined, or Can have different component configurations. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.

The following is a detailed introduction to a notification viewing method provided by the embodiment of this application based on application scenarios.

In some application scenarios, the electronic device 100 may obtain the first notification of the first application (for example, information, notification of application software, etc.) when displaying the first interface (for example, desktop, interface in application software, etc.). etc). In response to the first notification, the electronic device 100 may display a notification prompt window in the first area of the touch screen, and the notification prompt window includes part or all of the content of the first notification. The electronic device 100 may determine the user's gaze area based on gaze information (eg, the user's facial image, the user's eye image, the user's electrooculogram information, the user's eye movement information, etc.). After the electronic device 100 determines that the user's gaze area is located in the first area, the electronic device 100 may receive and respond to the user's request for a second area of the electronic device 100 (ie, an area other than the first area on the touch screen of the electronic device 100). Operation 1 (for example, sliding operation), display a floating window in the first interface. The floating window is used to display the interface of the first application. The interface of the first application includes the specific content of the first notification. Optionally, after determining that the user's gaze area is located in the first area (for example, the electronic device 100 detects that the user's gaze area is located in the first area, or the electronic device 100 detects that the duration of the user's gaze at the first area is greater than or equal to first time threshold, etc.), the electronic device 100 can also output gaze feedback. The gaze feedback can be changing the display color of the first interface, changing the color of the notification prompt window, or changing the color of the horizontal bar below the notification message, etc. Optionally, the electronic device 100 may also determine whether to display the interface of the first application in the floating window based on the distance moved on the touch screen in operation 1.

Exemplarily, as shown in FIG. 5A , the electronic device 100 displays a desktop interface 500 . The desktop interface 500 displays a page on which application icons are placed. The page includes multiple application icons (for example, a stock application icon, a calculator application icon, a browser application icon, a social application icon 501, etc.). A page indicator 502 is also displayed below the page where the application icon is placed to indicate the total number of pages on the desktop interface 500 and the positional relationship between the currently displayed page and other pages. For example, the desktop interface 500 may include three pages, and the white dot in the page indicator is located at the third position from the left, which may indicate that the currently displayed page is the third page from the left among the three pages. Optionally, a status bar 503 is also displayed above the page where the application icon is placed. The status bar 503 may include information such as a strength indicator of the communication signal, a power value, and time. Further optionally, there may be a dock area 504 below the page indicator, and the dock area 504 may include one or more tray icons (for example, a dialing application icon, a messaging application icon, a contact application icon, a camera application icons, etc.), one or more tray icons in the dock area can remain displayed when switching pages.

The electronic device 100 may receive and respond to the user's click operation on the social application icon 501, open the social application software, and display the social application interface 510.

As shown in FIG. 5B , the social application interface 510 includes one or more graphic and text contents (for example, graphic and text content 511 ), a range selection bar 512 and an auxiliary control bar 513 .

Graphic content 511 may include user avatar 511a, user name 511b, picture 511c, forwarding control 511d, comment control 511e and like control 511f. The user avatar 511a may be used to indicate the avatar of the user who posted the graphic content 511, and the user name 511b may be used to indicate the nickname of the user who posted the graphic content 511, and so on. The forwarding control 511d can be used to trigger the electronic device 100 to forward the graphic content 511, the comment control 511e can be used to trigger the electronic device 100 to display a comment input interface for the graphic content 511, and the like control 511f can be used to trigger the electronic device 100 to change the graphic content 511. The like status of 511 changes the color of the like control 511f. For example, when the color of the like control 511f is white as shown in FIG. 5B, the electronic device 100 can respond to the user's click operation on the like control 511f, change the color of the like control 511f from white to black, that is, as shown in FIG. The like status of article content 511 was changed from unliked to liked. In some embodiments, the graphic content 511 may also include more or less pictures, text, etc., which is not limited in this application.

The range selection column 512 may include one or more range options, such as a focus option, a recommendation option 512a, and a local option. The range option can be used to trigger the electronic device 100 to display one or more graphic contents corresponding to the range option. In some embodiments, the selected range option may be displayed in bold font, text in different colors, different background colors, etc. Taking FIG. 5B as an example, the recommended option 512a is displayed in bold font, that is, the recommended option 512a is in a selected state, indicating that one or more graphic and text contents displayed in the social application interface 510 are graphic and text contents recommended by the social application software. In some embodiments, the range selection bar 512 may also include one or more controls, such as a photo control 512b and an add control 512c. The camera control 512b can be used to trigger the electronic device 100 to display a shooting interface, and the add control 512c can be used to trigger the electronic device 100 to add one or more range options.

The auxiliary control bar 513 may include one or more auxiliary controls, such as a homepage control 513a, a search control 513b, a message control 513c, and a personal homepage control 513d. Among them, the homepage control 513a can be used to trigger the electronic device 100 to display the homepage of the social application software. The search control 513b may be used to trigger the electronic device 100 to display a search interface. The message control 513c can be used to trigger the electronic device 100 to display a message interface in the social application software. The personal homepage control 513d may be used to trigger the electronic device 100 to display the user's personal homepage in the social application software.

When the electronic device 100 displays the social application interface 510, the electronic device 100 can obtain the notification of the Changlian application. While the electronic device 100 can obtain the notification of the Changlian application, the notification prompt window 520 as shown in FIG. 5C is displayed in the first area of the touch screen. The notification prompt window 520 is used to prompt the user that the electronic device 100 has received the Changlian application. Even app notifications.

As shown in Figure 5C, the notification prompt window 520 may include an icon 520a. Optionally, the notification prompt window 520 may also include any one or more of the following: sender 520b, notification brief description 520c, and read mark control 520d. , application name 520e, notification time 520f, horizontal bar 520g. Among them, the icon 520a can be used to prompt the application software corresponding to the notification, that is, the Changlian application software. The application software name 520e can also be used to prompt that the application software corresponding to the notification is the Changlian application software. The sender 520b can be used to prompt the sender of the notification, that is, Xiaohong. Notification brief description 520c can be used to prompt the content of the notification, that is, "There is a good exhibition on Saturday, do you want to go see it?". It can be understood that when the number of words in the notification content exceeds a certain number of words (for example, 20 words), the notification summary 520c may only display a part of the text in the notification content, for example, only the first 15 words in the notification content may be displayed. Text, etc. In some embodiments, notification content may also be hidden. Mark as read control 520d may be used to trigger the electronic device 100 to mark the notification as read. The notification time 520f may be used to prompt the electronic device 100 to receive the notification, such as "just now". The horizontal bar 520g can be used to trigger the electronic device 100 to cancel the display of the notification prompt window 520 . In some embodiments, the notification prompt window 520 may also include more or less content than the notification prompt window 520 shown in FIG. 5C , which is not limited in this application.

When the electronic device 100 obtains the notification from the Changlian application, the electronic device 100 may obtain images of the user's eyes and/or face. As shown in FIG. 5D , the electronic device 100 can acquire an image of the user's eyes and/or face while displaying the notification prompt window. The electronic device 100 can determine the direction of the user's gaze based on the image of the user's eyes or face, and further determine whether the area the user is looking at is within the area (first area) where the notification prompt window 520 is located.

As shown in FIG. 5E , when the electronic device 100 detects that the user's gaze area is located in the first area, the electronic device 100 can change the display color of the social application interface 510 in the background layer of the notification prompt window 520 . In some embodiments, the electronic device 100 may also change the display color of the social application interface 510 in the background layer of the notification prompt window 520 when detecting that the user's gaze time on the first area exceeds a first time threshold (eg, 500 milliseconds).

After the electronic device 100 detects that the user's gaze area is located in the first area, or when the electronic device 100 outputs gaze feedback, the electronic device 100 may receive the user's response to an area other than the first area on the touch screen of the electronic device 100 (in this application It can also be called the sliding operation of the second area). In some embodiments, when the electronic device 100 detects that the user's sliding operation for the second area on the touch screen moves the distance on the touch screen not less than the first distance (for example, 0 pixels or 50 pixels, etc.) , as shown in Figure 5F, the electronic device 100 can move The notification prompt window 520 in the mobile social application interface 510 moves downward. Optionally, as shown in FIG. 5F , the notification prompt window 520 may be displayed in a blurred manner as it moves downward along with the user's sliding operation. It should be noted that in the embodiment of the present application, the first distance may also be written as the first movement distance.

In some embodiments, when the electronic device 100 detects that the duration of the user's sliding operation for the second area on the touch screen is not less than the first duration (for example, 1 microsecond or 1 millisecond, etc.), as shown in FIG. As shown in 5F, the electronic device 100 can move the notification prompt window 520 in the mobile social application interface 510 downward. In the embodiment of this application, the first duration can also be written as the second duration.

In some embodiments, when the electronic device 100 detects that the sliding distance moved on the touch screen is not less than a second distance (for example, 200 pixels), or when the electronic device 100 detects that the sliding operation continues, When the time is not less than the second duration (for example, 0.1 seconds), the notification prompt window 520 in the social application interface 510 can gradually become larger and expanded as the user slides down to form a floating window 530 as shown in Figure 5G. Wherein, the second distance is greater than the first distance, and the second duration is greater than the first duration. In the embodiment of the present application, the second distance can also be written as the second moving distance, and the second duration can also be written as the third duration.

As shown in Figure 5G, the floating window 530 may include an icon 531, and the icon 531 may be an application software icon corresponding to the notification, that is, a Changlian application software icon. Optionally, when the user continues to slide downward, the electronic device 100 may display a larger floating window than the floating window 530 shown in FIG. 5G in response to the sliding operation. In addition, optionally, the position of the floating window 530 can also move downward as the user continues to slide downward.

In other embodiments, when the electronic device 100 detects that the sliding distance moved on the touch screen is not less than a second distance (for example, 200 pixels), or when the electronic device 100 detects that the sliding operation moves When the duration is not less than the second duration (for example, 0.1 seconds), the electronic device 100 can cancel the display of the notification prompt window 520 and display the floating window 530 as shown in FIG. 5G.

When the electronic device 100 detects that the distance moved by the slide operation on the touch screen is not less than a third distance (for example, 500 pixels), or when the electronic device 100 detects that the duration of the slide operation is not less than a third duration. (for example, 1 second), the electronic device 100 can display the Changlian application interface 541 as shown in FIG. 5H in the floating window 530, where the third distance is greater than the second distance, and the third duration is greater than the second duration.

As shown in Figure 5H, the Changlian application interface 541 can be displayed in the floating window 530. The Changlian application interface 541 may include one or more messages, a sender identification 541b, and one or more controls for replying to messages (for example, a recording control 541c, a sending control 541d, a switching input control 541e, etc.).

The one or more messages include message 541a, which is used to prompt the specific content of the notification. The message 541a may include one or more of text, pictures, and voice. For example, the message 541a may be "There is a good exhibition on Saturday, do you want to go see it?"

The sender identification 541b is used to prompt the sender of the message 541a. The sender identification 541b may be the sender's avatar, or may be the sender's mobile phone number, user nickname, or other information that can prompt the sender's identity. The sender in the embodiment of this application refers to the electronic device that sends the message 541a.

The one or more controls for replying to a message may be used to reply to message 541a, that is, to send information to the sender of message 541a. The recording control 541c is used to trigger the electronic device 100 to record voice information. The sending control 541d is used to trigger the electronic device 100 to send the recorded voice message to the sender of the message 541a. The switching input control 541e is used to trigger the electronic device 100 to switch the current input mode from voice input to keyboard input, and display the keyboard and input field. The one or more controls for replying to messages may also include a picture control 541f, an envelope control 541g, an expression control 541h, a position control 541i, and an add control 541j. Among them, the picture control 541f can be used to send picture information. The envelope control 541g can be used to trigger the electronic device 100 to send the information to be sent in the form of envelope information. Expression control 541h can be used to send expressions. Location control 541i may be used to send location information of electronic device 100. Add Control 541j can be used to add additional controls.

The floating window 530 may also include one or more auxiliary controls, such as a full screen control 542, a minimize control 543, a drag control 544, and a close control 545. Among them, the full-screen control 542 can be used to trigger the electronic device 100 to open the Changlian application software and display the Changlian application interface 541 displayed in the floating window 530 in full screen. For details, please refer to the following description of the interface 600 shown in Figure 6B. This application is in This will not be described in detail yet. The minimization control 543 can be used to trigger the electronic device 100 to cancel the display of the floating window 530 and display a floating ball in the social application interface 510. The display size of the floating ball is much smaller than the display size of the floating window 530, and there can be a smooth display on the floating ball. Even application software icons. The drag control 544 can be used to trigger the electronic device 100 to move the floating window 530 along the user's sliding trajectory. The close control 545 can be used to cancel display of the floating window 530 .

It should be noted that after the electronic device 100 displays the floating window 530, the electronic device 100 may respond to the user's release operation and determine that the display position of the floating window 530 when the user releases the operation is the display position of the floating window 530.

Using the above notification viewing method, the electronic device 100 can also be based on the user's gaze area and the user's second area of the electronic device 100 For domain operations, check the notification content corresponding to the notification prompt window at the top. Since there is no need to operate the notification prompt window on the top of the electronic device 100, the user can easily view the notification content even in a scenario where the user operates with one hand.

In a possible implementation, when the electronic device 100 displays gaze feedback, the electronic device 100 can also receive and respond to the user's operation 2 (for example, click operation, long press operation) on the second area on the electronic device 100 , double-click operation or long-press operation with two fingers, etc.), open the application software corresponding to the notification prompt window, and display the interface of the application software.

Exemplarily, after receiving the notification from the Changlian application software, when the electronic device 100 detects that the area the user is looking at is within the first area, and the time the user is looking at the first area is not less than the first time threshold, the electronic device 100 100 may display social application interface 510 as shown in Figure 6A. The social application interface 510 is the same as the aforementioned social application interface 510 shown in FIG. 5E.

As shown in Figure 6A, the electronic device 100 can receive and respond to the user's long press operation on the second area, open the Changlian application software, and display the Changlian application interface 600 as shown in Figure 6B.

As shown in Figure 6B, the specific content of the Changlian application interface 600 can be referred to the relevant description of the Changlian application interface 541 displayed in the floating window 530 shown in Figure 5H, and will not be described again here.

In this way, the application software corresponding to the notification prompt window can be directly opened through a simpler operation, and the application software interface can be displayed. The time required to display the interface of the application software is shortened.

In some embodiments, when the electronic device 100 displays gaze feedback, the electronic device 100 may also receive and respond to the user's operation 3 on the second area on the electronic device 100 (for example, a double-click operation, a long press operation, or a two-finger long press operation). Press operation, etc.) to display the floating window 530 as shown in Figure 5H. It should be noted that operation 2 is different from operation 3.

In this way, you can also simplify the steps of viewing notifications and save time by defining operation 3.

In some application scenarios, after the electronic device 100 outputs gaze feedback, if the user does not want to view the specific content of the notification corresponding to the notification prompt window, the electronic device 100 can also respond to the user's operation 4 on the second area on the electronic device 100 (for example, , right swipe operation), cancel the display of the notification prompt window, for example, drag the notification prompt window to the right until it disappears completely, or cancel the display of the notification prompt window and display a floating ball, which can be used to trigger the electronic device 100 to display a notification A prompt window, or an interface used to trigger the electronic device 100 to display the first application. Optionally, in some embodiments, the electronic device 100 can also mark the notification corresponding to the notification prompt window as read in response to operation 4.

Exemplarily, after receiving the notification from the Changlian application software, when the electronic device 100 detects that the area the user is looking at is within the first area, and the time the user is looking at the first area is not less than the first time threshold, the electronic device 100 100 may display social application interface 510 as shown in Figure 7A. The social application interface 510 is the same as the aforementioned social application interface 510 shown in FIG. 5E.

As shown in FIG. 7A , the electronic device 100 may cancel the display of the notification prompt window 520 in response to the user's right sliding operation on the second area, and display the notification prompt window 701 as shown in FIG. 7B on the social application interface 510 .

As shown in FIG. 7B , the notification prompt window 701 is a part of the notification prompt window 520 shown in FIG. 7A , and the notification prompt window 701 is located farther in the social application interface 510 than the notification prompt window 520 is in the social application interface 510 . Keep Right.

When the electronic device 100 displays the social application interface 510 shown in FIG. 7B, if the user continues to perform a right sliding operation, that is, when the user continues to slide to the right, the electronic device 100 can receive and respond to the user's right sliding operation and continue to slide to the right. Drag the notification prompt window 701 until the notification prompt window 701 disappears completely. At this time, the electronic device 100 can cancel the display of the notification prompt window 701 and display the social application interface 510 shown in FIG. 7C. In some embodiments, the electronic device 100 may also change the notification prompt window 701 into a floating ball in response to the user's right sliding operation.

After detecting that the user's right-swipe operation on the electronic device 100 is completed, the electronic device 100 can cancel the display of the gaze feedback in the social application interface 510, change the display color of the social application interface 510, and display the social application interface 510 as shown in Figure 7D .

In some embodiments, when the electronic device 100 displays the social application interface 510 as shown in Figure 7C, the electronic device 100 can also cancel output of gaze feedback after the second time threshold (for example, 10 milliseconds) has passed, as shown in Figure 7D Social application interface 510 is shown.

Using the above method, when the user does not want to view the specific content of the notification, the electronic device 100 can also swipe away the notification prompt window based on the user's gaze area and the user's operation on the second area of the electronic device 100 . Similarly, since there is no need to operate the notification prompt window on the top of the electronic device 100, the user can easily perform the above operations even in a scenario where the user operates with one hand.

In some embodiments, after the electronic device 100 outputs the gaze feedback, if the user wants to view the specific content of the notification corresponding to the notification prompt window later, the electronic device 100 can also respond to the user's operation 5 on the second area on the electronic device 100 (for example, swipe left), mark the notification as a reminder later, cancel the display of the notification prompt window (for example, drag the notification prompt window to the left until it disappears completely), and after The notification prompt window is redisplayed after a period of time (for example, the fourth period of time). For specific implementation methods, reference may be made to the relevant content in the embodiments shown in FIGS. 7A to 7D .

In a possible implementation, when the electronic device 100 displays a notification prompt window, the electronic device 100 may also determine whether to cancel the display of the notification prompt window based on the duration of the user's gaze on the second area. In some embodiments, the electronic device 100 may also determine whether to cancel the display of the notification prompt window based on the display duration of the notification prompt window when detecting that the user is gazing at the second area.

Exemplarily, after receiving the notification from the Changlian application software, the electronic device 100 detects that the area the user is looking at is within the first area, and the time the user is looking at the first area is not less than the first time threshold. At this time, the electronic device 100 The device 100 may display a social application interface 510 as shown in FIG. 8A , which is the same as the social application interface 510 shown in FIG. 5E .

As shown in FIG. 8A , when the electronic device 100 detects that the user is gazing at the second area, and the gazing time is not less than the third time threshold, as shown in FIG. 8B , the electronic device 100 can cancel output of gaze feedback and change the social application interface 510 display color.

When the electronic device 100 detects that the user is gazing at the second area, and the gazing time is not less than the fourth time threshold, as shown in FIG. 8C , the electronic device 100 may cancel the display of the notification prompt window 520 . In this case, the fourth time threshold is greater than the third time threshold.

In a possible implementation, the electronic device 100 may also use the moment when the electronic device 100 displays the social application interface 510 as shown in FIG. 8B as a starting point, when it is determined that the time the user looks at the second area is greater than the fifth time threshold. , the notification prompt window 520 is cancelled, and the social application interface 510 shown in FIG. 8C is displayed.

In another possible implementation, when the electronic device 100 detects that the user is gazing at the second area and the display duration of the notification prompt window 520 is not less than the sixth time threshold, the social application interface 510 as shown in FIG. 8C is displayed.

In this way, the electronic device 100 can determine whether to cancel the display of the notification prompt window by detecting the duration of the user's gaze on the second area or by detecting the display duration of the notification prompt window when the user is gazing at the second area, without requiring the user to perform additional operations, simplifying the operating steps.

In some embodiments, after the electronic device 100 outputs the gaze feedback, the electronic device 100 may also perform the function of the first interface in response to the user's operation 6 on the electronic device 100 (eg, slide-up operation, etc.). It should be noted that this operation is different from any of the above operations (for example, operation 1, operation 2, operation 3, etc.).

For example, as shown in FIG. 9A , the electronic device 100 may display a social application interface 510 with gaze feedback. The social application interface 510 is the same as the aforementioned social application interface 510 shown in FIG. 5E .

The electronic device 100 may, in response to the user's upward sliding operation on the electronic device 100, slide down one or more graphic content in the social application interface 510 to display the graphic content 901 as shown in FIG. 9B.

As shown in FIG. 9B , the graphic and text content 901 is different from the graphic and text content in the social application interface 510 shown in FIG. 9A . Optionally, the social application interface 510 shown in FIG. 9B may also include a sliding bar 902, which may be used to trigger the electronic device 100 to slide and display the graphic content in the social application interface 510.

It can be understood that FIG. 9A to FIG. 9B are only exemplary illustrations. When the operation (for example, operation 6) received by the electronic device 100 is different from any of the aforementioned operations 1 to 5, no matter whether the user's gaze area is located in the third In a region, the electronic device 100 can receive and respond to the operation 6, and perform a response corresponding to the operation 6. This application does not limit the specific form of operation 6 and the response corresponding to operation 6.

In a possible implementation, the area on the touch screen of the electronic device 100 other than the first area may include a second area and a third area. Wherein, in response to the user's operation on the second area on the touch screen (for example, operation 1, operation 2, operation 3, etc.), the electronic device 100 may control the notification prompt window. In response to the user's operation on the third area (for example, operation 6), the electronic device 100 may perform the function of the first interface. For example, the second area may be the area 1002 shown in the above-mentioned FIG. 1, and the third area may be the area 1003 shown in the above-mentioned FIG. 1. As another example, the first area may be the top area of the touch screen, the second area may be the bottom area of the touch screen, the third area may be the middle area of the touch screen, and so on. This application does not limit the specific division of areas.

In a possible implementation, the electronic device 100 can also change the color of the notification prompt window, change the color of the horizontal bar under the notification message in the notification prompt window, flash the indicator light, light up the indicator light, play a prompt sound or vibrate. and other forms to output gaze feedback. In another possible implementation, the electronic device 100 may not output gaze feedback.

For example, when the electronic device 100 detects that the area the user is looking at is within the first area and the time the user looks at the area is not less than the first time threshold, the electronic device 100 can change the color of the notification prompt window 520, as shown in Figure The social application interface 510 shown in 10.

It should be noted that the embodiments shown in Figures 5A to 10 are only examples. In the embodiments of the present application, the interface for displaying the notification prompt window can also be the desktop, a negative screen interface or an interface in other application software. The notification prompt window The corresponding notification can also be a notification from other application software. This application is not limited here.

The following is a schematic flowchart of a notification viewing method provided by an embodiment of the present application.

As shown in Figure 11A, the specific process of the notification viewing method may include the following steps:

S1101. When the electronic device 100 displays the first interface, the electronic device 100 obtains the first notification of the first application and displays a notification prompt window in the first area of the touch screen.

The first interface may be an interface in application software, such as the social application interface 510 in the embodiment shown in FIG. 5B , or it may be a desktop or a one-screen interface displaying multiple application software icons, and so on.

The first notification may be a notification of the first application, such as a notification of the Changlian application software in the embodiment shown in FIG. 5C. In some embodiments, the first notification may also be a text message, a notification from a chat application software, a notification from a video application software, and so on.

The first area may be a top area on the touch screen of the electronic device 100 . In some embodiments, the first area may also be other areas on the touch screen.

The electronic device 100 may obtain the first notification by receiving the first notification sent by other electronic devices, or by obtaining the first notification pushed by the first application, and so on.

In response to the above-mentioned first notification, the electronic device 100 may display a notification prompt window on the first interface, and the notification prompt window includes part or all of the content of the first notification. For example, when the first notification is a notification from the Changlian application software, the notification prompt window may be the notification prompt window 520 in the social application interface 510 shown in FIG. 5C .

S1102. The electronic device 100 obtains gaze information. The gaze information includes any one or more of the user's facial image, the user's eye image, eye movement information, and electrooculogram information.

In some embodiments, when the electronic device 100 obtains the first notification, the electronic device 100 may obtain the user's image information through a camera. The user's image information may include the user's facial image information and/or the user's eye image information.

The electronic device 100 may respond to the first notification, turn on the front camera of the electronic device 100, and continuously obtain the user's image information through the front camera of the electronic device 100. For example, the electronic device 100 may respond to the first notification, at a fixed time Continuously obtain the user's image information at intervals.

In some embodiments, the front camera of the electronic device 100 can always remain on, that is, the electronic device 100 does not need to respond to the first notification and can continue to obtain the user's image information.

In other embodiments, the electronic device 100 can also obtain the user's eye movement information or electrooculogram information.

S1103: The electronic device 100 determines whether the user's gaze area is located in the first area based on the gaze information.

It should be noted that in the embodiment of the present application, the gaze area being located in the first area may include the following three situations: the gaze area completely coincides with the first region, the gaze area is within the first region, and the gaze area and the first region exist intersection. The relationship between the gaze area and the first area will be described below with the help of diagrams.

As shown in FIG. 11B , the first area may be area 2001 on the touch screen of the electronic device 100 , and the user's gaze area may also be area 2001 , that is, the first area completely overlaps with the user's gaze area. In this case, it can be determined that the user's gaze area is located in the first area.

As shown in FIG. 11C , the first area may be area 2001 on the touch screen of the electronic device 100 , and the user's gaze area may be area 2002 , that is, the user's gaze area may be within the first area. In this case, it can be determined that the user's gaze area is located in the first area.

As shown in FIG. 11D , the first area may be area 2001 on the touch screen of the electronic device 100 , and the user's gaze area may be area 2003 , that is, the user's gaze area intersects with the first area, and the intersection is area 2004 . In this case, it may also be determined that the user's gaze area is located in the first area.

It can be understood that the above-mentioned embodiments shown in FIGS. 11B to 11D are only examples. In some embodiments, the gaze area may also be a rounded rectangle, a circle, or other shapes, and the first area may also be a rounded rectangle, a circle, or the like. This application does not limit the specific shapes of the first area and the gaze area.

Exemplarily, after acquiring the user's image information, the specific process for the electronic device 100 to determine whether the user's gaze area is located in the first area based on the user's image information may include the following steps:

1. The electronic device 100 determines the gaze direction of the user's line of sight in a first coordinate system based on the user's image information. The first coordinate system is a coordinate system with the center point of the user's face or eyes as the origin.

2. The electronic device 100 determines the position and posture of the user's head relative to the front camera of the electronic device 100 based on the user's image information.

3. The electronic device 100 determines the gaze of the user's gaze in the second coordinate system based on the gaze direction of the user's gaze in the first coordinate system and the position and posture of the user's head relative to the front camera of the electronic device 100 direction, the second coordinate system is a coordinate system with the light point of the front camera of the electronic device 100 as the origin and the optical axis of the front camera as the Z axis.

4. The electronic device 100 determines the intersection point of the user's line of sight and the touch screen as the first position based on the gaze direction of the user's line of sight in the second coordinate system and the coordinates of the touch screen in the second coordinate system.

The coordinates of the touch screen in the second coordinate system may be data pre-stored in the electronic device 100 . In some embodiments, the electronic device 100 may also determine the coordinates of the touch screen in the second coordinate system based on the size of the touch screen of the electronic device 100 and the position of the front camera relative to the touch screen.

5. The electronic device 100 determines the user's gaze area based on the first location.

The electronic device 100 may use the first position as the center of the gaze area, and determine the gaze area as the length and width of the rectangular area with a specific length (such as the width of the touch screen, eye distance or viewing distance, etc.). It can be understood that the gaze area can also be a circular area or an area of other shapes, which is not limited in this application.

6. The electronic device 100 determines whether the user's gaze area is located in the first area.

When the positional relationship between the user's gaze area and the first area satisfies any of the conditions in the embodiments shown in FIGS. 11B to 11D , the electronic device 100 may determine that the user's gaze area is located in the first area.

It should be noted that the above method is only an example. In the embodiment of the present application, other line of sight estimation algorithms can also be used to determine the user's gaze area, which is not limited in this application.

When the gaze information obtained by the electronic device 100 is eye movement information or electrooculogram information, the electronic device 100 may also determine whether the user's gaze area is located in the first area based on the eye movement information or electrooculogram information. Among them, eye movement information is image information of eye movement or potential information generated by eye movement. When the eyeballs move, measurable potential changes are produced, called electrooculogram (eog), which can also be called electrooculogram information. By analyzing and calculating eye movement information or electrooculogram information, the eye movement trajectory can be obtained. Based on the movement trajectory of the eyeballs, the user's line of sight direction relative to the electronic device 100 can be calculated, and then the user's gaze area on the electronic device 100 can be determined.

When the electronic device 100 determines that the user is gazing at the first area, the electronic device 100 performs the following step S1104 or step S1105; when the electronic device 100 determines that the user is not gazing at the first area, the electronic device 100 performs step S1108 or step S1109.

S1104. The electronic device 100 outputs gaze feedback.

Step S1104 is an optional step.

In some embodiments, the electronic device 100 may determine whether to output gaze feedback based on a positional relationship between the user's gaze area and the first region. Gaze feedback is used to prompt the user that the gaze area is located in the first area.

Gaze feedback may be changing the display color of the first interface, such as the social application interface 510 shown in FIG. 5E. In some embodiments, the gaze feedback may also be to change the color of the notification prompt window in the first interface, such as the notification prompt window 520 in the embodiment shown in FIG. 10 . In some embodiments, the gaze feedback may also be a change in the controls in the notification prompt window, for example, changing the color of the horizontal bar below the notification message in the notification prompt window, and so on. In other embodiments, the gaze feedback can also be the brightness of the indicator light, the vibration of the electronic device 100, the playing of prompt sounds, etc., which is not limited in this application.

In other embodiments, the electronic device 100 may also determine whether to output gaze feedback based on the relationship between the duration of the user's gaze on the first area and the first time threshold.

For example, when the user looks at the first area, the electronic device 100 determines whether the time the user looks at the first area is less than a first time threshold (in the embodiment of the present application, the first time threshold can also be written as a first duration). . That is, taking the moment when the electronic device 100 determines that the user is gazing at the first area as a starting point, it is determined whether the user continues to gazing at the first area within the time period corresponding to the first time threshold. If the user continues to gaze at the first area during this time period, it is determined that the time for which the user gazes at the first area is greater than or equal to the first time threshold. If the user's gaze leaves the first area within this time period, it is determined that the time the user looks at the first area is less than the first time threshold.

When the time the user looks at the first area is less than the first time threshold, it means that the user is not interested in the first notification prompted by the notification prompt window, and the electronic device 100 executes step S1108 or step S1109.

When the time the user looks at the first area is greater than or equal to the first time threshold, it indicates that the user is interested in the first notification prompted by the notification prompt window, and the electronic device 100 executes step S1105.

In some embodiments, for example, when the electronic device 100 detects that the user's gaze area is located in the first area for the first time, when the user triggers the electronic device 100 to output gaze feedback for the first time, or when the electronic device 100 responds within a longer period of time (eg, one month). ) after gaze feedback is not output, re- When outputting gaze feedback, etc., in the above situations, the electronic device 100 may also display an operation prompt in the first interface. The operation prompt can be used to prompt the user. The operation prompt can be any one or more of text prompts, picture prompts, animation prompts, sound prompts, etc. The operation prompt is used to prompt the user to control the notification prompt window in the second area, that is, to prompt the user to move the notification prompt window and display the notification content through operations in the second area (such as operation 1, operation 2, operation 3, etc.) Or cancel the display of the notification prompt window, and other functions.

S1105. The electronic device 100 receives the user's operation 1 for the second area on the electronic device 100.

The second area is an area on the touch screen of the electronic device 100 other than the first area. The second area may include area 1002 and/or area 1003 in the embodiment shown in FIG. 1 . In some embodiments, the second area may be a middle area on the touch screen of the electronic device 100 or a bottom area on the touch screen.

In some embodiments, operation 1 may be the sliding operation in the embodiment shown in FIG. 5E. In the embodiment of this application, operation 1 can also be written as the first operation.

S1106. In response to operation 1, the electronic device 100 moves the display position of the notification prompt window downward in the first interface.

Step S1106 is an optional step.

In some embodiments, in response to operation 1, the electronic device 100 may detect the distance moved by user operation 1 in the first interface or the duration of operation 1, and based on the distance moved by user operation 1 on the touch screen or the duration of operation 1 The duration determines whether the notification prompt window is moved. For example, referring to the embodiment shown in Figure 5G, when the movement distance of user operation 1 on the touch screen is not less than the first distance (in the embodiment of this application, the first distance can also be written as the first movement distance) , the electronic device 100 can move the display position of the notification prompt window downward in the first interface. As another example, when the duration of operation 1 is not less than the first duration (for example, 1 microsecond or 1 millisecond), the electronic device 100 can move the display position of the notification prompt window downward in the first interface. It should be noted that in the embodiment of the present application, the first duration can also be written as the second duration.

In this way, by setting the first distance or the first duration, the notification prompt window can be prevented from being dragged downward due to the user's accidental touch, thereby improving the accuracy of the method and reducing unnecessary energy consumption.

S1107. In response to operation 1, the electronic device 100 displays a floating window. The floating window is used to display the interface of the first application. The interface of the first application includes the specific content of the first notification.

As shown in FIG. 5H , when the first notification is a notification from the Changlian application software, as shown in FIG. 5H , the electronic device 100 may display the Changlian application interface 541 in the floating window 530 . The specific content of the first notification is displayed in the Changlian application interface 541. For details, please refer to the relevant description in the embodiment shown in Figure 5H.

The user can view the specific content of the first notification and process the first notification in the interface displayed in the floating window.

Optionally, the electronic device 100 can also expand the notification prompt window into a floating window based on the movement distance of user operation 1 in the first interface or the duration of operation 1, or cancel the display of the notification prompt window and display the floating window. In any of the above situations, optionally, the display size of the floating window may also increase as the movement distance or duration increases. Regarding the specific content of the floating window, please refer to the relevant description in the embodiment shown in FIG. 5G.

In some embodiments, the electronic device 100 may also determine whether to display the interface of the first application in the floating window based on the distance moved by operation 1 on the touch screen or the duration of operation 1, such as Changlian shown in Figure 5H Application interface 541.

S1108: The electronic device 100 determines whether the time period in which the user has not looked at the first area is greater than or equal to the fourth time threshold.

Step S1108 is an optional step. In this embodiment of the present application, the fact that the user does not focus on the first area means that the user's focus area does not intersect with the first area, that is, the user's focus area is not located in the first area.

After the electronic device 100 determines that the user has not looked at the first area, the electronic device 100 can use the display time of the notification prompt window as a starting point to determine whether the user has never looked at the first area within the time period corresponding to the fourth time threshold.

In some embodiments, when the user briefly gazes at the first area and the electronic device 100 determines that the user has left the first area, the electronic device 100 may use the moment when the user's sight leaves the first area as a starting point to determine the fourth time threshold. Whether the user has never looked at the first area during the corresponding time period.

When the electronic device 100 determines that the user has never looked at the first area during the time period, that is, the time that the user has not looked at the first area is greater than or equal to the fourth time threshold, the electronic device 100 executes step S1110.

When the electronic device 100 determines that the time during which the user does not look at the first area is less than the fourth time threshold within the time period, the electronic device 100 executes step S1109.

S1109: The electronic device 100 determines whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.

When the user does not look at the first area, or when the time for which the user does not look at the first area is less than the fourth time threshold, the electronic device 100 may determine whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.

When the electronic device 100 determines that the display duration of the notification prompt window is greater than or equal to the sixth time threshold, the electronic device 100 may perform step S1110.

When the electronic device 100 determines that the display duration of the notification prompt window is less than the sixth time threshold, the electronic device 100 may perform step S1103.

It should be noted that in the embodiment of the present application, the sixth time threshold can also be written as the fifth duration.

S1110. The electronic device 100 cancels display of the notification prompt window.

For example, the interface after the electronic device 100 cancels display of the notification prompt window can refer to the social application interface 510 shown in FIG. 7D and FIG. 8C.

In some embodiments, after the electronic device 100 displays gaze feedback, when the electronic device 100 does not receive any operation from the user, and the electronic device 100 detects that the user's gaze leaves the first area (or detects that the user's gaze area is different from the first area). When there is no intersection in the first area), the electronic device 100 determines the length of time that the user continues not to look at the first area. When the user continues not to gaze at the first area for a length of time greater than or equal to the third time threshold, the electronic device 100 cancels outputting gaze feedback. When the length of time that the user continues not to look at the first area is greater than or equal to the fourth time threshold, the electronic device 100 cancels display of the notification prompt window. Optionally, after the electronic device 100 cancels the output of gaze feedback, the electronic device 100 may also use the moment when gaze feedback is turned off as a starting point to determine that the length of time the user has not gazed at the first area is not less than the fifth time threshold. Cancels the notification window. For specific methods and interfaces, please refer to the relevant content in the embodiment shown in FIGS. 8A to 8C.

In a possible implementation, after step S1103 or step S1104 shown in FIG. 11A , the electronic device 100 can receive and respond to the user's operation 2 (such as a single click operation, a double-click operation, a long press) on the second area. operation, etc.), open the application software corresponding to the first notification, and display the interface of the application software, such as the Changlian application interface 600 shown in Figure 6B. In some embodiments of the present application, the above operation 2 can also be written as the second operation. In other embodiments, operation 3 in the embodiment shown in FIG. 6A and FIG. 6B can also be written as the second operation.

In some embodiments, after step S1103 or step S1104 shown in FIG. 11A above, the electronic device 100 may also receive and respond to the user's operation 3 on the second area (such as a double-click operation, a long-press operation, etc.), The floating window in step S1107 shown in FIG. 11A is displayed.

In this way, the floating window can be quickly displayed through simpler operations, or the application software corresponding to the notification prompt window can be quickly opened and the interface of the application software can be displayed. The time required to display a floating window or display an application software interface is shortened.

In a possible implementation, when the user does not want to view the specific content of the first notification, the electronic device 100 can also cancel the display of the notification prompt window in response to the user's operation 4 on the second area, such as a right sliding operation. For specific operation methods and interfaces, please refer to the relevant content in the embodiment shown in FIGS. 7A to 7D .

In another possible implementation, when the user wants to view the specific content of the first notification later, the electronic device can also cancel the display of the notification prompt window in response to the user's operation 5 on the second area, such as a left sliding operation. , and then display the notification prompt window after a period of time. For specific operation methods and interfaces, please refer to the relevant content in the embodiment shown in FIGS. 7A to 7D .

It should be noted that in some embodiments of the present application, operation 4 can also be written as the third operation, and in other embodiments, operation 5 can also be written as the third operation.

In some embodiments, when the electronic device 100 outputs gaze feedback, the electronic device 100 can only respond to operations in the gaze operation set. The gaze operation set includes one or more operations defined in the gaze feedback state (for example, the above-mentioned operations 1 to Operation 5).

In other embodiments, when the electronic device 100 outputs gaze feedback, the electronic device 100 may also respond to an operation outside the gaze operation set and perform a response corresponding to the operation. For example, in the embodiment shown in FIGS. 9A to 9B , the electronic device 100 may slide down one or more graphic contents in the social application interface 510 in response to the user's upward sliding operation on the electronic device 100 , and so on.

As shown in Figure 12, the specific process of the electronic device 100 responding to operation 7 when gaze feedback is displayed may include the following steps:

S1201. When the electronic device 100 displays the first interface, the electronic device 100 obtains the first notification of the first application and displays a notification prompt window in the first area of the touch screen.

S1202. The electronic device 100 obtains gaze information, which includes any one or more of the user's facial image, the user's eye image, eye movement information, and electrooculogram information.

S1203: The electronic device 100 determines whether the user's gaze area is located in the first area based on the gaze information.

When the user's gaze area is located in the first area, the electronic device 100 executes step S1204 or step S1205.

When the user's gaze area is not located in the first area, the electronic device 100 executes step S1209 or step S1210.

Optionally, the electronic device 100 may also determine steps to be performed later based on the relationship between the duration of the user's gaze on the first area and the first time threshold.

When the time the user looks at the first area is greater than or equal to the first time threshold, the electronic device 100 executes the following step S1205.

When the time the user looks at the first area is less than the first time threshold, the electronic device 100 executes the following step S1210.

S1204. The electronic device 100 outputs gaze feedback.

Step S1204 is an optional step.

S1205: The electronic device 100 receives the user's operation 7 on the second area on the electronic device 100.

S1206: The electronic device 100 determines whether operation 7 belongs to the gaze operation set.

The set of gaze operations may include valid operations for the notification prompt window when the electronic device 100 outputs gaze feedback, such as one of operations 1, 2, 3, 4, and 5 in the above embodiment, or Multiple items. It can be understood that the set of gaze operations may be stored in the electronic device 100 in advance.

When the electronic device 100 determines that operation 7 does not belong to the gaze operation set, the electronic device 100 performs the following step S1207.

When the electronic device 100 determines that operation 7 belongs to the gaze operation set, the electronic device 100 performs the following step S1208.

S1207: The electronic device 100 determines that the action object of operation 7 is the first interface, and executes a response corresponding to operation 7.

In this case, the electronic device 100 may perform the function of the first interface (eg, the first function of the first interface) in response to operation 7 . For the specific content of operation 7 and the functions of the first interface, reference can be made to the relevant descriptions of operation 6 and the functions of the first interface in the embodiment shown in FIG. 9A to FIG. 9B , which will not be described again here.

S1208: The electronic device 100 determines that the action object of operation 7 is the notification prompt window, and executes a response corresponding to operation 7.

In this case, the electronic device 100 may respond to operation 7 by displaying the interface of the first application, displaying the floating window, or canceling the display of the notification prompt window, and so on.

S1209: The electronic device 100 determines whether the time period in which the user has not looked at the first area is greater than or equal to the fourth time threshold.

Step S1209 is an optional step. In this embodiment of the present application, the fact that the user does not focus on the first area means that the user's focus area does not intersect with the first area, that is, the user's focus area is not located in the first area.

When the user does not gaze at the first area for longer than or equal to the fourth time threshold, the electronic device 100 executes the following step S1211.

When the time period in which the user does not look at the first area is less than the fourth time threshold, the electronic device 100 performs the following step S1210.

S1210: The electronic device 100 determines whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.

When the display duration of the notification prompt window is greater than or equal to the sixth time threshold, the electronic device 100 executes the following step S1211.

When the display duration of the notification prompt window is less than the sixth time threshold, the electronic device 100 executes the above step S1203.

S1211. The electronic device 100 cancels display of the notification prompt window.

For details of steps S1201 to S1204 and steps S1209 to S1211 shown in FIG. 12 , please refer to the relevant content in the embodiment shown in FIG. 11A , and will not be described again here.

In this way, even if the electronic device 100 outputs gaze feedback, other functions performed by the electronic device 100 will not be affected. Moreover, since operation 7 does not belong to one or more operations defined in the gaze feedback state, the electronic device 100 will not cause misjudgment.

It should be noted that in some embodiments, for example, when operation 7 does not belong to the gaze operation set, operation 7 can also be written as the fourth operation. In some embodiments, the fourth operation may also include operation 6 in the embodiment shown in FIG. 9A.

The embodiment of the present application also provides a notification viewing method that can be executed by the system 10 shown in FIG. 2B. System 10 may include electronic device 100 and electronic device 200 .

As shown in Figure 13, the specific process of the system 10 executing the notification viewing method may include the following steps:

S1301. When the electronic device 100 displays the first interface, the electronic device 100 obtains the first notification of the first application and displays a notification prompt window in the first area of the touch screen.

S1302: The electronic device 100 sends request 1 to the electronic device 200, requesting the electronic device 200 to send gaze information.

Step S1302 is an optional step.

In response to the first notification, the electronic device 100 may also send a request 1 to the electronic device 200, requesting the electronic device 200 to send a request to the electronic device 100. Send gaze messages.

S1303. The electronic device 200 obtains the user's gaze information. The gaze information includes any one or more of the user's facial image, the user's eye image, eye movement information, and electrooculogram information.

The electronic device 200 may continuously obtain the gaze information in response to the request 1 sent by the electronic device 100 .

In some embodiments, the electronic device 200 may also continue to obtain the gaze information before receiving the request 1 sent by the electronic device 100 .

S1304: The electronic device 200 sends the gaze information to the electronic device 100.

The electronic device 200 may send the gaze information to the electronic device 100 after acquiring the gaze information.

It should be noted that steps S1303 and S1304 are both continuous steps, that is, during the entire notification viewing process, the electronic device 200 continues to obtain gaze information and continues to send gaze information to the electronic device 100 .

In some embodiments, the electronic device 200 may also send the latest collected gaze information obtained by the electronic device 200 to the electronic device 100 at fixed time intervals.

S1305: The electronic device 100 obtains the positional relationship between the electronic device 100 and the electronic device 200.

In some embodiments, the electronic device 100 can acquire an image of the electronic device 200 through a front-facing camera, and determine the positional relationship between the electronic device 200 and the electronic device 100 based on the image of the electronic device 200 .

In a possible implementation, the electronic device 100 may also send request 2 to the electronic device 200 , where the request 2 is used to request the electronic device 200 to send the image information of the electronic device 100 to the electronic device 100 . After receiving the image information of the electronic device 100 sent by the electronic device 200, the electronic device 100 can determine the positional relationship between the electronic device 100 and the electronic device 200 based on the image information of the electronic device 100.

In another possible implementation, the electronic device 100 can also measure the positional relationship between the electronic device 100 and the electronic device 200 based on Bluetooth, ultrasound, wifi, etc.

S1306: The electronic device 100 determines whether the user's gaze area is located in the first area based on the gaze information and the positional relationship between the electronic device 100 and the electronic device 200.

For example, taking the gaze information including the user's image information (the user's facial image and/or the user's eye image) as an example, the specific process for the electronic device 100 to determine whether the user's gaze area is located in the first area may include the following steps:

1. The electronic device 100 determines the gaze direction of the user's line of sight in a first coordinate system based on the user's image information. The first coordinate system is a coordinate system with the center point of the user's face or eyes as the origin.

2. The electronic device 100 determines the position and posture of the user's head relative to the electronic device 200 based on the user's image information.

3. The electronic device 100 determines the position and posture of the user's head relative to the electronic device 100 based on the position and posture of the user's head relative to the electronic device 200 and the positional relationship between the electronic device 100 and the electronic device 200 .

4. The electronic device 100 determines the gaze of the user's gaze in the second coordinate system based on the gaze direction of the user's gaze in the first coordinate system and the position and posture of the user's head relative to the front camera of the electronic device 100 direction, the second coordinate system is a coordinate system with the center of the touch screen of the electronic device 100 as the origin and the normal vector of the touch screen as the Z axis.

5. The electronic device 100 determines the intersection point of the user's line of sight and the touch screen as the first position based on the gaze direction of the user's line of sight in the second coordinate system and the coordinates of the touch screen in the second coordinate system.

6. The electronic device 100 determines the user's gaze area based on the first location.

7. The electronic device 100 determines whether the user's gaze area is located in the first area.

It should be noted that the above method is only an example. In the embodiment of the present application, other gaze estimation algorithms can also be used to determine whether the user's gaze area is located in the first area, or the user can be determined based on eye movement information or electrooculogram information. Whether the gaze area is located in the first area is not limited in this application.

In some embodiments, the electronic device 200 may also determine whether the user's gaze area is located in the first area, and send the determination result to the electronic device 100 . In this case, the electronic device 200 can determine the intersection point of the user's line of sight and the touch screen of the electronic device 100 based on the user's image information and the image information of the electronic device 100, and then determine whether the user's gaze area is located in the first area. Specifically, Reference may be made to the relevant content in the above embodiments.

In other embodiments, after acquiring the gaze information, the electronic device 200 may determine the user's gaze direction based on the gaze information, and send the user's gaze direction to the electronic device 100 . After receiving the user's line of sight direction, the electronic device 100 may The line direction and the positional relationship of the screen of the electronic device 100 relative to the electronic device 200 determine whether the user's gaze area is located in the first area.

Optionally, the electronic device 100 may also determine steps to be performed later based on the relationship between the duration of the user's gaze on the first area and the first time threshold.

For example, when the duration of the user's gaze on the first area is greater than or equal to the first time threshold, the electronic device 100 performs the following step S1307 or step S1308.

When the duration of the user's gaze on the first area is less than the first time threshold, the electronic device 100 executes the following step S1311 or step S1312.

S1307: The electronic device 100 outputs gaze feedback.

Step S1307 is an optional step.

S1308: The electronic device 100 receives the user's operation 1 for an area other than the first area on the electronic device 100.

S1309, in response to operation 1, the electronic device 100 moves the display position of the notification prompt window downward in the first interface.

Step S1309 is an optional step.

S1310. In response to operation 1, the electronic device 100 displays a floating window. The floating window is used to display the interface of the first application. The interface of the first application includes the specific content of the first notification.

S1311. The electronic device 100 determines whether the time period in which the user has not looked at the first area is greater than or equal to the fourth time threshold.

Step S1311 is an optional step. In this embodiment of the present application, the fact that the user does not focus on the first area means that the user's focus area does not intersect with the first area, that is, the user's focus area is not located in the first area.

When the user does not gaze at the first area for longer than or equal to the fourth time threshold, the electronic device 100 executes the following step S1313.

When the time period in which the user does not look at the first area is less than the fourth time threshold, the electronic device 100 executes the following step S1312.

S1312: The electronic device 100 determines whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.

When the display duration of the notification prompt window is greater than or equal to the sixth time threshold, the electronic device 100 executes the following step S1313.

When the display duration of the notification prompt window is less than the sixth time threshold, the electronic device 100 executes the above step S1306.

S1313. The electronic device 100 cancels display of the notification prompt window.

For details of step S1301, step S1307 to step S1313 shown in Figure 13, reference can be made to the relevant content in the above embodiment shown in Figure 11A, and will not be described again here.

It can be understood that the embodiments shown in FIG. 11A, FIG. 12 and FIG. 13 only illustrate the notification viewing method by taking the electronic device 100 receiving the first notification of the first application as an example. In the embodiment of the present application, the electronic device 100 receives the first notification of the first application as an example. The device 100 may also receive other notifications from the first application (for example, the second notification) or notifications from other applications (for example, the third notification from the second application), and use any one of the above notification viewing methods to view the notification, or Cancel the notification prompt window, etc. This application is not limited here.

In addition, it should be noted that in the embodiment of the present application, the notification prompt window may be a first notification prompt window, a second notification prompt window, a third notification prompt window, etc., and the floating window may refer to the first floating window. It can also be a second floating window or a third floating window, and so on. The electronic device 100 may also be referred to as a first electronic device, and the electronic device 200 may also be referred to as a second electronic device.

The following introduces the functional module diagram of an electronic device 100 provided in the embodiment of the present application.

As shown in FIG. 14 , the electronic device 100 may include a notification prompt module 1401, a display module 1402, a gaze information acquisition module 1403, a line of sight estimation module 1404, and a gesture recognition module 1405.

Among them, the notification prompt module 1401 can receive the first notification and send a notification reminder to the display module 1402 and the gaze information acquisition module 1403.

The display module 1402 may display the notification prompt window in response to the notification reminder sent by the notification prompt module 1401. The display module 1402 can also display or close gaze feedback based on the gaze estimation result sent by the gaze estimation module 1404, or close the notification prompt window, and so on. For example, when the gaze estimation result received by the display module 1402 is that the time the user gazes at the first area is not less than the first time threshold, the display module 1402 can output gaze feedback and change the background color of the interface or the color of the notification prompt window. For another example, when the gaze estimation result received by the display module 1402 is that the time the user has not looked at the first area is not less than the fourth time threshold, the display module 1402 can cancel the display of the notification prompt window, and so on. The display module 1402 may also execute a corresponding response based on the operation recognition result (eg, operation 1, operation 2, etc.) sent by the gesture recognition module 1405.

The gaze information acquisition module 1403 can acquire gaze information, such as the user's image information, eye movement information, or electrooculogram information. The gaze information acquisition module 1403 may respond to the notification reminder sent by the notification reminder module 1401 and continue to obtain gaze information. The gaze information acquisition module 1403 may also send gaze information to the gaze estimation module 1404.

In some embodiments, the gaze information acquisition module 1403 may also acquire image information of the electronic device 100 or image information of the electronic device 200 .

The gaze estimation module 1404 may determine whether the user's gaze area is located in the first area based on the gaze information. In some embodiments, the gaze estimation module 1404 may also be used to determine whether the time the user looks at the first area is not less than a first time threshold. In other embodiments, the gaze estimation module 1404 can also be used to determine whether the time when the user's gaze leaves the first area is not less than a third time threshold, and can also be used to determine whether the time when the user does not look at the first area is not less than a fourth time threshold. time threshold, etc. The gaze estimation module 1404 may send the above gaze estimation results (for example, the user gazes at the first area, the user's gaze area is not located in the first area, the user gazes at the first area for longer than the first time threshold, etc.) to the display module 1402 .

The gesture recognition module 1405 can receive and recognize the user's operation. For example, the gesture recognition module 1405 can receive and recognize the user's operation 1, and send the operation recognition result to the display module 1402, instructing the display module 1402 to perform a corresponding response.

It should be noted that the functional module diagram of the system 10 shown in FIG. 2B may also refer to the functional module diagram of the electronic device 100 shown in FIG. 14 . In the system 10 , the above-mentioned gaze information acquisition module 1403 in FIG. 14 may be located in the electronic device 200 , and the remaining functional modules are located in the electronic device 100 . In some embodiments, gaze estimation module 1404 may also be located in electronic device 200.

It can be understood that the functional module structure of the electronic device 100 shown in FIG. 14 is just an example. The electronic device 100 may also include more or fewer functional modules than shown in FIG. 14 , which is not limited in this application. .

The various embodiments of the present application can be combined arbitrarily to achieve different technical effects.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in this application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk (SSD)), etc.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments are implemented. This process can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, , may include the processes of the above method embodiments. The aforementioned storage media include: ROM, random access memory (RAM), magnetic disks, optical disks and other media that can store program codes.

In short, the above descriptions are only examples of the technical solutions of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made based on the disclosure of the present invention shall be included in the protection scope of the present invention.

Claims (20)

1. A notification viewing method is characterized by including:

   The first electronic device displays the first interface;

   The first electronic device obtains a first notification of the first application;

   In response to the first notification, the first electronic device displays a first notification prompt window in the first area of the touch screen, and the first notification prompt window includes part or all of the content of the first notification;

   After the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's first operation on a second area, and the second area is located on the touch screen. Areas other than the first area mentioned above;

   After the first electronic device detects the first operation, the first electronic device overlays and displays a first floating window on the first interface, and the first floating window includes the first application. An interface, the interface of the first application includes the content of the first notification.
2. The method of claim 1, wherein after the first electronic device determines that the user's gaze area is located in the first area, the method further includes:

   The first electronic device outputs gaze feedback, and the gaze feedback is used to prompt the user that the gaze area is located in the first area.
3. The method of claim 2, wherein the outputting gaze feedback includes any one or more of the following: changing the display color of the first interface, changing the display color of the first notification prompt window, changing The color of the control in the first notification prompt window, lighting of the indicator light of the first electronic device, vibration and playing of prompt sounds.
4. The method according to claim 2 or 3, characterized in that the first electronic device outputs gaze feedback, specifically including:

   When the first electronic device detects that the user gazes at the first area for longer than a first duration, the first electronic device outputs the gaze feedback.
5. The method according to any one of claims 1-4, characterized in that the first operation is a sliding operation;

   After the first electronic device receives the user's first operation on the second area, the method further includes:

   When the movement distance of the first operation on the touch screen is greater than the first movement distance, the first electronic device moves the display position of the first notification prompt window;

   When the movement distance of the first operation on the touch screen is greater than the second movement distance, the first electronic device expands the first notification prompt window into the first notification prompt window. The second moving distance is greater than the first moving distance.
6. The method according to any one of claims 1-4, characterized in that the first operation is a sliding operation;

   After the first electronic device receives the user's first operation on the second area, the method further includes:

   When the duration of the first operation is greater than the second duration, the first electronic device moves the display position of the first notification prompt window;

   When the duration of the first operation is greater than a third duration, the first electronic device expands the first notification prompt window into the first notification prompt window, and the third duration is greater than the second duration. .
7. The method according to any one of claims 1-6, characterized in that the method further includes:

   After the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives a second operation from the user for the second area;

   In response to the second operation, the first electronic device displays an interface of the first application.
8. The method according to any one of claims 1-6, characterized in that the method further includes:

   After the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's third operation for the second area;

   In response to the third operation, the first electronic device cancels display of the first notification prompt window.
9. The method according to claim 8, characterized in that, after the first electronic device cancels display of the first notification prompt window, the method further includes:

   The first electronic device redisplays the first notification prompt window after a fourth period of time;

   or,

   The first electronic device displays a floating ball;

   The first electronic device receives a user's input regarding the floating ball;

   In response to the user's input on the floating ball, the first electronic device displays the first notification prompt window or the first floating window.
10. The method according to any one of claims 1 to 9, characterized in that, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes:

    When the first electronic device receives the user's fourth operation on the second area, the first electronic device performs the first function corresponding to the first interface, and the fourth operation is different from the first operation.
11. The method according to any one of claims 1 to 10, characterized in that, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes:

    When the first electronic device determines that the user's gaze area changes to the second area and the display duration of the first notification prompt window is longer than the fifth duration, the first electronic device cancels display of the first notification. Prompt window.
12. The method according to any one of claims 1-11, characterized in that, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes:

    The first electronic device displays an operation prompt, and the operation prompt is used to prompt the user to control the first notification prompt window in the second area.
13. The method according to any one of claims 1-12, wherein the first area is a top area of the touch screen.
14. The method of claim 13, wherein the second area is a bottom area of the touch screen.
15. The method according to claims 1-14, characterized in that the method further includes:

    The first electronic device obtains gaze information, and the gaze information includes any one or more of the following: the user's eye image, the user's facial image, eye movement information, and electrooculogram information;

    The first electronic device determines that the user's gaze area is located in the first area based on the gaze information.
16. The method according to claim 15, wherein the first electronic device obtains gaze information, specifically including:

    The first electronic device receives the gaze information sent by the second electronic device.
17. The method according to claims 1-14, characterized in that the method further includes:

    The first electronic device receives a line of sight estimation result sent by the second electronic device, where the line of sight estimation result includes that the user's gaze area is located in the first area;

    The first electronic device determines that the user's gaze area is located in the first area based on the line of sight estimation result.
18. The method according to claims 1-14, characterized in that the method further includes:

    The first electronic device receives the user's line of sight direction sent by the second electronic device;

    The first electronic device determines that the user's gaze area is located in the first area based on the user's line of sight direction.
19. An electronic device, which is a first electronic device, is characterized in that it includes: one or more processors, and one or more memories; the one or more memories are coupled to the one or more processors, and the One or more memories are used to store computer program codes. The computer program codes include computer instructions. When the one or more processors execute the computer instructions, the terminal performs any of the above claims 1-18. The method described in one item.
20. A computer-readable storage medium including computer instructions, characterized in that when the computer instructions are run on a first electronic device, the first electronic device is caused to execute any one of the above claims 1-18. Methods.