WO2023185636A1

WO2023185636A1 - Image display method, and electronic devices

Info

Publication number: WO2023185636A1
Application number: PCT/CN2023/083430
Authority: WO
Inventors: 杨光宇; 彭权; 徐俊; 张强; 任丛雅旭
Original assignee: 华为技术有限公司
Priority date: 2022-03-30
Filing date: 2023-03-23
Publication date: 2023-10-05
Also published as: CN116931853A

Abstract

Provided in the present application are an image display method, and electronic devices. In the method, a first electronic device may shorten an image drawing period, so as to increase an image drawing amount, and on the basis of screen refresh rates of the first electronic device and a second electronic device, when a drawn image is transmitted to the above-mentioned two electronic devices for display, the first electronic device may make a transmission and display period of the image shorter when the image is transmitted to the second electronic device for display, such that the first electronic device may provide to the second electronic device a code stream which has a higher screen refresh rate than the first electronic device.

Description

Image display method and electronic device

This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on March 30, 2022, with application number 202210325714.4 and application name "Image Display Method and Electronic Device", the entire content of which is incorporated into this application by reference. middle.

Technical field

The embodiments of the present application relate to the technical field of terminal equipment, and in particular, to an image display method and electronic equipment.

Background technique

With the development of terminal technology, the refresh rate of display screens of electronic devices is getting higher and higher. The higher the refresh rate of the display screen of the electronic device, the smoother the screen display. As electronic devices have more and more functions, electronic devices can be used in various multi-screen simultaneous display scenarios, such as screen projection, video calls, video conferencing, live broadcasts, etc.

Currently, due to hardware limitations of the display screen of the electronic device, the refresh rates of the display screens of different electronic devices may be different. In a multi-screen simultaneous display scenario, when the local terminal shares a code stream to the peer terminal, the refresh rate of the code stream is limited by the hardware refresh rate of the local terminal's display.

Contents of the invention

In order to solve the above technical problems, this application provides an image display method and electronic device. In this method, the image drawing amount of the first electronic device can be increased, and based on the different refresh rates of the first electronic device and the second electronic device, the drawn image can be sent to the two electronic devices in different display cycles. device, thereby providing the second electronic device with a code stream higher than the screen refresh rate of the first electronic device.

In a first aspect, embodiments of the present application provide an image display method, applied to a first electronic device, where the first electronic device is communicatively connected to a second electronic device. The method includes: in response to the received image display request, the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate; wherein the first refresh rate The refresh rate of the first display screen of the first electronic device, the second refresh rate of the second display screen of the second electronic device, the first refresh rate is less than the second refresh rate ; The first electronic device draws multiple frames of first images according to the first cycle; the first electronic device displays the corresponding part of the first image to the first display screen according to the second cycle; The first electronic device sends the corresponding part or all of the first image to the second electronic device according to a third period; wherein the second period is greater than the third period.

Exemplarily, the communication connection method between the first electronic device and the second electronic device may include but is not limited to at least one of the following: Bluetooth, Wi-Fi, server, etc.

Exemplarily, both the first display screen and the second display screen support multiple refresh rate gears. The first refresh rate is a refresh rate corresponding to the refresh rate gear of the first display screen, and the second refresh rate is a refresh rate corresponding to the current refresh rate gear of the first display screen. The refresh rate corresponding to the refresh rate gear of the second display screen.

For example, the multiple refresh rates (FPS) supported by the first display screen are 20, 40, and 60 respectively. Among them, the first display screen The maximum FPS is 60. The multiple refresh rates supported by the second display are 60, 90, and 120 respectively. Among them, the maximum FPS of the second display is 120.

In some embodiments, both the first refresh rate and the second refresh rate are adjustable, and can be adjusted at the refresh rate level supported by the corresponding display screen.

For example, the traditional image drawing cycle is limited to the first refresh rate of the first display screen. The traditional image drawing period may be the signal period T0 of the vertical synchronization signal (VSync signal) sampled by the first electronic device from the hardware. When the first refresh rate is changed, the traditional image drawing cycle can be changed, thereby changing the number of image drawing frames of the first electronic device in unit time (for example, 1 s). In this embodiment, the image drawing cycle can be shortened to a first period to increase the number of image drawing frames of the electronic device per unit time. The first period may not be limited to the first refresh rate of the first display screen. .

For example, when drawing multiple frames of first images according to the first cycle, the first electronic device can render the images according to the first cycle, and synthesize the rendered images according to the first cycle, so as to follow the first cycle. In the first cycle, multiple first frames of images that can be displayed on the screen are obtained. Wherein, the first electronic device can obtain a frame of the first image to be displayed on the screen within a time period corresponding to a first period.

In some embodiments, after receiving the image display request, the first electronic device may, in response to the image display request, display the first display screen and the second display screen of the second electronic device according to the shortened first period. Provide differentiated delivery. For example, the first image is sent and displayed according to the above-mentioned second cycle and third cycle respectively.

For example, the image display request may be a multi-screen simultaneous display request. Then as long as the first electronic device receives a multi-screen simultaneous display operation with the second electronic device, the mobile phone responds to the image display request according to the above solution.

Taking the first electronic device as a mobile phone and the second electronic device as a smart TV as an example, the user triggers a screen projection operation on the mobile phone (an example of an image display request), and the mobile phone can respond to the screen projection operation and automatically follow the above method to The first image to be displayed on the screen is displayed on the display screens of the mobile phone and the TV in different cycles. In a multi-screen simultaneous display scenario, the first electronic device automatically sends images with differentiated display cycles according to the respective screen refresh rates of the first electronic device and the second electronic device connected through communication, so that the first electronic device can The refresh rate of the image sent and displayed by the second electronic device may be higher than the first refresh rate of the first electronic device.

For example, the image display request may also be an adaptive refresh request. For example, the system setting interface of the first electronic device has a preset switch, or the first electronic device has a preset virtual button, or the first electronic device has a switching icon, etc. By operating the above-mentioned preset switch, or the preset virtual button, or the switching icon, the user can cause the first electronic device to receive an adaptive refresh request, thereby drawing the first electronic device to be displayed according to the above method. The image is transmitted and displayed with differentiated display cycles according to the respective screen refresh rates of the first electronic device and the second electronic device connected through communication, so that the refresh rate of the image transmitted by the first electronic device to the second electronic device is Can be higher than the first refresh rate of the first electronic device.

In a possible implementation, when the first electronic device performs the method of the first aspect, the application may not be distinguished. As long as it is an image to be displayed by the first electronic device, the method is performed according to the above method.

According to a first aspect, the first electronic device displays the corresponding part of the first image to the first display screen according to a second cycle, including: the first electronic device based on the first refresh rate and the The first period is determined to determine the second period; the first electronic device displays the corresponding part of the first image to the second period according to the second period. A first display screen, wherein the first image is refreshed and displayed on the first display screen at the first refresh rate. The first electronic device may combine the screen refresh rate of the first electronic device and the modified image drawing cycle of the first electronic device to determine the display cycle (second cycle) of sending the image to the first electronic device, so as to be able to The second cycle displays the first image of the drawing portion of the first electronic device to the first display screen. Then, on the first electronic device, the image can be refreshed and displayed according to the screen refresh rate of the first electronic device, and the refresh rate of the image is consistent with the screen refresh rate of the first electronic device.

According to the first aspect, or any implementation of the first aspect above, the first electronic device includes a first display screen object; the first electronic device displays the corresponding part of the first image according to a second cycle. To the first display screen, the method includes: the first electronic device sends a first vertical synchronization signal to the first display screen object according to the second period, so that the first display screen object follows the second period. The second cycle acquires part of the first image; wherein the first display screen object is used to manage the input data and output data of the first display screen; the first electronic device uses the first display screen object , display the corresponding part of the first image to the first display screen according to the second cycle. The first electronic device can send the drawn part of the first image to the first display screen object according to the second cycle, so that on the first display screen corresponding to the first display screen object, the first image can be displayed according to the first refresh rate. show. The refresh rate of the first image displayed by the first electronic device is consistent with the screen refresh rate of the first electronic device.

According to the first aspect, or any implementation of the above first aspect, the first electronic device sends the corresponding part or all of the first image to the second electronic device for display according to a third cycle, including : The first electronic device determines a third period based on the second refresh rate and the first period; the first electronic device converts the corresponding part or all of the first image according to the third period. Sent to the second electronic device for display, wherein the first image is displayed on the second display screen at the second refresh rate. The first electronic device may combine the screen refresh rate of the second electronic device and the modified image drawing cycle of the first electronic device to determine the display cycle (third cycle) for sending the image to the second electronic device, so that it can be based on In the third cycle, the first electronic device draws part or all of the first image and sends it to the second electronic device. Then on the second electronic device, the image drawn by the first electronic device can be refreshed and displayed according to the screen refresh rate of the second electronic device. The refresh rate of the image is consistent with the screen refresh rate of the second electronic device and is not affected by the first electronic device. Screen refresh rate limits for electronic devices.

According to the first aspect, or any implementation of the first aspect above, the first electronic device includes a second display screen object; the first electronic device converts the corresponding part or all of the first display screen object according to the third cycle. Sending the image to the second electronic device for display includes: the first electronic device sends a second vertical synchronization signal to the second display screen object according to the third cycle, so that the second display screen The object acquires part or all of the first image according to the third cycle; wherein the second display screen object is used to manage the input data and output data of the second display screen; the first electronic device passes the The second display screen object sends the corresponding part or all of the first image to the second electronic device according to the third cycle to display on the second display screen. The first electronic device may send part or all of the first image drawn by the first electronic device to the second display screen object according to the third cycle, so that on the second display screen corresponding to the second display screen object, the first image Can be displayed at a second refresh rate. The refresh rate of the first image displayed by the second electronic device is consistent with the screen refresh rate of the second electronic device.

According to the first aspect, or any implementation of the above first aspect, the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate, including: the first When the electronic device acquires the image drawing cycle sampled from the first display screen, the first electronic device modifies the image drawing cycle to a first refresh rate based on the first refresh rate and the second refresh rate. period, wherein the image drawing period before modification is the reciprocal of the first refresh rate. The first electronic device may shorten the image drawing cycle based on the relationship between the first refresh rate and the second refresh rate every time the image drawing cycle is sampled from the first display screen, so that the updated image drawing cycle The amount can satisfy the second refresh rate of the second electronic device when displaying images. The first electronic device modifies the image drawing cycle every time the image drawing cycle is sampled from the first display screen of the hardware, and can dynamically implement calibration of the image drawing cycle based on the hardware signal.

According to the first aspect, or any implementation of the above first aspect, the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate, including: the first The electronic device determines a third refresh rate based on the least common multiple of the first refresh rate and the second refresh rate; the first electronic device shortens the image drawing cycle to the A first period, wherein the first period is the reciprocal of the third refresh rate. In this embodiment, corresponding to the acquired multiple frames of first images, the time intervals between different frame images displayed by the first electronic device and the second electronic device are uniform, which can make the first electronic device and the second electronic device The picture display quality of electronic equipment is optimal and the best display performance can be achieved.

According to the first aspect, or any implementation of the above first aspect, the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate, including: the first The electronic device determines a first period based on the maximum refresh rate among the first refresh rate and the second refresh rate; the first electronic device shortens the image drawing period to the first period; wherein, The first period is the reciprocal of the second refresh rate. In this embodiment, corresponding to the acquired multi-frame first image, the second electronic device with a screen refresh rate of the maximum refresh rate can be used to display the same time interval between frames in the image, so that Improve screen display quality.

According to the first aspect, or any implementation of the above first aspect, the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate, including: the first The electronic device determines a fourth refresh rate based on the first refresh rate and the second refresh rate, wherein the fourth refresh rate is greater than the maximum refresh rate among the first refresh rate and the second refresh rate; An electronic device shortens the image drawing cycle to the first cycle based on the fourth refresh rate, wherein the first cycle is the reciprocal of the fourth refresh rate. In this embodiment, corresponding to the acquired multiple frames of the first image, the requirement for the image rendering capability of the first electronic device can be reduced to a certain extent, so as to reduce the load of the first electronic device.

According to the first aspect, or any implementation of the above first aspect, the first image includes a second image, wherein the ratio of the number of the second image to the number of the first image is the The ratio of the first refresh rate to the second refresh rate; the first electronic device displays the corresponding part of the first image according to the second cycle To the first display screen, the method includes: the first electronic device displays the second image to the first display screen according to a second cycle; and the first electronic device displays the corresponding part or Sending all the first images to the second electronic device includes: the first electronic device sending the multiple frames of first images to the second electronic device according to a third cycle.

For example, the FPS of the first display screen is 60 and the FPS of the second display screen is 120. In an exemplary period of time, the first electronic device can draw 120 frames of the first image, where all 120 frames of the first image are drawn. The images of 60 frames out of 120 frames (the second images here) are sent to the first display screen for display. In this way, differentiated display of the first electronic device and the second electronic device is achieved.

According to the first aspect, or any implementation of the above first aspect, the first electronic device responds to the received image display request and shortens the image drawing cycle to a third refresh rate based on the first refresh rate and the second refresh rate. One cycle includes: the first electronic device receives an image display request for the first application; the first electronic device responds to the image display request, based on the first refresh rate and the second refresh rate, The image drawing cycle of the first application is shortened to the first cycle, wherein the first image is an image to be displayed by the first application on the first display screen.

For example, the first application may be an application with a screen casting function, an application with a screen sharing function, a music playing application, a video playing application, a photo album application, etc. This application does not limit this.

In a possible implementation, the image display request may not be an operation triggered by an application, but may be an operation triggered by the system of the first electronic device. For example, the image display request may be a quick access to the mobile phone system (such as screen casting). Quick entry) operation. For example, the shortcut entry may include, but is not limited to, the preset virtual keys, preset switches, switching icons, etc. exemplified in the first aspect.

In this embodiment, the first electronic device can shorten the image drawing cycle of the first application, and send the image to be displayed of the first application to the first electronic device and the second electronic device for display according to different display cycles. This enables application display content to be displayed at different refresh rates on different electronic devices.

According to the first aspect, or any implementation of the above first aspect, the first electronic device includes a first camera, and the first image is an image collected by the first camera.

For example, the first electronic device can display the images captured by the first camera (the former camera or the rear camera) in real time on the first electronic device and the second device at different refresh rates in scenes such as video calls, video conferences, and live broadcasts. Electronic equipment.

According to the first aspect, or any implementation of the above first aspect, the first application includes any one of the following applications: a video call application, a video conferencing application, and a live broadcast application.

According to the first aspect, or any implementation of the above first aspect, the number of the second electronic devices is one or more.

For example, when there are multiple second electronic devices, in any of the above embodiments, when determining relevant information in combination with the second refresh rate, the second refresh rates of each of the multiple second electronic devices may be combined. , to determine the relevant The information and methods are the same and will not be repeated here.

In a possible implementation, the number of first electronic devices may be one or more.

In a second aspect, embodiments of the present application provide an electronic device. The electronic device includes: a memory and a processor, the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the first aspect and a method in any implementation of the first aspect.

For technical effects corresponding to the second aspect, please refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be described again here.

In a third aspect, embodiments of the present application provide an image display system. The system includes: the electronic device described in the second aspect.

For technical effects corresponding to the third aspect, please refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be described again here.

In a fourth aspect, embodiments of the present application provide a computer-readable medium for storing a computer program. When the computer program is run on an electronic device, the electronic device causes the electronic device to execute the first aspect and the first aspect. method in any embodiment.

For technical effects corresponding to the fourth aspect, please refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be described again here.

In a fifth aspect, embodiments of the present application provide a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from the memory of the electronic device and provide them to the processor. The processor sends the signal, and the signal includes computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device is caused to execute the first aspect and any one of the implementation manners of the first aspect. Methods.

For technical effects corresponding to the fifth aspect, please refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be described again here.

In a sixth aspect, embodiments of the present application provide a computer program product containing instructions. When the computer program product is run on a computer, the computer is caused to execute the first aspect and any one of the embodiments of the first aspect. method in.

For technical effects corresponding to the sixth aspect, please refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be described again here.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. , for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative labor.

Figure 1 is one of the structural schematic diagrams of an exemplary electronic device;

Figure 2 is a schematic diagram of the software structure of an exemplary electronic device;

Figure 3a is a schematic diagram of a process of displaying a frame of image on an exemplary electronic device;

Figure 3b is a schematic diagram of an application scenario of the electronic device;

Figure 3c is a schematic diagram illustrating the interaction of two electronic devices;

Figure 4 is a schematic diagram of a screen projection scenario of an electronic device in traditional technology;

Figure 5a is a schematic diagram of an exemplary image display process of an electronic device;

Figure 5b is a schematic diagram of an application scenario of the electronic device;

Figure 5c is a schematic diagram of an application scenario of the electronic device;

Figure 5d is a schematic diagram of an application scenario of the electronic device;

Figure 5e is a schematic diagram of an exemplary frame drawing and display process of an electronic device;

Figure 5f is a schematic diagram of an exemplary frame drawing and display process of an electronic device;

Figure 6a is a schematic diagram of an exemplary frame drawing and display process of an electronic device;

Figure 6b is a schematic diagram of an exemplary frame drawing and display process of an electronic device;

Figure 6c is a schematic diagram of an exemplary frame drawing and display process of an electronic device;

Figure 6d is a schematic diagram of an exemplary frame drawing and display process of an electronic device;

Figure 7a is a schematic diagram of an exemplary image display process of an electronic device;

Figure 7b is a schematic diagram of an exemplary image display process of an electronic device;

Figure 7c is a schematic diagram of an exemplary image display process of an electronic device;

Figure 8 is a schematic diagram of an application scenario of an exemplary electronic device;

Figure 9 is a schematic diagram of an application scenario of an exemplary electronic device;

Figure 10 is a schematic diagram of an application scenario of an exemplary electronic device;

Figure 11 is a schematic diagram of an application scenario of an exemplary electronic device;

Figure 12 is a schematic structural diagram of a device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

The term "and/or" in this article is just an association relationship that describes related objects, indicating that three relationships can exist. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and they exist alone. B these three situations.

The terms “first” and “second” in the description and claims of the embodiments of this application are used to distinguish different objects, rather than to describe a specific order of objects. For example, the first target object, the second target object, etc. are used to distinguish different target objects, rather than to describe a specific order of the target objects.

In the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "such as" in the embodiments of this application should not be interpreted as Interpreted as more preferred or advantageous than other embodiments or designs. Rather, use of the words "exemplary" or "such as" is intended to present the concept in a concrete manner.

In the description of the embodiments of this application, unless otherwise specified, the meaning of “plurality” refers to two or more. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

FIG. 1 shows a schematic structural diagram of an electronic device 100 . It should be understood that the electronic device 100 shown in FIG. 1 is only an example of an electronic device. Alternatively, the electronic device 100 may be a terminal, which may also be called a terminal device. The terminal may be a cellular phone or a tablet computer. Devices with media data playback functions such as (pads), wearable devices or Internet of Things devices are not limited by this application. It should be noted that the electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in Figure 1 may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.

The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2. Mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, And subscriber identification module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) wait. Among them, different processing units can be independent devices or integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100 . The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 110 . If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. Interfaces may include integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, pulse code modulation (PCM) interface, universal asynchronous receiver and transmitter (universal asynchronous receiver/transmitter (UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, user Identity module (subscriber identity module, SIM) interface, and/or universal serial bus (universal serial bus, USB) interface, etc.

The USB interface 130 is an interface that complies with the USB standard specification, and may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through them. This interface can also be used to connect other electronic devices, such as AR devices, etc.

It can be understood that the interface connection relationships between the modules illustrated in the embodiments of the present application are only schematic illustrations and do not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

The charging management module 140 is used to receive charging input from the charger. Among them, the charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through the wireless charging coil of the electronic device 100 . While the charging management module 140 charges the battery 142, it can also provide power to the electronic device through the power management module 141.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, wireless communication module 160, etc. The power management module 141 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters. In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: Antenna 1 can be reused as a diversity antenna for a wireless LAN. In other embodiments, antennas may be used in conjunction with tuning switches.

The mobile communication module 150 can provide solutions for wireless communication including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be disposed in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.

A modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor communicates via the audio device (not limited to speaker 170A, receiver 170B etc.) output sound signals, or display images or videos through the display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110 and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (bluetooth, BT), and global navigation satellites. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.

The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is an image processing microprocessor and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used to display images, videos, etc. Display 194 includes a display panel. The display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light). emitting diode (AMOLED), flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. In other embodiments, the electronic device 100 may also be a projector. The electronic device may or may not include a display screen 194 based on the display characteristics of the projector.

The electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

Camera 193 is used to capture still images or video. The object passes through the lens to produce an optical image that is projected onto the photosensitive element.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. Such as saving music, videos, etc. files in external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes instructions stored in the internal memory 121 to execute various functional applications and data processing of the electronic device 100 . The internal memory 121 may include a program storage area and a data storage area. Among them, the stored program area can store an operating system, at least one application program required for a function (such as a sound playback function, an image playback function, etc.). The storage data area may store data created during use of the electronic device 100 (such as audio data, phone book, etc.). In some embodiments, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash memory (universal flash storage, UFS), etc.

The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .

The buttons 190 include a power button, a volume button, etc. Key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .

The motor 191 can generate vibration prompts. The motor 191 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback. For example, touch operations for different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. The motor 191 can also respond to different vibration feedback effects for touch operations in different areas of the display screen 194 . Different application scenarios (such as time reminders, receiving information, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

The indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to indicate messages, missed calls, notifications, etc.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of this application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100 .

FIG. 2 is a software structure block diagram of the electronic device 100 according to the embodiment of the present application.

The layered architecture of the electronic device 100 divides the software into several layers, and each layer has clear roles and division of labor. The layers communicate through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime and system libraries, and kernel layer.

The application layer can include a series of application packages.

As shown in Figure 2, the application package can include camera, gallery, calendar, calling, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 2, the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, logic manager, etc.

A window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc.

The logic manager may include at least one display screen object, and manage the input and output of the corresponding display screen (not limited to the display screen of this electronic device) through each display screen object.

As shown in FIGS. 1 and 2 , for the display screen 194 in the working state in the electronic device 100 of the present application (named after the first electronic device), the electronic device 100 can create a corresponding display screen 194 in the logic manager. display object. By way of example, the display screen 194 on which the electronic device 100 operates may include at least one display screen.

For example, the first electronic device may have a single screen (including a home screen), then the logical manager may include a display screen object marked as the home screen.

For example, the first electronic device may have a double-sided screen (including a main screen and a back screen), then the logical manager may include a display screen object marked as the main screen and a display screen object marked as the back screen.

For example, the first electronic device may have a folding screen (including a first screen and a second screen), then the logical manager may include a display screen object marked as the first screen and a display screen marked as the second screen. object.

In some embodiments, the first electronic device of the present application can be connected to at least one second electronic device (where the second electronic device includes a display screen) through HDMI (High Definition Multimedia Interface), and the first If the electronic device and the second electronic device are in a multi-screen simultaneous display scenario, the logic manager may further include at least one display screen object marked as a wired screen.

Exemplarily, the electronic device 1 of the present application is connected to the electronic device 2 and the electronic device 3 through HDMI. The logic manager may further include a display screen object labeled wired screen 1 (corresponding to the display screen of the electronic device 2) and a display object labeled wired screen 1. The display screen object of the wired screen 2 (corresponding to the display screen of the electronic device 3).

In some embodiments, the first electronic device of the present application can communicate with at least one third electronic device in a wireless manner, and the first electronic device and the third electronic device are in a multi-screen simultaneous display scenario. The logical manager may then further include at least one display screen object labeled virtual screen.

For example, the electronic device 1 of the present application communicates with the electronic device 4 through Wi-Fi and uses a screen projection protocol. Then the logical manager may further include a display object labeled virtual screen 1 (corresponding to the display of the electronic device 4 Screen).

In some embodiments, when the first electronic device is in a screen recording scene, the logic manager may further include a display screen object marked as a virtual screen, and the display screen object marked as a virtual screen is the first screen object for recording. A display simulated by the screen of an electronic device (such as a home screen).

Content providers are used to store and retrieve data and make this data accessible to applications. Said data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

The view system includes visual controls, such as controls that display text, controls that display pictures, etc. A view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide communication functions of the electronic device 100 . For example, call status management (including connected, hung up, etc.).

The resource manager provides various resources to applications, such as localized strings, icons, pictures, layout files, video files, etc.

The notification manager allows applications to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be notifications that appear in the status bar at the top of the system in the form of charts or scroll bar text, such as notifications for applications running in the background, or notifications that appear on the screen in the form of conversation windows. For example, text information is prompted in the status bar, a beep sounds, the electronic device vibrates, the indicator light flashes, etc.

The system library and runtime layer include system libraries and Android Runtime. System libraries can include multiple functional modules. For example: surface manager (surface manager), media libraries (Media Libraries), 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), Android layer synthesis module (Android SurfaceFlinger), etc. The 3D graphics library is used to implement three-dimensional graphics drawing, image rendering, composition and layer processing, etc. The Android runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core library contains two parts: one is the functional functions that need to be called by the Java language, and the other is the core library of Android. The application layer and application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and application framework layer into binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection and other functions.

The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, composition, and layer processing.

2D Graphics Engine is a drawing engine for 2D drawing.

The layer synthesis module (SurfaceFlinger) is used to receive the signal period of the VSync signal sampled by the display screen 194 in Figure 1 .

In some embodiments, the layer synthesis module is also used to modify the signal period, perform layer synthesis according to the modified signal period, and differentiate the synthesized images to be displayed on the screen to different display screen objects. The specific processing process will be described in detail in the following embodiments.

For example, the layer composition module may be an Android layer composition module.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

It can be understood that the components included in the system framework layer, system library and runtime layer shown in Figure 2 do not constitute specific limitations on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently.

Currently, as users have higher and higher requirements for the smoothness of the display screen of electronic devices, the refresh rate of the display screen of electronic devices is also getting higher and higher. Among them, the picture (static or dynamic picture) displayed on the display screen of the electronic device consists of static pictures that are constantly refreshed frame by frame. The higher the refresh rate of an electronic device, the more information the display can display in 1 second. The greater the number of static images, the smoother the images displayed on the display.

In the traditional solution, taking the electronic device as a mobile phone as an example, the mobile phone uses a single buffer to display images. In the single buffer solution, the image drawing process and the screen refresh process of the image use the same buffer. If the refresh rate of the screen is inconsistent with the drawing frequency of the image, it may cause two picture areas to be displayed on the screen at the same time, where the two picture areas belong to different image frames, causing the image displayed on the screen to appear split.

In order to solve the problem of single buffer, the electronic device of the present application uses double buffer to display images. Exemplarily, FIG. 3a illustrates the process of an electronic device using a double buffer to display each frame of image.

As shown in Figure 3a, taking the electronic device as a mobile phone as an example, the mobile phone may include an application program, a layer composition module, a logic manager, as well as a buffer A for drawing images and a buffer B for displaying images. Among them, the application here is the application to which the interface currently displayed on the mobile phone screen belongs. For example, the application program may be an application installed on the mobile phone, or may be a system application (for example, an application that provides a mobile phone desktop), which is not limited in this application.

For example, as shown in Figure 3b, the main interface displayed on the mobile phone screen may include the icon interface 201 shown in (1) of Figure 3b, the system notification bar interface 202 shown in (2) of Figure 3b, and the system notification bar interface 202 shown in (2) of Figure 3b. The status bar interface 203 shown in (4). The icon interface 201 may include a desktop background image 2013 and a plurality of icons. Here, application icons, power icons, network icons, etc. are shown. In order to display the main interface on the mobile phone screen, the mobile phone can execute the process shown in Figure 3a to realize the display of one frame of the main interface.

As shown in Figure 3a, in order to transfer the drawn image in buffer A to buffer B for display, a VSync (vertical synchronization) signal is introduced in the electronic device. The layer synthesis module can receive the signal period T0 of the VSync signal, and the layer synthesis module can send the VSync signal to the above-mentioned application program, so that the application program draws the image in the buffer A, wherein each time the application program receives a VSync signal, execute Drawing of a frame of image.

For example, based on the number of windows included in the image that needs to be drawn, the application can call the CPU or GPU to create a corresponding number of queues in buffer A to achieve parallel drawing of different layers in one frame of image. For example, in order to realize the drawing of the main interface shown in Figure 3b, the application program can call the CPU to render window 1 in the queue 11 in the buffer to draw the status bar interface 203 shown in (4) of Figure 3b, and Window 2 is rendered in queue 12 to draw the system notification bar interface 202 shown in (2) of Figure 3b, and window 3 is rendered in queue 13 to draw the icon interface 201 shown in (1) of Figure 3b. The desktop background image 2013 is rendered, and the window 4 is rendered in the queue 14 to draw each icon in the icon interface 201 shown in (1) of Figure 3b. The application can draw different windows in the main interface in parallel in buffer A by calling the CPU or GPU.

In some embodiments, every time the CPU completes rendering a window in a queue (for example, queue 11) in buffer A, it will write window 1 that has been completed in queue 11 to the queue in buffer B (here: Queue 21), the CPU can therefore continue to render the window 1 in the image of the next frame of the main interface in the queue 11 of the buffer A. The execution process of other queues in buffer A is the same to achieve parallel rendering of different windows of the image and improve the rendering efficiency of the image.

Continuing to refer to Figure 3b, the user can slide down from the left half area at the top of the screen shown in (1) of Figure 3b to display the system notification bar interface 202 shown in (2) of Figure 3b, and from The right half area at the top of the screen shown in (3) slides down to display the status bar interface 203 shown in (4) of Figure 3b. In other words, the system notification bar interface 202 and the status bar interface 203 are processed by layer synthesis. Then referring to Figure 3a, the application can call the CPU to render window 1 (here is the status bar) in queue 11 and window 2 (here is the system) rendered in queue 12 notification bar), written to the queue 21 in the buffer B to perform layer synthesis, so as to present the display of the status bar interface 203 and the system notification bar interface 202 as shown in FIG. 3b.

In some embodiments, the application can also call the CPU to write the window 3 (here, the desktop background image) that has been rendered in the queue 13 and the window 4 (here, the desktop icon) that has been rendered in the queue 14 to the buffer B. Queue 22 within.

For example, the layer synthesis module can call the CPU or GPU to perform layer synthesis on the status bar and system notification bar in the queue 21 , and write the layer-composited image into the queue 22 . And the layer synthesis module can perform hardware synthesis on the desktop background image, desktop icons, and layer-composited images in the queue 22, so as to obtain a frame of the main interface image that can be displayed on the screen in the queue 22. Here, it is with the mobile phone. The screen resolution of the image is the same size.

As can be seen from the process in Figure 3a, before a frame of image is displayed on the screen, it needs to be rendered by the application. The layer synthesis module synthesizes the rendered image (layer synthesis and hardware synthesis in sequence). deal with). Finally, the layer synthesis module provides the hardware-synthesized image to be displayed on the screen to the corresponding display screen object in the logic manager (for example, display screen object 1 corresponding to the main screen), and display screen object 1 can be read from the cache area B. The image to be uploaded is displayed on the screen, so that the main screen of the mobile phone can display any interface shown in (1) to (4) in Figure 3b.

In some embodiments, the layer synthesis module can also send the layer-synthesized image and the image to be hardware-synthesized obtained from buffer A to the hardware synthesis module (a module belonging to the hardware abstraction layer) to implement image synthesis. Hardware synthesis, and can read the hardware synthesis result of the image from the hardware synthesis module to provide it to the corresponding display object.

For example, the window manager in the frame layer of the application in Figure 2 can also be used to notify the layer management module of the mapping relationship, where the mapping relationship is between the rendered window in buffer A and the one displayed in the logical manager The mapping relationship of screen objects. The mutually mapped window and display screen object indicate that the window is used to send display to the display screen corresponding to the display screen object, that is, the window content required for a frame of image that the display screen needs to display.

For example, the layer management module can call the CPU (or GPU) to synthesize the windows required for each display screen object according to the above-mentioned mapping relationship provided by the window manager to obtain the on-screen image of each display screen object. . In different scenarios, the images to be displayed on the screen required by different display screen objects in the logic manager may be different or the same, and this application does not limit this. For example, in a multi-screen simultaneous display scenario, if the window content required by each display screen corresponding to each display screen object is the same, then the layer synthesis module can perform layer synthesis and hardware synthesis according to the windows required by each display screen object, so as to obtain A frame of image to be displayed on the screen to be sent to each display screen object.

In some embodiments, the layer management module can serially synthesize the images to be displayed on each display screen object, and this application does not limit this.

Continuing to refer to Figure 3a, after receiving the signal period T0, the layer synthesis module can not only send a VSync signal to the application program to notify the application program to render a frame of image, but also send a VSync signal to each in the logic manager. The display screen object is used to display the images to be displayed on the screen in buffer B. For example, the layer synthesis module can send a VSync signal to each display screen object in the logic manager to notify the display screen object to read its respective frame of image to be displayed on the screen from the corresponding queue in buffer B.

As can be seen from the description of Figure 3a, in the double buffer scenario, the mobile phone can send a VSync signal to the application through the layer synthesis module to achieve the rendering, layer synthesis, hardware synthesis of a frame of image, and through Send a VSync signal to the display object in the logic manager to realize the display of a frame of image after hardware synthesis, thereby realizing double buffer data exchange.

Currently, the FPS (Frames per Seconds) of a mobile phone's display can have multiple levels, such as 10, 20, 40, and 60. Among them, the maximum FPS of a mobile phone's display is 60. When the FPS of the screen is at a certain gear, the layer synthesis module can sample the signal period of the VSync signal corresponding to the gear from the hardware (such as the display screen). For the convenience of explanation, the full text of this application uses the current value of the mobile phone's display screen. FPS is at the highest gear as an example to illustrate.

In traditional technology, when the layer synthesis module sends a VSync signal to the application and when it sends a VSync signal to each display object in the logic manager, the sending period of the VSync signal is the signal period T0 sampled from the hardware. . The signal period T0 is the reciprocal of the current FPS of the mobile phone screen. For example, the current FPS of the mobile phone's main screen is 60, then the signal period T0 is 1/60.

Based on the introduction of the FSP of the mobile phone display, it can be seen that the mobile phone display is limited by the hardware display, and its maximum FPS is 60. Then the signal period T0 is also limited by the hardware of the mobile phone screen, so that the application can only render up to 60 frames per second. . Moreover, in traditional technology, when the layer synthesis module sends VSync signals to each display screen object in the logic manager, it also sends VSync signals to each display screen object according to the unified signal period T0 to notify each display. The screen objects obtain their respective images to be displayed on the screen in buffer B. Then the display screen corresponding to each display screen object can only display up to 60 frames of images per second.

So in a multi-screen simultaneous display scenario, such as the scenario where the mobile phone projects to the TV as shown in Figure 3b, Figure 3c and Figure 4, in traditional technology, as shown in Figure 3c, the mobile phone shown in Figure 3b communicates with the router through Wi-Fi data connection, the TV and the router are connected through Wi-Fi data, then the user can operate the mobile phone in Figure 3b to project the content displayed on the mobile phone screen to the TV screen for display.

For example, as shown in Figure 4, the screen refresh rate of the mobile phone is 60 frames/s, the FPS of the display screen of the mobile phone is 60, and the FPS of the display screen of the TV is 120. For example, as the end that shares the code stream, the mobile phone can be called the local end, and as the end that receives the code stream, the TV can be called the opposite end.

For example, referring to (4) of Figure 3b, the status bar interface 203 may include multiple controls, such as icons for shortcut switches of multiple system functions, such as the "Airplane Mode" shortcut switch icon and the "Mobile Data" shortcut switch icon. , "Mute" shortcut switch icon, "Auto-rotate" shortcut switch icon, "Do Not Disturb" shortcut switch icon, "Screen Recording" shortcut switch icon, "Super Power Saving" shortcut switch icon, "Wireless Screen Mirroring" shortcut switch icon 2031 wait.

For example, the user clicks the "wireless screen projection" shortcut switch icon 2031, and the mobile phone can respond to the user operation and display the icon interface 201 shown in (5) of Figure 3b, wherein the icon interface 201 displays a floating window 2011. The window 2011 includes an icon 2012 of a screen-casting device searched by the mobile phone (here, the TV in the living room). For example, the user clicks the icon 2012, and the mobile phone can respond to the user operation by creating a display screen object 2 in the logic manager shown in Figure 3a, such as display2 marked as a virtual screen. The display screen object 2 here corresponds to the display screen shown in Figure 3a. The screen of the TV shown in Figure 4; in some embodiments, the logic manager also includes a display screen object 1, for example, display 1 marked as the main screen. The display screen object 1 here corresponds to the screen of the mobile phone shown in Figure 4.

Combined with the process of Figure 3a, after the mobile phone receives the signal period T0, in traditional technology, the mobile phone can respond to the user's operation of clicking the icon 2012 in (5) of Figure 3b, and send the VSync signal to the application according to the signal period T0, and Send the VSync signal to display screen object 1 and display screen object 2 according to the signal period T0. Combined with the process in Figure 3a, the application can draw up to 60 frames of images per second, and both the display screen object 1 corresponding to the main screen and the display screen object 2 corresponding to the virtual screen can obtain up to 60 frames of images per second.

For example, in the screen projection scenario shown in Figure 4, the mobile phone can transmit the image obtained by the display screen object 2 as a screen projection code stream to the TV through the Wi-Fi network, and display the image obtained by the display screen object 1 on the phone screen. As shown in Figure 4, the display content of the mobile phone screen is the icon interface 201 as shown in Figure 3b, and the display content of the TV screen is also the icon interface 201. Moreover, the mobile phone screen can display 60 frames of images per second, and the frequency of the code stream sent by the mobile phone to the TV as screen projection content is also 60 frames/s.

Although the current FPS of the TV screen is 120 (the maximum FPS of the TV screen is 120), because the frequency of drawing images at the local end is limited by the signal period T0 of the VSync signal sampled from the hardware, the TV as the peer end When displaying media data transmitted by the mobile phone (local end), the media data can only be displayed according to the screen refresh rate of the local end. However, the code stream transmitted by the peer cannot be displayed according to the peer's own screen refresh rate. Therefore, in a multi-screen simultaneous display scenario, when the screen refresh rate of the local end is lower than the screen refresh rate of the opposite end, the local end can only draw frames and send display according to the screen refresh rate of the local end, but the opposite end does not perform image processing. of drawing. In these cases, the local end with a lower refresh rate cannot provide a code stream higher than the local refresh rate to the opposite end.

In order to solve the problem in traditional technology that in a multi-screen simultaneous display scenario, the local end can only provide a code stream with the same refresh rate as the local end to the opposite end for display, but cannot provide a code stream that exceeds the local end's screen refresh rate, this application An electronic device is provided that can break through the screen refresh rate limit of the local terminal and draw frames to achieve high refresh rate display of media data on the opposite terminal.

For example, the multi-screen simultaneous display scenarios applied to the electronic device of the present application may include but are not limited to the following scenarios: screen projection, screen recording, video calls, live broadcasts, video conferencing, super terminals, multi-screen collaboration and other scenarios. For example, in various embodiments of the present application, in a multi-screen simultaneous display scenario, the number of local terminals is at least one, and the number of opposite terminals is at least one. Among them, the local end can be an electronic device that produces a code stream of media data and displays the code stream, and the opposite end can be an electronic device that receives the code stream produced by the local end and displays the code stream. For example, the media type of the media data may include but is not limited to: still images, videos, etc.

For example, in some scenarios, such as a screen recording scenario, the local terminal and the opposite terminal may be the same electronic device.

For example, in a screen casting scenario, the electronic device that performs the screen casting operation is the local end, and the electronic device that receives the screen casting code stream is the opposite end.

For example, in a video call scenario, the electronic device that shares the video picture is the local end, and the electronic device that receives the code stream of the video picture sent by the local end is the opposite end.

For example, in a live broadcast scenario, the electronic device that performs the live broadcast operation is the local end, and the electronic device that receives the code stream of the live video sent by the local end is the opposite end.

For example, in a video conference scenario, the electronic device that shares the video picture is the local end, and the electronic device that receives the code stream of the video picture sent by the local end is the opposite end.

For example, in a hyper terminal scenario, the electronic device that shares the video picture is the local end, and the electronic device that receives the code stream of the video picture sent by the local end is the opposite end.

In some embodiments, the local end and the opposite end may be electric cars, the electronic device sharing the video picture may be an electric car, and the electronic device receiving the video picture may be another electric car. For example, the video screen of this application can be shared or received through the screen of an electric car.

In some embodiments, the local end can be an electric car and the opposite end can be a mobile phone. The electric car can share video images to the mobile phone for multi-screen simultaneous display. In some embodiments, the local terminal can be a mobile phone, the opposite terminal can be an electric car, and the mobile phone can share Video images can be displayed simultaneously on multiple screens in electric vehicles.

In some embodiments, in response to the image display request, the electronic device sharing the image (or video screen) can serve as the local end, and the electronic device receiving the code stream (image stream or video stream) can serve as the peer end. Then, in different scenarios Below, the electronic device serving as the local end and the electronic device serving as the peer end are both adjustable. For example, the same electronic device can be switched from the local end to the opposite end, the same electronic device can also be switched from the opposite end to the local end, and the same electronic device can also be used as both the local end and the opposite end.

For example, mobile phone 1, mobile phone 2 and mobile phone 3 communicate and connect for video conference.

In one scenario, mobile phone 1 shares video footage to mobile phone 2 and mobile phone 3, then mobile phone 1 serves as the local end, and mobile phone 2 and mobile phone 3 are both opposite ends. The images displayed by the three mobile phones are all video images shared by mobile phone 1.

In another scenario, mobile phone 2 also starts to share the video screen, then mobile phone 1 and mobile phone 2 both share their respective video screens in the video conference. So in this scenario, mobile phone 2 also serves as the local end, mobile phone 3 is still the opposite end, and mobile phone 1 and mobile phone 2 both serve as the local end for sharing video images. In addition, mobile phone 1 can also receive and display the video screen shared by mobile phone 2, so mobile phone 1 can not only serve as the local end, but also serve as the opposite end. In the same way, mobile phone 2 can also receive and display the video screen shared by mobile phone 1, so mobile phone 2 not only serves as the local end, but also serves as the opposite end.

Figure 5a is a schematic diagram illustrating an exemplary image display process of an electronic device in a multi-screen simultaneous display scenario. The image display process can be understood in conjunction with the screen projection scenarios of Figures 5b to 5d. Among them, for the process of applying the drawing image and the layer synthesis module to synthesize the image in Figure 5a, as well as the data exchange process between the buffer area A and the buffer area B, please refer to the description in Figure 3a. The principles are similar and will not be described again here.

As shown in Figure 5a, the process may include the following steps:

S101. The layer synthesis module determines the first period T11 based on the current screen refresh rate of the local end (FPS0=1/T0) and the current screen refresh rate of the opposite end.

For example, please refer to Figure 5b. As shown in (1) of Figure 5b, the display interface 200 of the mobile phone may include at least one control. The control includes but is not limited to: a network icon control, a power icon control, an application icon control, etc., The application icon control may include the Huawei Video icon control 2013. When the user clicks the icon control 2013, the mobile phone may launch the Huawei Video application in response to the user operation and display the display interface 301 as shown in (2) of Figure 5b.

As shown in (2) of Figure 5b, the display interface 301 is the video playback interface of the Huawei video application. The display interface 301 may include at least one control, which includes but is not limited to: playback progress bar control 402, playback progress control 404, current Play time control 403, video duration control 406, play pause control 405, screen projection control 302.

For example, after the user clicks the playback pause control 405, the mobile phone can respond to the user operation by pausing or starting the playback of the video played in the display interface 301. For example, the playback duration control 406 is used to represent the total duration of the video played in the display interface 301 (here, it is 50 minutes). The current playback time control 403 is used to represent the current playback progress of the video played in the display interface 301, and the corresponding playback time point in the above total duration (referred to as the current playback time point, here is 0 minutes and 0 seconds). For example, when the user drags the playback progress control 404 along the playback progress bar control 402, the mobile phone can adjust the playback progress of the video played in the display interface 301 in response to the user's operation.

For example, as shown in (2) of Figure 5b, the user clicks the screen projection control 302. In response to the user operation, the mobile phone can display the display interface 301 shown in (3) of Figure 5b. The display interface 301 can further include: Control 303 displays the name of the device that can project the screen (here is the TV in the living room). In some embodiments, the control 303 also includes Option Control 304. The user can click the option control 304, and the mobile phone can respond to the user operation, as shown in Figure 5c and Figure 5d, to connect the mobile phone to the TV through Wi-Fi, and the mobile phone can display the video content on the mobile phone screen in real time through the screen projection protocol. The video stream is projected to the TV screen for simultaneous display.

Exemplarily, in response to the user clicking the option control 304 shown in (3) of Figure 5b, the mobile phone can obtain the screen of the TV from the TV through the device connection protocol (here, the screen casting protocol) between the opposite end and the local end. Refresh rate (for example, the peer's FPS for short). For example, the FPS of the opposite end is 120. In some embodiments, as shown in Figure 5a, the mobile phone can also create a display screen object 2 (for example, display 2 marked as a virtual screen) in the logic manager in response to the user's operation of the clickable option control 304. In some embodiments, the logical manager also includes a mobile phone display screen object 1 (for example, display 1 marked as the main screen), where the display screen object 1 is used to manage the input and output of the mobile phone screen. The display screen object 2 is used to manage the input and output of the TV screen shown in Figure 5c and Figure 5d.

In some embodiments, the layer synthesis module of the mobile phone can periodically obtain sampling data of the VSync signal period (such as 00001000001, where 1 indicates that the VSync signal needs to be sent) from the display hardware of the mobile phone (such as a liquid crystal display (LCD, Liquid Crystal Display) , 0 means no VSync signal needs to be sent), the layer synthesis module can regularly obtain the signal period T0 of the VSync signal from the LCD. As mentioned above, the signal period T0 is directly related to the FPS0 of the mobile phone's LCD, T0=1/FPS0. For example, if the FPS0 of the local terminal (such as a mobile phone) is 60, then T0=1/60.

In traditional technology, the layer synthesis module can only periodically send VSync signals to applications and display objects according to the signal period T0 sampled from the LCD, so that the image drawing rate and refresh rate are affected by the VSync sampled from the hardware locally. The signal period limit of the signal.

In the mobile phone according to the embodiment of the present application, the layer synthesis module can modify the signal period T0 (here 1/60) sampled from the local hardware based on the FPS of the opposite end obtained from the opposite end (here 120). , the first period T11 is obtained.

For example, as shown in the embodiment of Figure 3a, when the mobile phone performs image rendering (or drawing) and image synthesis, it is performed according to the signal period T0, so the signal period T0 is also called the image drawing period. In this embodiment, the layer synthesis module can modify the signal period T0 of the VSync signal sampled from the hardware, and the modified image drawing period is T11. Then when the mobile phone performs image rendering and image synthesis, it is performed according to T11. Then for each T11, the mobile phone can get one frame of image to be displayed on the screen, and then the mobile phone can draw multiple frames of images according to T11.

For example, the periodic signal in the form of hardware sent by the LCD to the layer synthesis module cannot be modified, but the layer synthesis module can convert the periodic signal in the form of hardware into a periodic signal in the form of software, and then, the software periodic signal T0 Make modifications so that the periodic signal in the modified software form is T11. Then the layer synthesis module can notify the application program to draw frames according to the first period T11, so that the drawing frequency of the application program is not limited by the periodic signal T0 sampled by the hardware.

In some embodiments, as the mobile phone runs longer, the CPU execution command is blocked, etc., which may cause the time interval between the signal period in the software form and the signal period in the hardware form to become larger and larger, resulting in software form. The periodic signal is inaccurate, causing problems such as screen blur on the mobile phone screen. In order to calibrate the periodic signal in the form of software, in a multi-screen simultaneous display scenario, the layer synthesis module can modify the periodic signal in the form of software every time it receives the periodic signal in the hardware form of the LCD, thereby achieving regular calibration. The effect of the signal period of the VSync signal prevents problems such as screen blur.

S104. The layer synthesis module sends the VSync signal to the application program according to the first period T11.

For example, as shown in Figure 5b, the application here is the application corresponding to the interface currently displayed on the mobile phone screen, here it is the Huawei video application.

S105, the application renders the image according to the first cycle T11.

S106, the layer synthesis module synthesizes the image according to the first cycle T11.

Through the above S105 and S106, the local terminal can draw multiple frames of images (for example, the first image) according to the first period T11.

For example, the current FPS of the local end is 60 and the current FPS of the opposite end is 120. When the layer synthesis module modifies the signal period of the VSync signal, it can modify it through two strategies, corresponding to the different modification strategies of the signal period. There are also differences in the strategies for displaying the main screen and the virtual screen. For example, strategy 1 may be a least common multiple strategy. Strategy 2 can be a strategy higher than the maximum FPS.

Based on Implementation Mode 1 of Strategy 1, the local FPS is 60, then T0=1/60, and the opposite FPS is 120. The layer synthesis module can determine the target FPS when modifying the signal period, where T11 is the reciprocal of the target FPS. . In this implementation, the target FPS can be 120, and the layer synthesis module can shorten the signal period of the VSync signal by half. In this regard, the layer synthesis module can shorten the signal period when sending the VSync signal to the application by half, T11=(1/2)*T0=1/120. In this way, the layer composition module can control the local application to draw images at a frequency of 120 frames/s. Since the application can draw one frame of image each time it receives a VSync signal, the signal period of the VSync signal (also called the image drawing period) is shortened by half, which can double the image drawing amount of the application and increase the image drawing amount. In the same way, when the layer synthesis module synthesizes images, it also synthesizes at a frequency of 120 frames/s.

Based on Implementation Mode 2 of Strategy 2, the local FPS is 60, then T0=1/60, and the opposite FPS is 120. The layer synthesis module can determine the target FPS when modifying the signal period, where T11 is the reciprocal of the target FPS. . In this embodiment, the target FPS may be a value higher than the local FPS and the opposite end FPS, for example, the target FPS is 180. T11=1/180, T11 is one third of T0. In this way, the layer composition module can control the local application to draw frames and render images at a frequency of 180 frames/s. Since the application can draw one frame of image each time it receives a VSync signal, shortening the signal period of the VSync signal by half can double the image drawing volume of the application and increase the image drawing volume. In the same way, when the layer synthesis module synthesizes images, it also synthesizes at a frequency of 180 frames/s.

It should be noted that, as mentioned above, the FPS of the display screen can have multiple gears. For example, the FPS of the mobile phone includes four FPS gears of 10, 20, 40, and 60. In this embodiment, the current FPS of the mobile phone is used. Taking the maximum gear (such as 60) as an example to illustrate the modification strategy of the signal period. However, when the current FPS of the mobile phone is at another FPS level, the signal period T0 received by the layer synthesis module from the display screen also changes. For example, the current FPS of the mobile phone is 40, then the signal period T0 = 1/40, then Based on the above strategy 1, when the layer synthesis module determines the first period T11, then T11=(1/3)*T0. The layer synthesis module can shorten the current signal period of the mobile phone to 1/3 of the original signal period T0. So that T11=1/120. The same applies to other strategies and will not be repeated here.

For example, after S101, the layer synthesis module may execute S102 and S103. This application does not limit the execution order of S102 and S103. In addition, this application does not limit the execution order between S102, S103 and S104. S104 is executed periodically according to T11. For example, S107 is executed after S102, and S108 is executed after S103.

S102, the layer synthesis module can determine the second period T12 to be displayed to the local terminal based on the current screen refresh rate of the local terminal and the first period T11.

S107, the layer synthesis module sends the VSync signal to the display screen object 1 according to the second period T12.

Then the display screen object 1 can read a frame of the image to be displayed on the screen synthesized by the layer synthesis module each time it receives the VSync signal. The display screen object 1 can send the frame image to the display driver, and the display driver can The image is sent to the mobile phone's display (such as LCD) for on-screen display.

S103. The layer synthesis module determines the third period T13 for displaying to the opposite end based on the current screen refresh rate of the opposite end and the first period T11.

S108, the layer synthesis module sends the VSync signal to the display screen object 2 according to the third period T13.

Then the display screen object 2 can read a frame of the image to be displayed on the screen synthesized by the layer synthesis module every time it receives the VSync signal, and send the frame image to the codec of the application framework layer to process the frame. The image is encoded to obtain the code stream. The local end can send the code stream to the opposite end, so that part or all of the multi-frame images drawn by the local end according to the first period T11 can be refreshed and displayed on the opposite end's display screen in the third period T13.

In some embodiments, the second period T12 is greater than the third period T13.

The local terminal can send the corresponding part of the first image to the display screen object 1 according to the second period T12 for display on the local terminal screen; the local terminal can also send the corresponding part or all of the first image according to the third period T13. The first image is sent to display screen object 2, so as to send part or all of the first image drawn by the local end according to the first period T11 to the opposite end, so as to display part or all of the first image on the opposite end's display screen. .

In some embodiments, the second period T12 is greater than the third period T13, and the third period T13 may be equal to the first period T11. Then within an exemplary period of time, the local terminal may draw multiple frames according to the first period T11. All the images in the image are sent to the display screen object 2 corresponding to the opposite end, and the end can send part of the multi-frame images drawn according to the first cycle T11 to the display screen object 1 corresponding to the local end. .

In some embodiments, the first period T11 may be greater than the third period T13, and the first period T11 may be greater than the second period T12. Then within an exemplary time period, the local terminal may draw multiple times according to the first period T11. frame image, according to the second period T12, part of the image in the multi-frame image is sent to the display screen object 1 corresponding to the local end, and according to the third period T13, part of the image in the multi-frame image is sent to the The display screen object 2 corresponding to the opposite end, wherein the image sent to the display screen object 1 by this end and the image sent to the display screen object 2 may have the same image.

For example, S102, S103, S107 and S108 can be understood in conjunction with the above-mentioned Embodiment 1 and 2 respectively:

In Embodiment 1, FIG. 5e is a schematic diagram illustrating an exemplary image drawn by an application and an image sent to different display screen objects.

In Embodiment 1, the target FPS is 120. For example, the Huawei video application draws images at a frequency of 120 frames/s, and the layer synthesis module performs image synthesis at a frequency of 120 frames/s, thereby obtaining the image to be displayed on the screen. , the P1 frame to the Pi frame shown here, where the Pi frame represents the image to be displayed on the screen (or to be sent for display) of the i-th frame generated by the layer synthesis module, and i is a positive integer.

In this embodiment 1, the FPS of the local end is 60, the FPS of the opposite end is 120, the FPS of the local end drawing images is 120, the signal period T11 of the VSync signal of the local end is 1/120, then the second period T12 sent to the local end is It can be 2 T11s. The layer synthesis module can send a VSync signal to the display screen object 1 corresponding to the main screen every two signal periods of the VSync signal (for example, 1/240) to notify the display screen object 1 to read from the buffer B. A frame of image to be displayed on the screen. For example, as shown in Figure 5e, the images sent to the main screen by the layer synthesis module may include P1 frame, P3 frame, P5 frame, and P7 frame. frames etc. In addition, in other cases, the images sent to the main screen by the layer synthesis module may include P2 frames, P4 frames, P6 frames, P8 frames, etc.

Since the FPS of the opposite end is 120 and the FPS of the local end drawing images is also 120, the layer synthesis module can send a VSync signal to the display screen object 2 corresponding to the virtual screen in each signal period of the VSync signal (for example, T13=T11). , to notify display screen object 2 to read a frame of image to be displayed on the screen from buffer B. For example, as shown in Figure 5e, the image sent to the virtual screen by the layer synthesis module is the same as the image synthesized by the application drawing and layer synthesis module, such as P1 frame to Pi frame.

For example, every time display screen object 1 reads a frame of image to be displayed on the screen from buffer B, it can send the frame image to the display driver, and the display driver can send the frame image to the display screen of the mobile phone (such as LCD). ) to display on the screen. For example, as shown in Figure 5c, the refresh rate of the image displayed on the display interface 301 of the mobile phone is 60 frames/s. In this way, although the application frame volume is doubled (120 frames/s), when the image synthesis module sends the display to the main screen of the mobile phone, it can send the display once every two VSync signal periods (for example, every two T11). As a result, the refresh rate of the image displayed on the display screen of the mobile phone (such as an example of this end) is the same as the FPS of the display screen of this end, and the FPS is 60.

For example, every time the display screen object 2 reads a frame of image to be displayed on the screen from buffer B, it can send the frame of image to the codec of the application framework layer to encode the frame of image to obtain the code. flow. For example, as shown in Figure 5c and Figure 5d, the mobile phone can send the code stream at 120 frames/s to the TV through the screen projection protocol. The TV can decode the code stream through the codec on the TV side to obtain the video stream. And the video stream is displayed on the TV screen, so that the refresh rate of the screencast content displayed on the TV screen is 120 frames/s.

Compared with Figure 4 in traditional technology, we can see that in traditional technology, the current FPS of the local end (taking the maximum gear 60 as an example) is used as the benchmark to draw frames, and a unified signal is used for the display objects of the end and the opposite end. The VSync signal is sent for display in period T0, so that the refresh rate of the image displayed on the mobile phone screen is the same as the refresh rate of the code stream sent to the TV, which is 60 frames/s. In this embodiment of the present application, the mobile phone can modify the signal period of the VSync signal of the local terminal based on the FPS of the opposite terminal, and draw frames according to the modified signal period, so that the amount of drawn frames is increased. In some embodiments, the mobile phone can also send displays to the home screen and the peer virtual screen with different signal cycles based on the modified signal cycle, so that the frequency of images sent to the home screen is consistent with the frequency of the images. The current FPS of the display screen (here is 60) is consistent, and the frequency of the code stream sent to the opposite end display screen is consistent with the current FPS of the opposite end display screen (here is 120). In a multi-screen simultaneous display scenario, the local end can provide a code stream that exceeds the maximum screen refresh rate of the local end to the opposite end to achieve high refresh rate display of media data on the opposite end.

It should be noted that in a multi-screen simultaneous display scenario, the media type of the shared code stream at the local end of the shared code stream may include but is not limited to: static images, videos, documents and other displayable data. That is to say, what the local end projects to the opposite end is a static image, not a video, and the image is also transmitted to the opposite end in the form of a code stream. The solution in Figure 5a of this application can also be used to realize the image on the opposite end. High refresh rate display of the peer.

For example, S102, S103, S107 and S108 can be understood in conjunction with the above-mentioned Embodiment 2:

In Embodiment 2, FIG. 5f is a schematic diagram illustrating an exemplary image drawn by an application and an image sent to different display screen objects.

In Embodiment 2, the target FPS is 180. For example, in a screen projection scenario, the Huawei video application shown in Figure 5b draws images at a frequency of 180 frames/s, and the layer synthesis module performs image rendering at a frequency of 180 frames/s. The image is synthesized to obtain the image to be displayed on the screen. The P1 frame to the Pi frame are shown here, where the definition of the Pi frame is the same as in Figure 5e The same, so we won’t go into details here.

Based on implementation mode 2 of strategy 2, the current FPS of the local end is 60, the current FPS of the opposite end is 120, and the FPS of the local end drawing images is 180 (higher than the maximum value of the local FPS and the opposite end's FPS, such as 120). The signal period of the VSync signal at the end is T11 = 1/180 = (1/3) * T0, then the second period T12 sent to the local end can be 3 T11, and the layer synthesis module can be every 3 signal periods of the VSync signal ( T12=3*T11) Send a VSync signal to display screen object 1 corresponding to the main screen to notify display screen object 1 to read a frame of image to be displayed on the screen from buffer B. For example, as shown in Figure 5f, the images sent to the main screen by the layer synthesis module may include P1 frames, P4 frames, P7 frames, etc. Since the FPS of the opposite end is 120 and the FPS of the local end drawing images is 180, when the layer synthesis module sends the display image to the virtual screen according to the third cycle T13, it will miss one frame after every two frames of the image are sent to the virtual screen. Send the show. Among them, as mentioned above, the layer synthesis module synthesizes one frame of image that can be sent for display every T11. For example, as shown in Figure 5f, the images sent to the virtual screen by the layer synthesis module include P1 frames, P2 frames, P4 frames, P5 frames, P7 frames, P8 frames, etc. Of course, the images sent to the virtual screen by the layer synthesis module are not limited to the example in Figure 5f, as long as the layer synthesis module misses one frame after sending two frames of images to the virtual screen. For example, the images sent to the virtual screen by the layer synthesis module can also be: P1 frame, P3 frame, P4 frame, P6 frame, P7 frame, P9 frame, etc. Alternatively, the images sent to the virtual screen by the layer synthesis module can also be P2 frames, P3 frames, P5 frames, P6 frames, P8 frames, P9 frames, etc.

Then, through the display of the difference between the main screen and the virtual screen in the second embodiment, the refresh rate of the image displayed on the local end can be 60, and the refresh rate of the image displayed on the opposite end can be 120. The similarities between Embodiment Mode 2 and Embodiment Mode 1 will not be repeated here. For the specific display sending process, please refer to the introduction of Embodiment Mode 1.

For example, the process of Figure 5a can also be understood based on Embodiment 3 to Embodiment 6 shown in Figures 6a to 6d. Among them, the principles of Figures 6a to 6d are similar to those of Figures 5e and 5f, and the similarities will not be described again.

In Figures 6a to 6d, different signal cycle modification strategies are taken as examples to describe the process of different displaying of the main screen and the virtual screen. In this application, different signal period modification strategies can be determined based on two factors: the load of the local terminal and the picture quality of the images displayed by the local terminal and the opposite terminal.

In Embodiment 3 to Embodiment 6, the current FPS of the local end is 60 (the maximum FPS of the display screen of the local end is 60), and the current FPS of the opposite end is 90 (the maximum FPS of the display screen of the opposite end is 90). , of course, when the FPS of the local end and the opposite end are other values, or when the number of opposite ends is multiple, for the scenario where the maximum FPS of the local end is smaller than the current FPS of the opposite end, the same method can be based on Embodiment 3 to Embodiment 6. principles to implement the technical solution in Figure 5a, which will not be described again here.

Exemplary implementation 3 based on least common multiple strategy 1:

Similar to the principle of Embodiment 1 above, the local end can set the target FPS to the least common multiple of the local end FPS (60) and the opposite end FPS (90), here it is 180, then T11 is 1/3 times of T0, so that the mobile phone displays For applications corresponding to the on-screen interface, when drawing images, the amount of image drawing triples.

As shown in Figure 6a, when the layer synthesis module sends the display to the main screen, it can send a VSync signal to the main screen every 3 signal cycles of the VSync signal, for example, every 3 T11 to send a frame of image to the main screen, so that this The refresh rate of the image displayed on the home screen of the client is the same as the FPS of the display of the client, which is 60. For example, as shown in Figure 6a, the images sent to the main screen are P1 frame, P4 frame, P7 frame, etc. in order. Alternatively, the images sent to the main screen by the layer synthesis module may be P2 frames, P5 frames, P8 frames, P11 frames, etc. in sequence. The layer composition module sends a VSync every 3 T11 Signal to notify display object 1 to read a frame of image from buffer B. Of course, the sequence of images sent to the main screen is not limited to the example in Figure 6a, as long as the effect of sending only one frame of image to the main screen for every three consecutive frames of images generated by the layer synthesis module is achieved.

As shown in Figure 6a, when the layer synthesis module sends the display to the virtual screen, it can send a VSync signal to the virtual screen every two signal cycles of the VSync signal, for example, every two T11 to send one frame of image to the virtual screen. . The refresh rate of the image displayed on the opposite end's screen is the same as the FPS of the opposite end's display, which is 90. For example, as shown in Figure 6a, the images sent to the virtual screen are P1 frame, P3 frame, P5 frame, P7 frame, etc. in order. Alternatively, the images sent to the virtual screen by the layer synthesis module may be P2 frames, P4 frames, P6 frames, P8 frames, etc. in sequence. For example, the layer synthesis module sends a VSync signal every 2 T11 to notify display screen object 2 to read a frame of image from buffer B. Of course, the sequence of images sent to the virtual screen is not limited to the example in Figure 6a, as long as the effect of sending only one frame of image to the main screen for every two consecutive frames of images generated by the layer synthesis module is achieved.

In this embodiment 3, the mobile phone can send a display to the virtual screen once every 2 signal cycles of the VSync signal, and to the main screen once every 3 signal cycles of the VSync signal, so that the main screen of the local end and the display screen of the opposite end (corresponding to here The time interval between frames in the image displayed on the virtual screen is uniform. For example, the mobile phone can obtain an even 60-frame image from the 180-frame image refreshed by the application and display it on the local end, and obtain an even 90-frame image for display on the opposite end. For example, in a multi-screen simultaneous display scenario, the images displayed at the local end and the opposite end are linear motion images, such as the image of throwing a curling stone (assuming that the curling motion is uniform), then in this embodiment 3, The linear motion images in the pictures displayed at the local end and the opposite end are uniform. The display quality of the pictures at the local end and the opposite end is optimal, and the best display performance can be achieved. However, the requirements for the image rendering capability of the local end are relatively high, resulting in a high load on the local end.

Exemplary implementation 4 of strategy 3 based on taking the maximum FPS among the local and peer FPS:

For example, the local end can set the target FPS to the maximum FPS of the local end FPS (60) and the opposite end FPS (90), here it is 90, then T11 is 2/3 times of T0, so that the mobile phone display screen displays For applications corresponding to the interface, when drawing images, the amount of image drawing increases.

As shown in Figure 6b, when the layer synthesis module sends the display to the main screen, it may not send the VSync signal to the main screen for one signal period in every three signal periods of the VSync signal. When the layer synthesis module sends the display image to the main screen according to the third period T12, it may miss one frame after sending two frames of images to the main screen. Among them, as mentioned above, the layer synthesis module synthesizes one frame of image that can be sent for display every T11. As shown in Figure 6b, the layer synthesis module can miss one frame after every two frames of images are sent to the main screen, so that the refresh rate of the image displayed on the local home screen is the same as the FPS of the local display screen, which is 60. For example, as shown in Figure 6a, the images sent to the main screen are P1 frame, P2 frame, P4 frame, P5 frame, P7 frame, P8 frame, etc. in order. In the layer composition module, two T11s out of every three T11s will send a VSync signal to the display screen object 1 respectively, but the remaining T11 will not send a VSync signal to the display screen object 1, so that the display screen object 1 will be out of control every three T11s. There are 2 T11s that can read one frame of image from buffer B respectively. Of course, the sequence of images sent to the main screen is not limited to the example in Figure 6a, as long as the effect of only sending 2 frames to the main screen in every 3 consecutive frames is achieved. For example, the images sent to the main screen by the layer synthesis module can also be: P1 frame, P3 frame, P4 frame, P6 frame, P7 frame, P9 frame, etc. Alternatively, the images sent to the main screen by the layer synthesis module can also be P2 frames, P3 frames, P5 frames, P6 frames, P8 frames, P9 frames, etc.

As shown in Figure 6b, when the layer synthesis module sends the display to the virtual screen, the signal period of each VSync signal (for example For example, T13=T11) all send a VSync signal to the display screen object 2 corresponding to the virtual screen to notify the display screen object 2 to read a frame of the image to be displayed on the screen from the buffer B. For example, as shown in Figure 6b, the image sent to the virtual screen by the layer synthesis module is the same as the image synthesized by the application drawing and layer synthesis module, such as P1 frame to Pi frame.

In this embodiment 4, the mobile phone can send and display each VSync signal cycle to the virtual screen once, so that the time interval between frames in the image displayed on the opposite end's screen is uniform, and the opposite end's image quality is optimal. . The mobile phone sends the display to the main screen every three signal periods of the VSync signal to the main screen, causing the time interval between frames in the image displayed on the local main screen to be uneven, with long and short frames. For example, in a multi-screen simultaneous display scenario, the images displayed at the local end and the opposite end are linear motion images, such as the image of throwing a curling stone (assuming that the curling motion is uniform), then in this embodiment 4, The linear motion image in the picture displayed by the opposite end is uniform, the picture display quality is optimal, and the best display performance can be achieved. However, the linear motion image in the picture displayed on this end is uneven (for example, the image display effect is that the curling stone advances 5 meters, 5 meters, 10 meters, 5 meters, 5 meters, and 10 meters in sequence). Although in this Embodiment 4, compared with Embodiment 3, there is loss between frames in the picture displayed by the local end, and the display effect is slightly worse, compared with Embodiment 3, this Embodiment 4 has better effects on the image displayed by the local end. The rendering capability requirements are relatively low, which can reduce the load requirements on the local end to a certain extent.

Exemplary, implementation 5 of strategy 2 based on higher than maximum FPS:

Similar to the principle of Embodiment 2 above, the local end can set the target FPS to a higher value than the maximum FPS (here 90) between the local end FPS (60) and the opposite end FPS (90), here it is 120, then T11 is 1/2 times of T0 makes the application corresponding to the interface displayed on the mobile phone display double the amount of image drawing when drawing images.

Of course, the target FPS is not limited to the 120 in the example here. It can be any value higher than 90. The specific value can be comprehensively determined based on factors such as the load requirements of the local end and the display performance of the local and opposite end displays. numerical value.

As shown in Figure 6c, when the layer synthesis module sends the display to the main screen, it can send a VSync signal to the main screen every 2 signal cycles of the VSync signal, for example, every 2 T11 to send a frame of image to the main screen, so that this The refresh rate of the image displayed on the home screen of the client is the same as the FPS of the display of the client, which is 60. For example, as shown in Figure 6c, the images sent to the main screen are P1 frame, P3 frame, P5 frame, P7 frame, etc. in order. For example, the layer synthesis module sends a VSync signal every 2 T11 to notify display screen object 1 to read a frame of image from buffer B. Of course, the sequence of images sent to the main screen is not limited to the example in Figure 6c, as long as the effect of sending only one frame to the main screen in every two consecutive frames is achieved.

As shown in Figure 6c, when the layer synthesis module sends the display to the virtual screen, it may not send the VSync signal to the virtual screen in one of every four signal cycles of the VSync signal. When the layer synthesis module sends the display image to the virtual screen according to the third period T13, it may miss one frame after sending three frames of images to the virtual screen. Among them, as mentioned above, the layer synthesis module synthesizes one frame of image that can be sent for display every T11. Then as shown in Figure 6c, the layer synthesis module can miss one frame after sending three frames of images to the virtual screen, so that the refresh rate of the image displayed on the opposite end's display screen is the same as the FPS of the opposite end's display screen, which is 90. For example, as shown in Figure 6c, the images sent to the virtual screen by the layer synthesis module are P1 frame, P2 frame, P3 frame, P5 frame, P6 frame, P7 frame, P9 frame, etc. in order. For example, in the layer composition module, 3 of every 4 T11s will send a VSync signal to display screen object 2, but the remaining T11 will not send a VSync signal to display screen object 2, so that every 4 T11s of display screen object 2 There are 3 T11s available from the buffer Area B reads one frame of image respectively. Of course, the sequence of images sent to the virtual screen is not limited to the example in Figure 6c, as long as the effect of only sending 3 frames to the virtual screen in every 4 consecutive frames is achieved. For example, the images sent to the virtual screen by the layer synthesis module can also be: P2 frame, P3 frame, P4 frame, P6 frame, P7 frame, P8 frame, P10 frame, P11 frame, P12 frame, etc.

In this embodiment 5, the mobile phone can send the display to the main screen once every two VSync signal cycles, so that the time interval between frames in the image displayed on the local screen is uniform, and the local image quality is optimal. . The local layer synthesis module sends the display to the virtual screen every four signal cycles of the VSync signal to the virtual screen, causing the time interval between frames in the image displayed by the opposite end's display screen to be uneven. , there are long and short frames. For example, in a multi-screen simultaneous display scenario, the images displayed at the local end and the opposite end are linear motion images, such as the image of throwing a curling stone (assuming that the curling motion is uniform), then in this embodiment 5, The linear motion image in the picture displayed on this end is uniform, the picture display quality is optimal, and the best display performance can be achieved. However, the linear motion image in the picture displayed by the opposite end is uneven (for example, the image display effect is that the curling stone advances 3 meters, 3 meters, 3 meters, 6 meters, 3 meters, 3 meters, 3 meters, 6 meters in sequence) . Although, in this Embodiment 5, compared with Embodiment 3, there is loss between frames in the picture displayed at the local end, and the display effect is slightly worse. However, in Embodiment 5, compared with Embodiment 4, the time interval between frames is shorter, the problem of long and short frames is alleviated, and the display effect is better. In addition, compared with Embodiment 3, this Embodiment 5 has relatively lower requirements on the image rendering capability of the local terminal, which can reduce the load requirements on the local terminal to a certain extent.

Exemplary implementation 6 of strategy 4 based on no modification of signal period:

For example, in this embodiment 6, the local end can set the target FPS to the same value as the local FPS (60), then T11=T0, that is, the signal period of the VSync signal is not modified, so that the interface displayed on the mobile phone display screen corresponds to When drawing an image, the image drawing amount remains unchanged. Among them, the FPS of the opposite end is 90.

As shown in Figure 6d, when the layer synthesis module sends the display to the main screen, it can send a VSync signal to the main screen every signal cycle of the VSync signal, for example, every T11, so as to send a frame of image to the main screen, so that the local main screen The refresh rate of the displayed image is the same as the FPS of the local display, which is 60. For example, as shown in Figure 6d, the images sent to the main screen are P1 frame, P2 frame, P3 frame, P4 frame, P5 frame, P6 frame, P7 frame...Pi frame, etc. in order. The layer composition module sends a VSync signal every T11 to notify display screen object 1 to read a frame of image from buffer B.

As shown in Figure 6d, when the layer synthesis module sends the display to the virtual screen, it can send a VSync signal to the virtual screen once in the signal cycle of the VSync signal to send a frame of image to the virtual screen once, and in the next VSync signal signal cycle, send the VSync signal to the virtual screen twice to display the next frame of image to the virtual screen twice, and the cycle continues. Then the refresh rate of the image displayed on the opposite end's display screen can be the same as the FPS of the opposite end's display screen, which is 90. For example, every two frames in every 90 frames of images displayed by the peer end are the same, and after an interval of one image frame, the other two frames of images are the same again. For example, as shown in Figure 6d, the images sent to the virtual screen by the layer synthesis module are P1 frame, P1 frame, P2 frame, P3 frame, P3 frame, P4 frame, P5 frame, P5 frame, P6 frame, etc. . The layer synthesis module will send a VSync signal to display screen object 2 in one T11, then 2 VSync signals to display screen object 2 in the next T11, and then another VSync signal to display screen object 2 in the next T11, lower right A T11 sends 2 VSync signals to the display object…. This allows display object 2 to read a frame of image from buffer B at one T11, and at the next T11 it can read an image from buffer B. Buffer B reads two frames of the same image. Of course, the sequence of images sent to the virtual screen is not limited to the example in Figure 6d, as long as the effect of two consecutive frames of the same image in every three consecutive frames sent to the virtual screen is achieved.

In this embodiment 6, the mobile phone can send and display each signal cycle of the VSync signal to the main screen once, so that the time interval between frames in the image displayed on the local screen is uniform, and the local image quality is optimal. Moreover, the signal period of the local VSync signal is not modified, so that the load on the local end is not increased, and the local load is optimal. However, there are two identical images in every three frames displayed on the local end. Compared with Embodiment 3, the screen display effect is slightly worse.

In practical applications, the local end can flexibly use any of the above strategies 1 to 4 to modify the signal period of the VSync signal, as well as the differentiated VSync signals are sent to the main screen and virtual screen at different cycles for display. In addition, the modification of the signal period and the different strategies for sending displays to the main screen and the virtual screen are not limited to the above-mentioned strategies 1 to 4, and may also include other strategies not shown, which are not limited in this application.

In some embodiments, the above strategies 1 to 4 are also applicable to scenarios where there are multiple local terminals and/or multiple peer terminals, as long as the FPS of the local terminal's display screen is lower than the FPS of the peer terminal's display screen. That’s it. For example, when there are multiple peers, the layer synthesis module of the local peer can distinguish the display screen object of the local peer and the display screen objects of each peer, so as to process the different display screen objects in the logic manager according to their respective signals. The display is sent periodically to enable the local end with a low refresh rate to provide a code stream higher than the refresh rate of the local end to the opposite end with a high refresh rate. In a multi-screen simultaneous display scenario, when the peer screen has higher display performance than the local screen, the refresh rate of the displayed content on the peer screen is not limited by the performance (refresh rate) of the local screen.

In the above embodiments of the present application, the local end can modify the signal period of the VSync signal of the local end based on the FPS of the opposite end, and control the application of drawing frames according to the modified signal period, so that the amount of drawing frames is increased. In addition, the mobile phone can also send the display to the local home screen and the remote virtual screen with different signal cycles based on the modified signal cycle, so that the frequency of images sent to the local display screen is consistent with the local display screen. The FPS is consistent, and the frequency of the code stream sent to the opposite display is consistent with the FPS of the opposite display. In a multi-screen simultaneous display scenario, the local end can provide a code stream that exceeds the screen refresh rate of the local end to the opposite end to achieve high refresh rate display of media data on the opposite end.

In some embodiments, referring to Figure 5a, the process of rendering layers on the application side and the process of compositing images on the layer synthesis module side are both executed according to the unified VSync signal period (signal period T11 here). , then when the local end sends the VSync signal to display screen object 1 according to the second period T12, and sends the VSync signal to display screen object 2 according to the third period T13, there is no need to check whether the corresponding response has been synthesized in buffer B. For the image to be displayed on the display screen object, you only need to notify the corresponding display screen object according to the set period to read the image to be displayed on the screen, because the layer synthesis module notifies each display screen in the logic manager Before an object reads its respective image to be displayed on the screen, the image to be displayed on each display screen object has been generated in the buffer B according to the first period T11.

In some embodiments, the VSync signal sent by the layer composition module to the application or logic manager side may be a VSync signal generated by the layer composition module. In addition, the layer composition module notifies the application to render the image, and notifies the display object to read the image from the buffer B. It is not limited to the method of sending the VSync signal. It can also be other notification methods, as long as the notification effect can be achieved. That’s it.

In addition, as shown in the description of the logic manager in Figure 2, based on different scenarios, the logic manager can include three types of display screen objects. One type is the display screen marked as the main screen or the back screen or the first screen or the second screen. Objects, one type is the display screen object marked as wired screen, and the other type is the display screen object marked as virtual screen. In the screen projection scenarios of Figures 5b to 5d, the above embodiment takes the display screen object 1 marked as the main screen and the display screen object 2 marked as the virtual screen in the logic manager as examples to illustrate the display shown in Figure 5a. The local end can provide a code stream that exceeds the local end's screen refresh rate (here, the FPS of the main screen) to the opposite end to realize the process of high refresh rate display of media data on the opposite end.

In the scenario of simultaneous display of multiple screens, the local terminal has many types of display screens, and the number of the local terminal and the opposite terminal is at least one. For example, the number of the local terminal and the opposite terminal can be multiple. Therefore, based on the multi-screen simultaneous display scenario, electronic devices for simultaneous display of media data are differentiated. The logical manager may include a display screen object labeled Home Screen, and/or a display screen object labeled Back Screen, and/or a display screen object labeled First Screen in Folding Screen, and/or a display screen object labeled First Screen in Folding Screen. A display screen object of the second screen, and/or at least one display screen object marked as a wired screen, and/or at least one display screen object marked as a virtual screen. Moreover, no matter what kind of display screen objects the logic manager includes, the process in Figure 5a can also be implemented, with similar principles and similar effects, which will not be described again here.

In some embodiments, in a multi-screen simultaneous display scenario, the communication connection method between the local end and the opposite end may include but is not limited to at least one of the following methods: logging in to the same account, connecting to the same Wi-Fi, Bluetooth connection, NFC (near field Communication, Near Field Communication), etc., this application does not impose restrictions on this.

In a possible implementation, in a multi-screen simultaneous display scenario, when the maximum FPS of the local display screen (for example, the local current FPS is the maximum FPS, such as 120) is higher than the maximum FPS of the opposite end's display screen (for example, the opposite end's current FPS is the maximum FPS) When the current FPS of the terminal is the maximum FPS, such as 60), in order to enable the local terminal to transmit the code stream with a refresh rate of 60 frames/s to the opposite terminal for on-screen display, for example, as described in the above embodiment, the local terminal When transmitting a frame of image to the opposite end, it needs to go through operations such as image rendering, layer synthesis, hardware synthesis, sending display objects to the display screen, and encoding. Then the local end can extend the cycle of any of the above operation links to This achieves the purpose of transmitting the code stream with a refresh rate of 60 frames/s to the opposite end for display on the screen. In a scenario where the maximum FPS of the local display is lower than the FPS of the opposite display, when sending the display to different display objects, the display should be sent with different display cycles to achieve the same refresh rate as the opposite display. The same code stream as the FPS of the screen is transmitted to the peer display. In this embodiment, the execution cycle can be extended in any of the above operation steps to achieve the effect of transmitting a code stream with the same refresh rate as the FPS of the opposite end display screen to the opposite end display. For example, this application does not limit the specific method of extending the period.

In a possible implementation, different applications installed in the electronic device (taking a mobile phone as an example) require different refresh rates. In traditional technology, when the layer synthesis module sends a VSync signal to any application, it sends the VSync signal according to the signal period T0 sampled from the hardware, so that the image drawing amount of different applications in unit time is the same, which is the screen of the mobile phone. The amount of refresh in unit time. However, the refresh rate required by each application may be different. For example, game applications have higher refresh rate requirements, while reading applications have lower refresh rate requirements. In traditional technology, a unified signal period T0 is used to control the refresh rate of different applications. Image drawing and screen refresh will cause waste of power consumption or loss of performance of the mobile phone.

To this end, the electronic device provided by this application can also set different signal periods of the VSync signal for different applications based on the refresh rates required by each application, and send the VSync signal according to the signal period corresponding to each application. to each application, so that each application can draw images according to their respective required refresh rates, so that the media data of different applications can be displayed on the electronic device at different refresh rates.

For example, as mentioned above, the FPS of the display screen of the mobile phone may have multiple levels, and then the FPS required by the application may be one of the multiple FPS levels supported by the display screen.

Exemplarily, FIG. 7a is a schematic diagram of a process in which an electronic device displays an image of Application 1. FIG. 7b is a schematic diagram of a process of the electronic device displaying the image of Application 2.

For example, combined with the scenario shown in Figure 8, as shown in Figure 7a, the process includes the following steps:

S4011, when application 1 is started, the layer synthesis module modifies the current screen refresh rate of the local terminal according to the screen refresh rate required by application 1, and obtains the signal period T21 of application 1.

For example, this terminal is a mobile phone, and the maximum refresh rate of the display screen of this terminal is 180, which can provide refresh rates of 60, 90, 120, 180 and other gears. This application does not limit the specific gears. For example, the current screen refresh rate (FPS) of the local terminal is 90. For example, the signal period T0 of the VSync signal sampled by the layer synthesis module in the mobile phone from the mobile phone display screen is 1/90. For example, if the FPS required by Application 1 is 120, then the layer synthesis module can set the target refresh rate corresponding to Application 1 to 120, and modify the signal period of the VSync signal corresponding to Application 1 from unified T0 to T21. , where T21 is 1/120, and the signal period of application 1 is modified to 3/4 times the original signal period T0 (T21=(3/4)*T0), so that the image drawing amount of application 1 is increased.

Exemplarily, as shown in (1) of FIG. 8 , the display interface 600 of the mobile phone includes at least one control, which includes a power icon, a network icon, an application icon, etc., where the application icon includes the icon 601 of the video application 1 . The user clicks the icon 601 of the video application 1, and the mobile phone can start the video application 1 in response to the user operation. For example, as shown in (2) of FIG. 8 , the display interface of the mobile phone is switched from the display interface 600 shown in (1) of FIG. 8 to the display interface 603 shown in (2) of FIG. 8 .

For example, after the video application 1 is started, the layer synthesis module can obtain the FPS required by the video application 1 (here 120), and receive the signal period T0 of the mobile phone screen sampling in the layer synthesis module Finally, the signal period T0 is modified, and the signal period of the VSync signal for video application 1 is modified from T0 to T21 (here 1/120).

S4012: The layer synthesis module sends the VSync signal to Application 1 according to the signal period T21 corresponding to Application 1.

S4013, Application 1 renders the image according to the signal period T21.

S4014, the layer synthesis module synthesizes the image according to the signal period T21.

S4015, the layer synthesis module sends the VSync signal to the display screen object 1 according to the signal period T21.

The specific implementation process of S4013 to S4015 can be described with reference to Figure 3a and will not be described again here.

Among them, display screen object 1 is the display screen object of the mobile phone's home screen. Each time display object 1 receives a VSync signal, it can read a frame of the screen image of the mobile phone to be displayed from the corresponding queue in buffer B, and send the read frame of the image to be displayed to The display driver can send the frame of the image to be displayed on the screen to the display screen (such as LCD) of the mobile phone for display on the screen. For example, as shown in (2) of FIG. 8 , the display interface 603 of the mobile phone includes the video picture 604 of the video application 1, and the refresh rate of the video picture is 120 frames/s. In this way, when the video application 1 is used as an application displayed on the display screen, the image drawing amount of the video application 1 can be improved without being limited by the signal period of the VSync signal sampled by the hardware of the mobile phone display screen (the current FSP of the mobile phone screen limit), thereby improving the display performance of the video screen of the video application 1 on the mobile phone.

Continuing from Figure 7a, the electronic device can also perform the process of Figure 7b. As shown in Figure 7b, the process can include the following steps:

S4021, when application 2 starts, the layer synthesis module modifies the current screen refresh rate of the local terminal according to the screen refresh rate required by application 2, and obtains the signal period T22 of application 2.

For example, the application 2 here is the video application 2. As shown in (2) of Figure 8, the user slides upward along the arrow from the bottom of the display interface 603. The mobile phone can respond to the user operation and run the video application 1 from the foreground. The state switches to the background running state, and the display interface 600 shown in (3) of Figure 8 is displayed. The user clicks the icon 602 of the video application 2 in the display interface 600 in (3) of Figure 8, and the mobile phone can start the video application 2 in response to the user operation.

For example, after the video application 2 is started, the layer synthesis module can obtain the FPS required by the video application 2 (here 60), and the layer synthesis module receives the signal period T0 of the mobile phone screen sampling Finally, the layer synthesis module modifies the signal period T0, changes the signal period of the VSync signal for video application 2 from T0 (here 1/90) to T22 (here 1/60), and applies 2 to video The signal period is modified to 3/2 times the original signal period T0 (T22=(3/2)*T0), so that the image drawing amount of video application 2 is reduced.

S4022: The layer synthesis module sends the VSync signal to application 2 according to the signal period T22 corresponding to application 2.

S4023, Application 2 renders the image according to the signal period T22.

S4024, the layer synthesis module synthesizes the image according to the signal period T22.

S4025, the layer synthesis module sends the VSync signal to the display screen object 1 according to the signal period T22.

The specific implementation process of S4023 to S4025 can be described with reference to Figure 3a and will not be described again here.

Among them, display screen object 1 is the display screen object of the mobile phone's home screen. Each time display screen object 1 receives a VSync signal 0, it can read a frame of the mobile phone's display screen image to be displayed on the screen from the corresponding queue in buffer B, and send the read frame of the image to be displayed on the screen. To the display driver, the display driver can send the frame of the image to be displayed on the screen to the display screen (such as LCD) of the mobile phone for display on the screen. For example, as shown in (3) and (4) of Figure 8, after the user clicks the icon 602, the mobile phone can respond to the user operation by changing the display interface of the mobile phone from the display interface shown in (3) of Figure 8. The display interface 600 is switched to the display interface 605 shown in (4) of FIG. 8 . The display interface 605 includes a video picture 606 played by the video application 2, and the refresh rate of the video picture is 60 frames/s. In this way, when the video application 2 is used as an application displayed on the display screen, the amount of image drawing of the video application 2 can be reduced, without the need to draw and display images according to the signal period T0 of the VSync signal of the hardware of the display screen of the mobile phone. Reduce the power consumption of mobile phones and reduce the performance loss of mobile phones when drawing images.

FIG. 7c is a schematic diagram illustrating an image display process for different applications by an electronic device in a split-window scenario. The implementation process of the same steps in FIG. 7c as in FIG. 7a and FIG. 7b is similar and will not be repeated here. Let’s not go into details.

For example, this split-window scenario means that the same display screen of the same electronic device can display the application interfaces of multiple applications at the same time, and the application interfaces of different applications are displayed in different windows.

Exemplarily, the split-window scene may include but is not limited to at least one of the following: split-screen scene, floating window scene, picture-in-picture scene, etc.

As shown in Figure 7c, the process may include the following steps:

S401: The layer synthesis module modifies the current screen refresh rate of the local terminal according to the screen refresh rate required by each window application, and obtains the respective signal periods T21 and T22 of the window application.

For example, the window application here includes application 1 and application 2. When the layer synthesis module modifies the signal period of the VSync signal corresponding to each window application, the layer synthesis module can be used after the window application is started. Module receives When the signal period of hardware sampling is T0, the signal period of the VSync signal applied by the window is modified from T0 to T21 and T22. For the method of determining T21 and T22, reference may be made to the embodiment of FIG. 7a and FIG. 7b, which will not be described again here.

S4013, Application 1 renders the image according to the signal period T21.

S4014: The layer synthesis module synthesizes the image of application 1 according to the signal period T21.

S4023, Application 2 renders the image according to the signal period T22.

S4024: The layer synthesis module synthesizes the image of application 2 according to the signal period T22.

S402: The layer synthesis module applies respective signal periods according to the windowing, and determines the display period T23.

For example, although different window applications have different image refresh rates, when the images to be displayed on the screen of different window applications need to be displayed according to a unified signal cycle (for example, the display cycle T23 here) Send the display to the same display object to ensure that there will be no lagging problem when the mobile phone display displays the video images of different window applications.

For example, S402 may be executed after receiving the user's windowing operation.

In some embodiments, if the mobile phone does not receive the user's window splitting operation, the layer synthesis module continues to process the application currently displayed on the display screen of the mobile phone in the manner of S4015 and S4025 in the embodiments of Figure 7a and Figure 7b. The generated image is sent for display according to the signal period corresponding to the application. It is not necessary to display the images generated by each window application according to the unified display period T23 set here for at least two window applications.

S409: The layer synthesis module sends the VSync signal to the display screen object 1 according to the display sending period T23.

The process shown in Figure 7c is explained below based on different windowing scenarios:

1. Split screen scene:

For example, in this scenario, application 1 is video application 1, application 2 is video application 2, the local terminal is a mobile phone, the maximum FPS of the mobile phone display is 180, and the current FPS of the mobile phone is 90 (T0=1/90), The required FPS for video application 1 is 120 and the required FPS for video application 2 is 60. Then the layer synthesis module respectively modifies the signal period T0 sampled from the hardware according to the FPS required by video application 1 and video application 2. It can be obtained that T21 is 1/120 and T22 is 1/60.

Exemplarily, with reference to Figure 9, as shown in (1) of Figure 9, the user clicks the icon 601 of the video application 1 in the display interface 600 to start the video application 1, as shown in (2) of the mobile phone. The display interface switches from the display interface 600 shown in (1) of Figure 9 to the display interface 603. The display interface 603 displays the video screen played by the video application 1 at the FPS required by the video application 1. For example, the display interface 603 The video picture 604 inside is refreshed and displayed at a frequency of 120 frames/s.

Referring to (2) of FIG. 9 , the user slides from the right edge (or the left edge) to the center of the display window and stays there. As shown in (3) of FIG. 9 , the mobile phone may display the sidebar 607 on the right edge (or the left edge) of the application interface (for example, the display interface 603 ) of the video application 1 in response to the user operation. Referring to (3) of Figure 9 , the sidebar includes one or more controls, such as an email application icon, a memo application icon, a gallery application icon, a file management application icon, a video application 2 icon 6071, etc. , the sidebar can also include an add option, which is used to add the icon of the specified application to the sidebar. It should be noted that the name, quantity and position of each application in (3) of Figure 9 are only illustrative examples, and are not limited in this application. In (3) of Figure 9, the video screen of video application 1 is in The mobile phone’s display still refreshes at a frequency of 120 frames/s.

Continuing to refer to (3) of Figure 9 , for example, the user can drag the icon 6071 of the video application 2 in the sidebar to the lower half area of the mobile phone display window and release it, for example, drag the icon 6071 in the direction of the dotted arrow. , where the user operation is a split-screen operation. As shown in (4) of Figure 9, the mobile phone can respond to the split-screen operation by splitting the display window of the mobile phone into two screens, including a display window 608 and a display window 609. As shown in (4) of Figure 9, for example, The display window 608 of the mobile phone is used to display the application interface of video application 1, and the display window 609 is used to display the application interface of video application 2.

Exemplarily, in response to the split-screen operation, the mobile phone may also perform the above-mentioned S402 to determine the respective images to be displayed on the video application 1 and the video application 2, and send a signal when displayed to the display screen object 1 marked as the main screen. period (T23). As mentioned above, the image drawing frequency of video application 1 is 120 frames/s, the signal period T21 of drawing images is 1/120, the image drawing frequency of video application 2 is 60 frames/s, and the signal period T22 of drawing images is 1 /60.

For example, the layer synthesis module can determine T23 based on T21 and T22. This application does not limit the strategy for determining T23, as long as the images generated by the windowing application are all sent for display according to the unified signal period T23.

In some embodiments, when determining the display sending period T23, the layer synthesis module can use the signal period corresponding to the focus window in the mobile phone as the display sending period T23.

For example, as shown in (4) of FIG. 9 , the display window 609 clicked by the user's finger is the display window 608 and the focus window in the image window 609 .

As shown in (4) of Figure 9, the focus window (here is the display window 609) is the display window of the video picture of the video application 2, then the signal period corresponding to the display window 609 is the signal period corresponding to the video application 2 (T22, here is 1/60).

For example, T23=T22, then the layer synthesis module can send 60 VSync signals to display screen object 1 per second. Each time display screen object 1 receives a VSync signal, it obtains the synthesis of video application 1 from buffer B. The image 1 to be displayed on the screen is a frame of image in the buffer B for display in the window 1 (the display window 608 in (4) of Figure 9), and the video is obtained from the buffer B. The image 2 to be displayed on the screen is synthesized by application 2. The image 2 to be displayed on the screen is a frame of image in buffer B for display in window 2 (display window 609 in (4) of Figure 9). As shown in (4) of FIG. 9 , the video image of video application 1 in the display window 608 is refreshed at a frequency of 60 frames/s, and the video image of video application 2 in the display window 609 is refreshed at a frequency of 60 frames/s.

2. Floating window scene:

For example, with reference to Figure 10 , the processes of (1) of Figure 10 and (2) of Figure 10 are the same as the processes of (1) of Figure 9 and (2) of Figure 9 , which will not be described in detail here. Refer to Figure 10 Description of (1) of 9 and (2) of Fig. 9 .

Referring to (3) of FIG. 10 , for example, the user can click the icon 6071 of the video application 2 in the sidebar, where the user operation is a split-screen operation. As shown in (4) of Figure 10, the mobile phone can respond to the split-screen operation by splitting the display window of the mobile phone, including the display window 603 (ie, the display interface 603) and the display window 610, as shown in (4) of Figure 10 As an example, the display window 603 of the mobile phone is used to display the application interface of the video application 1, and the display window 610 is used to display the application interface of the video application 1. Shows the application interface of video application 2. For example, the display window 610 is a floating window. For example, the display window 610 may include a maximize control 6102, a minimize control 6103, and a close control 6101.

For example, the user clicks the maximize control 6102, and the mobile phone can respond to the user operation by maximizing the display interface 610. The maximized display interface 610 serves as the only display interface of the mobile phone, and the display effect is as shown in (4) of Figure 8 The display interface 606 is similar, but the refresh rate of the display interface is different. Here, the image refresh rate of the display interface 610 is 120 frames/s.

In some embodiments, when determining the display sending period T23, the layer synthesis module may use the signal period corresponding to the focus window in the mobile phone as the display sending period T23.

For example, as shown in (4) of Figure 10, the display window 603 clicked by the user's finger is the focus window.

As shown in (4) of Figure 10, the focus window (display window 603 here) is the display window of the video picture of video application 1, then the signal period corresponding to the display window 603 is the signal period corresponding to video application 1 (T21, here is 1/120).

For example, T23=T21, then the layer synthesis module can send 120 VSync signals to display screen object 1 per second. Each time display screen object 1 receives a VSync signal, it obtains the synthesis of video application 1 from buffer B. The image 1 to be displayed on the screen is a frame of image in the buffer B for display in the window 1 (the display window 603 in (4) of Figure 10), and the video is obtained from the buffer B. The image 2 to be displayed on the screen is synthesized by application 2. The image 2 to be displayed on the screen is a frame of image in buffer B for display in window 2 (display window 610 in (4) of Figure 10). As shown in (4) of FIG. 10 , the video image of video application 1 in the display window 603 is refreshed at a frequency of 120 frames/s, and the video image of video application 2 in the display window 610 is refreshed at a frequency of 120 frames/s.

2. Picture-in-picture scene:

For example, in this scenario, Application 1 is a desktop application (which can be a system application), Application 2 is a video application 2, and the local terminal is a mobile phone. The maximum FPS of the mobile phone display is 180, and the current FPS of the mobile phone is 90 ( T0=1/90), the FPS required for desktop application is 90, and the FPS required for video application 2 is 60. Then the layer synthesis module respectively modifies the signal period T0 sampled from the hardware according to the FPS required by the desktop application and video application 2 respectively. It can be obtained that T21 is 1/90 and T22 is 1/60.

Illustratively, with reference to Figure 11 , the processes of (1) and (2) of Figure 11 are the same as the processes of (3) and (4) of Figure 8 , which will not be described again here. Refer to Figure 11 Description of (3) of 8 and (4) of Fig. 8 .

As shown in (1) of Figure 11, although the display interface 600 displays a static image, the images in the display interface 600 are still refreshed at a frequency of 90 frames/s. The image rendering and image synthesis process of the display interface 600 can be Refer to the illustration about Figure 3a, which will not be described again here. As shown in (2) of FIG. 11 , the video screen in the display interface 605 is refreshed and displayed at the FPS corresponding to the video application 2 (for example, 60 frames/s).

Exemplarily, in this implementation, the video application 2 supports the picture-in-picture function, refer to (2) of Figure 11 and Figure 11 (3), after the user slides upward from the bottom of the display interface 605, the mobile phone can display the video content in the display interface 605 of the video application 2 in the picture-in-picture window 611 in (3) of Figure 11. As shown in (3) of Figure 11, the display interface of the mobile phone includes a display interface 600 (display window 600) and a picture-in-picture window 611.

Exemplarily, the mobile phone may also respond to the user's sliding up operation (a split-screen operation) in (2) of Figure 11 to perform the above S402 to determine the respective images to be displayed on the desktop application and the video application 2, The signal period (T23) when sending the display to display object 1 marked as the main screen. As mentioned above, the image drawing frequency of the desktop application is 90 frames/s, and the signal period T21 of the image drawing is 1/90. The image drawing frequency of the video application 2 is 60 frames/s, and the signal period T22 of the image drawing is 1/90. 60.

For example, as shown in (3) of Figure 11, the picture-in-picture window 611 clicked by the user's finger is the focus window.

As shown in (4) of Figure 11, the focus window (picture-in-picture window 611 here) is the display window of the video picture of video application 2, then the signal period corresponding to the picture-in-picture window 611 is the signal period corresponding to video application 2 (T22, here 1/60).

For example, T23=T22, then the layer synthesis module can send 60 VSync signals to the display screen object 1 per second. Each time the display screen object 1 receives a VSync signal, it obtains the synthesized data for the desktop application from the buffer B. Image 1 to be displayed on the screen, image 1 to be displayed on the screen is a frame of image in buffer B for display in window 1 (display window 600 in (3) of Figure 11), and the video application is obtained from buffer B 2 synthesizes the image 2 to be displayed on the screen, and the image 2 to be displayed on the screen is a frame of image in buffer B for display in window 2 (picture-in-picture window 611 in (3) of Figure 11). As shown in (3) of FIG. 11 , the image of the desktop application in the display window 600 is refreshed at a frequency of 60 frames/s, and the video image of the video application 2 in the picture-in-picture window 611 is refreshed at a frequency of 60 frames/s.

In the embodiment of the present application, when the electronic device is in a split-screen scenario, the electronic device can respond to different split-window applications according to the signal periods corresponding to the refresh rates required by the split-screen applications (referred to as split-window applications). The signal period notifies the window application to draw the frame, and performs layer synthesis and hardware synthesis on the drawn layers of each window application according to the signal period of each window application, thereby obtaining the refresh rate corresponding to each window application. Image to be displayed on screen. Since the split-window applications are displayed on the same display screen, when the electronic device displays the images to be displayed on the screen of different split-window applications, they can be displayed according to a unified signal cycle, so that the images of different split-window applications can be displayed according to the same signal cycle. The same cycle is displayed on the mobile phone display.

It should be noted that in Figures 5e, 5f, and 6a to 6d, the same reference numerals represent the same objects, and in Figures 5b, 5d, 5d, and 8 to 11, the same reference numerals represent the same objects. The reference numerals represent the same objects. Therefore, the reference numerals in each drawing are not explained one by one. It is enough to refer to each other and will not be described again here.

It can be understood that, in order to implement the above functions, the electronic device includes corresponding hardware and/or software modules that perform each function. In conjunction with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions in conjunction with the embodiments for each specific application, but such implementations should not be considered as beyond the scope of this application.

In one example, FIG. 12 shows a schematic block diagram of a device 300 according to an embodiment of the present application. The device 300 may include: a processor 301 and a transceiver/transceiver pin 302, and optionally, a memory 303.

The various components of device 300 are coupled together by bus 304, which includes, in addition to a data bus, a power bus, a control bus, and a status signal bus. However, for the sake of clear explanation, various buses are referred to as bus 304 in the figure.

In some embodiments, memory 303 may be used for instructions in the aforementioned method embodiments. The processor 301 can be used to execute instructions in the memory 303, and control the receiving pin to receive signals, and control the transmitting pin to send signals.

The device 300 may be the electronic device or a chip of the electronic device in the above method embodiment.

All relevant content of each step involved in the above method embodiments can be quoted from the functional description of the corresponding functional module, and will not be described again here.

This embodiment also provides a computer storage medium that stores computer instructions. When the computer instructions are run on an electronic device, the electronic device executes the above related method steps to implement the image display method in the above embodiment.

This embodiment also provides a computer program product. When the computer program product is run on a computer, it causes the computer to perform the above related steps to implement the image display method in the above embodiment.

In addition, embodiments of the present application also provide a device. This device may be a chip, a component or a module. The device may include a connected processor and a memory. The memory is used to store computer execution instructions. When the device is running, The processor can execute computer execution instructions stored in the memory, so that the chip executes the image display method in each of the above method embodiments.

Among them, the electronic equipment, computer storage media, computer program products or chips provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the corresponding methods provided above. The beneficial effects of the method will not be repeated here.

Through the description of the above embodiments, those skilled in the art can understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be allocated to different modules according to needs. The functional module is completed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be combined or can be integrated into another device, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

A unit described as a separate component may or may not be physically separate. A component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or it can be Each unit physically exists alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.

Any contents of various embodiments of this application, as well as any contents of the same embodiment, can be freely combined. Any combination of the above is within the scope of this application.

Integrated units may be stored in a readable storage medium if they are implemented in the form of software functional units and sold or used as independent products. Based on this understanding, the technical solutions of the embodiments of the present application are essentially or contribute to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium , including several instructions to cause a device (which can be a microcontroller, a chip, etc.) or a processor to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program code.

The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Inspired by this application, many forms can be made without departing from the purpose of this application and the scope protected by the claims, all of which fall within the protection of this application.

The steps of the methods or algorithms described in connection with the disclosure of the embodiments of this application can be implemented in hardware or by a processor executing software instructions. Software instructions can be composed of corresponding software modules. Software modules can be stored in random access memory (Random Access Memory, RAM), flash memory, read only memory (Read Only Memory, ROM), erasable programmable read only memory ( Erasable Programmable ROM (EPROM), electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), register, hard disk, removable hard disk, compact disc (CD-ROM) or any other form of storage media well known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage media may be located in an ASIC. Additionally, the ASIC can be located in a network device. Of course, the processor and storage media can also exist as discrete components in the network device.

Those skilled in the art should realize that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media can be any available media that can be accessed by a general purpose or special purpose computer.

Claims

An image display method, characterized in that it is applied to a first electronic device, and the first electronic device is communicatively connected to a second electronic device; the method includes:

In response to the received image display request, the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate; wherein the first refresh rate is the first electronic device. The refresh rate of the first display screen of the device, the second refresh rate is the refresh rate of the second display screen of the second electronic device, the first refresh rate is less than the second refresh rate;

The first electronic device draws multiple frames of first images according to the first cycle;

The first electronic device displays the corresponding part of the first image to the first display screen according to a second cycle;

The first electronic device sends the corresponding part or all of the first image to the second electronic device according to a third period; wherein the second period is greater than the third period.
The method of claim 1, wherein the first electronic device displays the corresponding part of the first image on the first display screen according to a second cycle, including:

The first electronic device determines a second period based on the first refresh rate and the first period;

The first electronic device displays the corresponding part of the first image to the first display screen according to the second cycle, wherein the first image is displayed on the first display screen at the first refresh rate. display.
The method according to claim 1 or 2, characterized in that the first electronic device includes a first display screen object;

The first electronic device displays the corresponding part of the first image on the first display screen according to the second cycle, including:

The first electronic device sends a first vertical synchronization signal to the first display screen object according to the second period, so that the first display screen object acquires part of the first image according to the second period. ; Wherein, the first display screen object is used to manage the input data and output data of the first display screen;

The first electronic device displays the corresponding part of the first image on the first display screen according to the second cycle through the first display screen object.
The method according to any one of claims 1 to 3, characterized in that the first electronic device sends the corresponding part or all of the first image to the second electronic device for display according to a third cycle. ,include:

The first electronic device determines a third period based on the second refresh rate and the first period;

The first electronic device sends the corresponding part or all of the first image to the second electronic device for display according to the third cycle, wherein the first image is displayed at the second refresh rate. on the second display screen.
The method according to any one of claims 1 to 4, wherein the first electronic device includes a second display screen object;

The first electronic device sends the corresponding part or all of the first image to the second electronic device for display according to the third cycle, including:

The first electronic device sends a second vertical synchronization signal to the second display screen object according to the third period, so that the second display screen object obtains part or all of the third display screen object according to the third period. An image; wherein the second display screen object is used to manage the input data and output data of the second display screen;

The first electronic device sends the corresponding part or all of the first image to the second electronic device according to the third cycle through the second display screen object to display on the second display screen.
The method according to any one of claims 1 to 5, characterized in that the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate, including:

When the first electronic device acquires the image drawing cycle sampled from the first display screen, the first electronic device changes the image drawing cycle based on the first refresh rate and the second refresh rate. Modified to a first period, where the image drawing period before modification is the reciprocal of the first refresh rate.
The method according to any one of claims 1 to 6, characterized in that the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate, including:

The first electronic device determines a third refresh rate based on a least common multiple of the first refresh rate and the second refresh rate;

The first electronic device shortens the image drawing cycle to the first cycle based on the third refresh rate, where the first cycle is the reciprocal of the third refresh rate.
The method according to any one of claims 1 to 6, characterized in that the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate, including:

The first electronic device determines a first period based on a maximum refresh rate among the first refresh rate and the second refresh rate;

The first electronic device shortens the image drawing cycle to the first cycle;

Wherein, the first period is the reciprocal of the maximum refresh rate.
The method according to any one of claims 1 to 6, characterized in that the first electronic device shortens the image drawing cycle to a first cycle based on the first refresh rate and the second refresh rate, including:

The first electronic device determines a fourth refresh rate based on the first refresh rate and the second refresh rate, wherein the fourth refresh rate is greater than the maximum refresh rate among the first refresh rate and the second refresh rate. ;

The first electronic device shortens the image drawing cycle to the first cycle based on the fourth refresh rate, where the first cycle is the reciprocal of the fourth refresh rate.
The method according to any one of claims 1 to 9, characterized in that the first image includes a second image, wherein the ratio of the number of the second image to the number of the first image is The ratio of the first refresh rate to the second refresh rate;

The first electronic device displays the corresponding part of the first image on the first display screen according to the second cycle, including:

The first electronic device displays the second image to the first display screen according to a second cycle;

The first electronic device sends the corresponding part or all of the first image to the second electronic device according to the third cycle, including:

The first electronic device sends the multiple frames of first images to the second electronic device according to a third cycle.
The method according to any one of claims 1 to 10, characterized in that, in response to the received image display request, the first electronic device shortens the image drawing cycle based on the first refresh rate and the second refresh rate. For the first cycle, include:

The first electronic device receives an image display request for the first application;

The first electronic device responds to the image display request and shortens the image drawing cycle of the first application to the first cycle based on the first refresh rate and the second refresh rate, wherein the The first image is an image to be displayed by the first application on the first display screen.
An electronic device, characterized in that it includes: a memory and a processor, the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device The device performs the image display method according to any one of claims 1 to 11.
An image display system, characterized by comprising the electronic device according to claim 12.
A computer-readable storage medium, characterized in that it includes a computer program that, when the computer program is run on an electronic device, causes the electronic device to execute the image display method according to any one of claims 1 to 11 .
A chip, characterized in that it includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send the signals to the processor, and the The signal includes computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device is caused to perform the image display method according to any one of claims 1 to 11.
A computer program product containing instructions, characterized in that when the computer program product is run on a computer, it causes the computer to execute the image display method according to any one of claims 1 to 11.