CN113805832A

CN113805832A - Image data transmission method, device, terminal and medium

Info

Publication number: CN113805832A
Application number: CN202111078949.XA
Authority: CN
Inventors: 高延凯; 王月文; 苗守飞; 钟柳和
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-09-15
Filing date: 2021-09-15
Publication date: 2021-12-17
Also published as: WO2023040592A1

Abstract

The embodiment of the application discloses an image data transmission method, an image data transmission device, a terminal and a medium. The method comprises the following steps: transmitting the m-th frame image data to the DDIC, wherein m is a positive integer; determining the historical refresh frequency of the DDIC when the m-n to m-1 frame images are displayed, wherein n is an integer less than m and more than or equal to 2; in response to the historical refresh frequency meeting the display sending delay condition, performing display sending delay operation on the (m + 1) th frame of image data, wherein the display sending delay operation is used for delaying the transmission of the (m + 1) th frame of image data to the DDIC; in response to completion of the presentation delay operation, image data of the (m + 1) th frame is transmitted to the DDIC. In the embodiment of the application, a display sending delay mechanism is introduced, so that the problem of flicker and jitter of the picture due to DDIC refresh frequency hopping caused by fluctuation of the AP output frame rate is avoided, the stability of DDIC refresh frequency in the image display process is improved, and the effect of improving the image display quality is achieved.

Description

Image data transmission method, device, terminal and medium

Technical Field

The embodiment of the application relates to the technical field of display, in particular to an image data transmission method, an image data transmission device, a terminal and a medium.

Background

With the continuous development of display screen technology, more and more display screens capable of supporting high-refresh-frequency display are developed, and when a high-frame-rate application program is operated or in the sliding operation process, the fluency of a picture can be improved by setting the display screen to be in a high-refresh-frequency mode.

For a Display screen adopting an Application Processor (AP) -Display Driver chip (DDIC) -Display Panel (Panel) driving architecture, in an image Display process, the DDIC adaptively adjusts a refresh frequency according to an output frame rate (i.e., a rate of outputting image data) of the AP, so as to achieve adaptive frequency conversion.

However, the output frame rate of the AP may fluctuate within a certain range, which may cause fluctuation of the refresh frequency of the DDIC, and when the refresh frequency jumps, for example, when the refresh frequency jumps from 45Hz to 72Hz, the problem of image flicker and jitter may occur, which may affect the image display quality.

Disclosure of Invention

The embodiment of the application provides an image data transmission method, an image data transmission device, a terminal and a medium. The technical scheme is as follows:

in one aspect, an embodiment of the present application provides an image data transmission method, which is used for an AP, and the method includes:

transmitting the m-th frame image data to the DDIC, wherein m is a positive integer;

determining the historical refresh frequency of the DDIC when the m-n to m-1 frame images are displayed, wherein n is an integer less than m and more than or equal to 2;

in response to the historical refresh frequency meeting a display delay condition, performing display delay operation on the m +1 th frame of image data, wherein the display delay operation is used for delaying the transmission of the m +1 th frame of image data to the DDIC;

transmitting the m +1 frame image data to the DDIC in response to completion of the presentation delay operation.

In another aspect, an embodiment of the present application provides an image data transmission apparatus, including:

a transmission module for transmitting the m-th frame image data to the DDIC, m being a positive integer;

a first determining module, configured to determine a history refresh frequency of the DDIC when displaying images of m-n th to m-1 th frames, where n is an integer smaller than m and greater than or equal to 2;

a delay module, configured to perform a display sending delay operation on the m +1 th frame of image data in response to that the history refresh frequency satisfies a display sending delay condition, where the display sending delay operation is used to delay transmission of the m +1 th frame of image data to the DDIC;

the transmission module is further configured to transmit the image data of the (m + 1) th frame to the DDIC in response to completion of the display delivery delay operation.

On the other hand, an embodiment of the present application provides a terminal, where the terminal includes an AP, a display screen, and a DDIC, where the AP and the DDIC are connected through a Mobile Industry Processor Interface (MIPI), and the AP is configured to execute at least one instruction in a memory to implement the above image data transmission method.

In another aspect, the present application provides a computer-readable storage medium, which stores at least one instruction for execution by a processor to implement the image data transmission method as described above.

In another aspect, embodiments of the present application provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, so that the terminal executes the image data transmission method provided by the above aspect.

In the embodiment of the application, by introducing the display sending delay mechanism, after the AP transmits the m-th frame of image data to the DDIC, whether the display sending delay condition is met is determined based on the historical refresh frequency of the DDIC in the recent n-frame image display process, and when the display sending delay condition is met, the m + 1-th frame of image data is transmitted to the DDIC after the display sending delay operation is performed on the m + 1-th frame of image data, so that the problem that the DDIC refresh frequency jumps due to fluctuation of the AP output frame rate, and further the picture flickers and shakes is avoided, the stability of the DDIC refresh frequency in the image display process is improved, and the effect of improving the image display quality is achieved.

Drawings

FIG. 1 is a diagram illustrating an image display process under the AP-DDCI-Panel architecture;

FIG. 2 is a schematic diagram illustrating an image data transmission method according to an embodiment of the present disclosure;

FIG. 3 illustrates a flow chart of an image data transmission method shown in an exemplary embodiment of the present application;

FIG. 4 is a graph comparing refresh rates with and without the introduction of a presentation delay mechanism;

FIG. 5 is a flow chart illustrating a history refresh frequency determination process according to an exemplary embodiment of the present application;

FIG. 6 is a schematic diagram illustrating an implementation of a historical refresh frequency determination process, according to an exemplary embodiment of the present application;

FIG. 7 illustrates a flow chart of an image data transmission method according to another exemplary embodiment of the present application;

FIG. 8 is a schematic diagram of an implementation of the image data transmission method of FIG. 7;

FIG. 9 shows a flow chart of an image data transmission method shown in another exemplary embodiment of the present application;

FIG. 10 is a schematic diagram of an implementation of the image data transmission method shown in FIG. 9;

fig. 11 is a block diagram showing a configuration of an image data transmission apparatus according to an embodiment of the present application;

fig. 12 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

As shown in fig. 1, under an AP-DDIC-Panel architecture, an AP side first performs layer rendering by an Application (App), then performs layer composition on a rendered layer by a surfefinger (layer composer) to obtain image data, and further transmits (writes) the image data to the DDIC by MIPI. The DDIC stores the image data sent from the AP to the Buffer, and controls Panel to perform image refresh Display (Display) by scanning (reading) the image data in the Buffer. In implementing adaptive frequency conversion, the DDIC adaptively adjusts the refresh rate according to the output frame rate of the AP (i.e., the amount of image data that the AP delivers to the DDIC per unit time, or the rate at which the AP delivers image data to the DDIC). For example, DDIC may decrease the refresh rate when the output frame rate of the AP decreases, and may increase the refresh rate when the output frame rate of the AP increases.

In the self-adaptive frequency conversion process, the change of the refresh frequency within a short time in a small range can not affect the image display quality, and when the change of the refresh frequency within a short time in a large range, the problems of flicker, jitter and the like can occur, thereby affecting the image display quality.

For example, in some scenes, when the output frame rate of the AP is changed from 60Hz to 45Hz in a short time and then from 45Hz to 72Hz, the refresh frequency of the DDIC is changed from 60Hz to 45Hz, which does not cause the flicker of the picture, and when the refresh frequency of the DDIC is changed from 45Hz to 72Hz, the flicker of the picture occurs because the change amplitude of the refresh frequency is too large.

In order to solve the above technical problem, in the embodiment of the present application, an AP side introduces a display sending delay mechanism. Under the mechanism, as shown in fig. 2, the AP acquires a history refresh frequency of the DDIC during a display process of a latest n-frame image (that is, a refresh frequency of the DDIC when each frame image in the latest n-frame image is displayed), and performs a display sending delay condition detection on the history refresh frequency based on a DDIC refresh frequency stabilization algorithm, so that when the display sending delay condition is satisfied, a display sending delay operation is performed on next frame image data, a problem that the refresh frequency jumps greatly is avoided, an effect of stabilizing the DDIC refresh frequency is achieved, and a problem of display image flicker caused thereby is reduced.

For example, the AP obtains the historical refresh frequency of DDIC in the last two frames of image display process, and when the last two frames of image display process are detected, the historical refresh frequency of DDIC is 60Hz (refresh frequency of the last second frame of image) and 45Hz (refresh frequency of the last frame of image), if the AP directly transmits the next frame of image data to DDIC after completing the data preparation, the refresh frequency of DDIC will be changed to 72 Hz; after a display sending delay mechanism is introduced, the AP detects that the historical refreshing frequency of the DDIC meets a display sending delay condition, so that the next frame of image data is transmitted to the DDIC after a certain time delay, the refreshing frequency of the DDIC is changed into 60Hz, and the direct large-amplitude jump of the refreshing frequency of the DDIC from 45Hz to 72Hz is avoided.

The method provided by the embodiment of the application is applied to the terminal, and the AP in the terminal executes the image data transmission method. The terminal may include a smart phone, a tablet computer, a wearable device (such as a smart watch), a portable personal computer, a smart television, and the like, and the specific type of the terminal is not limited in the embodiments of the present application.

Referring to fig. 3, a flowchart of an image data transmission method according to an exemplary embodiment of the present application is shown. The method comprises the following steps:

step 301, transmitting the m-th frame image data to the DDIC, m being a positive integer.

In a possible implementation, the AP is connected to the DDIC through the MIPI, and after the image data preparation is completed, the AP transmits the image data to the DDIC through the MIPI, and the DDIC controls a display screen (Panel) to display the image based on the image data.

Step 302, determining the historical refresh frequency of the DDIC when displaying the m-n to m-1 frame images, wherein n is an integer less than m and greater than or equal to 2.

To avoid a jump in the refresh frequency of the DDIC, the AP needs to determine the historical refresh frequency of the DDIC during the display of the last n frames of images (i.e., the m-n th to m-1 th frames of images) before transmitting the next frame of image data (i.e., the m +1 th frame of image data) to the DDIC, so as to detect whether the display transmission delay condition is satisfied based on the historical refresh frequency in the following. With regard to specific implementation of determining the history refresh frequency on the DDIC side, the following embodiments will be described in detail.

In a possible implementation manner, during the image display process, the AP monitors the refresh frequency of the DDIC during the image display process of each frame in real time, and stores the refresh frequency corresponding to each of the n frames of images closest to the m-th frame (current display frame), that is, the AP stores the historical refresh frequency of the latest n frames. When the image data of the (m + 1) th frame is transmitted, the AP acquires the stored n historical refresh frequencies.

In one illustrative example, when m is 10 and n is 2, the AP determines a first history refresh frequency of the DDIC during display of the image of the 10 th frame and before transmission of the image data of the 11 th frame, and a second history refresh frequency of the DDIC during display of the image of the 8 th frame.

Step 303, in response to the historical refresh frequency satisfying the display delay condition, performing display delay operation on the m +1 th frame of image data, the display delay operation being used for delaying the transmission of the m +1 th frame of image data to the DDIC.

In some embodiments, the presentation delay condition is used to filter sporadic acceleration requests on the AP side, avoiding DDIC going directly from low to high refresh rates. When the image data preparation speed of the AP side changes suddenly in a short time, the AP generates a scattered acceleration request, and the speed of preparing the image data by the AP side is reduced after the scattered acceleration request, so that the image data cannot be kept for a long time.

In a possible implementation manner, the AP determines whether there is an image preparation delay condition in the latest n-frame image based on the historical refresh frequency of the latest n-frame image, and if there is an image preparation delay, it determines that the condition of the display delay is met, so as to perform the display delay operation on the m + 1-th frame image data, and avoid that when the m + 1-th frame image data is prepared in advance, the refresh frequency jumps and rises due to the fact that the AP immediately transmits the m + 1-th frame image data to the DDIC (because the previous frame display delay makes the issue interval between two adjacent frames of image data shorter, and thus the refresh frequency rises).

In the embodiment of the present application, the purpose of the display delay operation is to reduce the refresh frequency of DDIC, so as to avoid the rising of DDIC refresh frequency jump caused by the sudden increase of the AP-side image preparation speed when there is an image preparation delay in the latest n frames of images.

Optionally, the manner of performing the presentation delay operation on the m +1 th frame of image data may include a Skip Tear Effect (TE) signal (Skip TE) or a barrier mipi (mipi block), which will be described in detail in the following embodiments.

Optionally, when the history refresh frequency of the latest n frames does not satisfy the display delay condition, the AP does not need to perform display delay operation on the image data of the (m + 1) th frame, and transmits the image data of the (m + 1) th frame to the DDIC according to the conventional display logic.

Step 304, in response to the completion of the presentation delay operation, transmits the m +1 th frame image data to the DDIC.

In one possible implementation, after completing the presentation delay operation for the image data of the (m + 1) th frame, the AP transmits the image data of the (m + 1) th frame to the DDIC according to the TE signal output by the DDIC. After the transmission of the image data of the (m + 1) th frame is completed, the AP re-executes the above steps 302 to 303 before the transmission of the image data of the (m + 2) th frame, which is not described herein again.

Optionally, when the refresh frequency of the DDIC changes, in order to avoid that the frequency change affects the image display, the DDIC adjusts parameters according to display screen parameters corresponding to the refresh frequency in a frame register (a register in the DDIC for storing a corresponding relationship between the refresh frequency and the display screen parameters), where the adjusted display screen parameters may include a Gamma parameter and a Demura parameter, which is not limited in this embodiment.

In an illustrative example, as shown in fig. 4, in the case where no presentation delay mechanism is introduced, the refresh frequency of the DDIC jumps from 45Hz to 72Hz during the display of the 5 th and 6 th frame images, and jumps from 51Hz to 72Hz during the display of the 13 th and 14 th frame images.

After introducing the display sending delay mechanism, before sending the 7 th frame image data to the DDIC, the AP determines that the historical refresh frequencies of the DDIC in the display process of the 4 th frame image and the 5 th frame image are respectively 60Hz and 45Hz, so that the display sending delay condition is determined to be met, the display sending delay operation is carried out, the 7 th frame image data is transmitted to the DDIC in a delayed mode, and the refresh frequency of the DDIC in the display process of the 6 th frame image is reduced to 60Hz, namely in the process of displaying the 5 th frame image and the 6 th frame image, the refresh frequency of the DDIC is increased from 45Hz to 60Hz and is not directly increased to 72 Hz; for another example: after introducing the display delay mechanism, before sending the 15 th frame image data to DDIC, AP determines to meet the display delay condition based on the historical refresh frequencies of DDIC during the 12 th and 13 th frame image display processes being 60Hz and 51Hz respectively, so as to delay the transmission of the 15 th frame image data to DDIC, and reduce the refresh frequency of DDIC during the 14 th frame image display process to 60Hz, that is, during the display of the 13 th and 14 th frame images, the refresh frequency of DDIC is increased from 51Hz to 60Hz, but is not directly increased to 72 Hz.

In summary, in the embodiment of the present application, by introducing the display sending delay mechanism, after the AP transmits the mth frame of image data to the DDIC, it is determined whether the display sending delay condition is satisfied based on the historical refresh frequency of the DDIC in the recent n frames of image display, and when the display sending delay condition is satisfied, after the display sending delay operation is performed on the m +1 th frame of image data, the m +1 th frame of image data is transmitted to the DDIC, so as to avoid the problem that the DDIC refresh frequency jumps greatly due to fluctuation of the output frame rate of the AP, which further causes flicker and jitter of the picture, thereby being beneficial to improving the stability of the DDIC refresh frequency in the image display process, and achieving the effect of improving the image display quality.

In one possible embodiment, the DDIC outputs a TE signal when it completes image display based on image data transmitted by the AP and is ready to refresh an image of a next frame, and accordingly, the AP transmits image data of a next frame to the DDIC when it completes image data preparation of a next frame and detects a TE signal. In the embodiment of the present application, the TE signal output by the DDIC is a Multiple-TE (Multiple tearing effect) signal, that is, when a next frame image is ready to be refreshed, the DDIC continuously outputs a plurality of TE signals according to a preset frequency. When the AP detects the rising edge of the TE signal, image data is transmitted to the DDIC. By outputting a Multiple-TE signal (which is equivalent to increasing the probability that the AP can detect the rising edge of the TE signal), the AP can transmit the image data to the DDIC in time after completing the preparation of the image data, which is helpful for reducing the image display delay.

Optionally, the TE frequency of the Multiple-TE signal is the same as the Emission (EM) frequency of the display screen, or the EM frequency is an integer Multiple of the TE frequency. For example, when the EM frequency of the display screen is 360Hz, the TE frequency of the Multiple-TE signal is 360Hz, i.e., one Multiple-TE signal is output every 2.8ms (1000 ÷ 360), or the TE frequency of the Multiple-TE signal is 180Hz, i.e., one Multiple-TE signal is output every 5.6 ms. The TE frequency is not limited in the embodiments of the present application. Accordingly, the AP side can determine the historical refresh frequency of the DDIC by detecting the number of Multiple-TE signals output by the DDIC. In one possible implementation, as shown in FIG. 5, the process of determining the history refresh frequency may include the following steps.

Step 302A, for the m-n th to m-1 th frame images, acquiring the historical number of Multiple-TE signals output by the DDIC in the display process of each frame image.

For each frame image in the latest n frame images, the AP counts the quantity of Multiple-TE signals output by the DDIC in the display process of each frame image to obtain the historical quantity corresponding to each frame image, wherein the historical quantity is the quantity of the Multiple-TE signals detected when the AP transmits the data of two adjacent frame images.

The display process of one frame image comprises a process of frame scanning by the DDIC, and a process of waiting for next frame image data (keeping the currently displayed image frame in the process) by the DDIC after the frame scanning is finished, because the DDIC does not output Multiple-TE signals in the process of frame scanning, the interval between the Multiple-TE signals respectively output by the DDIC after the adjacent two frame image frame scanning is finished (namely, the interval between the Multiple-TE signals output by the DDIC after the previous frame scanning is finished and the Multiple-TE signals output by the DDIC after the next frame image frame scanning is finished) is obviously larger than the interval between two or more Multiple-TE signals output by the DDIC after the same frame scanning is finished. Based on the above characteristics, the AP can record the number of Multiple-TE signals during each frame of image display based on the time interval between adjacent Multiple-TE signals using a counter.

Alternatively, this step may include the following substeps.

Firstly, a counter is set.

The AP sets a counter with which the number of Multiple-TE signals output by the DDIC during display of each frame of an image is recorded. The initial count value of the counter is 0, and the counter can be set by calling a counting thread.

Illustratively, as shown in fig. 6, the AP starts a counter after completing the transmission of the image data corresponding to the image frame a.

After detecting the Multiple-TE signal output by the DDIC, the AP calculates a time interval (in the presence of the previous adjacent Multiple-TE signal) between the Multiple-TE signal and a previous adjacent Multiple-TE signal (i.e., the last received Multiple-TE signal), and detects whether the time interval is smaller than an interval threshold. If the value is less than the preset value, executing the step two, and if the value is more than the preset value, executing the step three.

Optionally, the interval threshold is determined based on a TE frequency of the Multiple-TE signal, where the interval threshold is slightly greater than 1/k when the TE frequency is k. For example, when the TE frequency is 360Hz, the interval threshold is 3 ms.

And secondly, in response to the detection of the Multiple-TE signal and the time interval between the Multiple-TE signal and the forward adjacent Multiple-TE signal is smaller than an interval threshold, updating the count value of the counter.

Optionally, the count value of the counter is updated, i.e. the current count value of the counter is + 1.

When the Multiple-TE signal is detected and the time interval between the Multiple-TE signal and the forward adjacent Multiple-TE signal is smaller than the interval threshold, it indicates that the currently detected Multiple-TE signal and the forward adjacent Multiple-TE signal are all output after the DDIC finishes scanning the frame of the same frame image, so as to add one to the count value of the counter.

Illustratively, as shown in fig. 6, the TE frequency is 360Hz, the interval threshold is 3ms, and the AP updates the count value of the counter to 1 when the Multiple-TE signal is first detected during the display of the image frame a; when the Multiple-TE signal is detected again, since the time interval between the detection of the Multiple-TE signal and the last Multiple-TE signal is 2.8ms, the time interval 2.8ms is less than the interval threshold 3ms, and therefore the AP updates the count value of the counter to 2. Similarly, during the display of the image frame B, the count value of the counter is updated from 1 to 4.

And thirdly, in response to the detection of the Multiple-TE signal and the time interval between the detected Multiple-TE signal and the forward adjacent Multiple-TE signal is larger than an interval threshold, determining the count value of the counter as the historical number, and setting the count value of the counter to be 1.

When a Multiple-TE signal is detected, and the time interval between the Multiple-TE signal and a forward adjacent Multiple-TE signal is greater than an interval threshold, it indicates that the currently detected Multiple-TE signal and the forward adjacent Multiple-TE signal are output after completing frame scanning of an adjacent frame image by the DDIC, so that the AP determines the current count value of the counter as the historical number corresponding to the current frame image, and sets the count value of the counter to 1, so as to count the historical number of the Multiple-TE signal in the next frame image display process.

Illustratively, as shown in fig. 6, when the TE frequency is 360Hz and the interval threshold is 3ms, and in the case that the count value of the counter is 2 during the display of the image frame a, when the Multiple-TE signal is detected again, since the time interval with the last Multiple-TE signal is greater than the time threshold, the AP determines the history number corresponding to the image frame a to be 2 and sets the count value of the counter to be 1; similarly, in the display process of the image frame B, in the case where the count value of the counter is 4, when the Multiple-TE signal is detected again, since the time interval with the last Multiple-TE signal is greater than the time threshold, the AP determines the history number corresponding to the image frame B to be 4 and sets the count value of the counter to 1.

Step 302B, determining a historical refresh frequency of the DDIC based on the historical number.

In a possible implementation manner, the terminal is preset with a corresponding relationship between the number of TE signals and the refresh frequency of the DDIC, and accordingly, after determining the historical number, the AP determines the historical refresh frequency of the DDIC corresponding to the historical number based on the corresponding relationship.

Illustratively, the corresponding relationship between the number of TE signals and the refresh frequency of the DDIC is shown in table one.

Watch 1

Number of TE signals	Refresh frequency of DDIC
		1	72Hz
2	60Hz
		3	51Hz
4	45Hz

With reference to the corresponding relationship shown in table one, as shown in fig. 6, the AP determines that the historical refresh frequency of the DDIC in the display process of the image frame a is 60Hz, and determines that the historical refresh frequency of the DDIC in the display process of the image frame B is 45 Hz.

It should be noted that, in other possible embodiments, the AP may also monitor the refresh frequency of the DDIC in other manners, which is not limited in this embodiment.

In a possible scenario, when the reference Frame rate during the operation of the foreground application is 60FPS (Frame Per Second), the refresh frequency of the DDIC should be designed to be Frame-stable with 60Hz as the target refresh frequency, that is, the target refresh frequency is matched with the reference Frame rate during the operation of the foreground application. The matching between the target refresh frequency and the reference frame rate means that a difference between the target refresh frequency and the reference frame rate is smaller than a threshold (for example, 5FPS), optionally, the target refresh frequency is equal to the reference frame rate, or the target refresh frequency is slightly larger than the reference frame rate, or the target refresh frequency is slightly smaller than the reference frame rate.

Wherein when the AP is ready for image data on time (i.e., at a frequency of 60Hz) or delayed from being ready (i.e., less than 60Hz), the DDIC may wait appropriately to ensure that the refresh frequency of the DDIC remains within a range no greater than the target refresh frequency (e.g., 45Hz to 60Hz) in most scenarios.

When the AP prepares the image data in advance (i.e. when there is an acceleration demand), in order to avoid the direct jump of the refresh frequency of the DDIC from the low refresh frequency to the high refresh frequency (i.e. from the refresh frequency lower than 60Hz to the refresh frequency higher than 60Hz), in the present application, the AP performs the display sending delay operation.

Since the refresh frequency hopping occurs under the condition that the historical refresh frequency is low and the current refresh frequency is high, in a possible implementation manner, after the AP acquires the historical refresh frequency, it is detected whether the historical refresh frequency is less than the target refresh frequency. And if the historical refresh frequency corresponding to at least one frame of image is less than the target refresh frequency, determining that the display sending delay condition is met, and further performing display sending delay operation on the (m + 1) th frame of image data.

And if the historical refresh frequency corresponding to each frame of image is greater than or equal to the target refresh frequency, determining that the display sending delay condition is not met, and transmitting the image data of the (m + 1) th frame to the DDIC according to the conventional display sending logic (namely, when the rising edge of the Multiple-TE signal is detected, the AP transmits the image data of the (m + 1) th frame to the DDIC).

Since the reference frame rates corresponding to different applications are different, different target refresh frequencies need to be set for different applications. In one possible embodiment, before image data transmission, the AP determines a reference frame rate of a foreground application, thereby setting a presentation delay condition based on the reference frame rate.

In an illustrative example, when the foreground application is a game application and the reference frame rate of the game application is 60FPS, the AP sets the presentation delay condition as: in the history refresh frequency corresponding to the latest 2 frames of images, the history refresh frequency is less than 60 Hz. As long as the historical refresh frequency corresponding to the image is less than 60Hz, the AP performs the display sending delay operation; if the historical refresh frequency corresponding to the two frames of images is not less than 60Hz, the AP does not need to carry out display delay operation.

Regarding the specific manner of performing the rendering delay operation on the image data, in a possible embodiment, when the history refresh frequency satisfies the rendering delay condition and the image data of the (m + 1) th frame is completely prepared, the AP skips the TE signal to implement the rendering delay. The following description will be made using exemplary embodiments.

Referring to fig. 7, a flowchart of an image data transmission method according to another exemplary embodiment of the present application is shown. The method comprises the following steps:

step 701, transmitting the m-th frame image data to the DDIC, wherein m is a positive integer.

Step 702, determining the historical refresh frequency of the DDIC when displaying the m-n to m-1 frame images, wherein n is an integer less than m and greater than or equal to 2.

The embodiments of steps 701 to 702 can refer to the above embodiments, and this embodiment is not described herein again.

Illustratively, as shown in FIG. 8, after the AP transmits image data for image frame C to the DDIC, it is determined that the historical refresh frequency of DDIC is 60Hz when image frame A is displayed and 45Hz when image frame B is displayed.

And 703, in response to the fact that the historical refresh frequency corresponding to at least one frame of image is smaller than the target refresh frequency, when the m +1 th frame of image data is prepared, determining the real-time continuous number of Multiple-TE signals in the display process of the m frame of image.

When the delayed display sending condition is met, the AP detects the real-time continuous number of Multiple-TE signals generated by the DDIC before the m +1 frame image data is transmitted to the DDIC (the m +1 frame image data is prepared), and further determines whether the display sending delay operation is needed.

In a possible implementation manner, the AP determines the real-time duration number of the Multiple-TE signal during the display of the mth frame image by acquiring the real-time count value of the counter, so as to determine the output position of the current Multiple-TE signal. The real-time continuous number is based on the real-time count value of the counter and the image frame corresponding to the real-time count value, wherein the update manner of the count value of the counter may refer to the above embodiments, and this embodiment is not described herein again.

Optionally, if the image frame corresponding to the real-time count value of the counter is an mth frame image, the real-time continuous number of Multiple-TE signals in the display process of the mth frame image is the real-time count value of the counter; and if the image frame corresponding to the real-time counting value of the counter is the m-1 frame image, the real-time continuous number of the Multiple-TE signals in the display process of the m frame image is 0.

Illustratively, as shown in fig. 8, when the image data of the image frame D is prepared, the DDIC performs frame scanning on the image frame C, and the image frame corresponding to the real-time count value of the counter is the image frame B instead of the image frame C, so that the real-time duration number of the Multiple-TE signal during the display of the image frame C is 0, that is, the image data of the image frame D is prepared before the DDIC outputs the Multiple-TE signal.

Step 704, in response to the real-time continuous number being smaller than the number threshold, performing a TE signal skipping operation, where the number threshold is set based on the target refresh frequency.

In order to avoid the refresh frequency from jumping to be higher than the target refresh frequency, the AP sets a number threshold value based on the target refresh frequency and detects whether the real-time continuous number is smaller than the number threshold value. If the real-time continuous number is smaller than the number threshold, the m +1 th frame of image is prepared in advance, and if the image data transmission is directly carried out based on the next Multiple-TE signal, the refreshing frequency is greatly jumped, so that the AP skips at least one Multiple-TE signal, and the effect of delaying display is achieved.

Optionally, when the DDIC scans an image at the target refresh frequency, after the image scanning is completed, the number of Multiple-TE signals generated before scanning the next frame of image is the number threshold-1, and correspondingly, the number of the Multiple-TE signals skipped when the TE signal skipping operation is performed is the difference between the number threshold and the real-time continuous number, that is, after the TE signal skipping operation is completed, the refresh frequency of the DDIC in the display process of the mth frame of image is the target refresh frequency.

In one possible implementation, in response to the real-time duration number being less than the number threshold, the AP determines a target skip number for the Multiple-TE signal based on a difference between the number threshold and the real-time duration number; when receiving the Multiple-TE signal output by the DDIC and the real-time skipping number does not reach the target skipping number, the AP skips the Multiple-TE signal, namely, image data is not transmitted to the DDIC according to the Multiple-TE signal. Further, the AP updates the real-time skip number, i.e. adds one to the real-time skip number, so as to compare the updated real-time skip number with the target skip number when the Multiple-TE signal is detected again.

Optionally, if the number of the real-time continuous images is not less than the number threshold, it indicates that the m +1 th frame of image is not prepared in advance, and image data transmission is directly performed based on the next Multiple-TE signal, and therefore large-amplitude jump of the refresh frequency is not caused.

Illustratively, as shown in fig. 8, when the target refresh frequency is 60Hz, the AP determines that the number threshold is 1 (which may be determined by looking up the correspondence between the refresh frequency and the number threshold). Since the number of real-time continuous Multiple-TE signals in the display process of the image frame C is 0, the AP skips the current Multiple-TE signal and transmits the m +1 th frame of image data to the DDIC when the rising edge of the next Multiple-TE signal is detected (i.e., transmits the m +1 th frame of image data to the DDIC with a delay of 2.8 ms).

Step 705, in response to the TE signal skip operation being completed, transmits the m +1 th frame image data to the DDIC.

Optionally, after the TE signal skipping operation is completed, the AP transmits image data of the (m + 1) th frame to the DDIC when detecting a next Multiple-TE signal, so that the refresh frequency of the DDIC in the display process of the image of the (m) th frame is the target refresh frequency.

Illustratively, as shown in fig. 8, after introducing the presentation delay mechanism, the AP skips the first Multiple-TE signal and transmits the m +1 th image data to the DDIC when detecting the rising edge of the second Multiple-TE signal. Through the TE signal skipping operation, the change situation of the DDIC refresh frequency is changed from 60Hz → 45Hz → 72Hz (without introducing the presentation delay mechanism) to 60Hz → 45Hz → 60Hz (introducing the presentation delay mechanism), and the jump of the refresh frequency is avoided. In addition, after the TE signal skips the operation, since the display sending interval between the current frame and the next frame is shortened, the refresh frequency corresponding to the display sending of the next frame is increased, thereby reducing the occurrence frequency of low refresh frequencies such as 45Hz and the like, and further improving the stability of the refresh frequency.

In another possible implementation, since data transmission is performed between the AP and the DDIC through the MIPI, when the history refresh frequency satisfies the display delay condition and the provision of the image data of the (m + 1) th frame is completed, the AP may implement the display delay by blocking the MIPI. The following description will be made using exemplary embodiments.

Referring to fig. 9, a flowchart of an image data transmission method according to another exemplary embodiment of the present application is shown. The method comprises the following steps:

step 901, transmitting the m-th frame image data to the DDIC, m being a positive integer.

Step 902, a first timer is started, wherein the MIPI is in a pass-through state within the timer duration of the first timer.

In a possible implementation, after transmitting the image data to the DDIC, the AP starts the first timer, and ensures that the MIPI is in the on state within the time length of the timer of the first timer, so that the AP can transmit instructions other than the image data to the DDIC through the MIPI during the frame scanning process, where the first timer may be started by the AP by invoking a timing thread.

Illustratively, as shown in fig. 10, the AP starts a first timer after transmitting image data for image frame C to the DDIC.

Step 903, determining the historical refresh frequency of the DDIC when displaying the m-n to m-1 frame images, wherein n is an integer less than m and greater than or equal to 2.

Illustratively, as shown in FIG. 10, after the AP transmits image data for image frame C to the DDIC, it is determined that the historical refresh frequency of DDIC is 60Hz when image frame A is displayed and 45Hz when image frame B is displayed.

And step 904, in response to the fact that the historical refresh frequency corresponding to at least one frame of image is less than the target refresh frequency, starting a second timer when the timer duration of the first timer is reached, and setting the MIPI to be in a blocking state within the timer duration of the second timer.

In this embodiment, when the first timer reaches the timer duration, the AP sets the MIPI from the access state to the blocking state, and starts the second timer, so as to ensure that the MIPI maintains the blocking state within the timer duration of the second timer. Since the MIPI is in the blocking state, the TE signal cannot be detected during the second timer period, and therefore the AP cannot transmit the image data of the (m + 1) th frame to the DDIC during the second timer period, thereby achieving the effect of delayed display.

Wherein the timer duration of the first timer and the timer duration of the second timer are set based on the target refresh frequency. In a possible implementation manner, when the target refresh frequency is i and the highest refresh frequency required by the DDIC in the foreground application running process is j, the timer duration of the first timer is less than 1/j, and the sum of the timer durations of the first timer and the second timer is greater than 1/j and less than 1/i, so that the MIPI enters the blocking state before the rising edge of the next TE signal, and the refresh frequency of the DDIC is the target refresh frequency in the mth frame image display process.

Illustratively, as shown in fig. 10, when the target refresh frequency of the foreground application running is 60Hz, and the highest refresh frequency required by the DDIC during the foreground application running is 72Hz, the AP sets the timer duration of the first timer to 13ms (less than 1000 ÷ 72 ═ 13.9ms), and sets the timer duration of the second timer to 2ms (the sum of the timer duration of the first timer and the timer duration of the second timer is 15ms, and 15ms is less than 1000 ÷ 60 ÷ 16.7 ms). During the second timer period, the AP cannot transmit image data for image frame D to the DDIC during this period, since the MIPI is in the blocking state.

Step 905, in response to reaching the timer duration of the second timer, sets the MIPI to the on state and transmits the image data of the (m + 1) th frame to the DDIC.

Optionally, when the duration of the timer of the second timer is reached, the AP resets the MIPI to the access state, and transmits image data of the (m + 1) th frame to the DDIC when a next TE signal is detected, so that the refresh frequency of the DDIC in the display process of the image of the mth frame is the target refresh frequency.

Illustratively, as shown in fig. 10, after the MIPI is restored to the on state after the timer duration of the second timer is reached, the AP transmits the image data of the image frame D to the DDIC when detecting the next TE signal output by the DDIC.

It should be noted that, the foregoing embodiment only takes two delayed rendering manners of skipping the TE signal and blocking the MIPI as an example, and in other possible implementations, the AP may delay the rendering timing by other manners, and the embodiment of the present application does not limit the specific delayed rendering manner.

Referring to fig. 11, a block diagram of an image data transmission device according to an embodiment of the present application is shown. The device includes:

a transmission module 1101 for transmitting the mth frame image data to the display driver chip DDIC, m being a positive integer;

a first determining module 1102, configured to determine a history refresh frequency of the DDIC when displaying m-n to m-1 th frame images, where n is an integer less than m and greater than or equal to 2;

a delay module 1103, configured to perform, in response to that the history refresh frequency satisfies a display delay condition, a display delay operation on the m +1 th frame of image data, where the display delay operation is used to delay transmission of the m +1 th frame of image data to the DDIC;

the transmission module 1101 is further configured to transmit the image data of the (m + 1) th frame to the DDIC in response to completion of the display delay operation.

Optionally, the AP is configured to perform image data transmission based on a rising edge of a Multiple tear effect Multiple-TE signal, where the Multiple-TE signal is output by the DDIC;

the first determining module 1102 includes:

an obtaining unit, configured to obtain, for the m-n to m-1 th frame images, a historical number of the Multiple-TE signals output by the DDIC in a display process of each frame image;

a determining unit, configured to determine the history refresh frequency of each frame image of the m-n th to m-1 th frame images by the DDIC based on the history number.

Optionally, the obtaining unit is configured to:

in response to the detection of the Multiple-TE signal and the time interval between the detected Multiple-TE signal and the forward adjacent Multiple-TE signal is smaller than an interval threshold, updating the count value of a counter, wherein the counter is used for recording the number of the Multiple-TE signals output by the DDIC in the display process of each frame of image;

in response to detecting the Multiple-TE signal and a time interval between the forward adjacent Multiple-TE signal and the interval threshold being greater than the interval threshold, determining the count value of the counter as the historical number and setting the count value of the counter to 1.

Optionally, the determining unit is configured to:

and determining the historical refresh frequency from the corresponding relation between the TE signal number and the refresh frequency based on the historical number.

Optionally, the frequency of the Multiple-TE signal output by the DDIC is a TE frequency, and the TE frequency is the same as a light-emitting EM frequency of the display screen, or the TE frequency is an integer Multiple of the EM frequency.

Optionally, the delay module 1103 is configured to:

and in response to the fact that the historical refreshing frequency corresponding to at least one frame of image is smaller than a target refreshing frequency, determining that the display sending delay condition is met, and performing the display sending delay operation on the (m + 1) th frame of image data, wherein the target refreshing frequency is matched with a reference frame rate in the foreground application running process.

Optionally, the AP is configured to perform image data transmission based on a rising edge of a Multiple-TE signal, where the Multiple-TE signal is output by the DDIC;

the delay module 1103 includes:

a first delay unit, configured to determine a number of real-time durations of the Multiple-TE signal during display of an mth frame image in response to completion of preparation of the m +1 th frame image data; and responding to the real-time continuous number smaller than a number threshold value, and performing TE signal skipping operation, wherein the number threshold value is set based on the target refreshing frequency.

Optionally, the first delay unit is configured to:

in response to the real-time duration number being less than the number threshold, determining a target skip number for the Multiple-TE signal based on a difference between the number threshold and the real-time duration number;

in response to receiving the Multiple-TE signal output by the DDIC and the real-time skip number does not reach the target skip number, performing skip processing on the Multiple-TE signal;

updating the real-time skip number.

Optionally, data transmission is performed between the AP and the DDIC through a mobile industry processor interface MIPI;

the device further comprises:

the system comprises a timing module, a first timer and a second timer, wherein the MIPI is in a pass-through state within the timer duration of the first timer;

the delay module 1103 includes:

a second delay unit, configured to start a second timer in response to reaching the timer duration of the first timer, and set the MIPI to a blocking state within the timer duration of the second timer;

wherein timer durations of the first timer and the second timer are set based on the target refresh frequency.

Optionally, the target refresh frequency is i, the highest refresh frequency required by the DDIC in the foreground application running process is j, and j is greater than i;

the timer duration of the first timer and the second timer is less than 1/j;

the sum of the timer duration of the first timer and the timer duration of the second timer is greater than 1/j and less than 1/i.

Optionally, the apparatus further comprises:

a second determining module, configured to determine the reference frame rate of the foreground application;

and the setting module is used for setting the display delay condition based on the reference frame rate.

In summary, in the embodiment of the present application, by introducing the display sending delay mechanism, after the AP transmits the mth frame of image data to the DDIC, it is determined whether the display sending delay condition is satisfied based on the historical refresh frequency of the DDIC in the recent n frames of image display, and when the display sending delay condition is satisfied, after the display sending delay operation is performed on the m +1 th frame of image data, the m +1 th frame of image data is transmitted to the DDIC, so as to avoid the problem of flicker and jitter of the picture due to the jump of the DDIC refresh frequency caused by the fluctuation of the AP output frame rate, which is helpful for improving the stability of the DDIC refresh frequency in the image display process, and achieving the effect of improving the image display quality.

Referring to fig. 12, a block diagram of a terminal 1200 according to an exemplary embodiment of the present application is shown. The terminal 1200 may be a smart phone, a tablet computer, a notebook computer, or the like. The terminal 1200 in the present application may include one or more of the following components: processor 1210, memory 1220, display screen module 1230.

The processor 1210 may include one or more processing cores, and the processor 1210 may be an AP as described in the above embodiments. The processor 1210, using various interfaces and lines to connect various parts throughout the terminal 1200, performs various functions of the terminal 1200 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1220, and calling data stored in the memory 1220. Alternatively, the processor 1210 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1210 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Neural-Network Processing Unit (NPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is responsible for rendering and drawing the content to be displayed by the touch screen module 1230; the NPU is used for realizing an Artificial Intelligence (AI) function; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 1210, but may be implemented by a single chip.

The Memory 1220 may include a Random Access Memory (RAM) or a Read-Only Memory (ROM). Optionally, the memory 1220 includes a non-transitory computer-readable medium. The memory 1220 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 1220 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments of the present application, and the like; the storage data area may store data (such as audio data, a phonebook) created according to the use of the terminal 1200, and the like.

The display module 1230 is a display module for displaying images, and is generally disposed on the front panel of the terminal 1200. The display screen module 1230 may be designed as a full-screen, curved screen, odd-shaped screen, double-sided screen, or folding screen. The display screen module 1230 can be designed to be a combination of a full-screen and a curved-surface screen, or a combination of a special-shaped screen and a curved-surface screen, which is not limited in this embodiment.

In the embodiment of the present application, the display screen module 1230 includes a DDIC1231 and a display screen 1232 (panel). The display screen 1232 may be an OLED display screen, which may be a Low Temperature Polysilicon (LTPS) AMOLED display screen or a Low Temperature Polysilicon Oxide (LTPO) AMOLED display screen.

The DDIC1231 is used to drive the display screen 1232 for image display. In addition, the DDIC1231 is connected to the processor 1210 through an MIPI interface, and is configured to receive image data and instructions sent by the processor 1210.

In a possible implementation manner, the display screen module 1230 further has a touch function, and through the touch function, a user can use any suitable object such as a finger, a touch pen, and the like to perform a touch operation on the display screen module 1230.

In addition, those skilled in the art will appreciate that the configuration of terminal 1200 illustrated in the above-described figures is not meant to be limiting with respect to terminal 1200, and that terminals may include more or less components than those illustrated, or some components may be combined, or a different arrangement of components. For example, the terminal 1200 further includes a microphone, a speaker, a radio frequency circuit, an input unit, a sensor, an audio circuit, a Wireless Fidelity (WiFi) module, a power supply, a bluetooth module, and other components, which are not described herein again.

The embodiment of the present application further provides a computer-readable storage medium, which stores at least one instruction, where the at least one instruction is used for being executed by a processor to implement the image data transmission method according to the above embodiment.

Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in the embodiments of the present application may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An image data transmission method for an application processor AP, the method comprising:

transmitting the mth frame image data to a display driving chip DDIC, wherein m is a positive integer;

2. The method of claim 1, wherein the AP is configured to perform image data transmission based on a rising edge of a Multiple tear effect Multiple-TE signal, the Multiple-TE signal being output by the DDIC;

the determining the historical refresh frequency of the DDIC when displaying the m-n th to m-1 th frame images comprises:

acquiring the historical number of the Multiple-TE signals output by the DDIC in the display process of each frame of image for the m-n to m-1 frames of images;

and determining the historical refreshing frequency of the DDIC for each frame image from the m-n th frame image to the m-1 th frame image based on the historical number.

3. The method according to claim 2, wherein said obtaining the historical number of the Multiple-TE signals during each frame of image display comprises:

4. The method of claim 2, wherein the determining the historical refresh frequency of the DDIC based on the historical number comprises:

5. The method of claim 2, wherein the DDIC outputs a Multiple-TE signal at a TE frequency that is the same as an emission EM frequency of a display screen, or wherein the EM frequency is an integer Multiple of the TE frequency.

6. The method according to any one of claims 1 to 5, wherein the performing, in response to the history refresh frequency satisfying the presentation delay condition, a presentation delay operation on the m +1 th frame image data comprises:

7. The method of claim 6, wherein the AP is configured to perform image data transmission based on a rising edge of a Multiple-TE signal, the Multiple-TE signal being output by the DDIC;

the performing the display delay operation on the m +1 th frame of image data includes:

in response to the m +1 th frame of image data being prepared, determining the real-time continuous number of the Multiple-TE signal in the display process of the m frame of image;

and responding to the real-time continuous number smaller than a number threshold value, and performing TE signal skipping operation, wherein the number threshold value is set based on the target refreshing frequency.

8. The method of claim 7, wherein said performing a TE signal skip operation in response to said real-time duration number being less than a number threshold comprises:

updating the real-time skip number.

9. The method of claim 6, wherein the data transmission between the AP and the DDIC is performed through a Mobile Industry Processor Interface (MIPI);

after the transmitting of the m-th frame of image data to the DDIC, the method further comprises:

starting a first timer, wherein the MIPI is in a pass-through state within the timer duration of the first timer;

in response to reaching the timer duration of the first timer, starting a second timer and setting the MIPI to a blocked state within the timer duration of the second timer;

10. The method according to claim 9, wherein the target refresh frequency is i, the highest refresh frequency required by the DDIC during the foreground application running process is j, and j is greater than i;

therefore, the time length of the timer of the first timer is less than 1/j;

11. The method of claim 6, further comprising:

determining the reference frame rate of the foreground application;

setting the presentation delay condition based on the reference frame rate.

12. An image data transmission apparatus, characterized in that the apparatus comprises:

the transmission module is used for transmitting the image data of the mth frame to the display driving chip DDIC, wherein m is a positive integer;

13. The apparatus of claim 12, wherein the AP is configured to perform image data transmission based on a rising edge of a Multiple tear effect Multiple-TE signal, the Multiple-TE signal being output by the DDIC;

the first determining module includes:

14. The apparatus of claim 13, wherein the obtaining unit is configured to:

15. The apparatus of claim 13, wherein the determining unit is configured to:

16. The apparatus of claim 13, wherein the DDIC outputs a Multiple-TE signal at a TE frequency that is the same as an emission EM frequency of a display screen, or wherein the EM frequency is an integer Multiple of the TE frequency.

17. The apparatus of any of claims 12 to 16, wherein the delay module is configured to:

18. The apparatus of claim 17, wherein the AP is configured to perform image data transmission based on a rising edge of a multiplex e-TE signal, the multiplex-TE signal being output by the DDIC;

the delay module includes:

19. The apparatus of claim 18, wherein the first delay unit is configured to:

updating the real-time skip number.

20. The apparatus of claim 17, wherein data transmission between the AP and the DDIC is performed by a Mobile Industry Processor Interface (MIPI);

the device further comprises:

the delay module includes:

21. The apparatus according to claim 20, wherein the target refresh frequency is i, the highest refresh frequency required by the DDIC during the foreground application running is j, and j is greater than i;

the timer duration of the first timer and the second timer is less than 1/j;

22. The apparatus of claim 17, further comprising:

23. A terminal, characterized in that the terminal comprises an application processor AP, a display screen and a display driver circuit chip DDIC, the AP and the DDIC are connected through a MIPI (mobile industry processor interface), and the AP is configured to execute at least one instruction in a memory to implement the image data transmission method according to any one of claims 1 to 11.

24. A computer-readable storage medium storing at least one instruction for execution by a processor to implement the image data transmission method of any one of claims 1 to 11.