CN114265568A

CN114265568A - Display processing unit, method, acceleration unit and system on chip

Info

Publication number: CN114265568A
Application number: CN202111489018.9A
Authority: CN
Inventors: 程茂林
Original assignee: Pingtouge Shanghai Semiconductor Co Ltd
Current assignee: Hangzhou C Sky Microsystems Co Ltd
Priority date: 2021-12-07
Filing date: 2021-12-07
Publication date: 2022-04-01
Anticipated expiration: 2041-12-07
Also published as: CN114265568B

Abstract

The embodiment of the application provides a display processing unit, a method, an acceleration unit and a system on a chip, wherein the display processing unit comprises: the device comprises a processing unit, an interface unit and a storage unit; the display device comprises a processing unit, a storage unit and a display unit, wherein the processing unit is used for processing pixels in an image to be displayed according to a first clock signal to obtain pixel data and storing the pixel data into the storage unit, the first clock signal is started when the data volume of the data stored in the storage unit is smaller than a first data volume threshold value, the first clock signal is stopped when the data volume of the data stored in the storage unit is larger than a second data volume threshold value, and the first data volume threshold value is smaller than the second data volume threshold value; and the interface unit is used for reading the pixel data from the storage unit according to the second clock signal and performing time sequence superposition on the read pixel data to obtain a pixel data stream for displaying the image to be displayed. The scheme can reduce the power consumption of the display system.

Description

Display processing unit, method, acceleration unit and system on chip

Technical Field

The embodiment of the application relates to the technical field of chips, in particular to a display processing unit, a display processing method, an acceleration unit and a system on a chip.

Background

With the development of the performance of electronic devices, the requirement on the resolution of the display system in the electronic devices such as notebook computers, mobile phones, tablet computers, smart watches, etc. is higher and higher, and along with the higher and higher power consumption of the display system, the problem of high power consumption of the display system in the electronic devices needs to be solved urgently. The display system always starts from the upper left corner of the image during the pixel display and moves horizontally forward, while the scanning point also moves downward at a slower rate. When the scanning point reaches the right edge of the image, the scanning point quickly returns to the left side, the scanning of the next line is restarted, and the process of returning the scanning point between the lines is called line blanking. After a frame of image is scanned by a scanning point, the scanning point needs to return to the upper left corner of the image from the lower right corner of the image, the scanning of a new frame of image is started, and the process that the scanning point returns to the upper left corner from the lower right corner of the image is called frame blanking.

At present, frame blanking is controlled in the process of pixel display, so that the power consumption of a display system in a frame blanking area is reduced, and the power consumption of the display system in electronic equipment is further reduced.

However, since the occupation ratio of the frame blanking in the total time of one frame image scanning is small, the effect of reducing the power consumption of the display system in the electronic device by controlling the frame blanking is poor.

Disclosure of Invention

In view of the above, embodiments of the present application provide a display processing scheme to at least partially solve the above problems.

According to a first aspect of embodiments of the present application, there is provided a display processing unit including: the device comprises a processing unit, an interface unit and a storage unit; the processing unit is used for processing pixels in an image to be displayed according to a first clock signal to obtain pixel data and storing the pixel data into the storage unit, wherein the first clock signal is turned on when the data volume of the data stored in the storage unit is smaller than a first data volume threshold value, the first clock signal is turned off when the data volume of the data stored in the storage unit is larger than a second data volume threshold value, and the first data volume threshold value is smaller than the second data volume threshold value; the interface unit is used for reading the pixel data from the storage unit according to a second clock signal, and performing time sequence superposition on the read pixel data to obtain a pixel data stream for displaying the image.

According to a second aspect of embodiments of the present application, there is provided a display processing method including: processing pixels in an image to be displayed according to a first clock signal to obtain pixel data, and storing the pixel data in the storage unit, wherein the first clock signal is turned on when the data volume of the data stored in the storage unit is smaller than a first data volume threshold, the first clock signal is turned off when the data volume of the data stored in the storage unit is larger than a second data volume threshold, and the first data volume threshold is smaller than the second data volume threshold; and reading the pixel data from the storage unit according to a second clock signal, and performing time sequence superposition on the read pixel data to obtain a pixel data stream for displaying the image to be displayed.

According to a third aspect of embodiments of the present application, there is provided an acceleration unit including: the display processing unit according to the first aspect; and the controller is used for controlling the display processing unit to work.

According to a fourth aspect of embodiments of the present application, there is provided an electronic apparatus, including: the acceleration unit according to the third aspect described above; and the scheduling unit is used for scheduling the accelerating unit to execute the image display task.

According to a fifth aspect of embodiments of the present application, there is provided a system on chip comprising the acceleration unit according to the third aspect described above.

According to the image display processing scheme provided by the embodiment of the application, the interface unit reads the pixel data from the storage unit according to the second clock signal, when the data volume of the pixel data in the storage unit is smaller than the first data volume threshold value, the first clock signal is turned on, the processing unit processes the pixel in the image to be displayed according to the first clock signal to obtain the pixel data, the pixel data is stored in the storage unit, when the data volume of the pixel data in the storage unit is larger than the second data volume threshold value, the first clock signal is turned off, and the processing unit suspends the processing of the pixel in the image to be displayed until the data volume of the pixel data in the storage unit is smaller than the first data volume threshold value. The processing unit is used for processing an image to be displayed, the interface unit is used for outputting data, the processing unit and the interface unit work according to different clock signals, rectification is carried out through the storage unit, the speed of generating pixel data by the processing unit is greater than the speed of outputting the pixel data by the interface unit, the first clock signal is turned off after the processing unit generates a certain amount of pixel data, a circuit used for image processing in the processing unit is enabled to pause, and compared with the situation that the circuit is only enabled to stop working in a frame blanking area, the power consumption of a display system can be effectively reduced.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings can be obtained by those skilled in the art according to the drawings.

FIG. 1 is a block diagram of an electronic device to which one embodiment of the present application is applied;

fig. 2 is an internal structural diagram of a scheduling unit and an acceleration unit inside an electronic device according to an embodiment of the present application;

FIG. 3 is an internal block diagram of a display processing unit according to an embodiment of the present application;

fig. 4 is an internal structural view of a display processing unit according to another embodiment of the present application;

fig. 5 is a flowchart of a display processing method according to an embodiment of the present application.

Detailed Description

The present application is described below based on examples, but the present application is not limited to only these examples. In the following detailed description of the present application, certain specific details are set forth in detail. It will be apparent to one skilled in the art that the present application may be practiced without these specific details. Well-known methods, procedures, and procedures have not been described in detail so as not to obscure the present application. The figures are not necessarily drawn to scale.

The following terms are used herein.

Line blanking: line Blanking, also known as Horizontal Blanking (HBlank), is a technique for making the return process from line to line non-displayed during image display to avoid the return line from line to line. In the process of image display, for each pixel point on the image, scanning points scan one by one from left to right, when the scanning points reach the right edge of the image, the scanning points quickly return to the left side to start scanning of the next line, and the return process of the scanning points between the lines is called line blanking.

Frame blanking: frame Blanking is also called Vertical Blanking (VBlank), which is a technique for preventing a return line from the lower right to the upper left of an image from being displayed when the image is displayed. In the process of image display, after a scanning point scans a frame of image, the scanning point returns to the upper left corner of the image from the lower right corner of the image to start scanning of a new frame of image, and the process of returning the scanning point to the upper left corner from the lower right corner of the image is called frame blanking.

First-in first-out memory: the FIFO is a memory that employs a FIFO queue access rule, and the FIFO is an access rule of the queue, and an element that is First added to the queue is First taken Out, and an element that is added to the queue is last taken Out.

Clock signals: clock signals (Clock signals) are the basis of sequential logic, which is a Signal quantity having a fixed period and being independent of operation, and are usually used in synchronous circuits to play the role of a timer to ensure that the related electronic components are operated synchronously.

Clock period: the clock signal has a fixed frequency and the clock period is equal to the inverse of the clock frequency.

An acceleration unit: in the case where the conventional processing unit is not efficient in some special-purpose fields (for example, displaying an image, processing an image, and the like), the processing unit designed to increase the data processing speed in these special-purpose fields is, in the embodiment of the present disclosure, mainly a special processing unit designed to increase the image display processing speed, release the computing power of the CPU, and reduce the power consumption of the CPU.

A scheduling unit: the processing unit for scheduling the acceleration unit and allocating the instruction sequence to be executed to the acceleration unit can adopt various forms of processors (CPUs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) and the like.

Electronic device

Fig. 1 shows a block diagram of an internal structure of an electronic device 100 according to an embodiment of the present application, and as shown in fig. 1, the electronic device 100 includes a memory 110, a scheduling unit cluster 140, and an acceleration unit cluster 150, which are connected by a bus. The cluster of scheduling units 140 includes a plurality of scheduling units 120. The acceleration unit cluster 150 includes a plurality of acceleration units 130. The acceleration unit 130 is a special processing unit designed to accelerate image processing and image display in the embodiment of the present application, and may be embodied as a processing unit specially designed for image display, a Graphics Processing Unit (GPU), an image display unit (DPU), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), and the like. The scheduling unit is a unit that schedules the acceleration units and allocates to each acceleration unit a sequence of instructions to be executed, and may take various forms such as a processor (CPU), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), and the like.

In the traditional architecture design of the processing unit, a control unit and a storage unit occupy a large part of space in the architecture, and the space occupied by a computing unit is insufficient, so that the traditional architecture design is very effective in logic control and is not efficient in large-scale parallel computing. Therefore, various special acceleration units have been developed to perform more efficient processing for increasing the operation speed for calculations of different functions and different fields. The acceleration unit is a processing unit dedicated to accelerating the image display processing speed, and adopts a structure that an image processing part and an interface part are separated, and is used for sending the processed image to an image display device for displaying. When an image is displayed, when the data volume of data stored in a storage unit is smaller than a first data volume threshold value, a first clock signal is started, pixels in the image to be displayed are processed according to the first clock signal, pixel data corresponding to the pixels are obtained, the obtained pixel data are stored in the storage unit, when the data volume of the data stored in the storage unit is larger than a second data volume threshold value, the first clock signal is closed, the image to be displayed is stopped from being processed, meanwhile, the pixel data are continuously read from the storage unit according to a second clock signal, time sequence superposition is carried out on the pixel data, and a pixel data stream used for displaying the image to be displayed is obtained. The image processing part and the interface part work according to different clock signals, and after a certain amount of pixel data is stored in the storage unit, the first clock signal is closed, so that the circuit for image processing stops working, the power consumption of the circuit for image processing can be saved, and the power consumption of a display system in the electronic equipment is further reduced.

The acceleration unit 130 is to accept the schedule of the scheduling unit 120. As shown in fig. 1, the memory 110 stores the image to be displayed, which is delivered to an acceleration unit 130 by a scheduling unit 120. The scheduling unit 120 may send an address in the memory 110 that requires the image to be displayed to the acceleration unit 130 in the form of an instruction. When the acceleration unit 130 displays an image, the memory 110 directly addresses the image to be displayed, and the image to be displayed is temporarily stored in the on-chip memory, so that the acceleration unit 130 can display the image to be displayed. The embodiment of the present application mainly focuses on the process of displaying the image to be displayed by the acceleration unit 120, and the process of displaying the image will be described in detail later.

Internal structure of dispatching unit and accelerating unit

How the scheduling unit 120 schedules the acceleration unit 130 to operate will be described in detail below with reference to the internal structures of the scheduling unit 120 and the acceleration unit 130 of fig. 2.

As shown in fig. 2, the scheduling unit 120 includes a plurality of processor cores 121 and a cache 122 shared by the plurality of processor cores 121. Each processor core 121 includes an instruction fetch unit 123, an instruction decode unit 124, an instruction issue unit 125, an instruction execution unit 126, and a memory control unit 127.

The instruction fetch unit 123 is configured to carry an instruction to be executed from the memory 110 to an instruction register (which may be one of the registers used for storing instructions in the register file 128 shown in fig. 2) through the storage control unit 127, and receive a next instruction fetch address or calculate the next instruction fetch address according to an instruction fetch algorithm, which includes: the address is incremented or decremented according to the instruction length.

After fetching the instruction, dispatch unit 120 enters an instruction decode stage in which instruction decode unit 124 decodes the fetched instruction according to a predetermined instruction format to obtain operand fetch information required by the fetched instruction in preparation for operation by instruction execution unit 126. The operand fetch information points, for example, to an immediate, register, or other software/hardware capable of providing source operands.

Instruction issue unit 125 is located between instruction decode unit 124 and instruction execution unit 126 for scheduling and control of instructions to efficiently allocate individual instructions to different instruction execution units 126, enabling parallel operation of multiple instructions.

After instruction issue unit 125 issues an instruction to instruction execution unit 126, instruction execution unit 126 begins executing the instruction. But if the instruction execution unit 126 determines that the instruction should be executed by an acceleration unit, it is forwarded to the corresponding acceleration unit for execution. For example, if the instruction is an instruction for image display, the instruction execution unit 126 no longer executes the instruction, but sends the instruction to the acceleration unit 130 via the bus to be executed by the acceleration unit 130. The scheduling unit 130 schedules the acceleration unit 130 to perform display processing of an image to be displayed.

The acceleration unit 130 includes a controller 131, a display processing unit 132, and a clock unit 137. The controller 131 receives the image to be displayed from the scheduling unit 120 and transmits the image to be displayed to the display processing unit 132. The controller 131 generates control instructions to the display processing unit 132 and the clock unit 137 to control the operations of the display processing unit 132 and the clock unit 137, respectively.

The clock unit 137 starts the first clock signal according to the data amount of the data stored in the storage unit of the display processing unit 132 when the data amount of the data stored in the storage unit is smaller than the first data amount threshold, and inputs the first clock signal to the display processing unit 132, so that the display processing unit 132 processes the image to be displayed according to the first clock signal, and closes the first clock signal when the data amount of the data stored in the storage unit is larger than the second data amount threshold.

The display processing unit 132 processes an image in an image to be displayed based on the first clock signal from the clock unit 137 to obtain pixel data, stores the pixel data in the storage unit, reads the pixel data from the storage unit according to the second clock signal, and performs time-series superposition on the read pixel data to obtain a pixel data stream for displaying the image to be displayed. The display processing unit 132 inputs the pixel data stream to the controller 131, and the controller 131 returns the pixel data stream to the scheduling unit 120.

The clock unit 137 turns on or off the first clock signal according to the data amount of the data stored in the storage unit included in the display processing unit 132, when the data amount of the data stored in the storage unit is smaller than the first data amount threshold, the clock unit 137 starts the first clock signal, the display processing unit 132 processes the pixels in the image according to the first clock signal to obtain the pixel data corresponding to the pixels, and stores the obtained pixel data in the storage unit, when the data amount of the data stored in the storage unit is larger than the second data amount threshold, the clock unit 137 turns off the first clock signal, the display processing unit 132 stops processing the image to be displayed, at the same time, the display processing unit 132 reads pixel data from the memory unit according to the second clock signal, and performing time sequence superposition on the pixel data to obtain a pixel data stream for displaying the image to be displayed. Since the portion for image processing and the portion for time-series superimposition in the display processing unit 132 operate according to different clock signals, the portion for image processing stops operating after the storage unit stores a certain amount of pixel data, power consumption of a circuit for image processing is saved, and power consumption of a display system in an electronic device can be reduced.

In one possible implementation, the clock unit 137 may be implemented by a Q-channel. The clock unit 137 may generate a second clock signal with a frequency smaller than that of the first clock signal, in addition to the first clock signal, and certainly, the second clock signal may also be generated by other clock sources, which is not limited in this embodiment of the present application.

System on chip

The embodiment of the present application further provides a system on chip, which includes the acceleration unit 130 in any of the above embodiments.

Display processing unit

Fig. 3 is an internal structural diagram of a display processing unit according to an embodiment of the present application. As shown in fig. 3, the display processing unit 132 includes a processing unit 133, an interface unit 134, and a storage unit 135. In the process of displaying an image, the processing unit 133 processes the image to be displayed based on the first clock signal, when the data amount of the data stored in the storage unit 135 is smaller than the first data amount threshold, the first clock signal is turned on, the processing unit 133 processes the pixel in the image to be displayed according to the first clock signal, obtains pixel data, and stores the obtained pixel data in the storage unit 135. When the data amount of the data stored in the storage unit 135 is larger than the second data amount threshold, the second data amount threshold is larger than the first data amount threshold, the first clock signal is turned off, and the processing unit 133 suspends the processing of the pixels in the image to be displayed. The interface unit 134 reads pixel data from the storage unit 135 according to the second clock signal, and performs time-series superposition on the read pixel data to obtain a pixel data stream for displaying an image to be displayed.

The second clock signal is determined by the refresh frequency of the external display device, and has a fixed frequency on the premise that the refresh frequency of the display device is fixed. The refresh frequency of the display device indicates the number of frames of the refreshed image per second, and the pixels are scanned one by one in the image display process, each frame of image comprises a plurality of pixels, and each clock period scans one pixel, so that the frequency of the second clock signal is greater than the refresh frequency of the display device. The interface unit 134 reads pixel data from the storage unit 135 according to the second clock signal and transmits the read pixel data to the display device, so that the display device displays each pixel in an image to be displayed one by one according to the received pixel data.

Since the frequency of the second clock signal is fixed, the interface unit 134 continuously reads the pixel data from the storage unit 135 according to the second clock signal, and in order to ensure that the interface unit 134 can acquire the pixel data according to the time sequence to ensure normal display of the image to be displayed, it is necessary to ensure that the pixel data stored in the storage unit 135 for the interface unit 134 to read is stored, and therefore, it is necessary to determine the frequency of the first clock signal according to the frequency of the second clock signal, and it is necessary to ensure that the speed of generating the pixel data according to the first clock signal by the processing unit 133 is greater than the speed of reading the pixel data by the interface unit 134.

The storage unit 135 is used for storing pixel data, and since the data capacity of the storage unit 135 is fixed and the speed of generating the pixel data by the processing unit 133 is faster than the speed of reading the pixel data by the interface unit 134, if the processing unit 133 continuously operates, the data amount of the pixel data stored in the storage unit 135 is increased. In order to ensure that the interface unit 134 can read necessary pixel data from the storage unit 135 and ensure that the pixel data in the storage unit 135 does not overflow, a first data amount threshold and a second data amount threshold are set for the storage unit 135 such that the first data amount threshold is smaller than the second data amount threshold. When the data amount of the pixel data in the storage unit 135 is smaller than the first data amount threshold, the first clock signal is turned on, and the processing unit 133 processes the pixel in the image to be displayed according to the first clock signal, generates pixel data, and stores the pixel data in the storage unit 135. As the processing unit 133 stores the pixel data into the storage unit 135, the data amount of the pixel data in the storage unit 135 is continuously increased, and when the data amount of the pixel data in the storage unit 135 is greater than the second data amount threshold, the first clock signal is turned off, and the processing unit 133 suspends the processing of the pixel in the image to be displayed, and does not store the pixel data into the storage unit 135 any more. The interface unit 134 continuously reads the pixel data from the storage unit 135 according to the second clock signal, after the processing unit 133 turns off the first clock signal, the data amount of the pixel data in the storage unit 135 gradually decreases until the data amount of the pixel data in the storage unit 135 is smaller than the first data amount threshold again, the first clock signal is turned on again, and the processing unit 133 processes the pixel data in the image according to the first clock signal again, generates the pixel data, and stores the pixel data in the storage unit 135.

When playing video by the display device, since the video data is composed of a plurality of frames of images, the processing unit 133 continuously processes the plurality of frames of images included in the video data, and therefore, the storage unit 135 may store therein pixel data corresponding to different frames of images, but the interface unit 134 reads the pixel data stored in the storage unit 135 in order to display the images included in the video data on a frame-by-frame basis.

The image received by the processing unit 133 is an image rendered by the GPU, and the processing of the image by the processing unit 133 includes scaling, rotating, denoising, and the like of the image.

In this embodiment, the interface unit 134 reads the pixel data from the storage unit 135 according to the second clock signal, when the data amount of the pixel data in the storage unit 135 is smaller than the first data amount threshold, the first clock signal is turned on, the processing unit 133 processes the pixel data in the image to be displayed according to the first clock signal to obtain the pixel data, and stores the pixel data in the storage unit 135, when the data amount of the pixel data in the storage unit 135 is larger than the second data amount threshold, the first clock signal is turned off, and the processing unit 133 suspends the processing of the pixel data in the image to be displayed until the data amount of the pixel data in the storage unit 135 is smaller than the first data amount threshold. The processing unit 133 is used for image processing, the interface unit 134 is used for data output, the processing unit 133 and the interface unit 134 operate according to different clock signals, rectification is performed through the storage unit 135, the speed of generating pixel data by the processing unit 133 is greater than the speed of outputting the pixel data by the interface unit 134, image processing is suspended after the processing unit 133 generates a certain amount of pixel data, a circuit used for image processing in the processing unit 133 is suspended, and compared with the situation that the circuit is stopped only in a frame blanking area, the power consumption of a display system can be effectively reduced.

In one example, the resolution of the display device is 1920 × 1080, the refresh rate of the display device is 60Hz, and the frame blanking region occupies about 4% of the total scanning time of one frame image, and if the frame blanking region is controlled, power consumption can be saved by about 4% compared with the case where the frame blanking region is not controlled. However, if the scheme of the embodiment of the present application is adopted, when the resolution of the display device is 1920 × 1080, the first clock signal is 445.5MHz, and the second clock signal is 148.5MHz, the time required for completing the processing of 1 frame image is 16.67ms, and the time required for completing the 1920 × 1080 operation by the processing unit 133 is 1920 × 1080 × 2.2(ns) ≈ 4.5ms, so that the time for the processing unit 133 to be in the off state is 16.67-4.5 ═ 12.17ms, and the time for the processing unit 133 to be in the off state accounts for 72% of the total time, so that the power consumption of the display processing unit 132 can be greatly reduced.

In a possible implementation manner, when processing the pixels in the image to be displayed, the processing unit 133 sequentially processes the pixels corresponding to the pixel data in the image to be displayed according to the sorting of the pixel data in the pixel data stream. When the image to be displayed is displayed, pixels included in the image to be displayed are scanned line by line from the upper left corner of the image to be displayed, and the interface unit 134 needs to sequentially overlap pixel data corresponding to each pixel according to the scanning sequence of the pixels in the image to be displayed to generate a pixel data stream, so that the display device can normally display the image to be displayed according to the pixel data stream. According to the ordering of the pixel data in the pixel data stream, the pixels in the image to be displayed are sequentially processed to obtain corresponding pixel data, that is, the order in which the processing unit 133 generates the pixel data is the same as the order in which the interface unit 134 reads the pixel data, so that the interface unit 134 can sequentially and timely read the required pixel data from the storage unit 135, the problems of image blockage, image tearing and the like are avoided, and the image to be displayed can be normally displayed.

When the display device scans the pixels in the image to be displayed line by line, the processing unit 133 processes the pixels in the ith line in the image to be displayed one by one according to the sequence from left to right, and after all the pixels included in the ith line are processed, processes the pixels in the (i + 1) th line in the image to be displayed one by one according to the sequence from left to right until all the pixels in each line in the image to be displayed are processed, and processes the pixels from the first line in the image to be displayed in the next frame. Correspondingly, the interface unit 134 sequentially reads the pixel data corresponding to each pixel in the ith row from the storage unit 135 for time sequence superposition according to the sequence of the ith row of pixels in the image to be displayed from the left to the right, sequentially reads the pixel data corresponding to each pixel in the (i + 1) th row from the storage unit 135 for time sequence superposition according to the sequence of the (i + 1) th row of pixels in the image to be displayed from the left to the right after performing time sequence superposition on the pixel data corresponding to each row of pixels in the image to be displayed, and performs time sequence superposition on the pixel data corresponding to the pixels from the first row of the image to be displayed in the next frame line by line according to the sequence from the left to the right after performing time sequence superposition on the pixel data corresponding to each row of pixels in the image to be displayed.

When the display device scans pixels in the image to be displayed in an interlaced manner, the processing unit 133 processes pixels in an ith row in the image to be displayed one by one according to a left-to-right sequence, after all pixels included in the ith row are processed, the pixels in an (i + 2) th row in the image to be displayed are processed one by one according to the left-to-right sequence, after all pixels included in an m-1 th row in the image to be displayed are processed, m is the number of rows of pixels in the image to be displayed, the processing unit 133 processes pixels in a 2 nd row in the image to be displayed one by one according to the left-to-right sequence until all pixels in the rows in the image to be displayed are processed, and the pixels are processed from a first row of the image to be displayed in a next frame. Correspondingly, the interface unit 134 sequentially reads the pixel data corresponding to each pixel in the ith row from the storage unit 135 for time-series superposition according to the sequence of the pixels in the ith row from the left to the right in the image to be displayed, sequentially reads the pixel data corresponding to each pixel in the (i + 2) th row from the storage unit 135 for time-series superposition according to the sequence of the pixels in the (i + 2) th row from the left to the right in the image to be displayed after the time-series superposition of the pixel data corresponding to each pixel in the (m-1) th row in the image to be displayed, sequentially reads the pixel data corresponding to each pixel in the (2) th row from the storage unit 135 for time-series superposition according to the sequence of the pixels in the (2) th row from the left to the right in the image to be displayed until the pixel data corresponding to each row of pixels in the image to be displayed are time-series superposed, and carrying out time sequence superposition on pixel data corresponding to the pixels line by line according to the sequence from left to right from the first line of the image to be displayed of the next frame.

In one possible implementation, the storage unit 135 may be a First-in First-out (FIFO) memory based on the display processing unit 132 shown in fig. 3. Fig. 4 is an internal structural view of a display processing unit according to another embodiment of the present application. As shown in fig. 4, the display processing unit 132 includes a processing unit 133, an interface unit 134, and a FIFO memory 136, the processing unit 133 stores pixel data obtained by processing pixels into the FIFO memory 136, and the interface unit 134 reads the pixel data from the FIFO memory 136 for time-series superimposition, obtaining a pixel data stream for displaying an image.

Because the clock signals adopted by the processing unit 133 and the interface unit 134 are different, the speed of generating the pixel data by the processing unit 133 is greater than the speed of outputting the pixel data by the interface unit 134, and the FIFO memory 136 is arranged between the processing unit 133 and the interface unit 134, and rectification is performed by the FIFO memory 136, so that the interface unit 134 can perform time sequence superposition based on the refresh frequency of the display device. Since the FIFO memory 136 adopts the queue access rule of first-in first-out, the element added to the queue first is taken out first, and the element added to the queue last is taken out, so long as the processing unit 133 processes the pixels in the image to be displayed according to the order required by the display device, and stores the pixel data obtained by the processing into the FIFO memory 136, it can be ensured that the interface unit 134 reads the pixel data from the FIFO memory 136 in the same order for time-series superposition, thereby ensuring that the display device can normally display the image to be displayed through the pixel data stream.

In a possible implementation manner, the data capacity of the FIFO memory 136 is larger than the data amount of the pixel data corresponding to one line of pixels in the image to be displayed, and the data capacity of the FIFO memory 136 may be any value larger than the data amount of the pixel data corresponding to one line of pixels in the image to be displayed, for example, the FIFO memory 136 may store the pixel data corresponding to 1.5 lines, two lines, three lines, or more lines of pixels in the image to be displayed.

The data capacity of the FIFO memory 136 is greater than the data amount of the pixel data corresponding to a row of pixels in the image to be displayed, so that the FIFO memory 136 can at least accommodate the pixel data corresponding to a row of pixels, and when the FIFO memory 136 has a large data capacity, the pixel data stored in the FIFO memory 136 can be supplied to the interface unit 134 for long-time sequence superposition, so that the first clock signal can be turned off for a long time each time, the frequency of starting and stopping the circuit in the processing unit 133 is reduced, and the power consumption of the display processing unit 132 is further reduced.

In addition, during the process of performing time sequence superposition on the pixel data, the interface unit 134 has a line blanking region and a frame blanking region, the line blanking region corresponds to a time period when the scanning point returns from the right edge of the ith line to the left edge of the (i + 1) th line, the frame blanking region corresponds to a time period when the scanning point returns from the lower right corner of the image to the upper left corner of the image, and obviously, the time length corresponding to the frame blanking region is longer than that corresponding to the line blanking region. The data capacity of the FIFO memory 136 is greater than the data amount of the pixel data corresponding to a row of pixels in the image to be displayed, so that the interface unit 134 experiences at least one row blanking area when performing the time sequence superposition after each time of turning off the first clock signal, which may also increase the time during which the interface unit 134 continuously operates based on the pixel data in the FIFO memory 136, thereby further increasing the duration after each time of turning off the first clock signal, reducing the frequency of turning on and off the circuit in the processing unit 133, increasing the time during which the circuit in the processing unit 133 does not operate, and further reducing the power consumption of the display processing unit 132.

In one possible implementation, the turning on and off of the first clock signal may be controlled by a status flag of the FIFO memory 136. The status of the FIFO memory 136 is identified as a first identification when the amount of data stored by the FIFO memory 136 is less than a first data amount threshold, and the status of the FIFO memory 136 is identified as a second identification when the amount of data stored by the FIFO memory 136 is greater than a second data amount threshold. When the clock unit 137 detects that the status flag of the FIFO memory 136 is the first flag, the clock unit 137 starts the first clock signal, so that the processing unit 133 processes the pixels in the image to be displayed according to the first clock signal. When the clock unit 137 detects that the status flag of the FIFO memory 136 is the second flag, the clock unit 137 turns off the first clock signal, so that the processing unit 133 stops processing the pixels in the image to be displayed.

Alternatively, the second data amount threshold is equal to the data capacity of the FIFO memory 136, i.e. the clock unit 137 turns off the first clock signal when the FIFO memory 136 is full of pixel data. Specifically, the clock unit 137 detects the status flag of the FIFO memory 136, and if the status flag of the FIFO memory 136 transitions to the second flag and the status flag of the FIFO memory 136 indicates that the duration of the second flag is greater than the preset time threshold, the clock unit 137 turns off the first clock signal.

In a possible implementation manner, after the processing unit 133 processes the pixels in the image to be displayed, the generated pixel data is not directly sent to the interface unit 134 for time-series superposition, but is stored in the storage unit 135 first, and then the interface unit 134 reads the pixel data from the storage unit 135 for time-series superposition, so as to ensure that the interface unit 134 reads the pixel data from the storage unit 135 in sequence for time-series superposition, a pixel data stream for normally displaying the image to be displayed can be generated, the processing unit 133 can process a plurality of pixels in the image to be displayed simultaneously, and the processing unit 133 can also generate the pixel data without ordering the pixel data in the pixel data stream.

When processing the pixels in the image to be displayed according to the first clock signal, the processing unit 133 performs parallel processing on at least two pixels in the image to be displayed in each clock cycle, so as to obtain pixel data corresponding to each pixel. The processing unit 133 includes at least two circuit structures for image processing, each of which can independently process one pixel in the image to be displayed in each clock cycle according to the first clock signal to obtain one pixel data, so that the processing unit 133 can generate a plurality of pixel data in each clock cycle and store all the generated pixel data in the storage unit 135. The processing unit 133 performs parallel processing on a plurality of pixels in the image to be displayed in each clock cycle, so that the time for generating pixel data is shortened, more pixel data can be generated in a shorter time, and the data amount of the pixel data in the storage unit 135 is larger than the second data amount threshold, so that the circuit for image processing in the processing unit 133 can be suspended for a longer time, the power consumption of the display processing unit 132 is reduced, and the power consumption of the display system in the electronic device is further reduced.

In one example, after the data amount of the pixel data in the storage unit 135 is smaller than the first data amount threshold, the first clock signal is turned on, and the processing unit 133 processes the pixel in the image to be displayed according to the first clock signal. If the processing unit 133 processes one pixel in the image to be displayed at each clock cycle and stores the pixel data generated by the processing into the storage unit 135, the elapsed time period T is required to make the data amount of the pixel data in the storage unit 135 larger than the second data amount threshold. If the processing unit 133 performs parallel processing on two pixels in the image to be displayed in each clock cycle and stores two pixel data generated by the processing into the storage unit 135, it takes a time period T/2 to make the data amount of the pixel data in the storage unit 135 larger than the second data amount threshold. If the processing unit 133 performs parallel processing on three pixels in the image to be displayed at each clock cycle and stores the three pixel data generated by the processing into the storage unit 135, it takes a time period T/3 to make the data amount of the pixel data in the storage unit 135 larger than the second data amount threshold.

In a possible implementation manner, before starting to display a series of images to be displayed, the processing unit 133 first selects a first clock signal with a frequency greater than that of a second clock signal according to the second clock signal, and after starting to display the images, the processing unit 133 processes one pixel in the images to be displayed in each clock cycle according to the first clock signal, so as to obtain corresponding pixel data, and store the corresponding pixel data in the storage unit 135.

Since different display devices may have different refresh frequencies, and the frequency of the second clock signal needs to match the refresh frequency of the display device, that is, when an image is displayed by the display devices having different refresh frequencies, the interface unit 134 needs to perform timing superposition on the pixel data according to the different second clock signals. After the second clock signal is determined, the processing unit 133 selects a clock signal with a frequency greater than that of the second clock signal from a plurality of preset clock signals as the first clock signal, and further processes the pixels in the image to be displayed according to the selected first clock signal. A plurality of clock signals with different frequencies are preset, the processing unit 133 selects a clock signal with a frequency greater than that of the second clock signal as the first clock signal according to the second clock signal, and the speed of generating the pixel data by the processing unit 133 according to the first clock signal is ensured to be greater than the speed of outputting the pixel data by the interface unit 134, so as to ensure that the image can be normally displayed.

The processing unit 133 may select, as the first clock signal, a clock signal having a frequency greater than that of the second clock signal and a frequency relatively close to that of the second clock signal when selecting the first clock signal. Because the frequency of the first clock signal is higher, the power consumption of the processing unit 133 is higher when the processing unit processes the pixel, and the clock signal with the frequency higher than the second clock signal and the frequency difference smaller than the second clock signal is selected as the first clock signal, when the processing unit 133 processes the pixel according to the first clock signal, not only can the speed of generating the pixel data be ensured to be higher than the speed of outputting the pixel data by the interface unit 134, but also the processing unit 133 can have lower power consumption, so that the power consumption of a display system in the electronic device can be further reduced.

In one possible implementation, the interface unit 134 stops reading of the pixel data from the storage unit 135 in the line blank region and the frame blank region of the image to be displayed according to the second clock signal.

In the process of displaying the image to be displayed based on the pixel data stream generated by the interface unit 134, the image to be displayed needs to be scanned line by line, a line blanking area and a frame blanking area exist in the scanning process, the interface unit 134 stops reading the pixel data from the storage unit 135 in the line blanking area and the frame blanking area, the image to be displayed can be displayed normally, and the power consumption of a circuit in the interface unit 134 can be reduced.

In one possible implementation, the interface unit 134 determines the number of pixel data read from the storage unit 135 according to the second clock signal, and then determines the line blank region and the frame blank region according to the number of pixel data read.

In the process of displaying the image to be displayed, the second clock signal is always applied to the interface unit 134, so the interface unit 134 can determine the number of pixel data read from the storage unit 135 according to the second clock signal, and the number of rows of pixels included in the image to be displayed and the number of pixels per row can be determined, so that the row blanking area and the frame blanking area can be determined according to the number of pixel data read from the storage unit 135, and further, the reading of the pixel data from the storage unit 135 is stopped in the row blanking area and the frame blanking area, thereby ensuring that the image to be displayed can be normally displayed.

The interface unit 134 counts the number of pixel data read from the storage unit 135 by a line counter and a frame counter, and the count values of the line counter and the true counter are each increased by 1 every time the interface unit 134 reads one pixel data from the storage unit 135. When the count value of the line counter is equal to the number of pixels included in one pixel line in the image to be displayed, the interface unit 134 determines to enter the line blanking area, and stops reading the pixel data from the storage unit 135 for the next n clock cycles, where the sum of the time lengths of the n clock cycles is equal to the time length corresponding to the line blanking area. When the count value of the frame counter is equal to the number of pixels included in the image to be displayed, the interface unit 134 determines that the frame blanking area is entered, and stops reading the pixel data from the storage unit 135 for the next m clock cycles, where the sum of the time lengths of the m clock cycles is equal to the time length corresponding to the frame blanking area, and n < m.

In one possible implementation manner, the first clock signal is turned on or off according to the data amount of the pixel data in the storage unit 135, the processing unit 133 intermittently processes the pixels in the image to be displayed, and when the pixels are not processed, the circuit for image processing in the processing unit 133 is stopped, so that the power consumption of the processing unit 133 is reduced. The processing unit 133 may gate the circuit for image processing based on a first clock signal, control the circuit for image processing to operate when the first clock signal is turned on, process pixels in an image to be displayed, and control the circuit for image processing to stop operating when the first clock signal is turned off. The processing unit 133 gates the circuit based on the first clock signal, which can ensure that the circuit in the processing unit 133 can accurately start or stop working as required, thereby ensuring that the pixel data generated by the processing unit 133 can be supplied to the interface unit 134 for timing superposition.

Display processing method

Fig. 5 is a flowchart of a display processing method according to an embodiment of the present application, where the display processing method is used in the display processing unit according to any of the embodiments, and as shown in fig. 5, the display processing method 500 includes:

step 501, processing pixels in an image to be displayed according to a first clock signal to obtain pixel data, and storing the pixel data in a storage unit, wherein the first clock signal is turned on when the data amount of the data stored in the storage unit is smaller than a first data amount threshold, the first clock signal is turned off when the data amount of the data stored in the storage unit is larger than a second data amount threshold, and the first data amount threshold is smaller than a second data amount threshold;

step 502, reading pixel data from the storage unit according to the second clock signal, and performing time sequence superposition on the read pixel data to obtain a pixel data stream for displaying an image to be displayed.

Since details of the display processing method are already described in detail in the display processing unit portion of the foregoing embodiment in conjunction with the schematic structural diagram, specific processes may refer to descriptions in the foregoing display processing unit embodiment, and are not described again here.

Commercial value of embodiments of the present application

When the technical problem that power consumption of a display system in electronic equipment is high is solved, a processing unit and an interface unit are separated, the processing unit processes pixels in an image to be displayed according to a first clock signal and stores pixel data obtained through processing into a storage unit, and the interface unit reads the pixel data from the storage unit according to a second clock signal to perform time sequence superposition to generate a pixel data stream for displaying the image to be displayed. Because the processing unit and the interface unit work according to different clock signals, the processing unit can rapidly process pixels according to a first clock signal with high frequency, after more pixel data are stored in the storage unit, the first clock signal is closed, the processing unit suspends the pixel processing, the interface unit is not influenced, time sequence superposition is continuously carried out according to a second clock signal, and in the process that the processing unit suspends the pixel processing, a circuit in the processing unit is in a work stopping state, so that the power consumption of the display processing unit can be effectively reduced, and the power consumption of a display system in the electronic equipment is further reduced.

It should be understood that the embodiments in this specification are described in a progressive manner, and that the same or similar parts in the various embodiments may be referred to one another, with each embodiment being described with emphasis instead of the other embodiments. In particular, as for the method embodiments, since they are substantially similar to the methods described in the apparatus and system embodiments, the description is simple, and the relevant points can be referred to the partial description of the other embodiments.

It should be understood that the above description describes particular embodiments of the present specification. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

It should be understood that an element described herein in the singular or shown in the figures only represents that the element is limited in number to one. Furthermore, modules or elements described or illustrated herein as separate may be combined into a single module or element, and modules or elements described or illustrated herein as single may be split into multiple modules or elements.

It is also to be understood that the terms and expressions employed herein are used as terms of description and not of limitation, and that the embodiment or embodiments of the specification are not limited to those terms and expressions. The use of such terms and expressions is not intended to exclude any equivalents of the features shown and described (or portions thereof), and it is recognized that various modifications may be made within the scope of the claims. Other modifications, variations, and alternatives are also possible. Accordingly, the claims should be looked to in order to cover all such equivalents.

Claims

1. A display processing unit comprising: the device comprises a processing unit, an interface unit and a storage unit;

the processing unit is used for processing pixels in an image to be displayed according to a first clock signal to obtain pixel data and storing the pixel data into the storage unit, wherein the first clock signal is turned on when the data volume of the data stored in the storage unit is smaller than a first data volume threshold value, the first clock signal is turned off when the data volume of the data stored in the storage unit is larger than a second data volume threshold value, and the first data volume threshold value is smaller than the second data volume threshold value;

the interface unit is used for reading the pixel data from the storage unit according to a second clock signal, and performing time sequence superposition on the read pixel data to obtain a pixel data stream for displaying the image to be displayed.

2. The display processing unit of claim 1, wherein the processing unit is configured to sequentially process pixels in the image to be displayed corresponding to the pixel data according to an ordering of the pixel data in the pixel data stream.

3. The display processing unit of claim 2, wherein the storage unit is a first-in-first-out memory.

4. The display processing unit of claim 3, wherein the data capacity of the FIFO memory is larger than the data amount of the pixel data corresponding to a row of pixels in the image to be displayed.

5. The display processing unit according to claim 1, wherein the processing unit is configured to perform parallel processing on at least two pixels in the image to be displayed in each clock cycle according to the first clock signal, and obtain the pixel data corresponding to each pixel.

6. The display processing unit according to claim 1, wherein the processing unit is configured to select, according to the second clock signal, a first clock signal with a frequency greater than that of the second clock signal from at least two preset clock signals, and process, according to the selected first clock signal, one pixel in the image to be displayed in each clock cycle to obtain the corresponding pixel data.

7. The display processing unit according to any one of claims 1 to 6, wherein the interface unit is configured to stop reading the pixel data from the storage unit in a line blanking region and a frame blanking region of the image to be displayed according to the second clock signal.

8. The display processing unit of claim 7, wherein the interface unit is configured to determine a number of the pixel data read from the storage unit according to the second clock signal, and determine the line blanking area and the frame blanking area according to the number.

9. A display processing method, comprising:

processing pixels in an image to be displayed according to a first clock signal to obtain pixel data, and storing the pixel data in the storage unit, wherein the first clock signal is turned on when the data volume of the data stored in the storage unit is smaller than a first data volume threshold, the first clock signal is turned off when the data volume of the data stored in the storage unit is larger than a second data volume threshold, and the first data volume threshold is smaller than the second data volume threshold;

and reading the pixel data from the storage unit according to a second clock signal, and performing time sequence superposition on the read pixel data to obtain a pixel data stream for displaying the image to be displayed.

10. An acceleration unit, comprising:

the display processing unit of any one of claims 1-8;

and the controller is used for controlling the display processing unit to work.

11. The acceleration unit of claim 10, further comprising:

the clock unit is used for starting a first clock signal input to the display processing unit when the data volume of the data stored in the storage unit in the display processing unit is smaller than a first data volume threshold value, and closing the first clock signal when the data volume of the data stored in the storage unit is larger than a second data volume threshold value, wherein the first data volume threshold value is smaller than the second data volume threshold value.

12. An electronic device, comprising:

an acceleration unit according to claim 10 or 11;

and the scheduling unit is used for scheduling the accelerating unit to execute the image display task.

13. A system on chip comprising an acceleration unit according to claim 10 or 11.