CN106488240A - Image processing system and image processing method - Google Patents

Abstract

The invention provides an image processing system and an image processing method, comprising an image processing module, a frame buffer coding module and a frame buffer. Each image block comprises a plurality of first type coding blocks and at least one second type coding block. The image processing module generates a first image processing result according to a plurality of first type coding blocks of the target image block. The frame buffer coding module generates a first frame buffer coding result according to the first image processing result. The frame buffer is provided with a first temporary storage area with at least one first random access point and a second temporary storage area with at least one second random access point for the target image block. Storing a first frame buffer encoding result into the first temporary storage region; at least one second type coded block of the target image block is stored in the second temporary storage area.

Description

Image processing system and image processing method

technical field

The present invention relates to image processing technology, and in particular to frame buffers for storing image data.

Background technique

With the vigorous development of various digital electronic products, most of the current audio-visual multimedia data exists in digital format. The digital files of audio-visual data are usually quite large, and the higher the resolution, the larger the data volume. For example, in applications such as video telephony or digital TV, the video and audio data is usually transmitted in a stream mode through the wireless network. Due to the high amount of data, the transmission bandwidth may be insufficient. Therefore, the digital audio-video coding technology is used at the transmitting end to encode the audio-video data before transmission. Currently, digital audio-video coding technologies widely used are MPEG-4, H.264, etc., for example.

When the receiving end receives the video-encoded data, the decoder decodes the encoded data and restores the video-audio data sent by the transmitting end. Many receivers use storage devices such as Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM) as buffers to temporarily store restored audio and video data for subsequent used by image processing programs.

The decoded audio-visual data includes multiple video frames, which are stored in the frame buffer. Each video frame is defined as having multiple image blocks, and each image block is used as a unit of video frame access for subsequent In the image processing program, for example, the video frame in FIG. 1(A) includes image blocks (0,0)˜(M,N), and the size of each image block is fixed.

However, due to the large amount of audio and video data, the frame buffer space consumed is also large; on the other hand, when performing image processing, if you want to increase the speed of accessing audio and video data from the frame buffer, you need to use bandwidth Larger, but also more expensive memory. As a result, the overall cost of the hardware system will undoubtedly increase.

Contents of the invention

To solve the above problems, the present invention proposes a new video frame buffer and dynamic image processing system.

A specific embodiment according to the present invention is an image processing system for performing an image processing procedure on a plurality of image blocks. Each of the plurality of image blocks includes a plurality of first-type coding blocks and at least one second-type coding block. The image processing system includes an image processing module, a frame buffer coding module, a frame buffer and a frame buffer controller. The image processing module applies image processing to a plurality of first-type coding blocks in a target image block to generate a set of first image processing results. The frame buffer encoding module applies frame buffer encoding to the set of first image processing results to generate a set of first frame buffer encoding results. The frame buffer is provided with a first temporary storage area and a second temporary storage area corresponding to the target image block. The first temporary storage area has at least one first random access point. The second temporary storage area has at least one second random access point independent of the at least one first random access point. The frame buffer controller enables the set of first frame buffer encoding results to be stored in the first temporary storage area in the frame buffer, and enables at least one second-type encoding block of the target image block to be stored in the This second scratchpad area in the framebuffer.

Another specific embodiment of the present invention is an image processing method, which is used to cooperate with a frame buffer to perform an image processing procedure for a plurality of image blocks. Each of the plurality of image blocks includes a plurality of first-type coding blocks and at least one second-type coding block. The frame buffer is provided with a first temporary storage area and a second temporary storage area corresponding to a target image block. The first temporary storage area has at least one first random access point. The second temporary storage area has at least one second random access point independent of the at least one first random access point. First, multiple first-type coding blocks in the target image block are subjected to image processing to generate a set of first image processing results. Next, the set of first image processing results are subjected to frame buffer encoding to generate a set of first frame buffer encoding results. Subsequently, the set of first frame buffer encoding results are stored in the first temporary storage area in the frame buffer, and at least one second-type encoding block of the target image block is stored in the frame buffer in the Second staging area.

The advantages and spirit of the present invention can be further understood through the following detailed description of the invention and the accompanying drawings.

Description of drawings

FIG. 1(A) is a schematic diagram of a frame divided into multiple image blocks.

FIG. 1(B) is a schematic diagram of an image block being divided into multiple coding blocks.

FIG. 2(A) presents a typical storage space planning method in the frame buffer.

FIG. 2(B) shows a planning method of the storage space in the frame buffer with data compression.

3(A)-3(C) are functional block diagrams of an image processing system according to an embodiment of the present invention.

Figures 4(A) to 4(D) take eight coded blocks of an image block as an example, showing the random access points, the first temporary storage area, and the second temporary storage area in the frame buffer according to the present invention Several examples of relative relationships.

FIG. 5 is a functional block diagram of an image processing system according to another embodiment of the present invention.

FIG. 6 presents a schematic diagram of dividing a coding block into multiple prediction blocks.

FIG. 7 is a flowchart of an image processing method according to an embodiment of the invention.

Symbol Description

300, 500: image processing system 31, 51: frame buffer

32, 52: frame buffer controller 33, 53: image decoding module

34, 54: image processing module 35, 55: frame buffer encoding module

410: the first temporary storage area 420: the second temporary storage area

S71～S74: process steps

It should be noted that the drawings of the present invention include functional block diagrams representing various interrelated functional modules. These drawings are not detailed circuit diagrams, and the connecting lines are only used to represent signal flow. Various interactions between functional elements and/or programs do not necessarily need to be achieved through direct electrical connections. Furthermore, the functions of individual components do not have to be allocated as shown in the drawings, and distributed blocks do not have to be realized by distributed electronic components.

detailed description

According to an embodiment of the present invention, each image block is further defined as a plurality of coding blocks, for example, the image block (0,0) is further defined as the coding block 0~code in FIG. 1(B) Block 7. In this embodiment, the coding block is the smallest unit for accessing video frames. FIG. 2(A) shows that an image block is stored in the frame buffer in units of encoding blocks. Taking the case where the data volumes of these coding blocks are each 64 bytes as an example, a storage space planning method in the frame buffer is presented. In this example, the eight coded blocks 0~7 of image block (0,0) (indicated by symbols EB _0(0,0) , EB _1(0,0) , ..., EB _7(0,0) represent, collectively referred to as [EB ₀ ˜EB ₇ ] _(0,0) ) are each stored in a memory space with a length of 64 bytes.

An embodiment according to the present invention is an image processing system 300, the functional block diagram of which is shown in FIG. 3(A). The image processing system 300 includes a frame buffer 31 , a frame buffer controller 32 , an image decoding module 33 , an image processing module 34 and a frame buffer encoding module 35 . The image processing system 300 is used for performing image processing procedures on a plurality of image blocks, such as image blocks (0,0)˜(M,N) in FIG. 1(A). The following paragraphs mainly take the division method of the image block in FIG. 1(A) as an example. Those skilled in the art of the present invention can understand through the following description that the scope of the present invention is not limited to the division method of the above blocks.

Please also refer to the structure of the image block in FIG. 2(A) and the image processing system 300 in FIG. 3(A). When performing an image processing procedure, it is common that, since the coded blocks 6 and 7 of the image block (0,0) are adjacent to the image block (1,0) in the original picture, the The target image block, that is, the image block (1,0), needs to use the image block (0,0) as a reference image block when the image processing procedure is performed in the image processing module 34 . More specifically, it refers to the data of the coding blocks 6 and 7 of the image block (0,0). Assuming that the image block (0,0) has been stored in the frame buffer 31 earlier, when the image processing module 34 performs an image processing program on the image block (1,0), such as a de-blocking program, the frame buffer controller 32 reads the data of the coding blocks 6 and 7 of the image block (0,0) from the frame buffer 31 to the image processing module 34 according to the request of the image processing module 34 . In one embodiment, in order to achieve the purpose of reading only part of the coding block instead of the entire image block, each of the eight memory spaces storing the coding blocks 0-7 of the image block (0,0) is assigned a The random access point (random access point) is like the random access points 0-7 in FIG. 2(A). Therefore, the frame buffer controller 32 can directly access any coded block of the image block (0,0) from the frame buffer 31 according to the request of the image processing module 34, without first having to read from the coded block located at an earlier address. Blocks are read.

In one embodiment of the present invention as shown in FIG. 3(A), before the decoded video and audio data is stored in the frame buffer 31, it will be frame buffer encoded by the frame buffer encoding module 35, and then the encoded audio and video data will be encoded by the frame buffer. The data is stored in the frame buffer 31 , which can save the transmission bandwidth required for accessing video and audio data from the frame buffer 31 . The frame buffer encoding referred to herein may be a compression method, which is not limited in any way, and the compression method shall not limit the scope of the present invention.

In addition, FIG. 2(B) presents a storage space planning method in the frame buffer with data compression. In this example, the eight pieces of compressed data (represented by symbols [EB ₀ ~EB ₇ ]” _(0,0)) corresponding to the eight coding blocks of the image block (0,0) have different sizes. For example As shown in FIG. 2(B), the compressed data [EB ₀ ˜EB ₇ ]” _(0,0) is continuously stored in the frame buffer from random access point 0 without interval. In this example, the sum of the compressed data EB ₀ ” _(0,0) and the compressed data EB ₁ ” _(0,0) is 64 bits, so they jointly occupy the first column starting from random access point 0 A memory space with a length of 64 bytes. The data size of the compressed data EB ₂ ” _(0,0) is less than 64 bits, so part of the compressed data EB ₃ ” _(0,0) (marked as EB _3A ” _(0,0) ) and the compressed The data EB ₂ ″ _{(0, 0)} jointly occupy the memory space of the second column with a length of 64 bytes. Similarly, another part of compressed data EB ₃ " _(0,0) (marked as EB _3B " _(0,0) ) and a part of compressed data EB ₄ " _(0,0) (marked as EB _4A " _{( 0,0)} )) together occupy the memory space of the third column length of 64 bytes. By analogy, the compressed data [EB ₀ ~EB ₇ ]” _(0,0) in the example of Figure 2(B) starts from random access point 0 and occupies a total of six columns of memory space (only a part of it is used instead of the entire sixth column).

As known to those skilled in the art to which the present invention pertains, the amount of data that can be read from the frame buffer each time is a fixed value, such as 32 bytes or 64 bytes. Taking the case where the amount of data that can be read out from the frame buffer each time is 32 bytes (corresponding to the half-column memory space) as an example, comparing FIG. 2(A) with FIG. 2(B), it can be seen that, If the data related to coding blocks 0-7 needs to be read out, the coding block shown in Figure 2(A) needs to be read a total of 512 bytes (8 columns) of data, while Figure 2(B) As shown, only 352 bytes (5.5 columns) of data need to be read from the random access point 0 of the encoded block that has been encoded by the frame buffer. Obviously, the effect of saving the amount of data transmission can be achieved by using the coded blocks encoded by the frame buffer as shown in FIG. 2(B). However, the disadvantage of this approach in Figure 2(B) is that when only the compressed data [EB ₆ ~EB ₇ ]” _(0,0) needs to be read out, because only one random access point 0 exists in The starting position of the frame buffer (that is, the starting position of the compressed data EB ₀ ” _(0,0) ), before the compressed data [EB ₆ ~ EB ₇ ]” _(0,0) is read out, The compressed data [EB ₀ ~EB ₅ ]” _(0,0) also needs to be read out completely, which leads to low reading efficiency and waste of reading bandwidth.

In another embodiment according to the present invention, the coding blocks included in the image blocks are divided into two types. More specifically, each image block includes a plurality of first-type coding blocks and at least one second-type coding block. The way these coding blocks are classified is based on the fact that when image processing is to be performed on a part of one image block (called a target image block), a part of another image block (called a reference image block) must also be processed. is read for the above image processing routines. Accordingly, a part of the target image block is regarded as a first-type coding block, and a part of the reference image block is regarded as a second-type coding block. When the second type of coded block of the reference image block needs to be read, but the first type of coded block of the reference image block does not need to be read, the second type of coded block can be read from the frame buffer register 31 is read individually. Taking the image coding standard H.264 and the coded block division method shown in Figure 1(B) as an example, the coded blocks 0-5 in each image block can be regarded as the first type of coded blocks, and the coded blocks Blocks 6, 7 can be regarded as a second type of coding block. Please refer to FIG. 3(A), in the image processing system 300 of an embodiment of the present invention, for each image block being processed, the frame buffer 31 is provided with a first temporary storage area corresponding to the first type of coding block and a second temporary storage area corresponding to the second type of coding block. The first temporary storage area has at least one first random access point, and the second temporary storage area has at least one second random access point. With different settings of random access points, when the first type of coding block corresponding to the same image block has not been read, the second type of coding block can still be read from the frame buffer 31 independently In other words, it is not necessary to read the first type of coding block before reading the second type of coding block. An example of the arrangement of the first random access point and the second random access point according to the present invention will be described later with reference to FIGS. 4(A) to 4(D).

As shown in FIG. 3(A), in the image processing system 300, the image decoding module 33 is responsible for performing image decoding on the input data I, so as to obtain the data of each coding block (such as the grayscale, chroma, and brightness of each pixel). …Wait). The relative position of each coding block in the associated image block is known in advance. Therefore, the image decoding module 33 can determine that each decoded coded block is the first type of coded block or the second type of coded block. The output terminals of the image decoding module 33 transmit different types of coding blocks to different signal paths. More specifically, for the same image block, the image decoding module 33 provides its first type of coding block to the image processing module 34, and through the control of the frame buffer controller 32, its second type of coding block Stored in the frame buffer 31. The frame buffer controller 32 is responsible for receiving the access requests of each module for the frame buffer 31, coordinating the right to use the data input and output terminals of the frame buffer 31, and providing the corresponding access start address according to the number of each image block. to the module making the request. For example, after receiving a request from the image decoding module 33 to store the second-type coded block of a certain image block into the frame buffer 31, the frame buffer controller 32 will store the second coded block corresponding to the image block into the frame buffer 31. The storage start address of the second-type coding block is provided to the image decoding module 33 , so that the one or more second-type coding blocks are stored in the second temporary storage area corresponding to the image block in the frame buffer 31 . After receiving the first-type coding blocks provided by the image decoding module 33 , the image processing module 34 performs image processing on these first-type coding blocks to generate a set of first image processing results. For example, the image processing performed by the image processing module 34 may be a deblocking procedure. Subsequently, the frame buffer encoding module 35 is responsible for applying frame buffer encoding to the set of first image processing results to generate a set of first frame buffer encoding results, and through the assistance of the frame buffer controller 32, the set of first The frame buffer encoding result is stored in the frame buffer 31 in the first temporary storage area corresponding to the image blocks to which the first-type encoding blocks belong.

In addition, FIG. 3(A) takes the case where the image processing system 300 processes the image block (1,0) in FIG. 1(A) as an example, and indicates the content of the signals transmitted between the modules. It is assumed that coding blocks 0 to 5 in each image block are coding blocks of the first type, and coding blocks 6 and 7 are coding blocks of the second type. Also, as mentioned above, it is assumed that the image processing module 34 needs to refer to a second type of coding block of a reference image block when performing image processing on a first type of coding block of a certain target image block. As shown in FIG. 3(A), after the image decoding module 33 completes the decoding of the image block (1,0), the first-type coding blocks [EB ₀ ˜EB ₅ ] of the image block (1,0) are _(1,0) is transmitted to the image processing module 34 , and the second type of coding block [EB ₆ ˜EB ₇ ] _(1,0) of the image block (1,0) is transmitted to the frame buffer 31 . If the image block (0,0) is the reference image block relative to the image block (1,0), and the two second-type coding blocks [EB ₆ ˜EB ₇ ] of the image block (0,0) ] _(0,0) has been stored in the frame buffer 31 by the image decoding module 33 earlier, then the image processing module 34 can make a request to the frame buffer controller 32 to obtain from the frame buffer 31 corresponding to the image block ( 0,0) from the second temporary storage area to read out the second-type coding blocks [EB ₆ -EB ₇ ] _{(0,0) of the image block (0,0)} . Next, the image processing module 34 and the frame buffer encoding module 35 are responsible for sequentially generating a set of first image processing results [EB ₀ -EB ₅ ]' _{(1,0) corresponding to the image block (1,0)} and A group of first frame buffer encoding results [EB ₀ ～EB ₅ ]” _(1,0) . Subsequently, this group of first frame buffer encoding results [EB ₀ ～EB ₅ ]” _(1,0) will be stored in the frame The buffer 31 corresponds to the first temporary storage area of the image block (1,0).

In one embodiment, when the image processing module 34 performs image processing on the first-type coding blocks [EB ₀ ˜EB ₅ ] _(1,0) of the image block (1,0), it also performs image processing on the image block The second-type coding blocks [EB ₆ ˜EB ₇ ] _{(0,0) of (0,0)} perform image processing to generate a set of second image processing results [EB ₆ ˜EB ₇ ]′ _(0,0) . Taking the case where the function of the image processing module 34 is deblocking as an example, the image processing module 34 may also refer to the first type of coding blocks [EB ₀ -EB ₅ ] _{(1 ,0)} and the second type of coding block [EB ₆ ～EB ₇ ] _{(0,0) of the image block (0,0)} to modify the second type of coding block [EB ₆ ~ EB ₇ ] _(0,0) . In this case, the above-mentioned second image processing result [EB ₆ ~EB ₇ ]' _(0,0) is the second type of coding block [EB ₆ ~EB ₇ ] _(0,0) after deblocking . As shown in FIG. 3(B), the group of second image processing results [EB ₆ ˜EB ₇ ]’ _(0,0) is stored back in the frame buffer 31 corresponding to the image block (0,0). 2. Temporary storage area. Alternatively, as shown in FIG. 3(C), the set of second image processing results [EB ₆ ˜EB ₇ ]’ _(0,0) is handed over to the frame buffer encoding module 35 for frame buffer encoding into a set of second frame buffer Encoding result [EB ₆ ˜EB ₇ ]” _(0,0) . Subsequently, the frame buffer encoding module 35 will make a request to the frame buffer controller 32 to make the second frame buffer encoding result [EB ₆ ˜EB ₇ ]” _{( 0,0)} is stored back into the second temporary storage area corresponding to the image block (0,0) in the frame buffer 31 . It should be noted that the frame buffer encoding method of the image data is known to those skilled in the art of the present invention, and will not be repeated here.

As mentioned earlier, the first type of coded block will be compressed by the frame buffer coding module 35 before being stored in the frame buffer 31 after the image processing, and the second type of coded block will be stored after the image processing. Before entering the frame buffer 31, it may also undergo compression processing by the frame buffer encoding module 35, as shown in FIG. 3(C), for example. 4(A) to 4(D) take the eight coded blocks 0-7 of the image block (1,0) as an example, showing the random access point and the first temporary storage area, the second temporary storage area in the frame buffer 31. Several examples of the relative relationship between the two staging areas. In these four illustrations, the first temporary storage area for storing the first type of coding block is marked as 410 , and the second temporary storage area for storing the second type of coding block is marked as 420 . In practice, the numbers of the first-type coding blocks and the second-type coding blocks can be known in advance, and the capacities of the first temporary storage area 410 and the second temporary storage area 420 can be set accordingly. Taking the High-Efficiency Video Coding (HEVC) specification and the encoding block division method shown in Figure 1(B) as an example, if the length and width of each image block are 32 pixels*16 pixels, then the eight encoding blocks 0～ 7 The size of the data volume before being compressed is 64 bytes each. In the case of using a lossless compression technique, the first temporary storage area 410 can be designed to contain six memory spaces with a length of 64 bytes, while the second temporary storage area 420 can be designed to contain two The length of the memory space is 64 bytes to ensure that no matter what the actual compression ratio is, the sum of the first temporary storage area 410 and the second temporary storage area 420 is sufficient to accommodate the complete data of one image block.

See Figure 4(A) first. In this example, the first temporary storage area 410 includes eight memory spaces with a length of 64 bytes. The first group of frame buffer encoding results [EB ₀ ˜EB ₅ ]” _(1,0) provided by the frame buffer encoding module 35 are respectively stored in the first temporary storage area 410. As can be seen from FIG. 4(A), this The five frame buffer encoding results [EB ₀ -EB ₅ ]" _(1,0) respectively correspond to an encoding block, and each is assigned a first random access point (1-0, 1) by the frame buffer controller 32 -1,...,1-5). The two second-type coding blocks [EB ₆ ˜EB ₇ ] _(1,0) provided by the image decoding module 33 are stored in the second temporary storage area 420 , and each is assigned a first Two random access points (2-0, 2-1). In this case, each module in the image processing system 300 can request the assistance of the frame buffer controller 32 to independently access the frame buffer encoding result [EB ₀ ~EB ₅ ]” _(1,0) and the encoding result from the frame buffer 31. Block [EB ₆ ~EB ₇ ] _(1,0) any one of the eight pieces of data.

See Figure 4(B). In this example, the first set of frame buffer encoding results [EB ₀ ˜EB ₅ ]” _(1,0) provided by the frame buffer encoding module 35 are continuously stored in the first temporary storage area 410 without intervals, and there is a first A random access point (1-0). That is to say, the memory space following the frame buffer encoding result EB ₀ ” _(1,0) will be used to store the frame buffer encoding result EB ₁ ” _{(1,0 )} , and so on. Until a certain column memory space is full, the frame buffer controller will start to store data in the next column memory space in the first temporary storage area 410. As shown in Figure 4 (B), the first temporary storage The first column memory space in the area 410 stores the frame buffer encoding result EB ₀ ″ _(1,0) and the frame buffer encoding result EB ₁ ″ _(1,0) at the same time, and the second temporary storage area 410 The column memory space simultaneously stores the frame buffer encoding result EB ₂ ″ _(1,0) and a part of the frame buffer encoding result EB ₃ ″ _(1,0) (marked as EB _3A ″ _(1,0) ). In this example, the first group of frame buffer encoding results [EB ₀ ~EB ₅ ]" _(1,0) occupies the first ~ fourth column memory spaces in the first temporary storage area 410 (only a part is used instead of All the fourth columns). On the other hand, two coded blocks [EB ₆ ˜EB ₇ ] _(1,0) are stored in a row of memory spaces in the second temporary storage area 420 respectively, and are frame buffered The controller 32 assigns a second random access point (2-0, 2-1). The benefit of this approach is that any coding area in the coding block [EB ₆ ~EB ₇ ] _(1,0) Blocks can be read individually.

See Figure 4(C). In this example, the frame buffer encoding result [EB ₀ ˜EB ₅ ]” _(1,0) provided by the frame buffer encoding module 35 is divided into five parts, each of which is stored in the first temporary storage area 410, and each is framed The buffer controller 32 assigns a first random access point (1-0, 1-1, ..., 1-5). On the other hand, two second-type coded blocks [EB ₆ ~EB ₇ ] _(1,0) are continuously stored in the second temporary storage area 420 without intervals, and share a second random access point (2-0).

See Figure 4(D). In this example, the frame buffer encoding results [EB ₀ ˜EB ₅ ]” _(1,0) provided by the frame buffer encoding module 35 are continuously stored in the first temporary storage area 410 without intervals, and share a first random Access point (1-0). Similarly, two second-type coded blocks [EB ₆ ˜EB ₇ ] _(1,0) that have not been encoded by the frame buffer are continuously stored in the second temporary storage area without intervals 420, and share a second random access point (2-0). In practice, the frame buffer coding module 35 can record whether each coding block is through frame buffer coding and the data volume of each frame buffer coding result, and store In the frame buffer 31 or in other temporary registers. In other words, the amount of data corresponding to each coded block in the frame buffer 31 can be a known number. Assuming that the frame buffer controller 32 can read from the frame buffer each time The amount of data read by the device 31 is fixed at 32 bytes (corresponding to the half column memory space in the figure). As far as the example shown in Fig. 4 (D) is concerned, it is intended to read the image block ( _1,0 ), the frame buffer controller 32 can estimate the frame buffer coding result [EB ₀ ~EB ₅ ] _{” ( 1 ,0)} occupies the first to fourth column memory spaces in the first temporary storage area 410, and uses more than half of the fourth column memory space. On the other hand, the coded blocks [EB ₆ -EB ₇ ] _(1,0) occupies the first column and the second column memory space in the second temporary storage area 420. Therefore, the frame buffer controller 32 should start from the first random access point 1-0 at first, read eight consecutive times By taking 32 bytes (a total of 256 bytes of data are read), the frame buffer encoding result [EB ₀ ~EB ₅ ]" _(1,0) can be read out. Then, starting from the second random access point 2-0, the frame buffer controller 32 reads 32 bytes four times in a row (a total of 96 bytes of data are read), and the coded block [ EB ₆ to EB ₇ ] _(1,0) are read out. On the other hand, when reading the coded block [EB ₆ ˜EB ₇ ] _(1,0) from the frame buffer 31, the frame buffer controller 32 may directly start from the second random access point 2-0, and perform four consecutive Read 32 bytes (a total of 96 bytes of data are read).

It should be noted that since the second type of coded block is stored in the second temporary storage area 420 different from the first temporary storage area 410, it can be read, modified, compressed and It is feasible to restore the second type of encoded blocks after compression. In addition, if there is no need to read any frame buffer encoding result [EB ₀ ～EB ₅ ]” _(1,0) separately, the frame buffer encoding result [EB ₀ ～EB ₅ ]” _{(1, 0)} Using the same first random access point will not cause problems, that is, there is no need to store each frame buffer encoding result in different memory spaces and assign a random access point to each.

It should be noted that giving the second-type coding blocks stored in the second temporary storage area 420 to the image processing module 34 as reference data is only a possibility of using the second-type coding blocks. These second-type coding blocks can also be used by other modules in the image processing system 300 .

FIG. 5 is a functional block diagram of an image processing system according to another embodiment of the present invention. The main difference between the image processing system 500 and the image processing system 300 shown in FIG. When the image processing module 54 needs data of some coding blocks, it makes a request to the frame buffer controller 52 to fetch the data from the frame buffer 51 . Similarly, the first image processing result generated by the image processing module 54 is compressed by the frame buffer encoding module 55 and then stored in the frame buffer 51 . Same as the frame buffer 31, the frame buffer 51 also has a first temporary storage area corresponding to the first type of coding block and a second temporary storage area corresponding to the second type of coding block for each processing image block. storage area. The first temporary storage area has at least one first random access point, and the second temporary storage area has at least one second random access point. Thereby, even if the first type of coding block corresponding to the same image block has not been read, the second type of coding block can still be read from the frame buffer 51 independently.

Those with ordinary knowledge in the technical field of the present invention can understand that the scope of the present invention is not limited to the number of blocks, data size, block division method, memory space allocation method, etc. in the above examples.

Practically, each coding block can be further divided into multiple prediction blocks as shown in FIG. 6 . In the example of FIG. 6, a coding block includes four prediction blocks. For example, assuming that the length and width of each image block 4 as the basic unit of dynamic image coding is 32 pixels*16 pixels, the length and width of each coding block can be 16 pixels*4 pixels, and each prediction area The length and width of a block may be 4 pixels*4 pixels. The prediction block can be set as a minimum compression/decompression unit block. That is to say, each prediction block among the plurality of prediction blocks has a corresponding piece of compressed information, such as a compressed bit length.

It should be noted that if the encoded block is compressed by frame buffer encoding before being stored in the frame buffer 31, 51, it can be decompressed by a frame buffer decoding module (not shown) before being taken out for use. In one embodiment, the frame buffer encoding modules 35 and 55 selectively apply frame buffer encoding to the image processing results output by the image processing modules 34 and 54, and determine whether to compress an image processing result according to a compression efficiency threshold . For example, assuming that the compression efficiency threshold is 70%, the frame buffer encoding module 35 may evaluate whether the compressed data volume is less than 70% of the uncompressed data volume before compressing an image processing result. If not, the frame buffer encoding module 35 does not apply compression to the image processing result. The advantage of this approach is that it can save the power and time required for frame buffer encoding/frame buffer decoding of image processing results with poor compression efficiency.

Another embodiment of the present invention is an image processing method, which is used to cooperate with a frame buffer to perform an image processing procedure for a plurality of image blocks, the flow chart of which is shown in FIG. 7 . Each of the plurality of image blocks includes a plurality of first-type coding blocks and at least one second-type coding block. The frame buffer is provided with a first temporary storage area and a second temporary storage area corresponding to a target image block. The first temporary storage area has at least one first random access point. The second temporary storage area has at least one second random access point independent of the at least one first random access point. First, step S71 is to perform image processing on a plurality of first-type coding blocks in the target image block to generate a set of first image processing results. Next, step S72 is to apply frame buffer encoding to the set of first image processing results to generate a set of first frame buffer encoding results. Subsequently, step S73 is to store the group of first frame buffer encoding results into the first temporary storage area in the frame buffer. Step S74 is to store at least one coding block of the second type of the target image block into the second temporary storage area in the frame buffer. It should be noted that step S74 may exist independently of steps S71 - S73 , that is, its execution time may not be limited by the execution time of steps S71 - S73 .

Those skilled in the art of the present invention can understand that the various possible changes described in the introduction of the image processing systems 300 and 500 can also be applied to the image processing method in FIG. 7 , and the details will not be repeated here. The concept of the present invention can be applied to video and audio compression coding technologies such as H.264, High Performance Video Coding (HEVC), but not limited thereto.

Through the above detailed description of the preferred embodiments, it is hoped that the features and spirit of the present invention can be described more clearly, rather than limiting the scope of the present invention by the preferred embodiments disclosed above. On the contrary, the intention is to cover various changes and equivalent arrangements within the scope of the claimed patent scope of the present invention.

Claims (20)

1. An image processing system for performing image processing on a plurality of image blocks, each of the plurality of image blocks includes a plurality of first-type coding blocks and at least one second-type coding region block, the image processing system includes: An image processing module, which performs image processing on a plurality of first-type coding blocks in a target image block to generate a set of first image processing results; A frame buffer encoding module, which applies frame buffer encoding to the set of first image processing results to generate a set of first frame buffer encoding results; A frame buffer, corresponding to the target image block, is provided with a first temporary storage area and a second temporary storage area, the first temporary storage area has at least one first random access point, and the second temporary storage area having at least one second random access point independent of the at least one first random access point; and a frame buffer controller for storing the group of first frame buffer encoding results into the first temporary storage area of the frame buffer, and causing at least one second type of encoding block of the target image block to be stored in the The second scratchpad area of the framebuffer. 2. The image processing system according to claim 1, wherein the frame buffer controller makes the set of first frame buffer encoding results from a first random access point in the at least one first random access point are continuously stored in the first temporary storage area without gaps. 3. The image processing system according to claim 1, wherein the group of first frame buffer encoding results comprises a plurality of frame buffer encoding results, and the first temporary storage area has a plurality of first random access points; The frame buffer controller enables the plurality of frame buffer encoding results to be stored in the first temporary storage area from different first random access points. 4. The image processing system as claimed in claim 1, wherein the frame buffer controller enables the at least one second-type coded block of the target image block from the at least one second random access point A second random access point is continuously stored in the second temporary storage area without interval. 5. The image processing system according to claim 1, characterized in that one, the target image block includes a plurality of second-type coding blocks, the second temporary storage area has a plurality of second random access points, the The frame buffer controller enables the plurality of coded blocks of the second type to be stored in the second temporary storage area from different second random access points. 6. The image processing system according to claim 1, wherein, corresponding to a reference image block adjacent to the target image block, the frame buffer is provided with a first reference temporary storage area and a second a reference temporary storage area, the first reference temporary storage area has at least one first random access point, and the second reference temporary storage area has at least one second random access point independent of the at least one first random access point ; when the image processing module performs image processing on the plurality of first-type coding blocks in the target image block, the frame buffer controller obtains from the at least one reference temporary storage area of the frame buffer The second random access point reads out at least one coded block of the second type from the reference image block and provides it to the image processing module for reference. 7. The image processing system according to claim 6, wherein the image processing module further performs image processing on the at least one coded block of the second type of the read reference image block to generate a set of The second image processing results make the group of second image processing results related to the group of first image processing results. 8. The image processing system according to claim 7, wherein the frame buffer encoding module also applies frame buffer encoding to the set of second image processing results to generate a set of second frame buffer encoding results; the frame The buffer controller causes the group of second frame buffer encoding results to be stored back into the frame buffer from the at least one second random access point in the frame buffer corresponding to the second reference temporary storage area of the reference image block device. 9. The image processing system according to claim 7, wherein the image processing module is based on the at least one second-type coding block of the reference image block and the plurality of the target image blocks The first type of encoding block is used to generate the set of first image processing results and the set of second image processing results. 10. The image processing system according to claim 1, wherein the frame buffer encoding module compresses the set of first image processing results according to a compression efficiency threshold to obtain the set of first frame buffer encoding results. 11. An image processing method, which is used to cooperate with a frame buffer to perform an image processing program on a plurality of image blocks, each of the plurality of image blocks includes a plurality of first-type coding blocks and At least one coding block of the second type, the frame buffer is provided with a first temporary storage area and a second temporary storage area corresponding to a target image block, and the first temporary storage area has at least one first random access point , the second temporary storage area has at least one second random access point, independent of the at least one first random access point, and the image processing method includes: (a) performing image processing on multiple first-type coding blocks in the target image block to generate a set of first image processing results; (b) applying frame buffer encoding to the set of first image processing results to generate a set of first frame buffer encoding results; (c) causing the set of first framebuffer encoding results to be stored in the first temporary storage area of the framebuffer; and (d) storing at least one second-type coding block of the target image block into the second temporary storage area of the frame buffer. 12. The image processing method according to claim 11, wherein the step (c) comprises: making the set of first frame buffer encoding results be accessed from a first random access point in the at least one first random access point Points are continuously stored in the first temporary storage area without gaps. 13. The image processing method according to claim 11, wherein the group of first frame buffer encoding results comprises a plurality of frame buffer encoding results, and the first temporary storage area has a plurality of first random access points; step (c) includes: enabling the plurality of frame buffer encoding results to be stored in the first temporary storage area from different first random access points. 14. The image processing method as claimed in claim 11, wherein step (d) comprises: making the at least one second-type coding block of the target image block from the at least one second random access point A second random access point is continuously stored in the second temporary storage area without interval. 15. The image processing method according to claim 11, wherein the target image block comprises a plurality of second-type coding blocks, and the second temporary storage area has a plurality of second random access points, the step (d) includes: allowing the plurality of second-type coded blocks to be stored in the second temporary storage area from different second random access points. 16. The image processing method according to claim 11, wherein, corresponding to a reference image block adjacent to the target image block, the frame buffer is provided with a first reference temporary storage area and a second a reference temporary storage area, the first reference temporary storage area has at least one first random access point, and the second reference temporary storage area has at least one second random access point independent of the at least one first random access point ; The image processing method further comprises before step (a): Read at least one second-type coding block in the reference image block from the at least one second random access point in the second reference temporary storage area of the frame buffer, and provide it to the step (a) performed Image Processing Reference. 17. The image processing method according to claim 16, further comprising: performing image processing on the at least one coded block of the second type of the read reference image block to generate a set of second image processing results, so that the set of second image processing results is related to the set of first image processing results result. 18. The image processing method according to claim 17, further comprising: Apply frame buffer encoding to the set of second image processing results to generate a set of second frame buffer encoding results; and The set of second frame buffer encoding results are stored back into the frame buffer from the at least one second random access point in the frame buffer corresponding to the second reference temporary storage area of the reference image block. 19. The image processing method according to claim 17, wherein step (a) comprises: combining the at least one second-type coding block of the reference image block and the target image block A plurality of coding blocks of the first type are used to generate the set of first image processing results and the set of second image processing results. 20. The image processing method according to claim 11, wherein step (b) comprises: compressing the set of first image processing results according to a compression efficiency threshold to obtain the set of first frame buffer encoding results.