CN102333212B - Bilinear two-fold upsampling method and system thereof - Google Patents

Abstract

The invention discloses a bilinear twice upsampling method and system, and belongs to the technical fields of image processing, video encoding and decoding, and the like. Existing bilinear upsampling methods are inefficient. The present invention firstly applies for four buffers, which are respectively the first buffer, the second buffer, the third buffer and the fourth buffer; then the image to be sampled is input to the first buffer; Perform horizontal pixel sampling, vertical pixel sampling and adjacent four-pixel sampling on the image to be sampled; store the horizontal pixel sampling result in the second buffer, store the vertical pixel sampling result in the third buffer, and store the adjacent four-pixel sampling result into the fourth buffer; finally, the pixels in the four buffers are interleaved, and the sampled image is output. The present invention is mainly applied to bilinear double upsampling processing on images.

Description

A bilinear twice upsampling method and system

technical field

The invention belongs to the technical fields of image processing, video encoding and decoding, etc., and specifically relates to a bilinear twice upsampling method and system.

Background technique

In recent years, with the continuous development of image processing and video codec technology, people have higher and higher requirements for the flexibility of image and video presentation. For example, zooming in and out of images, stretching and compressing videos, all of which involve up-down sampling technology. Especially in scalable video coding, up and down sampling is required inside the codec to support the feature of spatial scalability.

Image upsampling is the process of producing a higher resolution from a lower resolution image, that is, increasing the image resolution. There are many kinds of upsampling techniques, from the simplest nearest neighbor sampling to the more complex Lanczos sampling. Nearest neighbor sampling has the least amount of computation and is easy to implement, but the effect is the worst. The Lanczos sampling method is an alternative to BicubicResize, which can provide more accurate and sharper image quality, but its procedure is complicated and time-consuming. For image processing, it is reasonable to use more complex and time-consuming sampling methods in order to improve image quality. However, for video applications, the requirements of real-time and synchronization are the first. Bilinear upsampling is a relatively simple upsampling algorithm, and it is widely used because of its faster speed and better sampling effect.

The output pixel value of the bilinear sampling method is the average value of its 2×2 field sampling points in the input image, that is, the new pixel value is calculated by weighted average according to the pixel values of the four nearest surrounding points. It interpolates both horizontally and vertically based on the gray values of the surrounding 4 pixels. A typical application of bilinear upsampling is twice upsampling. The existing bilinear upsampling method is a general method for multiple upsampling, and multiplication is used to calculate the sampling result. Since the multiplication calculation is time-consuming and slow, the sampling efficiency is low.

Contents of the invention

Aiming at the defects in the prior art, the technical problem to be solved by the present invention is to provide a high-efficiency bilinear double upsampling method and system.

In order to solve the problems of the technologies described above, the technical scheme adopted in the present invention is as follows:

A bilinear twice upsampling method, comprising the following steps:

(1) Apply for four buffer zones, namely the first buffer zone, the second buffer zone, the third buffer zone and the fourth buffer zone;

(2) Input the image to be sampled into the first buffer;

(3) Traversing the image to be sampled, respectively performing horizontal pixel sampling, vertical pixel sampling and adjacent four-pixel sampling on the image to be sampled; storing the horizontal pixel sampling result in the second buffer, and storing the vertical pixel sampling result in the third buffer , storing the sampling results of adjacent four pixels into the fourth buffer;

(4) Arrange the pixels in the four buffers in a staggered manner, and output the sampled image.

A bilinear upsampling system by two times, including

A caching device for applying for a buffer, the caching device applies for four buffers, which are respectively the first buffer, the second buffer, the third buffer and the fourth buffer; the first buffer is used for caching For the image to be sampled, the second buffer is used for caching the pixels obtained by horizontal pixel sampling, the third buffer is used for caching the pixels obtained by vertical pixel sampling, and the fourth buffer is used for caching the pixels obtained by adjacent four-pixel sampling ;

An input device for inputting the image to be sampled into the first buffer;

It is used to perform horizontal pixel sampling, vertical pixel sampling and adjacent four-pixel sampling on the image to be sampled, and store the horizontal pixel sampling result in the second buffer, store the vertical pixel sampling result in the third buffer, and store the adjacent four-pixel The sampling result is stored in the sampling device of the fourth buffer zone;

An output device for interleaving the pixels in the four buffers and outputting the sampled image.

According to the method and system of the present invention, the image to be sampled is subjected to horizontal pixel sampling, vertical pixel sampling and adjacent four pixel sampling, and the sampling results are stored in different buffers, and then the pixels in the four buffers are arranged in a staggered manner. In this way, the efficiency of image bilinear upsampling is greatly improved. Moreover, in the sampling process, the multiplication calculation is replaced by calculating the average value of adjacent pixels, thereby reducing the calculation time and speeding up the sampling speed.

Description of drawings

Fig. 1 is the structural block diagram of described bilinear twice upsampling system in the specific embodiment;

Fig. 2 is a flow chart of the bilinear double upsampling method described in the specific embodiment;

Fig. 3 is a schematic diagram of buffer start address 16 byte alignment in the specific embodiment;

Fig. 4 is a schematic diagram of pixel distribution of an image to be sampled in a specific embodiment;

Fig. 5 is a schematic diagram of the staggered arrangement of pixels in four buffer areas in a specific embodiment.

Detailed ways

The present invention will be described in detail below in conjunction with specific embodiments and accompanying drawings.

FIG. 1 shows a structural block diagram of the bilinear twice upsampling system in this embodiment. The system includes a buffer device 12 , an input device 11 connected to the buffer device 12 and a sampling device 13 , and an output device 14 connected to the sampling device 13 .

The cache device 12 is used to apply for a buffer area. In this embodiment, four buffer areas are applied for, which are respectively named as the first buffer area (O buffer area), the second buffer area (H buffer area), and the third buffer area (V buffer area). ) and the fourth buffer (HV buffer). The O buffer is used to cache the image to be sampled, the H buffer is used to cache the pixels obtained by horizontal pixel sampling, the V buffer is used to cache the pixels obtained by vertical pixel sampling, and the HV buffer is used to cache the pixels obtained by adjacent four-pixel sampling.

The input device 11 is used for inputting the image to be sampled into the O buffer.

The sampling device 13 is used to perform horizontal pixel sampling, vertical pixel sampling and adjacent four-pixel sampling on the image to be sampled, and store the horizontal pixel sampling result into the H buffer, store the vertical pixel sampling result into the V buffer, and store the adjacent four pixel sampling results into the H buffer. The pixel sampling result is stored in the HV buffer.

The output device 14 is used for interleaving the pixels in the four buffers and outputting the sampled image.

FIG. 2 shows a flowchart of a method for performing bilinear upsampling by two times using the system shown in FIG. 1 . The method includes the following steps:

(1) The cache device 12 first applies for four buffers, which are respectively named O buffer, H buffer, V buffer and HV buffer. The size of each buffer is not smaller than the size of the image to be sampled, and is a multiple of 16.

After applying for four buffers, it also includes the operation of 16-byte alignment for the start address of each buffer. In this embodiment, the 16-byte alignment operation adopts the following method:

For 32-bit applications, first apply for an additional 19 bytes of space when applying for each buffer; then record the starting address P of the buffer, and align P+4 bytes to 16 bytes upwards to obtain Q. For 64-bit applications, when applying for each buffer, additionally apply for 23 bytes of space; record the starting address P of the buffer, and align P+8 bytes to 16 bytes to obtain Q. Among them, 4 bytes and 8 bytes are the size of the pointer. The end address of the buffer is delayed accordingly to ensure that the size of the buffer remains unchanged. After 16-byte alignment processing, for 32-bit applications, write P into the front 4 bytes of Q; for 64-bit applications, write P into the front 8 bytes of Q.

For example, as shown in Figure 3, set the starting position P of a certain buffer zone as 10, move P to the right by 4 bytes to reach the R position, and R is 14; compared with 16, the difference is 2 bytes, and then to After moving 2 bytes to the right, it reaches the Q position. At this time, the current starting position Q of the buffer is 16, which achieves the purpose of alignment. Since 19 or 23 bytes are applied for, the size of the buffer can remain the same.

When releasing the buffer, subtract 4 bytes from Q (8 bytes for 64-bit applications) to obtain P, and release P.

(2) The input device 11 inputs the image to be sampled (original image) into the O buffer as upsampling source data.

(3) The sampling device 13 traverses the image to be sampled, and respectively performs horizontal pixel sampling, vertical pixel sampling and adjacent four-pixel sampling on the image to be sampled. Store the horizontal pixel sampling results into the H buffer, store the vertical pixel sampling results into the V buffer, and store the adjacent four pixel sampling results into the HV buffer.

Taking the original image shown in Figure 4 as an example, its pixels are a 10×10 matrix, and the intersections of rows and columns represent pixels. The process of horizontal pixel sampling is as follows: traverse the image to be sampled, first calculate the average value of pixel 00 (the pixel in row 0, column 0, and the meaning of other pixels in the same way) and pixel 01, and store it in the H buffer The position of row 0, column 0; then calculate the average value of pixel 01 and pixel 02, and store it in the position of row 0, column 1 in the H buffer; then calculate the average value of pixel 02 and pixel 03, and store it in the H buffer The position of row 0, column 2; and so on. After the first row is processed, the second row is processed in the same manner as the first row, until all rows are processed, and the horizontal pixel sampling ends. The sampling results stored in the H buffer are shown in the table below:

A(00,01) A(01,02) A(02,03) A(03,04) A(04,05) A(05,06) A(06,07) A(07,08) A(08,09)

A(10, 11) A(11, 12) A(12, 13) A(13, 14) A(14, 15) A(15, 16) A (16, 17) A(17, 18) A(18, 19) A(20, 21) A(21, 22) A(22, 23) A(23, 24) A(24, 25) A(25, 26) A(26, 27) A(27, 28) A(28, 29) A(30,31) A(31,32) A(32,33) A(33,34) A(34,35) A(35,36) A(36,37) A(37,38) A(38,39) A(40, 41) A(41, 42) A (42, 43) A(43,44) A(44, 45) A(45, 46) A (46, 47) A(47, 48) A(48, 49) A(50, 51) A(51, 52) A(52, 53) A(53, 54) A(54, 55) A(55, 56) A(56, 57) A(57, 58) A(58, 59) A(60,61) A(61,62) A(62,63) A(63,64) A(64,65) A(65,66) A(66,67) A(67,68) A(68,69) A(70,71) A(71,72) A(72,73) A(73,74) A(74,75) A(75, 76) A(76,77) A(77,78) A(78,79) A(80,81) A(81,82) A(82,83) A(83,84) A(84,85) A(85,86) A(86,87) A(87,88) A(88,89) A(90,91) A(91,92) A(92,93) A(93,94) A(94,95) A(95,96) A(96,97) A(97,98) A(98,99)

Wherein, A represents the average value, A(00,01) represents the average value of pixel 00 and pixel 01, A(01,02) represents the average value of pixel 01 and pixel 02, and so on.

The process of vertical pixel sampling is as follows: traverse the image to be sampled, first calculate the average value of pixel 00 and pixel 10, and store it in the position of row 0 and column 0 in the V buffer; then calculate the average value of pixel 10 and pixel 20, and store it in The position of row 1, column 0 in the V buffer; then calculate the average value of pixel 20 and pixel 30, and store it in the position of row 2, column 0 in the V buffer; and so on. After the first column is processed, the second column is processed in the same manner as the first column, until all columns are processed, and the vertical pixel sampling ends. The sampling results stored in the V buffer are shown in the table below:

Wherein, A represents the average value, A(00, 10) represents the average value of pixel 00 and pixel 10, A(10, 20) represents the average value of pixel 10 and pixel 20, and so on.

When calculating the average value of pixels by vertical pixel sampling, the parallel calculation can be performed by using the parallel average instruction, which can improve the calculation efficiency. For example, the SSE (Streaming SIMD Extensions, Single Instruction Multiple Data Stream Extensions) instruction set or the PAVGB instruction in the SSE2 instruction set is used. If the PAVGB parallel averaging instruction in the SSE instruction set is used, one instruction can process 8 pixels synchronously; if the PAVGB parallel averaging instruction in the SSE2 instruction set is used, one instruction can simultaneously process 16 pixels.

The process of sampling adjacent four pixels is as follows: traverse the image to be sampled, first calculate the average value of pixel 00, pixel 01, pixel 10 and pixel 11, and store it in the position of row 0 and column 0 in the HV buffer. The so-called adjacent four pixels refer to four adjacent pixels that can form four vertices of a square, such as pixel 00 , pixel 01 , pixel 10 and pixel 11 in FIG. 4 . Then calculate the average value of pixel 01, pixel 02, pixel 11 and pixel 12, and store it in the position of row 0 and column 1 in the HV buffer. Then calculate the average value of pixel 02, pixel 03, pixel 12 and pixel 13, and store it in the position of row 0 and column 2 in the HV buffer. and so on. After the first and second rows have been processed, the second and third rows are processed in the same manner as the first and second rows. And so on until processing is complete.

Since the average value of two horizontally adjacent pixels has been calculated when performing horizontal pixel sampling, the result of horizontal sampling can be used when calculating the average value of four adjacent pixels. For example, to calculate the average value of pixel 00, pixel 01, pixel 10, and pixel 11, you can first take out the average value of pixel 00 and pixel 01 from the H buffer, and use A to represent it; then take the average value of pixel 10 and pixel 11, and use B means; then the average value of pixel 00, pixel 01, pixel 10 and pixel 11 can be obtained by calculating the average value of A and B. In this way, the calculation speed can be accelerated. The sampling results stored in the HV buffer are shown in the table below:

Among them, A represents the average value, A(00,01,10,11) represents the average value of pixel 00, pixel 01, pixel 10 and pixel 11, A(01,02,11,12) represents pixel 01, pixel 02, The average of pixel 11 and pixel 12, and so on.

When calculating the average value of pixels in adjacent four-pixel sampling, parallel computing can also be performed by using a parallel averaging instruction, for example, using a parallel averaging instruction in the SSE instruction set or the SSE2 instruction set. If the parallel average instruction in the SSE instruction set is used, one instruction can process 8 pixels synchronously; if the parallel average instruction in the SSE2 instruction set is used, one instruction can process 16 pixels synchronously.

(4) The output device 14 arranges the pixels in the four buffers in a staggered manner, and outputs the sampled image.

Insert the average value of these two pixels in the H buffer between two adjacent pixels in each row of the image in the O buffer, and insert the average of the two pixels in the V buffer between two adjacent pixels in each column value, interpolate the average value of the four pixels in the HV buffer at the center positions of the adjacent four pixels. After everything is arranged, the sampled image output is obtained.

For example, as shown in Figure 5, A, B, C, D, E, F, G, H, and I are the original image pixels in the O buffer, insert pixels A and B in the H buffer between pixels A and B , interpolate between pixels B and C the average of pixels B and C in the H buffer, and so on. Interpolate between pixels A and D the average of pixels A and D in the V buffer, between pixels D and G the average of pixels D and G in the V buffer, and so on. The average value of pixels A, B, D, and E at the centers of pixels A, B, D, and E in the HV buffer, and the centers of pixels B, C, E, and F into the HV buffer Average of pixels B, C, E, and F, and so on.

The pixels in the four buffers can be arranged in parallel by using the MMX (Multi Media eXtension) instruction set or the PUNPCKHBW and PUNPCKLBW parallel integer interleaving instructions in the SSE2 instruction set, which can improve the efficiency of pixel arrangement. If the parallel integer interleaving instruction in the MMX instruction set is used, one instruction can process 8 pixels synchronously; if the parallel integer interleaving instruction in the SSE2 instruction set is used, one instruction can process 16 pixels synchronously.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (12)

1. bilinearity twice top sampling method may further comprise the steps:

(1) four buffering areas of application are respectively first buffering area, second buffering area, the 3rd buffering area and the 4th buffering area; In described four buffering areas, the size of each buffering area is not less than the size for the treatment of sampled images, and is 16 multiple;

(2) will treat that sampled images is input to first buffering area;

(3) traversal is treated sampled images, treats sampled images respectively and carries out pixels across sampling, vertically pixel sampling and adjacent four pixels sampling; Deposit the pixels across sampled result in second buffering area, deposit vertical pixel sampling result in the 3rd buffering area, deposit the adjacent four pixels sampled result in the 4th buffering area;

(4) pixel in four buffering areas is carried out staggered, output sampling back image; It is as follows that pixel in four buffering areas is carried out staggered method:

Insert the mean value of these two pixels in second buffering area between two neighbors of the every row of image in first buffering area, between two neighbors of every row, insert the mean value of these two pixels in the 3rd buffering area, insert the mean value of these four pixels in the 4th buffering area at the center position of adjacent four pixels.

2. the method for claim 1, it is characterized in that: step also comprises the operation of the initial address of four buffering areas being carried out 16 byte-aligned in (1).

3. method as claimed in claim 2, it is characterized in that: the method for the address of each buffering area being carried out 16 byte alignment operation is as follows:

For 32 application programs, when each buffering area of application, additionally apply for 19 byte spaces;

The initial address p of log buffer will carry out the upwards alignment of 16 bytes after p+4 the byte;

Prolong after the end address of buffering area is corresponding, guarantee the big or small constant of buffering area.

4. method as claimed in claim 2, it is characterized in that: the method for the address of each buffering area being carried out 16 byte alignment operation is as follows:

For 64 application programs, when each buffering area of application, additionally apply for 23 byte spaces;

The initial address p of log buffer will carry out the upwards alignment of 16 bytes after the P+8 byte;

Prolong after the end address of buffering area is corresponding, guarantee the big or small constant of buffering area.

5. the method for claim 1 is characterized in that: the method that the sampling of pixels across described in the step (3) is adopted is as follows:

Traversal is treated sampled images, treats each row pixel of sampled images, calculates the mean value of adjacent two pixels, as the pixels across sampled result.

6. the method for claim 1 is characterized in that: vertically the method that adopts of pixel sampling is as follows described in the step (3):

Traversal is treated sampled images, treats each row pixel of sampled images, calculates the mean value of adjacent two pixels, as vertical pixel sampling result.

7. method as claimed in claim 6 is characterized in that: the mean value of adjacent two pixels of described calculating adopts parallel averaging instruction to carry out parallel computation.

8. the method for claim 1 is characterized in that: the method that the sampling of adjacent four pixels described in the step (3) is adopted is as follows:

Traversal is treated sampled images, treats four adjacent pixels of sampled images, and calculating mean value is as the adjacent four pixels sampled result.

9. method as claimed in claim 5 is characterized in that: the method that the sampling of adjacent four pixels described in the step (3) is adopted is as follows:

Obtain the mean value A of top two pixels in the adjacent four pixels and the mean value B of following two pixels, calculate the mean value of A and B, as the mean value of adjacent four pixels.

10. method as claimed in claim 9 is characterized in that: the mean value of described adjacent four pixels adopts parallel averaging instruction to carry out parallel computation.

11. method as claimed in claim 10 is characterized in that: describedly pixel in four buffering areas is carried out staggered method be: adopt the staggered instruction of parallel integer to walk abreast staggered to pixel in four buffering areas.

12. a bilinearity twice up-sampling system comprises

The buffer storage (12) that is used for the application buffering area, four buffering areas of described buffer storage (12) application are respectively first buffering area, second buffering area, the 3rd buffering area and the 4th buffering area; Described first buffering area is used for buffer memory and treats sampled images, described second buffering area is used for buffer memory pixels across sampling gained pixel, described the 3rd buffering area is used for the vertical pixel sampling gained of buffer memory pixel, and described the 4th buffering area is used for buffer memory adjacent four pixels sampling gained pixel; In described four buffering areas, the size of each buffering area is not less than the size for the treatment of sampled images, and is 16 multiple;

Be used for to treat that sampled images is input to the input unit of first buffering area (11);

Be used for treating sampled images and carry out pixels across sampling, vertically pixel sampling and adjacent four pixels sampling, and deposit the pixels across sampled result in second buffering area, deposit vertical pixel sampling result in the 3rd buffering area, the adjacent four pixels sampled result is deposited in the sampling apparatus (13) of the 4th buffering area;

And staggered for four buffering area pixels are carried out, the output device (14) of output sampling back image; It is as follows that output device carries out staggered mode with pixel in four buffering areas:

Insert the mean value of these two pixels in second buffering area between two neighbors of the every row of image in first buffering area, between two neighbors of every row, insert the mean value of these two pixels in the 3rd buffering area, insert the mean value of these four pixels in the 4th buffering area at the center position of adjacent four pixels.

Images