CN102214356A - NVIDIA graphics processing unit (GPU) platform-based best neighborhood matching (BNM) parallel image recovering method - Google Patents

Abstract

The invention discloses a parallel image restoration method based on NVIDIA GPU platform, comprising the following steps: step 100, scanning the image with bad blocks, finding the bad blocks and marking them; step 110 , according to the CUDA programming technology, design the BNM parallel algorithm based on the GPU platform; Step 120, according to the optimization technology of CUDA, combine the parameters of the GPU model adopted and the height of the image size and the bad block rate, parallelize the BNM designed in the step 110 The algorithm is improved; step 130, according to the optimization result of step 120, find the algorithm with the best performance to restore the image. The invention is based on the NVIDIA GPU platform and adopts CUDA parallel programming technology to realize the parallel image restoration algorithm of BNM.

Description

Optimal Neighborhood Matching Parallel Image Restoration Method Based on NVIDIA GPU Platform

technical field

The invention relates to an optimal neighborhood matching parallel image restoration method based on an NVIDIA GPU platform, relates to the field of graphics and images, in particular to image restoration.

Background technique

In recent decades, the research and application of digital image processing and network media have developed rapidly, and people have higher and higher requirements for the quality of digital images and videos. Transmission errors will inevitably occur during the transmission of images. For this reason, many methods have been proposed to improve the reliability of the channel and expect to obtain images and videos with guaranteed quality. Another way to solve this problem is to use the received image from the perspective of the image itself, and restore the damaged pixel with the normally received image pixel according to the correlation of the image itself. This method is the main idea of error concealment technology. Inspired by fractal theory, the optimal neighborhood matching algorithm utilizes the similarity between image blocks to restore and reconstruct images. The best neighborhood matching algorithm restores the image damaged during the transmission process, the image restoration effect is good, and a high-quality restored image can be obtained, but the algorithm itself has a large amount of calculation and the execution efficiency is not high.

In the field of graphics and images, error compensation technology (Error Concealment) can be used to restore images of bad blocks. Among many image restoration methods, the Best Neighborhood Matching (BNM, Best Neighborhood Matching) algorithm works better. This algorithm not only utilizes the pixel information of adjacent blocks, but also utilizes the pixel information of remote blocks, so the restoration effect is better.

Figure 1 illustrates the operation process of the BNM algorithm. The black block represents a bad block (size x1*x2 pixels), and the pixels around the bad block ((x1+2)*(x2+2) pixels) are called boundaries window. With the boundary window as the center, a search area (called the search window) with a size of r1*r2 is formed. In the search area, the far-end window is continuously formed at a step of 1 pixel each time. The far-end window and the local window have the same Same shape and size. Find a remote window with the same size as the bad block and the best match within the bad block search window to replace the bad block. Search step by step from the upper left corner, move one step at a time, and find the best matching remote window block after the search is completed. In this way, all bad blocks in an image can be recovered one by one.

Since NVIDIA launched the graphics processor G80 (including 128 stream processor SPs, and the latest G200 includes 240 stream processor SPs) in 2006, GPUs have been more powerful than CPUs in certain large-scale parallel computing applications. It is said that the performance improvement can reach more than 100 times. Especially since NVIDIA launched CUDA SDK 1.1, a development platform for GPUs in May 2008, parallel computing based on GPU platforms has been promoted on a large scale. CUDA provides a unified computing device architecture for GPU computing, enabling users to easily integrate GPU programming into traditional programming tools (such as: Visual Studio, gcc, etc.) and languages (such as: C, C++, FORTRAN, etc.). In just one year, CUDA has been applied to accelerate many problems in the field of large-scale parallel computing, such as in image processing, physical model simulation (such as computational fluid dynamics), engineering and financial simulation and analysis, biomedical engineering, database, It has good applications in data mining, searching, sorting, etc., and has achieved 1-2 orders of magnitude acceleration in many applications.

GPU has more transistors for data processing instead of processing data cache and instruction control, which means that GPU has huge parallel computing capabilities. In a GPU, a single data processing unit is a stream processor (SP), 8 stream processors form a stream processor group (SM), a GPU has multiple stream processor groups, and each stream processor group has 8 stream processors, some caches (texture memory, constant memory, shared memory) and two special function units (SFU). The off-chip global memory is used to store data and realize data transfer between CPU and GPU.

CUDA (Compute Unified Device Architecture, Compute Unified Device Architecture) is used as a parallel programming language for GPU. CUDA programming calls the CPU a host, and the GPU as a coprocessor is called a device. In CUDA programming, multiple threads are executed on the GPU at the same time. Multiple threads form a thread block, and multiple blocks are organized into a grid. In addition, every 32 threads form a warp. The optimization techniques commonly used in CUDA programming include reasonable grid configuration, enough warps on each SM to hide access delays, combined access to global memory, use of shared memory, use of texture memory and constant memory, and reasonable use of registers etc.

The Best Neighborhood Matching (BNM) algorithm has a better recovery effect among all image restoration algorithms, but it has a large amount of calculation and needs to improve the operation efficiency. At present, there are many researches on BNM at home and abroad, mainly for the improvement of the full search method of BNM, and different algorithms are proposed, such as: Two-Step Best Neighborhood Matching (Two-Step Best Neighborhood Matching, TSBNM) algorithm , BNM algorithm based on evolution strategy (ES_BNM), jump and look best neighborhood matching (Jump and Look Best Neighborhood Matching, JLBNM) algorithm, improved jump and look BNM algorithm (Improved JLBNM, IJLBNM), etc. These improvements are only based on the improvement of the algorithm on the CPU. Although the performance has been improved several times, it is becoming more and more difficult to further improve the performance.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention provides a BNM parallel image restoration method based on a GPU platform. For images with bad blocks, the BNM parallel error compensation algorithm based on the GPU platform is used for image restoration.

A kind of optimal neighborhood matching parallel image restoration method based on NVIDIA GPU platform, comprising the following steps:

Step 100, scanning the image with bad blocks, finding the bad blocks and marking them;

Step 110, according to CUDA programming technology, design the BNM parallel algorithm based on GPU platform;

Step 120, improve the BNM parallel algorithm designed in step 110 according to the optimization technology of CUDA, in combination with the parameters of the GPU model adopted and the image size and the level of bad block rate;

Step 130, according to the optimization result of step 120, find the algorithm with the best performance to restore the image.

In the image restoration method, in the step 120, one of the following methods or a combination thereof is used for optimization:

A1, according to the resources of the selected GPU, set the grid, block, and thread structures of the parallel program so that it can hide the access delay of the global memory.

A2, the data in the global memory accessed by threads of the same warp satisfies coalescing access.

A3, use shared memory to store data used multiple times by each thread in the same thread block, and reduce access to global memory through the use of shared memory.

A4, each stream processor group SM on the GPU has a large number of registers, and the access to the global memory is reduced by fully using the registers to store data used multiple times in a thread.

A5. The search radius in the BNM algorithm determines the recovery effect, that is, the PSNR value. Setting the search radius not only meets high quality, but also improves performance; at the same time, setting the search threshold reasonably can also improve the recovery speed.

In the image restoration method, in step A1, the resource of the GPU refers to the total number of stream processors SP in the selected GPU, the capacity of the global memory, the access delay, the computing power, the maximum number of blocks allowed by each grid, The maximum number of threads allowed per block, the maximum number of threads allowed per stream processor group SM.

In the image restoration method, in step A2, the warp refers to a group of threads (32 threads) running simultaneously in the same thread block, and a warp is a thread scheduling unit.

In the image restoration method, in step A3, the shared memory refers to a cache structure on the GPU. The shared memory is located in the GPU chip, and its access speed is fast. Fully using the shared memory can improve the parallel computing speed.

In the image restoration method, in step A4, the threshold means that a specific value is given during the BNM search process, and the search is stopped once the obtained MSE value reaches the threshold.

The invention is based on the NVIDIA GPU platform and adopts CUDA parallel programming technology to realize the parallel image restoration algorithm of BNM. At the same time, because the traditional serial BNM algorithm is slow in execution, it cannot meet the real-time recovery in video playback, but the present invention can meet the real-time recovery of high-definition images.

Description of drawings

The operation process of Fig. 1BNM algorithm;

Fig. 2 uses the parallel BNM algorithm structural diagram of combining memory access;

Fig. 3 uses the parallel BNM algorithm structural diagram of shared memory;

Fig. 4 uses the parallel BNM algorithm structural diagram of shared memory;

Fig. 5 is a structural diagram of the specific implementation of the present invention.

Detailed ways

The technical solutions of the present invention will be described in detail below in conjunction with the accompanying drawings and preferred embodiments. The following examples are only used to illustrate and explain the present invention, but not to limit the technical solution of the present invention.

The present invention mainly realizes the parallel image restoration algorithm based on the GPU platform, and in the specific implementation process, it mainly uses various optimization techniques in CUDA rationally, and combines the specifically selected GPU models.

For example, we choose the GPU of NVIDIA GeForce GTX275 and the i7920CPU of Intel Corporation. The detailed parameters are shown in Table 1.

Table 1 Test environment

Referring to Figure 5, the BNM parallel image restoration process based on the GPU platform includes the following steps:

Step 100, scanning the image with bad blocks, finding the bad blocks and marking them;

For example, we choose an image of 1024*1024 pixels for testing, assuming that the bad block size is 8*8 pixels (x1*x2), and the bad block rate is 15%, then the total number of bad blocks is n=(1024*1024)/( 8*8)*15%=2458, the boundary window is (x1+2)*(x2+2)=(8+2)*(8+2)=10*10 pixels, and the search window is r1*r2= 80*80 pixels.

Step 110, according to CUDA programming technology, design the BNM parallel algorithm based on GPU platform;

For the above example, the P_BNM algorithm is designed, and each thread handles the recovery of a bad block, that is, finds the best matching block (block with the smallest MSE value) in the 80*80 search window, and replaces it. In the process of searching, one pixel is moved at a time, so each thread has to compare 80*80 blocks with bad blocks (calculate the MSE value of the two). Suppose the number of threads in each thread block is m=128, then the total number of thread blocks is (n+m-1)/m=(2458+128-1)/128=20. The performance of the P_BNM algorithm is only 1.42 times higher (Intel i7vs GTX 275), mainly because the data accessed by threads in the same warp cannot meet the combined access, and there are only 20 thread blocks, while there are 30 on a stream processor group SM Stream processors are not conducive to the rational use of resources.

In this design method, there are as many threads as there are bad blocks, assuming that the number of bad blocks is n, and the number of threads in each thread block is m (the range of m is determined by the selected GPU model, and different types of GPU The maximum number of threads allowed for each thread block is also different), then the number of thread blocks in each grid is (n+m-1)/m (rounded). That is, the design of the thread block is (n, 1, 1), and the design of the grid is ((n+m-1)/m, 1, 1)

Step 120, improve the BNM parallel algorithm designed in step 110 according to the optimization technology of CUDA, in combination with the parameters of the GPU model adopted and the image size and the level of bad block rate;

In the BNM parallel algorithm based on the GPU platform proposed by the present invention, the optimization method in the CUDA programming technology is adopted, and the execution performance of the serial BNM algorithm is improved by the optimization technology, and the specific optimization process is given as an example as follows:

P_BNM_CA algorithm:

The P_BNM algorithm does not consider the optimization technology of CUDA, and only performs simple parallelism. Since the P_BNM algorithm cannot meet the merged access, the performance is not very good, and the merged access can be satisfied by further subdividing the tasks of each thread - the P_BNM_CA algorithm.

When using the P_BNM_CA algorithm to restore an image, in an image with bad blocks (each bad block is an n1*n2 block), the distribution of bad blocks is random, so the distance between bad blocks and bad blocks The data does not have any dependencies, which conforms to the characteristics of GPU parallel processing. Simultaneously in order to meet the combined access of the global memory, as shown in Figure 2, the number of bad blocks set by the present invention is the number of thread blocks in the grid, and each thread block processes a bad block, and the number of threads of each thread block is the BNM algorithm The search radius in (assuming that the search radius of the BNM algorithm is R, the number of threads in each thread block is also R), each thread processes a column in the search range, so that the data accessed by the 32 threads constituting the same warp is A method that can satisfy global memory coalescing access. Suppose a thread block handles the recovery process of a bad block. The size of each thread block is 80. Each thread in each thread block only processes the search work of one column within the 80*80 search range. Each thread finds a column in a column. The block with the smallest MSE, 80 threads in the same thread block and the block with the smallest MSE are the best matching blocks. The total number of thread blocks is n=2458.

This method makes the data in the search window accessed by threads in the same warp have continuity, so it satisfies the combined access well. At the same time, the total number of thread blocks is 2458, which can make good use of the stream processor group SM. Computing power, the performance of this algorithm reaches 18.38 times of the basic BNM parallel algorithm.

In the recovery process of each bad block, in the 80*80 search window, it is necessary to calculate the MSE of 80*80 replacement windows and the bad block window, and the pixel value of the bad block window is used every time, 8* The number of pixels in a circle of 8 windows is 36. Therefore, we can use shared memory to store these 36 pixel values. The method is P_BNM_CA_SH, and the performance of this method is increased to 22.22 times.

P_BNM_CA_SH algorithm:

The CUDA programming model recommends the use of shared memory to improve performance. In the serial BNM algorithm, each search within the search range of R*R must be compared with (n1+n2+2)*2 pixels around the n1*n2 bad block to find its MSE value. As shown in Figure 3, P_BNM_CA_SH transfers the value of storing these (n1+n2+2)*2 pixels to the shared memory on the basis of the P_BNM_CA algorithm using the combined memory access technology, and reduces the access to the global shared memory. Improve performance.

In addition, as shown in Figure 4, in each comparison process, the pixels in the upper row and the lower row will be accessed multiple times (generally n1 times), so shared memory can be used to store these pixels to reduce the need for global memory access to improve performance.

P_BNM_CA_SH_R algorithm:

Based on the P_BNM_CA_SH method, the performance can be further improved by modifying the search radius (at the same time, setting the threshold reasonably during the search can also improve the speed of recovery)——P_BNM_CA_SH_R. Appropriately reducing the search radius will not significantly reduce the quality of image restoration . For the above example, when the search radius is changed to 40 pixels, the performance is increased to 61.23 times, while the PSNR value is reduced from 34.70 to 34.56, which is only reduced by 0.14. Using P_BNM_CA_SH_R to restore the 1024*1024 pixel image takes only 28ms, which satisfies the real-time restoration very well.

Step 130, according to the above optimization steps, find the algorithm with the best performance to restore the image.

For the above example, the P_BNM_CA_SH_R method has the best performance. Therefore, the P_BNM_CA_SH_R method can be used to restore the 1024*1024 pixel-sized image with 15% damage.

It can be seen from the above test results provided by the present invention that the powerful parallel computing capability of the GPU can be used to speed up image restoration operations by tens or hundreds of times.

The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention, All should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (8)

1. the best neighborhood matching parallel image restoration methods based on NVIDIA GPU platform is characterized in that, may further comprise the steps:

Step 100 scans the image that bad piece is arranged, and finds the local of bad piece and it is carried out mark;

Step 110, according to the CUDA programming technique, design is based on the BNM parallel algorithm of GPU platform;

Step 120, according to the optimisation technique of CUDA, the height of the parameter of the GPU model that combination is simultaneously adopted and image size and bad piece rate improves the BNM parallel algorithm of design in the step 110;

Step 130, according to the optimization result of step 120, finding wherein, the best algorithm of performance recovers image.

2. image recovery method according to claim 1 is characterized in that, in the described step 120, adopts one of following method or its combination to be optimized:

A1 according to the resource of selected GPU, is provided with grid, the piece of concurrent program, the structure of thread, enables to hide the access delay of global storage.

A2, the data of the global storage of the thread accesses of same warp satisfy to merge visits.

A3 uses shared storage to deposit the data that each thread repeatedly uses in the same thread block, is used for reducing visit to global storage by making of shared storage.

A4 has a large amount of registers on each the stream handle group SM on the GPU, deposit the data of repeatedly using in the thread by sufficient use register and reduce visit to global storage.

A5, the search radius in the BNM algorithm have determined that the effect of recovering is the PSNR value, and the setting search radius had both satisfied high quality, and the lifting of performance is arranged again; Simultaneously rationally the threshold value during setting search also can improve the speed of recovery.

3. image recovery method according to claim 2, it is characterized in that, in the steps A 1, the resource of described GPU is meant stream handle SP sum among the selected GPU, the capacity of global storage, access delay, computing power, the largest block number that each grid allows, the maximum thread that each piece allows, the maximum thread that each stream handle group SM allows.

4. image recovery method according to claim 2 is characterized in that, in the steps A 2, described warp refers in same thread block one group of thread (32 threads) of operation simultaneously, and warp is the thread of thread.

5. image recovery method according to claim 2 is characterized in that, in the steps A 3, described shared storage refers to a kind of cache structure on the GPU, shared storage is positioned at the GPU sheet, and its access speed is fast, uses shared storage can improve parallel computing velocity fully.

6. image recovery method according to claim 2 is characterized in that, in the step f), described PSNR value refers to the signal to noise ratio (S/N ratio) of image.

7. image recovery method according to claim 2 is characterized in that, in the step f), described threshold value refers to a given specific value in the BNM search procedure, in case the value of the MSE that tries to achieve just stops search after reaching threshold value.

8. image recovery method according to claim 2 is characterized in that MSE is meant the mean variance of the piece of two identical sizes in the image.

Images