CN108876736A - A kind of image alias removing method based on FPGA - Google Patents

Abstract

The invention discloses a method for eliminating the ladder effect of an image based on FPGA, comprising the following steps: step A, performing grayscale conversion on the RGB image input to the FPGA to obtain a grayscale signal with a value of 0-255; step B, converting the converted The image preprocessing of the mean value filter is carried out on the grayscale image to reduce the sharp transformation of the grayscale value of the image; step C, the wavelet decomposition is performed on the grayscale image, and it is decomposed into a low-frequency component containing the basic information of the original _image and a low-frequency component containing the detailed information of the original image. The high-frequency component I _HH of the information, and decompose I _LL again to obtain its low-frequency component I _LL ; step D, use the Canny operator to process I _LH1 , and perform image reconstruction, and finally obtain image information that eliminates the staircase effect. This method is based on FPGA and eliminates the step effect of the input image signal. While ensuring the real-time performance of the image processing algorithm, it can also make full use of the customization functions of the FPGA IP core to meet more specific needs.

Description

An FPGA-Based Elimination Method of Image Staircase Effect

technical field

The invention belongs to the technical field of image processing, and relates to an FPGA-based image step effect elimination method, in particular, to a method for realizing image edge thinning by using FPGA, and in particular to the combination of wavelet transform and Canny operator to realize image Elimination of the ladder effect.

Background technique

Denoising is a basic problem in image processing. The general idea is to repair or restore noisy images by studying the characteristics of specific noises. It has important applications in image denoising, image segmentation, and image edge detection. Among them, the edge structure and texture information of the image can reflect the basic characteristics and important information of the image content, while the traditional filtering model always leads to the loss of edge information to a certain extent in the process of image denoising processing, so looking for an effective The method of image denoising effect and protecting edge information is very important. Due to the lack of prior information, the denoising problem is often ill-conditioned, so it is necessary to use mathematical methods such as partial differential equations (PDE), which can accurately reflect the constraint relationship between the derivatives of unknown variables with respect to time and space variables. By establishing the "energy function" first, and then obtaining Euler's equation by the variational method, the corresponding PDE is established by analogy with a certain physical process.

With the development of computer technology, traditional image denoising technologies are mostly implemented by PCs. Even if the denoising effect is relatively obvious, considering the operation of PC-based systems, its shortcomings are obvious, such as high cost, large size, poor stability, etc. , and FPGA has the advantages of parallel pipeline, this mechanism endows it with higher speed performance, and there are abundant logic resource units in its chip, FPGA structure design is flexible, flexible and changeable, which can guarantee real-time in most cases The research and development cycle based on FPGA is relatively short, and special IP cores can be designed based on requirements to facilitate the expansion of the system structure. At the same time, the joint development of FPGA, high-performance single-chip microcomputer and DSP can take into account the processing speed and computing performance, which can give Users bring a more smooth and high-quality sensory experience.

Contents of the invention

The purpose of the present invention is to provide a FPGA-based image step effect elimination method, based on the FPGA, to eliminate the step effect of the input image signal. The image processing algorithm is implemented on the FPGA hardware platform. While ensuring the real-time performance of the image processing algorithm, it can also make full use of the customization functions of the FPGA IP core to meet more specific needs. The combination of FPGA and image processing technology improves the practicality of system design, and with the continuous improvement of FPGA performance, its processing speed is getting faster and faster, more and more functional modules are integrated inside, and the performance of detection methods will become more and more it is good.

In order to achieve the above object, the solution of the present invention is:

A method for eliminating image ladder effect based on FPGA, comprising the steps:

Step A, the RGB image that is input to FPGA is carried out gray-scale conversion, obtains the gray-scale signal of value 0-255;

Step B, performing image preprocessing on the converted grayscale image by means of filtering to reduce the sharp transformation of the grayscale value of the image;

Step C, decomposing the grayscale image into a low-frequency component I _LL containing basic information of the original image and a high-frequency component I _HH containing detailed information of the original image, and decomposing I _LL again to obtain its low-frequency component I _LL ;

In step D, use the Canny operator to process _ILH1 and perform image reconstruction, and finally obtain image information that eliminates the staircase effect.

In the above steps A and B, the RGB image signal is input to the FPGA from the outside, and the clock synchronization signal is performed; the RGB color space image is passed through the conversion program, and the converted grayscale image is connected to the image mean filter program through the clock synchronization signal .

The concrete process of above-mentioned step B is:

Step B1, the input data is cached, the two-dimensional transformation processor reads the data from the FIFO cache, and selects a mask with a size of 5×5, and slides the mask to perform image traversal to achieve filtering;

Step B2, treat the 5×5 mask as a two-dimensional array, based on the continuous rows of row data, first perform the summation operation of 5 rows of data, take 4 consecutive shots of the input data, and add the current data to form 5-shot data , after 3 clocks of pairwise addition operation, the sum of 5 consecutive data is obtained;

Step B3, when performing the summation operation of 5 row data sums in the column direction, when the first row data sum is obtained, the row data cache is performed, and the summation result of the one-dimensional row direction is taken into the row cache, and wait until the subsequent When 4 pieces of data arrive, the summation results of 5 rows of data are summed in the column direction, and its structure is the same as that of the row summation;

In step B4, the operation of dividing by 25 is converted into 9 shift operations and 7 addition operations, and finally an image mean filter with a mask of 5×5 is completed.

The concrete process of above-mentioned step C is:

Step C1, the image is decomposed into low-frequency component I _LL and high-frequency component I _HH after a row transformation;

Step C2, transforming the low frequency component I _LL again to obtain two low frequency components I _LL1 , I _LH1 , transforming the high frequency component I _HH again to obtain two high frequency components I _HL1 , I _HH1 ;

In step C3, repeat step C1 for the low-frequency component I _LL1 to complete the required wavelet decomposition column transformation.

In the above-mentioned step D, the concrete process of utilizing the Canny operator to deal with I _LH1 is:

Step D11, the smoothing process uses a Gaussian kernel with a mask of 5×5 and a standard deviation of 1.4 to perform Gaussian filtering to obtain smoothing component information;

Step D12, using the Sobel operator to window the smoothed image, and filter the image horizontally and vertically through a 3×3 mask to calculate the modulus and direction of the gradient;

Step D13, on the basis of the 3×3 mask of the modulus and direction of the gradient calculated by the aforementioned Sobel operator, find out the local maximum value of the pixel, so that the gray value corresponding to the non-maximum point is set to 0 ;

Step D14, using a double threshold method to eliminate the ladder effect through hysteresis threshold segmentation.

In the above step D13, the method of finding the local maximum value of the pixel point is: divide the 8 pixels into 8 quadrants with the middle pixel as the center, and interpolate the intersection points of the oblique line and the 8 gradient directions through the center pixel point respectively , so as to determine whether the maximum value of the central pixel is determined by interpolation and comparison.

In the above step D, the specific process of image reconstruction is:

Step D21, bringing the low-frequency component I _LH10 processed by the Canny operator into the reconstruction module;

Step D22, in the reconfiguration module, configure the frequency-divided clock CLK1 by two, which is responsible for coordinating the control data flow for alternate calculation and output in the prediction and update module;

Step D23, when the current pixel is being predicted and calculated, the next pixel data is directly input to the update module for calculation, and then the next pixel data is input to the prediction module for calculation;

In step D24, the pixel data is input to the update module for calculation and output after being predicted and calculated, and the data that directly enters the update calculation can be output only after participating in the prediction.

After adopting the above scheme, compared with the prior art, the present invention has the following technical effects:

(1) In terms of the complexity of the method, the method requires less information, and the method is simple. It only needs to convert the color image to grayscale. Perform image reconstruction;

(2) In terms of timeliness of the method, because the starting point of the present invention requires little information, the complexity of implementation is low, thereby reducing the processing time of the method;

(3) FPGA-based image detection methods have broad application prospects, and different IP cores can be customized according to specific needs, and the design results can be reused;

(4) Based on the parallelism of FPGA, the algorithm can be realized at high speed, and complex processing can be realized;

(5) FPGA has strong expansibility, and FGPA is designed based on hardware language, which has excellent portability.

Description of drawings

Fig. 1 is a flow chart of the present invention;

Fig. 2 is that FPGA among the present invention realizes two-dimensional summation decomposition diagram;

Fig. 3 is that FPGA among the present invention realizes the summation schematic diagram of continuous data flow;

Fig. 4 is that FPGA among the present invention realizes two-dimensional wavelet transform schematic diagram;

Fig. 5 is that FPGA among the present invention realizes the schematic diagram of Canny operator operation;

Fig. 6 is that FPGA among the present invention realizes Sobel operator operation schematic diagram;

Fig. 7 is a schematic diagram of the gradient direction of the Canny operator implemented by FPGA in the present invention;

Fig. 8 is that FPGA among the present invention realizes wavelet transform logic diagram;

Fig. 9 is that FPGA among the present invention realizes wavelet reconstruction structural diagram;

Fig. 10 is a schematic diagram of an image for eliminating the staircase effect in the present invention.

Detailed ways

The technical solutions and beneficial effects of the present invention will be described in detail below in conjunction with the accompanying drawings.

The present invention provides a kind of FPGA-based image ladder effect elimination method, comprises the steps:

In step A, the RGB image input to the FPGA is converted into a grayscale to obtain a grayscale signal with a value of 0-255 for subsequent processing;

In the step A, the RGB space includes color signals of three channels, and the main signal of the human eye cognition image is a brightness signal, so the three channel signals of the RGB space are converted into a single-channel grayscale signal by a classical transformation formula, and at the same time It is easy for the FPGA to perform addition, multiplication, and shift operations.

In step B, the converted grayscale image is subjected to image preprocessing of mean value filtering, and the purpose of reducing noise is achieved by reducing the sharp transformation of the grayscale value of the image;

In steps A and B, the RGB image signal is input to the FPGA from the outside, and a clock synchronization signal is performed; the RGB color space image is converted through a conversion program, and the converted grayscale image is connected to the image mean filter program through a clock synchronization signal ;

Step C, decomposing the grayscale image into a low-frequency component (I _LL ) containing most of the basic information of the original image and a high-frequency component (I _HH ) containing detailed information of the original image;

Step D, using the Canny operator to process the low-frequency component (I _LH1 ) of the grayscale image obtained by wavelet decomposition, and perform image reconstruction, and finally obtain image information that eliminates the staircase effect.

As the further optimization scheme of the FPGA-based image ladder effect elimination method of the present invention, its detailed operation process is as follows: the flow block diagram of the FPGA-based image ladder effect elimination method of the present invention is as shown in Figure 1, and mainly utilizes the multiplier in the FPGA chip , internal memory, logic unit and other resources realize the color space conversion of video signal, image preprocessing, Canny edge detection, morphological erosion operation and the final outline simple extraction under the control of line, field and other synchronous signals of video signal. Finally, an edge video signal with single-pixel precision is output.

Considering the main clock synchronization information of FPGA, when clock synchronization signals such as pixel, row, and column signals arrive, the clock control program fully invokes on-chip resources such as FPGA built-in multipliers, memory, and important logic units, so that the input image data signal completes RGB The conversion from color space to grayscale color space, image mean filter preprocessing, image wavelet decomposition, Canny operator operation and wavelet reconstruction, etc., finally realize the output of eliminating the image ladder effect.

Wherein, the image data signal is transformed from the RGB color space to the grayscale color space, that is, the color transformation module, and its formula is:

Gray＝0.30×R+0.59×G+0.11×B(1)

The multiplier and adder are called inside the FPGA. Here, the three signals of the input RGB color space are respectively multiplied, the cached results are summed, and finally the summed results are shifted. At this time, the gray value of the image data pixel 0-255 is obtained, and the formula is:

Gray＝(300×R+590×G+110×B+500)＞＞10 (2)

The converted grayscale image is connected with the image mean filter preprocessing module through a clock synchronization signal. The image preprocessing module is mainly to reduce the sharp transformation of the gray value of the image and achieve the purpose of reducing noise. Here, the mask of the mean filter is 5×5, which is designed with a pipeline structure. The image is a two-dimensional array, and any two-dimensional calculation steps can be transformed into one-dimensional operations. The conversion structure diagram is shown in Figure 2. The row direction summation structure is shown in Figure 3. The 25-pixel averaging problem is transformed into an addition operation that FPGA is good at processing. The specific operation is as follows:

1) The input data is cached, and the two-dimensional transformation processor reads the data from the FIFO cache, and selects a mask with a size of 5×5, and slides the mask for image traversal to achieve filtering;

2) The 5×5 mask can be regarded as a two-dimensional array, based on the continuous rows of row data, so the summation operation of 5 rows of data is performed first, and the input data is continuously recorded for 4 shots, and the current data is added to form 5-shot data , after 3 clocks of pairwise addition operation, the sum of 5 consecutive data can be obtained;

3) When summing 5 rows of data in the column direction, when the sum of the first row data is obtained, the row data cache is required, and the summation result of the one-dimensional row direction is taken into the row cache, and it needs to wait until When the subsequent 4 data arrives, the summation results of the 5 row data are summed in the column direction, and its structure is the same as the row summation structure;

4) In order to prevent the FPGA from performing division operations, the operation of dividing by 25 is converted into 9 shift operations and 7 addition operations. Since the shift operation does not consume clock cycles, it is an overhead-free operation for the FPGA and is finally completed. The mask is a 5×5 image mean filter.

The above-mentioned processed image is controlled by a synchronous clock, input to the wavelet transform module, and wavelet decomposition is performed, and a weighted sum operation is performed on the pixel data in a 5×5 mask, involving addition and multiplication operations, here for two-dimensional The discrete wavelet transform uses fixed-point arithmetic. Here the impulse response and formula of the lifting wavelet transform are:

where h(t) is a discrete-time system function, the grayscale image is decomposed into two subbands of high frequency and low frequency after a row transformation, and then four frequency subbands are obtained through a column transformation; then the low frequency subband (I _LL1 ) Repeat the previous steps to complete the next-level wavelet decomposition column transformation method. The principle steps of the two-dimensional wavelet transformation are shown in Figure 4. The specific operation is as follows:

1) The image is decomposed into low-frequency components (I _LL ) and high-frequency components (I _HH ) after a row transformation;

2) Transform the low frequency component (I _LL ) again to obtain 2 low frequency components (I _LL1 , I _LH1 ), and transform the high frequency component (I _HH ) again to obtain 2 high frequency components (I _HL1 , I _HH1 ) ;

3) Repeat the previous steps for the low frequency component (I _LL1 ) to complete the required wavelet decomposition column transformation.

Among them, after the grayscale image is transformed by the three-level two-dimensional wavelet, the possible maximum value of the low-frequency sub-band coefficient is:

Among them, h(t) is the discrete-time system function, and L is the series.

After the above processing, the Canny operator module is called here to further process the low frequency component (I _LH1 ). As shown in Figure 5, the Canny operator operation needs to be processed in four steps: smoothing, gradient calculation, non-maximum suppression and hysteresis threshold segmentation. The detailed steps are as follows:

1) Smoothing process Gaussian filtering is performed with a Gaussian kernel with a mask of 5×5 and a standard deviation of 1.4 to obtain smoothing component information. The normalized rounding result of its mask is as follows:

2) Use the Sobel operator to window the smoothed image, and perform filtering processing in the horizontal and vertical directions of the image through a 3×3 mask. The calculation steps are shown in Figure 6. Let the filtering templates in the x and y directions be G _X and G _Y respectively, as shown in Table 1 and Table 2 respectively:

Table 1 Filter template G _X

-1 0 +1 -2 0 +2 -1 0 +1

Table 2 Filter template G _Y

+1 +2 +1 0 0 0 -1 -2 -1

Let g _x (x, y) be the filtering result in the x direction, and g _y (x, y) be the filtering result in the y direction, then the gradient modulus g(x, y) has the following formula:

The formula for calculating the gradient direction angle is:

3) When performing non-maximum suppression, on the basis of the 3×3 mask of the modulus and direction of the gradient calculated by the above Sobel operator, find the local maximum value of the pixel point, so that the corresponding non-maximum point The gray value is set to 0. As shown in Figure 7, the 8 pixels a ₀ ~ a ₈ are centered on the current pixel point a ₄ , and the 4 large quadrants are divided into 8 small quadrants, where the vector is the gradient direction of the current pixel point a4 _. In order to determine whether a ₄ is a regional maximum, the intersection points m ₁ and m ₂ in the gradient direction are linearly interpolated, and a ₂ and a ₅ are used to evaluate m ₁ interpolation, and a ₃ and a ₆ are used to evaluate m ₂ interpolation. Suppose the distance of a ₄ a ₅ is x, the distance of a ₅ m ₁ is y, and the interpolation function is f, then the interpolation result is:

Let the calculation result of this point be Result, then:

When mapping the above algorithm to FPGA, in order to eliminate the subtraction operation, do the following transformation:

x·f(m ₁ )=y·a ₂ +(xy)·a ₅ (11)

x·f(m ₂ )=y·a ₆ +(xy)·a ₃ (12)

Using the same method, the interpolation results in the other three cases can be obtained, as follows:

y·f(m ₃ )=x·a ₂ +(yx)·a ₁ (14)

y·f(m ₄ )=x·a ₆ +(yx)·a ₇ (15)

y·f(m ₅ )=x·a ₀ +(yx)·a ₁ (17)

y·f(m ₆ )=x·a ₈ +(yx)·a ₇ (18)

x·f(m ₇ )=y·a ₃ +(xy)·a ₀ (20)

x·f(m ₈ )=y·a ₅ +(xy)·a ₈ (21)

In order to eliminate the difference between x and y, the above formula is transformed as follows:

M _max f(m _X ) = M _min C ₀ +(M _max -M _min ) C ₁ (23)

M _max f(m _Y ) = M _min C ₂ +(M _max -M _min ) C ₃ (24)

In the above formula, M _max represents the larger value of the current x and y direction gradient values, M _min represents the smaller value of the current x and y direction gradient values, f(m _X ) and f(m _Y ) both represent For interpolation, C ₀ , C ₁ , C ₂ , and C ₃ respectively represent 4 interpolation elements. For 8 different small quadrants, the index number of the interpolation element can be obtained by querying Table 3:

table 3

Judging whether the central pixel is the maximum value or not by the interpolation results obtained above, and effectively suppressing the non-maximum value.

4) The ladder effect is eliminated by hysteresis threshold segmentation, and the double threshold method is used here. By setting two thresholds, the edge information of the image can be obtained based on the set high threshold, so the image will rarely have a staircase effect, but setting a higher threshold may cause the edge of the image to fail to close. Here, another low setting is used. threshold to avoid this situation, and finally obtain the Canny operator processing result.

Finally, the low-frequency component (I _LH10 ) processed by the Canny operator is brought into the reconstruction module. As shown in FIG. 8 , in the reconstruction module, a frequency-divided clock CLK1 is configured, which is responsible for coordinating the control data flow for alternate calculation and output in the prediction and update modules. Among them, the structural diagram of the wavelet reconstruction is shown in Figure 9. It can be seen from the figure that the prediction formula is:

From the knowledge of wavelet reconstruction, we can know that the addition and subtraction signs outside the brackets in the decomposition transformation process (26) and (27) can be exchanged, so the reverse prediction formula of this process is:

Among them, when the current pixel is being predicted and calculated, the next pixel data can be directly input into the update module for calculation, and then the next pixel data can be input into the prediction module for calculation; The updated and calculated data needs to participate in the prediction before it can be output, and finally reconstructed to obtain image data that eliminates the step effect, as shown in Figure 10.

The present invention aims at eliminating the step effect of the image based on the FPGA, suppresses the "block effect" of the image to the greatest extent and maintains detailed information such as the edge texture of the image, is based on a model combining wavelet and Canny algorithm, and uses wavelet theory as the time-frequency The analysis method uses the Canny algorithm to perform non-maximum value and hysteresis threshold segmentation processing, and then performs wavelet reconstruction on the processed image sub-bands to restore image noise while protecting image edge texture and other details, and is based on FPGA The parallelism and programmability of the system can meet the specific needs of the parameters, so as to expand the system accordingly, effectively suppress the ladder effect, have a good denoising effect, and the final effect is very ideal.

The above embodiments are only to illustrate the technical ideas of the present invention, and can not limit the protection scope of the present invention with this. All technical ideas proposed in accordance with the present invention, any changes made on the basis of technical solutions, all fall within the protection scope of the present invention. Inside.

Claims (7)

1. a kind of image alias removing method based on FPGA, it is characterised in that include the following steps：

Step A, the RGB image that will enter into FPGA carry out gradation conversion, obtain the grey scale signal that value is 0-255；

Conversion gained gray level image is carried out the image preprocessing of mean filter, reduces the sharp change of the gray value of image by step B It changes；

Gray level image is carried out wavelet decomposition by step C, is decomposed into the low frequency component I comprising original image essential information_LLAnd packet The high fdrequency component I of the details containing original image_HH, and to I_LLIt decomposes again and obtains its low frequency component I_LL；

Step D, using Canny operator to I_LH1Processing, and image reconstruction is carried out, it is final to obtain the image letter for eliminating alias Breath.

2. a kind of image alias removing method based on FPGA as described in claim 1, it is characterised in that：The step In A and step B, the input of RGB image signal, row clock synchronization signal of going forward side by side are carried out to FPGA from outside；To RGB color space figure As by conversion program, gray level image after conversion is attached by clock sync signal and Image Mean Filtering program.

3. a kind of image alias removing method based on FPGA as described in claim 1, it is characterised in that：The step The detailed process of B is：

Step B1, input data is cached, and two-dimensional transform processor reads data from FIFO caching, and selected size is 5 × 5 mask, and slide mask and carry out image traversal to realize filtering；

Step B2 regards 5 × 5 masks as a two-dimensional array, the continuous row based on row data, carries out 5 row data first Input data is continuously played 4 bats by sum operation, in addition current data forms 5 beat of data, is transported by the addition two-by-two of 3 clocks It calculates, obtains the sum of continuous 5 data；

Step B3, when carrying out 5 row data and progress column direction summation operation, when first row data is with acquiring, into Line data caching, takes capable caching for the summed result of one-dimensional line direction, until subsequent 4 data arrive, 5 row data Summed result carries out column direction summation, and structure is identical as row summation structure；

Operation divided by 25 is converted to 9 shifting functions and 7 sub-additions operates by step B4, and being finally completed mask is 5 × 5 Image Mean Filtering.

4. a kind of image alias removing method based on FPGA as described in claim 1, it is characterised in that：The step The detailed process of C is：

Step C1, image resolve into low frequency component I after the transformation of primary row_LLWith high fdrequency component I_HH；

Step C2, by low frequency component I_LLIt converts again and once obtains 2 low frequency component I_LL1, I_LH1, by high fdrequency component I_HHOne is converted again It is secondary to obtain 2 high fdrequency component I_HL1, I_HH1；

Step C3, then to low frequency component I_LL1Step C1 is repeated, wavelet decomposition rank transformation needed for completing.

5. a kind of image alias removing method based on FPGA as described in claim 1, it is characterised in that：The step In D, using Canny operator to I_LH1The detailed process of processing is：

Step D11, smoothing processing use mask for 5 × 5 and standard deviation be 1.4 Gaussian kernel carry out gaussian filtering, obtain smooth Component information；

Step D12 opens a window to smooth rear image using Sobel operator, by 3 × 3 masks in image level direction and Vertical Square To being filtered, modulus value and the direction of gradient are calculated；

Step D13, aforementioned Sobel operator calculate gradient modulus value and direction 3 × 3 mask on the basis of, find out pixel Point local maximum, so that gray value corresponding to non-maximum point is set to 0；

Step D14 eliminates alias by hysteresis threshold segmentation using dual-threshold voltage.

6. a kind of image alias removing method based on FPGA as claimed in claim 5, it is characterised in that：The step In D13, the method for finding out pixel local maximum is：By 8 pixels centered on intermediate pixel, 8 quadrants are separated into, and Cross central pixel point and do the intersection point of oblique line and 8 gradient directions and carry out interpolation respectively, thus by compared with interpolation judge center Whether the maximum value of pixel.

7. a kind of image alias removing method based on FPGA as described in claim 1, it is characterised in that：The step In D, the detailed process for carrying out image reconstruction is：

Step D21, by Canny operator treated low frequency component I_LH10It is brought into reconstructed module；

Step D22 configures two divided-frequency clock CLK1 in reconstructed module, is responsible for coordinated control data flow and is predicting and updating Interleaved computation and output are carried out in module；

Step D23, when current pixel is carrying out prediction calculating, next pixel data is directly inputted to update module calculating, then Input prediction module calculates next pixel data again；

Step D24, pixel data input update module after prediction calculates calculate output, are directly entered the data for updating and calculating It could be exported after participating in prediction.