CN103986922B - Image processing method - Google Patents

Abstract

The invention provides an image processing method, which comprises the following steps. And acquiring a source image of the color arrangement of the Bell diagram. First-order image processing is performed on a source image to produce a first luma chroma format image. And performing second-stage image processing on the source image to produce a second luma chroma format image. Then, noise filtering processing is carried out on the first luminance chrominance format image to generate a noise suppression image. The luminance image in the noise-suppressed image and the luminance image in the second luminance chrominance format image are weighted, and then the chrominance images in the second luminance chrominance format image are combined to generate a processed image. The noise-suppressed image has a higher degree of noise reduction than the second luminance chrominance format image.

Description

image processing method

technical field

The present invention relates to an image processing technology, and in particular to an image processing method for reproducing natural appearance reproduction of images.

Background technique

In the application of digital cameras, noise filtering of photosensitive elements or electronic signals is a very important part, especially for high-sensitivity (High ISO) digital images. In the process of digital image noise filtering, the detailed texture and gradient light and shadow changes of the object itself are often filtered out together, so that the detailed texture in the digital image presents a blocky distribution and loses its natural appearance.

In order to maintain the natural feeling of digital images, the post-processing for image noise filtering generally uses image sharpness enhancement algorithms to enhance the edges of objects in the image to achieve a sharpening effect. For the side effect of losing the natural appearance after noise filtering, the common way to deal with it is to adjust the parameters in the noise filtering algorithm to alleviate this effect, or to detect the smoothing area (smoothing area) and texture area ( texturearea) and sharpnessedge (sharpnessedge) to give different noise filtering parameters respectively.

For the excessively blurred image after noise filtering, although the sharpness enhancement algorithm can be used to sharpen the edge or texture area of the object, the image detail is increased. However, in the noise filtering algorithm, according to the smooth area, texture area and sharp edge of the image, giving different noise filtering parameters respectively will cause the difference of each area in the image to become larger, because the noise filtering of different areas Parameters tend to vary to some degree. In this way, it will feel more unnatural visually. However, if the noise filtering parameters are adjusted to weaken the differences in each area, the problem that the noise cannot be filtered out will be formed. Therefore, how to present the natural appearance of the image after noise filtering is a problem to be solved.

Contents of the invention

The present invention provides an image processing method, which performs post-processing on the noise-filtered image, thereby retaining more image detail information and reproducing the natural sense of the image.

An image processing method of the present invention includes the following steps. Get the source image for the color arrangement of the Bayer pattern. A first stage of image processing is performed on the source image to produce a first luma-chroma format image. And performing second-order image processing on the source image to generate a second luma-chroma format image. Then, a denoise process is performed on the first luminance-chrominance format image to generate a noise-suppressed image. After weighting the luminance image in the noise-suppressed image and the luminance image in the second luminance-chroma format image, merging the chrominance (chrominance) image in the second luminance-chroma format image to generate a processed image . The noise reduction degree of the noise-suppressed image is higher than the noise reduction degree of the second luma-chroma format image.

In an embodiment of the present invention, the above-mentioned step of performing the first-order image processing includes first performing Bayer denoise processing on the source image, and then performing color interpolation (color interpolation) processing and color reconstruction (color reproduction) processing , to produce a first luma-chroma format image.

In an embodiment of the present invention, the step of performing the second-level image processing includes directly performing color interpolation processing and color reconstruction processing on the source image to generate a second luminance-chroma format image.

In an embodiment of the present invention, after the step of performing the second-level image processing on the source image to generate the second luminance-chroma format image, it further includes converting the second luminance-chroma format image to separate the luminance Image and chroma image. Next, a corresponding noise map (noisemap) is generated for the luminance image. Furthermore, the luminance image, the noise image and the chrominance image are blended to generate output images with noises of different characteristics.

In an embodiment of the present invention, the image size of the aforementioned noise map is the same as that of the brightness image.

In an embodiment of the present invention, the aforementioned noise map includes a plurality of positive or negative values, and each value corresponds to each pixel in the luminance image.

In an embodiment of the present invention, the step of generating a corresponding noise map for the luminance image includes: determining a respective noise value range (noiserange) and a respective noise offset (noiseoffset) for each pixel in the luminance image, A single-point noise map (singlepixelnoisemap) is thus generated; a respective blur mask (blurmask) is determined for each pixel in the brightness image to generate an image mask indication map; and the corresponding single-point noise is generated according to the image mask indication map map is blurred to produce the noise map described above.

In an embodiment of the present invention, each noise value range and each noise offset are determined for each pixel in the luminance image, and the step of generating the above-mentioned single-point noise map includes: according to the block of the luminance image The noise value range is determined by the characteristics and the brightness value; the noise value is generated by a random number within the noise value range; the look-up table is performed according to the brightness value of the brightness image to obtain the noise offset; and each noise value of each pixel is added to each noise value offset to obtain the single-point noise map described above.

In an embodiment of the present invention, the step of determining each blur mask for each pixel in the luminance image to generate the above-mentioned image mask indication map includes: according to the block characteristics of the luminance image and the luminance value of each pixel, Selecting a corresponding blur mask set from a blur mask database; and selecting a blur mask by random number from each blur mask set for each pixel, so as to form the above-mentioned image mask indication map.

In an embodiment of the present invention, the above-mentioned blur mask database stores a plurality of blur masks with different sizes and styles.

Based on the above, the image processing method provided by the present invention can output a processed image that contains image detail information and achieves the effect of filtering noise, and this method can avoid the block effect generated after general noise filtering and eliminate the sense of image discontinuity , so that the processed image has a natural look.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail with reference to the accompanying drawings.

Description of drawings

FIG. 1 is a block diagram of an image processing device according to a first embodiment of the present invention;

FIG. 2 is a flowchart of an image processing method according to the first embodiment of the present invention;

3 is a block diagram of an image processing device according to a second embodiment of the present invention;

FIG. 4 is a flowchart of an image processing method according to a second embodiment of the present invention;

Fig. 5 is a detailed implementation of the noise generating module according to the second embodiment of the present invention;

Fig. 6 is a detailed implementation manner of generating a corresponding noise map for a luminance image according to the second embodiment of the present invention.

Explanation of reference signs:

100, 300: image processing device;

110: image acquisition module;

120: a first noise filtering module;

130: color reconstruction module;

140: a second noise filtering module;

150, 330: merge modules;

200, 400: method flow;

310: image separation module;

320: noise generation module;

510: single-point noise determination unit;

512: random number generator;

514: mixer;

520: fuzzy mask decision unit;

530: fuzzy processing unit;

Img1～Img9: image;

S210～S250: each step of the image processing method;

S410-S440: each step of the image processing method.

detailed description

first embodiment

In this embodiment, in order to avoid the unnatural feeling of the image after noise filtering, the source image is separated into two different image processing processes to generate two result images, one of which contains more noise and image details information, the other image is more blurred, and finally the two resulting images are weighted and summed to produce a processed image with noise filtering effect and more image detail information.

FIG. 1 is a block diagram of an image processing device according to a first embodiment of the present invention. The image processing device 100 is, for example, a digital camera, a SLR camera, a digital video camera, or other electronic devices with image processing functions such as smart phones, tablet computers, notebook computers, and desktop computers, and is not limited to the above.

Referring to FIG. 1 , the image processing device 100 includes an image acquisition module 110 , a first noise filtering module 120 , a color reconstruction module 130 , a second noise filtering module 140 and a combining module 150 . Wherein, the image acquisition module 110 includes components such as a lens, a photosensitive element, etc., to acquire images. The color reconstruction module 130, the first and second noise filtering modules 120, 140, and the combining module 150 may be functional modules implemented by hardware and/or software. The hardware may include central processing unit, chipset, microprocessor and other hardware devices with image computing and processing functions, or a combination of the above hardware devices, while the software may be an operating system, driver program, etc.

FIG. 2 is a flowchart of an image processing method according to the first embodiment of the present invention. Please refer to FIG. 2, the method 200 of this embodiment is applicable to the image processing device 100, and the detailed steps of this embodiment are described below with each module in the image processing device 100:

First, in step S210, the image acquisition module 110 is used to acquire the source image Img1 of the Bayer pattern color arrangement.

Next, in step S220 , a first-stage image processing is performed on the source image Img1 to generate a first luminance and chrominance format image (ie, a YCbCr format image, hereinafter referred to as the first YCbCr image Img2 ). Wherein, step S120 can also be divided into sub-steps S222 and S224. That is to say, the steps of the first-stage image processing include using the first noise filtering module 120 to perform Bell noise filtering on the source image Img1 (step S122 ). Afterwards, the color reconstruction module 130 performs color interpolation processing and color reconstruction processing to generate the first YCbCr image Img2.

In detail, since each pixel in the source image Img1 of the Bell diagram color arrangement has only one color of the red channel (Rchannel), green channel (Gchannel) or blue channel (Bchannel), it is not the RGB format for general display image or YCbCr format image. Therefore, the color reconstruction module 130 performs color interpolation processing to generate a three primary color image for general display. Furthermore, in order to present the color correctly, the color reconstruction module 130 also performs color reconstruction processes such as black offset, RGB gain adjustment, color correction, and gamma correction. After that, the color reconstruction module 130 converts and outputs the first YCbCr image Img2.

Next, in step S230 , the second noise filtering module 140 also performs noise filtering on the first YCbCr image Img2 to generate a blurred noise-suppressed image Img3 . Object details or textures in the noise-suppressed image Img3 usually disappear with the noise filtering process, and human eyes usually feel unnatural.

On the other hand, as described in step S240 , the color reconstruction module 130 performs second-level image processing on the source image Img1 to generate the second YCbCr image Img4 . The step of performing the second-level image processing includes the color reconstruction module 130 directly performing color interpolation processing and color reconstruction processing on the source image Img1 to generate a second YCbCr image Img4 with clearer details. Wherein, the partial processing performed by the color reconstruction module 130 (for example, color correction) will change the noise distribution, resulting in an unnatural sense of noise (for example, the noise in certain color-saturated areas is relatively high). Therefore, the color reconstruction module 130 will adjust the corresponding parameters of the color correction. Compared with the first-stage image processing, the correction intensity is much weaker, so that the image luminance noise of the second YCbCr image Img4 maintains a natural feeling. In other words, the degree of noise reduction of the noise-suppressed image Img3 is higher than that of the second YCbCr image Img4.

In step S250, the merging module 150 performs weight processing (also called weighted sum ( weightingsum) operation). Finally, the merging module 150 merges the chrominance images in the second YCbCr image to generate a processed image Img5 containing more object detail information. It should be noted that since this weighting process will also mix noises into the output processed image Img5, these noises can alleviate some unnatural feeling of block-like appearance after noise filtering and make the image more natural.

Since human eyes have different perceptions of bright and dark noises, when performing the weighted sum operation at the end, the combining module 150 will refer to the brightness value of the noise-suppressed image Img3 to determine weights, so as to generate a processed image Img5 that is better perceived by human eyes. For example, for a pixel with a brightness value of 100 in the noise-suppressed image Img3, the weight of the combined module 150 to mix the noise-suppressed image Img3 and the second YCbCr image Img4 is, for example, set to 80:20; Therefore, for pixels with a brightness value of 10 in the noise-suppressed image Img3, the weight of the merging module 150 to mix the noise-suppressed image Img3 and the second YCbCr image Img4 is set to 90:10, for example. In one embodiment, the weights set for different luminance values can be pre-set by those skilled in the art and stored in a table form, so that the combining module 150 can be stored in the memory unit (not shown in FIG. 1 ). Quickly query the corresponding weight setting value in .

In this embodiment, since the first-level image processing flow and the second-level image processing flow have many identical or similar processing steps, the same hardware (such as the color reconstruction module 130) can be used but multiple calculations, To reduce the cost of hardware design. This embodiment not only allows the noise-filtered image to retain image detail information, but also eliminates the sense of discontinuity in image quality, and improves the naturalness and image quality of the processed image.

second embodiment

In this embodiment, in order to avoid the unnatural feeling of the image after noise filtering, it is also possible to randomly generate many noise points in the image and mix these noise points with different characteristics into the image after noise filtering , in order to achieve the technology of natural reproduction.

FIG. 3 is a block diagram of an image processing device according to a second embodiment of the present invention. The image processing device 300 is, for example, a digital camera, a SLR camera, a digital video camera, or other electronic devices with image processing functions such as smart phones, tablet computers, notebook computers, and desktop computers, and is not limited to the above.

Referring to FIG. 3 , the image processing device 300 includes an image separation module 310 , a noise generation module 320 and a combination module 330 . The image separation module 310 is used for separating the received input image into a luma image and a chrominance image. The noise generating module 320 is used for generating a noise map (noisemap). The merging module 330 blends the luma image, the noise image, and the chrominance image to generate output images with noises of different characteristics. Each of the above-mentioned modules may be a functional module realized by hardware and/or software. The hardware may include central processing unit, chipset, microprocessor and other hardware devices with image computing and processing functions, or a combination of the above hardware devices, while the software may be an operating system, driver program, etc.

FIG. 4 is a flowchart of an image processing method according to a second embodiment of the present invention. Please refer to FIG. 4, the method 400 of this embodiment is applicable to the image processing device 300, and the detailed steps of this embodiment are described below with each module in the image processing device 300:

In step S410, the image separation module 310 receives an input image Img6. Wherein, the input image Img6 is, for example, the second YCbCr image Img4 in the first embodiment. Next, in step S420, the image separation module 310 converts the input image Img6 to separate a luminance image (ie, Y channel image) Img7 and a chrominance image (ie, CbCr channel image) Img8. Wherein, the luminance image Img7 is sent to the noise generation module 320 and the merging module 330 respectively; the chrominance image Img8 is directly sent to the merging module 330 .

In step S430 , the noise generating module 320 generates a corresponding noise image Img9 for the luminance image Img7 . Among them, the image size (size) of the noise image Img9 is the same as that of the brightness image Img7. The noise image Img9 is a value including a plurality of positive or negative signs, and each value corresponds to each pixel in the luminance image Img7.

Finally, in step S440 , the merging module 330 mixes the luminance image Img7 , the noise image Img9 , and the chrominance image Img8 to generate an output image Img10 with noises of different characteristics. Specifically, the merging module 330 first adds the brightness value of each pixel in the brightness image Img7 to the corresponding noise value in the noise map Img9 to generate a brightness image containing noise points. Next, the merging module 330 mixes the luma image containing noise points with the chrominance image Img8 to generate a natural output image Img10 . Wherein, the input image Img10 can be used as an image input to the merging module 150 in the first embodiment, for example.

It should be noted that adding noise points to the luminance image Img7 is equivalent to canceling the function of noise filtering, so these noise points cannot be randomly generated randomly, and must conform to the user's natural viewpoint. That is to say, it is necessary to refer to the preference of human vision. In this embodiment, noise points of different sizes, different intensities, and different trends are added according to the brightness of the image, so that the user can have a better visual experience. How the noise generation module 320 of this embodiment generates the corresponding noise image Img9 for the brightness image Img7 will be described in detail below with reference to FIG. 5 and FIG. 6 .

FIG. 5 is a detailed implementation of the noise generating module 320 according to the second embodiment of the present invention. Referring to FIG. 5 , the noise generation module 320 includes a single-point noise determination unit 510 , a blur mask (blurmask) determination unit 520 and a blur processing unit 530 . Wherein, the single point noise determining unit 510 further includes a random number generator 512 and a mixer 514 .

Fig. 6 is a detailed implementation manner of generating a corresponding noise map for a luminance image according to the second embodiment of the present invention. Please refer to Fig. 5 and Fig. 6 together below.

First, in step S610, the noise generating module 320 receives a brightness image Img7. The luminance image Img7 is respectively sent to the single-point noise determination unit 510 and the blur mask determination unit 520 for processing.

In step S620, the single-point noise determining unit 510 first determines a noise range and a noise offset for each pixel in the luminance image, thereby generating a single-pixel noise map. Specifically, the single-point noise determining unit 510 first determines the noise value range (-TH _range , TH _range ) according to the block characteristics of the brightness image Img7 and the brightness value of each pixel, wherein TH _range is a positive number greater than 0. Next, a noise value is randomly generated by the random number generator 512 , wherein the noise value falls within the noise value range (−TH _range , TH _range ). Next, the mixer 514 adds a corresponding noise offset to the noise value to control the average noise intensity. After each pixel in the brightness image Img7 undergoes the above processing, a single pixel noise map (single pixel noise map) can be obtained. Wherein, the noise offset can be pre-set in the form of a table by those skilled in the art, and the noise offset can be obtained by looking up the table according to the luminance value of the luminance image Img7. The function of noise offset also includes enhancing the contrast of the image, so that the bright part of the image is brighter and the dark part is darker.

On the other hand, in step S630 , the blur mask determining unit 520 determines a respective blur mask (blur mask) for each pixel in the luminance image to generate an image mask indication map. In detail, since the numerical variation of each noise in a single-point noise map is different from the apparent distribution of natural noise, natural noise may cluster to form a circular or other shaped distribution. Therefore, it is also necessary to use an image mask on the single-point noise map to generate different styles of noise. The so-called image mask is a blur mask.

Because the noise style in the image is slightly different with the brightness, and there will be multiple noise styles at the same brightness. Therefore, the blur mask determining unit 520 can firstly select a corresponding blur mask set from a blur mask database according to the block characteristics of the brightness image and the brightness value of each pixel. The blur mask database stores multiple blur masks with different sizes and styles. For example, the blur mask can be a standard N*N mask, where N is a positive integer greater than 0. Each fuzzy mask is given a number in the fuzzy mask database, assuming that there are 10 groups of fuzzy masks in the fuzzy mask database, the numbers are 1-10 respectively.

For example, Table 1 is a brightness and blur mask set table according to the second embodiment. Please refer to the following table 1, corresponding to the brightness value of 0, for example, you can use the blur mask numbered 1, 2 or 5; corresponding to the brightness value of 1, for example, you can use the blur mask numbered 2, 3 or 4; and so on .

Table 1

Brightness value Blur mask collection 0 1, 2, 5 1 2, 3, 4 2 1, 3 ... ... 255 10

The blur mask determining unit 520 then performs table lookup according to the brightness value to randomly select a blur mask corresponding to the pixel from the corresponding blur mask set, and record its number. After recording its corresponding blur mask number for each pixel, the image mask indication map of this embodiment is generated.

Returning to FIG. 6 , in step S640 , the blurring processing unit 530 performs blurring processing on the corresponding single-point noise map according to the image mask indication map, so as to generate a noise map Img9 with different characteristics.

To sum up, the present invention adopts a dual image processing path, the first-stage image processing is to generate blurred images after noise filtering, and the second-stage image processing is to retain image detail information and/or clear images with different characteristic noises . By considering the brightness and darkness of the image, the two images are weighted and mixed to output a processed image that contains image detail information and achieves the effect of filtering noise. In addition, the image processing method adopted in the present invention can avoid the block effect produced after general noise filtering and eliminate the sense of image discontinuity, so that the processed image has a natural appearance.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (10)

1. An image processing method, characterized in that, comprising: receive a source image; performing a first-stage image processing on the source image to generate a first luma-chroma format image; performing a second-level image processing on the source image to generate a second luma-chroma format image; performing a noise suppression process on the first luma-chroma format image to generate a noise-suppressed image; and first performing weight processing on the luminance image in the noise-suppressed image and the luminance image in the second luminance-chroma format image, and combining the chrominance images in the second luminance-chroma format image to generate a processed image, wherein The noise reduction degree of the noise-suppressed image is higher than the noise reduction degree of the second luma-chroma format image. 2. The image processing method according to claim 1, wherein the step of performing the first-order image processing comprises: After a Bell noise filtering process is performed on the source image, a color interpolation process and a color reconstruction process are performed to generate the first luma-chroma format image. 3. The image processing method according to claim 2, wherein the step of performing the second-order image processing comprises: The color interpolation process and the color reconstruction process are directly performed on the source image to generate the second luma-chroma format image. 4. The image processing method according to claim 1, further comprising: converting the second luma-chroma format image to separate a luma image and a chroma image; generating a corresponding noise map for the luminance image; and The luma image, the noise map and the chrominance image are blended to generate an output image with noise of different characteristics. 5. The image processing method according to claim 4, wherein the image size of the noise map is the same as the image size of the brightness image. 6 . The image processing method according to claim 4 , wherein the noise map includes a plurality of positive or negative values, and each value corresponds to each pixel in the brightness image. 7. The image processing method according to claim 4, wherein the step of generating a corresponding noise map for the luminance image comprises: determining a respective noise value range and a respective noise offset for each pixel in the luminance image, thereby generating a single-point noise map; determining a respective blur mask for each pixel in the luminance image to generate an image mask map; and Fuzzing the corresponding single-point noise map according to the image mask indication map to generate the noise map. 8. The image processing method according to claim 7, characterized in that first, each of the noise value ranges and each of the noise offsets are determined for each of the pixels in the luminance image, thereby generating the single-point noise map The steps include: determining the noise value range according to a block characteristic of the brightness image and a brightness value; Randomly generating a noise value within the noise value range; performing a table lookup according to the luminance value of the luminance image to obtain the noise offset; and Adding each noise offset to each noise value of each pixel to obtain the single-point noise map. 9. The image processing method according to claim 7, wherein the step of determining each of the blur masks for each of the pixels in the brightness image to generate the image mask indication map comprises: selecting a corresponding fuzzy mask set from a fuzzy mask database according to a block characteristic of the luminance image and a luminance value of each pixel; and A fuzzy mask is randomly selected from each of the fuzzy mask sets for each of the pixels to form the image mask indication map. 10. The image processing method according to claim 9, wherein a plurality of blur masks with different sizes and styles are stored in the blur mask database.