JP6631076B2 - Image processing apparatus, program, and image processing method - Google Patents

Description

  The present invention relates to an image processing device, a program, and an image processing method.

  Conventionally, a noise reduction process has been disclosed in which an edge of an image is extracted and an input image or a smoothed image is output depending on whether or not a pixel is an edge to reduce image noise (for example, Patent Document 1). ).

Japanese Patent No. 5460173

  In the method of selecting an input image or a smoothed image depending on whether or not a pixel is an edge as described in Patent Literature 1, an unnatural image in which an outline appears is generated.

  Therefore, an object of the present invention is to provide an image processing device, a program, and an image processing method capable of reducing a noise of an image and generating a more natural image.

The present invention solves the above problem by the following means.
A first invention is a blurred image generating means for performing a blurring process on an input image to generate a blurred image, and performing edge extraction on the input image to generate an edge extracted image. A process image generating means for performing a blurring process to generate an edge blurred image, and synthesizing the input image and the blurred image based on a pixel value of the edge blurred image generated by the process image generating means. And an image generating means for generating a new image.
According to a second aspect, in the image processing apparatus according to the first aspect, the image processing apparatus further includes a noise-removed image generating unit that generates a noise-removed image by removing noise from the input image. And combining the blurred image with the blurred image to generate a new image.
A third invention is the image processing device according to the first invention or the second invention, wherein the image generating means generates a new image based on a luminance value of the edge blurred image. An image processing device.
A fourth invention is the image processing apparatus according to any one of the first invention to the third invention, wherein the luminance value of the input image and the blur value for each pixel are based on the luminance value of the edge blur image. And a brightness value calculating unit that calculates a new brightness value using the brightness value of the image, wherein the image generating unit calculates a color component of the input image and a color of the blurred image based on the calculated new brightness value. An image processing apparatus for generating a new image using at least one of the color components.
In a fifth aspect based on the image processing apparatus according to the fourth aspect, the image generating means is configured to output one of the color component of the input image and the color component of the blurred image based on the calculated new threshold value of the luminance value. An image processing apparatus characterized in that one is selected and a new image is generated by adding a color component of a selected pixel to the new luminance value.
A sixth invention is a program for causing a computer to function as the image processing device according to any one of the first invention to the fifth invention.
In a seventh aspect, a blurring process is performed on an input image to generate a blurred image, an edge is extracted from the input image to generate an edge-extracted image, and then the edge-extracted image is blurred. And generating an edge blurred image, and combining the input image and the generated blurred image based on a pixel value of the generated edge blurred image to generate a new image. .

  According to the present invention, it is possible to provide an image processing apparatus, a program, and an image processing method capable of reducing a noise of an image and generating a more natural image.

FIG. 2 is a functional block diagram of the image processing apparatus according to the embodiment. FIG. 3 is a diagram for explaining a relationship between images generated by the image processing apparatus according to the embodiment. 5 is a flowchart illustrating an image generation process in the image processing apparatus according to the embodiment. FIG. 3 is a diagram illustrating an example of a generated image in the image processing apparatus according to the embodiment. 9 is a flowchart illustrating a pre-stage noise removal process in the image processing apparatus according to the present embodiment. FIG. 2 is a diagram for explaining pixels of an input image of the image processing apparatus according to the embodiment. 5 is a flowchart illustrating an image combining process in the image processing apparatus according to the embodiment.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. This is only an example, and the technical scope of the present invention is not limited to this.
(Embodiment)
<Image processing device 1>
FIG. 1 is a functional block diagram of an image processing apparatus 1 according to the present embodiment.
FIG. 2 is a diagram for explaining the relationship between the images generated by the image processing apparatus 1 according to the present embodiment.
The image processing device 1 is a device that generates a new image from which noise has been removed from an input image.

The image processing device 1 is, for example, a personal computer (PC). The image processing device 1 may be a portable device having functions of a computer typified by a tablet terminal or a smartphone. Further, the image processing device 1 may be a device such as a camera having an imaging unit.
As illustrated in FIG. 1, the image processing device 1 includes a control unit 10, a storage unit 20, an input unit 30, a display unit 32, and a communication unit 39.
The control unit 10 is a central processing unit (CPU) that controls the entire image processing apparatus 1. The control unit 10 executes various functions in cooperation with the above-described hardware by appropriately reading and executing an operating system (OS) and an application program stored in the storage unit 20.

The control unit 10 includes a blurred image generation unit 11 (blurred image generation unit), a process image generation unit 12 (process image generation unit), a noise removal image generation unit 13 (noise removal image generation unit), and a luminance value calculation unit. 14 (luminance value calculation means) and an image generation unit 15 (image generation means).
The blurred image generator 11 performs a blurring process on the input image to generate a blurred image.
The process image generating unit 12 performs edge extraction on the input image to generate an edge extracted image, and then performs a blurring process on the edge extracted image to generate an edge blurred image.
The noise-removed image generation unit 13 generates a processing source image (noise-removed image) obtained by removing noise from the input image.
The brightness value calculation unit 14 calculates a new brightness value for each pixel from the brightness value of the pixel of the processing source image and the brightness value of the pixel of the blurred image based on the edge signal intensity of the edge blurred image.
The image generating unit 15 generates a new image by combining the processing source image and the blurred image based on the edge signal strength of the edge blurred image.
The details of each process will be described later.

The storage unit 20 is a storage area such as a hard disk and a semiconductor memory device for storing programs, data, and the like necessary for the control unit 10 to execute various processes.
The storage unit 20 includes a program storage unit 21 and an image storage unit 22.
The program storage unit 21 is a storage area for storing the image processing program 21a.
The image processing program 21a is a program for performing various functions executed by the control unit 10 of the image processing apparatus 1. In this example, the image processing program 21a is installed in the image processing apparatus 1 in advance.
The image storage unit 22 is a storage area for storing various images used in the image processing device 1. The image storage unit 22 includes an input image storage unit 23, an intermediate image storage unit 24, and a generated image storage unit 25.

As shown in FIG. 2, the input image storage unit 23 is a storage area for storing the input image 40. The image processing device 1 receives and stores the input image 40 from an external device (for example, a camera) via the communication unit 39, for example.
The intermediate image storage unit 24 is a storage area for storing a process image generated during the image generation processing. The intermediate image storage unit 24 stores a blurred image 50, an edge blurred image 52, and a processing source image 54. Although details will be described later, the blurred image 50, the edge blurred image 52, and the processing source image 54 are images generated by performing different processes on the input image 40, respectively.
The generated image storage unit 25 is a storage area that stores the output image 60. Although the details will be described later, the output image 60 is an image generated by referring to the edge blurred image 52 and synthesizing the blurred image 50 and the processing source image 54.

Referring back to FIG. 1, the input unit 30 is an input device including a mouse, a keyboard, and the like.
The display unit 32 is a display including a liquid crystal panel or the like.
Note that the input unit 30 and the display unit 32 may be an integrated device, and a touch panel display having a display function and a function of detecting a touch input by a finger or the like from a user who operates the image processing apparatus 1 It may be.
The communication unit 39 is an interface unit for performing communication with an external device (such as a camera, not shown) via a communication network.

<Process of Image Processing Apparatus 1>
Next, the processing of the image processing apparatus 1 will be described.
FIG. 3 is a flowchart illustrating an image generation process in the image processing apparatus 1 according to the present embodiment.
FIG. 4 is a diagram illustrating an example of an image generated by the image processing apparatus 1 according to the present embodiment.
FIG. 5 is a flowchart illustrating a pre-stage noise removal process in the image processing apparatus 1 according to the present embodiment.
FIG. 6 is a diagram for explaining pixels P of the input image 40 of the image processing device 1 according to the present embodiment.
FIG. 7 is a flowchart illustrating an image combining process in the image processing apparatus 1 according to the present embodiment.

The image generation processing illustrated in FIG. 3 is processing for reducing noise of the input image 40 and generating a more natural output image 60. In the image generation process, for example, a user selects one input image 40 stored in the input image storage unit 23 of the image processing apparatus 1 during execution of the image processing program 21a, and instructs to start the process. Is started.
Here, FIG. 4A is an example of the input image 40. In this example, the input image 40 is an image of a rose flower as a subject, but may be any image.
In step S (hereinafter, referred to as “S”) 10 in FIG. 3, the control unit 10 (blurred image generation unit 11) generates a blurred image 50 using the input image 40. The control unit 10 generates the blurred image 50 using, for example, a known moving average filter that is one of the smoothing filters. The process using the moving average filter performed here is a process of allocating the average value of the pixel values around the target pixel to the target pixel, and can smoothly change the gray value of the image.
FIG. 4B shows a blurred image 50 obtained by performing a blurring process on the input image 40 shown in FIG. 4A. The blurred image 50 is smoothed by performing the blurring process when the input image 40 has noise. Therefore, the blurred image 50 is an image having an effect as if noise has disappeared.

Returning to FIG. 3, in S <b> 11, the control unit 10 (process image generation unit 12) generates an edge blurred image 52 using the input image 40. Specifically, first, the control unit 10 (process image generation unit 12) performs edge extraction on the input image 40 to generate an edge extracted image (not shown). The control unit 10 generates an edge extracted image using, for example, a known Sobel filter. The edge extraction process performed here is a process of extracting, as an edge, a portion in the image where the gray value changes abruptly.
Next, the control unit 10 (process image generating unit 12) performs a blurring process on the generated edge extracted image to generate an edge blurred image 52. The blurring process is the same process as S10 described above. The control unit 10 generates the edge blurred image 52 from the edge extracted image using, for example, a known moving average filter.

The edge blurred image 52 generated here is referred to when the output image 60 is generated. The control unit 10 generates the edge blurred image 52 by, for example, down-sampling the input image 40, performing edge extraction processing and blurring processing on the image, and restoring the image to the size of the input image 40. You may. By doing so, the control unit 10 can improve processing efficiency as compared with performing processing directly on the input image 40.
FIG. 4C shows an edge blurred image 52 obtained by performing an edge extraction process and a blurring process on the input image 40 shown in FIG. 4A. Since the edge blurred image 52 is a blurred image of the edge extracted image, it is possible to make a smooth region between the region extracted as the edge and the region in contact with the region.

Returning to FIG. 3, in S13, the control unit 10 (the noise-removed image generation unit 13) performs a preceding-stage noise removal process.
Here, the pre-stage noise removal processing will be described with reference to FIG. This pre-stage noise removal process is a process for removing impulse noise.
In S20 of FIG. 5, the control unit 10 acquires one of the pixels P constituting the input image 40 and sets the acquired pixel as a target pixel Pa (i, j).
As shown in FIG. 6, the input image 40 can be represented as a pixel P (1, 1) at the upper left, with x pixels arranged in the right direction and y pixels arranged in the lower direction.

  Returning to FIG. 5, in S21, the control unit 10 determines whether or not there is a pixel P to the left and right of the pixel of interest Pa (i, j). In other words, the control unit 10 determines whether or not the input image 40 includes the pixel P (i−1, j) and the pixel P (i + 1, j). When there are pixels P on the left and right of the pixel of interest Pa (i, j) (S21: YES), the control unit 10 shifts the processing to S22. On the other hand, when there are no pixels P on the left and right of the target pixel Pa (i, j) (S21: NO), the control unit 10 shifts the processing to S23.

  In S22, the control unit 10 performs a signal value calculation process of the pixel of interest Pa (i, j). Specifically, the control unit 10 determines the pixel of interest Pa (i, j) and the pixels P (i−1, j) and P (i + 1, j) on the left and right of the pixel of interest Pa (i, j). , RGB signal values. Then, the control unit 10 determines the signal value M (i−1, j) of the pixel P (i−1, j), the signal value Ma (i, j) of the pixel of interest Pa (i, j), and the pixel P When the intensity of the signal value M (i + 1, j) of (i + 1, j) has any of the irregularities, that is, the signal value Ma (i, j) of the pixel of interest Pa (i, j) is When the signal value M of the pixel P is lower or higher than the signal value M of the pixel P, the signal value Ma (i, j) of the target pixel Pa (ij) is set to the average value of the signal values M of the left and right pixels P. .

Specifically, a new signal value Ma (i, j) of the target pixel Pa (i, j) is calculated by the following equation.

  Thereby, the control unit 10 can remove the impulse noise included in the input image 40. Here, the impulse noise is also referred to as salt-and-pepper noise. For example, an input image 40 captured and acquired by a digital camera may include noise during photography. This is because residual heat (thermal noise) from a previous image capturing may remain in the image sensor of the camera. In such a case, random noise occurs in the input image 40. Due to random noise, the input image 40 includes an image that shows an abnormal numerical value in pixel units. By the above-described processing, the pixels that exhibit abnormal numerical values on a pixel-by-pixel basis can be leveled. For this reason, the control unit 10 can remove the impulsive noise, and can change the image that gives the mottled feeling existing in the input image 40 to an image that has disappeared.

In S23, the control unit 10 determines whether or not all the pixels P of the input image 40 have been set to the target pixel Pa (i, j). If all the pixels P of the input image 40 have been set to the target pixel Pa (i, j) (S23: YES), the control unit 10 shifts the processing to S24. On the other hand, if all the pixels P of the input image 40 have not been set to the target pixel Pa (i, j) (S23: NO), the process returns to S20, and the pixels P that have not yet been processed are set to the target pixel Pa ( i, j) and the subsequent processing is repeated.
In S24, the control unit 10 applies the calculated signal value Ma to each pixel P of the input image 40, generates a processing source image 54, and stores it in the intermediate image storage unit 24. After that, the control unit 10 ends this processing, and returns the processing to FIG.

Returning to FIG. 3, in S13, the control unit 10 (the luminance value calculation unit 14, the image generation unit 15) performs an image synthesis process.
Here, the image combining process will be described with reference to FIG. This image synthesis processing is processing for generating an image from which luminance unevenness has been removed.
In S30 of FIG. 7, the control unit 10 acquires one pixel P of the processing source image 54 and sets it as a target pixel Pb (i, j).

In S31, the control unit 10 (the luminance value calculating unit 14) calculates the luminance value Br (i, j) of the target pixel Pb (i, j).
The luminance value Br (i, j) of the target pixel Pb (i, j) is calculated by the following equation, for example.

  In this equation, the weight of the G luminance value among the RGB signal values is increased. This takes into account human visual characteristics. Note that the calculation of the luminance value Br (i, j) is not limited to the above expression, and may be, for example, a value obtained by averaging the luminance values of the RGB signal values.

In S32, the control unit 10 (the luminance value calculating unit 14) calculates the luminance value BlBr (i, j) of the pixel Pc (i, j) at the same position as the target pixel Pb (i, j) in the blurred image 50. calculate.
The luminance value BlBr (i, j) of the pixel Pc (i, j) is calculated by the same formula as the luminance value Br (i, j).
In S33, the control unit 10 (the luminance value calculating unit 14) calculates the luminance value Mask (i, j) of the pixel Pd (i, j) at the same position as the target pixel Pb (i, j) in the edge blurred image 52. Is extracted to make the edge signal strength. The luminance value Mask (i, j) is represented by a numerical value from 0 to 255.

In S34, the control unit 10 (the brightness value calculation unit 14) calculates a new brightness value NBr (i, j) with reference to the edge signal strength.
The luminance value NBr (i, j) is calculated by, for example, the following equation.

  This equation is such that if the edge signal strength of the pixel Pd of the edge blurred image 52 is strong, the luminance value of the processing source image 54 is strongly reflected, and if the edge signal strength of the pixel Pd of the edge blurred image 52 is weak, the brightness of the blurred image 50 is Indicates that the luminance value is strongly reflected. In FIG. 4C, the white portion has a high edge signal intensity, and the black portion has a low edge signal intensity.

In S35, the control unit 10 (image generation unit 15) generates a new pixel Pn (i, j) by adding a color component to the new luminance value NBr (i, j). At that time, the control unit 10 changes a color component to be added depending on whether or not the luminance value NBr (i, j) is equal to or larger than the threshold value k. The threshold value k can be set to an arbitrary value from 0 to 255 depending on a parameter, for example. The pixel Pn (i, j) can be calculated by the following equation. Note that the color component is a luminance value Br (i, j) (or a luminance value BlBr (i, j)) from the RGB signal values of the target pixel Pb (i, j) (or pixel Pc (i, j)). Is subtracted from the RGB signal value.

  If the new luminance value NBr (i, j) is equal to or greater than the threshold value k, this equation is obtained by adding the color component of the processing source image 54 to the new luminance value NBr (i, j). If present, this indicates that the color component of the blurred image 50 is added to the new luminance value NBr (i, j).

In S36, the control unit 10 determines whether or not all the pixels P of the processing source image 54 have been set to the target pixel Pb (i, j). When all the pixels P of the processing source image 54 have been set to the target pixel Pb (i, j) (S36: YES), the control unit 10 shifts the processing to S37. On the other hand, if all the pixels P of the processing source image 54 have not been set to the target pixel Pb (i, j) (S36: NO), the process returns to S30, and the pixels of the processing source image 54 that have not been processed yet P is set as a target pixel Pb (i, j), and the subsequent processing is repeated.
In S37, the control unit 10 (image generation unit 15) generates an image including the generated new pixels Pn as the output image 60, and stores the generated image in the generated image storage unit 25. After that, the control unit 10 returns the processing to FIG. 3, and thereafter ends this processing.

As described above, according to the image processing apparatus 1 of the present embodiment, the following effects can be obtained.
(1) The information of the input image 40 (actually, the processing source image 54) is mainly used for the portion where the edge signal intensity of the edge blurred image 52 is strong, and the luminance is mainly used for the portion where the edge signal intensity is weak. Since the information of the blurred image 50 from which the unevenness (noise) has been removed is used, as a result, an image in which both noise removal and image clarity can be obtained can be obtained.
The edge blurred image 52 is an image obtained by blurring the edge extracted image. The edge signal intensity has a value of 0 to 255, and if the edge signal intensity is high, the input image 40 (actually, the processing source image 54) If the ratio of pixel values is increased and if the edge signal intensity is low, the ratio of pixel values of the blurred image 50 is increased. Generate Therefore, an image having a more natural texture can be generated as compared with the alternative selective synthesis processing of the input image and the blurred image.
(2) By making the processing source image 54 an image from which the impulse noise has been removed from the input image 40 and using the image for synthesis, the effect of (1) can be further enhanced.

  The embodiments of the present invention have been described above, but the present invention is not limited to the above-described embodiments. Further, the effects described in the embodiments merely enumerate the most preferable effects resulting from the present invention, and the effects according to the present invention are not limited to those described in the embodiments. Note that the above-described embodiment and the modified examples described below can be used in appropriate combinations, but detailed description is omitted.

(Modified form)
(1) In the present embodiment, the blurring process uses a known moving average filter, and the edge extraction process uses a known Sobel filter. However, the present invention is not limited to this. Any image processing technique may be used as long as a blurred image and an edge-extracted image can be generated. For example, a Laplacian filter may be used for the edge detection processing.
(2) In the present embodiment, an example has been described in which a new luminance value is calculated, and a value obtained by adding the color component of the processing source image or the blurred image to the calculated new luminance value is used as a new pixel. However, the present invention is not limited to this. For example, alpha blending of the processing source image and the blurred image may be performed using the new luminance value as the α value. Even in such a case, it is possible to remove an image noise and generate an image having a natural texture without luminance unevenness.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Control part 11 Blurred image generation part 12 Process image generation part 13 Noise removal image generation part 14 Brightness value calculation part 15 Image generation part 20 Storage part 21a Image processing program 40 Input image 50 Blurred image 52 Edge blurred image 54 Source image 60 Output image Br Luminance value BlBr Luminance value k Threshold value M, Ma Signal value Mask Luminance value NBr Luminance value P, Pc, Pd Pixel Pa, Pb Target pixel

Claims (5)

A blur image generating means for performing a blur process on the input image to generate a blur image,
A process image generating unit that performs edge extraction on the input image to generate an edge extracted image, and then performs a blur process on the edge extracted image to generate an edge blurred image;
For each pixel, a new brightness value is calculated using the brightness value of the input image and the brightness value of the blurred image based on the brightness value of the edge blurred image generated by the process image generating means. Brightness value calculating means,
Based on the new luminance value threshold calculated by the luminance value calculating means, one of the color component of the input image and the color component of the blurred image is selected, and the new luminance value Image generation means for generating a new image by providing a color component ;
An image processing apparatus comprising: The image processing device according to claim 1,
A noise-removed image generating unit configured to generate a noise-removed image in which noise is removed from the input image,
The image generating unit uses the noise-removed image generated by the noise-removed image generating unit instead of the input image, and selects one of a color component of the noise-removed image and a color component of the blurred image. Generating a new image by adding the color component of the selected pixel to the new luminance value ;
An image processing apparatus characterized by the above-mentioned. A program for causing a computer to function as the image processing device according to claim 1 . Perform blur processing on the input image to generate a blurred image,
After performing edge extraction on the input image to generate an edge extracted image, perform a blur process on the edge extracted image to generate an edge blurred image,
For each pixel, based on the generated brightness value of the edge blurred image , calculate a new brightness value using the brightness value of the input image and the brightness value of the blurred image,
Based on the calculated threshold value of the new luminance value, one of the color component of the input image and the color component of the generated blurred image is selected, and the color component of the selected pixel is selected as the new luminance value. by applying, to generate a new image,
Image processing method. Perform blur processing on the input image to generate a blurred image,
After performing edge extraction on the input image to generate an edge extracted image, perform a blur process on the edge extracted image to generate an edge blurred image,
Generating a noise-removed image from which noise has been removed for the input image,
For each pixel, based on the generated brightness value of the edge blurred image, calculate a new brightness value using the brightness value of the input image and the brightness value of the blurred image,
Based on the calculated threshold value of the new luminance value, one of the color component of the generated noise-removed image and the color component of the generated blurred image is selected, and the selected pixel is used as the new luminance value. By generating a new image by adding the color components of
Image processing method.