JP4380399B2 - Imaging apparatus, noise reduction apparatus, noise reduction method, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, a noise reduction apparatus, a noise reduction method, and a program that can effectively remove noise contained in video signals and image signals.

  In Patent Document 1 below, a level difference between a target pixel to be subjected to noise reduction (noise reduction processing) and peripheral pixels around the target pixel is compared with a threshold value. An invention of a noise reduction device based on a nonlinear smoothing method using ε filtering that replaces a pixel of interest is described.

JP-A-6-86104

  The noise reduction using ε filtering will be described with reference to FIG. FIG. 9A shows values of a plurality of adjacent pixels in the flat portion of the image. For example, it is assumed that the detection area is set to three pixels on the left and right for the target pixel 101 to be subjected to noise reduction, and smoothing is performed. The pixels (pixel 102 to pixel 107) in the detection region have a difference (level difference) from the pixel level when there is no noise. This level difference is displayed as noise. Here, if the absolute value of the level difference between the target pixel and each pixel in the detection region is within the threshold value set around the value of the target pixel, the target pixel 101 and the pixels in the detection region (pixel 102 to pixel 107). The pixel level addition average is calculated and the filter processing is performed. Here, the left-right direction is taken as an example, but the same applies to the up-down and diagonal directions as long as the detection frame forms a planar area.

  In noise reduction using ε filtering, a pixel whose absolute value of level difference between the target pixel and a pixel in the detection region is within a certain threshold is determined to be correlated with respect to the signal component, and the target pixel and the target pixel An average of pixel levels of pixels determined to have a correlation is calculated. In noise reduction using ε filtering, pixels having a large level difference from the pixel of interest such as an edge portion (contour portion) are not used. Therefore, in the flat portion of the image as shown in FIG. 9A, noise reduction can be performed while smoothing the edge and suppressing deterioration of the frequency characteristics as much as possible.

  However, the conventional noise reduction described above has the following problems. FIG. 9B shows the value of each pixel in the ramp portion of the image where the level gradually changes in the plane. As shown in FIG. 9B, among the plurality of pixels in the detection region, the pixels whose absolute value of the level difference from the target pixel is within the threshold are the pixel 204 and the pixel 205. Even if smoothing is performed by the pixel of interest 201 and the pixel whose threshold value is within the detection region in the ramp portion of the image, the number of pixels used for calculating the addition average is reduced compared to the flat portion of the image. Therefore, there is a problem that the effect of noise reduction in the lamp portion is reduced.

  In addition, in an image in which a flat portion and a lamp portion are mixed, the noise reduction effect is obtained in the flat portion, but the effect is reduced in the lamp portion, and as a result, the noise feeling of the lamp portion is conspicuous. there were.

  Further, in the conventional noise reduction, there is a problem that the center of gravity of the target pixel is shifted after noise reduction depending on which peripheral pixel is used for noise reduction, and linearity in the image is remarkably lost. Furthermore, there has been a problem that the noise reduction effect cannot be expected when, for example, impulse-like protruding noise is present on the target pixel.

  Accordingly, an object of the present invention is to provide an image pickup apparatus that can perform good noise reduction even in a ramp portion of an image, and can perform noise reduction uniformly even in either a flat portion or a ramp portion of an image, A noise reduction apparatus, a noise reduction method, and a program are provided. Another object of the present invention is to provide an imaging device, a noise reduction device, a noise reduction method, and a program that do not shift the center of gravity of a pixel of interest after noise reduction and that can expect a noise reduction effect even for protruding noise. is there.

In order to solve the above-described problem, the invention according to claim 1 is directed to a noise reduction device that performs noise reduction based on an average value of a target pixel to be noise-reduced and a plurality of peripheral pixels located around the target pixel. In the provided imaging device,
Means for taking out the target pixel and a plurality of peripheral pixels;
A comparing means for performing a second derivative using the pixel of interest and a plurality of surrounding pixels and comparing the second derivative with a threshold;
As a result of comparison by the comparison means, when the secondary differential value is larger than the threshold, it is determined that a plurality of peripheral pixels cannot be used for noise reduction, and when the secondary differential value is less than the threshold, the plurality of peripheral pixels are used for noise reduction. Discriminating means for discriminating that it is possible ;
An arithmetic means for adding the pixel value of the target pixel and the pixel values of peripheral pixels determined to be usable for noise reduction by the determination means, and dividing the addition result by the addition number;
The image pickup apparatus includes an output unit that outputs the pixel value of the target pixel by replacing the pixel value calculated by the calculation unit .

The invention according to claim 4 is a noise reduction device that performs noise reduction based on an average value of a target pixel to be subjected to noise reduction and a plurality of peripheral pixels located around the target pixel.
Means for taking out the target pixel and a plurality of peripheral pixels;
A comparing means for performing a second derivative using the pixel of interest and a plurality of surrounding pixels and comparing the second derivative with a threshold;
As a result of comparison by the comparison means, when the secondary differential value is larger than the threshold, it is determined that a plurality of peripheral pixels cannot be used for noise reduction, and when the secondary differential value is less than the threshold, the plurality of peripheral pixels are used for noise reduction. Discriminating means for discriminating that it is possible ;
An arithmetic means for adding the pixel value of the target pixel and the pixel values of peripheral pixels determined to be usable for noise reduction by the determination means, and dividing the addition result by the addition number;
The noise reduction device includes an output unit that outputs the pixel value of the target pixel by replacing the pixel value calculated by the calculation unit .

The invention according to claim 7 is a noise reduction method in which noise reduction is performed by an average value of a target pixel to be subjected to noise reduction and a plurality of peripheral pixels located around the target pixel.
A step of taking out the target pixel and a plurality of peripheral pixels;
Performing a second derivative using the pixel of interest and a plurality of surrounding pixels, and comparing the second derivative with a threshold;
If the secondary differential value is greater than the threshold value as a result of the comparison in the comparison step, it is determined that multiple peripheral pixels cannot be used for noise reduction. If the secondary differential value is less than or equal to the threshold value, multiple peripheral pixels are used for noise reduction. A determination step for determining that it is possible ;
A calculation step of adding the pixel value of the target pixel and the pixel values of peripheral pixels determined to be usable for noise reduction by the determination step, and dividing the addition result by the addition number;
This is a noise reduction method including an output step of outputting the pixel value of the pixel of interest by replacing the pixel value calculated by the calculation step .

Invention, a computer according to claim 1 0,
A pixel of interest for noise reduction, and a step of extracting a plurality of peripheral pixels located around the pixel of interest;
Performing a second derivative using the pixel of interest and a plurality of surrounding pixels, and comparing the second derivative with a threshold;
If the secondary differential value is greater than the threshold value as a result of the comparison in the comparison step, it is determined that multiple peripheral pixels cannot be used for noise reduction. If the secondary differential value is less than or equal to the threshold value, multiple peripheral pixels are used for noise reduction. A determination step for determining that it is possible ;
A calculation step of adding the pixel value of the target pixel and the pixel values of peripheral pixels determined to be usable for noise reduction by the determination step, and dividing the addition result by the addition number;
This is a program for executing a process including an output step of outputting a pixel value of a target pixel by replacing the pixel value calculated in the calculation step .

  According to the present invention, a noise reduction effect can be obtained with respect to a target pixel to be subjected to noise reduction. Also, unlike the conventional method of detecting by comparing the level difference with the threshold value, the comparison target with the threshold value is a second-order differential result, so that sufficient noise reduction can be performed even in the ramp part, and in the flat part Noise reduction without variation in the noise reduction effect can be achieved even in the case of a part.

According to the inventions according to claim 2, claim 5, claim 8 and claim 11 , even if impulse noise or the like is applied to the pixel of interest or the pixels around the pixel of interest, smoothing processing performed before detection Therefore, it is possible to realize noise reduction that is not easily influenced by noise.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, the imaging apparatus is described as a digital video camera. However, the imaging apparatus is not limited to a digital video camera, and can be applied to other imaging apparatuses (for example, a digital still camera, a camcorder, etc.).

  FIG. 1 shows a signal processing system 11 of a digital video camera according to an embodiment of the present invention. The signal processing system 11 includes an image sensor 21, a delay line (DL (Delay Line)) block 22, a noise reduction processing block 23, and a camera signal processing block 24.

  The image pickup device 21 is, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) imager, and photoelectrically converts an image of a subject and outputs an image signal. The output image signal is supplied to the DL block 22, and the output image signal of the DL block 22 is supplied to the noise reduction processing block 23.

  The image signal subjected to noise reduction in the noise reduction processing block 23 is supplied to the camera signal processing block 24. The camera signal processing block 24 converts the image output signal into a luminance signal and a color difference signal, performs resolution conversion when the input image exceeds the number of pixels of the video size (horizontal direction 720 pixels × vertical direction 480 lines), and outputs the video size. It is a signal processing block which outputs the amount of pixel information. The camera signal processing block 24 adaptively adjusts the image size after resolution conversion to an arbitrary size so that the output of the digital video camera is a still image and the output destination is a personal computer or the like. It has a function that can be converted. From the camera signal processing block 24, a luminance signal and a color difference signal of an image signal subjected to camera signal processing are output.

  It should be noted that the camera signal processing block 24 may be configured to be positioned in front of the noise reduction processing block 23 as shown in FIG. In this case, the image signal output from the image sensor 21 is supplied to the camera signal processing block 24 via the DL block 22. In the camera signal processing block 24, camera signal processing is performed on an image signal that has not been subjected to noise reduction, and an image signal converted into a luminance signal and a color difference signal is supplied to the noise reduction processing block 23. In the noise reduction processing block 23, noise reduction processing is performed on each of the supplied luminance signal and color difference signal.

  There are n types (n: positive integer) of color information of the signal supplied to the noise reduction processing block 23, and noise is detected for m types (m: positive integer, n ≧ m) of the n types. When performing reduction, m noise detection processes are performed in parallel. Here, of the m types of color information, when it is known that the spatial phase is the same for k types (k: positive integer, m ≧ k), if the spatial phase is equal, the pixel level Since (color level) will show a similar change, m-k + 1 noise detection processes can be performed in parallel to reduce the number of processes. Here, the noise reduction processing block 23 requires input of an arbitrary number of lines delayed in the vertical direction, and this vertical delay is processed in the DL block 22.

  In the parallel processing described above, for example, when there are three types of inputs of the first color signal, the second color signal, and the third color signal, and three types of noise reduction processing are desired to be performed independently, the noise reduction processing is performed in parallel. Do it independently. In addition, when there are three types of inputs, a first color signal, a second color signal, and a third color signal, and the spatial phase of the first color signal and the second color signal overlaps, noise detection processing is performed. The first color signal and the third color signal are applied to the first color signal, the noise correction is performed on the first color signal and the second color signal based on the detection result of the first color signal, and the third color signal is obtained. Based on the detection result, noise correction may be performed on the third color signal.

  Next, an example of the noise reduction processing block 23 will be described. As shown in the frame of FIG. 1 and FIG. 2, the noise reduction processing block 23 includes a processing block (LPF block 31) using LPF (Low Pass Filter), a secondary differentiation processing block 32, a pixel discrimination block 33, a threshold calculation block. 34, a symmetric processing block 35, an interpolation pixel addition / division block 36. Although details will be described later, since the processing in the symmetric processing block 35 can also be performed in the secondary differentiation processing block 32, the symmetric processing block 35 is indicated by a broken line in FIGS.

  First, the process of the LPF block 31 will be described. FIG. 3 shows an example in which the G component is extracted from the RGB components of the Bayer color filter array (Bayer array) provided in the image sensor 21. In this embodiment, the detection area 41 is set to an area of 5 pixels in the vertical direction and 5 pixels in the horizontal direction, and the center pixel Gc is set as a target pixel to be subjected to noise reduction. Within the detection region, a total of four second-order differential directions are defined in the horizontal direction, the vertical direction, and the diagonal direction (two directions). The second order differentiation is performed by a 3-tap high-pass filter centered on the target pixel. The detection area can be arbitrarily set, for example, 7 lines in the vertical direction and 9 pixels in the horizontal direction. As for the secondary differentiation, the direction of the secondary differentiation can be arbitrarily selected as long as LPF can be performed in the direction orthogonal to the secondary differentiation. In the LPF processing block 31, smoothing and interpolation processing is performed on each pixel in the detection region 41 in a direction orthogonal to the second-order differential direction.

FIG. 4 shows the smoothing process in this embodiment, and the broken line shows the secondary differential direction for performing the secondary differentiation. Smoothing is performed in a direction orthogonal to the second-order differential direction. For example, smoothing is performed on the arrangement of the pixel G ₁₁ , the pixel G _31, and the pixel G _{51 that} are orthogonal to the secondary differential direction. Smoothing is performed by a method of adding pixel levels of each pixel and dividing by the number of additions.

In the LPF block 31, not only smoothing but also interpolation processing is performed. At a position where no pixel exists in the second-order differential direction, interpolation is performed with the addition average of the two pixels as the value of the non-existing pixel. For example, the pixel G ₃₂ is interpolated by the average value of the pixel G ₂₂ and the pixel G ₄₂ . Further, the pixel G ₃₄ is interpolated by the average value of the pixel G ₂₄ and the pixel G ₄₄ .

The LPF block 31 also performs smoothing in the direction orthogonal to the second-order differential direction in the vertical direction and the left-right diagonal direction as well. Further, pixels that are located at positions such as the pixel G ₁₁ , the pixel G ₁₅ , the pixel G ₅₁ , and the pixel G ₅₅ and cannot be smoothed only by the pixels in the detection region are smoothed by using pixels outside the detection region. It is possible to arbitrarily perform or not perform smoothing.

  As described above, by performing smoothing and interpolation processing on each pixel in the detection area in the LPF block 31, impulse noise protruding to each pixel is calculated when calculating the second derivative value. Even in the case of deterioration, the detection accuracy can be prevented from deteriorating. Further, by performing smoothing in a direction uncorrelated with the secondary differential direction, the reliability of the result of the secondary differential processing block 32 can be improved.

After smoothing and interpolation processing, secondary differentiation is performed in the secondary differentiation processing block 32. FIG. 5 shows an arrangement of pixels, for example, in the horizontal direction for performing second order differentiation. The central pixel Gc is a pixel of interest that performs noise reduction, and the pixel G ₃₁ , pixel G ₃₂ , pixel G _34, and pixel G ₃₅ are peripheral pixels. As described above, the pixel G ₃₂ and the pixel G ₃₄ are pixels interpolated from the pixels in the direction orthogonal to the secondary differential direction.

In this embodiment, the case where it is determined whether or not the pixel G ₃₁ can be used for noise reduction will be described. First, second order differentiation is performed among the pixel G ₃₂ , the pixel Gc, and the pixel G ₃₄ (Process 1). The 3-tap coefficient for performing the second order differentiation can be fixed as (−1, 2, −1), or can be set adaptively.

The absolute value of the second-order differential value performed between the pixel G ₃₂ , the pixel Gc, and the pixel G _{34 in} the process 1 is compared with a threshold value in the pixel determination block 33. The threshold value is a threshold value calculated by the threshold value calculation block 34, calculated based on the pixel level of the target pixel, or an arbitrarily set threshold value. For example, for the purpose of reducing light shot noise, the threshold value is set to a constant multiple of the square root of the target pixel level. For the purpose of reducing light shot noise and random noise, the threshold value is set to a value obtained by adding another constant (a constant or variable indicating a random noise level) to a constant multiple of the square root of the target pixel level.

  The reason why the second-order differentiation is used unlike the conventional level difference is to prevent the number of pixels that can be used for noise reduction also in the ramp portion. That is, a constant value corresponding to the slope is maintained when the ramp portion changing by a linear function is first-order differentiated, and the value becomes zero when the constant value is second-order differentiated. Therefore, when the secondary differential value is equal to or less than the threshold value, it means that the signal component of the pixel has a correlation, and it can be determined that it can be used for noise reduction.

Comparison of results in the pixel decision block 33, if the second derivative the result is greater than the threshold value, determines that the pattern of the edge exists between the pixels G ₃₂ and the pixel Gc or pixel Gc and the pixel G _34, pixel G _31, It is determined that the pixel G ₃₁ cannot be used for noise reduction of the pixel Gc without performing a second-order differentiation process between the pixel Gc and the pixel G ₃₅ . Even if it is determined that there is a correlation with the pixel of interest, a pixel at a position across a different picture area or edge with respect to a certain pixel of interest is the original pixel level or noise at the pixel level. It is not used for noise reduction because it is not possible to distinguish whether it is included or it is determined that there is a correlation.

When the secondary differential result is less than or equal to the threshold, it is assumed that the secondary differential result has detected a noise component. If the secondary differentiation result by the process 1 is not more than the threshold value, the secondary differentiation is performed among the pixel G ₃₁ , the pixel Gc, and the pixel G ₃₅ (process 2). If the result of the second derivative by the process 2 is equal to or less than the threshold value, it is determined that there is no pattern edge between the pixel G ₃₁ and the pixel Gc, in other words, a flat portion, and the pixel G ₃₁ is a noise of the target pixel Gc. Determine that it can be used for reduction. If second derivative the result is greater than the threshold value, it is determined that the pattern of the edge exists between the pixels G ₃₁ and between the pixel Gc or pixel Gc and the pixel G _35, pixel G ₃₁ can not be used for noise reduction of the pixel of interest Gc Is determined.

  The secondary differentiation is preferably performed from a pixel in the vicinity of the target pixel Gc. By performing the second order differentiation from the inside to the outside with respect to the target pixel Gc, the processing steps of the second order differentiation can be reduced. Further, by using pixels as close as possible, it is possible to perform second order differentiation while taking advantage of the frequency characteristics of the image.

  In this embodiment, the second order differentiation is performed on three pixels including the target pixel. However, an arbitrary number of pixels such as five pixels can be set.

Second derivative, the process 1 and process not only 2, pixel Gc, pixel G _31, second derivative (process 3) and carried out between the pixels G _32, pixel Gc, pixel G _34, between the pixel G ₃₅ The secondary differentiation (processing 4) to be performed can also be performed. In this embodiment, the detection area by the second derivative is set to 5 pixels in the horizontal direction. However, when the detection area becomes wider, preliminary detection such as processing 3 and processing 4 is effective in increasing detection accuracy. Is.

As described above, the second order differentiation is performed by overlapping the second order differential region as much as possible between the pixel to be used to determine whether it can be used for noise reduction and the target pixel. However, since the secondary differential region cannot be overlapped when the pixel to be determined whether it can be used for noise reduction is an adjacent pixel between the target pixels, the secondary differential process is performed only once. For example, in the pixel of interest is pixel Gc, pixels of the discrimination object is a case where a positional relationship of the pixel G _32.

  Note that the interpolation pixel is used only for second-order differential detection, and is not a target for determining whether it can be used for noise reduction. In other words, noise reduction is not performed using the interpolation pixel.

  The second-order differential processing is performed in the same way in the vertical direction and the left-right diagonal direction, and pixels that can be used for noise reduction are detected from the pixels located around the target pixel. By making the comparison target with the threshold value a secondary differential result from the conventional level difference, the effect of noise reduction can be obtained even in the ramp portion of the image.

Next, processing of the symmetric processing block 35 will be described. In the symmetry processing block 35, the symmetry of a pixel determined to be usable for noise reduction is determined. The symmetry is determined for the following reason. For example, it is assumed that the pixel determination block 33 determines that the pixel G ₃₁ can be used for noise reduction and the pixel G ₃₅ cannot be used for the target pixel Gc. In this case, if noise reduction by the pixel G ₃₁ is executed, the center of gravity of the target pixel Gc after noise reduction may be shifted.

FIG. 6 shows an example of symmetry point processing. For example, as a result of the second order differentiation in all directions, the pixel discrimination block 33 can use the pixel G ₁₅ , the pixel G ₂₂ , the pixel G ₄₄ , and the pixel G ₅₃ for the target pixel Gc for noise reduction. Assume that it has been determined. In the symmetric processing, it is determined whether or not there is a pixel at a point-symmetrical position around the target pixel Gc among the pixels that can be used for noise reduction.

As shown in FIG. 6, the pixel G ₂₂ and the pixel G ₄₄ are in point-symmetric positions with the target pixel Gc as the center, and since both pixels are determined to be usable for noise reduction, they are used for noise reduction. The other pixels G ₁₅ and G ₅₃ have no pixels that are determined to be usable for noise reduction at point-symmetrical positions, and may cause a shift in the center of gravity of the pixel of interest and degrade linearity. Is determined to be unusable.

Note that the symmetry processing may be performed in the secondary differentiation processing block 32. That is, the secondary differentiation may be performed on peripheral pixels that are at least point-symmetrical with respect to the target pixel. For example, as shown in FIG. 5, whether or not the pixels (for example, the pixel G ₃₂ and the pixel G ₃₄ ) at symmetrical positions can be used for noise reduction by performing at least the second differentiation of the processing 1 and the processing 2. May be discriminated.

In the interpolation pixel addition / division block, all the values of the pixels that can be used for noise reduction and the pixel value of the target pixel Gc obtained by the series of processes described above are added and divided by the number of additions to obtain noise and noise. Only the determined pixels are smoothed to generate a noise reduction effect. In the case where a pixel that can be used for noise reduction is not present at all, the pixel level of the pixel of interest G _C is output as it is.

  FIG. 7 is a flowchart showing the flow of noise reduction processing according to an embodiment of the present invention. In step S1, the presence / absence of a pixel missing in the vertical direction, the horizontal direction, and the left / right diagonal direction is determined with the target pixel to be noise reduced as the center. Here, the color of the pixel is the same color or an arbitrary color type with little difference in pixel level.

  In the example described with reference to FIG. 3, attention is paid to the G component of the RGB three primary color signals. This is because the G component is the main component of the luminance signal and the human visibility to the G component is low. The reason is that it is high or there are many G pixels on the image sensor. The R component and the B component may be detected independently, or only the R component and only the B component may be detected. It is also possible to apply the detection result of the G component to the R component and the B component, or apply the detection result of the arbitrary color component to the arbitrary color component.

  If there is a missing pixel, the operation proceeds to step S2. In step S2, pixel interpolation is performed from the direction orthogonal to the secondary differential direction to the missing pixel position. After performing the pixel interpolation, the operation returns to step S1.

  When there is no missing pixel or after the missing pixel is interpolated, in step S3, smoothing by LPF is performed from the direction orthogonal to the second order differentiation with the target pixel to be noise reduced as the center. After the smoothing by the LPF, the operation proceeds to step S4.

  Next, in step S4, the second order differentiation is performed in the vertical, horizontal, and diagonal directions with respect to the target pixel for all the detection pixels within the detection frame. The secondary differentiation is performed using, for example, three pixels between the pixels having a point-symmetric positional relationship with the target pixel as the center. When the interval between the end pixel and the target pixel is two pixels or more, the second order differentiation may be performed for all combinations located between the end pixel and the target pixel. The second-order differential direction can be set to any direction as long as LPF is possible in a direction orthogonal to the second-order differential direction.

  In step S5, the threshold value is calculated based on the value of the pixel level for which noise reduction is to be performed. Alternatively, an arbitrary threshold value may be set.

  When the secondary differentiation process is finished in step S4, the absolute value of the secondary differentiation result calculated in step S4 and the threshold value calculated in step S5 are compared in step S6.

  In step S7, attention is focused on the target pixel for determining whether it can be used for noise reduction. This target pixel is sequentially shifted to a different pixel, and the process continues until all pixels in the detection frame including the interpolated pixel are focused except for the target pixel.

  It is determined whether or not there is a result of the secondary differential result larger than the threshold value between the pixel of interest in step S7 and the pixel of interest (step S8). If the secondary differential value is larger than the threshold value, the operation proceeds to step S9, and it is determined that the pixel focused at that time cannot be used for noise reduction, and the operation proceeds to step S11.

  As a result of the determination in step S8, if there is no result whose secondary differential result is larger than the threshold value between the focused pixel and the focused pixel, it is determined that the focused pixel at that time can be used for noise reduction, and the operation is step Proceed to S11.

  In the operation of step S11, it is determined whether or not all the detection pixels within the detection frame are focused. If there is still a pixel to be detected with attention, the operation proceeds to step S12, and it is determined that there is a pixel to be detected with new attention, and the processing from step S7 is performed again.

  In step S13, when the processing from step S1 to step S12 is completed and the detection is completed, all the pixels determined to be usable for noise reduction and the target pixel are added, and the noise of the target pixel is divided by the addition number. Reduction is performed and the process ends.

  In the flowchart shown in FIG. 7, the second order differentiation is performed in consideration of the symmetry of surrounding pixels (step S4). However, a symmetry process may be performed separately.

  FIG. 8 shows a flowchart in the case of performing symmetric processing. Since the process from step S1 to step S12 is the same, description is abbreviate | omitted. After paying attention to all the detection pixels in the detection frame in step S11, the process proceeds to step S21.

  In step S21, it is determined whether or not there is a pixel that can be used for noise reduction (NR) among the detected pixels. If it is determined that there is no pixel that can be used for noise reduction, the noise reduction process ends. In this case, the pixel level of the target pixel is output as it is. If it is determined that there is a pixel that can be used for noise reduction, the operation proceeds to step S22.

  In step S22, it is further determined whether or not there is a pixel having a point-symmetric positional relationship with respect to the pixel of interest among the pixels that can be used for the noise reduction determined in step S21. If there is no pixel that is in a point-symmetrical positional relationship, noise reduction is not performed and the process ends. If there are pixels in point symmetry, these pixels are determined as pixels that can be used for noise reduction, and the operation proceeds to step S13.

  In step S13, the pixel of interest is added to all of the pixels that are determined to be usable for noise reduction and that have a point-symmetrical positional relationship with the pixel of interest as the center, and is divided by the addition number, thereby reducing the noise of the pixel of interest. And the process ends.

  The present invention can be variously modified and applied without departing from the gist of the present invention, and is not limited to the above-described embodiment. For example, the noise reduction described above can be operated at an arbitrary position during camera signal processing, and can also be applied to primary color, complementary color, raw color raw data or luminance signal, and color difference signal. .

1 is a block diagram of an imaging apparatus according to an embodiment of the present invention. It is a block diagram of the other example of the imaging device in one Embodiment of this invention. It is a figure which shows the arrangement | sequence of G component of the image in one Embodiment of this invention. It is a basic diagram which shows the smoothing process and interpolation process in one Embodiment of this invention. It is a basic diagram which shows the secondary differentiation process in one Embodiment of this invention. It is a basic diagram which shows the symmetrical process in one Embodiment of this invention. It is a flowchart of the noise reduction in one Embodiment of this invention. It is another example of the flowchart of the noise reduction in one Embodiment of this invention. It is a basic diagram which shows the noise reduction process using an epsilon filter.

Explanation of symbols

23 Noise reduction processing block 24 Camera signal processing block 31 LPF
32 Secondary differentiation processing block 33 Pixel discrimination block 34 Threshold calculation block 35 Symmetric processing block 36 Interpolated pixel addition / division block

Claims (12)

In an imaging device including a noise reduction device that performs noise reduction according to an average value of a target pixel to be subjected to noise reduction and a plurality of peripheral pixels located around the target pixel,
Extraction means for extracting the target pixel and the plurality of peripheral pixels;
A comparison means for performing a second derivative using the target pixel and the plurality of surrounding pixels, and comparing the second derivative with a threshold value;
As a result of comparison by the comparison means, when the secondary differential value is larger than the threshold value, it is determined that the plurality of surrounding pixels cannot be used for noise reduction, and when the secondary differential value is equal to or less than the threshold value, Discriminating means for discriminating that peripheral pixels can be used for noise reduction ,
Arithmetic means for adding the pixel value of the target pixel and the pixel values of peripheral pixels determined to be usable for noise reduction by the determination means, and dividing the addition result by the addition number;
An image pickup apparatus comprising: an output unit that outputs the pixel value of the target pixel by replacing the pixel value calculated by the calculation unit. Furthermore, smoothing processing means for smoothing the target pixel and the plurality of peripheral pixels in a direction orthogonal to the secondary differential direction for performing the secondary differentiation,
The imaging apparatus according to claim 1, further comprising an interpolation unit that interpolates the missing pixel when there is a missing pixel in the secondary differential direction.   The imaging apparatus according to claim 1, wherein the second-order differentiation is performed on the plurality of peripheral pixels having a point-symmetric positional relationship with respect to at least the pixel of interest. In a noise reduction device that performs noise reduction by an average value of a target pixel to be noise-reduced and a plurality of peripheral pixels located around the target pixel,
Extraction means for extracting the target pixel and the plurality of peripheral pixels;
A comparison means for performing a second derivative using the target pixel and the plurality of surrounding pixels, and comparing the second derivative with a threshold value;
As a result of comparison by the comparison means, when the secondary differential value is larger than the threshold value, it is determined that the plurality of surrounding pixels cannot be used for noise reduction, and when the secondary differential value is equal to or less than the threshold value, Discriminating means for discriminating that peripheral pixels can be used for noise reduction ,
Arithmetic means for adding the pixel value of the target pixel and the pixel values of peripheral pixels determined to be usable for noise reduction by the determination means, and dividing the addition result by the addition number;
A noise reduction device comprising: output means for replacing the pixel value of the pixel of interest with a pixel value calculated by the arithmetic means and outputting the pixel value . Furthermore, smoothing processing means for smoothing the target pixel and the plurality of peripheral pixels in a direction orthogonal to a secondary differential direction for performing the secondary differentiation,
The noise reduction device according to claim 4 , further comprising an interpolation unit that interpolates the missing pixel when there is a missing pixel in the secondary differential direction. 5. The noise reduction device according to claim 4 , wherein the second-order differentiation is performed on the plurality of peripheral pixels having a point-symmetric positional relationship with respect to at least the pixel of interest. In the noise reduction method in which noise reduction is performed by an average value of a target pixel to be noise-reduced and a plurality of peripheral pixels located around the target pixel,
A step of taking out the target pixel and the plurality of surrounding pixels;
Performing a second derivative using the pixel of interest and the plurality of surrounding pixels, and comparing the second derivative with a threshold;
As a result of the comparison in the comparison step, when the secondary differential value is larger than the threshold value, it is determined that the plurality of surrounding pixels cannot be used for noise reduction, and when the secondary differential value is equal to or less than the threshold value, A determination step for determining that peripheral pixels can be used for noise reduction ;
A calculation step of adding the pixel value of the pixel of interest and the pixel values of peripheral pixels determined to be usable for noise reduction by the determination step, and dividing the addition result by the addition number;
A noise reduction method comprising: an output step of outputting the pixel value of the pixel of interest by replacing the pixel value calculated by the calculation step . Further, a smoothing step for smoothing the target pixel and the plurality of peripheral pixels in a direction orthogonal to a secondary differential direction for performing the secondary differentiation,
The noise reduction method according to claim 7 , further comprising an interpolation step of interpolating the missing pixel when there is a missing pixel in the secondary differential direction. The noise reduction method according to claim 7 , wherein the second-order differentiation is performed on the plurality of peripheral pixels having a point-symmetric positional relationship with respect to at least the pixel of interest. On the computer,
A target pixel to be subjected to noise reduction, and a step of taking out a plurality of peripheral pixels located around the target pixel;
Performing a second derivative using the pixel of interest and the plurality of surrounding pixels, and comparing the second derivative with a threshold;
As a result of the comparison in the comparison step, when the secondary differential value is larger than the threshold value, it is determined that the plurality of surrounding pixels cannot be used for noise reduction, and when the secondary differential value is equal to or less than the threshold value, A determination step for determining that peripheral pixels can be used for noise reduction ;
A calculation step of adding the pixel value of the pixel of interest and the pixel values of peripheral pixels determined to be usable for noise reduction in the determination step, and dividing the addition result by the addition number;
A program for executing a process including an output step of outputting the pixel value of the target pixel by replacing the pixel value calculated by the calculation step . A smoothing step in which the processing further smoothes the target pixel and the plurality of surrounding pixels in a direction orthogonal to a secondary differential direction for performing the secondary differentiation;
The program according to claim 10, further comprising an interpolation step of interpolating the missing pixel when there is a missing pixel in the secondary differential direction. The program according to claim 10, wherein the second-order differentiation is performed on the plurality of peripheral pixels having a point-symmetric positional relationship with respect to at least the target pixel.

Images