JP6324235B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method.

  In recent years, a short-exposure image (hereinafter also simply referred to as “short-exposure image”) and a long-exposure image (hereinafter also simply referred to as “long-exposure image”) are continuously photographed and combined. An imaging function called WDR (wide dynamic range) or HDR (high dynamic range) that obtains an image that captures a dynamic range that exceeds the dynamic range that the sensor can shoot is increasing. Such a photographing function is particularly effective in a scene with a very large contrast ratio such as a composition of backlight.

  As a technique for obtaining a WDR signal, a technique is generally adopted in which a plurality of images are continuously photographed and combined while changing the exposure time. At this time, the obtained dynamic range may reach 16 bits to 20 bits or more. The WDR signal is output after being range compressed from 8 bits to 12 bits in accordance with the display capability of the display or printer.

  In such a WDR signal, noise in a region where a short exposure image is used often becomes a problem. Even if it is a short-exposure image, the signal-to-noise ratio is high and the noise is not very noticeable if it is a properly exposed region, but the region that appears almost black in the short-exposure image is used for amplification. Will amplify a signal that is not very different from the noise level, and the noise will be very noticeable. In particular, the object boundary where the brightness changes rapidly becomes a transition region where the short exposure image use region and the long exposure image use region are switched, and the two images are mixed and used. In a region where two images are mixed and used, pixel values of a short exposure image having a low S / N ratio are used, so noise is mixed in the composite image.

  As a technique for reducing noise in the short-exposure image use region of the WDR signal, there are techniques as disclosed in the following Patent Documents 1 to 4.

  The technique described in Patent Document 1 is a technique for performing composition after clipping a pixel value equal to or less than a threshold value of a short exposure image. This masks a dark part where the noise is conspicuous, so that the noise can be greatly suppressed.

  The technique described in Patent Document 2 is a technique for increasing the intensity of noise reduction processing for a short-exposure image as the ratio of the short-exposure image used for the composite image is high or the luminance value of the composite image is large. As a result, the noise of the short-exposure image can be reduced.

  The technique described in Patent Document 3 uses a short exposure image for synthesis and a long exposure image after creating a short exposure image for synthesis with reduced noise by photographing and synthesizing a plurality of short exposure images. This is a technique for generating a composite image.

  The technique described in Patent Document 4 captures several images with different exposure times in advance, analyzes the noise amount of each image, grasps the noise situation, and the noise amount of the composite image is a desired noise level. Exposure settings are made so that the amount is within the range.

JP 2009-152669 A JP 2000-69355 A JP 2012-239077 A JP 2010-200197 A

  However, when the technique described in Patent Document 1 is used, the texture information included in the pixel level equal to or lower than the threshold value is completely erased, so that the texture is weakened in the boundary region between the short exposure image and the long exposure image. Phenomenon that looks unnatural. In addition, since the clip level is a direct current component, if the pixel value of the long exposure image in the boundary area is near 0, a luminance difference is generated so that the periphery of the short exposure image use area appears slightly bright. Is also a problem. When the exposure ratio is as large as several tens to several tens of times, the image quality problem pointed out may be reduced, but an exposure ratio that is too large will cause the short exposure image use area and the long exposure image use area to change smoothly. In order to reduce the smoothness of the power region, this technique cannot be applied to all scenes.

  Further, in the technique described in Patent Document 2, the object of noise reduction processing is the entire short exposure image, and not only the noise reduction effect but also the adverse effect of resolution reduction spreads to the entire short exposure image use region. As a result, the problem is that the noise reduction effect cannot be strengthened. In addition, even in a short exposure image, if the signal level is high, the noise is not so conspicuous, and the noise is conspicuous in the boundary area between the short exposure image use area and the long exposure image use area where the low signal level is amplified and used. Tend. In the boundary area, the use ratio of short-exposure images is low, and the brightness value of the composite image is also small. Therefore, it is estimated that even if this technology is used, there is almost no effect in reducing noise in the boundary area. Is done.

  Further, the technique described in Patent Document 3 has a problem in that, when an object is moving, a synthesis artifact is generated at the time of generating a short exposure image for synthesis. In addition, if the noise reduction effect is to be increased, it is necessary to shoot several images with only a short exposure image. Therefore, if this technology is applied to a moving image, the frame rate is greatly reduced. It is.

  Further, the technique described in Patent Document 4 has a problem in that the dynamic range is lowered instead of reducing the noise.

  Therefore, the present invention provides a technique for more effectively performing noise reduction processing in a technique for synthesizing a short exposure image and a long exposure image.

  According to an embodiment of the present invention, in an image processing apparatus that synthesizes a short exposure image and a long exposure image, noise reduction processing for the short exposure image is selected for each pixel based on a pixel value of the short exposure image. An image processing apparatus is provided, comprising: a selection unit; a processing unit that performs the noise reduction processing for each pixel; and a synthesis unit that combines the short-exposure image and the long-exposure image subjected to the noise reduction processing. The

  According to such a configuration, noise in the boundary area between the short-exposure image use area and the long-exposure image use area, which is a conspicuous area in the composite image, is effectively reduced by reducing dark noise in the short-exposure image. Can be done. Further, according to such a configuration, a natural composite image can be obtained with little reduction in resolution, which is an adverse effect of noise reduction processing.

  More specifically, the selection unit selects a first noise reduction process when the pixel value falls below a threshold value, and selects the first noise reduction process when the pixel value exceeds the threshold value. A second noise reduction process different from that may be selected.

  The second noise reduction process may be a weaker noise reduction process or no noise reduction process than the first noise reduction process.

  The image processing apparatus includes a motion detection unit that detects a motion region and a non-motion region, and the selection unit, when the pixel belongs to the non-motion region, when the pixel value falls below a threshold value, One noise reduction process may be selected, and the second noise reduction process may be selected when the pixel value exceeds the threshold value.

  According to such a configuration, it is possible to suppress a decrease in resolution of the motion region by performing weak noise reduction processing on the motion region. In particular, when a short-exposure image is used in a motion region in order to suppress artifacts and blur that occur when a moving object is synthesized, it is possible to suppress a reduction in resolution of the short-exposure image. Further, in the non-motion area, it is possible to effectively reduce noise in the boundary area between the short exposure image use area and the long exposure image use area, and it is possible to suppress a decrease in resolution of the non-motion area.

  More specifically, the selection unit may select a third noise reduction process when the pixel belongs to the motion region.

  According to another embodiment of the present invention, in an image processing method in an image processing apparatus that combines a short exposure image and a long exposure image, noise reduction processing for the short exposure image is performed for each pixel based on a pixel value. And a step of performing the noise reduction process for each pixel, and a step of synthesizing the short-exposure image and the long-exposure image on which the noise reduction process has been performed. The

  According to this method, noise in the boundary area between the short-exposure image use area and the long-exposure image use area, which is a conspicuous area in the composite image, is effectively reduced by reducing dark noise in the short-exposure image. Can be done. Further, according to such a configuration, a natural composite image can be obtained with little reduction in resolution, which is an adverse effect of noise reduction processing.

  As described above, according to the present invention, it is possible to perform noise reduction processing more effectively in the technique of combining a short exposure image and a long exposure image.

It is a figure which shows the example of each image which concerns on a general WDR technique. It is a figure which shows the function structure of the image processing apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the detailed structural example of a noise reduction part. It is a figure which shows the structural example of each of a weak filter and a strong filter. 6 is a flowchart illustrating an operation example of a selection unit of the image processing apparatus according to the first embodiment of the present invention. It is a figure for demonstrating the example of the effect which the 1st Embodiment of this invention show | plays. It is a figure which shows the function structure of the image processing apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows the detailed structural example of a noise reduction part. It is a flowchart which shows the operation example of the selection part of the image processing apparatus which concerns on the 2nd Embodiment of this invention.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

  In the present specification and drawings, a plurality of constituent elements having substantially the same functional configuration may be distinguished by attaching different alphabets after the same reference numeral. However, when it is not necessary to particularly distinguish each of a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are given.

(Outline of the embodiment)
First, an outline of an embodiment of the present invention will be described. FIG. 1 is a diagram illustrating an example of each image according to a general WDR technique. Each image in FIG. 1 is a scene taken outside from the room on a sunny day. In the short-exposure image Img-a, a cloud outside the window is visible, but the room appears very dark. In the long exposure image Img-b, the room and the person are properly exposed, but the window is saturated. When these two images are combined according to the use image selection information Img-c, a combined image Img-d is obtained. In the use image selection information Img-c, the white area indicates an area that uses a short exposure image, and the black area indicates an area that uses a long exposure image.

  If the composition is performed in this way, the visibility increases both outdoors and indoors, but the window frame portion R1 where the short exposure image use area and the long exposure image use area are switched becomes noisy. In the short exposure image Img-a, the signal level of the portion other than the window is very small, and the noisy signal obtained by amplifying the pixel value of the short exposure image Img-a and the pixel value of the long exposure image are obtained. If they are mixed and used in the composite image Img-d, noise will be conspicuous only around the window frame portion R1 as in the composite image Img-d.

  In the technique according to the first embodiment of the present invention, synthesis is performed after applying a strong noise reduction process only to the dark part of the short exposure image Img-a. Thereby, noise in the boundary region between the dark part and the bright part, which has been conspicuous conventionally, can be suppressed. Further, in the technology according to the second embodiment of the present invention, when combining the technology according to the first embodiment of the present invention and the moving object synthesis artifact reduction processing, strong noise reduction processing is performed in a short non-motion region. It is applied to the dark part of the exposure image, and weak noise reduction processing is applied to the motion area. Thereby, it is possible to obtain the effect of reducing the boundary noise while preventing the resolution of the moving object from being lowered.

(First embodiment)
First, a first embodiment of the present invention will be described. First, the functional configuration of the image processing apparatus 1A according to the first embodiment of the present invention will be described. FIG. 2 is a diagram showing a functional configuration of the image processing apparatus 1A according to the first embodiment of the present invention. As illustrated in FIG. 2, the image processing apparatus 1A includes a sensor 10, a frame memory 20, a use image selection unit 30, a noise reduction unit 50A, a WDR synthesis unit 70, and a range compression unit 80. Hereinafter, the functions of the respective functional blocks provided in the image processing apparatus 1A will be sequentially described in detail.

  The image processing apparatus 1A changes the exposure setting of the sensor 10 and continuously shoots two images. Here, the short exposure shooting is performed first, and then the long exposure shooting is performed. However, long exposure photography may be performed first and then short exposure photography. The short-exposure image and the long-exposure image photographed in this way are written in the frame memory 20 as a pair. Shooting of the long exposure image and the short exposure image and writing of the captured long exposure image and the short exposure image to the frame memory 20 are continuously performed.

  In the example shown in FIG. 2, the image processing apparatus 1A has one common system for outputting the long exposure image and the short exposure image, and the sensor 10 sometimes outputs the long exposure image and the short exposure image. Although the output is divided, the long exposure image and the short exposure image may be output simultaneously. In such a case, the image processing apparatus 1A only needs to have two systems, that is, a system for outputting a long exposure image from the sensor 10 and a system for outputting a short exposure image. Each shutter time is determined by, for example, a dynamic range of a subject to be imaged, a sensor specification, and the like.

  In the embodiments of the present invention, the terms short exposure image and long exposure image are used, but these terms do not limit the absolute exposure time of each of the two captured images. Therefore, when two images with different exposure times are taken, an image with a relatively short exposure time corresponds to a short exposure image and an image with a relatively long exposure time is a long exposure. Corresponds to an image.

  The sensor 10 is configured by an image sensor that forms an image of light from the outside on the light receiving plane of the image sensor, photoelectrically converts the imaged light into a charge amount, and converts the charge amount into an electric signal. The type of the image sensor is not particularly limited, and may be, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

  The use image selection unit 30 refers to the short exposure image and the long exposure image read from the frame memory 20, detects the saturation state and movement of each of the long exposure image and the short exposure image, and detects the short exposure image and the long exposure image. Use image selection information for selecting one of the images as the use image is generated. As an algorithm for selecting either the short exposure image or the long exposure image, various algorithms are assumed.

  For example, since a region that has been saturated in the long exposure image is highly likely not to be saturated in the short exposure image, the short exposure image may be selected as the use image of the region. However, with this process alone, artifacts such as a double outline may occur in an area where there is a large movement. For this reason, a process of detecting a motion and reducing a phenomenon that a contour becomes double based on the motion may be performed. An algorithm for selecting either a short exposure image or a long exposure image including such processing is not particularly limited.

  As described above, the use image selection information may be a set of binary data indicating which of the short exposure image and the long exposure image is to be selected. It may be a set of mixing ratios indicating whether to mix at a ratio. For example, the use image selection unit 30 may increase the mixing ratio of the short-exposure images as the degree of saturation of the long-exposure images increases. In addition, the use image selection unit 30 may increase the mixing ratio of the short exposure image as the movement of the short exposure image or the long exposure image increases. The algorithm for calculating the mixing ratio of the short exposure image and the long exposure image is not particularly limited.

  The noise reduction unit 50A reduces noise from the short exposure image. FIG. 3 is a diagram illustrating a detailed configuration example of the noise reduction unit 50A. As illustrated in FIG. 3, the noise reduction unit 50A includes a processing unit 51 and a selection unit 52A. The processing unit 51 performs noise reduction processing for each pixel. In the example illustrated in FIG. 3, the weak filter 51 a and the strong filter 51 b are illustrated as subjects that perform noise reduction processing for each pixel. However, the weak filter 51a and the strong filter 51b do not limit absolute strength. Therefore, of the two filters having different strengths, the filter having a relatively low strength corresponds to the weak filter 51a, and the filter having a relatively high strength corresponds to the strong filter 51b. In addition, the weak filter 51a does not need to perform a noise reduction process.

  FIG. 4 is a diagram illustrating a configuration example of each of the weak filter 51a and the strong filter 51b. FIG. 4 shows an example in which the weak filter 51a is a 5-tap filter and the strong filter 51b is a 13-tap filter. The weak filter 51a and the strong filter 51b may calculate the pixel value of the center tap by calculating an average of the pixel value of the center tap and the pixel value of the peripheral tap. As in this example, the number of peripheral taps may be increased as the noise reduction intensity is increased.

  However, the strengths of the weak filter 51a and the strong filter 51b may be varied in any way. For example, the tap size is fixed, and the weak filter 51a and the strong filter 51b are calculated by calculating an average of pixel values of one or more taps whose difference from the pixel value of the center tap is smaller than a predetermined value. The pixel value of the tap may be calculated. In such a case, the magnitude of the predetermined value corresponds to the strength of noise reduction processing.

  Alternatively, the tap size may be fixed, a weight coefficient corresponding to the distance from the center tap may be defined for each tap, and noise reduction may be performed by a convolution operation between the pixel value of each tap and the weight coefficient. In this case, the difference in weight coefficient distribution is the strength of noise reduction processing. In this way, there are various methods for noise reduction processing. However, if two types of noise reduction processing can be prepared, there are strengths of noise reduction processing, and there are also adverse effects such as deterioration of texture and sharpness. These two types of noise reduction processing can be applied to the embodiment of the present invention.

  The selection unit 52A selects noise reduction processing for the short exposure image for each pixel based on the pixel value of the short exposure image. For example, the selection unit 52A selects the strong filter 51b when the pixel value falls below the threshold value, and selects the weak filter 51a when the pixel value exceeds the threshold value. When the pixel value is equal to the threshold value, either the weak filter 51a or the strong filter 51b may be selected. The short-exposure image that has been subjected to noise reduction processing by the selected filter is output to the WDR synthesis unit 70.

  With such a configuration, noise included in a dark portion of a short-exposure image is strongly reduced, while weak noise reduction is applied to an area where noise is not so conspicuous from an intermediate area to a high luminance area. Securing resolution is prioritized over noise reduction processing. Note that the optimum threshold value may be determined depending on the set exposure ratio, the degree of noise, and the like. In the noise reduction unit 50A, a short exposure image in which dark part noise is strongly reduced is used for WDR synthesis.

  The WDR synthesis unit 70 synthesizes the short-exposure image and the long-exposure image that have been subjected to the noise reduction processing by the noise reduction unit 50A based on the use image selection information generated by the use image selection unit 30, thereby performing WDR synthesis. Generate an image. Specifically, the WDR composition unit 70 refers to the use image selection information, uses the short exposure image for the short exposure image use region, and uses the long exposure image for the long exposure image use region. Is generated. At the time of composition, it is preferable to normalize one of the images by multiplying by a gain corresponding to the exposure ratio.

  The combining method by the WDR combining unit 70 is not particularly limited. For example, it is assumed that a value indicating that a long exposure image is selected is “0” and a value indicating that a short exposure image is selected is “1”. In such a case, the WDR synthesizing unit 70 sets α as the mixing ratio constituting the used image selection information, and α × (pixel value of the short exposure image) + (pixel value) corresponding to the long exposure image and the short exposure image. 1−α) × (pixel value of the long exposure image) can be calculated, and the calculation result can be used as a combined image (WDR image).

  The range compression unit 80 performs compression processing on the WDR image generated by the WDR synthesis unit 70 so that the bit range of the image signal having a wide dynamic range falls within a predetermined bit range. If the resolution of the sensor 10 is 12 bits, the WDR signal is expanded to about 16 bits. However, the range compression unit 80 adjusts this signal according to the input specifications for general-purpose image signal processing at the subsequent stage, for example, Compress again to 12 bits. As such compression processing, tone mapping according to a look-up table (LUT) may be used, but any method may be used.

  The subsequent stage of the range compression unit 80 is connected to, for example, an image processing engine including a demosaic unit that generates an RGB plane from Bayer data, a contour enhancement unit, and color management. Therefore, it is preferable that the data amount of the output signal from the range compression unit 80 is adjusted so as to match the size of the input data to the image processing engine (for example, about 12 bits). If the data size is simply reduced, the image is converted into a dark image. Therefore, it is preferable that the high luminance side is strongly compressed so as to approximate human visual characteristics.

  In the above, an example in which one of the two types of noise reduction processing is selected depending on whether the threshold value is higher or lower than one threshold value has been described, but two or more threshold values may be provided. For example, when two threshold values are provided, one of the three types of noise reduction processing may be selected depending on which of the three regions divided by these threshold values corresponds.

  In the above description, an example in which two types of images (short exposure image and long exposure image) having different exposure amounts are taken and combined has been described. However, the type of image used for composition is not particularly limited. For example, the present embodiment can be applied to an example in which three or more types of images having different exposure amounts are captured and combined.

  Here, an operation example of the selection unit 52A of the image processing apparatus 1A according to the first embodiment of the present invention will be further described. FIG. 5 is a flowchart showing an operation example of the selection unit 52A of the image processing apparatus 1A according to the first embodiment of the present invention. Note that the operation example of the selection unit 52A illustrated in FIG. 5 is merely an example, and the operation of the selection unit 52A is not limited to the example illustrated in FIG.

  As illustrated in FIG. 5, the selection unit 52A performs filter selection as described below for each pixel constituting the short exposure image. That is, when the pixel value of the short exposure image exceeds the threshold value (“No” in Step S11), the selection unit 52A selects the weak filter 51a (Step S12). On the other hand, when the pixel value of the short exposure image is below the threshold value (“Yes” in step S11), the selection unit 52A selects the strong filter 51b (step S13). The WDR synthesis unit 70 outputs a short exposure image with noise reduced by the selected filter. Thereafter, the long-exposure image and the short-exposure image whose noise is reduced by the selected filter are combined by the WDR combining unit 70.

  Then, the example of the effect which the 1st Embodiment of this invention has is demonstrated. FIG. 6 is a diagram for explaining an example of the effect achieved by the first embodiment of the present invention. Referring to FIG. 6, a composite image Img-e generated by the image processing apparatus 1A according to the first embodiment of the present invention is shown. Referring to the composite image Img-e, the noise in the boundary region between the short exposure image use region and the long exposure image use region, which is a region where noise is conspicuous in the composite image, is effectively reduced by reducing the dark noise of the short exposure image. Has been reduced.

  In addition, a natural composite image is obtained with little reduction in resolution, which is an adverse effect of noise reduction processing. In addition, compared with the prior art, the image processing apparatus 1A according to the first embodiment of the present invention can be applied to moving images, the dynamic range does not decrease, and the motion region synthesis artifact reduction processing It is also characterized by high affinity.

(Second Embodiment)
Subsequently, a second embodiment of the present invention will be described. In the first embodiment of the present invention, the example in which the noise reduction process is selected without referring to the motion detection information has been described. However, for example, if a strong noise reduction process is performed on the motion region, the resolution of the motion region may be greatly reduced (the texture of the moving object may be greatly attenuated).

  For example, when a short exposure image is used in the motion region in the WDR synthesis unit 70 in order to suppress artifacts and blur that occur when a moving object is synthesized, the resolution of the short exposure image is greatly reduced. There is a possibility. Thus, in the second embodiment of the present invention, an example in which noise reduction processing is selected using motion detection information will be described.

  FIG. 7 is a diagram showing a functional configuration of an image processing apparatus 1B according to the second embodiment of the present invention. As shown in FIG. 7, the image processing apparatus 1B according to the second embodiment of the present invention detects motion in the subsequent stage of the frame memory 20 compared to the image processing apparatus 1A according to the first embodiment of the present invention. A unit 40 is further provided. Moreover, the function of the noise reduction part 50 differs compared with 1 A of image processing apparatuses which concern on the 1st Embodiment of this invention. Hereinafter, the motion detection unit 40 and the noise reduction unit 50B will be mainly described.

  The motion detection unit 40 detects motion and generates motion detection information. Although the motion detection method is not limited, when motion is detected from a short exposure image and a long exposure image, the difference is calculated after normalizing by multiplying one of the images by a gain according to the exposure ratio. It is good. Further, the method for generating motion detection information is not particularly limited. For example, the motion detection unit 40 may generate the motion detection information based on the relationship between the motion detected based on the short exposure image and the long exposure image and the threshold value.

  Specifically, the motion detection unit 40 may detect a difference between pixel values or gradients of corresponding regions in the short-exposure image and the long-exposure image, and detect a region where the difference is larger than a threshold as a motion region. On the other hand, the motion detection unit 40 may detect a region where the difference is smaller than the threshold as a non-motion region. An area where the difference is the same as the threshold value may be detected as any area. The motion detection unit 40 may generate such a detection result as motion detection information.

  The noise reduction unit 50B reduces noise from the short exposure image. FIG. 8 is a diagram illustrating a detailed configuration example of the noise reduction unit 50B. As illustrated in FIG. 8, the noise reduction unit 50B includes a processing unit 51 and a selection unit 52B. When the pixel belongs to the non-motion region, the selection unit 52B selects the strong filter 51b when the pixel value falls below the threshold value. As a result, as in the first embodiment of the present invention, it is possible to effectively reduce the noise in the boundary area between the short exposure image use area and the long exposure image use area, which is an area where noise is conspicuous in the composite image. Is possible.

  In addition, when the pixel belongs to the non-motion region, the selection unit 52B selects the weak filter 51a when the pixel value exceeds the threshold value. As a result, as in the first embodiment of the present invention, a weak noise reduction is applied to a region where noise is not so conspicuous, whereby priority is given to ensuring the resolution over the noise reduction processing. Note that, similarly to the first embodiment of the present invention, the weak filter 51a may not perform noise reduction processing.

  On the other hand, when the pixel belongs to the motion region, the selection unit 52B selects the weak filter 51a. As a result, the possibility that the resolution of the moving area is greatly reduced is reduced (the possibility that the texture of the moving object is greatly attenuated) is reduced. Here, an example in which the weak filter 51a is selected when the pixel belongs to the motion region will be described. However, if the filter is weaker than the strong filter 51b, a filter different from the weak filter 51a may be selected. .

  Here, an operation example of the selection unit 52B of the image processing apparatus 1B according to the second embodiment of the present invention will be further described. FIG. 9 is a flowchart showing an operation example of the selection unit 52B of the image processing apparatus 1B according to the second embodiment of the present invention. Note that the operation example of the selection unit 52B illustrated in FIG. 9 is merely an example, and the operation of the selection unit 52B is not limited to the example illustrated in FIG.

  As illustrated in FIG. 9, the selection unit 52B performs filter selection as described below for each pixel constituting the short exposure image. That is, when the pixel of the short exposure image belongs to the motion region (“Yes” in step S10), the selection unit 52B selects the weak filter 51a (step S14). On the other hand, when the pixel of the short exposure image belongs to the non-motion region (“No” in step S10), the selection unit 52B proceeds to step S11.

  When the pixel value of the short-exposure image exceeds the threshold value (“No” in step S11), the selection unit 52B selects the weak filter 51a (step S12). On the other hand, when the pixel value of the short exposure image is below the threshold value (“Yes” in step S11), the selection unit 52B selects the strong filter 51b (step S13). The WDR synthesis unit 70 outputs a short exposure image with noise reduced by the selected filter. Thereafter, the long-exposure image and the short-exposure image whose noise is reduced by the selected filter are combined by the WDR combining unit 70.

  FIG. 9 shows an example in which the weak filter 51a is selected when the pixel belongs to the motion region. However, if the filter is weaker than the strong filter 51b, a filter different from the weak filter 51a is selected. It may be done as already described.

  Then, the example of the effect which the 2nd Embodiment of this invention has is demonstrated. According to the second embodiment of the present invention, by performing a weak noise reduction process on the motion region, it is possible to suppress a decrease in resolution of the motion region. In particular, when a short-exposure image is used in a motion region in order to suppress artifacts and blur that occur when a moving object is synthesized, it is possible to suppress a reduction in resolution of the short-exposure image. Further, in the non-motion area, it is possible to effectively reduce noise in the boundary area between the short exposure image use area and the long exposure image use area, and it is possible to suppress a decrease in resolution of the non-motion area.

  Heretofore, the first embodiment and the second embodiment of the present invention have been described. According to the first embodiment of the present invention, in the image processing apparatus 1A that combines a short-exposure image and a long-exposure image, noise reduction processing for the short-exposure image is selected for each pixel based on the pixel value of the short-exposure image. The image processing apparatus 1A includes a selection unit 52A that performs processing, a processing unit 51 that performs noise reduction processing for each pixel, and a WDR synthesis unit 70 that combines the short-exposure image and the long-exposure image that have undergone noise reduction processing. Is provided.

  According to such a configuration, noise in the boundary area between the short-exposure image use area and the long-exposure image use area, which is a conspicuous area in the composite image, is effectively reduced by reducing dark noise in the short-exposure image. Has been. In addition, a natural composite image is obtained with little reduction in resolution, which is an adverse effect of noise reduction processing. In addition, compared with the prior art, the image processing apparatus 1A according to the first embodiment of the present invention can be applied to moving images, the dynamic range does not decrease, and the motion region synthesis artifact reduction processing It is also characterized by high affinity.

  In addition, according to the second embodiment of the present invention, the motion detection unit 40 that detects the motion region and the non-motion region is provided, and the selection unit 52B has a pixel value that is a threshold value when the pixel belongs to the non-motion region. The image processing device 1B is provided that selects the first noise reduction process when the pixel value is lower than the threshold value and selects the second noise reduction process when the pixel value exceeds the threshold value.

  According to such a configuration, it is possible to suppress a decrease in resolution of the motion region by performing weak noise reduction processing on the motion region. In particular, when a short-exposure image is used in a motion region in order to suppress artifacts and blur that occur when a moving object is synthesized, it is possible to suppress a reduction in resolution of the short-exposure image. Further, in the non-motion area, it is possible to effectively reduce noise in the boundary area between the short exposure image use area and the long exposure image use area, and it is possible to suppress a decrease in resolution of the non-motion area.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  Wide dynamic range technology is particularly important as a network camera differentiation technology. The embodiment of the present invention can effectively reduce the noise in the boundary area between the short exposure image use area and the long exposure image use area, which has been a big problem in the prior art. In addition, the embodiment of the present invention can suppress the adverse effects such as a reduction in resolution of moving objects and still areas, can be realized with a simple configuration and a small circuit scale, and does not affect the frame rate, and has a moving image performance. This technology has many excellent points, such as the ability to reduce the dynamic range, the dynamic range does not decrease, and the motion region synthesis artifact reduction processing.

1 (1A, 1B) Image processing device 10 Sensor 20 Frame memory 30 Used image selection unit 40 Motion detection unit 50 (50A, 50B) Noise reduction unit 51 Processing unit 51a Weak filter 51b Strong filter 52 (52A, 52B) Selection unit 70 WDR synthesis unit 80 Range compression unit

Claims (2)

In an image processing apparatus that combines a short exposure image and a long exposure image,
A motion detector for detecting a motion region and a non-motion region;
A selection unit that selects a noise reduction process for the short exposure image for each pixel based on a pixel value of the short exposure image;
A processing unit that performs the noise reduction processing for each pixel;
A combining unit that combines the short-exposure image and the long-exposure image that have undergone the noise reduction processing;
With
When the pixel belongs to the non-motion region, the selection unit selects a first noise reduction process when the pixel value is lower than a threshold value, and the first noise when the pixel value is higher than the threshold value. A second noise reduction process that is different from the noise reduction process is selected, and if the pixel belongs to the motion region, a third noise reduction process is selected,
The second noise reduction process is a noise reduction process weaker than the first noise reduction process, and is executed by the same filter as the filter on which the third noise reduction process is executed.
Image processing device. In an image processing method in an image processing apparatus that combines a short exposure image and a long exposure image,
Detecting a motion region and a non-motion region;
Selecting noise reduction processing for the short exposure image for each pixel based on a pixel value;
Performing the noise reduction processing for each pixel;
Combining the short-exposure image and the long-exposure image that have undergone the noise reduction processing;
With
When the pixel belongs to the non-motion region, the first noise reduction process is selected when the pixel value is below a threshold value, and the first noise reduction process is selected when the pixel value is above the threshold value. Select a different second noise reduction process, and if the pixel belongs to the motion region, select a third noise reduction process,
The second noise reduction process is a noise reduction process weaker than the first noise reduction process, and is executed by the same filter as the filter on which the third noise reduction process is executed.
Image processing method.