JP6121234B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method.

  In recent years, a short-exposure image (hereinafter also simply referred to as “short-exposure image”) and a long-exposure image (hereinafter also simply referred to as “long-exposure image”) are continuously photographed and combined. An imaging function called WDR (wide dynamic range) or HDR (high dynamic range) that obtains an image that captures a dynamic range that exceeds the dynamic range that the sensor can shoot is increasing. Such a photographing function is particularly effective in a scene with a very large contrast ratio such as a composition of backlight.

  Furthermore, in recent years, for the purpose of extending the dynamic range that can be photographed, a technique for photographing and combining three or more images with different exposures has appeared. For example, Patent Document 1 discloses a method of smoothly synthesizing three images shot with different exposures.

  In such a method, the reference exposure image is converted into a grayscale image, and the grayscale image is divided into three regions, a light region, a dark region, and an intermediate region, by gradation value threshold processing, and each of the three regions is divided. Colors are added and averaged filter processing is performed, and then three images are synthesized according to the color component of each pixel. The threshold value of the gray value is determined by score calculation for all the pixels of the reference exposure image using the edge detection result and the boundary of the region divided according to the gray value.

JP 2009-10566 A JP 2005-72965 A

  Here, in the case of using the method of dividing the area using the reference exposure image, it is possible to cope with the case where EV (Exposure Value) is shaken by about ± 1 and the image is taken. However, when such a technique is applied to a shooting function called WDR in which the exposure amount is changed by an order of several times, the high-brightness whiteout and the low-brightness blackout are severe, and the standard exposure image It becomes difficult to appropriately determine an image to be used as a composition target on the basis of one sheet.

  In addition, if the number of images used for composition is increased in order to further expand the dynamic range that can be shot, it is more difficult to determine the image to be composed, and an appropriate composition result can be obtained in a high-luminance or low-luminance region. Not likely to be.

  In addition, since it is necessary to prepare a grayscale image and an RGB plane for weighting, a point to use edge detection, and a calculation for all pixels, it is necessary to finish processing for all pixels. There is a point that it is considered that a very large circuit scale is required if it is executed by hardware, such as a point that a synthesized image cannot be output. Even if the same function is realized by software, it is considered that a large amount of calculation is required and a large processing delay occurs.

  Therefore, the present invention provides a technique capable of obtaining a natural synthesized image by smoothly connecting image boundaries when synthesizing using three or more images. Furthermore, the present invention provides a technique capable of reducing the circuit scale and processing delay further when combining three or more images.

  According to an embodiment of the present invention, an identification value for identifying each of N (N is a natural number of 3 or more) images photographed at different exposure amounts is set to a predetermined numerical value with zero being a lower limit value. With respect to selection information that is defined in a range having a numerical value multiplied by 1 as an upper limit, and a selection value that indicates one of the identification values for identifying each of the N images is defined corresponding to each pixel A processing unit for performing a predetermined process, a selection process for selecting two images from the N images based on a selection value defined in the selection information subjected to the predetermined process, An image processing apparatus is provided, comprising: a combining unit configured to combine each pixel with a combining process for combining corresponding pixels of each of the images.

  With this configuration, in the vicinity of a boundary where a plurality of images photographed with different exposure amounts are adjacent, an intermediate value between the selection values of the plurality of adjacent images is generated as a selection value. Therefore, it is possible to obtain a composite image in which the mixture ratio is smoothly changed and the vicinity of the boundary is smoothly connected.

  Further, by referring to selection information as one parameter, it is possible to select two images to be combined for each pixel from three or more images having different exposure amounts. Therefore, when configured with hardware, it is possible to implement with a smaller circuit scale, and when implemented with software, it is possible to reduce the computation load.

  For example, the identification value for identifying each of the N images is a value obtained by multiplying the predetermined numerical value by each integer from 0 to N-1 and assigned in ascending or descending order of exposure amount. It may be. According to such a configuration, it is possible to assign the identification values in the order of the exposure amount by setting the intervals between the adjacent identification values at equal intervals, and therefore the selection value defined in the selection information subjected to the predetermined processing. It is possible to easily calculate the mixing ratio using.

  The predetermined process is preferably a process for greatly blurring selection information. If it does so, it will become possible to connect the boundary vicinity more smoothly. For example, the predetermined process may be a process for applying a low-pass filter.

  A method for selecting the two images for each pixel is not particularly limited. For example, the synthesis unit uses two identification values having a smaller difference from the selection value defined in the selection information subjected to the predetermined process as identification values for identifying each of the two images. You may specify. In this way, the two images may be selected for each pixel.

  Further, for example, the combining unit determines a mixing ratio according to a difference value between a selection value defined in the selection information subjected to the predetermined process and an identification value of each of the two images, The corresponding pixels of each of the two images may be synthesized based on the mixing ratio. In this way, the mixing ratio can be calculated for each pixel.

  According to another embodiment of the present invention, an identification value for identifying each of N (N is a natural number of 3 or more) images photographed with different exposure amounts has a predetermined numerical value with zero as a lower limit value. A selection value that is defined in a range in which a numerical value obtained by multiplying N-1 by N-1 is set as an upper limit value, and a selection value indicating one of identification values for identifying each of the N images is defined corresponding to each pixel. A step of applying a predetermined process to the information; a selection process of selecting two images from the N images based on a selection value defined in the selection information subjected to the predetermined process; An image processing method is provided, including a step of combining each pixel with a combining process of combining corresponding pixels of each of the two images.

  According to this method, in the vicinity of a boundary where a plurality of images photographed with different exposure amounts are adjacent, an intermediate value between the selection values of the plurality of adjacent images is generated as a selection value. Therefore, it is possible to obtain a composite image in which the mixture ratio is smoothly changed and the vicinity of the boundary is smoothly connected.

  Further, by referring to selection information as one parameter, it is possible to select two images to be combined for each pixel from three or more images having different exposure amounts. Therefore, when configured with hardware, it is possible to implement with a smaller circuit scale, and when implemented with software, it is possible to reduce the computation load.

  As described above, according to the present invention, it is possible to provide a technique capable of obtaining a natural synthesized image by smoothly connecting image boundaries when synthesizing using three or more images. Is possible. Furthermore, according to the present invention, when combining using three or more images, it is possible to reduce the circuit scale and the processing delay.

It is a figure which shows the function structural example of the image processing apparatus which concerns on embodiment of this invention. It is a figure which shows the example of the various information in the WDR synthetic | combination technique which concerns on embodiment of this invention. It is a figure which shows the example of the flow of operation | movement of the image processing apparatus which concerns on embodiment of this invention.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

  In the present specification and drawings, a plurality of constituent elements having substantially the same functional configuration may be distinguished by attaching different alphabets after the same reference numeral. However, when it is not necessary to particularly distinguish each of a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are given.

  In the present specification, as an example of N (N is a natural number of 3 or more) images photographed with different exposure amounts, three images of a short exposure image, a middle exposure image, and a long exposure image are arranged in order of increasing exposure amount. An example of taking an image will be mainly described. However, as will be described later, the image processing apparatus 1 according to the embodiment of the present invention can also be applied when four or more images are captured.

  First, a functional configuration of the image processing apparatus 1 according to the embodiment of the present invention will be described. FIG. 3 is a diagram showing a functional configuration of the image processing apparatus 1 according to the embodiment of the present invention. As shown in FIG. 3, the image processing apparatus 1 includes a sensor 10, a short exposure image memory 21, a medium exposure image memory 22, a long exposure image memory 23, a use image selection unit 30, an LPF (Low Pass Filter) 40, and WDR composition. Unit 50 and gradation compression unit 60. Hereinafter, the function of each functional block included in the image processing apparatus 1 will be described in detail sequentially.

  The image processing apparatus 1 changes the exposure setting of the sensor 10 and continuously shoots three images. Here, the short exposure shooting is performed first, followed by the medium exposure shooting, and then the long exposure shooting. Assumed to be performed. The short exposure image taken by the short exposure is written in the short exposure image memory 21. When the short exposure shooting is completed, the image processing apparatus 1 changes the exposure setting and performs medium exposure shooting. The middle exposure image taken by the middle exposure is written in the middle exposure image memory 22. The image processing apparatus 1 changes the exposure setting when the medium exposure shooting is completed, and performs the long exposure shooting. The long exposure image taken by the long exposure is written in the long exposure image memory 23.

  In the example illustrated in FIG. 1, the image processing apparatus 1 has one common system for outputting a long exposure image, a medium exposure image, and a short exposure image, and the sensor 10 includes a long exposure image and a medium exposure image. Although the short exposure image is output in a time division manner, the long exposure image, the medium exposure image, and the short exposure image may be taken simultaneously. In such a case, if the image processing apparatus 1 has three systems, a system for outputting a long exposure image from the sensor 10, a system for outputting a medium exposure image, and a system for outputting a short exposure image. Good. Each shutter time is determined by, for example, a dynamic range of a subject to be imaged, a sensor specification, and the like.

  Note that the readout operation for each of the short-exposure image, the medium-exposure image, and the long-exposure image can vary depending on how the system is assembled in the case of hardware implementation. For example, each of the short-exposure image, the medium-exposure image, and the long-exposure image may be stored once in a memory such as a DRAM (Dynamic Random Access Memory) and then read again at the time of synthesis.

  Here, in the embodiments of the present invention, the terms short exposure image, medium exposure image, and long exposure image are used, but these terms limit the absolute exposure time of each of the three captured images. It is not a thing. Therefore, when three images with different exposure times are taken, the three images are named as a short exposure image, a medium exposure image, and a long exposure image in order of increasing exposure time.

  The “short exposure image”, “medium exposure image”, and “long exposure image” shown in FIG. 2 show examples of the short exposure image, the medium exposure image, and the long exposure image, respectively. Referring to the “short-exposure image”, the scenery outside the window with very high brightness is photographed with appropriate exposure. However, the other areas are underexposed, almost dark and have low visibility.

  Referring to the “medium exposure image”, the area on the shelf is photographed with appropriate exposure, and the presence of the mobile phone can be visually recognized. However, the outside of the window is overexposed and overexploded, and the bottom of the shelf is still underexposed and the visibility is low. Referring to the “long exposure image”, the area under the dark shelf is photographed with appropriate exposure, and the presence of the bag can be visually recognized. However, the outside of the window and on the shelf becomes overexposed and the visibility becomes low.

  Returning to FIG. 1, the description will be continued. The sensor 10 is configured by an image sensor that forms an image of light from the outside on the light receiving plane of the image sensor, photoelectrically converts the imaged light into a charge amount, and converts the charge amount into an electric signal. The type of the image sensor is not particularly limited, and may be, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

  The use image selection unit 30 refers to the short exposure image read from the short exposure image memory 21, the medium exposure image read from the medium exposure image memory 22, and the long exposure image read from the long exposure image memory 23. Saturation state and movement of each of the exposure image, the medium exposure image, and the long exposure image are detected, and selection information for selecting one of the short exposure image, the medium exposure image, and the long exposure image as the use image is generated. . As an algorithm for selecting one of the short exposure image, the medium exposure image, and the long exposure image, various algorithms are assumed.

  For example, since a region that has been saturated in the long exposure image is highly likely not to be saturated in the medium exposure image, the medium exposure image may be selected as the use image of the region. However, when the area is saturated even in the medium exposure image, the short exposure image may be selected as the use image of the area. However, with this process alone, artifacts such as a double outline may occur in an area where there is a large movement. Therefore, a process of detecting a motion and reducing a phenomenon that the contour becomes double may be performed. An algorithm for selecting any one of the short exposure image, the medium exposure image, and the long exposure image including such processing is not particularly limited.

  In this specification, the identification values D0 to D2 for identifying each of the long exposure image, the medium exposure image, and the short exposure image are “0” as a lower limit value and N−1 (this example) Then, since there are three images, the value is defined in a range having an upper limit value obtained by multiplying 3-1 = 2). In the selection information described above, a selection value indicating any one of the identification values D0 to D2 for identifying each of the long exposure image, the medium exposure image, and the short exposure image is defined corresponding to each pixel. The predetermined numerical value A is not particularly limited, but if it is a power of 2, as will be described later, division by the distance between identification values can be performed by a simple bit shift operation.

  The identification values D0 to D2 for identifying each of the long exposure image, the medium exposure image, and the short exposure image are obtained by multiplying a predetermined numerical value A by “0”, “1”, and “2”. The values may be assigned in ascending or descending order. In the example shown below, the predetermined numerical value A is “256”, and “0” as a result of multiplying “256” by “0” is assigned as the identification value D0 for identifying the long exposure image. The case will be described.

  Similarly, in the example shown below, “256” as a result of multiplying “256” by “1” is assigned as an identification value D1 for identifying the middle exposure image, and “256” is assigned to “256”. A case where “512” as a result of multiplying “2” is assigned as the identification value D2 for identifying the short-exposure image will be described.

  “Selection information” illustrated in FIG. 2 indicates an example of selection information generated by the use image selection unit 30. This “selection information” is selected as “long exposure image” as a use image of a region that is not saturated in “long exposure image”, and is saturated in “long exposure image”. Adopted the method of selecting “Medium exposure image” as the use image of the unsaturated area and selecting “Short exposure image” as the use image of the saturated area in the “Medium exposure image”. Generated when.

  In this “selection information”, a black area indicates that a “long exposure image” is selected, and each selection value of the black area is “0”. A gray area indicates that “medium exposure image” is selected, and each selection value of the gray area is “256”. Further, the white area indicates that the “short exposure image” is selected, and each selection value of the white area is “512”.

  Returning to FIG. 1, the description will be continued. The LPF 40 performs processing on the selection information defined as described above. In the following, the case where the predetermined process is a process for applying a low-pass filter will be described as an example, but the predetermined process is not particularly limited. The low pass filter process corresponds to a process of removing high frequency components from the selection information. With this process, the selection information is greatly blurred, and when the WDR composition unit 50 performs composition, it is possible to smoothly connect the boundaries of the short exposure image, the medium exposure image, and the long exposure image.

  The bit accuracy of the output signal from the LPF 40 should be sufficiently ensured so that the boundaries between the short exposure image, the medium exposure image, and the long exposure image are smoothly connected. Various methods are assumed for the method of blurring the selection information, and it is not particularly limited. However, it is desirable that the blurring method is determined in consideration of the blurring effect, the calculation amount, the circuit scale, and the like. “Selection information + LPF” shown in FIG. 2 shows an example of selection information in which the LPF 40 performs processing for applying a low-pass filter to “selection information”.

  Referring to “selection information + LPF” shown in FIG. 2, in the vicinity of the boundary between two regions having different selection values in “selection information”, an intermediate value between the two regions is output as a selection value. ing. On the other hand, in a region where a certain selection value is continuous in the “selection information”, the selection value in that region is maintained as it is.

  The WDR synthesis unit 50 selects two images from a short exposure image, a medium exposure image, and a long exposure image based on a selection value defined in the selection information subjected to the low-pass filter processing, and 2 A combining process for combining corresponding pixels of each of the images is performed for each pixel.

  For example, in the selection process, the WDR synthesis unit 50 selects two images from the short exposure image, the medium exposure image, and the long exposure image based on the selection value defined in the selection information subjected to the low-pass filter processing. It is recommended to select a pixel. More specifically, the WDR synthesizing unit 50 identifies two identification values having a smaller difference from the selection value defined in the selection information subjected to the low-pass filter process, for identifying each of the two images. What is necessary is just to specify as a value.

  Further, for example, in the synthesis process, the WDR synthesis unit 50 determines the mixing ratio according to the difference value between the selection value defined in the selection information subjected to the low-pass filter process and the identification value of each of the two selected images. And the corresponding pixels of the two images are synthesized based on the mixing ratio. For example, the distance from the selection value to the identification value for the distance between the identification values of each of the two selected images may be set as each mixing ratio.

  For example, for each pixel, the WDR synthesizing unit 50 sets the selection value as Sel, the pixel value of the short exposure image as Ps, the pixel value of the middle exposure image as Pm, and the pixel value of the long exposure image as Pl. Can be synthesized based on the following equation (1) or equation (2).

IF (256 <Sel ≦ 512)
Out = Ps × (512-Sel) / (512-256)
+ Pm × (Sel-256) / (512-256) (1)

IF (0 ≦ Sel ≦ 256)
Out = Pm × (Sel-256) / (256-0)
+ Pl × Sel / (256-0) (2)

  For example, it is assumed that there is a pixel whose selection value Sel is “300”. The selection value Sel “300” is larger than the identification value “256” for identifying the middle exposure image and is equal to or smaller than the identification value “512” for identifying the short exposure image. In this case, the two identification values that satisfy the condition of the expression (1) and have a smaller difference from the selection value Sel are “256” and “512”. Therefore, the WDR synthesis unit 50 may specify the two identification values “256” and “512” as identification values for identifying each of the two images.

  In such a case, the distance between the identification values of the two selected images is “512” − “256” = “256”, and the distance from the selection value Sel “300” to the identification value “256” is “ 300 ”−“ 256 ”=“ 44 ”, and the distance from the selection value Sel“ 300 ”to the identification value“ 512 ”is“ 512 ”−“ 300 ”=“ 212 ”. Therefore, the WDR composition unit 50 calculates the mixing ratio of the medium exposure image as “44” / “256” and the mixing ratio of the short exposure image as “212” / “256”, and these mixing ratios and the respective pixel values are calculated. Can be used to calculate the combined pixel value Out based on equation (1).

  As in this example, an intermediate value between “256” and “512” is selected by the LPF 40 in the vicinity of the boundary between the area where the short exposure image is used and the area where the medium exposure image is used. Is generated as Therefore, the WDR synthesizing unit 50 can smoothly change the mixing ratio of the pixel values of the short exposure image and the medium exposure image. As a result, it is possible to obtain a WDR composite image in which the vicinity of the boundary between images having different exposure amounts is smoothly connected.

  Further, for example, it is assumed that there is a pixel having a selection value Sel of “200”. The selection value Sel “200” is equal to or greater than an identification value “0” for identifying a long exposure image, and is equal to or less than an identification value “256” for identifying a middle exposure image. In such a case, the two identification values that satisfy the condition of Expression (2) and have a smaller difference from the selection value Sel are “0” and “256”. Therefore, the WDR synthesis unit 50 may specify these two identification values “0” and “256” as identification values for identifying each of the two images.

  In this case, the distance between the identification values of the two selected images is “256” − “0” = “256”, and the distance from the selection value Sel “200” to the identification value “0” is “ 200 ”−“ 0 ”=“ 200 ”, and the distance from the selection value Sel“ 200 ”to the identification value“ 256 ”is“ 256 ”−“ 200 ”=“ 56 ”. Therefore, the WDR synthesis unit 50 calculates the mixing ratio of the medium exposure image as “56” / “256”, and the mixing ratio of the long exposure image as “200” / “256”. Can be used to calculate the combined pixel value Out based on equation (2).

  As in this example, in the vicinity of the boundary between the area where the long exposure image is used and the area where the medium exposure image is used, an intermediate value between “0” and “256” is selected by the LPF 40. Is generated as Therefore, the WDR synthesizing unit 50 can smoothly change the mixing ratio of the pixel values of the long exposure image and the medium exposure image. As a result, it is possible to obtain a WDR composite image in which the vicinity of the boundary between images having different exposure amounts is smoothly connected.

  In the above example, when the selection value Sel is “256”, the condition of the above expression (2) is satisfied, but instead of the condition of the above expression (2), the above expression (1 Each condition may be changed so that the condition of

  The gradation compressing unit 60 performs compression processing on the WDR image generated by the WDR synthesizing unit 50 so that the bit range of the image signal having a wide dynamic range falls within a predetermined bit range. The subsequent stage of the gradation compression unit 60 is connected to an image processing engine including, for example, a demosaic unit that generates an RGB plane from Bayer data, a contour enhancement unit, and color management. Therefore, it is preferable that the data amount of the output signal from the gradation compression unit 60 is adjusted so as to match the size of the input data to the image processing engine (for example, about 12 bits). If the data size is simply reduced, the image is converted into a dark image. Therefore, it is preferable that the high luminance side is strongly compressed so as to approximate human visual characteristics.

  The “composite image” illustrated in FIG. 2 is an example of a composite image output by the image processing apparatus 1 according to the embodiment of the present invention. Referring to this example, it is understood that a discontinuous portion is not conspicuous even in the vicinity of a boundary where images having different exposure amounts are adjacent to each other, and a natural composite image is obtained.

  Further, according to the image processing apparatus 1 according to the embodiment of the present invention, by referring to selection information as one parameter, two images to be synthesized are selected for each pixel from three or more images having different exposure amounts. The mixing ratio can be calculated for each pixel. Therefore, according to the image processing apparatus 1 according to the embodiment of the present invention, when configured with hardware, the image processing apparatus 1 can be implemented with a smaller circuit scale, and when implemented with software. Can reduce the computation load.

  In the above, the WDR composition using three images with different exposure amounts has been mainly described, but the WDR composition using four or more images can be extended using the same method. For example, when four or more images are used, “768” and “0” are assigned as identification values for identifying each of the short exposure image and the long exposure image, and each of the two images having an intermediate exposure amount is identified. It is only necessary to assign “512” and “256” as identification values for this.

  Even when combining is performed using the selection information subjected to the processing for applying the low-pass filter, the convolution operation may be performed in the same manner only by changing the portion for selecting the two images to be combined. In this way, even if the number of images used for composition is increased to five, six,..., The method according to the embodiment of the present invention can be expanded and dealt with.

  FIG. 3 is a diagram illustrating an example of an operation flow of the image processing apparatus 1 according to the embodiment of the present invention. With reference to FIG. 3, a case where a short exposure image, a medium exposure image, and a long exposure image are output to each of the use image selection unit 30 and the WDR synthesis unit 50 will be described as an example. As shown in FIG. 3, first, the use image selection unit 30 sets the identification values D1 to D3 for identifying each of the short exposure image, the medium exposure image, and the long exposure image to a predetermined numerical value A with zero as a lower limit value. N-1 (in this example, since there are three images, 3-1 = 2) is defined in a range having an upper limit value, and a selected value indicating any one of these identification values D1 to D3 Generates selection information defined corresponding to each pixel (step S1).

  Subsequently, the LPF 40 performs low-pass filter processing as an example of predetermined processing on the selection information (step S2). The WDR synthesis unit 50 selects two images from a short exposure image, a medium exposure image, and a long exposure image based on a selection value defined in the selection information subjected to the low-pass filter processing, and 2 A combining process for combining corresponding pixels of each of the images is performed for each pixel (step S3). As described above, the composite image is preferably compressed by the gradation compression unit 60.

  According to the embodiment of the present invention, the identification value for identifying each of N (N is a natural number greater than or equal to 3) images photographed with different exposure amounts is set to a predetermined numerical value with N as a lower limit. For selection information that is defined in a range in which a numerical value obtained by multiplying is set as an upper limit, and a selection value that indicates one of the identification values for identifying each of the N images is defined corresponding to each pixel A processing unit for performing a predetermined process, a selection process for selecting two images from the N images based on a selection value defined in the selection information subjected to the predetermined process, and the two sheets There is provided an image processing apparatus comprising: a combining unit that performs combining processing for combining the corresponding pixels of each of the images for each pixel.

  With this configuration, in the vicinity of a boundary where a plurality of images photographed with different exposure amounts are adjacent, an intermediate value between the selection values of the plurality of adjacent images is generated as a selection value. Therefore, it is possible to obtain a composite image in which the mixture ratio is smoothly changed and the vicinity of the boundary is smoothly connected.

  Further, by referring to selection information as one parameter, it is possible to select two images to be combined for each pixel from three or more images having different exposure amounts. Therefore, when configured with hardware, it is possible to implement with a smaller circuit scale, and when implemented with software, it is possible to reduce the computation load.

  In the embodiment according to the present invention, the pixel values of two images are always used when synthesizing a certain pixel, whereas the prior art disclosed in Japanese Patent Application Laid-Open No. 2009-10565 is used. In the technology, it seems that there are cases where a composite result for one pixel is obtained using three images. In this respect, it may be pointed out that the synthesis performance according to the embodiment of the present invention may be degraded.

  However, in such a prior art, it is assumed that shooting is performed by changing EV by about ± 1, and in that case, it may be meaningful to combine three images. However, for example, when compositing images shot with exposure differences of several times as in the embodiment of the present invention, the difference in exposure is too large, meaning that three images are used for compositing. Is sparse. On the contrary, it can lead to mixing unnecessary information. For this reason, it is considered that this method of selecting and combining two images is reasonable and is not inferior to the prior art.

  Japanese Patent Application Laid-Open No. 2005-72965 discloses a method of smoothly connecting two images of short-time exposure and long-time exposure by averaging the short-time exposure and long-time exposure. In such a method, one synthesis parameter is defined for each pixel, and pixel values of two images are linearly combined for each pixel. According to this method, two images can be smoothly connected, but three or more images cannot be combined.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  As the signal to be processed by the method according to the embodiment of the present invention, it is most preferable to use Bayer data. However, the signal used as the signal to be processed is not limited to Bayer data, and may be RGB data. Alternatively, YUV data may be used.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Sensor 21 Short exposure image memory 22 Medium exposure image memory 23 Long exposure image memory 30 Use image selection part 40 LPF (processing part)
50 WDR synthesis unit (synthesis unit)
60 Gradation compression unit D0, D1, D2 Identification value A Predetermined numerical value Sel selection value Out Pixel value after composition

Claims (6)

An identification value for identifying each of N (N is a natural number of 3 or more) images photographed with different exposure amounts is set to a numerical value obtained by multiplying a predetermined numerical value by N-1 with zero as a lower limit value. A processing unit that performs a predetermined process on selection information that is defined by a range and that has a selection value that is defined corresponding to each pixel and that indicates one of the identification values for identifying each of the N images. ,
A selection process for selecting two images from the N images based on a selection value defined in the selection information subjected to the predetermined process and a corresponding pixel of each of the two images are synthesized. A combining unit that performs combining processing on each pixel;
An image processing apparatus comprising: The identification value for identifying each of the N images is a value obtained by multiplying the predetermined numerical value by each integer from 0 to N-1 and assigned in ascending or descending order of exposure amount. ,
The image processing apparatus according to claim 1. The predetermined process is a process of applying a low-pass filter.
The image processing apparatus according to claim 1. The synthesizing unit specifies two identification values having a smaller difference from the selection value defined in the selection information subjected to the predetermined processing as identification values for identifying each of the two images. ,
The image processing apparatus according to claim 1. The synthesizing unit determines a mixing ratio according to a difference value between a selection value defined in the selection information subjected to the predetermined process and an identification value of each of the two images, and based on the mixing ratio To synthesize the corresponding pixels of each of the two images,
The image processing apparatus according to claim 1. An identification value for identifying each of N (N is a natural number of 3 or more) images photographed with different exposure amounts is set to a numerical value obtained by multiplying a predetermined numerical value by N-1 with zero as a lower limit value. Performing a predetermined process on selection information defined in a range and having a selection value defined corresponding to each pixel, which is one of identification values for identifying each of the N images,
A selection process for selecting two images from the N images based on a selection value defined in the selection information subjected to the predetermined process and a corresponding pixel of each of the two images are synthesized. Performing synthesis processing for each pixel;
Including an image processing method.