KR20140133407A - Syestem and method for image processing - Google Patents

Abstract

The present invention provides a technique capable of obtaining a natural composite image by smoothly bonding the boundaries of three or more images and of reducing processing delay by making smaller circuit size when image composition is performed using the three or more images. According to an embodiment of the present invention, an image processing apparatus (1) is provided. The image processing apparatus comprises an LPF (40) and a WDR composite unit (50). The LPF performs LPF processing of selection information, in which an identification value for identifying each of the N (N is a natural number greater than or equal to three) number of images captured with different exposures is defined as a range using zero as a lower limit and using, as an upper limit, a value where a predetermined value A is multiplied by N-1, and in which a selection value indicating any one of the identification values for identifying the N number of images is defined to correspond to each pixel. The WDR composite unit performs a selection of two images from the N number of images based on the selection value defined in the LPF processed selection information, and a composition of pixels corresponding to each of the two images, with respect to each pixel.

Description

[0001] DESCRIPTION [0002] IMAGE PROCESSING APPARATUS AND IMAGE PROCESSING METHOD [

The present invention relates to an image processing apparatus and an image processing method.

(Hereinafter, simply referred to as a " short exposure image ") and an image of long time exposure (hereinafter, simply referred to as a "long exposure image" WDR (wide dynamic range) or HDR (high dynamic range), which obtains an image capturing a dynamic range, is increasing in number. The shooting function has a particularly large effect in a scene having a very high contrast ratio, such as a composition of backlight.

In recent years, there has also appeared a technique of photographing and synthesizing three or more images whose exposure has been changed for the purpose of expanding a photographable dynamic range. For example, Patent Document 1 discloses a technique of smoothly synthesizing three images photographed by changing the exposure.

In this method, the reference exposure image is converted into a gray-scale image, and the gray-scale image is divided into three areas of a bright area, a dark area and an intermediate area by threshold value processing of a density value, After filtering, three images are synthesized according to the color components of each pixel. The threshold value of the density value is determined by the score calculation for the phoneme of the reference exposure image using the boundary of the area divided according to the edge detection result and the density value.

[Prior Art Literature]

Patent Document 1: JP-A-2009-10566

Patent Document 2: JP-A No. 2005-72965

Here, when the technique of dividing the area using the reference exposure image is used, it can be coped with if the EV (Exposure Value) is about ± 1. However, when the above technique is applied to a photographing function called WDR in which the photographing is largely photographed with an order of several times the exposure amount, the whitewash form of the high luminance portion and the phenomenon of black marking are severe, The image to be used can not be properly determined.

In addition, if the number of images to be used in the synthesis is increased in order to further extend the photographable dynamic range, the determination of the synthesis object image becomes more difficult, and the possibility that proper synthesis results can not be obtained in a high luminance or low luminance region becomes high.

In addition, it is necessary to prepare an edge detection, a gray scale image, and an RGB plane for weighting. Since the telephone needs to be calculated with a score, it is possible to output a composite image There is a point that a very large circuit scale is needed to perform in hardware such as lack. Even if the same function is realized by software, a large amount of calculation is required and a large processing delay is considered to occur.

Thus, the present invention provides a technique for smoothly connecting boundaries of images when composing using three or more images, and obtaining a natural composite image. Further, the present invention provides a technique for further reducing the processing delay by further reducing the circuit scale when synthesizing using three or more images.

According to one embodiment of the present invention, the discrimination value for discriminating each of N (N is a natural number of 3 or more) images photographed at different exposure amounts is a value obtained by multiplying a predetermined numerical value by N-1 to an upper limit value A processing unit for performing a predetermined process on selection information defined in a range corresponding to each pixel and a selection value indicating any one of identification values for identifying each of the N pieces of images; And a combining unit for executing a combining process for combining corresponding pixels of each of the two images with respect to each of the pixels, There is provided an image processing apparatus comprising:

With the above arrangement, at the vicinity of a boundary between a plurality of images photographed at different exposure amounts, an intermediate value of the selection value of each of the adjacent plural images is generated as the selected value. Therefore, a composite image in which the vicinity of the boundary is smoothly connected can be obtained by smoothly changing the mixing ratio.

With reference to the selection information as one parameter, two images to be combined can be selected for each pixel from three or more images having different exposure amounts. Therefore, when the circuit is configured by hardware, it can be mounted with a smaller circuit scale, and when it is mounted with software, the computation load can be further reduced.

For example, the identification value for identifying each of the N images may be a value obtained by multiplying the predetermined value by each integer from 0 to N-1 in ascending or descending order of the exposure amount. According to the above configuration, the identification value can be assigned in the order of the exposure amount with the interval between the adjacent identification values being equal, so that the blending ratio can be calculated easily by using the selection value defined in the predetermined processed information .

It is preferable that the predetermined processing is processing for largely grading the selection information. This makes it possible to smoothly connect the vicinity of the boundary. For example, the predetermined process may be a low-pass filter process.

The method of selecting the two images for each pixel is not particularly limited. For example, the combining section may specify, as an identification value for identifying each of the two images, two identification values having a smaller difference from the selection values defined in the predetermined processed selection information. In this way, the two images may be selected for each pixel.

For example, the combining unit may determine a blending ratio in accordance with a difference value between a selection value defined in the predetermined processed selection information and an identification value of each of the two images, and based on the blending ratio, May be synthesized for each pixel of the image. In this manner, the mixing ratio can be calculated for each pixel.

According to another embodiment of the present invention, the discrimination value for discriminating each of N (N is a natural number of 3 or more) images photographed at different exposure amounts is a value obtained by multiplying a predetermined value by N-1, The method comprising the steps of: performing a predetermined process on selection information defined as a range defined by a range of pixel values of a plurality of pixels and a selection value indicating one of identification values for identifying each of the N images, Executing a selection process of selecting two images from the N images based on the selection values defined in the information and a combining process of combining the pixels of the two images with each other for each pixel Is provided.

According to the above method, an intermediate value of the selection value of each of the adjacent plural images is generated as the selected value in the vicinity of the adjacent boundary between a plurality of images photographed at different exposure amounts. Therefore, a composite image in which the vicinity of the boundary is smoothly connected can be obtained by smoothly changing the mixing ratio.

With reference to the selection information as one parameter, two images to be combined can be selected for each pixel from three or more images having different exposure amounts. Therefore, when the circuit is configured by hardware, it can be mounted with a smaller circuit scale, and when it is mounted with software, the computation load can be further reduced.

As described above, according to the present invention, it is possible to provide a technology for smoothly connecting boundaries of images when synthesizing using three or more images, thereby obtaining a natural composite image. Further, according to the present invention, in the case of synthesizing using three or more images, the circuit scale can be further reduced and the processing delay can be further reduced.

1 is a diagram showing an example of a functional configuration of an image processing apparatus according to an embodiment of the present invention.
2 is a diagram showing examples of various kinds of information in the WDR synthesis technique according to the embodiment of the present invention.
3 is a diagram showing an example of the operational flow of the image processing apparatus according to the embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description and drawings, constituent elements having substantially the same functional configuration are denoted by the same reference numerals, and redundant description will be omitted.

Also, in the present specification and drawings, a plurality of constituent elements having substantially the same functional configuration may be distinguished by attaching different alphabets to the same reference characters. However, when it is not necessary to distinguish each of a plurality of components having substantially the same functional configuration, only the same reference numeral is attached.

In the present specification, as an example of images of N (N is a natural number of 3 or more) images taken at different exposure amounts, three images of a single exposure image, a medium exposure image, and a long exposure image are taken in order of decreasing exposure dose Will be mainly described. However, the image processing apparatus 1 according to the embodiment of the present invention is also applicable to the case of taking four or more images as described later.

First, the functional configuration of the image processing apparatus 1 according to the embodiment of the present invention will be described. 3 is a diagram showing the functional configuration of the image processing apparatus 1 according to the embodiment of the present invention. 3, the image processing apparatus 1 includes a sensor 10, a short exposure image memory 21, a middle exposure image memory 22, a long exposure image memory 23, a used image selection unit 30 ), An LPF (Low Pass Filter) 40, a WDR synthesis unit 50, and a gray scale compression unit 60. Hereinafter, the function of each functional block included in the image processing apparatus 1 will be described in detail.

The image processing apparatus 1 changes the exposure setting of the sensor 10 so as to successively take three images. In this case, the exposure is firstly performed in the short exposure, then the middle exposure, and then the long exposure . The short-exposure image photographed by the exposure is written in the short-exposure image memory 21. The image processing apparatus 1 changes the exposure setting when the short exposure imaging is completed, and performs middle exposure imaging. And the middle exposure image is written in the middle exposure image memory 22. [ The image processing apparatus 1 changes the exposure setting when the middle exposure imaging is completed, and performs long exposure imaging. The long exposure image photographed by long exposure is written in the long exposure image memory 23. [

In the example shown in Fig. 1, the image processing apparatus 1 has one common system for outputting a long exposure image, a middle exposure image, and a short exposure image, in which the sensor 10 detects a long exposure image, However, the long exposure image, the intermediate exposure image, and the short exposure image may be simultaneously photographed. In this case, the image processing apparatus 1 may have three systems: a system for outputting a long-exposure image from the sensor 10, a system for outputting a heavy exposure image, and a system for outputting a short exposure image. The respective shutter times are determined, for example, by the dynamic range of the subject, the sensor specifications, and the like.

In addition, the read operation of each of the short exposure image, the middle exposure image, and the long exposure image may be changed depending on how the system is woven in the case of hardware implementation. For example, each of the short exposure image, the middle exposure image, and the long exposure image may be temporarily stored in a memory such as DRAM (Dynamic Random Access Memory) and read out again at the time of composition.

Here, in the embodiment of the present invention, terms such as a single exposure image, a medium exposure image, and a long exposure image are used, but these terms do not limit the absolute exposure time of each of the three photographed images. Therefore, when three images having different exposure times are photographed, they are named as a single exposure image, a medium exposure image, and a long exposure image in the order of short exposure time among the three images.

The "short exposure image", the "middle exposure image", and the "long exposure image" shown in FIG. 2 show examples of the short exposure image, the middle exposure image, and the long exposure image, respectively. Referring to the " single-exposure image ", the scenery outside the window having a very high luminance was photographed at an appropriate exposure. However, the other areas became underexposed and became almost black and visibility was lowered.

Referring to the " intermediate exposure image ", the area on the shelf is photographed with proper exposure, and the presence of the mobile phone can be recognized. However, the outside of the window is exposed with overexposure, and under the shelf is still under exposed, so the visibility is low. Referring to the " long exposure image ", the area under the dark shelf can be photographed with proper exposure to visually recognize the presence of the bag. However, out-of-window and over-the-shelf areas are exposed and become less visible.

Returning to Fig. 1, description will be continued. The sensor 10 is constituted by an image sensor which forms an image on the light receiving plane of the image pickup element from outside and photoelectrically converts the formed light into a charge quantity and converts the charge quantity into an electric signal. The type of the image sensor is not particularly limited, and may be, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

The used image selection unit 30 selects the single exposure image read from the short exposure image memory 21, the medium exposure image read out from the medium exposure image memory 22, and the long exposure image read from the long exposure image memory 23 A saturation state, a motion, and the like of each of the short exposure image, the middle exposure image, and the long exposure image with reference to the image to generate selection information for selecting either the short exposure image, the middle exposure image, or the long exposure image as the used image do. Various algorithms are assumed as algorithms for selecting either the short exposure image, the medium exposure image, or the long exposure image.

For example, since a saturated region in a long exposure image is highly unlikely to be saturated in a medium exposure image, a heavy exposure image can be selected as a used image in the corresponding region. However, if the area is also saturated in the middle exposure image, a short exposure image may be selected as the used image of the area. However, with this processing alone, artefacts can occur such as a double contour in a region with large motion. Therefore, a process of detecting a motion and reducing a phenomenon of a double contour may be performed. The algorithm for selecting either the short exposure image, the middle exposure image, or the long exposure image including the above process is not particularly limited.

In this specification, the identification values (D0 to D2) for identifying each of the long exposure image, the middle exposure image, and the short exposure image are set to N-1 (in this example, 3 " = 2) so that the upper limit value is defined as a range. The selection information is defined in correspondence with each pixel to a selection value indicating one of identification values (D0 to D2) for identifying each of a long exposure image, a middle exposure image, and a short exposure image. The predetermined value A is not particularly limited, but if it is the power of 2, division by the distance between the discrimination values can be performed by a simple bit shift operation as described later.

The discrimination values (D0 to D2) for discriminating each of the long exposure image, the middle exposure image and the short exposure image are obtained by multiplying the predetermined value A by "0", "1" and "2" in ascending or descending order As shown in Fig. In the following example, a case where "0" as a result of multiplying the predetermined value A by "256" and "256" by "0" is assigned as an identification value D0 for identifying a long exposure image is explained .

Similarly, in the example shown below, "256" as a result of multiplying "256" by "1" is assigned as an identification value D1 for identifying a middle exposure image, and "256" is multiplied by "2" Quot; 512 " is assigned as an identification value D2 for identifying a single exposure image will be described.

The " selection information " shown in Fig. 2 shows an example of the selection information generated by the used image selector 30. Fig. This " selection information " indicates that the "long exposure image" is selected as the used image of the region not saturated in the "long exposure image", the use of the region saturated in the "long exposure image" Quot; middle exposure image " is selected as the image, and the " short exposure image " is selected as the used image of the saturated region in the " medium exposure image ".

In this " selection information ", the black area indicates that " long exposure image " is selected and each selection value in the black area is " 0 ". The gray area indicates that "medium exposure image" is selected, and each selection value of the gray area is "256". Further, the white area indicates that "short exposure image" is selected, and each selection value of the white area is "512".

Returning to Fig. 1, description will be continued. The LPF 40 processes the selection information thus defined. Hereinafter, the description will be continued by exemplifying the case where the predetermined process is the low-pass filter process, but the predetermined process is not particularly limited. The low-pass filter process corresponds to the process of removing high-frequency components from the selection information. By this processing, the selection information is greatly graded so that the boundaries of the short exposure image, the middle exposure image, and the long exposure image can be smoothly connected when synthesized by the WDR combining section 50. [

The bit precision of the output signal from the LPF 40 should be ensured so that the boundary of each of the short exposure image, the middle exposure image and the long exposure image is smoothly connected. The gradation method of the selection information is not particularly limited as various methods are assumed, but it is preferable that the gradation method is determined in consideration of the gradation effect, the calculation amount, the circuit scale, and the like. The " selection information + LPF " shown in Fig. 2 shows an example of selection information in which the LPF 40 performs low-pass filter processing on " selection information ".

Referring to the " selection information + LPF " shown in Fig. 2, in the vicinity of the boundary between two areas having different selection values in the " selection information ", intermediate values of the selection values of the two areas are output as selected values. On the other hand, in a region where certain selection values are continuous in the " selection information ", the selected value of the region is maintained as it is.

The WDR combining section 50 performs selection processing for selecting two images from the short exposure image, the middle exposure image, and the long exposure image based on the selection value defined in the low-pass filtered selection information, And performs synthesis processing for each pixel to synthesize the corresponding pixels of each image.

For example, in the selection process, the WDR synthesis unit 50 generates two images from the short exposure image, the middle exposure image, and the long exposure image based on the selection values defined in the low-pass filtered selection information, . More specifically, the WDR combining unit 50 specifies two identification values having a smaller difference from the selection values defined in the low-pass-filtered selection information as identification values for identifying each of the two images do.

For example, in the combining process, the WDR combining section 50 determines the blending ratio according to the difference value between the selection value defined in the low-pass-filtered selection information and the identification value of each of the two selected images , And a corresponding pixel of each of the two images may be synthesized based on the blending ratio. For example, the distances from the selected values to the respective identification values for the distances between the identification values of the two selected images may be determined as the respective mixing ratios.

For example, the WDR synthesis section 50 sets the selection value to Sel for each pixel, sets the pixel value of the short exposure image to Ps, sets the pixel value of the middle exposure image to Pm, sets the pixel value of the long exposure image to Pl , And when the synthesized pixel value is Out, it can be synthesized on the basis of the following expression (1) or (2).

[Equation 1]

IF (256 <

Out = Ps * (512-Sel) / (512-256) + Pm * (Sel-256) / (512-256)

&Quot; (2) "

IF (0? Sel? 256)

Out = Pm * (Sel-256) / (256-0) + Pl * Sel / (256-0)

For example, assume that a pixel with a selection value Sel of " 300 " exists. This selection value Sel " 300 " is larger than the identification value " 256 " for identifying the middle exposure image and is equal to or smaller than the identification value " 512 " In this case, the two identification values that satisfy the condition of the expression (1) and whose difference from the selection value Sel is smaller are "256" and "512". Therefore, the WDR combining unit 50 may specify these two identification values "256" and "512" as identification values for identifying each of the two images.

In this case, the distance between the identification values of the selected two images is "512" - "256" = "256", and the distance from the selection value Sel "300" to the identification value "256" = 44, and the distance from the selection value Sel "300" to the identification value "512" is "512" - "300" = "212". Therefore, the WDR combining section 50 calculates the mixing ratio of the intermediate exposure image as " 44 " / " 256 " and the mixing ratio of the single exposure image as " 212 " / " 256 " The synthesized pixel value Out can be calculated based on the expression (1).

As shown in this example, in the vicinity of the boundary between the region where the short exposure image is used as the used image and the region where the heavy exposure image is used as the used image, an intermediate value of "256" and "512" is generated as a selection value by the LPF 40 do. Therefore, the WDR combining section 50 can smoothly change the mixing ratio of the pixel values of the short exposure image and the middle exposure image. As a result, it is possible to obtain a WDR synthesized image in which the vicinity of the boundary between the images of different exposure amounts is smoothly connected.

For example, it is assumed that a pixel having a selection value Sel of "200" exists. The selection value Sel " 200 " is equal to or larger than the identification value " 0 " for identifying the long exposure image, and is equal to or smaller than the identification value " 256 " In this case, the two identification values which satisfy the condition of the expression (2) and whose difference from the selection value Sel is smaller are "0" and "256". Therefore, the WDR combining section 50 may specify the two identification values " 0 " and " 256 " as identification values for identifying each of the two images.

In this case, the distance between the identification values of the two selected images is "256" - "0" = "256", and the distance from the selection value Sel "200" to the identification value "0" is "200" = 200, and the distance from the selection value Sel "200" to the identification value "256" is "256" - "200" = "56". Therefore, the WDR combining section 50 calculates the mixing ratio of the middle exposure image to "56" / "256" and the mixing ratio of the long exposure image to "200" / "256" The synthesized pixel value Out can be calculated based on the equation (2).

As shown in this example, in the vicinity of the boundary between the region where the long exposure image is used as the used image and the region where the heavy exposure image is used as the used image, the LPF 40 generates an intermediate value of "0" do. Therefore, the WDR combining section 50 can smoothly change the mixing ratio of the pixel values of the long-exposure image and the middle-exposure image. As a result, it is possible to obtain a WDR synthesized image in which the vicinity of the boundary between the images of different exposure amounts is smoothly connected.

In the above example, when the selection value Sel is " 256 ", the condition of the equation (2) is satisfied, but the condition of the equation (1) is changed instead of the condition of the equation It is also good.

The gradation compression unit 60 executes a compression process for converging the bit range of the image signal having a wide dynamic range to a predetermined bit range on the WDR image generated by the WDR synthesis unit 50. [ The rear end of the gradation compression section 60 is connected to an image processing engine including a demosaic section, an outline emphasis section, color management, and the like for generating RGB planes from Bayer data, for example. Therefore, it is preferable that the data amount of the output signal from the gradation compression section 60 is adjusted to be suitable for the input data size to the image processing engine (for example, about 12 bits). Since only the data size is simply reduced to a dark image, the high-luminance side may be strongly compressed so as to approximate human visual characteristics.

The "composite image" shown in FIG. 2 is an example of a composite image output by the image processing apparatus 1 according to the embodiment of the present invention. With reference to this example, it is understood that a discontinuous portion is not conspicuous even in the vicinity of a border adjacent to the images of different exposure amounts, and a natural composite image is obtained.

With reference to the selection information as one parameter, according to the image processing apparatus 1 of the embodiment of the present invention, two images to be combined can be selected for each pixel from three or more images having different exposure amounts, The ratio can be calculated for each pixel. Therefore, according to the image processing apparatus 1 of the embodiment of the present invention, when the image processing apparatus 1 is configured by hardware, the image processing apparatus 1 can be mounted with a smaller circuit scale, Can be made small.

In the above description, WDR synthesis using three images with different exposure amounts is mainly described. However, WDR synthesis using four or more images can be extended using the same technique. For example, when four or more images are used, " 768 " and " 0 " are assigned as discrimination values for discriminating each of the short exposure image and long exposure image, and an identification Quot; 512 " and " 256 "

Even when synthesizing using the selection information subjected to the low-pass filter processing, the convolution operation can be performed in the same manner as that for changing the portion for selecting two images to be combined. In this way, if the image used for the synthesis is 5 sheets, 6 sheets, The method according to the embodiment of the present invention can be extended and coped with.

3 is a diagram showing an example of the operation flow of the image processing apparatus 1 according to the embodiment of the present invention. A case where a short exposure image, a middle exposure image, and a long exposure image are outputted to the used image selector 30 and the WDR combining section 50 will be described with reference to FIG. 3 as an example. As shown in Fig. 3, the priority image selecting unit 30 sets the identification values D1 to D3 for identifying each of the short exposure image, the middle exposure image, and the long exposure image to 0 (3-1 = 2 because the number of images is three in this example) is set to an upper limit value, and a selection value indicating one of the identification values D1 to D3 corresponds to each pixel (Step S1).

Subsequently, the LPF 40 performs a low-pass filter process as an example of a predetermined process on the selection information (step S2). The WDR synthesis section 50 performs a selection process of selecting two images from a short exposure image, a middle exposure image, and a long exposure image based on the selection values defined in the low-pass filtered selection information, A synthesis process for synthesizing each corresponding pixel is executed for each pixel (step S3). As described above, the synthesized image is preferably compressed by the tone compression section 60. [

According to the embodiment of the present invention, the discrimination value for discriminating each of N (N is a natural number of 3 or more) images photographed at different exposure amounts is a value obtained by multiplying a predetermined value by N-1 with 0 being the lower limit value A processing unit that performs a predetermined process on selection information defined in a range corresponding to each pixel and that has a selection value indicating any one of identification values for identifying each of the N pieces of images; A selecting process of selecting two images from the N pieces of images based on the defined selection values and a synthesizing section executing synthesis processing for synthesizing the pixels corresponding to the two images with respect to each pixel An image processing apparatus is provided.

With the above arrangement, in the vicinity of the adjacent boundary between a plurality of images photographed at different exposure amounts, an intermediate value of the selection values of each of the adjacent plurality of images is generated as the selected value. Therefore, a composite image in which the vicinity of the boundary is smoothly connected can be obtained by smoothly changing the mixing ratio.

With reference to the selection information as one parameter, two images to be combined can be selected for each pixel from three or more images having different exposure amounts. Therefore, when the circuit is configured by hardware, it can be mounted with a smaller circuit scale, and when the circuit is mounted by software, the computation load can be reduced.

In the embodiments of the present invention, pixel values of two images are always used when synthesizing a pixel, whereas in the technique disclosed in Japanese Patent Laid-Open Publication No. 2009-10566, three images are used The synthesis result for one pixel may be obtained. From this point of view, it may be pointed out that the performance of synthesis according to the embodiment of the present invention is not deteriorated.

However, in the above prior art, it is assumed that the EV is changed by about ± 1, and in this case, there may be a meaning of synthesizing three images. However, for example, when images photographed with exposure differences of several times as in the embodiment of the present invention are synthesized, the exposure difference is too large and the meaning of using three images for synthesis is faded. Not only that, but you can also mix unnecessary information. Therefore, it is considered that this method of selecting and synthesizing two sheets is reasonable and does not fall behind the prior art.

Japanese Patent Application Laid-Open No. 2005-72965 discloses a method of smoothly connecting two images, that is, a short-time exposure and a long-time exposure, by averaging the short-time exposure and the long-time exposure. In this method, one synthesis parameter is defined for each pixel, and the pixel values of two images are linearly combined for each pixel. According to the above method, two images can be smoothly connected but three or more images can not be synthesized.

Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited thereto. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims and their equivalents. .

As the signal to be processed by the method according to the embodiment of the present invention, it is most preferable to use the Bayer data, but the signal used as the signal to be processed is not limited to the Bayer data and may be RGB data or YUV data .

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, medium, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified into one or more software modules for performing the processing according to the present invention, and vice versa.

The specific acts described in the present invention are, by way of example, not intended to limit the scope of the invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. Also, unless explicitly mentioned, such as " essential ", " importantly ", etc., it may not be a necessary component for application of the present invention.

The use of the terms " above " and similar indication words in the specification of the present invention (particularly in the claims) may refer to both singular and plural. In addition, in the present invention, when a range is described, it includes the invention to which the individual values belonging to the above range are applied (unless there is contradiction thereto), and each individual value constituting the above range is described in the detailed description of the invention The same. Finally, the steps may be performed in any suitable order, unless explicitly stated or contrary to the description of the steps constituting the method according to the invention. The present invention is not necessarily limited to the order of description of the above steps. The use of all examples or exemplary language (e.g., etc.) in this invention is for the purpose of describing the present invention only in detail and is not to be limited by the scope of the claims, It is not. It will also be appreciated by those skilled in the art that various modifications, combinations, and alterations may be made depending on design criteria and factors within the scope of the appended claims or equivalents thereof.

Accordingly, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all ranges that are equivalent to or equivalent to the claims of the present invention as well as the claims .

1: Image processing device
10: Sensor
21: Short exposure image memory
22: Medium exposure image memory
23: Long exposure image memory
30: Used image selection unit
40: LPF (processing section)
50: WDR combining section (combining section)
60: gradation compression section
D0, D1, D2: Identification value
A:
Sel: Selection value
Out: Pixel value after synthesis

Claims (10)

The identification value for identifying each of the N (N is a natural number of 3 or more) images photographed at different exposure amounts is defined as a range in which a value obtained by multiplying a predetermined numerical value by N-1 is set as an upper limit value, And a discrimination value for discriminating each of the pixels of the image, which is defined in correspondence with each pixel,
A selection process of selecting two images from the N pieces of images based on the selection values defined in the predetermined processed selection information and a combining process of combining corresponding pixels of the two images with each pixel For example,
And the image processing apparatus. The method according to claim 1,
Wherein the identification value for identifying each of the N pieces of images is a value that is obtained by multiplying each of the integers from 0 to N-1 to the predetermined numerical value in an ascending or descending order of the exposure amount. The method according to claim 1,
Wherein the predetermined processing is low pass filter processing. The method according to claim 1,
Wherein the combining section specifies two identification values having a smaller difference from a selection value defined in the predetermined processed selection information as an identification value for identifying each of the two images. The method according to claim 1,
Wherein the combining section determines a blending ratio in accordance with a difference value between a selection value defined in the predetermined processed selection information and an identification value of each of the two images and determines, based on the blending ratio, And the corresponding pixels of the pixel are combined. The identification value for identifying each of the N (N is a natural number of 3 or more) images photographed at different exposure amounts is defined as a range in which a value obtained by multiplying a predetermined numerical value by N-1 is set as an upper limit value, A step of performing a predetermined process on selection information in which a selection value indicating any one of identification values for identifying each of the plurality of pixels is defined for each pixel,
A selection process of selecting two images from the N pieces of images based on the selection values defined in the predetermined processed selection information and a combining process of combining corresponding pixels of the two images with each pixel , ≪ / RTI >
And the image processing method. The method according to claim 6,
Wherein the identification value for identifying each of the N pieces of images is a value obtained by multiplying each of the integers from 0 to N-1 to the predetermined numerical value in an ascending or descending order of the exposure amount. The method according to claim 6,
Wherein the predetermined processing is low pass filter processing. The method according to claim 6,
Wherein the step of executing the combining processing for each pixel comprises:
And identifies two identification values each having a smaller difference from the selection value defined in the predetermined processed selection information as an identification value for identifying each of the two images. The method according to claim 6,
Wherein the step of performing the combining processing for each pixel comprises:
Wherein the blending ratio is determined in accordance with the difference value between the selection value defined in the predetermined processed selection information and the identification value of each of the two images, and based on the blending ratio, Is synthesized.

Images