JP2009017583A - Image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device which can obtain gradation conversion characteristics, with fewer times of operations, using smaller memories. <P>SOLUTION: The image processing device synthesizes a wide dynamic range image from multiple images of different exposures and includes a feature amount calculating section 10 having a local domain extraction portion 20 which extracts a local domain of a prescribed size for each image, a first signal extraction portion 22 which calculates an equivalent signal for the luminance in a grid point shifting horizontally and vertically by half a pixel, and a second signal extraction portion 23, which calculates an edge component of the equivalent signal of luminance; a proper exposure extracting section 11 which extracts a proper exposure domain, based on the equivalent signal of luminance; a gradation correction section 12, which performs gradation correction to the proper exposure domain based on the edge component; and an image-compositing interpolation section which generates a wide dynamic range image by compositing the gradation-corrected proper exposure domain. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that generates one wide dynamic range image from a plurality of images captured under different exposure conditions.

  2. Description of the Related Art Conventionally, there has been proposed an image processing apparatus that generates a single wide dynamic range image by combining a plurality of images captured under different exposure conditions.

  As an example of such a thing, Japanese Patent Application No. 11-338551 divides each image into an appropriate exposure area and an improper exposure area, and performs gradation correction for each appropriate exposure area. An image processing apparatus that generates a single wide dynamic range image by synthesizing appropriate exposure areas for each image is described. Further, as an example of a device to which this image processing apparatus is applied, a subject with a wider dynamic range is described. A super latitude digital camera capable of imaging is described.

  In more detail, such a digital camera basically performs the following processing.

  First, for example, imaging is performed using a single-plate CCD in which RGB color filters are arranged in a Bayer array (see FIG. 4A showing the embodiment of the present invention), and R pixels in the imaging signal are captured. An R image for one screen is created by interpolating the signal, and similarly, a signal for the G pixel is interpolated, and a signal for the B pixel is interpolated to obtain a G image for each screen. A so-called three-plate process for creating a B image is performed.

  Next, based on these RGB signals, the following calculation is performed for each pixel to calculate the luminance signal Y and the luminance difference signals Cb and Cr.

Y = 0.29900R + 0.58700G + 0.14400B
Cb = −0.16874R−0.33126G + 0.50000B
Cr = 0.50000R-0.41869G-0.0811B
Thereafter, an edge component in the screen is extracted by applying a filter such as Laplacian to the luminance signal Y among these, and further, binarization processing for comparison with a predetermined threshold is performed to detect an edge.

  Thereafter, the appearance frequency of the edge with respect to the luminance is created as a histogram, further converted into a cumulative histogram or the like, and a process of obtaining a gradation conversion curve based on histogram flattening is performed.

This histogram flattening is a technique based on the premise that the main subject has many edges and non-main parts such as the background have few edges.
Japanese Patent Laid-Open No. 10-023324 Japanese Patent Application No. 11-355650

  In the general processing as described above, in order to calculate the gradation conversion characteristics, an image memory for each of three RGB screens created from a single plate signal and a luminance signal Y created from these RGB three plate signals are stored. The image memory for five screens, that is, the memory for one screen and the memory for one screen for storing the edge component extracted from the luminance signal Y, is required.

In addition, the calculation necessary for calculating the gradation conversion characteristics is as follows. First, when calculating a 3-plate RGB signal from a single-plate signal,
R signal: (addition 1, division 1) x number of pixels x 3/4
G signal: (addition 3, division 1) x number of pixels / 2
B signal: (addition 1, division 1) x number of pixels x 3/4
Only the calculation is necessary.

Next, when calculating the luminance signal Y from these RGB signals, from the arithmetic expression of the luminance signal Y,
Y signal: (addition 2, multiplication 3) × the number of pixels needs to be calculated.

Furthermore, when calculating the edge component from the luminance signal,
Edge component: (addition 10, subtraction 2, multiplication 4, logic 4) × the number of pixels is required.

And when binarizing the extracted edge component,
Binarization: (Logic 1) × Calculation of the number of pixels is required.

In addition, when calculating a histogram from edges detected by binarization,
Histogram: (addition 1, subtraction 2, division 2, logic 1) × the number of pixels needs to be calculated.

Thus, until the histogram is calculated as a total,
The calculation of (addition 16, subtraction 4, multiplication 7, division 4, logic 6) × number of pixels is required.

In the case of a CCD having a complementary color filter (see FIG. 5A showing the embodiment of the present invention), before the histogram is similarly calculated,
(Addition 16, subtraction 4, multiplication 7, division 5, logic 7) x number of pixels and memory for 6 screens are required.

  As described above, when conventional general processing is performed, a large working memory is required and a large number of operations are required, which requires cost and processing time. Further, when considering such a processing circuit as an IC, a large-capacity memory becomes a bottleneck and it cannot be said that it is suitable for a single chip.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an image processing apparatus capable of obtaining tone conversion characteristics with a smaller number of operations and a smaller memory.

In order to achieve the above object, an image processing apparatus according to an aspect of the present invention processes an image group made up of a plurality of images captured under different exposure conditions on the same subject to produce one wide dynamic range image. An image processing device to generate,
Local region extraction means for extracting a local region of a predetermined size centered on a pixel position outside the integer coordinates for each image in the image group;
From the pixel information in the local region, a first luminance equivalent signal that is a signal corresponding to a luminance signal at a pixel position outside one integer coordinate, and a vertical or horizontal direction from the pixel position outside the one integer coordinate. The second luminance equivalent signal, which is a signal corresponding to the luminance signal at the pixel position outside the integer coordinates existing at the position separated by one pixel, is obtained as an image feature amount, and the obtained first luminance is obtained. Edge calculation means for calculating an edge component at a pixel position outside the one integer coordinate as a feature quantity of an image using a differential filter shown as a difference between the equivalent signal and the second luminance equivalent signal;
Edge calculation repetition means for performing edge component calculation processing by the edge calculation means for each of the extracted local regions and calculating an edge component at a pixel position outside the integer coordinates on the image as a feature amount of the image;
By comparing each of the luminance-corresponding signals obtained by the edge calculating means with a predetermined value, it is determined whether or not the pixel corresponding to the pixel position outside the integer coordinates is appropriate exposure. A proper exposure extraction means for extracting a proper exposure area composed of a set of pixels determined to be present;
Gradation correction means for performing gradation correction of the appropriate exposure area based on the feature amount;
A synthesizing unit that generates a single wide dynamic range image by synthesizing an appropriate exposure area that has been tone-corrected by the tone correcting unit and interpolating into a three-plate state;
It has.

  An image processing apparatus according to the present invention calculates a feature amount at a pixel position outside an integer coordinate, extracts an appropriate exposure area based on an image signal obtained in the process, and performs gradation correction based on the feature amount. Therefore, there is an advantage that it is possible to reduce the number of calculations and the amount of memory until gradation correction is performed.

Embodiments of the present invention will be described below with reference to the drawings.
1 to 6 show an embodiment of the present invention, and FIG. 1 is a block diagram showing a basic configuration of an electronic camera.

  In the present embodiment, the image processing apparatus of the present invention is applied to an electronic camera. For simplicity, two images of a short-time exposure image and a long-time exposure image are combined to create a single wide dynamic range image. However, the present invention can be applied to a case where a larger number of images are synthesized.

  This electronic camera is composed of a single-plate color CCD having an electronic shutter function, a CCD 4 for photoelectrically converting a subject image and outputting it as an image signal, and a lens system for forming a subject image on the CCD 4 1, a diaphragm 2 for controlling the passage range of the light beam that has passed through the lens system 1, a low-pass filter 3 that is an optical filter for removing unnecessary high-frequency components from the light beam that has passed through the diaphragm 2, and An A / D converter 5 for converting an analog image signal which has been output from the CCD 4 and then subjected to removal of noise components by a correlated double sampling circuit (not shown) and then amplified to a digital signal, and this The image data for one screen digitized by the A / D converter 5 is accumulated, and an image by long exposure and an image by short exposure are stored. The image data is read out from the first image buffer 6a and the second image buffer 6b to be stored and the first image buffer 6a used for accumulating the photometry and focus detection data. Image data is read from the first image buffer 6a and the photometric evaluation section 7 for controlling the aperture diameter of the diaphragm 2 and the electronic shutter of the CCD 4 so as to obtain a brightness distribution and obtain an appropriate exposure at the time of imaging. An in-focus detection unit 8 that performs detection and controls an AF motor 9 to be described later based on the detection result, and an AF lens of the lens system 1 is driven by the in-focus detection unit 8 to drive an object on the CCD 4 An edge component, which is a feature value, is extracted from the AF motor 9 for forming an image and the image data read from the first and second image buffers 6a and 6b. Based on a feature amount calculation unit 10 serving as a collection amount calculation unit and a signal corresponding to a luminance signal related to long-time exposure output from the feature amount calculation unit 10, the pixel is appropriate for each pixel constituting the entire screen. A proper exposure extraction unit 11 serving as a proper exposure extraction unit that extracts and outputs divided image information based on the result, and an edge output from the feature amount calculation unit 10 A gradation correction unit 12 that is a gradation correction unit that generates gradation conversion curves based on the components and performs gradation conversion of the appropriate exposure area output from the appropriate exposure extraction unit 11, and output from the gradation correction unit 12 A composite means for generating a single wide dynamic range image by synthesizing a proper exposure area related to long exposure after gradation conversion and a proper exposure area related to short exposure and interpolating into a three-plate state. The result of the detection by the synthesizing interpolation unit 13, the output unit 14 that outputs the wide dynamic range image synthesized by the image synthesis interpolation unit 13 to, for example, a recording medium or a display device, and the photometric evaluation unit 7 or the in-focus detection unit 8 And a control unit 15 that controls the entire electronic camera including the feature amount calculation unit 10, the gradation correction unit 12, the image synthesis interpolation unit 13, and the output unit 14.

  Next, FIG. 2 is a block diagram showing a detailed configuration of the feature amount calculation unit 10.

  The feature amount calculation unit 10 reads out image data from the first image buffer 6a storing a long-time exposure image, generates a signal corresponding to a luminance signal, and outputs the signal to the appropriate exposure extraction unit 11. The edge component, which is the feature amount of the long exposure image and the short exposure image, is sequentially calculated and output to the gradation correction unit 12. For example, a local region extraction for extracting a local region of 5 × 5 pixels is performed. A local region extracting unit 20 as a means, a working buffer 21 for storing image data of the extracted local region, a first signal extracting unit 22 as an edge calculating unit for extracting a first signal corresponding to a luminance signal, and features And a second signal extraction unit 23 serving as an edge calculation means for extracting a second signal as an edge component which is a quantity.

  FIG. 3 is a block diagram showing a detailed configuration of the gradation correction unit 12.

  The gradation correction unit 12 performs gradation correction so that the long exposure image and the short exposure image are combined by the subsequent image composition interpolation unit 13 to be suitable for a wide dynamic range image. A histogram creation unit 30 that is a histogram creation unit that creates an edge histogram by discriminating an edge from the edge component output from the feature amount calculation unit 10 for the appropriate exposure region extracted by the proper exposure extraction unit 11, and the edge histogram A conversion curve calculation unit 31 that is a gradation conversion curve calculation unit that calculates a gradation conversion curve based on the conversion, and a conversion that performs gradation correction of the image data output from the feature amount calculation unit 10 based on the calculated gradation conversion curve And a conversion unit 32 as means.

  Next, FIG. 6 is a flowchart showing a calculation process of gradation conversion characteristics, and FIG. 4 is a diagram showing means for calculating edge components in the primary color filter CCD having the Bayer array.

  First, the case where the CCD 4 made of a single plate is provided with a color filter array having a primary color Bayer arrangement as shown in FIG.

  The subject image formed on the CCD 4 is captured a plurality of times under different exposure conditions, and as described above, for example, imaging by long exposure and imaging by short exposure are performed in this order. Are sequentially output as image signals.

  These image signals are converted into digital signals by the A / D converter 5 and then stored in the first image buffer 6a and the second image buffer 6b, respectively.

  The photometric evaluation unit 7 and the in-focus detection unit 8 send the AE information and the AF information to the control unit 17 based on the long-exposure image data stored in the first image buffer 6a. Output.

  On the other hand, the feature amount calculation unit 10 sequentially reads the image data stored in the first image buffer 6a and the second image buffer 6b in a predetermined direction along the pixel array (step S1), thereby generating a video signal. Is scanned (step S2).

  Then, the local region extraction unit 20 extracts a local region signal having, for example, 5 × 5 pixels as shown in FIG. 4A (step S3), and the local region signal is converted into a subsequent gradation correction unit. 12 for use (step S4) and temporarily stored in the work buffer 21.

  The first signal extraction unit 22 extracts the first signal corresponding to the luminance signal from the local region data as shown in FIG. 4A stored in the work buffer 21 (step S5).

  That is, in the first signal extraction unit 22, the luminance signal at the position that becomes the lattice point between the pixels arranged in the vertical and horizontal directions, that is, the pixel position that is shifted by 0.5 pixels in the vertical and horizontal directions, is obtained. The G signal is substituted for interpolation and extracted.

  Specifically, as shown in FIG. 4B, the relative luminance equivalent signal of the lattice point between the pixel G1 and the pixel G3 is calculated as G1 + G3. Similarly, the pixel G1 and the pixel G4 The relative luminance equivalent signal of the grid point between G1 + G4, the relative luminance equivalent signal of the grid point between the pixel G3 and the pixel G6 is G3 + G6, and the relative luminance of the grid point between the pixel G4 and the pixel G6 A typical luminance equivalent signal is calculated as G4 + G6, etc.

  The luminance equivalent signal substituted with the G signal should be divided by 2 to obtain an average value, but in order to extract a proper exposure area in the proper exposure extraction unit 11 in the subsequent stage, a relative signal is used. However, in order to suffice, the number of operations is reduced by omitting division by two.

  When the luminance-corresponding signal as the first signal is calculated in this way, it is output from the first signal extraction unit 22 to the appropriate exposure extraction unit 11 (step S6).

  Next, the second signal extraction unit 23 extracts an edge component (second signal), which is a feature amount, from local area data as shown in FIG. 4A stored in the work buffer 21. (Step S7).

  That is, the second signal extraction unit 23 extracts an edge component as the second signal by applying a second-order differential filter such as Laplacian. As is well known, the two-dimensional Laplacian is composed of a linear combination of a second-order derivative in the x direction and a second-order derivative in the y direction. Therefore, when the edge component at the point indicated by symbol c among the lattice points as shown in FIG. 4C is obtained by Laplacian, second-order differences are taken in the x direction and the y direction with respect to point c. This is done by linear combination.

First, the second-order difference in the x direction is
(C−b) − (d−c) = 2c−b−d
And the second-order difference in the y direction is
(C−a) − (e−c) = 2c−a−e
In the end,
c edge = | 4c−a−b−d−e |
Can be calculated.

When the point I1 in FIG. 4A is the point c, the luminance equivalent signals at the lattice points a, b, c, d and e are as follows:
a = (G1 + G4)
b = (G3 + G6)
c = (G4 + G6)
d = (G4 + G7)
e = (G6 + G9)
Therefore, if these are substituted, the edge component of the point I1 is
I1 = | 2 (G4 + G6) -G1-G3-G7-G9 |
It can be seen that

In the local region of 5 × 5 pixels as shown in FIG. 4A, the edge components of the points I2, I3, and I4 can be obtained.
I2 = | 2 (G4 + G7) -G2-G5-G6-G9 |
I3 = | 2 (G6 + G9) -G4-G7-G8-G11 |
I4 = | 2 (G7 + G9) -G4-G6-G10-G12 |
It becomes.

  By repeating such processing in 5 × 5 pixel units at intervals of two pixels, it is possible to calculate edge components of the entire screen excluding 2 rows and 2 columns around the screen.

  As can be seen from the equations for calculating the edge components of the points I1, I2, I3, and I4, the quantities in parentheses multiplied by the coefficient 2 are the luminance equivalent signals at the points I1, I2, I3, and I4. It has become. That is, the first signal that is a luminance equivalent signal is obtained in the process of calculating the edge component that is the feature amount.

  The number of operations in such processing is as follows. For each grid point, when the first signal, which is a luminance-corresponding signal, is calculated, addition is performed once, and in the process of calculating the edge component continuously, subtraction is performed four times. Only one multiplication and one logical operation that takes an absolute value are required.

  In addition, since this processing does not require branch processing according to the pixel position as in the case of interpolating single-plate data into three-plate data, a processor or the like that can perform recent multi-stage pipeline processing, etc. It is possible to perform processing with a deep number of stages by effectively using, and the processing speed can be improved.

  The second signal output from the second signal extraction unit 23 is input to the gradation correction unit 12.

  The histogram creation unit 30 of the gradation correction unit 12 receives the edge component output from the second signal extraction unit 23 and the appropriate exposure area information output from the appropriate exposure extraction unit 11 according to the command of the control unit 15. Based on the above, an edge histogram is calculated (step S8).

  Here, the appropriate exposure extraction unit 11 performs an appropriate exposure area related to the long exposure based on the first signal which is a luminance equivalent signal obtained from the image data of the long exposure read from the first image buffer 6a. The other portions are set as appropriate exposure areas for short-time exposure.

  In other words, the appropriate exposure extraction unit 11 compares the signal level of the luminance equivalent signal at the lattice point created by interpolating the G signal as described above with a predetermined value, so that the pixel has appropriate exposure. The set of pixels determined to be appropriate exposure is set as the appropriate exposure area in the long-time exposure image, and the result is output to the histogram creating unit 30.

  In this way, the histogram creation unit 30 extracts the edge by comparing the edge component in the appropriate exposure region with the predetermined threshold value. In the binarization process by comparison with the threshold value at this time, one logical operation is performed for the number of pixels.

  Further, the histogram creation unit 30 creates an edge histogram indicating the appearance frequency with respect to the luminance level of the pixels constituting the edge. In the calculation of the edge histogram, for example, each operation of addition, subtraction 2, division 2, and logic 1 is performed for the number of pixels.

  Thus, when the histogram creation unit 30 finishes scanning all the pixels in the appropriate exposure region and creates an edge histogram (step S9), the histogram creation unit 30 outputs the edge histogram to the conversion curve calculation unit 31.

  The conversion curve calculation unit 31 generates a target histogram by convolving the edge histogram using a Gaussian kernel or the like, and integrates the target histogram and the edge histogram output from the histogram creation unit 30 to accumulate the histogram. Based on these, a tone curve serving as gradation correction characteristics is calculated (step S10).

  The conversion unit 32 performs gradation correction on the image data output from the local region extraction unit 20 based on the tone curve obtained from the conversion curve calculation unit 31 in accordance with a command from the control unit 15, thereby correcting the gradation. The subsequent image data is output to the image composition interpolation unit 13.

  As described above, the image composition interpolation unit 13 receives the image data related to the appropriate exposure area of the long-time exposure subjected to the gradation correction and the image data related to the appropriate exposure area of the short-time exposure corrected in the gradation. One wide dynamic range image is generated by synthesizing and interpolating into a three-plate state.

  The generated wide dynamic range image is then output from the output unit 14.

Thus, in the case of the single-color primary plate CCD 4, the edge histogram is calculated from the image data in total,
(Addition 2, subtraction 6, multiplication 1, division 2, logic 3) x The number of pixels is calculated.
Since (addition 16, subtraction 4, multiplication 7, division 4, logic 6) × the number of operations can be considerably reduced as compared with the number of pixels, the processing time can be reduced and the processing time can be shortened. It becomes possible.

  Furthermore, since the memory for storing the data in the middle of the calculation is only required for one screen for storing the calculated edge component, the conventional example described above requires a memory for five screens. Since the circuit area can be reduced and the substrate area can be reduced by reducing the circuit scale, it is a great advantage when the processing circuit is made into a one-chip IC.

  Further, the processing as described above is not applied only to a single-plate CCD having a primary color filter array, but to a process having a complementary color filter array as shown in FIG. Can be applied similarly. FIG. 5 is a diagram showing a means for calculating an edge component in the complementary color filter CCD.

  That is, the local region extraction unit 20 extracts a local region of 5 × 5 pixels as shown in FIG. 5A, and positions that are lattice points between pixels arranged vertically and horizontally, that is, vertically and horizontally. A luminance equivalent signal at a pixel position shifted by 0.5 pixels is calculated using an interpolation operation as shown in FIG.

  That is, by adding the four pixel data around each lattice point, the luminance equivalent signals for the four lattice points in FIG. + C1 + M4 + Y4), (C1 + G2 + Y4 + M5), etc.

When Laplacian is used, the edge at the lattice point c is as shown in FIG. 5C, which is the same as FIG.
c edge = | 4c−a−b−d−e |
Therefore, the edge components of points I1, I2, I3 and I4 in FIG.
I1 = | 2 (M5 + Y4 + G2 + C1)
-M2-M4-Y1-Y5-G1-G5-C2-C4 |
I2 = | 2 (M5 + Y5 + G2 + C2)
-M2-M6-Y2-Y4-G3-G5-C1-C5 |
I3 = | 2 (M5 + Y4 + G5 + C4)
-M4 -M8 -Y5 -Y7 -G2 -G4 -C1 -C5 |
I4 = | 2 (M5 + Y5 + G5 + C5)
-M6 -M8 -Y4 -Y8 -G2 -G6 -C2 -C4 |
It becomes.

  After that, the luminance equivalent signal is binarized to create a histogram, as described above.

In this way, in the case of the complementary color single-chip CCD 4, the edge histogram is calculated from the image data in total,
(Addition 4, subtraction 10, multiplication 1, division 2, logic 3) × the number of pixels is calculated, and the conventional number of operations as described above,
Since (addition 16, subtraction 4, multiplication 7, division 5, logic 7) × the number of operations can be considerably reduced as compared with the number of pixels, the processing time can be reduced and the processing time can be shortened. It becomes possible.

  Further, since the memory for storing the data in the middle of calculation only needs one screen of memory for storing the calculated edge component, the conventional example described above requires six screens of memory. Since the circuit area can be reduced and the substrate area can be reduced by reducing the circuit scale, it is a great advantage when the processing circuit is made into a one-chip IC.

  According to such an embodiment, since gradation correction can be performed with a smaller number of computations, the processing time can be reduced by reducing the load on the processing circuit. Furthermore, since the memory for storing the data in the middle of the calculation is only required for one screen, the required memory amount can be greatly reduced, and the circuit area can be reduced to reduce the board area. Therefore, this is a great advantage when the processing circuit is made into a one-chip IC.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the spirit of the invention.

1 is a block diagram showing a basic configuration of an electronic camera according to an embodiment of the present invention. The block diagram which shows the detailed structure of the feature-value calculation part of the said embodiment. The block diagram which shows the detailed structure of the gradation correction | amendment part of the said embodiment. The figure which shows the means to calculate the edge component in the primary color system filter CCD of a Bayer arrangement in the said embodiment. The figure which shows the means which calculates the edge component in complementary color system filter CCD in the said embodiment. 6 is a flowchart showing calculation processing of gradation conversion characteristics in the embodiment.

Explanation of symbols

4 ... CCD
6a... First image buffer 6b. Second image buffer 10... Feature amount calculation unit (feature amount calculation means)
11 ... Proper exposure extraction unit (Proper exposure extraction means)
12 ... gradation correction part (gradation correction means)
13: Image composition interpolation unit (composition means)
DESCRIPTION OF SYMBOLS 14 ... Output part 15 ... Control part 20 ... Local area extraction part (local area extraction means)
21 ... Working buffer 22 ... First signal extracting unit (edge calculating means)
23: Second signal extraction unit (edge calculation means)
30 ... Histogram creation section (histogram creation means)
31: Conversion curve calculation unit (tone conversion curve calculation means)
32 ... Conversion unit (conversion means)

Claims (16)

An image processing apparatus that generates a single wide dynamic range image by processing an image group consisting of a plurality of images captured under different exposure conditions for the same subject,
Local region extraction means for extracting a local region of a predetermined size centered on a pixel position outside the integer coordinates for each image in the image group;
From the pixel information in the local region, a first luminance equivalent signal that is a signal corresponding to a luminance signal at a pixel position outside one integer coordinate, and a vertical or horizontal direction from the pixel position outside the one integer coordinate. The second luminance equivalent signal, which is a signal corresponding to the luminance signal at the pixel position outside the integer coordinates existing at the position separated by one pixel, is obtained as an image feature amount, and the obtained first luminance is obtained. Edge calculation means for calculating an edge component at a pixel position outside the one integer coordinate as a feature quantity of an image using a differential filter shown as a difference between the equivalent signal and the second luminance equivalent signal;
Edge calculation repetition means for performing edge component calculation processing by the edge calculation means for each of the extracted local regions and calculating an edge component at a pixel position outside the integer coordinates on the image as a feature amount of the image;
By comparing each of the luminance-corresponding signals obtained by the edge calculating means with a predetermined value, it is determined whether or not the pixel corresponding to the pixel position outside the integer coordinates is appropriate exposure. A proper exposure extraction means for extracting a proper exposure area composed of a set of pixels determined to be present;
Gradation correction means for performing gradation correction of the appropriate exposure area based on the feature amount;
A synthesizing unit that generates a single wide dynamic range image by synthesizing an appropriate exposure area that has been tone-corrected by the tone correcting unit and interpolating into a three-plate state;
An image processing apparatus comprising: The gradation correction means is
For each pixel constituting the appropriate exposure area, by extracting an edge based on a comparison result between the edge component at a pixel position corresponding to the pixel and a predetermined value, a luminance signal of the pixel having the extracted edge; Histogram creation means for creating a histogram showing a correspondence relationship with the appearance frequency of pixels having the luminance signal;
A gradation conversion curve calculating means for calculating a gradation conversion curve based on the histogram;
Conversion means for performing gradation conversion using the gradation conversion curve;
The image processing apparatus according to claim 1, further comprising:   The image processing apparatus according to claim 1, wherein the plurality of images are images obtained using a single-plate image sensor. The plurality of images are images obtained using a single-plate imaging device having a primary color Bayer array filter,
The pixel position outside the integer coordinates is a position shifted by 1/2 pixel in the horizontal direction and the vertical direction for each image in the image group,
The edge calculation unit is configured to output the pixel at one target lattice point as a pixel position outside the one integer coordinate, which is a position shifted by 1/2 pixel from the pixel information in the local region included in the obtained image. The first luminance-corresponding signal and the above-mentioned adjacent lattice points as pixel positions outside the other integer coordinates, which are lattice points that are present in a position separated by one pixel in the vertical or horizontal direction from the one target lattice point. The second luminance equivalent signal is obtained, and based on the obtained first luminance equivalent signal and second luminance equivalent signal, an edge at the one target lattice point as a pixel position outside the one integer coordinate is obtained. The image processing apparatus according to claim 3, wherein a component is calculated.   5. The edge calculation unit obtains the first luminance equivalent signal at the one target lattice point using pixel information of two G components sharing the one target lattice point. An image processing apparatus according to 1.   5. The image according to claim 4, wherein the edge calculating unit obtains the second luminance equivalent signal at the adjacent grid point using pixel information of two G components sharing the adjacent grid point. Processing equipment.   7. The image processing apparatus according to claim 5, wherein each of the two G component pixels is disposed adjacent to each other in an oblique direction.   8. The edge calculation unit obtains the first luminance equivalent signal or the second luminance equivalent signal by adding the pixel information of the two G components. An image processing apparatus according to 1. There are four adjacent grid points with respect to the one target grid point,
The edge calculating means calculates the first luminance equivalent signal at the one target lattice point as c, and calculates the second luminance equivalent signal at each adjacent lattice point corresponding to the one target lattice point as a. , B, d, and e, the edge component at the pixel position outside the one integer coordinate is determined according to the difference between the first luminance equivalent signal and the second luminance equivalent signal. 5. The edge component at the one target lattice point as the pixel position outside the one integer coordinate is calculated by substituting into the equation | 4c−a−b−d−e |. The image processing apparatus according to any one of 8. There are four adjacent grid points with respect to the one target grid point,
The one target lattice point is any one of four lattice points constituting a central pixel in the local region,
The edge calculating means calculates the first luminance equivalent signal at the one target lattice point as c, and calculates the second luminance equivalent signal at each adjacent lattice point corresponding to the one target lattice point as a. , B, d, and e, the edge component at the pixel position outside the one integer coordinate is determined according to the difference between the first luminance equivalent signal and the second luminance equivalent signal. 9. The edge component at the one target lattice point as the pixel position outside the one integer coordinate is calculated by substituting into the equation | 4c−a−b−d−e |. The image processing apparatus described. The plurality of images are images obtained using a single-plate image sensor having a complementary color filter,
The pixel position outside the integer coordinates is a position shifted by 1/2 pixel in the horizontal direction and the vertical direction for each image in the image group,
The edge calculation unit is configured to output the pixel at one target lattice point as a pixel position outside the one integer coordinate, which is a position shifted by 1/2 pixel from the pixel information in the local region included in the obtained image. The first luminance-corresponding signal and the above-mentioned adjacent lattice points as pixel positions outside the other integer coordinates, which are lattice points that are present in a position separated by one pixel in the vertical or horizontal direction from the one target lattice point. The second luminance equivalent signal is obtained, and based on the obtained first luminance equivalent signal and second luminance equivalent signal, an edge at the one target lattice point as a pixel position outside the one integer coordinate is obtained. The image processing apparatus according to claim 3, wherein a component is calculated.   The said edge calculation means calculates | requires the said 1st brightness | luminance equivalent signal in the said 1 attention grid point using the four pixel information which shares the said 1 attention grid point. Image processing device.   The image processing apparatus according to claim 11, wherein the edge calculation unit obtains the second luminance equivalent signal at the adjacent grid point using four pieces of pixel information sharing the adjacent grid point.   14. The edge calculation unit obtains the first luminance equivalent signal or the second luminance equivalent signal by adding pixel information of each of the four pixels. Image processing apparatus. There are four adjacent grid points with respect to the one target grid point,
The edge calculating means calculates the first luminance equivalent signal at the one target lattice point as c, and calculates the second luminance equivalent signal at each adjacent lattice point as a, b, d, and e, respectively. Thereafter, the calculation result is expressed by a mathematical expression | 4c−a−b−d in which an edge component at a pixel position outside the one integer coordinate is determined according to a difference between the first luminance equivalent signal and the second luminance equivalent signal. The image processing according to claim 11, wherein an edge component at the one target lattice point as a pixel position outside the one integer coordinate is calculated by substituting into −e |. apparatus. There are four adjacent grid points with respect to the one target grid point,
The one target lattice point is any one of four lattice points constituting a central pixel in the local region,
The edge calculating means calculates the first luminance equivalent signal at the one target lattice point as c, and calculates the second luminance equivalent signal at each adjacent lattice point corresponding to the one target lattice point as a. , B, d, and e, the edge component at the pixel position outside the one integer coordinate is determined according to the difference between the first luminance equivalent signal and the second luminance equivalent signal. 15. The edge component at the one target lattice point as a pixel position outside the one integer coordinate is calculated by substituting into the equation | 4c-a-bd-e |. The image processing apparatus described.

Images