JP3361957B2 - Method and apparatus for extending the dynamic range of an image - Google Patents

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION The present invention relates to a method and apparatus for extending the dynamic range of an image.

[0002]

2. Description of the Related Art Expanding the dynamic range of an image is indispensable for image processing of an outdoor scene by, for example, a camera, and 10 ³ to 10 ³ can be used to image one object including the sun and shade outdoors. ^It requires a dynamic range of ⁴ orders. However, existing CCD (charge storage device) cameras have a dynamic range of 10 ² and cannot obtain clear images.

A typical prior art is, for example, Japanese Patent Application Laid-Open No. 2-
As disclosed in Japanese Laid-Open Patent Application No. 174470 and Japanese Patent Laid-Open No. 6-105224, an image having a different exposure amount depending on a diaphragm and a shutter speed of a camera for capturing an object or a charge storage time of a CCD (charge storage device) is simply fitted. It has a configuration of complicated composition. As shown in FIG.
In this prior art, when the horizontal axis is set to the coordinates of pixels in the horizontal direction of the image in which the subject is photographed, the exposure amount range P1 is an intermediate exposure amount and the exposure amount range P2 is larger than that.
And the images obtained in the range P3 of the exposure amount smaller than the intermediate exposure amount are the characteristics L1 to L shown in relation to the density value on the vertical axis.
3 is obtained respectively. These characteristics L1 to L3 depend on the amount of exposure and the characteristics of the camera.

In this prior art, there is a problem that pseudo contours 1 and 2 are generated due to a density difference between images by fitting images with large and small exposure amounts into images with intermediate exposure amount.

[0005]

[0006]

[0007]

[0008]

[0009]

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method and apparatus for extending the dynamic range in which the density value changes naturally without the occurrence of pseudo contours in the image.

[0010]

The present invention comprises: (a) a first step of storing in a first memory first image data having a plurality of density values imaged with a certain exposure amount; and (b) a first step. A second step of storing in the second memory second image data having a plurality of density values obtained by imaging the same subject as the first image data with an exposure amount larger than the exposure amount of the image data; A third step of storing, in a third memory, third image data having a plurality of density values obtained by imaging the same subject as the first image data with an exposure amount smaller than the exposure amount of the image data; Within the dynamic range of the density values of the first image data stored in the memory, the density value of the first image data is a predetermined first density value Fa on one side corresponding to the darker one, and the brightness value Is on the other side corresponding to the brighter one It is determined whether or not it is within a specified range between the two density values Fd, and at least a part of the first image data in the first memory is brighter than the second density value Fd in the specified range Fa to Fd. There are one case, a second case in which at least a part of the first image data in the first memory is darker than the first density value Fa in the specified range Fa to Fd, and both the first and second cases. Third case and first case
A fourth step of determining a fourth case in which all of the first image data in the memory exists within a prescribed range Fa to Fd; and (e) for each density value of the first image data stored in the first memory. The fifth step of creating a histogram representing the number of pixels, and (f) the third density value F
a sixth step of defining b as a first density value having a density value greater than or equal to a first number of pixels which is predetermined from the first density value Fa to a bright side within the dynamic range; and (g) a fourth density by the created histogram. Value Fc
Is defined as a first density value having a density value equal to or greater than a second number of pixels determined in advance in the dynamic range from the second density value Fd, and (h) the second case or the fourth step by the fourth step. In the case of 3 cases, the 8th of extracting the coordinate of the pixel of the 1st image data stored in the 1st memory which has the density value of the 1st density value Fa and the 3rd density value Fb determined by the 6th step Step (i) a ninth step of extracting a pixel in the second memory having the same coordinates as the coordinates extracted in the eighth step, and (j) a ninth step, whereby the first and third density values F
a tenth step of obtaining the first and second average values F2a and F2b of the density values of the pixels extracted from the second memory for each a and Fb, and (k) the first density value Fa of the tenth step. First density value differences α2, α2 = Fa-F2a with respect to the first average value F2a are obtained, and further, the gradient (F2 of the density values of the second image data (F2
an eleventh step of obtaining first exposure amount coefficients β2m and β2m = (Fb-Fa) / (F2b-F2a), which are ratios of the gradient (Fb-Fa) of the density value of the first image data to b-F2a), (L) First density value F in the first memory in the ninth step
The density value of the pixel of the second memory having the same coordinates as the coordinates of each pixel having a density value darker than a is F2old.
, The first corrected density value F2 of the corrected image data
new, F2new = Fa + (F2old + α2-Fa) × β2m in the twelfth step, and (m) in the first case or the third case by the fourth step, the second concentration value Fd and the seventh step are determined. A thirteenth step of extracting the coordinates of the pixel stored in the first memory having the fourth density value Fc, and (n) a pixel of the third memory having the same coordinates as the coordinates extracted in the thirteenth step. And the third and fourth average values F1d of the density values of the pixels extracted from the third memory for each of the second and fourth density values Fd, Fc by (14) , F1c respectively, and (p) the second density value difference α1, α1 = F relating to the second density value Fd and the third average value F1d in the fifteenth step. Seeking -F1d, further, the slope of the density value of the third image data (F1
a 16th step of obtaining second exposure amount coefficients β1m, β1m = (Fd−Fc) / (F1d−F1c), which are ratios of the gradient (Fd−Fc) of the density value of the first image data to d−F1c), (Q) Second density value F of the first memory in the fourteenth step
The density value of the pixel of the third memory having the same coordinates as the coordinates of each pixel having a density value lighter than d is F1ol.
The second correction density value F of the corrected image data
1new, F1new = Fd + (F1old + α1-Fd) × β1m in the 17th step, and (r) in the second case or the third case by the fourth step, the first to second density values Fa to in the first memory Image data within the specified range of Fd is directly derived,
Image data whose brightness is darker than the first density value Fa in the first memory is replaced with the first corrected density F2new calculated by the second calculation means to derive the data, and in the case of the first case or the second case, First to second density values Fa to Fd in one memory
The image data within the specified range of is derived as it is, and the image data whose brightness is brighter than the second density value Fd of the first memory is replaced with the second corrected density value F1new calculated by the fourth calculating means. In the case of the fourth case, the image data in the first memory is directly derived.
And a step of expanding the dynamic range of the image.

Further, according to the present invention, (a) a first memory for storing first image data having a plurality of density values imaged at a certain exposure amount, and (b) an exposure amount larger than the exposure amount of the first image data. A second memory for storing second image data having a plurality of density values obtained by imaging the same subject as the first image data, and (c) an exposure amount smaller than the exposure amount of the first image data, A third memory for storing third image data having a plurality of density values obtained by imaging the same subject as the first image data; and (d) within a dynamic range of the density values of the first image data stored in the first memory. Then, the density value of the first image data is a predetermined first density value Fa on one side corresponding to the dark side and a second density value Fd on the other side corresponding to the bright side. To determine if it is within the specified range between A first case in which at least a part of the first image data in the first memory is brighter than the second density value Fd in the specified range Fa to Fd, and at least a part of the first image data in the first memory is a specified range. A second case in which the brightness is darker than the first density value Fa of Fa to Fd, a third case in which both the first and second cases are established, and a first case
Range determining means for determining a fourth case in which all of the first image data in the memory exists within the prescribed range Fa to Fd; and (e) for each density value of the first image data stored in the first memory. A means for creating a histogram representing the number of pixels; and (f) a third density value Fb in response to the output of the histogram creating means, the first density value Fa being greater than or equal to a predetermined number of pixels which is predetermined to be brighter in the dynamic range. A third density value setting means for determining the first density value having the density value of, and (g) a fourth density value Fc in response to the output of the histogram creating means from the second density value Fd to a darker area within the dynamic range. A fourth density value setting means defined as a first density value having a density value greater than or equal to a second number of pixels determined in advance, and (h) in response to the output of the range determination means, in the case of the second case or the third case, 1 First extraction means for extracting the coordinates of the pixel of the first image data stored in the first memory having the density value Fa and the third density value Fb determined by the third density value setting means. (I) second extracting means for extracting pixels in the second memory having the same coordinates as the coordinates extracted by the first extracting means, and (j) first and third densities in response to the output of the second extracting means. First average value calculating means for obtaining first and second average values F2a, F2b of the density values of the pixels extracted from the second memory for each value Fa, Fb; and (k) first average value calculating means. In response to the output of the first density value Fa and the first average value F2a, the first density value difference α2, α2 = Fa−F2a is obtained, and the gradient (F2) of the density value of the second image data (F2
a first calculation means for obtaining first exposure amount coefficients β2m and β2m = (Fb-Fa) / (F2b-F2a), which are ratios of the gradient (Fb-Fa) of the density value of the first image data to b-F2a). , (L) in response to the output of the second extracting means,
The density value of the pixel of the second memory having the same coordinates as the coordinates of each pixel having the density value darker than the density value Fa is F
2old, in response to the output of the first correction density value F2new, F2new = Fa + (F2old + α2-Fa) × β2m of the corrected image data, and (m) the range determination means, In the case of the first case or the third case, the coordinates of the pixel stored in the first memory having the density value of the second density value Fd and the fourth density value Fc determined by the fourth density value setting means are extracted. Third extracting means, (n) fourth extracting means for extracting pixels of the third memory having the same coordinates as the coordinates extracted by the third extracting means, and (o) responding to the output of the fourth extracting means, Second average value calculating means for obtaining the third and fourth average values F1d, F1c of the density values of the pixels extracted from the third memory for each of the second and fourth density values Fd, Fc; 2 average In response to an output of the calculation means, a second density value difference and to a second density value Fd and the third average value F1d [alpha] 1, determine the [alpha] 1 = Fd-F1d, further inclination of the density value of the third image data (F1
a third calculation means for obtaining second exposure amount coefficients β1m and β1m = (Fd-Fc) / (F1d-F1c), which are ratios of the gradient (Fd-Fc) of the density value of the first image data to d-F1c). (Q) in response to the output of the fourth extracting means,
When the density value of the pixel of the third memory having the same coordinates as the pixel having the density value whose brightness is brighter than the density value Fd is F1old, the second correction density values F1new and F1new of the image data after correction are set. = Fd + (F1old + α1-Fd) × β1m, and (r) a substitution unit 23, which responds to the output of the range determination unit and is the first memory in the case of the second case or the third case. The image data within the specified range of the first to second density values Fa to Fd is directly derived, and the image data whose brightness is darker than the first density value Fa of the first memory is calculated by the second calculation means. In the case of the first case or the second case, the image data within the specified range of the first to second density values Fa to Fd in the first memory is directly used as it is. While leaving the image data is brighter brightness than the second density value Fd of the first memory, 4
Second correction density value F1new calculated by the calculation means
In the fourth case, the image data in the first memory is replaced with a replacement unit 23 for deriving the image data as it is.

According to the present invention, a plurality of (for example, three) screens of image data obtained by imaging the same subject with different exposure amounts are used to synthesize a plurality of images with different exposure amounts. It is possible to obtain an image with an expanded dynamic range in which both the density value and the exposure amount ratio are corrected. Density value matching is performed using the image data for the plurality of screens, the exposure ratio is obtained from the image data for the plurality of screens, and density conversion is performed using these. In obtaining the density value α2, for example, one side of the dynamic range of the first image data with an intermediate exposure amount is shifted to one side of the dynamic range (that is, the brightness or illuminance of the subject is low, that is, the brightness is dark, and therefore, the image is captured with a large exposure amount. The density value Fa represented by the image data on the area side) and the image data on the other side of the dynamic range (that is, on the area side obtained with an intermediate exposure amount) of the second image data imaged with the large exposure amount described above are It is obtained by calculating the difference from the density value F2a represented. Further, the exposure amount coefficient β2m, which is the ratio of the exposure amount, is calculated from, for example, the first image data,
Amount of change in density value of two subject parts (Fb-Fa) based on image data on one side of the dynamic range
And the ratio of the dynamic range of the second image data for the same two subject portions to the change amount (F2b−F2a) of the density value based on the image data on the other side. In this way, the level matching of the first and second image data is performed, and the image data on the one side of the dynamic range in the first image data is replaced by correcting and calculating the density value F2old of the second image data. By this calculation, the dynamic range can be further expanded to the one side of the dynamic range of the first image data. In the above description and the following description, the description is mainly made in relation to the image data having the intermediate exposure amount and the image data having the large exposure amount. The same applies to image data with a small exposure amount. That is, in obtaining the density value α1, for example, one side of the dynamic range of the first image data with the intermediate exposure amount (that is, the brightness or illuminance of the subject is high, that is, the brightness is bright, and therefore the image is captured with a small exposure amount. Image data of the density value Fd represented by the image data of the region of interest) and the second image data imaged with the above-described small exposure amount, which is closer to the other side of the dynamic range (that is, the region of the region obtained with an intermediate exposure amount). It is obtained by calculating the difference from the density value F1d represented by. Also, the exposure dose coefficient β1 which is the ratio of the exposure dose
m is the amount of change (Fd−Fc) in the density value of the two subject portions based on the image data of the first image data that is closer to one side of the dynamic range, and is the same for the two subject portions. Is calculated by calculating the ratio of the dynamic range of the second image data to the change amount (F1d−F1c) of the density value based on the image data on the other side. In this way, the levels of the first and second image data are adjusted, and the image data on the one side of the dynamic range in the first image data is replaced by correcting and calculating the density value F1old of the second image data. By this calculation, the dynamic range can be further expanded to the one side of the dynamic range of the first image data.

Although the image data may be monochrome, for example, it may be color image data for each color of red R, green G and blue B.

According to the invention, the first extraction means
For example, on the side corresponding to one of the exposure amounts of the first image data stored in the first memory obtained by imaging with an intermediate exposure amount, for example, on the side of the second image data captured with a large exposure amount. Extracting the coordinates of a pixel in the first memory having a plurality of, for example, two density values Fa, Fb,
In the second extraction means, the pixels of the second memory having the same coordinates as the extracted coordinates of the pixels of the first memory, in other words, the pixels of the second memory in the same subject portion are extracted by the second extraction means, and the second extraction means is used. The density values of the pixels of the second memory having the coordinates obtained by are obtained, and the average values F2a and F2b thereof are obtained. The density value difference α2 is obtained based on the average value F2a corresponding to the density value Fa that is closer to the outside and corresponds to the darker one in the dynamic range.
Further, the average value F corresponding to the density value Fa on the outer side and the density value Fb on the inner side which is brighter than that
The exposure amount coefficient β2m is obtained using 2a and F2b. By using the average values F2a and F2b in this manner, the accuracy of the density value difference α2 and the exposure amount coefficient β2m can be increased, and the error due to the difference in the exposure amount can be reduced. This also applies to the density difference α1 and the exposure amount coefficient β1m.

[0015]

According to the present invention, one of the density values Fa and Fd in the first image data is used to determine the one density value F.
By setting a specified range closer to the inner side of the dynamic range than a and Fd, the density value closest to the one density value Fa, Fd having a density value equal to or more than a predetermined number of pixels is set to the other within the specified range. The predetermined concentration values Fb and Fc are defined as This makes it possible to calculate and obtain the exposure amount coefficients β2m and β1m with high accuracy.

According to the present invention, the first image data is image data obtained by capturing with an intermediate exposure amount, and the second image data is image data obtained by capturing with a large exposure amount. , The third image data is image data obtained by picking up an image with a small exposure amount, so that the dynamic ranges on both sides of the first image data are matched with the density values and the exposure amount ratios are matched, The dynamic range can be expanded to both upper and lower sides.

In the present invention, the first, second and third image data are all color image data for each color of red R, green G and blue B, and the fourth to tenth steps, and The eleventh step of obtaining the first density value difference α2, the thirteenth to fifteenth steps, and the sixteenth step of obtaining the second density value difference α1 are executed for each color, and in the eleventh step, the first exposure is performed. The quantity coefficient β2m is the average value β2R for each color,
It is calculated as an average value of β2G and β2B, and in the twelfth step, the first corrected density value F for each color of red R, green G and blue B is obtained.
2newR2, F2newG2, F2newB2 with respect to the image data of the second memory, the gain kR2 of red R is kR2 =
0.8, green G gain kG2 = 1.0, blue B gain kB2
= 1.0 and output as F2newR2 = F2newR × kR2 F2newG2 = F2newG × kG2 F2newB2 = F2newB × kB2, and in the 16th step, the second exposure amount coefficient β
1m is obtained as an average value of the average values β1R, β1G, β2B for each color, and in the 17th step, the second correction density values F1newR1, F1n for each color of red R, green G and blue B are obtained.
ewG1, F1newB1 with respect to the image data in the third memory, the red R gain kR1 = 1.0, and the green G gain kG.
1 = 1.0 and a gain B of blue B kB1 = 0.8 are set, and F1newR1 = F1newR × kR1 F1newG1 = F1newG × kG1 F1newB1 = F1newB × kB1 is output.

Further, in the present invention, the first, second and third image data are all color image data for each color of red R, green G and blue B, and range determining means and histogram creating means are provided. , Third and fourth density value setting means, and first
~ Fourth extracting means, first and second average value calculating means,
The first calculation means for obtaining the first density value difference α2 and the third calculation means for obtaining the second density value difference α1 operate for each color, and the first calculation means calculates the first exposure amount coefficient β2m. , As the average value of the average values β2R, β2G, β2B for each color,
The second calculation means has a first color temperature correction means, and the first color temperature correction means has first correction density values F2newR2, F2newG2, F2new for each color of red R, green G and blue B.
B2 is set to red R gain kR2 = 0.8, green G gain kG2 = 1.0, and blue B gain kB2 = 1.0 with respect to the image data in the second memory, and F2newR2 = F2newR × kR2 F2newG2 = F2newG × kG2 F2newB2 = F2newB × kB2, and the third calculating means outputs the second exposure amount coefficient β1m.
Is calculated as an average value of the average values β1R, β1G, β2B for each color, and the fourth calculation means has a second color temperature correction means, and the second color temperature correction means has red R, green G and blue B Second correction density values F1newR1, F1newG for each color of
1, F1newB1 with respect to the image data of the third memory, the gain R of red R is kR1 = 1.0, and the gain of green G is kG1 =
The characteristic is that the gain is set to 1.0 and the gain of blue B is set to kB1 = 0.8, and F1newR1 = F1newR × kR1F1newG1 = F1newG × kG1F1newB1 = F1newB × kB1.

According to the present invention, when the first and second image data are color image data, the exposure amount coefficient β2
m and β1m are individual exposure amount coefficients β2R, β2G, and β2B obtained from the image data for each color R, G, and B;
β1R, β1G, β1B are average values, whereas the density value differences α2, α1 are independent values α2R, α2G, α2B independent for each color R, G, B; α1R, α1G, α.
1B, the density value F of the corrected image data for each color
2new and F1new are calculated and obtained. As a result, even if there are variations in the sensitivity difference and the amplification factor of the image of each color R, G, B, the color reproducibility is excellent,
Moreover, it is possible to reduce the influence of the black level difference among the colors R, G, B, and thereby prevent the pseudo contour from occurring.

According to the present invention, the color temperature correction of the density values F2new and F1new of the corrected image data is performed. When the second image data in which the subject is dark and therefore the aperture diameter is large and the charge accumulation time is adjusted to be long to increase the exposure amount is used, since the color temperature of the second image data is low, the red gain is reduced. The blue gain is lowered and corrected. On the contrary, for a bright subject, the exposure amount is adjusted to be small. In this case, since the color temperature of the second image data is high, the blue gain is lower than the red gain.

A CCD (charge storage device) camera (especially a video camera other than a special purpose camera for industrial use, etc.) described later does not sufficiently correct the color temperature, and when a very bright white subject is photographed. There is a bluish tint or redness when shooting a very dark white subject. In order to prevent this, gain is applied to faithfully reproduce the original color regardless of the exposure amount.

According to the present invention, the first and second image data are color image data for each color. In this case, the exposure amount coefficient β2m used for obtaining the density values F2new and F1new of the corrected image data. , Β1m
Is the exposure amount coefficient β2R, β2G, β2 obtained for each color.
B: The average value of β1R, β1G, and β1B. As a result, even if the sensor sensitivity and the amplification factor for each color R, G, B vary, the color reproducibility is excellent and each color R, G, B
It is possible to reduce the influence of the black level difference between B and prevent the pseudo contour from occurring. The density value difference α2 is the density value F2ne of the corrected image data for each color R, G, B.
Used to find w, F1new.

[0024]

[0025]

[0026]

[0027]

According to the present invention, when the first and second image data are color image data, color temperature correction is performed as described above, and the exposure amount in a bright portion and a dark portion in one subject is increased. The difference in color temperature due to the difference in is corrected.

[0029]

[0030]

[0031]

1 is a block diagram showing the overall configuration of an embodiment of the present invention. The camera 7, which is an image pickup means, images the same subject with different exposure amounts to obtain image data for a plurality of screens. This image data is image data for color, and includes image data for each of red R, green G, and blue B. The image data for each color R, G, B from the camera 7 is digitized in the analog / digital conversion circuit 8 and converted into an 8-bit digital signal for each pixel for each color R, G, B. The digitized image data from the analog / digital conversion circuit 8 is supplied to the dynamic range expansion device 9 of the present invention, where density value matching is performed on a plurality of digitized images having different exposure amounts. At the same time, the screen composed of the image data for each color R, G, B is synthesized while correcting the exposure amount coefficients β2m and β1m described later.

In this way, the dynamic range is expanded. The image data expanded by the dynamic range expansion device 9 can be given to the image processing device to perform color analysis and the like. The image data with the expanded dynamic range is also supplied to the dynamic range compression device 10 for each color R, G, B combination, and the dynamic range expanded image of the combined image is displayed, for example, in a color cathode ray tube. Alternatively, its dynamic range is compressed for application to an output device such as a liquid crystal display means.

The camera 7 has a condenser lens 12 for forming an image of a subject on the image pickup device 11. Image sensor 11
Is composed of, for example, a CCD (charge storage element), and a total of three pixels for each color R, G, B constitutes one combination, and a large number of such combinations are arranged in a matrix. Each pixel is provided with a filter 13 for colors R, G, B. In order to set the exposure amount, an exposure amount setting means 14 including a diaphragm and a shutter is provided, and the charge storage time of the image sensor 11 is adjusted. Image sensor 11
The image data of each pixel is given to the analog / digital conversion circuit 8 and digitized as described above.

The CCD camera has a lux of 10 ⁰ to 10 ³
You can shoot only in the range of brightness of the order.

The range of brightness that can be photographed is called the dynamic range of the CCD camera.

Outdoors and indoors, 10 ⁴ to 10 ⁵ (lux) in direct sunlight and 10 ¹ to 10 in dark shadows
³ (lux), and when shooting a subject in direct sunlight and a subject in a dark shadow with a constant exposure amount, the former is filled with white and the latter is only filled with black. Both of these are outside the dynamic range, and the area where no fill is generated is the inside of the dynamic range.

FIG. 2 is a block diagram showing the structure of the dynamic range expansion device 9. The digitized image data conversion from the analog / digital conversion circuit 8 is input for each color R, G, B, and the memories M1 to M3 for each screen.
To the image switching circuit 15. Memory M1
Stores image data for one screen imaged with an intermediate exposure amount. Image data captured with a large exposure amount is stored in the memory M2. Image data taken with a small exposure amount is stored in the memory M3. In FIG. 2, solid lines represent image data for each color R, G, B, and broken lines represent signals for control other than image data. The density value of the image data having the intermediate exposure amount stored in the memory M1 is defined by a predetermined density value Fa on the dark side of the dynamic range and another predetermined density value Fd on the bright side. The determination circuit 16 determines whether or not the image data exists outside the range. When at least a part of the image data stored in the memory M1 exists outside the specified range, the arithmetic units 17 and 18 calculate the density value differences α2 and α1 by the arithmetic circuits 19 and 20, and the exposure amount coefficient β2m,
β1m is calculated by calculating circuits 21 and 22.

If all the image data stored in the memory M1 is within the specified range, the image data is
It is directly passed through the replacement circuit 23 without being calculated, and is supplied to the image processing device and the dynamic range compression device 10. When the determination circuit 16 determines that the image data exists outside the specified range, each of the arithmetic units 17 and 18 performs the above-mentioned arithmetic operation using the image data of the memory M1 and the image data of the memories M2 and M3. . Thus, the density value differences α2, α1 and the exposure amount coefficients β2m, β1m obtained by the arithmetic circuits 17, 18
Among the image data stored in the memory M1 and supplied to the density conversion circuits 24 and 25, the density value of a pixel having a density value outside the specified range is converted into corrected image data to obtain a dynamic range. Expand. The new corrected image data thus obtained is used in the color temperature correction circuit 2
The correction calculation is performed in 6 and 27, and is given to the replacement circuit 23. Therefore, the image data within the specified range is stored in the memory M1.
The image data stored in is used as it is, and the image data out of the specified range is replaced with the image data given from the color temperature correction circuits 26 and 27, thus the image in which the dynamic range of one screen is expanded. Data is derived.

FIG. 3 is a diagram showing the relationship between the dynamic range of the image data from the camera 7 and the density value of the output image data when the exposure amount is changed. The horizontal axis of FIG. 3 represents the coordinates of pixels in the horizontal direction of the subject. This subject is configured for the convenience of description such that the brightness or illuminance of the subject increases in the horizontal direction from the left to the right, and the vertical axis in FIG. The scale is graduated from dark to light. The vertical axis of FIG. 3 is
The density value of the image data for each color R, G, B is shown, and the reference numeral DR shows the dynamic range of the camera 7. Image data captured with an intermediate exposure amount has the characteristic indicated by L4, image data captured with a large exposure amount has the characteristic indicated by line L5, and an image captured with a smaller exposure amount The data has the characteristic shown by line L6. It can be seen that even if the brightness of the subject is the same, the density value of the image data differs depending on the value of the exposure amount, and the inclination of the density value also differs.

Luminance represents the intensity of light when the subject is luminous, and illuminance represents the intensity of light when the subject is not luminous but has brightness due to illumination. Generally called brightness. In the Munsell color system, the lightness is one of the three attributes together with the hue and the saturation for defining a color, and is a concept different from the brightness, illuminance and brightness referred to in the present invention.

FIG. 4 is a diagram for explaining a subject. FIG. 4 (1) shows a subject. As described above, the subject is configured so that the brightness or the illuminance smoothly changes from dark to bright as it moves from left to right in the horizontal direction. In the vertical direction of the subject in FIG. 4A, the brightness of the subject is the same. FIG. 4B shows the brightness corresponding to the coordinates of the horizontal pixels of the subject shown in FIG. Figure 4
The mutual interval ΔB1 between the vertical lines in (1) corresponds to the brightness of the subject, and indicates that the brightness increases as ΔB1 increases.

FIG. 5 is a diagram for explaining the operation when the camera 7 takes an image with an intermediate exposure amount, and FIG. 6 is a diagram for explaining the operation when the camera 7 takes an image with a large exposure amount. FIG. 7 is a diagram for explaining the operation when the camera 7 takes an image with a small exposure amount. Image data of the subject obtained by the large and small of these exposure conditions are shown in FIGS. 5 (1), 6 (1) and 7 (1), and are stored in the memories M1, M2 and M3 described above. The density values shown in FIGS. 5 (2), 6 (2) and 7 (2) are stored.

With reference to FIG. 5 (2), in the memory M1, the density values obtained by the camera 7 at the positions A and D of the coordinates of the pixels in the horizontal direction of the subject indicated by the horizontal axis are the reference numerals Fa and Fd. The density values Fa to Fd are within the specified range. The density value of the dynamic range DR is set to a total of 256 gradations of 0 to 255 in this embodiment, for example. Concentration value less than Fa and F
Outside the specified range exceeding d, according to the present invention, the memory M2
6 (2) using the image data imaged under the large exposure amount condition, and the memory M3 using the image data imaged under the small exposure amount condition illustrated in FIG. 7 (2). , Are corrected respectively. FIG. 5 (1),
The interval ΔB2 between the vertical lines in FIGS. 6 (1) and 7 (1) indicates the density value, and the larger the interval ΔB2, the more
Indicates that the density value is large, that is, bright.

The memory M1 has a density value f1 for each of the pixel coordinates (x1, y1) to (xm, yn) for each color R, G, B.
~ Fr are stored correspondingly, which means that another memory M
The same applies to 2 and M3.

[0045]

[Table 1]

FIG. 8 is a histogram obtained according to the invention for image data of one color, for example red R, in memory M1. The horizontal axis of FIG. 8 represents the density value,
The vertical axis of FIG. 8 represents the number of pixels, that is, the frequency.

FIG. 9 is a block diagram showing a simplified operation of the entire dynamic range expansion device 9 shown in FIG. The dynamic range extending device 9 performs the first operation 28 on the image data stored in the memory M1, performs the second operation 29 on the image data stored in the memory M2, and stores the image data in the memory M3. The third operation 30 is performed on the image data being displayed. Specific steps a1 to a9; a10 to a17; a20 to a of the first to third operations 28 to 30.
27 is shown in FIGS. 10 to 12, respectively.

Of the exposure amount imaged by the camera 7,
The large and small image data are stored in the memory M as described above.
Stored in 1, M2, M3. First, the image data of the memory M1 for each color R, G, B is read in step a1, and it is determined in step a2 whether or not the image data is within a specified range of density value Fa or more and density value Fd or less. The determination circuit 16 makes a determination. All of the image data in the memory M1 exists within the specified range Fa to Fd,
When it is determined that it does not exist outside the specified range,
In step a3, the operation of expanding the dynamic range is not performed, and the operation ends.

When it is determined in step a4 that at least a part of the image data in the memory M1 is out of the specified range, in the next step a5, the image data outside the specified range is classified into cases 1 to 3. . In case 1, at least a part of the image data in the memory M1 is within the specified range Fa to
This is the case when the Fd exceeds the upper limit Fd. Case 2
In this case, at least a part of the image data in the memory M1 exists below the lower limit density value Fa. Case 3 is a case in which both Cases 1 and 2 above are established, and
That is, at least a part of the image data in the memory M1 exists above the density value Fd and also exists below the density value Fa.

In step a6, the above-mentioned histogram described with reference to FIG. 8 is created for the image data in the memory M1. This histogram represents the frequency, which is the number of pixels for each density value of the image data in the memory M1.

In step a7, with respect to the image data in the memory M1, in the case of Case 1 or Case 3, the density value Fc which is equal to or lower than the density value Fd at the upper limit of the specified range and is equal to or higher than a predetermined specified frequency Q is displayed in the histogram. Search using. That is, Fc is set as the first density value having a density value equal to or more than the predetermined number of pixels Q in the dynamic range (left side in FIG. 8) from the density value Fd.

In step a8, with respect to the image data in the memory M1, a density value Fb having a specified frequency Q or more and a density value Fa at the lower limit of the specified range or more is searched. This is performed in case 2 or 3 above. This density value Fb is the density value F
It is determined as the first density value having a density value equal to or more than the predetermined number of pixels Q from a to the inside of the dynamic range (on the right side of FIG. 8). The specified frequency Q of each concentration value Fc, Fb is
They may be different from each other. Thus, the density values Fa, Fd
The density values Fc and Fb are determined based on the specified frequency Q and the specified frequency Q.

At step a9, the coordinates of the pixels having the density values Fa to Fd obtained in each of the cases 1 to 3 are calculated as
It is extracted based on the above-mentioned Table 1.

In step a9, coordinates having density values less than Fa and more than Fd are also extracted based on the image data in the memory M1.

In case 2 or 3 described above, FIG.
Further, the arithmetic processing is performed based on the stored contents of the memory M2 in which the image data captured with the large exposure amount in 1 is stored. This FIG. 11 shows the image memory M2.
4 is a flowchart for explaining a processing operation of image data stored in the storage device. At step a10, of the image data stored in the memory M1 captured with an intermediate exposure amount, the memory M2 having the same coordinates as the pixel of the memory M1 having the density values Fa and Fb obtained at step a9 described above. The coordinates in the coordinates are extracted and the density value in the memory M2 is read. This extraction result is shown in Table 2.
It is as follows. The coordinates of M2 in Table 2 are
Memory M having density values Fa and Fb in memory M1
It is the coordinate of the pixel of 2.

[0056]

[Table 2]

At step a11, the respective density values Fa, Fb
For each time, the average values F2a and F2b of the density values of the pixels extracted from the memory M2 are calculated and calculated based on the equations 1 and 2.

[0058]

[Equation 1]

Fi is a memory M2 having the same coordinates as the coordinates having the density value Fa of the image data in the memory M1.
, And the total number of pixels is q. Further, fj is the density value in the memory M2 having the same coordinates as the pixel having the density value Fb of the image data in the memory M1, and r is the total number of the pixels.

At step a12, the density value difference α2 is calculated and obtained. α2 = Fa−F2a (3) At step a13, the exposure amount coefficient β2 is calculated and obtained. β2 = (Fb−Fa) / (F2b−F2a) (4) The above steps a1 to a13 are performed for each color R, G, B of the image data.

At step a14, an average value β2m of the average values β2R, β2G and β2B of each color R, G and B obtained by the equation 4 is calculated and obtained. β2m = (β2R + β2G + β2B) / 3 (5) At step a15, the density value of the pixel of the image data in the memory M2 having the same coordinates as the pixel of the image data of the memory M1 having the density value less than the density value Fa. The density value F2new of the corrected image data is calculated based on Expression 6 using F2old and the calculated density value difference α2 and the exposure amount coefficient β2m. F2new = Fa + (F2old + [alpha] 2-Fa) * [beta] 2m (6) The arithmetic unit 17 executes steps a5 to a14 described above, and the density conversion circuit 24 executes the calculation of step a15 described above. In step a16, the color temperature correction circuit 26 calculates the color temperature of the density value F2new of the corrected image data obtained by the equation 6.

In order to correct the color temperature, the memories provided in the color temperature correction circuit 26 include the memories M2 and M3.
Corresponding to the image data of each color, as shown in Table 3, each color R, G, B
The gains kR, kG, kB are set for each.

[0063]

[Table 3]

The image data stored in the memory M2 is image data obtained by picking up an image with a large amount of exposure because the subject is dark. In this case, since the color temperature is low, the gain kR2 of red R is obtained. Other colors G, B gain kG
2, kB2 is set to a smaller value than kB2, and, for example, as shown in Table 3, kR2 = 0.8 and kG2 = kB2 = 1.0. The density value F2 for each color thus color temperature corrected
newR2, F2newG2, F2newB2 is expressed by Equation 7
~ 9.

F2newR2 = F2newR × kR2 (7) F2newG2 = F2newG × kG2 (8) F2newB2 = F2newB × kB2 (9) In step a17, the memory M is used in the replacement circuit 23.
The image data within the specified range of the density values Fa to Fd of 1 is derived as it is without replacement, and for the image data less than the density value Fa, the image data of the memory M2 is calculated as described above, and the above formula is used. The density value of the corrected image data for each color obtained in 7 to 9 is derived. In this way, the dynamic range is expanded to the one with the larger exposure amount.

In the case of case 1 or 3 described above, FIG.
The operation shown in 2 is performed. FIG. 12 is a flow chart for explaining the operation in case 1 or 3. Steps a20 to a27 in FIG. 12 correspond to steps a10 to a17 in FIG. 8 described above, and the image data stored in the memory M3 of the subject imaged with a small exposure amount is used.

In step a20, the density values Fd and Fc obtained in step a9 are selected from the image data stored in the memory M1 which has been imaged with an intermediate exposure amount.
Memory M having the same coordinates as the pixels of the memory M1 having
The coordinates at coordinates 3 are extracted, and the density value in the memory M3 is read in the same manner as in Table 2 above.

At step a21, the density values Fc, Fd
The average values F1d and F1c of the density values of the pixels extracted from the memory M3 are calculated for each time based on Expressions 1 and 2.

[0069]

[Equation 2]

Fk is a memory M3 having the same coordinates as the coordinates having the density value Fd of the image data in the memory M1.
, And the total number of pixels is u. Further, fh is the density value in the memory M3 having the same coordinates as the pixel having the density value Fc of the image data in the memory M1, and v is the total number of the pixels. At step a22, the density value difference α1 is calculated and obtained. α1 = Fd−F1d (12) In step a23, the exposure amount coefficient β1 is calculated and obtained. β1 = (Fd−Fc) / (F1d−F1c) (13) Regarding the above steps a20 to a23,
This is performed for each color R, G, B of the image data. Step a24
Then, the average value β1 for each color R, G, B obtained by Equation 13
The average value β1m of R, β1G, and β1B is calculated and obtained. β1m = (β1R + β1G + β1B) / 3 (14) At step a25, the density value of the pixel of the image data in the memory M3 having the same coordinates as the pixel of the image data of the memory M1 having the density value less than the density value Fa. Using F1old, the density value F1new of the corrected image data is calculated and calculated based on Expression 15 using the calculated density value difference α1 and the exposure amount coefficient β1m. F1new = Fd + (F1old + [alpha] 1-Fd) * [beta] 1m (15) In the arithmetic unit 17, the steps a20 to a24 described above are executed, and the density conversion circuit 24 executes the calculation in step a25 described above. In step a26, the color temperature correction circuit 26 performs the color temperature correction calculation of the density value F1new of the corrected image data obtained by Expression 15. Memory M3
The image data stored in the image data is image data obtained by picking up a bright subject and therefore with a small exposure amount. In this case, since the color temperature is high, the gain kB1 for blue B is set to another color R. , G gains kR1 and kG1 are set to be smaller, and for example, kB1 =
Set 0.8 and kR1 = kG1 = 1.0. The density value F1new for each color R, G, B whose color temperature has been corrected in this way
R1, F1newG1, and F1newB1 are obtained by Expressions 16 to 18. F1newR1 = F1newR × kR1 (16) F1newG1 = F1newG × kG1 (17) F1newB1 = F1newB × kB1 (18) In step a27, the memory M is used in the replacement circuit 23.
The image data within the specified range of the density values Fa to Fd of 1 is derived as it is without replacement, and for the image data exceeding the density value Fd, the image data of the memory M3 is calculated as described above, and the above formula is used. The density value of the corrected image data for each color obtained in 16 to 18 is derived. In this way, the dynamic range is expanded to the one with the larger exposure amount.

FIG. 13 shows a dynamic range expansion device 9
It is a figure for demonstrating the image data obtained by. The density value in the specified range of density values Fa to Fd in the area P1 of the image data obtained with the intermediate exposure amount and the slope of the density value are the areas P2 and
In P3, the density matching and the correction calculation using the exposure amount coefficient are performed, and in this way, the dynamic range in which the characteristics L5 and L6 are continuous in a straight line to the characteristic L4 is achieved. In the above-described embodiment, the dynamic range DR
Has a density value of 0 to 255, and Fa = 1, for example.
It may be 0, Fb = 20, Fc = 230, and Fd = 240.

When the densities are quantized from 0 to 255, Fa is the low density value (for example, density value 10) of the image taken with the intermediate exposure amount, and Fb is the intermediate exposure amount. The density value is slightly higher than Fa of the captured image (for example, density value 20), and Fc is a density value slightly lower than Fd of the image captured at an intermediate exposure amount (for example, density value 230). Yes, Fd is a high density value (for example, a density value of 240) of an image taken with an intermediate exposure amount.
Is.

If there is not a sufficient number of pixels between Fa and Fb, Fb is changed to a higher density value and a predetermined number of samples (for example, 10% of the total number of pixels of an image taken with an intermediate exposure amount) is used. ) Is repeated.

If there is not a sufficient number of pixels between Fc and Fd, Fc is changed to a lower density value and a predetermined number of samples (for example, 10% of the total number of pixels of an image taken with an intermediate exposure amount) is used. ) Is repeated.

The meanings of β2m and β1m will be supplementarily described. With respect to L4, L5, and L6 in FIG. 3 in which the exposure amounts are different, the ratio of the amount of change in the density due to the difference in the exposure amount is used by using the same subject portion of the image in which the exposure amount is different, with reference to L4 having the intermediate exposure amount. Is to calculate. Of course, in addition to such a configuration, it is possible to calculate using the aperture and storage time, but the precision of the aperture and the linearity of the image brightness with respect to the exposure amount of each CCD camera vary depending on the manufacturer and model. Therefore, in this embodiment, first-order linear approximation is performed from the captured image.

In the dynamic range compression device 10 to which the corrected image data from the dynamic range expansion device 9 is given, for example, when the dynamic range in a cathode ray tube or a liquid crystal display device is narrow as indicated by reference numeral 31 in FIG. , So that the relative relationship of brightness is maintained, and even if it is a wide dynamic range image that is biased toward the dark side or the bright side as a whole, an easy-to-see image converted to an appropriate level should be obtained. You can In the other embodiment of the present invention, the range 31 of the dynamic range is, for example, 0 to 0 like the camera 7.
It may be in the range of 255 density values.

[0077]

According to the present invention, by being able to extend the dynamic range, the present invention can be implemented in connection with, for example, digital still cameras and video cameras, and has a wide dynamic range such as outdoor scenes under clear weather. You can shoot when you need the desired subject, and without distinction of forward light and backlight,
Shooting becomes possible. Moreover, according to the present invention, since a pseudo contour hardly occurs while having a wide dynamic range, the present invention can be advantageously applied to image processing such as contour line extraction. Further, when the image data is image data for color, there is almost no color misregistration, and therefore color analysis can be performed. Further, according to the present invention, the pseudo-contour generation and the unnatural change in the density value due to the shooting condition, the characteristics of the camera for shooting, the analog / digital conversion characteristics, etc. A variety of optical systems, which are not designed for, and thus not specifically designed for, or characterized
The present invention is also applicable to a combination of a camera and an analog / digital converter of a video signal, and achieves an excellent effect that the present invention can be implemented in general. Moreover, according to the present invention, even if the difference between the bright portion and the dark portion of the image data is large, the change in the density value at each portion can be expressed, and therefore the appearance with the gradation change in the density can be expressed. It becomes possible to output as a natural image.

According to the present invention, for the density values F2old and F1old of the second image data, the density value differences α2 and α1 are obtained.
Also, the density values F2new and F1new of the corrected image data can be obtained by calculation from the exposure amount coefficients β2m and β1m.

According to the present invention, the other density values Fb, Fc within the dynamic range than the one density values Fa, Fd for setting the specified range of the first image data are equal to or more than the predetermined number of pixels. Therefore, the exposure amount coefficient β
The calculation of 2m and β1m can be performed with high accuracy. According to the present invention, for example, the dynamic range of the first image obtained with an intermediate exposure amount is corrected above and below the dynamic range of the first image using the second and third image data with a large exposure amount and a small exposure amount, respectively, and the density value is adjusted. By using the exposure amount ratio while performing the above, density conversion can be performed and the dynamic range can be sufficiently expanded. According to the present invention, when the image data is color image data,
The exposure amount coefficients β2m and β1m are the exposure amount coefficient β2 for each color.
R, β2G, β2B; average values of β1R, β1G, β1B, and thus excellent color reproducibility even when there are variations in sensor sensitivity difference and amplification factor for each color R, G, B, It is possible to reduce the influence of the black level difference among the colors R, G, and B and prevent the pseudo contour from occurring. In this case, the density value difference is independently and individually used for each color R, G, B.

According to the present invention, the corrected image data F2
Since the color temperatures of the new and F1new are corrected according to the magnitude of the exposure amount of the second image data, it is possible to prevent the image data having a large exposure amount and thus a low color temperature from appearing reddish. It is possible to prevent image data having a small exposure amount and thus a high color temperature from appearing bluish.

According to the present invention, the second value used for calculating the density value differences α2, α1 and the exposure amount coefficients β2m, β1m.
The average values F2a and F2b of the image data; F1d and F1
By using c, it is possible to calculate and obtain a highly accurate value with a small error, and therefore it is possible to obtain the density values F2new and F1new of the corrected image data with high accuracy.

According to the present invention, as described above, the exposure amount coefficients β2m, β1m in the color image data are the exposure amount coefficients β2R, β2G, β2B; β1R, β1 for each color.
Since it is the average value of G and β1B, it is possible to reduce errors as described above and perform highly accurate calculation.

[0083]

[0084]

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a dynamic range expansion device 9.

FIG. 3 is a diagram showing the relationship between the brightness of a subject and the density value of output image data when the exposure amount of image data by the camera 7 is changed.

FIG. 4 is a diagram illustrating a subject used to describe an embodiment of the present invention.

5A and 5B are diagrams for explaining image data imaged with a medium exposure amount when the image of the subject in FIG.

FIG. 6 is a diagram for explaining image data captured with a large exposure amount when the subject of FIG. 4 (1) is captured by the camera.

FIG. 7 is a diagram for explaining image data captured with a small exposure amount when the camera of FIG. 4 (1) images the subject.

FIG. 8 is a histogram obtained according to the present invention.

FIG. 9 is a dynamic range extending device 9 shown in FIG.
3 is a block diagram showing the entire operation of FIG.

FIG. 10 is a flowchart specifically showing an operation 28 in FIG.

FIG. 11 is a flowchart specifically showing an operation 29 of FIG.

FIG. 12 is a flowchart showing the operation 30 of FIG. 9 in detail.

FIG. 13 is a diagram for explaining image data obtained by the dynamic range expansion device 9.

FIG. 14 is a diagram showing a dynamic range extension result according to the first prior art.

FIG. 15 is a diagram showing a result of dynamic range extension according to the second prior art.

FIG. 16 is a diagram showing a result of dynamic range extension according to the third prior art.

[Explanation of symbols]

7 camera 8 analog / digital conversion circuit 9 Dynamic range extender 10 Dynamic range compressor 11 Image sensor 12 Condensing lens 15 Image switching circuit 17, 18 arithmetic circuit 23 Replacement circuit 24,25 density conversion circuit 26,27 Color temperature correction circuit M1, M2, M3 memory

─────────────────────────────────────────────────── ─── Continuation of front page (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N 5/222-5/257 H04N 5/30-5/335 H04N 5/58

Claims (4)

(57) [Claims] 1. A first step of storing, in a first memory, first image data having a plurality of density values imaged with a certain exposure amount, and (b) larger than the exposure amount of the first image data. A second step of storing in the second memory second image data having a plurality of density values obtained by imaging the same subject as the first image data, and (c) less than the exposure amount of the first image data. A third step of storing in the third memory the third image data having a plurality of density values obtained by imaging the same subject as the first image data with the exposure amount of (d) the first stored in the first memory. Within the dynamic range of the density value of the image data, the density value of the first image data is a predetermined first density value Fa on one side corresponding to the darker brightness and the other one corresponding to the lighter brightness. Specification between the second density value Fd on the side It is determined whether the brightness is within the range, and at least a part of the first image data in the first memory is brighter than the second density value Fd in the specified range Fa to Fd. At least a part of one image data is darker than the first density value Fa in the specified range Fa to Fd
The case, the third case in which both the first and second cases are established, and all of the first image data in the first memory are within the specified range Fa to F.
a fourth step of determining a fourth case existing within d, and (e) a fifth step of creating a histogram representing the number of pixels for each density value of the first image data stored in the first memory, (F) From the created histogram, the third density value F
a sixth step of defining b as a first density value having a density value greater than or equal to a first number of pixels which is predetermined from the first density value Fa to a bright side within the dynamic range; and (g) a fourth density by the created histogram. Value Fc
Is defined as a first density value having a density value equal to or greater than a second number of pixels determined in advance in the dynamic range from the second density value Fd, and (h) the second case or the fourth step by the fourth step. In the case of 3 cases, the 8th of extracting the coordinate of the pixel of the 1st image data stored in the 1st memory which has the density value of the 1st density value Fa and the 3rd density value Fb determined by the 6th step Step (i) a ninth step of extracting a pixel in the second memory having the same coordinates as the coordinates extracted in the eighth step, and (j) a ninth step, whereby the first and third density values F
a tenth step of obtaining the first and second average values F2a and F2b of the density values of the pixels extracted from the second memory for each a and Fb, and (k) the first density value Fa of the tenth step. First density value differences α2, α2 = Fa-F2a with respect to the first average value F2a are obtained, and further, a gradient (Fb) of the density value of the first image data with respect to a gradient (F2b-F2a) of the density value of the second image data. -Fa) first exposure amount coefficient β2m, β2m = (Fb-Fa) / (F2b-F2a) in the 11th step, and (L) in the 9th step, the first density value in the first memory F
The density value of the pixel of the second memory having the same coordinates as the coordinates of each pixel having a density value darker than a is F2old.
, The first corrected density value F2 of the corrected image data
new, F2new = Fa + (F2old + α2-Fa) × β2m in the twelfth step, and (m) in the first case or the third case by the fourth step, the second concentration value Fd and the seventh step are determined. A thirteenth step of extracting the coordinates of the pixel stored in the first memory having the fourth density value Fc, and (n) a pixel of the third memory having the same coordinates as the coordinates extracted in the thirteenth step. And the third and fourth average values F1d of the density values of the pixels extracted from the third memory for each of the second and fourth density values Fd, Fc by (14) , F1c respectively, and (p) the second density value difference α1, α1 = F relating to the second density value Fd and the third average value F1d in the fifteenth step. -F1d is obtained, and the second exposure amount coefficients β1m and β1m = (1d), which are the ratios of the gradient (Fd-Fc) of the density value of the first image data to the gradient (F1d-F1c) of the density value of the third image data. Fd−Fc) / (F1d−F1c) in the 16th step, and (q) the second density value F in the first memory in the 14th step.
The density value of the pixel of the third memory having the same coordinates as the coordinates of each pixel having a density value lighter than d is F1ol.
The second correction density value F of the corrected image data
1new, F1new = Fd + (F1old + α1-Fd) × β1m in the 17th step, and (r) in the second case or the third case by the fourth step, the first to second density values Fa to in the first memory The image data within the specified range of Fd is derived as it is, and the image data whose brightness is darker than the first density value Fa of the first memory is replaced with the first correction density F2new calculated by the second calculation means. The first case of the first memory in the case of the first case or the second case.
~ Image data within the specified range of the second density value Fa ~ Fd,
In the case of the fourth case, the image data, which is derived as it is and whose brightness is brighter than the second density value Fd in the first memory, is replaced with the second corrected density value F1new calculated by the fourth calculation means. And a step of deriving the image data of the first memory as it is, a method for extending a dynamic range of an image. 2. The first, second and third image data are color image data for each color of red R, green G and blue B, and the fourth to tenth steps and the first density value The eleventh step of obtaining the difference α2, the thirteenth to fifteenth steps, and the sixteenth step of obtaining the second density value difference α1 are performed for each color.
In the eleventh step, the first exposure amount coefficient β2m is obtained as the average value of the average values β2R, β2G, and β2B of each color, and in the twelfth step, the first exposure amount coefficient β2m of each color of red R, green G, and blue B is calculated. 1 correction density value F2newR2, F2newG2, F2
newB2 is the gain kR2 = 0.8 of red R and the gain kG2 = 1.
0, the gain of blue B is set to kB2 = 1.0, F2newR2 = F2newR × kR2 F2newG2 = F2newG × kG2F2newB2 = F2newB × kB2 is output, and in the 16th step, each second exposure amount coefficient β1m is output. Is calculated as an average value of β1R, β1G, and β2B, and in the 17th step, the second correction density values F1newR1, F1newG1, and F1 for each color of red R, green G, and blue B are calculated.
For the image data in the third memory, newB1 is a gain of red R kR1 = 1.0, a gain of green G kG1 = 1.
The dynamic range of the image according to claim 1 is expanded by setting the gain kB1 of 0 and blue B to 0.8, and outputting as F1newR1 = F1newR × kR1F1newG1 = F1newG × kG1F1newB1 = F1newB × kB1. how to. 3. A first memory for storing first image data having a plurality of density values imaged with a certain exposure amount, and (b) an exposure amount larger than the exposure amount of the first image data. A second memory for storing second image data having a plurality of density values obtained by imaging the same subject as the first image data; and (c) a first image with an exposure amount smaller than that of the first image data. A third memory that stores third image data having a plurality of density values obtained by imaging the same subject as the data; and (d) within a dynamic range of the density values of the first image data stored in the first memory, The density value of one image data is between a predetermined first density value Fa on one side corresponding to the dark side and a second density value Fd on the other side corresponding to the bright side. It is judged whether it is within the specified range, and the first memory A first case in which at least a part of the first image data is brighter than the second density value Fd in the specified range Fa to Fd, and at least a part of the first image data in the first memory is in the specified range Fa to Fd. The second darker than the first density value Fa
The case, the third case in which both the first and second cases are established, and all of the first image data in the first memory are within the specified range Fa to F.
range determining means for determining the fourth case existing within d, (e) means for creating a histogram representing the number of pixels for each density value of the first image data stored in the first memory, and (f) ) In response to the output of the histogram creating means, the third density value Fb is determined as the first density value having a density value greater than or equal to the first pixel number, which is determined in advance from the first density value Fa to the bright side within the dynamic range. In response to the output of the density value setting means and (g) the histogram creating means, the fourth density value Fc has a density value equal to or more than a predetermined second pixel number in the dark side within the dynamic range from the second density value Fd. In the case of the second case or the third case, the first density value Fa and the third density value setting means are used in response to the output of the fourth density value setting means defined as the first density value and (h) the range determining means. Fixed First extracting means for extracting the coordinates of the pixel of the first image data stored in the first memory having the density value of the determined third density value Fb; and (i) the coordinates extracted by the first extracting means. Second extracting means for extracting pixels of the second memory having the same coordinates as, and (j) extracting from the second memory for each of the first and third density values Fa and Fb in response to the output of the second extracting means. First average value calculating means for obtaining first and second average values F2a, F2b of the density values of the selected pixels, and (k) a first density value Fa in response to the output of the first average value calculating means. First density value differences α2, α2 = Fa-F2a with respect to the first average value F2a are obtained, and further, a gradient (Fb) of the density value of the first image data with respect to a gradient (F2b-F2a) of the density value of the second image data. -Fa) ratio of the first exposure amount coefficient β2 , Β2m = (Fb-Fa) / a first computing means for obtaining (F2b-F2a), (L) in response to an output of the second extraction means, the first first memory
The density value of the pixel of the second memory having the same coordinates as the coordinates of each pixel having the density value darker than the density value Fa is F
2old, in response to the output of the first correction density value F2new, F2new = Fa + (F2old + α2-Fa) × β2m of the corrected image data, and (m) the range determining means, In the case of the first case or the third case, the coordinates of the pixel stored in the first memory having the density value of the second density value Fd and the fourth density value Fc determined by the fourth density value setting means are extracted. Third extracting means, (n) fourth extracting means for extracting pixels of the third memory having the same coordinates as the coordinates extracted by the third extracting means, and (o) responding to the output of the fourth extracting means, Second average value calculating means for obtaining the third and fourth average values F1d, F1c of the density values of the pixels extracted from the third memory for each of the second and fourth density values Fd, Fc; 2 average In response to the output of the calculation means, the second density value difference α1, α1 = Fd−F1d between the second density value Fd and the third average value F1d is obtained, and further, the gradient of the density value of the third image data (F1d− A third calculation means for obtaining second exposure amount coefficients β1m and β1m = (Fd-Fc) / (F1d-F1c), which are ratios of the gradient (Fd-Fc) of the density value of the first image data with respect to (F1c). q) in response to the output of the fourth extracting means,
When the density value of the pixel of the third memory having the same coordinates as the pixel having the density value whose brightness is brighter than the density value Fd is F1old, the second correction density values F1new and F1new of the corrected image data are obtained. = Fd + (F1old + α1-Fd) × β1m, and (r) a replacement unit 23, which is responsive to the output of the range determination unit, and is the first memory in the case of the second case or the third case. First of
~ Image data within the specified range of the second density value Fa ~ Fd,
In the first case or the second case, the image data, which is derived as it is and whose brightness is darker than the first density value Fa in the first memory, is replaced with the first corrected density F2new calculated by the second calculation means. In case, the first of the first memory
~ Image data within the specified range of the second density value Fa ~ Fd,
In the case of the fourth case, the image data, which is derived as it is and whose brightness is brighter than the second density value Fd in the first memory, is replaced with the second corrected density value F1new calculated by the fourth calculation means. An apparatus for expanding the dynamic range of an image, comprising: a replacement unit 23 for deriving the image data of the first memory as it is. 4. The first, second and third image data are color image data for each color of red R, green G and blue B, and range determining means, histogram creating means and third And the fourth density value setting means, the first to fourth extraction means, the first and second average value calculating means, the first calculating means for obtaining the first density value difference α2, and the second density value difference α1. The third calculating means to be obtained operates for each of the colors, and the first calculating means obtains the first exposure amount coefficient β2m as the average value of the average values β2R, β2G, and β2B of each color, and the second calculating means. Has a first color temperature correction means, and the first color temperature correction means has first correction density values F2newR2, F2newG2, F for each color of red R, green G and blue B.
2newB2 with respect to the image data in the second memory, the red R gain kR2 = 0.8, and the green G gain kG2 = 1.
0, the gain of blue B is set to kB2 = 1.0, F2newR2 = F2newR × kR2 F2newG2 = F2newG × kG2F2newB2 = F2newB × kB2 is output, and the third calculating means outputs the second exposure amount color β1m for each color. The average value β1R, β1G, β2B is calculated as an average value, and the fourth calculation means has a second color temperature correction means, and the second color temperature correction means is provided for each color of red R, green G and blue B. Second corrected density values F1newR1, F1newG1, F
1newB1 with respect to the image data of the third memory, the gain R of red R is kR1 = 1.0, the gain of green G is kG1 = 1.
The dynamic range of the image according to claim 3 is expanded by setting the gain kB1 of 0 and blue B to 0.8, and outputting as F1newR1 = F1newR x kR1 F1newG1 = F1newG x kG1 F1newB1 = F1newB x kB1. Device to do.