JP2002232777A - Imaging system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging system capable of imparting a suitable gray- scale width to a main object in an image regardless of an area and a position of the object. SOLUTION: In an imaging system generating one image with wide dynamic range from more than one image imaged at different exposures, the system comprises an estimation part 14 for main object range to assume a main object range among images from a focus-locked focusing detecting range, a suitable exposure extracting part 13 to extract a suitable exposure range based on the level of an image signal image-by-image among a group of the images, a conversion characteristic calculation part 15 to execute a gray-scale correction to deliver the gray-scale width mainly to the assumed main object range for the extracted suitable exposure range, a brightness corrective part 16, a color- difference corrective part 17, a Y/C combining part 18 and an image combining par 20 to generate one image with wide dynamic range by combining the suitable exposure range of each image executed the gray-scale correction through the parts 15-18.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup system for gradation-compressing a wide dynamic range image obtained by synthesizing a plurality of images having different exposure amounts and outputting the image, or a method for increasing the dynamic range of one photographed image. More specifically, the present invention relates to an imaging system that obtains a high-quality output image by appropriately controlling a gradation width assigned to a main subject.

[0002]

2. Description of the Related Art Various imaging systems for electronically capturing an image of a subject have been proposed, and an electronic camera is an example.

A general CCD image pickup device used in such an electronic camera has a characteristic that a dynamic range relating to gradation is narrower than a silver halide film used in a conventional silver halide camera. In the case where an image of a subject having a large difference in contrast is taken, for example, when the sun and the shade under the scorching sun are imaged simultaneously, if one of them is imaged so as to have an appropriate exposure, the other is an image in which black or white is overexposed. It may be.

[0004] Even in such a photographing scene having a large difference in brightness, a plurality of images having different exposure amounts are combined to generate one wide dynamic range image so that both the bright and dark portions are appropriately reproduced. A technology to do this has been proposed.

[0005] However, the wide dynamic range image obtained in this manner cannot be directly handled by a display output system such as a normal monitor or a printer, so that it is necessary to perform gradation compression.

[0006] For example, Japanese Patent Laid-Open No. 2000-50151 discloses an image pickup technique capable of imaging the same subject with a plurality of different exposure amounts and generating image signals for a plurality of screens with different exposure amounts. Means for generating a wide dynamic range composite image by synthesizing image signals for a plurality of screens having different exposure amounts obtained by the imaging means. Determining means for determining whether or not an appropriate combined image is obtained based on the output of the determining means; and switching control means for switching on or off a generation processing operation of the combined image generating means based on an output of the determining means. An image pickup apparatus is described, and image signals for a plurality of screens having different exposure amounts obtained by imaging the same subject with a plurality of different exposure amounts are provided. Is read from a recording medium in which synthesis information used to generate a wide dynamic range synthesized image by synthesizing the file is read as a single file, and image signals for a plurality of screens in the file are read out from the storage medium. An external synthesizing device including a synthesizing means for generating a wide dynamic range synthesized image by synthesizing based on synthesis information in a file is described.

[0007] Further, in an ordinary electronic camera that obtains one image by one image pickup, the technology of gradation compression is used. That is, when performing digital signal processing,
Since there is a possibility that image quality may be degraded due to digit loss, the gradation bit (bit) of the input image may be set wider than the gradation bit of the output image in order to prevent this.
When this setting is made, gradation compression is finally performed. Therefore, in a conventional electronic camera, this gradation compression is generally processed by a fixed γ conversion.

Further, there is known a technique of obtaining a higher quality image by performing a contrast correction process on one image.

As such a technique, for example, Japanese Patent Application Laid-Open
In Japanese Patent Application Laid-Open No. 55520/1990, image data is input, a gradation frequency distribution is obtained, and a contrast correction parameter is set based on the gradation frequency distribution. There is described a technology for performing image level conversion based on the.

[0010]

As described above, in the prior art, the gradation is compressed by averaging the entire gradation width or by performing γ conversion. However, with such a means for fixedly compressing, it is not guaranteed that a main subject such as a person has an appropriate gradation, so that a preferable image is not always obtained. Further, when such a means of gradation compression is used, there is a tendency that a flat image lacking in contrast as a whole is obtained.

On the other hand, there has been proposed means for obtaining a histogram from an image and adaptively compressing the gradation. However, since the histogram is based on the histogram, a subject having a large area tends to be provided with a larger number of gradations. is there. For this reason, when a main subject such as a person is photographed in a small size in the screen, the gradation of the main subject tends to be lost, and an optimal image cannot be obtained.

In order to solve such a problem, there has been proposed a means of using a histogram of the edge portion on the assumption that the main subject has many edge portions. For example, when the main subject is a person's face, Contrary to the above assumption, since there are few elements serving as edges, a remarkable improvement effect cannot be obtained.

Thus, there is a demand for a technique capable of performing gradation compression such that the gradation of a main subject such as a person becomes more appropriate.

The present invention has been made in view of the above circumstances, and it is possible to provide an appropriate gradation width and perform higher-quality gradation compression irrespective of the area or position of a main subject in an image. It is intended to provide an imaging system.

[0015]

In order to achieve the above object, an imaging system according to a first aspect of the present invention processes an image group consisting of a plurality of images of the same subject captured under different exposure conditions. An imaging system for generating a wide dynamic range image, comprising: an estimating unit for estimating a main subject area in a captured image; and extracting an appropriate exposure area based on an image signal level for each image in the image group. Extracting means; correcting means for performing tone correction so that a tone width is distributed to the main subject area estimated by the estimating means with respect to the proper exposure area extracted by the extracting means; And synthesizing means for generating one wide dynamic range image by synthesizing an appropriate exposure area of each image subjected to gradation correction by the means.

[0016] The imaging system according to the second invention comprises:
An imaging system that generates an image in which a gradation is corrected by adjusting a gradation width of a captured image, comprising: an estimation unit configured to estimate a main subject area in the captured image; and a main unit estimated by the estimation unit. Correction means for performing the gradation correction of the captured image so that the gradation width is mainly distributed to the subject area.

Further, in the imaging system according to a third aspect of the present invention, in the imaging system according to the first or second aspect, the estimating means may include an image at a central portion of the screen, which is a focus detection area when focus is locked. A target image setting means for setting as a target image, a matching means for matching a target image set by the target image setting means with an image captured at the time of actual shooting, and a matching with the target image by the matching means. Region setting means for setting the region in the main photographed image determined to be the maximum as the main subject region.

An imaging system according to a fourth aspect of the present invention is the imaging system according to the first or second aspect, wherein the pre-imaging is performed a plurality of times before the actual imaging is performed. A flash unit for irradiating illumination light with a different light emission amount for each pre-photographing, a storage unit for storing a plurality of images related to illumination with different light emission amounts obtained by the pre-photographing, and stored in the storage unit. Distance calculating means for calculating distance information by reading an image at the time of pre-shooting and calculating the ratio of the luminance signal of each image, and is equivalent to a focusing distance at the time of main shooting based on the distance information calculated by the distance calculating means. Area setting means for setting an area at a short distance as a main subject area.

According to a fifth aspect of the present invention, in the imaging system according to the first or second aspect, the pre-photographing is performed prior to the main photographing, and the estimating means is obtained by the pre-photographing. Storage means for storing an image; difference means for reading an image at the time of pre-shooting stored in the storage means to obtain difference information from an image obtained by actual shooting; difference information obtained from the difference means Area setting means for setting, as a main subject area, an area having a difference equal to or greater than a predetermined threshold value based on the threshold value.

According to a sixth aspect of the present invention, in the imaging system according to the first or second aspect, the estimating unit is specified by an image display unit having a touch panel function and the touch panel function of the image display unit. Area setting means for setting an area as a main subject area.

According to a seventh aspect of the present invention, in the imaging system according to the first or second aspect, the estimating means includes: an image display means divided into a plurality of areas of a predetermined size; And an area setting means for setting the area selected by the selecting means as a main subject area.

An imaging system according to an eighth aspect of the present invention is the imaging system according to the first or second aspect, wherein the estimating means obtains an infrared distribution state of a subject area substantially corresponding to a captured image. A sensor unit, a candidate region setting unit that sets a region equal to or greater than a predetermined threshold as a candidate region based on the infrared component status acquired by the infrared sensor unit, and a candidate region set by the candidate region setting unit. The apparatus comprises a skin color detecting means for detecting a skin color component, and an area setting means for setting an area determined to include the skin color component by the skin color detecting means as a main subject area.

An imaging system according to a ninth aspect of the present invention is configured such that, in the imaging system according to the first or second aspect, it is possible to perform imaging by remote control using a remote control device. Receiving means for receiving a remote control signal transmitted from the remote control device; direction calculating means for calculating a transmitting direction of the remote control signal received by the receiving means; and transmitting direction calculated by the direction calculating means. Region setting means for setting a region of a predetermined size as a main subject region.

In the imaging system according to a tenth aspect, in the imaging system according to the first or second aspect, the correction means may be arranged such that the weight of the main subject area is heavier than an area other than the main subject area. Weighting factor setting means for setting such weighting factors, feature quantity calculating means for calculating feature quantities from the captured images, weighting factors set by the weighting factor setting means and calculated by the feature quantity calculating means A histogram creating means for creating a weighted histogram based on the characteristic amount; and a tone conversion curve calculating means for calculating a tone conversion curve based on the histogram created by the histogram creating means. is there.

[0025]

Embodiments of the present invention will be described below with reference to the drawings. 1 to 4 show a first embodiment of the present invention. FIG. 1 is a block diagram showing a configuration of an imaging system. FIG. 2 is a block diagram showing a configuration example of a main subject area estimating unit. 3 is a diagram illustrating a configuration example of an operation display unit, and FIG. 4 is a block diagram illustrating a configuration of a conversion characteristic calculation unit.

The imaging system according to the first embodiment is configured as, for example, an electronic camera.

First, the configuration of the imaging system according to the first embodiment will be described with reference to FIG.

This image pickup system includes a lens system 1 for forming an image of a subject on a CCD 4 described later, and an aperture 2 for defining a passing range of a light beam passing through the lens system 1.
A low-pass filter 3 for removing unnecessary high-frequency noise from a light beam passing through the lens system 1, and a light image formed after passing through the low-pass filter 3 is photoelectrically converted and output as an analog electric signal For example, a CCD 4 such as a single-plate color CCD having an electronic shutter function, an A / D converter 5 for converting an analog video signal output from the CCD 4 into a digital signal, and an A / D converter An image buffer 6 for accumulating image data output from the image buffer 5; a luminance and a focus position of a subject are detected based on the image data output from the image buffer 6; And outputs a lens drive control signal to an AF motor 8 to be described later, and outputs a strobe light emission control signal to a flash drive to be described later. 23, an AF motor 8 for driving a focus lens included in the lens system 1 based on a lens drive control signal output from the imaging control unit 7, and an output from the imaging control unit 7. Estimating means for irradiating the object with illumination light based on a strobe light emission control signal.
And an interpolator 9 for converting single-plate image data output from the image buffer 6 into three-plate image data.
A work buffer 10 for accumulating image data in the three-plate state output from the interpolation unit 9, and various operation switches as will be described later, and based on the image data output from the work buffer 10, An operation display unit 11 that is an estimating unit that displays an image for designating a subject area, a Y / C separation unit 12 that separates image data output from the work buffer 10 into a luminance signal Y and a color difference signal C, A proper exposure extracting section 13 as an extracting means for extracting a proper exposure area based on the luminance signal Y output from the Y / C separating section 12; and a main subject in the image based on the image data output from the image buffer 6. Main subject region estimating unit 14 which is an estimating unit for estimating
And a conversion means for calculating the conversion characteristic of the proper exposure area extracted by the proper exposure extraction unit 13 while weighting the main subject estimated by the main subject area estimation unit 14 so that the gradation is appropriate. A characteristic calculator 15;
A luminance correction unit 16 as correction means for performing gradation correction of the luminance signal Y based on the conversion characteristic calculated by the conversion characteristic calculation unit 15 for the appropriate exposure area extracted by the appropriate exposure extraction unit 13; Separation unit 12 and brightness correction unit 1
A color difference correction unit 17 serving as a correction unit for performing gradation correction of the color difference signal C output from the Y / C separation unit 12 based on the luminance signal before and after the conversion output from the Y / C 6 and a theoretical limit model where colors can exist. And a Y / C synthesizing unit 1 for synthesizing the luminance signal Y after gradation correction output from the luminance correction unit 16 and the color difference signal C after gradation correction output from the color difference correction unit 17.
8, a work buffer 19 for storing image data of a proper exposure area whose gradation has been corrected, and a plurality of image data relating to different exposures stored in the work buffer 19. An image synthesizing unit 20 that synthesizes one wide dynamic range image by synthesizing based on the proper exposure area information, and a wide dynamic range image synthesized by the image synthesizing unit 20 is stored in, for example, a memory card or a hard disk. An output unit 21 for outputting to a storage medium or an external device; an imaging control unit 7, an interpolation unit 9, an operation display unit 11, a Y / C separation unit 12, a proper exposure extraction unit 13, and a main subject region estimation unit 14 Conversion characteristic calculation unit 15, Y / C synthesis unit 18, image synthesis unit 20, output unit 21
And a control unit 22 such as a microcomputer that collectively controls each circuit in the imaging system including information obtained from the information and the like.

Next, the operation of the thus configured imaging system will be described with reference to the signal flow shown in FIG.

As described above, when a shutter button (not shown) provided in the image pickup system, for example, an electronic camera, is half-pressed, the apparatus enters a pre-photographing mode performed prior to the main photographing.

The subject image formed by the lens system 1 through the aperture 2 and the low-pass filter 3 is CC
The signal is converted into an electric signal by D4 and output as an analog video signal. This video signal is converted into a digital signal by the A / D converter 5 and then converted into a digital signal.
Is transferred to and stored.

The size of the image buffer 6 is such that the gradation width of the digitized signal in this embodiment is, for example, 1
If it is set to 0 bit, the capacity is such that the image data of the gradation width can be stored for the required continuous shooting.

The image data stored in the image buffer 6 is subsequently read out and transferred to the image pickup controller 7 as a video signal.

The imaging control unit 7 obtains a luminance level in the image, sets an aperture value and a shutter speed for achieving proper exposure, controls the aperture 2 and the CCD 4, and controls the edge intensity in the image. Is detected and the lens system 1 is driven via the AF motor 8 so as to maximize the edge strength, thereby controlling to obtain a focused image. After the lens system 1 is driven to the in-focus position in the pre-photographing mode, the in-focus position is fixed and maintained while the shutter button is half-pressed, resulting in a so-called focus lock state. During this time, the photographer can freely change the framing.

Next, the full photographing is performed by fully pressing the shutter button. The first image (short-time exposure image) is shot under a predetermined exposure ratio, for example, 1/8 of the exposure condition obtained by the imaging control unit 7, and A / D conversion is performed. After being converted into a digital signal by the device 5, it is transferred to the image buffer 6 and stored.

Subsequently, a second image (long-time exposure image) is photographed according to the exposure conditions obtained by the imaging control unit 7, and is converted into a digital signal by the A / D converter 5; The data is transferred to the buffer 6 and stored.

The interpolator 9 operates according to the control of the controller 22.
The video signals in the single-plate state stored in the image buffer 6 are sequentially read, and a known three-plate state signal is generated by performing known interpolation processing, white balance processing, enhancement processing, and the like, and transferred to the work buffer 10. I do.

The operation display unit 11 reads the long-time exposure image on the work buffer 10 for display under the control of the control unit 22.

Under the control of the control unit 22, the Y / C separation unit 12 reads the three-plate signals on the working buffer 10 in the order of a long-time exposure image and a short-time exposure image, and reads them in the order of a luminance signal Y and a chrominance signal. C.

In the luminance signal relating to the long-time exposure image, the overexposure area (inappropriate exposure area) is extracted by the appropriate exposure extraction section 13 and the signal of the other area (appropriate exposure area) is converted into the conversion characteristic calculation section. 15 and the luminance correction unit 16.

In the case of the luminance signal relating to the short-time exposure image, the luminance signal corresponding to the inappropriate exposure area is transferred to the conversion characteristic calculating section 15 and the luminance correcting section 16 by the appropriate exposure extracting section 13. Is done.

The main subject area estimating section 14 includes a control section 22
According to the above control, the image at the time of pre-shooting and the long-time exposure image are read from the image buffer 6, a main subject area is obtained from these images as described later, and the obtained result is transferred to the conversion characteristic calculation unit 15. .

The conversion characteristic calculating section 15 creates a cumulative histogram of the luminance signal sent from the proper exposure extracting section 13 by weighting the main subject area sent from the main subject area estimating section 14. , A conversion curve for performing gradation conversion based on the cumulative histogram is generated.

The brightness correction section 16 is provided with the conversion characteristic calculation section 1.
Based on the conversion curve sent from 5, a luminance signal for a long-time exposure image and a luminance signal for a short-time exposure image are converted. The converted luminance signal is transferred to the color difference correction unit 17 and the Y / C synthesis unit 18.

The color difference correction section 17 is provided with the Y / C separation section 12.
, And receives the converted luminance signal from the luminance correction unit 16. And
A correction coefficient for correcting the color difference signal is calculated based on the luminance signal before and after the conversion and the theoretical limit model in which the color can exist, and the correction coefficient is converted into the color difference signal sent from the Y / C separation unit 12. Multiply the coefficient.

The corrected color difference signal is supplied to the Y / C
And is combined with the converted luminance signal sent from the luminance correction unit 16 and stored in the working buffer 19 as a normal video signal in the order of a long exposure image and a short exposure image. .

The image synthesizing unit 20 controls the long-time exposure image after gradation conversion and the short-time exposure image after gradation conversion based on the information sent from the proper exposure extraction unit 13 under the control of the control unit 22. The image is synthesized to generate one wide dynamic range image and transferred to the output unit 21.

Next, a first example of the configuration of the main subject area estimating section 14 will be described with reference to FIG.

The main subject area estimating section 14 in this configuration example reads the pre-photographed image and the long-time exposure image from the image buffer 6 under the control of the control section 22, and stores them in the pre-photographed image buffer 32. A switching unit 31 for transferring the image to the main photographed image buffer 35, a pre-photographed image buffer 32 for storing the pre-photographed image sent from the switching unit 31, and a memory for storing the pre-photographed image buffer 32. A trimming unit as a target image setting unit that cuts out an area at the center of the screen of a predetermined size (an area which is a focus detection area used at the time of the focus lock and is considered to be a main subject of imaging) from an image obtained during pre-imaging 33, a target image buffer 34 for storing a target image cut out from the trimming unit 33, A captured image buffer 35 for storing the long-time exposure image, which is, the target image buffer 3
A matching unit 36 that performs a pattern matching process between the target image considered to be the main subject on the main image 4 and the long-time exposure image on the main photographed image buffer 35 and transfers the result to an area setting unit 37; Under the control of the control unit 22, an area in the long-time exposure image determined to have the highest matching by the matching unit 36 is set as a main subject area, and the result is transferred to the conversion characteristic calculation unit 15. Area setting unit 3 as means
7 are provided.

Since the image read from the image buffer 6 by the main subject area estimating section 14 is a single-chip image, the pattern matching processing by the matching section 36 is also performed in a single-chip state. Although the accuracy of the matching is slightly reduced by this, it can be said that it is still enough to perform the function of obtaining the approximate area of the main subject. Of course, the present invention is not limited to this, and it is also possible to perform a pattern matching process based on, for example, image information stored in the working buffer 10 after the interpolation process is performed to form a three-plate state.

Next, an example of the configuration of the conversion characteristic calculator 15 will be described with reference to FIG.

Under the control of the control unit 22, the conversion characteristic calculating unit 15 calculates a weighting factor such that the weight of the main subject area estimated by the main subject area estimating unit 14 becomes heavier than the other areas. Under the control of the weighting factor calculation unit 70 as a weighting factor setting unit to be set and the control unit 22, a luminance value of each exposure image is read from the proper exposure extraction unit 13 in the order described below, and a known feature amount for performing edge extraction is used. An edge extraction unit 71 serving as a calculation unit, and a pixel having an edge strength equal to or greater than a predetermined threshold value is selected from the edge image extracted by the edge extraction unit 71, and multiplied by a weight coefficient output from the weight coefficient calculation unit 70. A histogram creation unit 72 as a histogram creation unit for creating a histogram by performing weighting on the histogram;
And a conversion curve calculation unit 73 that is a gradation conversion curve calculation unit that generates a conversion curve by accumulating the histograms output from the second unit 2 and transfers the conversion curve to the luminance correction unit 16.

The conversion characteristic calculating section 15 first reads the luminance signal relating to the long exposure image from the appropriate exposure extracting section 13 and creates a conversion curve relating to the long exposure image as described above. Then, a conversion curve for the short-time exposure image is created in the same manner by reading a luminance signal for the short-time exposure image.

In this way, with the configuration as shown in FIG. 2A, the subject for which focus lock has been performed during the pre-shooting is determined as the main subject, and with the configuration as shown in FIG. Can be subjected to tone conversion. As a result, since the gradation width assigned to the main subject increases, a subjectively preferable image can be obtained.

Next, a second example of the configuration of the main subject area estimating unit 14 will be described with reference to FIG.

In the example shown in FIG. 2B, a plurality of stroboscopic images having different light emission amounts are photographed in the pre-shooting mode.
From these, a parameter proportional to the shooting distance is obtained, and an object at a distance equal to the focus distance is set as a main subject.

In other words, the main subject area estimating section 14 in this configuration example, under the control of the control section 22, controls the pre-shooting image when the strobe light is emitted from the image buffer 6 and the pre-shooting when the strobe light is not emitted. Switching unit 41 for reading the current image and transferring these single-plate images to the first image buffer 42 and the second image buffer 44, respectively.
A first image buffer 42 as storage means for storing an image at the time of pre-shooting when the strobe sent from the switching unit 41 is fired, and reading image data from the first image buffer A luminance calculating unit 43 that calculates a luminance signal after performing the linear interpolation processing of
A second image buffer 44 as storage means for storing an image at the time of pre-shooting when the strobe sent from 1 is not fired, and image data is read out from the second image buffer 44 to perform a known linear interpolation process. And a luminance calculation unit 45 that calculates a luminance signal after performing the calculation, and a distance calculation that obtains distance information by dividing the luminance signal from the luminance calculation unit 43 and the luminance signal from the luminance calculation unit 45 as described later. In accordance with the control of the dividing unit 46 and the control unit 22, a main subject region is set based on the distance information obtained by the dividing unit 46, and the region setting unit 47 is a region setting unit that outputs the main subject region to the conversion characteristic calculating unit 15. , And is configured.

The principle of setting the main subject area by the area setting section 47 based on the distance information is obtained by dividing the luminance signal between when the strobe light is emitted and when the strobe light is not emitted by the dividing section 46. Will be described.

The reflectance of the object is r, the angle between the normal line on the object surface and the optical axis of the lens system 1 is α1, the normal line on the object surface and the external light source other than the strobe from the object surface. Assuming that the angle between the heading direction and α2, the brightness of the strobe light is Ls, the brightness of an external light source other than the strobe is La, and the distance to the subject is d, the irradiation direction of the strobe light and the optical axis of the lens system 1 are described. Are almost the same, the pixel value V1 of the image at the time of flash emission is

V1 = r × cos (α1) × Ls / d ^ 2 + r × cos
(α2) × La. Here, the symbol “^” represents a power.

On the other hand, the pixel value V of the image when the strobe light is not emitted
2 is

## EQU2 ## V2 = r.times.cos (.alpha.2) .times.La.

The pixel value V1 obtained by the above equation (1),
When the ratio with the pixel value V2 obtained by the above equation 2 is obtained,

V1 / V2 = {1 / d ^ 2} × {cos (α1) / cos (α)
2)} × {Ls / La} +1, where Ls / La can be regarded as a constant term. Therefore, V1 / V2 obtained from Expression 3 is the object distance except for the influence of the direction of the surface. It will be inversely proportional to the square of d. That is, the distribution of V1 / V2 in the shooting screen is a distribution dependent only on the subject distance d. Therefore, the region setting unit 47 can extract a region at the focal distance as a main subject by extracting a region equivalent to the value of Expression 3 at the focal position.

Next, a third example of the configuration of the main subject area estimating unit 14 will be described with reference to FIG.

In the example shown in FIG. 2C, when the main subject is a person, a slight movement occurs in the main subject with the passage of time. An area is obtained and set as a main subject.

The main subject area estimating section 14 in this configuration example controls the image buffer 6 under the control of the control section 22.
A switching unit 51 for reading an image at the time of the pre-shooting and a long-time exposure image of the mainshooting, and transferring these single-chip images to the first image buffer 52 and the second image buffer 54, respectively.
A first image buffer 52 serving as a storage unit for storing the image at the time of the pre-shooting sent from the switching unit 51; and image data read out from the first image buffer 52 and subjected to a known linear interpolation process. A luminance calculating section 53 for calculating a luminance signal later; and a second image buffer 54 as a storage means for storing the long-time exposure image sent from the switching section 51
A luminance calculating unit 55 that reads out image data from the second image buffer 54 and performs a known linear interpolation process to calculate a luminance signal; a luminance signal from the luminance calculating unit 53 and a luminance calculating unit 55 Subtraction unit 56 as a difference means for obtaining a moving area by subtracting the luminance signal of
Under the control of the control unit 22, a main subject region is set by extracting a moving region within a predetermined range from the moving region obtained by the subtracting unit 56 and setting the extracted moving region as a person region. And an area setting unit 57 that is an area setting unit that outputs to the unit 15.

When the moving area is obtained by the subtraction unit 56, it is preferable to remove the influence of shifting the entire image due to camera shake or the like.

Further, in the above description, the main subject is detected by simply extracting only the moving area. However, by performing matching with skin color data and shape data, a more accurate person detection is performed. Is also possible.

Here, in the examples shown in FIGS. 2A to 2C, the imaging system automatically finds the main subject area, but the main subject area is manually designated. It is also possible.

FIG. 3 shows a specific configuration example of the operation display section 11 capable of designating a main subject area by such manual operation.
This will be described with reference to FIG.

First, the operation display section 11 of the configuration example shown in FIG. 3A displays various kinds of information relating to the imaging system, and reads out image data stored in the work buffer 10 to display an image. A liquid crystal 61 serving as an image display unit having a touch panel function and capable of inputting by a pen, and a liquid crystal 0N / N for setting whether to display the liquid crystal 61 or not.
It consists of a 0FF switch 62, a pen input switch 63 serving as an image display means for switching whether or not to enable the pen input to the liquid crystal 61, and a numeric keypad used for inputting numbers and the like to the imaging system. Numeric keys 64
And a selection key 65 as a selection means used when selecting an item or the like displayed on the liquid crystal 61, and an image display means for determining an item selected by the selection key 65, and a determination as a selection means. And a switch 66.

When the main subject area is manually designated using the operation display section 11 having such a configuration, the video signal is transmitted from the work buffer 10 by operating the liquid crystal 0N / 0FF switch 62. It is read and displayed on the liquid crystal 61.

Then, the pen input switch 63 is operated to shift to a pen input mode, and a closed curve is drawn while the tip of a pen (not shown) is brought into contact with the liquid crystal 61 to draw a closed curve.
Specify the main subject area.

Thereafter, when the decision switch 66 is pressed, the control section 22 extracts the inside of the designated closed curve as a main subject area and transfers it to the main subject area estimation section 14 as described above. Process.

Next, the operation display section 11 of another configuration example as shown in FIG. 3B has substantially the same configuration as that shown in FIG. 3A, but has a touch panel function. Liquid crystal 6 as a normal image display means instead of the liquid crystal 61
Image display means for setting a mode in which the image displayed on the liquid crystal 68 is divided into blocks and at least one of the divided blocks is selected instead of the pen input switch 63 The difference is that a block selection switch 69 is provided.

When the main subject area is manually designated using the operation display section 11 having such a configuration, the video signal is transmitted from the work buffer 10 by operating the liquid crystal 0N / 0FF switch 62. It is read and displayed on the liquid crystal 68.

When the mode is shifted to the block selection mode by the block selection switch 69, the image displayed on the liquid crystal 68 is divided into block areas of a predetermined size. The photographer operates the selection key 65 to move a block as a selection candidate up, down, left, and right, and presses a decision switch 66 when the block is moved over the main subject, thereby selecting a specific block area.

The control section 22 extracts the designated block area as the main subject area, and
4 to perform the above-described processing.

In the above description, a case has been described in which two images, a long-time exposure image and a short-time exposure image, are synthesized. However, this is merely an example, and it is assumed that a larger number of images with different exposure amounts are used. A configuration for synthesizing a wide dynamic range image is also possible, or the present invention can be applied to gradation correction even in the case of normal shooting in which one image is obtained by one exposure.

According to the first embodiment, the main subject area in the image is set automatically or manually, regardless of the area or position of the main subject in the image, so that the main subject has more floors. Since the tone conversion is performed so that a tone is assigned, it is possible to obtain an image on which a more subjectively preferable tone conversion has been performed.

FIGS. 5 and 6 show a second embodiment of the present invention. FIG. 5 is a block diagram showing a configuration of an imaging system. FIG. 6 is a block diagram showing a configuration example of a main subject area estimating section. FIG. In the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted. Only different points will be mainly described.

The imaging system according to the second embodiment eliminates the strobe 23, the operation display unit 11, the proper exposure extracting unit 13, the working buffer 19, and the image synthesizing unit 20 in the imaging system according to the first embodiment. Estimating means for detecting a two-dimensional infrared distribution image substantially corresponding to an image captured by being controlled by the imaging control unit 7 and outputting the detection result to the main subject region estimating unit 14; An infrared sensor 25 also serving as a means and a receiving means is newly added.

Subsequently, the operation of the second embodiment is basically the same as that of the first embodiment, and therefore, only different parts will be mainly described along the signal flow shown in FIG. I do.

By pressing a shutter button (not shown), one image is captured, and the image data is transferred to the image buffer 6.

In conjunction with this photographing, the imaging control section 7 causes the infrared sensor 25 to capture a two-dimensional infrared distribution image.
The result is sent from the infrared sensor 25 to the main subject area estimating unit 1.
4 is transferred.

On the other hand, the video signal stored in the image buffer 6 is converted into a three-plate signal by the interpolation unit 9 and transferred to the work buffer 10.

The Y / C separation section 12 converts the three signals on the work buffer 10 into a luminance signal Y under the control of the control section 22.
And a chrominance signal C, and the luminance signal Y is transferred to the conversion characteristic calculation unit 15 and the luminance correction unit 16.

The main subject area estimating section 14 obtains a main subject area based on information output from the infrared sensor 25 under the control of the control section 22 as described later, and sends it to the conversion characteristic calculating section 15. Forward.

The conversion characteristic calculation section 15 weights the cumulative histogram of the luminance signal Y sent from the Y / C separation section 12 to the main subject area sent from the main subject area estimation section 14. Then, a conversion curve for performing gradation conversion is generated based on the accumulated histogram.

The brightness correction section 16 is provided with the conversion characteristic calculation section 1.
The luminance signal Y is converted based on the conversion curve sent from the control unit 5 and transferred to the color difference correction unit 17 and the Y / C synthesis unit 18.

The color difference correction section 17 receives the luminance signal before conversion from the Y / C separation section 12 and
, And calculates a correction coefficient for correcting the color difference signal from the luminance signal before and after the conversion and the theoretical limit model in which the color can exist, and calculates the Y / Y
The color difference signal C sent from the C separation unit 12 is multiplied by the correction coefficient.

The Y / C synthesizing section 18 is provided with the color difference correcting section 17.
Is combined with the luminance signal converted by the luminance correction unit 16 and transferred to the output unit 21 as a normal video signal.

Here, an example of the configuration of the main subject area estimating section 14 will be described with reference to FIG.

The main subject area estimating section 14 binarizes the infrared distribution image sent from the infrared sensor 25 with a predetermined threshold value under the control of the control section 22. From the infrared distribution image transferred from the binarization processing unit 82 and the captured image read from the image buffer 6, an area having a certain temperature or higher is extracted as a candidate area of a main subject, and a linear interpolation process is performed on the candidate area. A candidate area extracting unit 81 as an area setting unit, an image buffer 83 for storing the candidate area transferred from the candidate area extracting unit 81, and a candidate area stored in the image buffer 83 are read out to generate a color difference signal. Y / C separation unit 84 to be calculated
And a skin color data recording unit 85 serving as a skin color detecting unit in which a range of color difference signals corresponding to the skin color is recorded; An area setting unit 8 serving as an area setting unit that transfers an area estimated to include a person by comparing the color difference signal transferred from the Y / C separation unit 84 to the conversion characteristic calculation unit 15 as a main subject area.
6 are provided.

Next, another example of the configuration of the main subject area estimating section 14 will be described with reference to FIG.

In the example shown in FIG. 6B, at the time of remote shooting using an infrared remote control device, the direction of the remote control device is detected, and a region of a predetermined size in that direction is set as a main subject region. It is like that.

That is, the main subject area estimating section 14
Is a binarization processing unit 91 that binarizes an infrared distribution image output from the infrared sensor 25 with a predetermined threshold value under the control of the control unit 22, and an infrared ray transferred from the binarization processing unit 91. The maximum intensity block extraction unit 92, which is a direction calculation unit that divides the distribution image into block regions of a predetermined size and sets a block having the maximum average value in each block as a candidate region, and the image buffer 6 under the control of the control unit 22 An area setting unit 93, which is an area setting unit that transfers the direction of the infrared remote control device as a main subject area to the conversion characteristic calculation unit 15 from the transferred signal and the candidate area transferred from the maximum intensity block extraction unit 92; It is configured to have.

Note that, in the above description, one exposure results in 1
Although the configuration in the normal case of capturing a single image has been described, the configuration of the second embodiment is not limited to this, and similarly to the first embodiment described above, the long exposure image and the short exposure image are used. The present invention can be applied to a case where two images or three or more images with different exposures are combined.

According to the second embodiment, substantially the same effects as those of the first embodiment can be obtained, and when the subject is a person, infrared rays are emitted due to body temperature or the like. The main subject area can be determined appropriately by using the infrared sensor with attention and comparing it with skin color data to distinguish it from other objects that emit infrared light. Then, by performing the gradation conversion with emphasis on the main subject, the gradation width assigned to the main subject increases, so that a subjectively preferable image can be obtained.

When performing remote shooting using an infrared remote control device, a photographer who also serves as a subject emits the infrared remote control device within a shooting screen, so that a red light detected by an infrared sensor is used. By detecting the maximum intensity in the outside distribution image, the direction of the subject can be specified.

The present invention is not limited to the above-described embodiment, and it goes without saying that various modifications and applications are possible without departing from the gist of the invention.

[0100]

As described above, according to the imaging system of the present invention, the main subject area and the proper exposure area of each image in a plurality of images taken under different exposure conditions are determined. By combining each image by performing gradation correction so that the gradation width is mainly distributed to the subject area, an appropriate gradation width is given to the main subject regardless of the area or position of the main subject in the image To be
High quality images can be obtained.

Further, according to the imaging system of the present invention, the main subject area in one photographed image is obtained, and the gradation correction is performed so that the gradation width is mainly distributed to the main subject. Accordingly, an appropriate gradation width is given to the main subject regardless of the area and position of the main subject in the image, so that a high-quality image can be obtained.

Further, according to the image pickup system of the present invention, the same effects as those of the first or second invention can be obtained, and the focus detection area when the focus is locked can be changed. Since the main subject area is obtained by matching with the photographed image, the main subject can be estimated by skillfully using the fact that the main subject is focus-locked.

According to the image pickup system of the present invention, the same effect as that of the first or second embodiment is obtained, and the distance information is obtained from the flash photographed images having different light emission amounts at the time of the pre-photograph. By determining an area at a distance equivalent to the focusing distance at the time of actual shooting as the main subject area, the main subject can be estimated with good accuracy.

According to the imaging system of the present invention, the same effects as those of the first or second embodiment can be obtained, and the motion can be obtained by taking the difference between the pre-photographed image and the main photographed image. By determining the area and setting it as the main subject area, the fact that a person or the like has a slight movement can be skillfully used to estimate the main subject.

According to the imaging system of the present invention, the same effects as those of the first or second aspect can be obtained, and the main subject position can be manually adjusted by using image display means having a touch panel function. , An appropriate gradation width can be given to a region intended by the photographer.

According to the imaging system of the present invention, the same effects as those of the first or second aspect can be obtained, and the photographer's intention can be selected by selecting the area of the image display means. An appropriate gradation width can be given to the region to be touched, and the touch panel function is not required, so that the configuration can be made more inexpensively.

According to the imaging system of the present invention, the same effect as that of the first or second aspect is obtained, and the infrared sensor means emits infrared light of a predetermined or more. Is detected, and a region having a flesh color component is further detected in the region, so that a main subject in the case of a person can be appropriately estimated.

According to the ninth aspect of the present invention, the same effect as that of the first or second aspect is obtained, and the transmission direction of the remote control signal is calculated to determine the predetermined size of the transmission direction. By setting the region as the main subject region, it is possible to appropriately estimate the main subject that is considered to be remotely controlled using the remote control device.

According to the tenth aspect of the present invention, the same effect as that of the first or second aspect is obtained, and the gradation conversion curve is obtained from the edge histogram weighting the main subject area. By calculating, it is possible to give an appropriate gradation width to the main subject.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an imaging system according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration example of a main subject area estimation unit in the imaging system according to the first embodiment.

FIG. 3 is a diagram showing a configuration example of an operation display unit in the imaging system according to the first embodiment.

FIG. 4 is a block diagram illustrating a configuration of a conversion characteristic calculation unit in the imaging system according to the first embodiment.

FIG. 5 is a block diagram illustrating a configuration of an imaging system according to a second embodiment of the present invention.

FIG. 6 is a block diagram showing a configuration example of a main subject area estimation unit in the imaging system according to the second embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Lens system 2 ... Aperture 4 ... CCD 7 ... Imaging control part 9 ... Interpolation part 11 ... Operation display part (estimation means) 12 ... Y / C separation part 13 ... Appropriate exposure extraction part (extraction means) 14 ... Main subject area Estimating section (estimating means, area setting means) 15: conversion characteristic calculating section (correcting means) 16: luminance correcting section (correcting means) 17: color difference correcting section (correcting means) 18: Y / C synthesizing section 19: working buffer Reference Signs List 20 image synthesizing unit (synthesizing unit) 21 output unit 22 control unit 23 strobe (estimating unit, strobe unit) 25 infrared sensor (estimating unit, infrared sensor unit, receiving unit) 31, 41, 51 Switching unit 32: Pre-captured image buffer 33: Trimming unit (target image setting unit) 34: Target image buffer 35: Main captured image buffer 36: Matching unit (matching unit) 37, 47, 57: Area setting Determining sections (area setting means) 42, 52: First image buffer (storage means) 43, 45, 53, 55 ... Luminance calculation sections 44, 54 ... Second image buffer (storage means) 46: Dividing section (distance calculation means) 56 ... Subtractor (difference means) 61, 68 ... Liquid crystal (image display means) 62 ... Liquid crystal 0N / 0FF switch 63 ... Pen input switch (image display means) 65 ... Selection key (selection means) 66 ... Confirmation switch (image) Display means, selection means) 69: block selection switch (image display means) 70: weight coefficient calculation section (weight coefficient setting means) 71: edge extraction section (feature amount calculation means) 72: histogram creation section (histogram creation means) 73 ... Conversion curve calculation unit (gradation conversion curve calculation means) 81 ... Candidate area extraction unit (candidate area setting means) 82,91 ... Binarization processing unit 85 ... Skin color data recording unit ( Color detection means) 86 ... area setting unit (region setting means) 92 ... maximum intensity block extraction section (direction calculation means) 93 ... area setting unit (region setting means)

Claims (10)

[Claims] An imaging system for processing an image group consisting of a plurality of images of the same subject captured under different exposure conditions to generate one wide dynamic range image, wherein a main subject in the captured image is provided. Estimating means for estimating a region; extracting means for extracting an appropriate exposure area based on an image signal level for each image in the image group; and estimating means for the appropriate exposure area extracted by the extracting means. By synthesizing a corrector that performs tone correction and an appropriate exposure area of each image that has been subjected to tone correction by the corrector, so that the tone width is mainly distributed to the estimated main subject area, An image capturing system comprising: a synthesizing unit that generates one wide dynamic range image. 2. An imaging system for generating an image whose gradation is corrected by adjusting a gradation width of a captured image, comprising: an estimation unit configured to estimate a main subject region in the captured image; And a correcting means for performing tone correction of the captured image so that the tone width is distributed with emphasis on the main subject area estimated by the above. 3. The target image setting means for setting, as a target image, an image at the center of the screen, which is a focus detection area when focus is locked, and a target image set by the target image setting means. And a matching unit that performs matching with an image captured at the time of the main shooting, and an area in the main shooting image that is determined by the matching unit to have the maximum matching with the target image,
3. The imaging system according to claim 1, further comprising: an area setting unit configured to set the main subject area. 4. The imaging system according to claim 1, wherein a plurality of pre-photographing operations are performed prior to the main photographing operation. Means for storing a plurality of images related to illumination with different light emission amounts obtained by pre-photographing; reading out the images at the time of pre-photography stored in the storage means; A distance calculating means for calculating distance information by taking a ratio; and an area for setting, as a main subject area, an area at a distance equivalent to the in-focus distance at the time of actual photographing based on the distance information calculated by the distance calculating means. The imaging system according to claim 1, further comprising: a setting unit. 5. The imaging system according to claim 1, wherein said image capturing system performs pre-photographing prior to actual photographing. The estimating means includes a storage means for storing an image obtained by pre-photographing; A difference means for reading an image at the time of the pre-shooting and obtaining difference information from an image obtained by the main shooting; and, based on the difference information obtained from the difference means, an area having a difference equal to or greater than a predetermined threshold value. The imaging system according to claim 1, further comprising: an area setting unit that sets an object area. 6. The estimating means includes: image display means having a touch panel function; and area setting means for setting an area specified by the touch panel function of the image display means as a main subject area. The imaging system according to claim 1, wherein the imaging system is provided. 7. The estimating unit includes: an image display unit that is divided into a plurality of regions of a predetermined size; a selecting unit that selects at least one of the plurality of regions; 3. The imaging system according to claim 1, further comprising: an area setting unit that sets an area as a main subject area. 8. An infrared sensor means for acquiring an infrared distribution state of a subject region substantially corresponding to a captured image, and a predetermined infrared component state obtained from the infrared state obtained by the infrared sensor means. Candidate area setting means for setting an area equal to or larger than a threshold value as a candidate area; skin color detecting means for detecting a skin color component with respect to the candidate area set by the candidate area setting means; 3. An imaging system according to claim 1, further comprising: an area setting unit that sets an area determined to be a main subject area. 9. The imaging system according to claim 1, wherein the image capturing system is configured to be capable of performing remote control image capturing using a remote control device, and wherein the estimating unit receives a remote control signal transmitted from the remote control device. Receiving means; direction calculating means for calculating a transmitting direction of the remote control signal received by the receiving means; area setting means for setting a predetermined size area of the transmitting direction calculated by the direction calculating means as a main subject area; The imaging system according to claim 1, wherein the imaging system comprises: 10. A weighting factor setting means for setting a weighting factor such that a weight of the main subject region is heavier than a region other than the main subject region; And a histogram creating means for creating a weighted histogram based on the weighting factor set by the weighting factor setting means and the feature amount calculated by the feature amount calculating means. 3. The image pickup apparatus according to claim 1, further comprising: a gradation conversion curve calculation unit that calculates a gradation conversion curve based on the histogram created by the histogram creation unit. system.