JPH09107499A - Image pickup device and its picture processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To use plural picture signals different by exposures to composite a wide dynamic range picture. SOLUTION: A picture processing part 5 obtains a proper exposure based on the picture signal obtained by first photographing of an image pickup element 3 and the exposure at this time, and an image pickup element control part 7 performs second and following photographing with the proper exposure, over- exposure, and under-exposure by the image pickup element 3 based on the obtained proper exposure and stores respective picture signals in a memory 6. The picture processing part 5 synthesizes plural picture signals different in exposures stored in the memory 6 to generate one picture whose dynamic range is extended.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus using a solid-state image pickup element and an image processing method in the image pickup apparatus.

[0002]

2. Description of the Related Art In recent years, solid-state image pickup devices such as CCD image pickup devices have been widely used as image capturing devices for terrain vision cameras, video cameras, electronic cameras and the like. However, the dynamic range of incident light rays of a solid-state image pickup device is significantly narrower than that of a silver salt film, and depending on the image pickup conditions, blackening or whiteout of the screen may occur, resulting in a marked deterioration of image quality. Therefore, as a method of expanding the dynamic range of a solid-state image sensor, for example, Japanese Patent Laid-Open No. 63-30
As proposed in Japanese Patent No. 6778, a plurality of images having different exposure amounts are photographed for the same subject, and the image data of the plurality of images are combined by some calculation to expand the dynamic range. There is a method of obtaining one image.

On the other hand, conventionally, when the electronic camera is not provided with a dedicated photometric sensor, the first photographing (hereinafter referred to as pre-photographing) is performed in advance in order to determine the exposure amount.
At this time, the proper exposure amount is determined from the image signal obtained from the image pickup device and the exposure amount at the time of photographing, and then the second proper light photographing (hereinafter referred to as main photographing) is performed. At this time, the image signal obtained by the pre-shooting was used only for determining the exposure amount, and was then discarded.

In the case of performing the dynamic range expansion / combination described above by the photographing method for performing the pre-photographing, first, the pre-photographing is performed for photometry to obtain a plurality of image signals having different exposure amounts. A plurality of image signals are obtained by determining the exposure amount, and then performing the actual exposure with the appropriate exposure amount and the over exposure and under exposure with respect to the appropriate exposure amount.

[0005]

As described above, conventionally, the image signal obtained by the pre-shooting is used only for photometry and then discarded, so that it is necessary to obtain a plurality of images having different exposure amounts. Since it is necessary to perform the main photographing at least twice, there is a problem that the photographing takes time and trouble. Moreover, since one image is taken in the pre-photographing and two or more images are taken in the main photographing, there is a time difference between the first photographed image and the n-th photographed image. There was a problem that it was not synchronized.

Therefore, according to the present invention, when an image having a wide dynamic range is obtained by using a plurality of images having different exposures, an image pickup apparatus and an image processing in the image pickup apparatus can be shortened. It is intended to provide a way.

[0007]

According to a first aspect of the invention, an image pickup device for picking up an image of a subject and outputting an image signal,
Based on the image signal obtained by the first image pickup of the image pickup device and the exposure amount at the time of the image pickup, a proper exposure amount and an unsuitable exposure amount are obtained, and the obtained proper exposure amount or unsuitable exposure amount is used. A wide dynamic range is set using a control unit that causes the image pickup device to perform subsequent image pickup at least once, an image signal obtained by the first image pickup of the image pickup device, and an image signal obtained by the subsequent image pickup. And an image processing unit that generates an image signal.

According to a fifth aspect of the present invention, a step of obtaining an appropriate exposure amount and an inappropriate exposure amount based on the image signal obtained by the first image pickup of the image pickup device and the exposure amount at the time of the image pickup, Obtained by performing the subsequent image pickup of the image pickup device at least once using the obtained proper exposure amount or unsuitable exposure amount, the image signal obtained by the first image pickup of the image pickup device and the subsequent image pickup. And the step of generating an image signal having a wide dynamic range by using the generated image signal.

[0009]

According to the present invention, the dynamic range expansion synthesis is performed by performing the dynamic range expansion synthesis using the pre-photographing image signal for photometry and the image signal photographed with proper exposure or improper exposure based on the data. It is possible to avoid wasting image data for shooting. In particular, when combining two images of a proper exposure image and an improper exposure image, it is possible to combine the images with only one main shooting, and it is not necessary to newly shoot image signals with different exposure amounts, and the shooting time Can be shortened.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows a first embodiment of the present invention. In FIG. 1, 1 is a lens, 2 is an optical low-pass filter, 3 is an image pickup device such as CCD, and 4 is A / D
A converter, 5 is an image processing unit, 6 is a memory, and 7 is an image sensor control unit. Further, FIG. 2 is a diagram in which a method of reading from the image sensor 3 of the present image pickup apparatus is described together with an exposure amount.

Next, the operation will be described with reference to the drawings. First, pre-imaging is performed to determine the appropriate exposure amount. In this pre-shooting, in FIG. 2A, two field signals A and B are read from the image sensor 3. At this time, the image signal for photometry obtained from the image sensor 3 is digitized by the A / D converter 4 and then input to the image processing unit 5. Image processing unit 5
Determines the proper exposure amount by performing a calculation using the input image signal and the exposure amount at the time of shooting, and sends a signal indicating the proper exposure amount to the image sensor control unit 7. As a result, the image sensor control unit 7 can operate the image sensor 3 at an appropriate exposure amount at the next shooting. The image signal is stored in the memory 6. Then start the actual shooting. In this main shoot,
As shown in FIG. 2A, when the pre-photographing is an unsuitable exposure with an overexposure or underexposure, the next A and B fields are read with an appropriate exposure. If the pre-photographing is proper exposure, the illuminance distribution of the image data obtained by the pre-photographing is checked, and if the pre-image is a dark image, the exposure amount is overexposed and the A and B2 fields are photographed. . If the pre-image is a bright image, the exposure amount is set to be under, and the A and B2 fields are photographed. Also,
If the pre-image has an appropriate illuminance distribution, the exposure amount in the A field is changed to over, and the exposure amount in the B field is changed to under. The image signals thus read out from the image sensor 3 are stored in the memory 6 by the image processing unit 5 as in the pre-shooting.

Next, when there is a difference of a predetermined value or more between the exposure amount of the pre-image signal and the proper exposure amount or when the user desires, the dynamic range expansion synthesis is not performed and the pre-shooting is performed. An image signal of one image is created using the image signals of the two fields photographed in the main photographing. This image signal is converted into a luminance signal and a color difference signal and output.

In addition, when the dynamic range is enlarged and synthesized, a plurality of image signals having different exposure amounts are used.
A wide dynamic range image signal is synthesized and output. Here, as the dynamic range expanding method, Japanese Patent Application No. 5-281008, Japanese Patent Application No. 6-285067, Japanese Patent Application No. 6-285066, Japanese Patent Application No. 5-271942, etc., which the applicant previously applied, are applied. Synthesis is performed by the method described in.

FIG. 2B shows an example in which a 1-field signal is used for pre-shooting in order to shorten the shooting time. If the pre-photographing performed for the A field in FIG. 2B is an improper exposure, the next A and B2 fields are photographed with the proper exposure. If the pre-shot was a proper exposure, shoot the next A field with a proper exposure and
The field is photographed by changing the exposure amount to over or under according to the illuminance distribution of the pre-photographed image. The image signal thus captured is sent to the image processing unit 5.

When the dynamic range is not expanded, the image signals for two fields, which are taken with proper and the same exposure amount, from the three field signals with different exposure amounts obtained in the pre-shooting and the main shooting, are used. Create a single image. Further, when expanding the dynamic range, one wide dynamic range image is combined using the image signals for the above three fields.

As described above, by performing the process of synthesizing the pre-image signal for photometry and the image signal by proper or improper exposure, a plurality of image signals for synthesizing can be obtained by one photographing operation. Therefore, it is possible to obtain an image with an expanded dynamic range in a short time without wasting the pre-image signal for photometry. FIG.
By performing the pre-shooting in one field as in (b), the shooting time can be further shortened.

FIG. 3 shows a method of reading the image pickup device according to the second embodiment of the present invention. The configuration of the image pickup device is the same as that in FIG. A feature of the image pickup apparatus in the present embodiment is that dynamic range expansion synthesis is performed from three image signals of proper exposure, overexposure, and underexposure, and a wide illuminance range from a bright region to a dark region of a subject can be imaged. Different from the first embodiment. FIG. 4 is a flowchart showing the operation of the image pickup apparatus of this embodiment.

The read operation of the image sensor will be described below with reference to FIGS. First, pre-shooting is performed in step S1,
The pre-image is stored in the memory 6 in step S2, and at the same time, the appropriate exposure amount is determined in step S3. Next, step S4
If it is determined that the range is not expanded in step S5, the main shooting is performed in step S5 as illustrated in FIG.

Next, when it is determined in step S4 that the dynamic range is to be expanded, in step S6 the exposure amount during pre-shooting and the exposure amount during proper exposure shooting are compared, and if the difference is less than a certain value, 3 (a) by step S7 or FIG. 3 (b) by step S8 is imaged. In FIG. 3A, one frame is read for pre-shooting for photometry, and then two frame signals are read for main shooting. When the image data of the pre-shooting is over-exposure as compared with the proper exposure, the two image signals in the main shooting are shot by the proper exposure and the under-exposure. The three frame image signals thus obtained are combined in the same manner as in the first embodiment to expand the dynamic range. If the image signal for pre-shooting is underexposure, shooting is performed in the order of proper exposure and overexposure. If the pre-shooting image signal indicates proper exposure, overshooting and underexposure are performed in that order.

FIG. 3B shows an example in which pre-photographing is performed for one field and improper exposure in actual photographing is performed for one field in order to shorten the photographing time. In this case, the photographing time can be shortened by 2 fields as compared with FIG. First, one field is read in pre-shooting, and if it is over-exposure, two fields are taken in proper exposure in main shooting, and the next one field is taken in under-exposure. If the pre-exposure is underexposure, 2
The field is properly exposed, and the next field is overexposed. If the pre-shoot is the proper exposure,
The next one field is properly exposed, and then over and under are photographed in this order.

In step S6, the exposure amount in pre-shooting is compared with the exposure amount in proper-exposure shooting, and if the difference is more than a certain value, the image signal of pre-shooting cannot be used for synthesis, so step S9 After erasing the pre-image stored in the memory 6 in step S10, shooting is performed as shown in FIG. 3D in step S10. In other words, in addition to the proper exposure photographing, a plurality of different improper exposure amounts are newly photographed, and one wide dynamic range image is combined from these plural image signals.

By performing photographing by the exposure amount setting and reading method as described above, the image signal for photometry is not wasted, and three image signals having different exposure amounts are obtained in one main photographing operation. It is possible to synthesize the obtained image signals having an expanded dynamic range.

[0023]

As described above, according to the present invention, the dynamic range is expanded using the pre-photographing image signal for photometry and the image signal photographed with proper exposure or improper exposure based on the data. Since the composition is performed, the image capturing time can be shortened without wasting the image data of the pre-image capturing. In particular, there are two
In combining the images, it is not necessary to newly capture image signals with different exposure amounts in addition to the proper exposure image in order to expand the dynamic range.

Further, by reading out only the signal of one field from the image pickup device at the time of pre-photographing for photometry and combining it with the frame image signal of the main photographing, the photographing time can be further shortened.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an image pickup apparatus according to first and second embodiments of the present invention.

FIG. 2 is a sequence chart showing a reading method of the image sensor according to the first embodiment.

FIG. 3 is a sequence chart showing a reading method of the image sensor according to the second embodiment.

FIG. 4 is a flowchart showing the operation of the second embodiment.

[Explanation of symbols]

 3 image sensor 5 image processor 6 memory 7 image sensor controller

Claims (8)

[Claims] 1. An image pickup device for picking up an image of a subject and outputting an image signal, a proper exposure amount and an improper exposure based on an image signal obtained by the first image pickup of the image pickup device and an exposure amount at the time of picking up the image signal. And an image signal obtained by the first image pickup of the image pickup device, and a control means for performing the subsequent image pickup of the image pickup device at least once by using the obtained appropriate exposure amount or unsuitable exposure amount. And an image processing unit that generates an image signal having a wide dynamic range by using the image signal obtained by the subsequent imaging. 2. The image signal obtained by the first image pickup is not used when the difference between the exposure amount of the first image pickup and the exposure amount of the second and subsequent image pickups is a predetermined value or more. The image pickup apparatus according to claim 1. 3. The image pickup apparatus according to claim 1, wherein the image signal is a one-frame image signal. 4. The image pickup apparatus according to claim 1, wherein the image signal obtained by the first image pickup is a 1-field image signal. 5. A step of obtaining an appropriate exposure amount and an inappropriate exposure amount based on an image signal obtained by the first image pickup of the image pickup element and an exposure amount at the time of the image pickup, and the appropriate exposure amount obtained above or Using the step of performing subsequent imaging of the imaging device at least once using an improper exposure amount, the image signal obtained by the first imaging of the imaging device, and the image signal obtained by the subsequent imaging. And a step of generating an image signal having a wide dynamic range. 6. When the difference between the exposure amount at the time of the first imaging and the exposure amount at the time of the second and subsequent imaging is a predetermined value or more, the above-mentioned 1
The image processing method for an image pickup apparatus according to claim 5, wherein the image signal obtained in the second image pickup is not used. 7. The image processing method according to claim 5, wherein the image signal is a one-frame image signal. 8. The image processing method for an image pickup apparatus according to claim 5, wherein the image signal obtained by the first image pickup is a 1-field image signal.