JP4898151B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging technique, and more particularly to an imaging technique for performing focus adjustment.

  In recent years, a subject image is imaged on a semiconductor imaging device, for example, a CCD two-dimensional image sensor, by an imaging optical system and converted into an electrical signal, and the resulting moving image data is converted into a semiconductor memory, a magnetic disk, or a magnetic tape. So-called digital video cameras for recording on such recording media, and digital still cameras capable of recording not only still images but also moving images are becoming widespread. Furthermore, with current digital video cameras and digital still cameras, all the important tasks for shooting such as exposure determination and focusing are automated, and the possibility of shooting failure even by a person who is unskilled in camera operation is very low.

  The above-described CCD image sensor has an electronic shutter function that can freely set the shutter speed. Therefore, the user can shorten or lengthen the shutter time according to the movement of the subject and the lighting state. For example, when shooting a moving image of a dark subject with little illumination, the user sets a so-called low-speed shutter so that the speed of the electronic shutter is reduced in order to increase exposure. This speed is a shutter speed slower than 1/60 second of one field period of a normal video signal, and is set to 1/30 seconds or 1/15 seconds, for example. However, if the shutter speed is slow, the influence of camera shake tends to occur, and as a result, the image of the subject is blurred. As a countermeasure against this, there is a method of obtaining a photographed image with a long exposure time by repeating photographing several times with an exposure time that does not cause camera shake, and combining the images obtained by these photographing while correcting image misalignment. (For example, patent document 1).

  In Patent Document 1, detection means for detecting motion information between a plurality of screens, and image moving means for converting the position of each of the plurality of screens within plane coordinates based on the detection information of the detection means. The present invention relates to an imaging screen composition device characterized by comprising. That is, if the technique disclosed in this document is applied, when shooting a moving image of a dark subject in which sufficient exposure cannot be obtained unless a low-speed shutter is set, the normal 1/30 second or 1 / Instead of setting a low-speed shutter of 15 seconds or longer, for example, 1 field period 1/60 seconds of a normal video signal where the exposure time is such that camera shake does not occur, and then the same as the low-speed shutter exposure period By synthesizing a plurality of images obtained after shooting multiple times at 1/60 second intervals during the time interval, the exposure amount is the same as when a low-speed shutter is set and the influence of camera shake is reduced. No moving image can be obtained at the same time interval as the low-speed shutter exposure period.

Japanese Patent No. 3110797

  By the way, instead of setting a low-speed shutter as described above, an exposure time is set so that camera shake does not occur, and a plurality of images are synthesized after shooting multiple times during the same time interval as the low-speed shutter exposure period. Therefore, when trying to obtain a moving image that is not affected by camera shake and has the same exposure amount as when a low-speed shutter is set, the focusing performance of the above-described automatic focusing means is greatly affected. There was found. That is, the normal moving image focus adjusting unit extracts the sharpness signal extracted by extracting the focus sharpness signal obtained at every image capturing interval from the image sensor output signal in accordance with the focus degree of the lens unit. For example, a method of adjusting the focus by controlling the lens unit so that the sharpness signal amount is maximized based on the signal (so-called TV autofocus method) is adopted. However, in the case of shooting a moving image of a dark subject with little illumination, the amplitude of the image sensor output signal obtained for each of the multiple shots is small, resulting in the sharpness signal for focusing obtained from the image sensor. The amplitude is also small, that is, a signal sufficient to satisfy the focusing performance cannot be obtained (low contrast state).

  FIG. 4 shows an example of the signal accumulation amount in the image sensor, the signal amount output from the image sensor, and the amplitude of the sharpness signal obtained from the image sensor output in this case. In the figure, (3-1) shows the relationship between the amount of signal accumulated in the image sensor and the exposure period. Symbols e2 / o3 / e3... Are repetition numbers of even / odd fields of the exposure period on the image sensor. Even in the case of a dark subject, the exposure period T is about 1/60 second with no camera shake because there is no influence of camera shake by the subsequent image synthesizing means and the exposure amount is equivalent to that when a low-speed shutter is set. The signal is read every 1/60 seconds (3-2). If this signal is combined four times, an exposure photographed image (synthesized image) equivalent to that obtained when a low-speed shutter with an exposure time of 1/15 seconds can be obtained. However, as shown in (3-3), the extracted sharpness signal is small in accordance with the amount of signal stored in the image sensor and is not sufficient to satisfy the focusing performance.

  Note that (3-4), (3-5), and (3-6) are output from the image sensor and the signal accumulation amount in the image sensor when the exposure period is set to a normal low-speed shutter of 1/15 second. 3 is an example of the signal amount and the amplitude of the sharpness signal obtained from the image sensor output. In this case, the amplitude of the sharpness signal is naturally increased, but the contrast blur component due to long-time exposure remains in the output of the image sensor, that is, the sharpness signal, and the focusing performance is greatly deteriorated.

  Here, for example, if the image composition processing as described above is not performed, in the case of a dark subject, the gain of the low-amplitude image sensor output signal obtained every 1/60 seconds of a normal video signal field period By increasing the amplitude of the sharpness signal, the contrast can be ensured although the amount of noise is large. However, as a result, the apparent exposure amount is increased by amplifying the image sensor output signal, so that it is impossible to acquire a high-quality moving image with a small amount of noise by the image composition as described above.

  Therefore, the present invention has been made in view of the above-described problems, and the object of the present invention is to improve the focusing performance in automatic focusing when acquiring a moving image without the influence of camera shake by the multiple image synthesis method. It aims at providing the technique which can prevent that a fall generate | occur | produces.

An image pickup apparatus according to the present invention detects a motion vector of a plurality of images output from a solid-state image sensor that inputs an object image formed through a lens unit that performs focus adjustment and outputs an image signal. a motion vector detecting means in accordance with the motion vector of the plurality of images, if based-out subject to the photometric value obtained by the photometric operation is dark, a synthesizing means for synthesizing images by increasing the number of images to be synthesized, the An extraction unit that extracts a sharpness signal corresponding to the degree of focus of the lens unit with respect to a subject from a signal corresponding to an extraction region of an image before synthesis, and a motion vector detected by the motion vector detection unit An adding means for adding the sharpness signal corresponding to the controlled extraction area in accordance with the number of combined images, and based on the sharpness signal added by the adding means And having a focus control means for performing focus adjustment by controlling the lens unit.
The imaging method of the present invention is an imaging method of an imaging apparatus having a solid-state imaging unit that inputs a subject image formed through a lens unit for performing focus adjustment and outputs an image signal. unit according to the motion vector of the plurality of images are output, the synthesis if based-out object is dark the photometric value obtained by the photometry operation, the synthesis step for synthesizing image by increasing the number of images to be synthesized, the image An extraction step for extracting a sharpness signal corresponding to the degree of focus of the lens unit with respect to the subject from a signal corresponding to the extraction region of the image before the image processing, and the extraction region controlled according to the motion vector An addition step of adding the sharpness signal according to the composite number, and controlling the lens unit based on the added sharpness signal to adjust the focus. And having a Cormorant focus control step.

By correcting and synthesizing a plurality of images, it is possible to suppress the influence of camera shake when shooting a moving image with a low-speed shutter and improve the accuracy of the captured moving image. In addition, by adjusting the focus by adding a sharpness signal, it is possible to improve the focusing performance when shooting a moving image of a dark subject with little illumination.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram showing a configuration example of a digital video camera according to an embodiment of the present invention. The light beam (photographing light) incident from the photographing lens 11 is imaged on the imaging unit 17 through the shutter 12a after the light amount is limited by the diaphragm 13a. The imaging unit 17 includes a two-dimensional image sensor such as a CCD.

  The photographic lens 11 is composed of a plurality of optical lens groups, and some or all of these lens groups receive a driving force from an autofocus (AF) driving motor 14a and move on the optical axis 10 to obtain a predetermined alignment. The focus is adjusted by stopping at the focal position. The AF drive motor 14a is driven by receiving a drive signal from the focus drive unit 14b. Also, some optical lens groups of the photographic lens 11 move on the optical axis 10 in response to the driving force from the zoom drive motor 15a, and change the photographic angle of view by stopping at a predetermined zoom position. The zoom drive motor 15a is driven by receiving a drive signal from the zoom drive unit 15b.

  The diaphragm 13a has a plurality of diaphragm blades, and these diaphragm blades are actuated by receiving a driving force from the diaphragm driving unit 13b to change the aperture area (diaphragm aperture) serving as a light passage aperture. The shutter 12a has a plurality of shutter blades, and these shutter blades open and close an opening serving as a light passage by receiving a driving force from the shutter driving unit 12b. Thereby, the light beam incident on the imaging unit 17 is controlled. The shutter 12a is used when taking a still image with a digital video camera.

  Driving of the image sensor driving unit 16, the focus driving unit 14b, the zoom driving unit 15b, the aperture driving unit 13b, and the shutter driving unit 12b is controlled by the imaging control unit 18. The shooting control unit 18 receives operation signals from the aperture operation unit 13c, the zoom operation unit 15c, and an image synthesis image stabilization operation unit 120, which will be described later, and the operation signals according to the shooting state of the camera. Are respectively supplied to the image sensor driving unit 16, the focus driving unit 14b, the zoom driving unit 15b, the aperture driving unit 13b, and the shutter driving unit 12b to set shooting conditions and perform shooting. Note that the aperture diameter of the aperture 13a is normally set automatically on the camera side at the time of shooting, so that the aperture operation unit 13c is not necessary, but is provided when the photographer arbitrarily sets the shooting conditions. ing.

  The imaging control unit 18 measures subject luminance (photometry) by a photometric signal 130 based on an image signal taken into the signal processing unit 111 described later, and based on the photometric result, the aperture diameter of the aperture 13a and the image sensor The electronic shutter timing (exposure time) of the drive unit 16 is determined.

  The video (image) signal output from the imaging unit 17 is converted from an analog signal to a digital signal by the A / D conversion unit 110 and input to the signal processing unit 111. The signal processing unit 111 performs signal processing such as forming a luminance signal and a color signal on the input signal to form a color moving image video signal.

  Then, the video signal signal-processed by the signal processing unit 111 is input to the image correction unit 117 via the signal switching unit 112. The image correction unit 117 performs gamma correction and compression processing on the input signal.

  The signal of the image correction unit 117 is input to the display unit 118 and the recording unit 119, and the captured moving image is displayed on the display unit 118 and recorded on the video recording unit 119.

  In the operation described above, if the subject to be photographed is dark and the exposure time is long, there is a risk of camera shake at a normal low-speed shutter. Therefore, the photographer operates the image composition image stabilization unit 120 to perform image composition image stabilization. The system is turned on and the operation is switched to the following operation, such as the flow by the imaging control unit 18 shown in FIG.

  First, the photometric operation is started (step S301). The electronic shutter timing (exposure time) of the image sensor driving unit 16 and the aperture diameter of the aperture 13a are set based on the photometric value obtained by the photometric operation. Generally, the subject is dark under the above-described shooting conditions. Is fully open and the exposure time is selected to be long exposure. Therefore, the long exposure time is changed to a plurality of continuous short exposure times so that the exposure amount added and combined by the multiple image composition method described later is equivalent to the exposure amount obtained when the low-speed shutter is set. To divide. When the exposure time is divided in such a short time, each image obtained by exposure is underexposed, but these images are less influenced by camera shake.

  Then, a moving image with improved exposure and camera shake is obtained by adding and synthesizing a plurality of consecutive images at the same repetition cycle as the low-speed shutter cycle.

  The image signal output from the imaging unit 17 is converted into a digital signal by the A / D conversion unit 110 and then subjected to signal processing by the signal processing unit 111. When the image synthesis anti-vibration operation unit 120 is operated to notify the imaging control unit 18 to turn on the image synthesis anti-vibration system, the image data from the signal processing unit 111 is transmitted via the signal switching unit 112 to the image. The data is input to the storage unit 113 (step S302). The image storage unit 113 has a storage area and a storage capacity for a plurality of images corresponding to the number of exposure time divisions, stores a plurality of images taken in consecutive video fields, and further inputs image data. It will be overwritten depending on. The deviation detection unit 114 extracts the same feature point in the image stored in the image storage unit 113, and detects the motion vector of the continuous image by determining the position coordinates of each feature point in the shooting screen ( Step S303). The coordinate conversion unit 115 corrects the respective differences (coordinate conversion) according to the position coordinates of the feature points in each image to match the feature point positions. That is, the video is sequentially moved in the opposite direction to the detected motion vector, and next, a good video whose shake is corrected by the image synthesis unit 116 is superimposed and synthesized (step S304). As a result, it is possible to correct a camera shake generated between fields and generate a good video signal.

  At the same time, the imaging control unit 18 drives the focus driving unit 14b to output the pre-image-combination sharpness signal extraction output 140 from the signal processing unit 111 or the post-image-combination sharpness signal extraction from the image composition unit 116 described later. Based on the sharpness signal formed and extracted by one of the outputs 150, the in-focus position of the photographing lens 11 is obtained.

  The imaging control unit 18 outputs an output signal 140 for extracting the sharpness signal before image synthesis from the signal processing unit 111 and an output signal for extracting the sharpness signal after image synthesis from the image synthesis unit 116. 150 is supplied. Then, in the shooting control unit 18, the amplitude value of the sharpness signal generated from the output 140 for extracting the sharpness signal before image synthesis is compared with a predetermined threshold, and the shooting subject is dark under the shooting conditions as described above. When the amplitude value is smaller than a predetermined value as in the case of the above, the output 140 for extracting the sharpness signal before image synthesis is not selected as the signal source for extracting the sharpness signal, and the image Control is performed to select the output 150 for extracting the combined sharpness signal (step S305). Frequency peak evaluation value, luminance level peak evaluation value, Max-Min evaluation value within the distance measurement frame for which the position / size is determined for autofocus based on the selected output for extracting the sharpness signal after image synthesis with a large amplitude Is generated and a hill-climbing type automatic focus adjustment operation is performed (step S306).

  It should be noted that at the time of the selection switching, the selection operation has an appropriate hysteresis with respect to the threshold value, and frequent switching operation does not occur when the amplitude value of the sharpness signal from the pre-image-combining sharpness signal extraction output 140 is near the threshold value. To control.

  FIG. 2 shows an example of the signal accumulation amount in the image sensor, the signal amount output from the image sensor, and the amplitude of the sharpness signal obtained from the image sensor output in the embodiment of the present invention. In the figure, (2-1) shows the relationship between the amount of signal accumulated in the image sensor and the exposure period. Symbols e2 / o3 / e3... Are repetition numbers of even / odd fields of the exposure period on the image sensor. Even in the case of a dark subject, the image synthesizing unit 116 in the subsequent stage has no influence of camera shake, and furthermore, the exposure amount T is about 1/60 seconds with no camera shake in order to obtain a moving image equivalent to the case where a low-speed shutter is set. The time is set, and the signal is read every 1/60 seconds (2-2). If this signal is synthesized four times, as shown in (2-3), an exposure photographed image (synthesized image) equivalent to that obtained when a low-speed shutter with an exposure time of 1/15 seconds can be obtained. When the amplitude value of the sharpness signal is smaller than a predetermined value, high-precision evaluation value data for autofocus is generated in order to obtain a sharpness signal with a large amplitude and high S / N from the synthesized image. (2-4).

  As described above, according to the present embodiment, the lens unit 11 for performing the focus adjustment and the subject image formed via the lens unit 11 are input, and the electronic shutter is continuously released to temporally. A solid-state imaging device 17 that outputs an image signal composed of a plurality of continuous images, a motion vector detection means 114 that detects a motion vector for the image immediately before the plurality of images, and a motion vector detection Based on each motion vector detected by the means 114, composition means 115 and 116 for composing each image by correcting the shift of each image, and the degree of focus of the lens means 11 based on the synthesized image Extraction means 116 for extracting the sharpness signal 150 and focus control means 18, 14a, 14b for controlling the lens means and adjusting the focus based on the extracted sharpness signal. .

  The extraction unit 111 extracts a sharpness signal 140 corresponding to the degree of focus of the lens unit 11 based on the pre-combination image. The focus control means 18, 14a, 14b selects the sharpness signal 140 based on the pre-combination image or the sharpness signal 150 based on the composited image, and controls the lens means 11 based on the selected sharpness signal to focus. Make adjustments.

  When the amplitude value of the sharpness signal 140 based on the pre-combination image is larger than the threshold value, the focus control unit 18 controls the lens unit 11 on the basis of the sharpness signal 140 based on the pre-combination image to adjust the focus. When the amplitude value of the sharpness signal 140 based on this image is smaller than the threshold value, the lens unit 11 is controlled based on the sharpness signal 150 based on the combined image to perform focus adjustment.

  By correcting and synthesizing a shift between a plurality of temporally continuous images, it is possible to suppress the influence of camera shake when shooting a moving image with a low-speed shutter and improve the accuracy of the captured moving image. In addition, by extracting the sharpness signal based on the synthesized image and adjusting the focus, it is possible to improve the focusing performance when shooting a moving image of a dark subject with little illumination.

  In an imaging device that can acquire moving image images that have the same exposure as when a low-speed shutter is set, but that is not affected by camera shake, by using a multiple-image composition method, it prevents the focusing performance from degrading in the auto-focusing means. For this purpose, it is possible to provide an imaging apparatus including means for extracting a sharpness signal for performing focus adjustment from a composite image after correcting the overlay processing of each video.

  In the present embodiment, the sharpness signal for performing focus adjustment is extracted from the composite image after correcting the overlay processing of each image. Therefore, camera shake correction is performed when shooting a dark subject with little illumination. Since the focus control of the television autofocus method can be performed by the image whose composite signal amplitude and contrast are recovered after the synthesis, the focusing performance can be improved as compared with the conventional case. Further, the accuracy of the captured moving image can be improved by correcting the camera shake when the moving image is captured with the low-speed shutter.

  Next, another technique capable of obtaining an appropriate focusing performance even in a low contrast state where a signal sufficient to satisfy the focusing performance cannot be obtained will be described. That is, the technique described here is also suitable for capturing a dark subject moving image with little illumination when the amplitude of the sharpness signal for focusing obtained from the image sensor is small, as described above. This is another technique that can achieve the focal performance.

  This technique has a configuration example similar to the configuration example of the digital video camera described in FIG. The difference is the extraction timing of the sharpness signal and the handling of the extracted sharpness signal. This technique will be described with reference to FIG.

  First, similar to step S301 in FIG. 3, a photometric operation is started (step S501). The electronic shutter timing (exposure time) of the image sensor driving unit 16 and the aperture diameter of the aperture 13a are set based on the photometric value obtained by this photometry operation, but here the subject is dark so the aperture is fully open and the exposure time is long. Time exposure is about to be selected. Therefore, similarly to the above description, one image to be acquired and one image to be acquired are underexposed due to the exposure control for the multiple image composition method.

  The image signal output from the imaging unit 17 is converted into a digital signal by the A / D conversion unit 110 and then subjected to signal processing by the signal processing unit 111. When the image synthesis anti-vibration operation unit 120 is operated to notify the imaging control unit 18 that the image synthesis anti-vibration system is to be turned on, the image data from the signal processing unit 111 is signal-switched as in step S302. The image is input to the image storage unit 113 via the unit 112 (step S502).

  As in step S303, the deviation detection unit 114 extracts the same feature point in the image stored in the image storage unit 113, and determines the position coordinates of each feature point in the shooting screen, thereby detecting successive image points. A motion vector is detected (step S503). The coordinate conversion unit 115 corrects the respective differences (coordinate conversion) according to the position coordinates of the feature points in each image to match the feature point positions. That is, the image is sequentially moved in the opposite direction to the detected motion vector, and then the image composition unit 116 superimposes and composites the good image with the shake corrected (step S504). Thereby, similarly to the above-described technique, it is possible to correct a camera shake generated between fields and generate a good video signal.

  Here, in the present technology, the imaging control unit 18 forms and extracts a sharpness signal from the pre-image-combined sharpness signal extraction output 140 from the signal processing unit 111 while driving the focus driving unit 14b (step S510). .

  However, since the brightness level of the acquired image is low, the acquired sharpness signal also has a low level. Therefore, a sharpness signal is added corresponding to the image composition in step S504 (S511). Here, the sharpness signal is added in accordance with the number of exposure time divisions described above. By this addition, a high-level sharpness signal can be acquired. Then, a hill-climbing automatic focusing operation is performed based on the added sharpness signal (step S512).

  Here, the extraction of the sharpness signal (step S510) may control the extraction area of the sharpness signal in accordance with the vector detection result in step S503. That is, by moving the sharpness signal extraction region in the video signal based on the detected vector, it is possible to extract the sharpness signal in the region corresponding to the image to be combined in step S504.

  In this way, by extracting the sharpness signal from the video before image synthesis and adding the sharpness signal, the focusing performance can be improved when shooting a moving image of a dark subject with little illumination. .

  In the description of FIG. 5, the sharpness signal is extracted in step S510 after the vector detection in step S503. However, as shown in FIG. 6, the sharpness signal is extracted in parallel with the vector detection (step S520). ), The sharpness signal may be added (step S521), and the focus adjustment operation may be performed (step S522). In this case, the sharpness signal extraction region cannot be moved using the above-described vector. However, by extracting the sharpness signal from the video before image synthesis and adding the sharpness signal in this way, it is possible to improve the focusing performance when shooting a moving image of a dark subject with little illumination. Can do.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

1 is a configuration diagram of a digital video camera according to an embodiment of the present invention. It is a timing chart which shows the amplitude amount of the sharpness signal by embodiment of this invention. It is a control flowchart in case the image composition anti-vibration system in embodiment of this invention is ON. It is a timing chart which shows the amplitude amount of the conventional sharpness signal. It is another control flowchart in case the image composition anti-vibration system in embodiment of this invention is ON. It is another control flowchart in case the image composition anti-vibration system in embodiment of this invention is ON.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Optical axis 11 Shooting lens 12a Shutter 12b Shutter drive part 12c Release operation part 13a Aperture 13b Aperture drive part 13c Aperture operation part 14a AF drive motor 14b Focus drive part 15a Zoom drive motor 15b Zoom drive part 15c Zoom operation part 16 Imaging element drive Unit 17 imaging unit 18 imaging control unit 110 A / D conversion unit 111 signal processing unit 112 signal switching unit 113 image storage unit 114 displacement detection unit 115 coordinate conversion unit 116 image composition unit 117 image correction unit 118 display unit 119 video recording unit 120 Image composition type image stabilization operation unit 130 Photometric signal 140 Output for extracting sharpness signal before image composition 150 Output for extracting sharpness signal after image composition

Claims (4)

A solid-state imaging device for inputting a subject image formed via a lens means for performing focus adjustment and outputting an image signal;
Motion vector detection means for detecting motion vectors of a plurality of images output by the solid-state imaging device;
According to the motion vector of the plurality of images, if based-out object is dark to the photometric value obtained by the photometry operation, and synthesizing means for synthesizing images by increasing the number of images to be synthesized,
Extraction means for extracting a sharpness signal corresponding to the degree of focus of the lens means with respect to a subject from a signal corresponding to the extraction region of the image before the synthesis;
Adding means for adding the sharpness signal corresponding to the extraction region controlled according to the motion vector detected by the motion vector detecting means in accordance with the number of combined images;
An image pickup apparatus comprising: a focus control unit that controls the lens unit based on a sharpness signal added by the addition unit and performs focus adjustment.   The imaging apparatus according to claim 1, wherein the extraction unit extracts the sharpness signal using a motion vector detected by the motion vector detection unit. An imaging method of an imaging apparatus having a solid-state imaging unit that inputs a subject image formed through a lens unit for performing focus adjustment and outputs an image signal,
The solid according to the motion vectors of a plurality of images imaging unit outputs, when based-out object is dark the photometric value obtained by the photometry operation, the synthesis step for synthesizing image by increasing the number of images to be synthesized,
An extraction step of extracting a sharpness signal corresponding to the degree of focus of the lens unit with respect to a subject from a signal corresponding to an extraction region of the image before the image is synthesized;
An addition step of adding the sharpness signal corresponding to the extraction region controlled in accordance with the motion vector in accordance with the number of combined images;
A focus control step of controlling the lens unit based on the added sharpness signal and performing focus adjustment.   4. The imaging method according to claim 3, wherein in the extraction step, the sharpness signal is extracted using the motion vector.

Images