JP2007274220A - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and an imaging method capable of photographing an image without a camera shake and an object image deletion. <P>SOLUTION: The imaging apparatus 100 for imaging a plurality of images by applying time division to an exposure time in one photographing operation is characterized in including: a comparison region setting section 158 that establishes a comparison region to image data in one exposure time; a memory 134 being an image storage section for storing one image in the comparison region; a deviation detection section 160 that detects other image equivalent to the one image from the image data in other exposure time so as to detect a relative positional deviation between the one image and the other image; and an image synthesizing section 118 that cancels the relative positional deviation to synthesize the images so that the one image may be coincident with the another image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method.

  There is an imaging apparatus that electronically corrects camera shake by synthesizing a plurality of images taken continuously in time so as to cancel image shift caused by camera shake or the like.

  In order to correct camera shake, it is necessary to set an area to be compared in each image to be synthesized and detect the amount of deviation of the area. Here, when the main subject is moving, for example, when taking a picture of a moving car or train, the amount of shift differs between the main subject and the background. Here, if the image is synthesized with the background as a comparison target instead of the main subject, the background is not blurred but the main subject is blurred.

  As a method for solving this problem, for example, there is Patent Document 1. Patent Document 1 describes a technique in which a moving subject is detected by fixing a camera at the time of shooting, and the subject is compared.

JP 2004-158905 A

  However, in the method described in Patent Document 1, when the camera pans the camera in accordance with the moving subject, the background is recognized as a moving subject and the background is not blurred, but the subject is blurred. There was a problem.

  Therefore, the present invention has been made in view of such a problem, and the object of the present invention is to ensure that the main subject is reliably detected regardless of whether the main subject has moved or whether the photographer has panned. A new and improved imaging device and imaging that can detect an image without camera shake or an image with no subject flowing by detecting and synthesizing images so as to cancel out the amount of movement and movement of the subject It is to provide a method.

  In order to solve the above-described problem, according to an aspect of the present invention, there is provided an imaging apparatus that captures a plurality of images by time-sharing an exposure time in one shooting operation: A comparison area setting section for setting a comparison area from among; an image storage section for storing one image in the comparison area; and detecting another image corresponding to the one image from image data at another exposure time; A displacement detection unit that detects a relative displacement between one image and another image; an image composition that cancels the relative displacement and synthesizes the images so that the one image matches another image And an imaging device.

  According to such a configuration, the comparison area setting unit sets a comparison area from image data at one exposure time, the image storage unit stores one image in the comparison area, and the deviation detection unit stores one image and the other. The relative position shift in the comparison area with the image of the image is detected, and the image composition unit synthesizes the image so that the one image matches the other image. As a result, it is possible to shoot an image free from camera shake or an image in which the subject does not flow by synthesizing the images so as to cancel out the shake and movement amount of the subject.

  The comparison area may be a focus detection area. According to such a configuration, the image is synthesized by the image composition unit so that the subject in focus in the focus detection region is used as the comparison region and the subject is matched. As a result, by compositing images so as to cancel out the blurring and movement amount of the focused subject in the focus detection area, it is possible to capture an image with no camera shake or an image where the focused subject does not flow.

  In order to solve the above-mentioned problem, according to another aspect of the present invention, there is provided an imaging method for imaging a plurality of images by time-sharing an exposure time in one imaging operation: image data at one exposure time A comparison region setting step for setting a comparison region from among the above; a focused image storage step for storing one image in the comparison region; and another image corresponding to one image among image data at other exposure times A displacement detection step for detecting and detecting a relative misalignment between one image and another image; canceling the relative misalignment and synthesizing the image so that the one image matches the other image And an image synthesizing step. An imaging method is provided.

  According to such a configuration, the comparison area setting step sets a comparison area from the image data at one exposure time, the image storage step stores one image in the comparison area, and the shift detection step detects one image and the other. A relative positional shift in the comparison area with the image is detected, and the image synthesis step synthesizes the image so that one image matches the other image. As a result, it is possible to shoot an image free from camera shake or an image in which the subject does not flow by synthesizing the images so as to cancel out the shake and movement amount of the subject.

  As described above, according to the present invention, the main subject is reliably detected regardless of whether the main subject has moved or whether the photographer has panned, and the image is synthesized so as to cancel the blur. Thus, it is possible to provide an imaging apparatus and an imaging method capable of capturing an image free from camera shake and an image in which a subject does not flow.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is an explanatory diagram for explaining an imaging apparatus according to an embodiment of the present invention. As shown in FIG. 1, an imaging apparatus 100 according to an embodiment of the present invention includes a zoom lens 102, a diaphragm 104, a focus lens 106, driving devices 102a, 104a, and 106a, and a CCD (Charge Coupled Devices). The element 108, an amplifier-integrated CDS (Correlated Double Sampling) circuit 110, an A / D converter 112, an image input controller 114, an image signal processing unit 116, a compression processing unit 120, an LCD (Liquid Crystal Display). ) Driver 123, LCD 124, timing generator 126, CPU (Central Processing Unit) 128, operation unit 132, shutter button 133, memory 134, VRAM (V an image random access memory) 136, a media controller 138, a recording medium 140, and motor drivers 142a, 142b, 142c.

  Hereinafter, the configuration of the imaging apparatus 100 according to an embodiment of the present invention will be described with reference to FIG.

  The zoom lens 102 is a lens in which the focal length is continuously changed by being moved back and forth in the optical axis direction by the driving device 102a. The zoom lens 102 changes the size of the subject and shoots. The diaphragm 104 adjusts the amount of light entering the CCD element 108 by the driving device 104a when taking an image. The focus lens 106 adjusts the focus of the subject by being moved back and forth in the optical axis direction by the driving device 106a.

  In the present embodiment, only one zoom lens 102 and focus lens 106 are shown, but the number of zoom lenses 102 may be two or more, and the number of focus lenses 106 may be two or more. Good.

  The CCD element 108 is an element for converting light incident from the zoom lens 102, the diaphragm 104, and the focus lens 106 into an electric signal. In this embodiment, the incident light is controlled by the electronic shutter to adjust the time for extracting the electric signal. However, the incident light may be controlled by using the mechanical shutter to adjust the time for extracting the electric signal.

  Although the CCD element 108 is used in the present embodiment, the present invention is not limited to such an example, and a CMOS (Complementary Metal Oxide Semiconductor) element may be used instead of the CCD element 108, or other image sensors may be used. May be. Since the CMOS element can convert the image light of the subject into an electrical signal at a higher speed than the CCD element, it is possible to shorten the time from when the subject is photographed to when the image is combined.

  The CDS circuit 110 is a circuit in which a CDS circuit that is a kind of sampling circuit that removes noise from the electrical signal output from the CCD element 108 and an amplifier that amplifies the electrical signal after removing the noise are integrated. . In the present embodiment, the imaging apparatus 100 is configured using a circuit in which a CDS circuit and an amplifier are integrated. However, the CDS circuit and the amplifier may be configured as separate circuits.

  The A / D converter 112 converts the electrical signal generated by the CCD element 108 into a digital signal, and generates raw image data (image data).

  The image input controller 114 controls input of raw image data (image data) generated by the A / D converter 112 to the memory 134.

  The image signal processing unit 116 performs light amount gain correction and white balance adjustment on the electrical signal output from the CCD element 108 and the image synthesized by the image synthesis unit 118.

  The image composition unit 118 performs composition of a plurality of captured images. The form of the image composition unit may be a circuit for performing image composition, or a computer program for performing image composition.

  The compression processing unit 120 performs compression processing for compressing the image combined by the image combining unit 118 into image data of an appropriate format. The image compression format may be a reversible format or an irreversible format. As an example of an appropriate format, it may be converted into a JPEG (Joint Photographic Experts Group) format or JPEG2000 format.

  The LCD 124 performs live view display before performing a shooting operation, various setting screens of the imaging apparatus 100, display of captured images, and the like. Display of image data and various types of information of the imaging apparatus 100 on the LCD 124 is performed via the LCD driver 122.

  The timing generator 126 inputs a timing signal to the CCD element 108. The shutter speed is determined by the timing signal from the timing generator 126. In other words, the drive of the CCD element 108 is controlled by the timing signal from the timing generator 126, and the image signal from the subject is incident within the time that the CCD element 108 is driven, thereby generating an electrical signal that is the basis of the image data. The

  The CPU 128 issues a signal-related command to the CCD element 108, the CDS circuit 110, and the like, and issues an operation-related command for the operation of the operation unit 132. In the present embodiment, only one CPU is included, but the signal-related command and the operation-related command may be executed by different CPUs.

  The operation unit 132 is provided with members for operating the imaging apparatus 100 and performing various settings during shooting. The members arranged in the operation unit 132 are arranged with a power button, a cross key for selecting a photographing mode and a photographing drive mode, and setting an effect parameter, a selection button, and the like. The shutter button 133 is for performing a photographing operation, and focuses the subject in a half-pressed state (hereinafter, the half-pressed state of the shutter button 133 is also referred to as “S1 state”), and is fully pressed (hereinafter referred to as “S1 state”). , The state where the shutter button 133 is fully pressed is also referred to as “S2 state”).

  The memory 134 is an example of the image storage unit of the present invention, and temporarily stores captured images and images synthesized by the image synthesis unit 118. The memory 134 has a storage capacity sufficient to store a plurality of images. Reading and writing of images to and from the memory 134 is controlled by the image input controller 114.

  The VRAM 136 holds contents to be displayed on the LCD 124, and the resolution and maximum color number of the LCD 124 depend on the capacity of the VRAM 136.

  The recording medium 140 is an example of an image recording unit, and records captured images and images synthesized by the image synthesis unit 118. Input / output to / from the recording medium 140 is controlled by the media controller 138. As the recording medium 140, a memory card, which is a card-type storage device that records data in a flash memory, can be used.

  The motor drivers 142a, 142b, and 142c control the driving devices 102a, 104a, and 106a that operate the zoom lens 102, the diaphragm 104, and the focus lens 106. By operating the zoom lens 102, the diaphragm 104, and the focus lens 106 via the motor drivers 142a, 142b, and 142c, the size of the subject, the amount of light, and the focus are adjusted.

  The imaging apparatus according to the embodiment of the present invention has been described above with reference to FIG. Next, the configuration of the CPU of the imaging apparatus according to the embodiment of the present invention will be described using FIG.

  FIG. 2 is an explanatory diagram illustrating the configuration of the CPU 128 according to the embodiment of the present invention. As shown in FIG. 2, the CPU 128 according to the first embodiment of the present invention includes a proper AE calculation unit 152, an exposure control unit 154, a focus position detection unit 156, and a comparison area setting unit 158. And a detection unit 160.

The appropriate AE level calculation unit 152 performs automatic exposure with the imaging apparatus 100 and acquires an EV (Exposure Value) value. Based on the acquired EV value, an appropriate aperture value and shutter speed set is determined. The EV value is changed by changing the aperture value and the shutter speed by setting EV = 0 as the amount of light with which an appropriate exposure can be obtained when the aperture value is F1 and the shutter speed is 1 second. The EV value can be obtained by EV = log ₂ (F ² / T) where F is the aperture value and T is the shutter speed. Therefore, the EV value increases as the shutter speed increases at the same aperture value, and the EV value increases as the aperture value increases at the same shutter speed. The appropriate AE level calculation unit 152 calculates an AF evaluation value and an AE (Auto Exposure) evaluation value of the captured image.

  The exposure control unit 154 determines the exposure time for photographing the subject based on the exposure time based on the shutter speed calculated by the appropriate AE level calculation unit 152. Based on the determined exposure time, the incident time of the image light from the subject to the CCD element 108 is controlled.

  The focus position detection unit 156 detects the focus position of the subject from the AF evaluation value of the generated image data when the image light from the subject is incident on the CCD element 108 under the control of the exposure control unit 154. is there.

  The comparison area setting unit 158 sets an image included in the in-focus area of the first image as comparison area data when combining a plurality of images shot by one shooting operation. When the images are combined by the image combining unit 118, the images are combined so that the comparison area data set by the comparison area setting unit 158 matches.

  The deviation detection unit 160 detects a relative positional deviation amount between the comparison area data in the first image and the comparison area data in the second and subsequent images. When the images are synthesized by the image synthesis unit 118, the images are synthesized so as to cancel out the shift amount of the relative position detected by the shift detection unit 160.

  The configuration of the CPU of the imaging apparatus according to the embodiment of the present invention has been described above with reference to FIG. Next, an imaging method according to an embodiment of the present invention will be described.

  FIG. 3 is a flowchart illustrating an imaging method according to an embodiment of the present invention. First, the subject is focused before photographing the subject. In order to perform the subject focusing process, first, the shutter button 133 is set to the S1 state, and the S1 operation in the S1 state is started (step S110). When the S1 operation of the shutter button 133 is started, the image signal processing unit 116 calculates an AF evaluation value by performing exposure processing of the subject while operating the focus lens 106 from the short distance side to the long distance side of the subject. (Step S112). Then, the focus position detection unit 156 detects the focus position of the subject corresponding to the peak of the AF evaluation value (step S114), and drives the focus lens 106 to a position where the drive device 106a focuses on the subject. The focusing process is completed (step S116).

  If the photographer has set the image capturing apparatus 100 to an automatic exposure mode in which shooting conditions such as a shutter speed (exposure time) and an aperture value are automatically set at the time of shooting, the subject is moved by the S1 operation of the shutter button 133. When the focusing process is performed, the appropriate AE level calculation unit 152 sets appropriate shooting conditions such as a shutter speed and an aperture value. Of course, the photographer may manually set the shutter speed and / or aperture value by operating the operation unit 132 without performing automatic exposure. When the photographer manually sets the shutter speed, the appropriate AE level calculation unit 152 automatically sets the appropriate aperture value, and when the aperture value is set manually, the appropriate AE level is set. The calculation unit 152 automatically sets the shutter speed by calculating an appropriate shutter speed.

  In the present embodiment, as shown in FIG. 4, when an image 170 including a train 172 and a mountain 174 as a subject is photographed, as a result of performing the S1 operation of the shutter button 133, the aperture value is set to F4.0. Suppose the sensitivity (gain) is set to ISO 100 and the shutter speed is set to 1/60 seconds. The set of aperture value, sensitivity, and shutter speed is an example, and is not limited to this. Further, when the shutter button 133 is set to capture five images at a time during the S2 operation, the exposure time per sheet is set to 1/60 ÷ 5 = 1/300 seconds. It should be noted that the number of images to be taken at one time during the S2 operation of the shutter button 133 may be set by operating the operation unit 132, and the limit shutter speed at which camera shake that can be calculated from the focal length of the subject does not occur. It may be determined from the camera shake limit shutter speed. The camera shake limit shutter speed can be obtained in approximately 1 / f (seconds) when the focal length of the focus lens 106 is f (mm).

  The in-focus position of the subject is detected by calculating an AF evaluation value for the image data in the focus detection area 176 in the image 170. The focus position is detected by the so-called hill climbing method.

  When the focus lens 106 is driven to the in-focus position of the subject, the S1 operation of the shutter button 133 ends, and the CPU 128 determines whether or not the shutter button 133 is not pressed (step S118). If the S1 operation of the shutter button 133 is finished and the shutter button 133 is not pressed, the process is finished. On the other hand, if the S1 operation of the shutter button 133 has not ended, it is determined whether the shutter button 133 is in the S2 state and the S2 operation is started (step S120). If the shutter button 133 is not in the S2 state, the process returns to step S118. On the other hand, if the shutter button 133 has been depressed to enter the S2 state, the subject exposure process is started and an imaging operation is performed (step S122). The imaging operation of the subject is performed at the shutter speed or aperture value set by automatic exposure or set manually.

  When the imaging operation is started, the image light from the subject enters the CCD element 108 for the set exposure time. When the set exposure time has elapsed, the image light is stopped from entering the CCD element 108 and image data is read from the CCD element 108 (step S124). The read image data is temporarily stored in the memory 134.

  When the reading of the image data is completed, it is determined whether or not the set number of images has been captured (step S126). If capturing of the set number of images has not been completed, the process returns to step S122, and exposure processing is performed again with the set exposure time. On the other hand, if the capturing of the set number of images has been completed, a plurality of captured images are combined.

  In the present embodiment, the photographer photographs a scene where the train 172 is traveling from right to left as time passes by a single photographing operation using the imaging device 100. As a result of photographing, five images 170a, 170b, 170c, 170d, and 170e as shown in FIGS. 5A to 5E are photographed.

  When combining a plurality of images, first, a comparison region of images is set (step S128). In the image comparison area in the present embodiment, the image of the focus detection area in the first image taken first among a plurality of images is set. In the image 170 shown in FIG. 4, the focus detection area 176 is a comparison area.

  When the image comparison area is set, the shift detection unit 160 then compares the relative value between one image in the comparison area of the first image and another image corresponding to the first image in the second and subsequent sheets. The amount of deviation is detected (step S130). The amount of deviation is detected depending on which part of the second and subsequent images the comparison area in the first image 170a is included in. A known technique such as a pattern matching method can be used to detect the comparison area in each of the second and subsequent images.

  When the amount of deviation from the comparison area in the first image 170a in each image is detected, the image composition unit 118 synthesizes each image so that the comparison areas in each image coincide with each other in order to cancel the image deviation. , An image is generated (step S132).

  An example of a composite image generated by the image composition unit 118 is shown in FIG. The synthesized image 180a shown in FIG. 6 is generated as a result of synthesizing five images 170a to 170e as shown in (a) to (e) of FIG. In the images 170a to 170e, the train 172 moves from right to left as time passes. However, since the head portion of the train 172 is set as the comparison area of the first image 170a, the head portion of the train 172 matches. By synthesizing the images 170a to 170e as described above, an image 180a in which the train 172 matches and the background mountain 174 flows can be generated as shown in FIG.

  In the conventional method of shooting a moving subject by combining a plurality of images, a plurality of images including the moving subject are captured with a camera fixed as in Patent Document 1, and the image is moved from the plurality of images. There is a method of detecting a moving subject and compositing so that the moving subject matches.

  However, with the method shown in Patent Document 1, a moving subject cannot be shot unless the camera is fixed, and it is not possible to take a picture while shaking the camera (panning) as in conventional panning. Can not. If the subject's movement speed is slow, it is possible to focus on the subject. However, when shooting a subject that moves at a high speed, such as a car or train, the subject is centered due to the time lag of the operation of the imaging device. If it is difficult to capture in the focus detection area of the camera or if it is taken while shaking the camera, the background is mistaken for the main subject, and as shown in FIG. 7, an image 180b in which the main subject is blurred instead of the background is taken. I will.

  Therefore, as in the present embodiment, in an imaging apparatus that can capture a plurality of images with a single shooting operation, the main subject does not have to move by focusing on the main subject at the time of shooting the first image. Even if the imaging device is panned or not, the main subject intended by the photographer can be reliably detected, and an image in which the main subject is not blurred can be taken.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, some of the components of the imaging apparatus 100 in the above embodiment may be a computer program stored in the imaging apparatus 100, or may be hardware such as a microprocessor that executes functions of each unit. Good.

  In the above embodiment, the focus detection area is used as the comparison area. However, an area other than the focus detection area may be set as the comparison area. Further, there may be two or more comparison areas. If the imaging apparatus includes a plurality of focus detection areas, a part or all of the plurality of focus detection areas may be set as the comparison areas. By setting a plurality of comparison areas, the amount of relative positional deviation can be detected more accurately.

  The present invention is applicable to an imaging apparatus and an imaging method.

It is explanatory drawing explaining the structure of the imaging device which concerns on one Embodiment of this invention. It is explanatory drawing explaining the structure of CPU which concerns on one Embodiment of this invention. It is a flowchart explaining the imaging method which concerns on one Embodiment of this invention. It is explanatory drawing which shows an example of an image. It is explanatory drawing which shows an example of the image image | photographed continuously. It is explanatory drawing which shows an example of the synthesized image which concerns on one Embodiment of this invention. It is explanatory drawing which shows the conventional synthesized image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Imaging device 102 Zoom lens 102a, 104a, 106a Drive device 104 Aperture 106 Focus lens 108 CCD element 110 CDS circuit 112 A / D converter 114 Image input controller 116 Image signal processing unit 118 Image composition unit 120 Compression processing unit 122 LCD driver 124 LCD
126 Timing Generator 128 CPU
132 Operation unit 133 Shutter button 134 Memory 136 VRAM
138 Media controller 140 Recording media 142a, 142b, 142c Motor driver 152 Appropriate AE level calculation unit 154 Exposure control unit 156 Focus position detection unit 158 Comparison area setting unit 160 Deviation detection unit 170, 170a, 170b, 170c, 170d, 170e, 180a, 180b Image 176 Focus detection area

Claims (3)

An imaging apparatus that captures a plurality of images by time-sharing an exposure time in one shooting operation:
A comparison area setting unit for setting a comparison area from image data at one exposure time;
An image storage unit for storing one image in the comparison region;
A displacement detection unit that detects another image corresponding to the one image from image data at another exposure time, and detects a relative displacement between the one image and the other image;
An image composition unit that composes an image such that the one image and the other image coincide with each other by canceling the relative displacement;
An imaging device comprising:   The imaging apparatus according to claim 1, wherein the comparison area is a focus detection area. An imaging method for capturing a plurality of images by time-sharing exposure time in one shooting operation:
A comparison area setting step for setting a comparison area from image data at one exposure time;
An image storing step for storing one image in the comparison region;
A displacement detection step of detecting another image corresponding to the one image from image data at another exposure time, and detecting a relative displacement between the one image and the other image;
An image combining step of canceling the relative positional shift and combining the images so that the one image matches the other image;
An imaging method comprising:

Images