JP2009159380A - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of providing appropriate photographed images in which camera shakes and object blurs are suppressed by appropriately determining photographing conditions. <P>SOLUTION: The imaging apparatus 100 for imaging an object by an imaging device includes: a motion detection part 101 for detecting the motion vector of the imaging apparatus; an object motion calculation part 103 for calculating the motion vector of the object on the basis of image data acquired by the imaging device; an ISO sensitivity control part 104 for comparing the motion vector of the imaging apparatus detected by the motion detection part 101 with the motion vector of the object calculated by the object motion calculation part 103, determining whether or not to change ISO sensitivity on the basis of a comparison result and controlling the ISO sensitivity; and an imaging processing part 110 for performing imaging processing on the basis of the ISO sensitivity controlled by the ISO sensitivity control part 104. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method.

  In recent years, an image pickup apparatus such as a digital camera incorporates a camera shake correction function for correcting image blur caused by movement of a hand when a shutter is released. Various techniques such as an optical camera shake correction technique and an electronic camera shake correction technique have been proposed as the camera shake correction technique.

  In addition, a technique for discriminating subject blur due to movement of the subject itself and increasing ISO sensitivity to increase shutter speed and suppress subject blur has been proposed. However, there is a concern that increasing the ISO sensitivity increases noise generated during imaging and causes image quality degradation.

  In Patent Documents 1 to 5, by combining both the above-described camera shake correction technology and subject blur suppression technology, an appropriate image that suppresses deterioration of image quality due to increasing ISO sensitivity while suppressing both camera shake and subject blurring. Techniques for obtaining the above are disclosed.

JP 2007-206583 A JP 2007-243250 A JP 2007-243251 A JP 2007-243759 A JP 2007-41570 A

  However, even the techniques disclosed in Patent Documents 1 to 5 have insufficient points to suppress blurring. For example, when shooting a subject moving at a very fast speed, the camera itself may also move. In such a case, the ISO sensitivity is increased by a method disclosed in Patent Documents 1 to 5. It is not possible to obtain an appropriate image only by adjusting.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to obtain an appropriate photographed image in which camera shake and subject blurring are suppressed by appropriately determining the photographing situation. It is an object of the present invention to provide a new and improved imaging apparatus and imaging method capable of performing the above.

  In order to solve the above-described problem, according to an aspect of the present invention, an imaging device that captures an image of a subject using an imaging device, a motion detection unit that detects a motion vector of the imaging device, and an image acquired by the imaging device The subject motion calculation unit that calculates the motion vector of the subject based on the data, the motion vector of the imaging device detected by the motion detection unit, and the motion vector of the subject calculated by the subject motion calculation unit are compared, and the comparison result An ISO sensitivity control unit that determines whether to change the ISO sensitivity based on the ISO sensitivity, and an imaging processing unit that performs an imaging process based on the ISO sensitivity controlled by the ISO sensitivity control unit An imaging device is provided.

  With such a configuration, it is possible to appropriately determine the shooting situation and prevent image quality deterioration due to increasing ISO sensitivity when unnecessary.

  The ISO sensitivity control unit changes the ISO sensitivity when it is determined that the motion vector of the imaging device calculated by the motion detection unit and the motion vector of the subject calculated by the subject motion calculation unit are not in the same direction. You may make it do. Thereby, it is possible to prevent the ISO sensitivity from being changed when the subject and the imaging apparatus are both moving in the same direction.

  The ISO sensitivity control unit compares the magnitude of the motion vector of the imaging device with the magnitude of the motion vector of the subject when it is determined that the motion vector of the imaging device and the motion vector of the subject are not in the same direction. Based on the above, it may be determined whether to change the ISO sensitivity. Thereby, it is possible to prevent the ISO sensitivity from being changed due to the difference in the magnitude of movement between the subject and the imaging device.

  Further, the ISO sensitivity control unit compares the motion vector of the imaging device and / or the motion vector of the subject with a predetermined threshold value, and determines whether to change the ISO sensitivity based on the comparison result. Good. Thereby, it is possible to prevent the ISO sensitivity from being changed according to the size of the subject or the movement of the imaging apparatus.

  In addition, a shake correction processing unit that corrects the shake of the image data based on the motion vector of the imaging device detected by the motion detection unit may be further provided. Accordingly, it is possible to obtain a more appropriate image by using the camera shake correction by the shake correction processing unit together.

  In order to solve the above-described problem, according to another aspect of the present invention, there is provided an imaging method in an imaging apparatus for imaging a subject with an imaging device, a motion detection step for detecting a motion vector of the imaging apparatus, and imaging A subject motion calculation step for calculating a motion vector of the subject based on image data acquired by the element, a motion vector of the imaging device detected in the motion detection step, and a motion vector of the subject calculated in the subject motion calculation step Based on the comparison result in the vector comparison step, whether or not to change the ISO sensitivity is determined, and the ISO sensitivity control step for controlling the ISO sensitivity, and the ISO controlled in the ISO sensitivity control step An imaging step for performing imaging processing based on sensitivity. Imaging method and is provided.

  As described above, according to the present invention, it is possible to obtain an appropriate captured image in which camera shake and subject blur are suppressed by appropriately determining the shooting situation.

  Exemplary embodiments of an imaging apparatus and an imaging method according to the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  First, an imaging apparatus according to an embodiment of the present invention will be described. The image pickup apparatus according to the present embodiment includes a shake correction function that corrects a shake of a photographed image due to a movement of a photographer's hand or a movement of the image pickup apparatus itself. Hereinafter, a digital still camera will be described as an example of an imaging apparatus, but the present invention is not limited to such an example. The imaging apparatus according to the present invention may be, for example, an apparatus having a moving image shooting function such as a digital video camera, or may be a communication apparatus or an information processing apparatus such as a mobile phone having an imaging function. .

(Hardware configuration of digital still camera 100)
FIG. 1 is a diagram showing an outline of a hardware configuration of a digital still camera 100 as an imaging apparatus according to an embodiment of the present invention. As illustrated in FIG. 1, the digital still camera 100 according to the present embodiment includes an imaging processing unit 110, an image processing unit 120, and a control unit 130. Details of each part will be described below.

(Imaging processing unit 110)
The imaging processing unit 110 includes an optical block 111, a driving unit 112, an imaging element 113, and an analog processing circuit 114. The optical block 111 includes a lens group for imaging a subject, an aperture adjustment mechanism, a focus adjustment mechanism, a zoom mechanism, a shutter mechanism, a flash mechanism, and the like. The drive unit 112 is controlled in response to a control signal from the control unit 130, and electrically drives the aperture adjustment mechanism, the zoom mechanism, the shutter mechanism, and the like of the optical block 111.

  The image sensor 113 is configured by, for example, a solid-state image sensor such as a charge coupled device (CCD), and an image through the optical block 111 is formed on an image formation surface thereof. Further, the image sensor 113 receives the timing signal generated by the analog processing circuit 114, converts the subject image formed on the imaging surface into an imaging signal, and supplies the image signal to the analog processing circuit 114.

  The analog processing circuit 114 performs processing for converting an analog imaging signal from the imaging element 113 into a digital signal. The analog processing circuit 114 is an AGC (Auto Gain Control) that automatically adjusts the gain of an image pickup signal from the image sensor 113, an AD (Analog Digital) converter that converts an analog signal into a digital signal, and a control unit according to a shutter operation. A timing generator or the like that generates an image capture timing signal to the image sensor 113 based on a control signal supplied from 130 is provided internally. The analog processing circuit 114 also has an ISO sensitivity adjustment function that adjusts the ISO sensitivity by increasing or decreasing the gain in AGC under the control of the control unit 130. The digital imaging signal output by the analog processing circuit 114 is stored in a storage medium such as the RAM 134 as RAW image data.

(Image processing unit 120)
The image processing unit 120 reads out the RAW image data output from the analog processing circuit 114 and stored in the RAM 134 and the like, performs digital image processing, and image formats such as JPEG (Joint Photographic Experts Group) and TIFF (Tagged Image File Format). And recorded in a storage medium such as the memory card 143. The image processing unit 120 performs various types of image processing such as pixel interpolation processing, gamma correction processing, and white balance processing on the RAW image data.

(Control unit 130)
As shown in FIG. 1, the control unit 130 is an operation input for connecting an MPU (Micro Processing Unit) 131, a ROM (Read Only Memory) 133, a RAM (Random Access Memory) 134, and an operation input unit 141. An interface 135, a display control unit 136 for connecting the display unit 142, a memory card interface 137 for loading the memory card 143, an angular velocity sensor 138, and an external input / output interface 139 are connected via the system bus 132. It is configured by being connected.

  The control unit 130 controls the imaging processing unit 110 and the image processing unit 120 to control the entire digital still camera 100 so that the imaging process is executed. In particular, the control unit 130 has a blur correction function for detecting the movement of the subject and the digital still camera 100 itself, and appropriately determining the shooting situation from the detected movement and performing the blur correction. Details of the blur correction function of the control unit 130 will be described later. Hereinafter, the configuration of the control unit 130 will be described.

  The MPU 131 supplies various control signals generated by executing the control program recorded in the ROM 133 to each unit, and controls the entire digital still camera 100. The ROM 133 stores programs for controlling the imaging processing unit 110, the image processing unit 120, the display control unit 136, and the like, and the memory card interface 137. The RAM 134 is a storage medium that stores RAW image data output from the analog processing circuit 114 of the imaging processing unit 110 described above. The RAM 134 is also used as a work area for the MPU 131.

  The operation input unit 141 connected to the operation input interface 135 includes a plurality of modes such as a mode switching key for switching a shooting mode, a zoom adjustment key, a key for exposure adjustment, a shutter key, and a display adjustment key for the display unit 142. A key is provided. The operation input interface 135 transmits an operation signal from the operation input unit 141 to the MPU 131. The MPU 131 determines the operation content based on the operation signal input from the operation input interface 135, and performs control processing according to the determination result.

  The display unit 142 connected to the display control unit 136 is configured by a liquid crystal display including a TFT (Thin Film Transistor) and the like. The display control unit 136 is controlled by the MPU 131 and causes the display unit 142 to display an image processed by the image processing unit 120 and an image read from the memory card 143.

  The memory card interface 137 writes the image data output from the image processing unit 120 to the memory card 143. The memory card interface 137 reads image data from the memory card 143 and supplies it to the image processing unit 120 and the like. The memory card 143 is a storage medium that is read and written by the memory card interface 137, and mainly records captured image data, attribute information associated with the image data, and the like.

  The angular velocity sensor 138 is configured by a sensor that can detect the magnitude of the angular velocity, such as a gyro sensor, and detects the angular velocity in each direction according to the movement of the digital still camera 100. The angular velocity detected by the angular velocity sensor 138 is input to the MPU 131 and used to detect the movement of the digital still camera 100.

The external input / output interface 139 is an interface for connecting an external device and the digital still camera 100 to input / output data, and outputs the data stored in the memory card 143 of the digital still camera 100 to the external device. Or an interface for inputting data input from an external device to the digital still camera 100.
Heretofore, the hardware configuration of the digital still camera 100 according to the present embodiment has been described.

(Configuration of image stabilization function)
Next, the blur correction function executed by the control unit 130 of the digital still camera 100 will be described with reference to FIG. FIG. 2 is a functional block diagram showing a schematic configuration for realizing the blur correction function of the control unit 130 of the digital still camera 100. As shown in FIG. 2, the shake correction function of this embodiment includes a motion detection unit 101, a camera shake correction processing unit 102, a subject motion calculation unit 103, and an ISO sensitivity control unit 104.

  2 may be realized by a software program that executes the function of each unit, which is stored in the ROM 133 or the like and can be executed by the MPU 131, or a dedicated hardware. It may be realized by wear. The software program may be executed by reading out a program stored in a computer-readable storage medium such as the ROM 133 as described above, or may be provided to the digital still camera 100 via a network or the like. It may be a thing.

(Motion detection unit 101)
The motion detection unit 101 detects the motion of the digital still camera 100 itself based on the angular velocity detected by the angular velocity sensor 138. The motion detection unit 101 calculates a motion vector of the digital still camera 100 and inputs the motion vector to the camera shake correction processing unit 102 and the ISO sensitivity control unit 104.

(Shake correction processing unit 102)
The camera shake correction processing unit 102 obtains the amount of camera shake based on the motion of the digital still camera 100 detected by the motion detection unit 101, and controls the imaging processing unit 110 and the image processing unit 120 to perform camera shake correction. The camera shake correction method performed here may be, for example, so-called optical camera shake correction in which the drive unit 112 of the imaging processing unit 110 is controlled to move the correction lens provided in the optical block 111. Or what is called electronic camera-shake correction | amendment which corrects the blurring of an image by trimming the image obtained by the image pick-up element 113 may be sufficient.

  In addition, here, the camera shake correction processing unit 102 will be described as calculating the amount of camera shake based on the motion of the imaging device detected by the angular velocity of the imaging device detected by the angular velocity sensor 138, but subject motion calculation The blur amount may be calculated based on the movement of the through image calculated by the unit 103.

(Subject motion calculation unit 103)
The subject motion calculation unit 103 calculates the motion of the subject from the motion of the through image, and calculates a motion vector. The subject motion calculation unit 103 obtains a subject motion vector by comparing image data (RAW image data) captured by the image sensor 113 and stored in a storage medium such as the RAM 134 with image data stored immediately before. The motion vector is calculated using, for example, a conventionally known moving image analysis method such as detecting a luminance value of a pixel constituting image data and calculating a motion vector based on a change in luminance value from the immediately preceding image data. Can be done.

(ISO sensitivity control unit 104)
The ISO sensitivity control unit 104 controls the ISO sensitivity based on the motion vector of the digital still camera 100 detected by the motion detection unit 101 and the subject motion vector calculated by the subject motion calculation unit 103. First, the ISO sensitivity control unit 104 determines whether to control the ISO sensitivity by comparing the motion vector of the digital still camera 100 with the motion vector of the subject. Specifically, if the motion vector of the digital still camera 100 and the motion vector of the subject are in the same direction, it is determined that the digital still camera 100 is moving in the same direction together with the subject, and correction for subject blur is not performed. That is, it is determined not to change the ISO sensitivity. On the other hand, if the motion vector of the digital still camera 100 and the motion vector of the subject are not in the same direction, it is determined that the subject is shaken, and the ISO sensitivity is determined to be changed. Here, as a method of determining whether or not both motion vectors are in the same direction, the angle formed by both vectors is compared with a predetermined threshold, and if the angle is smaller than the threshold, the direction is the same. It may be determined that the direction is not the same if the angle is large.

  Further, the magnitude of the vector together with the direction of the motion vector may be used for the determination. For example, the motion vector of the subject may be compared with a predetermined threshold value, and the comparison with the motion vector of the digital still camera 100 described above may be performed only when the motion vector is larger than the threshold value. Thus, the determination based on the vector direction can be performed only when the subject is moving at a speed higher than a constant speed. Similarly, a threshold value may be provided for the motion vector of the digital still camera 100, and the determination based on the vector direction may be performed only when the motion vector is larger than the threshold value. Alternatively, the determination based on the magnitude of the motion vector may be performed on both the subject and the digital still camera 100.

  Also, as a result of the determination based on the vector direction, when it is determined that the digital still camera 100 and the subject are moving in the same direction, it is determined whether the subject is blurred by comparing the magnitudes of the vectors of both. You may do it. For example, when the movement of the digital still camera 100 is significantly smaller than the movement of the subject, it may be determined that the subject is shaken and the ISO sensitivity may be changed.

  If it is determined by the above-described method that the blur is a subject blur due to the movement of the subject, the ISO sensitivity control unit 104 performs control so that the subject blur is suppressed by increasing the ISO sensitivity of the image sensor 113. In addition, the ISO sensitivity may be increased, and the shutter speed and the aperture value may be adjusted so as to achieve an appropriate exposure with the sensitivity. When increasing the ISO sensitivity, the ISO sensitivity is determined based on the motion vector of the subject calculated by the subject motion calculation unit 103. Here, the ISO sensitivity control method increases the sensitivity by controlling the gain in the AGC included in the analog processing circuit 114 to increase. However, another method may be used. For example, a method of increasing the sensitivity by driving the image sensor 113 to add pixels may be used, or a method of increasing the ISO sensitivity by performing high-precision digital image processing in the image processing unit 120 may be used.

  The configuration for executing the shake correction function in the digital still camera 100 has been described above.

(Example of background image generation processing)
Next, an example of imaging processing executed by the digital still camera 100 according to the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing the flow of imaging processing when imaging is performed using the blur correction function in the digital still camera 100 according to the present embodiment.

  First, in step S201, imaging of a subject by the imaging element 113 is started, and image data that has been captured and stored in a storage medium such as the RAM 134 is displayed on the display unit 142 (starting display of a through image). In step S203, when the angular velocity of the digital still camera 100 is detected by the angular velocity sensor 138 and notified to the MPU 131, the motion detection unit 101 calculates the motion vector of the digital still camera 100.

  Next, in step S205, the MPU 131 confirms whether or not the camera shake correction function is ON. If ON, the process proceeds to step S207, and the camera shake correction processing unit 102 performs camera shake correction. For example, the camera shake correction processing unit 102 performs the camera shake correction processing by controlling the driving unit 112 so as to move the correction lens provided in the optical block 111 according to the motion vector calculated by the motion detection unit 101. Execute.

Next, in step S <b> 209, the subject motion calculation unit 103 calculates a subject motion vector from the motion of the through image and inputs the motion vector to the ISO sensitivity control unit 104. Next, in step S <b> 211, the ISO sensitivity control unit 104 compares the motion vector of the digital still camera 100 input from the motion detection unit 101 with the motion vector of the subject input from the subject motion calculation unit 103. Here, it is determined whether or not both motion vectors are in the same direction. If it is determined that the motion vectors are in the same direction, it is determined that there is no subject blur, and imaging processing is performed without changing the ISO sensitivity. .
On the other hand, if it is determined in step S211 that the motion vector of the digital still camera 100 and the subject is not in the same direction, it is determined that there is subject blur, and the ISO sensitivity control unit 104 changes the ISO sensitivity in step S213. Is called. The ISO sensitivity control unit 104 controls to increase the ISO sensitivity by increasing the gain in the AGC of the analog processing circuit 114 based on the subject motion vector calculated by the subject motion calculation unit 103.

  Thereafter, in step S215, an imaging process is executed with the ISO sensitivity controlled by the ISO sensitivity control unit 104. As described above, the digital still camera 100 according to the present embodiment compares the movement of the digital still camera 100 with the movement of the subject on the through image, and the digital still camera 100 moves in the same direction as the subject. In this case, it is determined that there is no subject blur and the ISO sensitivity is not adjusted. As a result, when the subject is moving at a high speed and the digital still camera 100 is also moving with the subject, it is possible to prevent image quality deterioration due to unnecessarily increasing the ISO sensitivity. In addition, it is possible to obtain a more appropriate captured image by using together the scene determination and the ON / OF control of the camera shake correction realized by the conventional technology.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

It is a block diagram which shows the hardware constitutions of the digital still camera concerning one Embodiment of this invention. It is a functional block diagram which shows schematic structure for implement | achieving the blurring correction function of the digital still camera concerning the embodiment. It is a flowchart which shows the flow of the imaging process performed by the digital still camera concerning one Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Digital still camera 101 Motion detection part 102 Camera shake correction process part 103 Subject motion calculation part 104 ISO sensitivity control part 110 Imaging process part 111 Optical block 112 Drive part 113 Imaging element 114 Analog processing circuit 120 Image processing part 130 Control part 131 MPU
132 System bus 133 ROM
134 RAM
135 Operation Input Interface 136 Display Control Unit 137 Memory Card Interface 138 Angular Velocity Sensor 139 External Input / Output Interface 141 Operation Input Unit 142 Display Unit 143 Memory Card

Claims (6)

An imaging device for imaging a subject with an imaging device,
A motion detector for detecting a motion vector of the imaging device;
A subject motion calculator that calculates a motion vector of the subject based on image data acquired by the imaging device;
The motion vector of the imaging device detected by the motion detector is compared with the motion vector of the subject calculated by the subject motion calculator, and it is determined whether to change the ISO sensitivity based on the comparison result An ISO sensitivity control unit for controlling the ISO sensitivity;
An imaging processing unit that performs imaging processing based on the ISO sensitivity controlled by the ISO sensitivity control unit;
An imaging apparatus comprising: The ISO sensitivity control unit
The ISO sensitivity is changed when it is determined that the motion vector of the imaging device calculated by the motion detection unit and the motion vector of the subject calculated by the subject motion calculation unit are not in the same direction. The imaging device according to claim 1. The ISO sensitivity control unit
When it is determined that the motion vector of the imaging device and the motion vector of the subject are not in the same direction, based on the result of comparing the magnitude of the motion vector of the imaging device and the magnitude of the motion vector of the subject, The imaging apparatus according to claim 2, wherein it is determined whether to change the ISO sensitivity. The ISO sensitivity control unit
The motion vector of the imaging device and / or the motion vector of the subject is compared with a predetermined threshold value, and it is determined whether to change the ISO sensitivity based on the comparison result. 4. The imaging device according to any one of 3.   The image processing apparatus according to claim 1, further comprising a shake correction processing unit that corrects a shake of the image data based on a motion vector of the imaging device detected by the motion detection unit. Imaging device. An imaging method in an imaging apparatus for imaging a subject with an imaging device,
A motion detection step of detecting a motion vector of the imaging device;
A subject motion calculation step of calculating a motion vector of the subject based on image data acquired by the imaging device;
A vector comparison step of comparing the motion vector of the imaging device detected in the motion detection step with the motion vector of the subject calculated in the subject motion calculation step;
An ISO sensitivity control step for determining whether to change the ISO sensitivity based on the comparison result in the vector comparison step and controlling the ISO sensitivity;
An imaging step of performing an imaging process based on the ISO sensitivity controlled in the ISO sensitivity control step;
The imaging method characterized by including.

Images