JP2012090216A - Imaging device and control method for imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device to solve the problem of conventional techniques in which for example when a main subject is small with respect to a screen, inability to acquire a sufficient motion vector increases an error to thereby affect correction, the imaging device being configured to determine reliability of a motion vector and not to perform shake correction when the reliability is low.SOLUTION: An imaging device comprises: a first imaging optical system for imaging a subject to output a first image signal; a second imaging optical system for imaging the subject to output a second image signal; means for detecting a motion vector from the second image signal; and control means for performing motion correction on the first image signal according to the detected motion vector, comprises a function for determining reliability of the motion vector from the second image signal, and comprises control means that does not perform motion correction with the motion vector when the reliability of the motion vector is low. Also, the reliability of the motion vector is determined from the ratio of the subject to an image.

Description

  The present invention relates to an imaging apparatus and a method for controlling the imaging apparatus.

  When shooting an object using an imaging device, if the imaging device is not fixed to a tripod or the like, for example, if it is hand-held or taken from inside a running car, the imaging screen will be blurred. In order to correct the shake of the image pickup screen, as disclosed in Patent Document 1, the image is picked up by an image pickup device having a large image pickup surface in advance, the image shake is detected from the picked up image, and the image shake is read out from the image pickup device. There is a disclosure of a method of controlling the readout of the image sensor so as to correct the image and cutting out an image.

  In the above-mentioned patent, since a feedback system having a time delay when detecting blur from a read image is configured, the frequency characteristic of blur correction is lowered as a basic characteristic of the system, and during exposure, Cannot detect a motion vector and cannot be used for taking a still image. In order to reduce this time delay and acquire and correct a motion vector even during still image shooting, the first imaging element for imaging and, for example, a pixel that can be read out at high speed, as described in Patent Document 2, for example. There is a disclosure of a method using a second image sensor for detecting a motion vector of ¼.

JP-A-4-91573 Japanese Patent Laid-Open No. 11-1555096

However, in the prior art disclosed in the above-mentioned patent document, a sufficient motion vector cannot be acquired depending on the size of the main subject, the focal length, the contrast of the image, and the like, and the error becomes large, which affects the correction. For example, when the second image sensor is a wide-angle single focus, the accuracy of the motion vector is low on the telephoto side.
Accordingly, an object of the present invention is to provide an imaging apparatus that determines the reliability of a motion vector and does not perform blur correction when the reliability is low.

  In order to solve the above problems, in the present invention, a first imaging optical system that images a subject and outputs a first image signal, and a second imaging optical that images the subject and outputs a second image signal. A system, means for detecting a motion vector from the second image signal, and control means for performing motion correction on the first image signal according to the detected motion vector, And a control unit that does not perform motion correction using the motion vector when the motion vector reliability is low.

  The reliability of the motion vector is determined from the ratio of the subject image.

  According to the present invention, it is possible to provide a captured image that is not blurred even in a situation where it is difficult to acquire a motion vector.

It is a figure which shows the structural example of the imaging device of this embodiment. It is a figure which shows the example of the operation processing flow of the imaging device which concerns on Example 1 of this invention. FIG. 3 is a diagram illustrating Example 1. It is a figure which shows the example of the operation processing flow of the imaging device which concerns on Example 2 of this invention. FIG. 5 is a diagram illustrating an example of camera shake correction processing according to the first embodiment, for example, camera shake correction processing in step S2, step S6, and step S9 in FIG. .

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration example of an imaging apparatus according to the present embodiment. The imaging apparatus shown in FIG. 1 includes a first imaging optical system 1, a second imaging optical system 2, a control unit 3, a video signal processing unit 4, a display unit 5, and a plurality of imaging optical systems. A vector detection circuit 6, a zoom switch 7, an external input / output terminal unit 8, a shutter release switch 9, a storage unit 10, a power supply unit 11, and an angular velocity sensor 12 are provided.

  The first imaging optical system 1 and the second imaging optical system 2 have the same configuration. Specifically, the first imaging optical system 1 includes a zoom lens 13, a shutter / diaphragm unit 14, a shift lens 15, a focus lens 16, an imaging element 17, an imaging signal processing unit 18, a zoom drive control unit 19, a shutter / control unit. An aperture drive control unit 20, a shift lens drive control unit 21, and a focus drive control unit 22 are provided. The second imaging optical system 2 includes a zoom lens 23, a shutter / aperture unit 24, a shift lens 25, a focus lens 26, an imaging element 27, an imaging signal processing unit 28, a zoom drive control unit 19, and a shutter / aperture drive control unit. 20, a shift lens drive control unit 21 and a focus drive control unit 22.

  The optical system provided in each imaging optical system has a three-group configuration. A description will be given using the first imaging optical system 1. The zoom lens 13 is a first group lens involved in zoom control. The shift lens 15 is a second group lens that performs shake correction. The focus lens 16 is a third group lens that performs focus adjustment processing. The zoom lens 13 is configured to be able to change the position along the optical axis direction, and changes the magnification. The zoom drive control unit 19 drives the zoom lens 13. A shutter / aperture unit 14 arranged at the rear stage of the zoom lens 13 adjusts the exposure amount. The shutter / aperture drive control unit 20 is an exposure adjustment unit that controls the drive of the shutter / aperture unit 14 to adjust the exposure amount, that is, the exposure adjustment process. The shift lens 15 is disposed so as to be able to change the position in a plane substantially perpendicular to the optical axis, and constitutes a shake correction optical system. The shift lens drive control unit 21 controls driving of the shift lens 15. The focus lens 16 is a focus adjustment lens. The focus lens 16 is configured so that its position can be changed along the optical axis direction. The focus drive control unit 22 has a function as a focus adjustment unit that drives and controls the focus lens 16 to execute a focus adjustment process. These configurations are the same in the second imaging optical system.

  The image sensors 17 and 27 receive the light image that has passed through the first group lens to the third group lens, and convert the received light image into an electrical signal. Here, the imaging element 17 used in the first imaging optical system 1 is an imaging unit of the present invention and corresponds to a video signal generation unit. The imaging element 27 used in the second imaging optical system 2 is an imaging element that captures an image for detecting a motion vector, and has a ¼ pixel number of the imaging element 17 for photographing. . The imaging signal processing units 18 and 28 convert the electrical signals output from the imaging elements 17 and 27, respectively, into video signals, and output the converted video signals as image data.

  The video signal processing unit 4 performs predetermined processing on the image data output from the imaging signal processing unit 18 to obtain an image signal that can be displayed on the display unit 5, and outputs the image signal. The zoom lens 13 to the video signal processing unit 4 have a function as image capturing means for capturing image data. Further, based on the image data obtained from the imaging signal processing unit 28, the motion vector detection unit 6 acquires a motion vector. The zoom lens 23 to the motion vector detection unit 6 have a function as an imaging unit that captures image data at a speed higher than that during exposure of the first imaging optical system 1 and acquires a motion vector. In this embodiment, the number of pixels of the image sensor for motion vector detection has the number of pixels of 1/4, but the number of pixels can be arbitrarily selected.

  The control unit 3 controls the entire system. Specifically, the control unit 3 includes zoom drive control units 19 and 29, shutter / aperture drive control units 20 and 30 provided in each imaging optical system, shift lens drive control units 21 and 31, focus drive control unit 22 and 32, the imaging elements 17 and 27, the imaging signal processing units 18 and 28, the video signal processing unit 4 and the motion vector detection unit 6 are controlled. The control unit 3 controls the display unit 5, the external input / output terminal unit 7, the storage unit 9, and the angular velocity sensor 12. The control unit 3 executes processing according to a program interpreted and executed by a CPU (Central Processing Unit) (not shown) or the like.

  As an operation peculiar to the present embodiment, the control unit 3 issues a command to the shift lens drive control units 21 and 31 based on the shake amount detected by the motion vector detection unit 6 or the angular velocity sensor 12 described later. Further, the control unit 3 instructs the display unit 5 to display the image signal output from the video signal processing unit 4 on the display screen. The display unit 5 displays an image signal on the screen for each imaging optical system. The display unit 5 executes screen display processing in accordance with instructions from the control unit 3.

The motion vector detection unit 6 detects a motion vector of the image from the image data output by the imaging signal processing unit 28 and transmits the information to the control unit 3. As a motion vector detection method, the present embodiment employs a matching method in which an image in the current field and an image in the previous field are compared and contrasted for each predetermined representative point or pixel. Specifically, the image of the previous field is shifted horizontally and vertically in the screen with respect to the image of the current field, and the motion vector is obtained with a shift amount and a direction in which both match most closely. FIG. 3 shows temporal relationships of images output from the image sensor 17 for photographing and the image sensor 27 for motion vector detection for explaining the motion vector detection of the present embodiment. When the image pickup device 17 for shooting takes a single still image, the image sensor 27 for motion vector detection has 4 pixels because the number of pixels is 1/4 compared to the image pickup device 17 for shooting as described above. One image can be acquired. A motion vector is detected by pattern matching of these four images. A command can be issued to the shift lens drive control unit 31 in accordance with the detected motion vector, and an imaging apparatus capable of correcting blur during shooting can be realized.
The zoom switch 7 is an operation means for operating the zoom lenses 13 and 23. The zoom switch 7 inputs an operation signal in response to a user operation input, and transmits the input operation signal to the control unit 3. The external input / output terminal unit 8 mediates communication with an external device (not shown). Specifically, a video signal and an audio signal are input to the external input / output terminal unit 8. In addition, a video signal and an audio signal are output from the external input / output terminal unit 8.

  The shutter release switch 9 is configured such that a first switch (hereinafter referred to as “SW1”) and a second switch (hereinafter referred to as “SW2”) are sequentially turned on in accordance with the amount of pressing. Specifically, when the user presses the shutter release button 9 about half, SW1 is turned on. When the user further depresses the shutter release button 9 to the end, SW2 is turned on. Then, a signal indicating that SW1 and SW2 are turned on is transmitted to the control unit 3.

  The storage unit 10 stores image data output from the imaging signal processing unit 18, image signals output from the video signal processing unit 4, and the like. The storage unit 10 also includes a memory device for storing a control program for the imaging apparatus according to the present embodiment, which is interpreted and executed by the control unit 3. The power supply unit 11 supplies a power supply voltage to each processing unit of the imaging apparatus illustrated in FIG. The angular velocity sensor 12 is a sensor having a dual configuration for the horizontal direction and the vertical direction.

  The control method of the imaging apparatus of this embodiment is implement | achieved by each process part with which the imaging apparatus shown in FIG. The functions of the imaging apparatus shown in FIG. 1 are realized by a CPU and a computer program executed on the CPU. The computer program can be stored in a computer-readable recording medium, provided by being recorded on the recording medium, or provided by transmission / reception using a network via a communication interface.

  Next, automatic focus adjustment (AF) processing, automatic exposure (AE) processing, and zoom drive control processing by the imaging apparatus shown in FIG. 1 will be described. As described above, SW1 and SW2 are sequentially turned on in accordance with the pressing amount of the shutter release switch 9. When a signal indicating that the SW1 of the shutter release switch 9 is turned on is input to the control unit 3, the control unit 3 instructs the focus drive control units 22 and 32 to determine the index determined by the control unit 3. The AF process is executed for the target. Upon receiving the instruction from the control unit 3, the focus drive control units 22 and 32 drive the focus lens to execute the AF process for the index. Further, the control unit 3 instructs the shutter / aperture drive control units 20 and 30 to execute the AE process for the index determined by the control unit 3. Specifically, the shutter / aperture drive control units 20 and 30 drive the shutter / aperture units 14 and 24 to set the exposure amount to an appropriate value.

When SW2 is turned on and a signal indicating that SW2 is turned on is input to the control unit 3, the control unit 3 instructs the image pickup device 17 to electrically generate an optical image received by the image pickup device 17. Convert to signal. The imaging signal processing unit 18 outputs image data based on the electrical signal. The video signal processing unit 4 outputs an image signal to be displayed on the display unit 5 based on the image data obtained from the imaging signal processing unit 18. The control unit 3 stores the image data output from the imaging signal processing unit 18 and the image signal output from the video signal processing unit 4 in the storage unit 10.
When the user operates the zoom switch 7, the operation signal is input to the control unit 3, and the control unit 3 gives instructions to the zoom drive control units 19 and 29. The zoom drive control units 19 and 29 move the zoom lenses 13 and 23 to the zoom position instructed by the control unit 3. Thereby, zoom drive control processing is executed.

  Hereinafter, an operation processing flow of the imaging apparatus according to the first embodiment of the present invention will be described with reference to FIG. In the figure, “YES” indicates a positive determination result, and “NO” indicates a negative determination result.

  FIG. 2A shows an example of the entire operation process of the imaging apparatus. First, the control unit 3 of the imaging apparatus shown in FIG. 1 confirms that the shooting mode has been set according to the user's operation input, and starts the shooting process. First, in step S1, a flag (Flag) for determining the reliability of the motion vector is initialized. In step S2, camera shake correction processing is performed. In next step S3, the control unit 3 determines whether SW1 of the shutter release switch 8 is turned on. If the control unit 3 determines that SW1 has been turned on, the process proceeds to step S4. If it is determined that SW1 is not in the on state, that is, it is in the off state, the process returns to step S2 and the camera shake correction process is continued. In step S4, motion vector reliability determination processing is performed. This process will be described later.

  In step S5, preparations for photographing such as AE processing and AF processing are performed. Subsequently, in step S6, camera shake correction processing is performed again. In the next step S7, the control unit 3 determines whether SW2 of the shutter release switch 9 is turned on. If the control unit 3 determines that SW2 is turned on, the process proceeds to step S8. When the control unit 3 determines that SW2 is not in an on state, that is, is in an off state, the control unit 3 returns to step S3 and determines whether SW1 is in an on state. In step S8, the imaging optical system of the imaging apparatus starts shooting. Even during shooting, camera shake correction processing is performed in step S9. This is because, since the image sensor 27 for motion vectors is provided separately from the image sensor 17 for photographing, camera shake correction can be performed even during exposure. In step S10, it is determined whether the exposure time has elapsed. If the exposure time has not yet elapsed, the process returns to step S9 to perform camera shake correction processing. If the exposure has elapsed, in step S11, the imaging optical system executes image processing such as image capture. Then, the video signal processing unit 4 generates and outputs an image signal that can be displayed on the display unit 5. In step S12, the control unit 3 stores the image signal in the storage unit 10 and ends the shooting.

  FIG. 2B shows an example of the motion vector reliability determination process in step S4 of FIG. Here, a method for determining the reliability of the motion vector according to what proportion of the captured image the main subject area occupies will be described.

  First, in step S21, a main subject area is determined. For example, in the case of an imaging device having a face detection function, the main subject may be determined based on the information. FIG. 3 shows two patterns of images in which the main subject is determined based on the face area. FIG. 3A shows a case where the subject is small, and FIG. 3B shows a case where the subject is large. Reference numeral 301 denotes a captured image, and 302 denotes a main subject area.

  In step S22, the proportion of the image occupied by the main subject region 302 is calculated. In step S23, it is determined whether or not the ratio of the main subject area 302 obtained in step S22 exceeds a predetermined value. In the present embodiment, the above ratio is 1/9 of the captured image. Here, as shown in FIG. 3A, when the ratio of the main subject region 302 does not satisfy 1/9 of the photographed image, it is determined that the reliability of the motion vector is low. The flag for judging the reliability is set to 1, and the process is terminated. This is because if the main subject area is small, the size of the obtained motion vector is also small, and errors are likely to be carried. On the other hand, if it accounts for 1/9 or more as shown in FIG. 3B, it is determined that the reliability of the motion vector is high, and the processing is terminated without changing the flag.

  Here, the reliability of the motion vector is determined from the proportion of the main subject in the image, but it may also be determined from the contrast or focal length of the image.

  FIG. 2C shows an example of camera shake correction processing in step S2, step S6, and step S9 in FIG. In this example, it is assumed that there is a means capable of detecting the shake amount from the angular velocity sensor 12 in addition to the motion vector.

  First, in step S31, a flag for determining the reliability of the motion vector is read. If the flag remains 0, it is determined that a highly reliable motion vector can be obtained, the motion vector is calculated in the next step S32, the blur amount is calculated in step S33, and the process proceeds to step S35. If the flag is not 0, it is determined that the reliability of the motion vector is low, and the shake amount is acquired from the angular velocity sensor 12 in step S34. In step S35, based on the amount of shake obtained in step S33 or step S34, the control unit 3 issues a command to the shift lens drive control units 21 and 31 to drive the shift lenses 15 and 25 and perform shake correction. The process is terminated.

  Here, when it is determined that the reliability of the motion vector is low, information from the angular velocity sensor is used. However, other methods may be used, or camera shake correction may not be performed.

  In the first embodiment, the method for determining the presence / absence of camera shake correction processing from the reliability of the motion vector has been described. However, in an imaging apparatus having a mode in which settings are selected according to the situation, camera shake correction processing is performed depending on the mode. You may judge the decision.

  Hereinafter, with reference to FIG. 4, an operation processing flow of the imaging apparatus for determining presence / absence of camera shake correction according to a mode according to the second embodiment of the present invention will be described. In the figure, “YES” indicates a positive determination result, and “NO” indicates a negative determination result.

  FIG. 4 shows an example of the entire operation process of the imaging apparatus. First, the control unit 3 of the imaging apparatus shown in FIG. 1 confirms that the shooting mode has been set according to the user's operation input, and starts the shooting process. First, in step S41, a flag (Flag) for determining the reliability of the motion vector is initialized. In a succeeding step S2, it is determined whether or not the macro mode is set. If it is determined that the mode is the macro mode, the process proceeds to step S44 as it is. If not, the process proceeds to step S43 and the flag is set to 1. This is because there is a strong influence of shift blur, which is easy to obtain a motion vector during macro shooting, but other factors cause more blur. In the next step S44, blur correction processing is performed.

  In step S3, the control unit 3 determines whether SW1 of the shutter release switch 8 is turned on. When the control unit 3 determines that the SW1 is turned on, the process proceeds to step S46. If it is determined that SW1 is not on, that is, is off, the process returns to step S41. Since the processing from step S46 to the end of shooting, which is the processing after it is determined that SW1 is in the on state, is the same as the processing from step S4 to the end of shooting described in the first embodiment, the description is omitted.

Here, the macro mode is given as a mode for determining the presence or absence of blur correction processing, but there is no problem in other modes. For example, a mode in which the presence / absence of shake correction can be simply selected may be used.
Also, a setting mode in which the shutter speed is fast such as outdoors during the day may be used. This is to prevent the motion vector calculation process from being in time for the shutter speed.

  In the first embodiment and the second embodiment, the method of acquiring the camera shake amount using either the motion vector or the output information of the physical sensor when performing the camera shake correction processing has been described. However, the camera shake amount is based on both pieces of information. You may get

  Hereinafter, with reference to FIG. 5, an operation processing flow of the imaging apparatus that acquires and corrects the amount of camera shake from the information of both the motion vector and the physical quantity sensor according to the third embodiment of the present invention will be described. In the figure, “YES” indicates a positive determination result, and “NO” indicates a negative determination result.

  FIG. 5 illustrates an example of the camera shake correction process described in the first embodiment, for example, the camera shake correction process in step S2, step S6, and step S9 in FIG.

  First, the shake amount is acquired from the angular velocity sensor 12 in step S61. In step S62, a flag for determining the reliability of the motion vector is read. If the flag remains 0, it is determined that a highly reliable motion vector can be obtained, the motion vector is calculated in the next step S63, the shake amount is calculated in step S64, and the process proceeds to step S65. If the flag is not 0, it is determined that the reliability of the motion vector is low, and the process proceeds to step S65.

  In step S65, based on the amount of shake obtained in this flow, the control unit 3 issues a command to the shift lens drive control units 21 and 31, drives the shift lenses 15 and 25 to perform shake correction, and ends the process. . Here, when it is determined that the reliability of the motion vector is low, correction is performed based only on the information of the angular velocity sensor, and when it is determined that the motion vector is reliable, the amount of shake obtained from the angular velocity sensor Then, correction is performed based on both the blur amount obtained from the motion vector.

Here, the information of both the motion vector and the angular velocity sensor is used, but other sensors may be used, and a plurality of detection means may be used. In addition, the information from the angular velocity sensor is always used, but it may be used depending on the situation.
As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

DESCRIPTION OF SYMBOLS 1 1st imaging optical system 2 2nd imaging optical system 3 Control part 11 Angular velocity sensor 12 Motion vector detection part 13 Shift lens 20 Shift lens drive control part

Claims (9)

A first imaging optical system that images a subject and outputs a first image signal;
A second imaging optical system that images a subject and outputs a second image signal;
Means for detecting a motion vector from the second image signal;
Control means for performing motion correction on the first image signal in accordance with the detected motion vector,
A function of determining the reliability of the motion vector from the second image signal;
An imaging apparatus comprising: control means that does not perform motion correction by the motion vector when the reliability of the motion vector is low. The imaging apparatus according to claim 1, wherein the reliability of the motion vector is determined from a ratio of an image of a subject. The imaging apparatus according to claim 1, wherein the reliability of the motion vector is determined from image contrast. The imaging apparatus according to claim 1, wherein the reliability of the motion vector is determined from a frame rate of the first imaging optical system. A first imaging optical system that images a subject and outputs a first image signal;
A second imaging optical system that images a subject and outputs a second image signal;
Means for detecting a motion vector from the second image signal;
Control means for performing motion correction on the first image signal in accordance with the detected motion vector,
When shooting, it has a mode in which settings are selected according to the situation,
An imaging apparatus comprising: a control unit that does not perform motion correction using the motion vector depending on the mode. The imaging apparatus according to claim 5, wherein the mode is a macro mode. The imaging apparatus according to claim 5, wherein the mode is a mode in which motion correction is not performed. It has a physical sensor that physically detects the amount of shaking of the imaging device,
When the reliability of the motion vector is low and motion correction by the motion vector is not performed,
6. The imaging apparatus according to claim 1, further comprising a control unit that performs motion correction on the first image signal based on an output signal of the physical sensor. A physical sensor that physically detects the amount of shaking of the imaging device;
Control means for performing motion correction on the first image signal based on the output signal of the physical sensor,
Perform motion correction on the first image signal based on information from a plurality of shake amount detection means,
The imaging apparatus according to claim 1, further comprising a control unit that performs motion correction without using the motion vector information when the reliability of the motion vector is low.