JP4597073B2 - Camera shake correction method, camera shake correction apparatus, and imaging apparatus - Google Patents

Description

  The present invention relates to a camera shake correction method, a camera shake correction apparatus, and an imaging apparatus that correct camera shake occurring in image data sequentially obtained from an image sensor at a predetermined frame rate.

  For example, when shooting a video with a digital still camera or video camera with a video shooting function in hand, the position of the still object image shown in the image of a certain frame is the position of the next frame when the hand holding the camera shakes. It moves on the screen, making it difficult to see the video. Therefore, a movement vector of an image of the next frame is detected with respect to an image of a certain frame, and camera shake correction is performed so that the image does not shake.

FIG. 4 is an explanatory diagram illustrating the principle of detecting a movement vector when performing camera shake correction.
In a digital camera, image data with the maximum resolution (hereinafter referred to as maximum image data) obtained from an image sensor is sequentially input to the camera shake correction apparatus at a predetermined frame rate during moving image shooting. In the camera shake correction apparatus, as shown in FIG. 4A, image data of a predetermined size that is cut out for recording as one frame of a moving image in the maximum image data 61 of the nth frame obtained from the image sensor. Hereinafter, a cutout position of the recording image data) is determined, and the image data 66 at the cutout position is cut out and recorded on a recording medium or the like. In this case, if the image data in the block 62 indicated by the predetermined address in the entire image data 61 of the nth frame shown in FIG. 4A is the reference image data, the entire n + 1th frame shown in FIG. 6B. If the comparison image data cut out by the block 64 indicated by the same predetermined address as the block 62 in the image data 63 is the same as the reference image data, the whole image data 63 is not shaken with respect to the whole image data 61. Become.

  However, if camera shake has occurred, the reference image data in the block 62 and the comparison image data cut out in the block 64 do not match. Therefore, in the camera shake correction apparatus, the block 64 in the entire image data 63 is converted into the X direction (horizontal direction) and the Y direction (vertical direction) like the blocks 64a, 64b, 64c,. ), The comparison image data in each of the blocks 64a, 64b, 64c,... Is compared with the reference image data, and the comparison image data having the highest correlation with the reference image data is cut out. Find the position.

  In the example shown in FIG. 4B, if the comparison image data having the highest correlation with the reference image data in the block 62 is cut out in the block 64c, the difference between the entire image data 63 and the entire image data 61 is “k”. This k becomes a movement vector between the entire image data 61 and the entire image data 63. Therefore, in the camera shake correction apparatus, as shown in FIG. 4C, the recording image data cutout position set in the entire image data 63 is moved by the movement vector k from the position where the image data 66 is cut out. Then, if the image data 67 is cut out from this cut-out position and recorded on a recording medium or the like, image data in which camera shake is corrected can be obtained. Since the cutout position can be shifted within the range between the end of the recording image data cut out from the image data of the first frame and the end of the whole image data, this range performs camera shake correction. This is the range of camera shake correction that can be performed.

  However, if the cutout position of the image data 67 is greatly shifted from the center of the whole image data 63 and the cutout position of the image data 67 reaches the corner of the whole image data 63, the cutout position cannot be moved any more. Camera shake correction cannot be performed. Further, the corners of the whole image data 63 are images of the periphery of the lens, and when the lens is small, the surrounding image is distorted. Therefore, it is not preferable to cut out the corner portion of the whole image data 63.

  Conventionally, a technique that limits the function of camera shake correction by changing the response characteristics of the camera shake correction device according to the correction angle, and reduces the discontinuous movement of image data that occurs due to the limited range of camera shake correction. Is known (see, for example, Patent Document 1).

JP 2001-42376 A

  However, in the above conventional apparatus, a compact digital camera is used in order to correct the movement of image data by moving the lens group constituting the camera shake correction unit in a direction orthogonal to the optical axis and decentering the optical axis. There is a situation that it is difficult to install at low price.

  The present invention has been made in view of the above circumstances, and is a camera shake correction capable of reducing discontinuous movement of image data caused by a limited camera shake correction range with a simple configuration. It is an object to provide a method, a camera shake correction device, and an imaging device.

The camera shake correction method of the present invention corrects camera shake occurring in the image data by controlling the first cutout position of the recording image data cut out from the image data sequentially obtained from the image sensor at a predetermined frame rate. A movement vector calculating step for obtaining a movement vector between the image data of the nth frame (n is a natural number) and the image data of the ( n + 1 ) th frame , and a movement based on the movement vector. A second cut-out position calculating step for obtaining a second cut-out position of the recording image data to be cut out from the image data of the (n + 1) th frame for canceling, and a third of the recording image data cut out from the image data of the first frame . A deviation amount calculating step for obtaining a deviation amount between the cutout position of the second cutout position and the second cutout position; By correcting the second cut-out position Flip, and a first cut-out position setting step of setting the first cut-out position of the recording image data cut out from the n + 1 th frame of the image data.

  In the camera shake correction method of the present invention, in the first cutout position setting step, the correction of the second cutout position is performed according to a function of the shift amount and the correction value of the second cutout position with respect to the shift amount. I do.

  In the camera shake correction method of the present invention, the function has a dead zone in which the correction value becomes zero when the shift amount is smaller than a threshold value.

The camera shake correction apparatus of the present invention corrects camera shake occurring in the image data by controlling the first cutout position of the recording image data cut out from the image data sequentially obtained from the image sensor at a predetermined frame rate. And a movement vector calculation means for obtaining a movement vector between image data of n (n is a natural number) frame and the image data of n + 1 frame , and a movement based on the movement vector. Second cut-out position calculating means for obtaining a second cut-out position of the recording image data to be cut out from the image data of the (n + 1) th frame in order to cancel, and a second cut-out position of the recording image data cut out from the image data of the first frame Shift amount calculating means for obtaining a shift amount between the cutout position of 3 and the second cutout position, and according to the shift amount, By correcting the second extraction position, and a first cut-out position setting means for setting the first cut-out position of the recording image data cut out from the n + 1 th frame of the image data.

  In the camera shake correction apparatus according to the present invention, the first cutout position setting unit corrects the second cutout position according to a function of the shift amount and the correction value of the second cutout position with respect to the shift amount. I do.

  In the camera shake correction apparatus of the present invention, the function has a dead zone in which the correction value becomes zero when the shift amount is smaller than a threshold value.

  The imaging device of the present invention is an imaging device having a moving image shooting function, and includes the camera shake correction device.

  According to the present invention, there is provided a camera shake correction method, a camera shake correction device, and an imaging device that can reduce discontinuous movement of image data that occurs due to a limited camera shake correction range with a simple configuration. can do.

FIG. 1 is a block diagram showing a schematic configuration of a digital camera as an example of an imaging apparatus for explaining an embodiment of the present invention.
A digital camera 100 shown in FIG. 1 includes an imaging unit 1 including a photographing lens 1a and an imaging element 1b, an analog signal processing unit 2, an A / D conversion unit 3, a driving unit 4, a digital signal processing unit 6, A compression / decompression processing unit 7, a display unit 8, a system control unit (CPU) 9, an internal memory 10, a media interface 11, a recording medium 12, an operation unit 13, and a camera shake correction device 15 are provided. The digital signal processing unit 6, the compression / decompression processing unit 7, the display unit 8, the system control unit 9, the internal memory 10, the media interface 11, and the camera shake correction device 15 are connected to the system bus 14.

  The imaging unit 1 images a subject by an optical system including a photographing lens 1a and an imaging element 1b such as a CCD type or CMOS type solid-state imaging element, and outputs an analog imaging signal. The analog signal processing unit 2 performs predetermined analog signal processing on the imaging signal obtained by the imaging unit 1. The A / D conversion unit 3 converts the analog signal processed by the analog signal processing unit 2 into a digital signal.

  When the digital camera 100 is set to a photographing mode (a mode in which a subject can be photographed and photographed image data can be recorded), the driving unit 4 is driven by a solid-state imaging device 1b, The analog signal processing unit 2 and the A / D conversion unit 3 are driven. The digital camera 100 can set a moving image shooting mode and a still image shooting mode.

  The digital signal processing unit 6 performs digital signal processing corresponding to the operation mode set by the operation unit 13 on the digital signal from the A / D conversion unit 3 to generate photographed image data. The processing performed by the digital signal processing unit 6 includes black level correction processing (OB processing), linear matrix correction processing, white balance adjustment processing, gamma correction processing, and synchronization processing. The digital signal processing unit 6 is configured by a DSP, for example. When the moving image shooting mode is set, the digital signal processing unit 6 sequentially outputs, for example, maximum resolution image data at a predetermined frame rate, and stores these in the internal memory 10.

  The compression / decompression processing unit 7 performs compression processing on the captured image data generated by the digital signal processing unit 6 and performs expansion processing on the compressed image data obtained from the recording medium 12.

  The display unit 8 includes an LCD display device, for example, and displays an image based on photographed image data that has been photographed and subjected to digital signal processing. An image is also displayed based on the image data obtained by decompressing the compressed image data recorded on the recording medium 12. In addition, it is possible to display a through image in the shooting mode, various states of the digital camera, information on operations, and the like.

  The system control unit 9 is mainly configured by a processor that operates according to a predetermined program, and performs overall control of the entire digital camera 100 including shooting operations.

  The internal memory 10 is, for example, a DRAM, and is used as a work memory for the camera shake correction device 15, the digital signal processing unit 6, and the system control unit 9, and temporarily stores captured image data recorded on the recording medium 12. It is also used as a buffer memory for storing and a buffer memory for image data for display on the display unit 8. The media interface 11 inputs / outputs data to / from a recording medium 12 such as a memory card.

  The operation unit 13 performs various operations when using the digital camera, and includes a release button (not shown) for instructing photographing.

FIG. 2 is a block diagram showing a schematic configuration of the camera shake correction apparatus 15 shown in FIG.
Movement vector between the image stabilizer 15 at time t _{n + 1 (n} is a natural number), the maximum image data obtained from the image sensor 1b at the time t _n, at time t _{n + 1} to the maximum image data obtained from the image pickup device 1b obtaining a moving vector calculation unit 21 to obtain the B n _{+ 1,} the cut-out position K _{n + 1} of the recording image data to be extracted from the maximum image data obtained from the image sensor 1b at the time t _{n + 1} in order to offset the movement based on movement vector B _{n + 1} a cutout position calculating section 22, at time t ₁ and the deviation amount calculating section 23 for obtaining the deviation amount L between the position K _{n + 1} Log and cut-out position K1 of the recording image data cut out from the maximum image data obtained from the imaging element 1b, shift the position K _{n + 1} cut in accordance with the shift amount determined by the amount calculator 22 corrects, shooting at time t _{n + 1} Set the clipping position K _{'n + 1} of the recording image data actually cut out from the maximum image data obtained from the element 1b, and a cutout position setting section 24 for outputting a clipping position signal of the recording image data in the internal memory 10 Prepare.

The cutout position K1 corresponds to the third cutout position in the claims, the cutout position Kn _{+ 1} corresponds to the second cutout position in the claims, and the cutout position _{K′n + 1} corresponds to the first cutout position in the claims. It corresponds to.

The movement vector calculation unit 21 calculates the movement vector B _{n + 1} by a known method.

Cutout position calculation unit 22, a moving vector _{B n + 1,} based on the extraction position K or extraction position K _{'n + 1,} obtaining the cut-out position _{K n + 1.}

The deviation amount calculation unit 23 compares the center of the cutout position K1 and the center of the cutout position _{Kn + 1} to obtain the deviation amount L. The shift amount L is represented by a numerical value indicating how many pixels are shifted in the horizontal direction and how many pixels are shifted in the vertical direction.

Cutout position setting section 24, the shift amount L, in accordance with the function of the correction value of the cut-out position K _{n + 1} for the shift amount L, it corrects the extraction position K _{n + 1,} the cut-out position K _{'n + 1} after the correction, the time t _{n + 1} is set as the cut-out position of the recording image data actually cut out from the maximum image data obtained from the image sensor 1b. The recording image data is cut out from the maximum image data obtained from the image sensor 1b at time t _{n + 1} by the cut-out position set here, and this is recorded on the recording medium 12. The recording image data may be cut out and recorded by the cut-out position setting unit 24, the system control unit 9, or the like.

This function has a dead zone in which the correction value becomes zero when the deviation amount L is equal to or less than the threshold. For example, a function as shown in FIG. 3 is used. FIG. 3 is a diagram illustrating a function of the horizontal component Lx of the shift amount and the correction value. In FIG. 3, the horizontal axis represents the horizontal component of the shift amount, and the vertical axis represents the correction value of the cutout position K _{n + 1} . The sign of the horizontal component Lx of the shift amount is attached to distinguish whether the cutout position _{Kn + 1} is shifted to the right or left in the horizontal direction when the cutout position K1 is used as a reference. That is, when the cutout position Kn _{+ 1} is shifted to the right in the horizontal direction when the cutout position K1 is used as a reference, the shift amount Lx is positive, and when the cutout position Kn _{+ 1} is shifted to the left in the horizontal direction, the shift amount Lx. Is negative. Cutout position setting section 24, the cutout position K _{n + 1,} by shifting the position K _{n + 1} cut by adding the correction value corresponding to the shift amount L, the correction of the cut-out position K _{n + 1.}

In the function shown in FIG. 3A, the correction value when the absolute value of the deviation amount Lx is smaller than 2 is 0 (dead zone), and the correction value when the deviation amount Lx is 2 or more is −1. The correction value when the deviation Lx is −2 or less is +1. In the function shown in FIG. 3B, the correction value when the absolute value of the deviation amount Lx is smaller than 2 is 0 (dead zone), and the correction value when the deviation amount Lx is 2 or more decreases monotonously from 0. However, the correction value when the deviation amount Lx is −2 or less increases monotonously. Note that the cutout position setting unit 24 corrects the vertical direction of the cutout position _{Kn + 1} by using a function as shown in FIG.

Incidentally, the cutout position setting section 24 for the maximum image data obtained from the image sensor 1b at time t _1, regardless of the above-described processing, sets the extraction position K1 predetermined. The extraction position K1 is normally set at time t ₁ in the central portion of the maximum image data obtained from the image sensor 1b.

  Next, the operation of the camera shake correction device 15 will be described. Hereinafter, a case where the cutout position setting unit 24 corrects the cutout position using the function illustrated in FIG. 3A will be described as an example.

When the first maximum image data obtained from the image sensor 1b at time t ₁ is input, the cutout position setting unit 24 sets the position K1 cut. As a result, the recording image data is cut out from the cut-out position K1 set as the first maximum image data, and the cut-out recording image data is recorded on the recording medium 12.

Next, the second largest image data obtained from the image sensor 1b at time t ₂ is entered, motion vector calculation unit 21, between the first maximum image data and the second maximum image data calculating the moving vector _{B 2.}

Then, the cutout position calculating section 22, based on the position K1 Log and motion vector B _2, cut out of the recording image data to be cut out from the second up image data to compensate for movement by the moving vector B ₂ calculating the position _{K 2.}

Then, the deviation amount calculating section 23 calculates a deviation amount L1 between a center of the cut-out position K ₂ of the cut-out position K1.

Then, the cutout position setting section 24 corrects the position K ₂ cut in accordance with the displacement amount L1, the extraction position K of the recording image data actually cut out from the maximum image data obtained from the image sensor 1b at time t ₂ setting the _'2.
For example, if the horizontal and vertical components of the displacement amount L1 is 3, respectively, cut-out position K of the corrected addition position correction value -1 in the horizontal direction and the vertical direction of the cut-out position K _{2 '2} and then, if the horizontal and vertical components of the displacement amount L1 was respectively 1, cut-out position K _'2 of the corrected horizontal and a value obtained by adding the correction value 0 in the vertical direction of the cut-out position K ₂ and To do. As a result, the recording image data is cut out from the cut-out position K ′ ₂ set as the second maximum image data, and the cut-out recording image data is recorded on the recording medium 12.

Next, the third maximum image data obtained from the image sensor 1b at time t ₃ is input, motion vector calculation unit 21, between the second maximum image data and the third maximum image data calculating the moving vector _{B 3.}

Next, the cut-out position calculation unit 22 records image data to be cut out from the third maximum image data in order to cancel the movement based on the movement vector B ₃ based on the movement vector B ₃ and the cut-out position K ′ _2. calculating a segmentation position _{K 3.}

Then, the deviation amount calculating section 23 calculates a deviation amount L2 between a center of the cutout position K ₃ of the cutout position K1.

Then, the cutout position setting section 24 corrects the position K ₃ cut according to the deviation amount L2, extraction position K of the recording image data actually cut out from the maximum image data obtained from the image sensor 1b at time t ₃ 'Set to ₃ . As a result, the recording image data is cut out from the cut-out position K ′ ₃ set as the third maximum image data, and the cut-out recording image data is recorded on the recording medium 12.

Even when the maximum image data is input at time t ₄ later, repeat this process.

As described above, when the cutout position K1 set for the first maximum image data is determined, the cutout position _{Kn + 1} is sequentially moved thereafter. A value indicating how far the cutout position Kn _{+ 1} is away from the cutout position K1 is the shift amount L. When the shift amount L is 0, the cutout position Kn _{+ 1} matches the cutout position K1. means.

If the amount of deviation L is, for example, 10 in the horizontal direction and 10 in the vertical direction, it means that the cutout position K _{n + 1} is around the entire image data. Conventionally, since the cut out image data for recording from the extraction position K _{n + 1,} it is likely to cut-out position K _{n + 1} from sticking to the end of the entire image data. On the other hand, according to the present embodiment, when the cut-out position K _{n + 1} is away from the center, the cut-out position of the recording image data is set to the end of the whole image data because it is corrected to approach the center. It can be suppressed compared with the past. Therefore, it is possible to reduce the discontinuous movement of the image caused by the limited correction range with a simple configuration.

1 is a block diagram showing a schematic configuration of a digital camera as an example of an imaging apparatus for describing an embodiment of the present invention. The block diagram which shows schematic structure of the camera-shake correction apparatus shown in FIG. The figure which shows the function for correcting the clipping position Explanatory drawing which shows the principle which detects the movement vector when performing camera shake correction

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up part 1a Shooting lens 1b Image pick-up element 2 Analog signal processing part 3 A / D conversion part 4 Drive part 6 Digital signal processing part 7 Compression / decompression processing part 8 Display part 9 System control part 10 Internal memory 11 Media interface 12 Recording medium 13 Operation unit 14 System bus 15 Camera shake correction device 21 Movement vector calculation unit 22 Cutout position calculation unit 23 Deviation amount calculation unit 24 Cutout position setting unit

Claims (7)

A camera shake correction method for correcting camera shake occurring in the image data by controlling a first cutout position of recording image data cut out from image data sequentially obtained from an image sensor at a predetermined frame rate,
a movement vector calculation step for obtaining a movement vector between the image data of the nth frame (n is a natural number) and the image data of the ( n + 1 ) th frame ;
A second cut-out position calculating step for obtaining a second cut-out position of the recording image data to be cut out from the image data of the (n + 1) th frame in order to cancel the movement based on the movement vector;
A shift amount calculating step for obtaining a shift amount between the third cutout position of the recording image data cut out from the image data of the first frame and the second cutout position;
A first cutout position setting step of correcting the second cutout position according to the shift amount and setting the first cutout position of the recording image data cut out from the image data of the (n + 1) th frame. Camera shake correction method. The camera shake correction method according to claim 1,
In the first cutout position setting step, a camera shake correction method for correcting the second cutout position according to a function of the shift amount and a correction value of the second cutout position with respect to the shift amount. The camera shake correction method according to claim 2,
A camera shake correction method, wherein the function has a dead zone in which the correction value becomes zero when the deviation amount is smaller than a threshold value. A camera shake correction device that corrects camera shake occurring in the image data by controlling a first cutout position of recording image data cut out from image data sequentially obtained from an image sensor at a predetermined frame rate,
movement vector calculation means for obtaining a movement vector between image data of n (n is a natural number) frame and the image data of n + 1 frame ;
Second cut-out position calculating means for obtaining a second cut-out position of recording image data to be cut out from the image data of the (n + 1) th frame in order to cancel the movement based on the movement vector;
A deviation amount calculating means for obtaining a deviation amount between a third cutout position of the recording image data cut out from the image data of the first frame and the second cutout position;
First cut position setting means for correcting the second cut position according to the shift amount and setting the first cut position of the recording image data cut out from the image data of the (n + 1) th frame. Camera shake correction device. The camera shake correction device according to claim 4 ,
The camera shake correction apparatus in which the first cutout position setting unit corrects the second cutout position according to a function of the shift amount and the correction value of the second cutout position with respect to the shift amount. The camera shake correction device according to claim 5,
The camera shake correction apparatus according to claim 1, wherein the function has a dead zone in which the correction value becomes zero when the shift amount is smaller than a threshold value. An imaging device having a video shooting function,
An imaging device comprising the camera shake correction device according to claim 4.

Images