JP5415975B2 - Motion compensation apparatus and method - Google Patents

Description

  The present invention relates to a motion correction apparatus and method for obtaining a motion between images from a plurality of different image information and correcting the display position of the image from the motion information.

  When shooting with a device held in a hand, such as a video camera or a digital camera, there is a possibility of blurring in the direction shown in FIG.

Here, the case where the camera 1 translates in the horizontal direction is taken as the X axis, the case where the camera 1 translates in the vertical direction is taken as the Y axis, and the case where the camera 1 is translated in the front-rear direction is taken as the Z axis.
Further, the rotational movement about the X axis is the θx axis, the rotational movement about the Y axis is the θy axis, and the rotational movement about the Z axis is the θz axis.

  Conventionally, as an electronic image blur correction method by obtaining a movement amount between different images and moving an image display area in a direction opposite to the movement amount, two axes of an X axis and a Y axis of the image are used. What is corrected is common.

  2A to 2D are diagrams illustrating examples of image changes caused by movements of two axes, the X axis and the Y axis, the Z axis, the θz axis, the θx axis, and the θy axis.

FIG. 2A shows an image of X-axis and Y-axis correction.
When the camera is actually shaken, it is the X axis and the Y axis that are most likely to occur as the moving direction. Therefore, there are the most shake correction devices and methods that only need to correct these axes.

In recent years, a method for correcting other axes has been proposed in order to perform correction with higher accuracy.
FIG. 2B shows a case where a deviation occurs in the Z-axis (zoom) direction.
Since the Z axis is shifted from the direction of the optical axis of the camera, the size of the subject changes when this axis moves.

FIG. 2C shows a case where a deviation occurs in the θz axis (rotation) direction.
In the figure, the center of the screen is the center of rotation, but there is a pattern in which the center of rotation is outside the screen range.
As a method for correcting both the Z axis and the θz axis, a technique disclosed in Patent Document 1 has been proposed.

FIG. 2D shows a case where blurring occurs in the θx-axis (X-axis rotation) and θy-axis (Y-axis rotation) directions.
Regarding the blurring of this axis, in the figure, the center of the screen is the center of rotation of the θx axis and the θy axis, but there is a pattern in which the center of blurring is outside the screen range.
As a method for correcting the θx axis and the θy axis, a technique disclosed in Patent Document 2 has been proposed.

Patent No. 3770271 JP 2006-222933 A

As described above, in the natural world, there is a possibility that 6-axis blur occurs, and so far, proposals on how to detect and correct the movement amount in each axis have been made individually.
When detecting and correcting the movement amount of a certain axis, the best accuracy is obtained when there is no movement of the other axes. However, the actual blur in the natural world is a mixture of multiple blurs. Become.
For this reason, when blur correction is performed for each axis, the accuracy changes depending on the order in which the movement amount is detected and corrected. In some cases, it is difficult to perform highly accurate correction due to erroneous detection or the influence of a moving subject.

  However, it has not been proposed so far as to what correction should be performed when detecting and correcting the movement amount of each of the six axes in order.

  It is an object of the present invention to provide a motion correction apparatus and method capable of realizing high-accuracy correction by removing erroneous detection and removing the influence of a moving subject when correcting 6-axis blurring. is there.

  A motion correction apparatus according to a first aspect of the present invention includes a motion detection unit that detects motion between temporally continuous images, and the position and / or size of the image so as to cancel the motion detected by the motion detection unit. A motion correction unit including a function of correcting the image, wherein the motion detection unit and the motion correction unit are: a horizontal axis direction of the image, a rotation direction about the horizontal axis, a vertical axis direction, and a rotation about the vertical axis. A function for dividing a direction, an optical axis direction in which an image is picked up, and a rotation direction about the optical axis into two groups, a first group and a second group, and detecting and correcting motion for each divided group The first group includes four directions of a horizontal axis direction, a vertical axis direction, an optical axis direction, and a rotation direction about the optical axis, and the second group includes a rotation direction about the horizontal axis, And vertical axis Comprises two direction of rotation of the attached, after the detection and correction of motion for each direction of the first group, to detect and correct the motion in each direction of the second group.

  Preferably, the motion detection unit and the motion correction unit perform motion detection and correction in the optical axis direction and in the rotation direction with respect to the optical axis when performing motion detection and correction in the first group. Thereafter, motion detection and correction in the horizontal axis direction and the vertical axis direction are performed.

  Preferably, the order of the direction in which the motion detection unit and the motion correction unit perform the detection and correction of the motion of the first group is not fixed.

  Preferably, the motion detection unit performs motion detection in the rotation direction and the optical axis direction about the optical axis based on the spatial frequency information.

  Preferably, the motion detection unit obtains a plurality of pieces of motion information in motion detection in the rotational direction about the horizontal axis and the rotational direction about the vertical axis, and excludes information of a large variation amount, The motion correction unit performs motion correction using the remaining motion information after being excluded by the motion detection unit.

  The motion correction method according to the second aspect of the present invention includes a motion detection step for detecting a motion between temporally continuous images, and the position and / or size of the image so as to cancel the motion detected by the motion detection unit. A motion correction step including a function of correcting the image, and in the motion detection step and the motion correction step, the horizontal axis direction of the image, the rotation direction about the horizontal axis, the vertical axis direction, and the rotation about the vertical axis Dividing the six directions of the direction, the optical axis direction where the image is captured, and the rotation direction about the optical axis into two groups of the first group and the second group, and performing motion detection and correction for each of the divided groups, The first group includes four directions of a horizontal axis direction, a vertical axis direction, an optical axis direction, and a rotation direction about the optical axis, and the second group is a rotation direction about the horizontal axis. And in the motion detection step and the motion correction step, motion detection and correction are performed for each direction of the first group, and then each direction of the second group is detected. Detects and corrects movements.

  According to the present invention, when correcting the 6-axis blur, it is possible to eliminate the erroneous detection and to remove the influence of the moving subject, thereby realizing a highly accurate correction.

It is a figure for demonstrating the direction axis | shaft of camera shake. It is a figure which shows the example of the change of the image by the motion of an X-axis, a Y-axis, and an axis other than that. 1 is a block diagram illustrating a configuration example of an image processing device employing an image motion correction device according to a first embodiment of the present invention. FIG. 3 is a first diagram illustrating a positional relationship between amplitude and phase information when an actual image is Fourier-transformed and arranged in a frequency space. FIG. 2 is a second diagram showing the positional relationship between amplitude and phase information when an actual image is Fourier-transformed and arranged in a frequency space. It is a figure which shows typically the moving amount | distance with respect to the horizontal frequency and vertical frequency of (theta) z axis and Z axis. It is a figure for demonstrating the case where the θx axis moves. It is a figure for demonstrating the process which detects and correct | amends the movement amount of (theta) x axis | shaft and (theta) y axis | shaft using the characteristic that it moves centering on the center of an image after correcting other 4 axes. 4 is a flowchart for explaining an operation of the image processing apparatus according to the first embodiment. It is a block diagram which shows the structural example of the image processing apparatus which employ | adopted the image motion correction apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows an image when only the moving amount | distance of (theta) x remains after 4-axis correction. A process in which a value larger than the assumed movement amount of the θz axis and the θy axis is erroneously detected or is determined to be a moving subject, and is treated as being out of correction data, and the movement amount is detected and corrected. It is a flowchart for demonstrating.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 3 is a block diagram illustrating a configuration example of an image processing apparatus employing the image motion correction apparatus according to the first embodiment of the present invention.

As shown in FIG. 3, the image processing apparatus 100 includes an image input unit 110, a θz-axis and Z-axis movement amount calculation unit 120, a θz-axis and Z-axis correction unit 130, an X-axis and Y-axis movement amount calculation unit 140, And an X-axis and Y-axis correction unit 150.
The image processing apparatus 100 includes a θx-axis and θy-axis movement amount calculation unit 160, an abnormal value determination and movement amount calculation unit 170, a θx-axis and θy-axis correction unit 180, and an image output unit 190.

The image processing apparatus 100 is configured to be able to correct almost all blurring of images existing in the natural world.
The image processing apparatus 100 has a horizontal axis direction (X-axis direction), a rotation direction (θx-axis direction) about the horizontal axis (X-axis), a vertical axis direction (Y-axis direction), and a vertical axis (Y-axis). Rotation direction (θy axis direction), optical axis direction (Z axis) where the image is taken, and rotation direction (θz axis direction) about the optical axis (Z axis) It has a function of dividing into two groups of GRP2 and performing motion detection and correction for each divided group.
In the image processing apparatus 100, the first group GRP1 includes a horizontal axis direction (X-axis direction), a vertical axis direction (Y-axis direction), an optical axis direction (Z-axis direction), and a rotation direction about the optical axis (θz axis). Includes four directions.
The second group GRP2 includes two directions: a rotation direction about the horizontal axis (θx axis direction) and a rotation direction about the vertical axis (θy axis direction).
The image processing apparatus 100 has a function of detecting and correcting movement in each direction of the second group GRP2 after detecting and correcting movement in each direction of the first group GRP1. .

When the image processing apparatus 100 according to the first embodiment detects and corrects motion in the first group GRP1, the optical axis direction (Z-axis direction) and the rotation direction about the optical axis (θz-axis direction) ) Motion and correction are performed, and then motion detection and correction are performed in the horizontal axis direction (X-axis direction) and vertical axis direction (Y-axis direction).
The image processing apparatus 100 has a function of performing motion detection in the rotation direction (θz axis) and the optical axis direction (Z axis) about the optical axis based on the spatial frequency information.
In addition, the image processing apparatus 100 obtains the amount of movement of the object at the four corners of the image from information between different consecutive images, and performs projection conversion on the image based on the obtained amount of movement, so that all the moving directions can be obtained. It is configured to enable correction.

  The image input unit 110 receives an input signal based on the captured image (video), and inputs the uncorrected image IM1 that has not yet been corrected, as the θz axis and Z axis movement amount calculation unit 120, and the θz axis and Z axis. This is supplied to the correction unit 130.

  The θz-axis and Z-axis movement amount calculation unit 120 obtains (using) the movement amounts of the θz-axis and Z-axis of the uncorrected image IM1 based on the spatial frequency information, and the result is sent to the θz-axis and Z-axis correction unit 130. Supply.

The θz-axis and Z-axis correction unit 130 determines the position and / or size of the image so as to cancel the movement according to the movement amounts of the θz-axis and the Z-axis obtained by the θz-axis and Z-axis movement amount calculation unit 120. Correct.
The θz-axis and Z-axis correction unit 130 supplies the corrected image to the X-axis and Y-axis movement amount calculation unit 140 and the X-axis and Y-axis correction unit 150 as a biaxial correction image IM2.

  The X-axis and Y-axis movement amount calculation unit 140 obtains (using) the movement amounts of the X-axis and Y-axis of the biaxial correction image IM2 based on the spatial frequency information, and obtains the result as the X-axis and Y-axis correction unit 150. To supply.

The X-axis and Y-axis correction unit 150 corrects the position of the image so as to cancel the movement according to the X-axis and Y-axis movement amounts obtained by the X-axis and Y-axis movement amount calculation unit 140.
The X-axis and Y-axis correction unit 150 supplies the corrected image as a 4-axis correction image IM4 to the θx-axis and θy-axis movement amount calculation unit 160 and the θx-axis and θy-axis correction unit 180.

  The θx-axis and θy-axis movement amount calculation unit 160 obtains the movement amounts of the θx-axis and the θy-axis of the 4-axis corrected image IM4, and obtains a plurality of pieces of movement information in the movement detection in the θx-axis direction and the θy-axis direction. The result is supplied to the abnormal value determination and movement amount calculation unit 170.

  The abnormal value determination and movement amount calculation unit 170 receives the information obtained by the θx-axis and θy-axis movement amount calculation unit 160, and obtains the movement amount information excluding the information of the abnormal value having a large variation amount as the θx axis and the θy axis. This is supplied to the correction unit 180.

The θx-axis and θy-axis correction unit 180 is configured to cancel the movement according to the information on the movement amounts of the θx axis and the θy axis from which the abnormal value information is excluded by the abnormal value determination and movement amount calculation unit 170. Correct the position and / or size.
The θx-axis and θy-axis correction unit 180 supplies the corrected image to the image output unit 190 as a six-axis correction image IM6.

  Hereinafter, blur correction on six axes in the first embodiment will be described in detail with reference to FIGS.

First, when 6-axis correction is performed, the same processing is performed for the movement amount detection / correction algorithm because the X-axis and Y-axis are different only in vertical and horizontal directions.
For the same reason, the same processing is performed for the algorithm for correcting the θx axis and the θy axis.
Therefore, when considering the order of correction processing, it is only necessary to consider the order of four processes of the X axis, Y axis, Z axis, θz axis, θx axis, and θy axis.

Generally, when correcting each axis, it is more accurate to detect the movement amount in a situation where there is no misalignment of the other axis than in the situation where the other axis is misaligned.
However, for the θz axis and the Z axis, the movement amount of the X axis and the Y axis can be ignored by obtaining the movement amount using the spatial frequency information, so that an accurate movement amount can be calculated. .
Therefore, in the first embodiment, the movement amount detection / correction of the θz axis and the Z axis is executed first.
4 and 5 are diagrams showing the positional relationship between the amplitude and the phase information when the real image is Fourier-transformed and arranged in the frequency space.
FIG. 6 is a diagram schematically showing the movement amount with respect to the horizontal frequency and the vertical frequency of the θz axis and the Z axis.

The DC component of the image as the origin of coordinates, how have the components of the horizontal frequency f _H and the vertical frequency f _V becomes the angular position from the origin. Therefore, as shown in FIG. 6, the change amount of the angle becomes the movement amount of the θz axis.
Further, as the distance from the origin is increased from near to far, the low-frequency to high-frequency components are indicated. As shown in FIG. 6, the amount of movement of the Z axis can be obtained from the distance difference information from the origin of this component. it can.

  By converting the image data into the frequency space in this way, accurate information on the θz axis and the Z axis can be obtained by eliminating movement amount information on the X axis and the Y axis.

Next, in the first embodiment, the X axis and the Y axis are corrected.
When the θz axis and the Z axis are corrected, the remaining blur axes become the four axes of the X axis and the Y axis, and the θx axis and the θy axis.
When these two are compared, the change amount of the θx axis and the θy axis is smaller than the change amount due to the movement of the X axis and the Y axis.

FIG. 7 is a diagram for explaining a case where the θx axis moves.
Here, as an example, a case where the θx axis is moved as shown in FIG. 7 will be described.
In FIG. 7, when the camera 200 is shaken from the top to the bottom of the θx axis, the angle of view moves from the triangular portion of the solid line A to the triangular portion of the broken line B.
In this case, the image position to be captured is an area as shown on the right side of the figure. In the upper part of the screen, the subject position is relatively close, and in the lower part of the screen, the subject position is relatively far away. Therefore, the image IM moves downward, the upper part of the screen is small, and the lower part of the screen is large. It becomes a trapezoid shape.

In other words, for example, when blurring as shown in FIG. 7 occurs in the vertical direction, only the θx axis moves as the camera 200, but the Y axis movement is larger as an image to be captured. Become.
In a camera having a lens with a general focal length of 35 mm (35 mm silver salt film size conversion), the vertical field angle is about 37 degrees, and when a movement amount of about 3.5 degrees occurs on the θx axis, The amount of movement of the Y axis is 10%, and the zoom magnification changes by 1.8%.
Although blurring is likely to occur in the case of a telephoto lens, in the case of telephoto, the amount of movement of the Y axis relative to the amount of movement of the θx axis is further increased.
Therefore, the degree of change in the θx axis and θy axis that affects the correction accuracy of the θz axis and the Z axis is smaller than that of the X axis and the Y axis.
In addition, it is advantageous to correct the X-axis and the Y-axis having a larger amount of change than the θx-axis and the θy-axis first to reduce the error.
Therefore, it is possible to accurately detect and correct the movement amount of the X axis and the Y axis.

Therefore, in the present embodiment, the movement amount detection and correction of the θx axis and the θy axis are finally performed among the six axes.
In this case, since the other four axes have already been corrected, the center of the image moves. Here, the movement amount detection and correction of the θx axis and the θy axis are performed using this characteristic.

  FIG. 8 is a diagram for explaining processing for detecting and correcting the movement amount of the θx axis and the θy axis by using the characteristic that the center of the image moves after the other four axes are corrected.

First, an image movement amount detection area for obtaining the movement amount of the four corners of the image is set.
Next, in each region, a movement amount at the coordinates is calculated from a plurality of movement amount detections.
And each moving amount calculated | required in each area | region is converted into the moving amount in four corners.
For example, as shown in FIG. 8, the coordinates of the center of the image are (0, 0), the coordinates of one corner of the image are (x, y), and the amount of movement obtained at the coordinates (m, n) for obtaining the amount of movement. Assuming that (Xm, Yn) and (Xc, Yc) are the movement amounts of the image center obtained by the above method, the movement amounts (XM, YN) at the corners of the four corners of the image are obtained as follows.

After that, a process for eliminating abnormal movement amount information is performed from the movement amounts of the plurality of feature points obtained, and the movement amounts at the four corners of the image are calculated by obtaining an average value.
More specifically, a value larger than the assumed movement amount of the θz axis and the θy axis is determined as a false detection or a moving subject from the obtained movement amounts of the plurality of feature points. The movement amount information is excluded as an abnormal value. This specific process will be described in detail later.
Thereafter, the moving amount at the four corners of the image is calculated by calculating an average value for the remaining values.

Thereafter, by performing projection conversion so that the coordinates are returned in the direction opposite to the movement amount based on the obtained movement amount, it is possible to correct all the movement directions.
Projection transformation is represented by the following equation, where the coordinates before transformation are (x, y) and the coordinates after transformation are (x ′, y ′).

  FIG. 9 is a flowchart for explaining the operation of the image processing apparatus according to the first embodiment.

First, captured images (videos) that are temporally continuous are acquired through the image input unit 110 (ST1, ST2), and the output image has not yet been corrected, and the θz axis and Z axis movement are performed as the uncorrected image IM1. The amount calculation unit 120 and the θz axis and Z axis correction unit 130 are supplied.
The θz-axis and Z-axis movement amount calculation unit 120 obtains the θz-axis and Z-axis movement amounts of the uncorrected image IM1 based on the spatial frequency information (ST3), and the result is sent to the θz-axis and Z-axis correction unit 130. Supplied.
In the θz-axis and Z-axis correction unit 130, the position and / or size of the image so as to cancel the movements according to the movement amounts of the θz-axis and the Z-axis obtained by the θz-axis and Z-axis movement amount calculation unit 120. Is corrected. In this example, the θz axis is corrected (ST4), and the Z axis is corrected (ST5). However, the order may be reversed.
From the θz-axis and Z-axis correction unit 130, the corrected image is supplied to the X-axis and Y-axis movement amount calculation unit 140 and the X-axis and Y-axis correction unit 150 as a biaxial correction image IM2.

In the X-axis and Y-axis movement amount calculation unit 140, the X-axis and Y-axis movement amounts of the biaxial correction image IM2 are obtained based on the spatial frequency information (ST6), and the result is the X-axis and Y-axis correction unit. 150.
In the X-axis and Y-axis correction unit 150, the position and / or size of the image is canceled out in accordance with the movement amounts of the X-axis and Y-axis obtained by the X-axis and Y-axis movement amount calculation unit 140. Is corrected (ST7).
From the X-axis and Y-axis correction unit 150, the corrected image is supplied as the 4-axis correction image IM4 to the θx-axis and θy-axis movement amount calculation unit 160 and the θx-axis and θy-axis correction unit 180.

The θx-axis and θy-axis movement amount calculation unit 160 obtains the movement amounts of the θx-axis and θy-axis of the 4-axis corrected image IM4 (ST8), and a plurality of pieces of motion information are detected in the motion detection in the θx-axis direction and the θy-axis direction. And the result is supplied to the abnormal value determination and movement amount calculation unit 170.
The abnormal value determination and movement amount calculation unit 170 receives the information obtained by the θx axis and θy axis movement amount calculation unit 160, excludes abnormal value information having a large variation amount, and the movement amount information after the exclusion becomes θx. Is supplied to the shaft and θy-axis correction unit 180 (ST9).
In the θx-axis and θy-axis correction unit 180, the image is canceled so as to cancel the movement according to the information on the movement amount of the θx-axis and the θy-axis from which the information on the abnormal value is excluded by the abnormal value determination and movement amount calculation unit 170. Is corrected in position and / or size (ST10).
From the θx-axis and θy-axis correction unit 180, the corrected image is supplied to the image output unit 190 as a six-axis correction image IM6, and image output is performed (ST11).
Then, the processes of steps ST2 to ST11 are repeated.

As described above, according to the first embodiment, rotation about the optical axis is performed in electronic blur correction in which blur (movement amount) of an image is detected and corrected from information between different consecutive images. Direction blur (θz-axis direction) and zoom direction (Z-axis direction) are detected and corrected first, then blurs in the X-axis direction and Y-axis direction are detected and corrected. Processing is performed in the order of detecting and correcting the blur of the Y-axis rotation, that is, the blur in the θx-axis and θy-axis directions.
Thereby, it becomes possible to correct with respect to all the moving directions with high precision.
Further, as a method of detecting the movement amount in the θz-axis direction and the Z-axis direction, the movement amount is obtained using the spatial frequency information, so that correction of other axes can be performed with high accuracy.
In addition, when detecting shakes in the rotation of the X-axis and the Y-axis, high-precision shake correction is obtained by obtaining a plurality of pieces of movement information and excluding those that have an abnormally large value. Can be performed.

[Second Embodiment]
FIG. 10 is a block diagram illustrating a configuration example of an image processing apparatus employing the image motion correction apparatus according to the second embodiment of the present invention.

The difference between the image processing apparatus 100A according to the second embodiment and the image processing apparatus 100 according to the first embodiment is that the order of the directions in which the first group of motions is detected and corrected is unfixed and arbitrary. There is.
Correspondingly, in the image processing apparatus 100A of FIG. 10, the movement amount calculation unit and the correction unit include the X axis, Y axis, θz axis, and Z axis movement amount calculation unit 210 and the X axis, Y axis, θz axis, and A Z-axis correction unit 220 is shown.

As described above, when six-axis correction is performed, the same processing is performed for the movement amount detection and correction algorithm because only the vertical and horizontal differences are made for the X-axis and the Y-axis.
For the same reason, the same processing is performed for the algorithm for correcting the θx axis and the θy axis.
Therefore, when considering the order of correction processing, it is only necessary to consider the order of four processes of the X axis, Y axis, Z axis, θz axis, θx axis, and θy axis.

  In the second embodiment, the first group GRP1A is corrected by correcting the θx axis and the θy axis after detecting and correcting the blur of the four axes of the X axis, the Y axis, the Z axis, and the θz axis. The four-axis correction method (order) is not limited.

  The reason why the movement amount of the θx axis and the θy axis becomes small after detecting and correcting the shake of the four axes of the X axis, the Y axis, the Z axis, and the θz axis is as follows. .

As described above, when the blurring as shown in FIG. 7 occurs in the vertical direction, the camera 200 moves only on the θx axis, but as the image to be captured, the movement on the Y axis becomes larger. .
In a camera having a general lens with a focal length of 35 mm, the vertical field angle is about 37 degrees, and when a movement amount with a field angle ratio of 10% occurs on the Y axis, the movement amount of θx is about 3. 5 degrees.
Although blurring is likely to occur in the case of a telephoto lens, similarly in the case of telephoto, the movement amount of θx is further reduced when a movement amount with an angle of view ratio of 10% occurs on the Y axis.

FIG. 11 is a diagram showing an image when only the moving amount of θx remains after the 4-axis correction.
Due to the movement of θx, the movement amount on the screen becomes the position of the intersection of the solid line and the dotted line in FIG.
Under the same conditions as in FIG. 7, when the movement amount of θx is 3.5 degrees, the movement amount on the screen is the movement amount of only the θx axis when moving far from the movement amount of 10% of the Y axis. Is about 2.5%. When moving closer, it becomes even smaller.

Thus, the movement amount of the θz axis and the θy axis is smaller than the movement amount of the X axis and the Y axis, and the movement of the θz axis and the θy axis is determined from the focal length of the lens and the movement amount information of the X axis and the Y axis. It is possible to assume how much the quantity should be below.
Therefore, the value larger than the assumed movement amount of the θz axis and the θy axis is erroneously detected or is determined to be a moving subject, and is treated as a target for correction data, thereby detecting the movement amount. Accuracy can be improved.

  Here, a value larger than the assumed movement amount of the θz axis and the θy axis is erroneously detected or determined to be a moving subject, and the movement amount is detected and corrected by treating it as out of correction data. The processing in the case will be described.

  FIG. 12 shows that a value larger than the assumed movement amount of the θz axis and the θy axis is erroneously detected or is determined to be a moving subject, and is treated as an object of correction data, and the movement amount is detected and corrected. It is a flowchart for demonstrating the process in the case of performing.

Detection points are acquired from the processing results of the θx-axis and θy-axis movement calculation unit 160 (ST21), and the amount of movement of the detection points is obtained (ST22).
Abnormal value determination and movement amount calculation section 170 determines whether or not the detected movement amount is larger than the assumed detection value (ST23).
If the detected movement amount is smaller than (or less than) the assumed detection value, it is determined that the detected movement amount is not an abnormal value, and the detected movement amount is registered as movement amount data (ST24).
When the detected movement amount is larger than the assumed detection value, the detected movement amount is not registered as the movement amount data because it is an abnormal value.
The processes from step ST22 are repeated until the above process completes the detection of the movement amounts of all feature points (ST25).
Then, the processing from step ST21 is repeated until the detection of the feature points at the four corners of the image is completed (ST26).
When the detection of the feature points at the four corners of the image is completed in step ST26, the movement amounts of the four corners of the image are obtained (ST27), and blur correction is performed by projection conversion (ST28).

  This process can also be applied to the first embodiment.

As described above, according to the second embodiment, in electronic blur correction in which blurring (movement amount) of an image is detected and corrected from information between different images that are temporally continuous, When performing 6-axis blurring correction that corrects the moving direction, the horizontal axis direction (X-axis direction), vertical axis direction (Y-axis direction), optical axis direction (Z-axis direction), and optical axis, regardless of the order After detecting and correcting the blurring of the four axes in the four directions in the θz-axis direction and correcting the blurring of the rotations in the X-axis and the Y-axis (bluring in the θx-axis and θy-axis directions).
Accordingly, it is possible to perform blur correction with high accuracy by utilizing the fact that the image after the 4-axis correction moves about the center of the screen.
In addition, when detecting the rotational shake of the X-axis and the Y-axis, a plurality of movement amount information is obtained. Correction can be performed.

  DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus, 110 ... Image input part, 120 ... θz-axis and Z-axis movement amount calculation part, 130 ... θz-axis and Z-axis correction part, 140 ... X-axis and Y Axis movement amount calculation unit, 150 ... X axis and Y axis correction unit, 160 ... θx axis and θy axis movement amount calculation unit, 170 ... Abnormal value determination and movement amount calculation unit, 180 ... θx Axis and θy axis correction unit, 190... Image output unit, 210... X axis, Y axis, θz axis and Z axis movement amount calculation unit, 220... X axis, Y axis, θz axis and Z axis Correction unit.

Claims (10)

A motion detector that detects motion between temporally continuous images;
A motion correction unit including a function of correcting the position and / or size of the image so as to cancel the motion detected by the motion detection unit,
The motion detection unit and the motion correction unit are
The six directions of the horizontal axis direction of the image, the rotation direction about the horizontal axis, the vertical axis direction, the rotation direction about the vertical axis, the optical axis direction where the image is captured, and the rotation direction about the optical axis are defined as the first group. Dividing into two groups of the second group, and having a function of detecting and correcting motion for each of the divided groups,
The first group includes four directions of a horizontal axis direction, a vertical axis direction, an optical axis direction, and a rotation direction about the optical axis,
The second group includes two directions of a rotation direction about the horizontal axis and a rotation direction about the vertical axis,
A motion correction device that detects and corrects motion in each direction of the second group after detecting and correcting motion in each direction of the first group. The motion detection unit and the motion correction unit are
When performing motion detection and correction in the first group,
After detecting and correcting the movement of the optical axis direction and the rotational direction of the optical axis,
The motion correction apparatus according to claim 1, wherein motion detection and correction are performed in a horizontal axis direction and a vertical axis direction. The motion detection unit and the motion correction unit are
The motion correction apparatus according to claim 1, wherein the order of directions in which the first group of motions is detected and corrected is not fixed. The motion detector is
The motion correction device according to any one of claims 1 to 3, wherein motion detection in the rotation direction and the optical axis direction is performed based on spatial frequency information with respect to the optical axis. The motion detector is
In the motion detection in the rotational direction about the horizontal axis and the rotational direction about the vertical axis, after obtaining a plurality of motion information, exclude information of a large amount of variation,
The motion correction unit is
The motion correction apparatus according to claim 1, wherein motion correction is performed using the remaining motion information after being excluded in the motion detection unit. A motion detection step for detecting motion between temporally continuous images;
A motion correction step including a function of correcting the position and / or size of the image so as to cancel the motion detected by the motion detection unit,
In the motion detection step and the motion correction step,
The six directions of the horizontal axis direction of the image, the rotation direction about the horizontal axis, the vertical axis direction, the rotation direction about the vertical axis, the optical axis direction where the image is captured, and the rotation direction about the optical axis are defined as the first group. Dividing into two groups of the second group, performing motion detection and correction for each of the divided groups,
The first group includes four directions of a horizontal axis direction, a vertical axis direction, an optical axis direction, and a rotation direction about the optical axis,
The second group includes two directions of a rotation direction about the horizontal axis and a rotation direction about the vertical axis,
In the motion detection step and the motion correction step,
A motion correction method that detects and corrects motion in each direction of the second group and then detects and corrects motion in each direction of the second group. In the motion detection step and the motion correction step,
When performing motion detection and correction in the first group,
After detecting and correcting the movement of the optical axis direction and the rotational direction of the optical axis,
The motion correction method according to claim 6, wherein the motion is detected and corrected in the horizontal axis direction and the vertical axis direction. In the motion detection step and the motion correction step,
The motion correction method according to claim 6, wherein the order in which the first group of motions is detected and corrected is unfixed. In the motion detection step,
The motion correction method according to any one of claims 6 to 8, wherein the rotation direction of the optical axis and the motion detection in the optical axis direction are performed based on spatial frequency information. In the motion detection step,
In motion detection in the rotational direction about the horizontal axis and the rotational direction about the vertical axis, information on a large amount of variation is excluded after obtaining a plurality of motion information,
In the motion correction step,
The motion correction method according to claim 6, wherein motion correction is performed using the remaining motion information after being excluded in the motion detection step.

Images