JP2010166257A - Camera shake correcting device, imaging device, camera shake correcting method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera shake correcting device, an imaging device, a camera shake correcting method, and a program capable of suppressing recording of unnatural images accompanied with camera shake correction. <P>SOLUTION: The camera shake correcting device includes a movement detector, a camera shake corrector 15, a photography operation determiner 32, and a camera shake determiner 36. The movement detector (video image codec 16 and sensor group 38) detects the amount of movements of a video camera 10. The camera shake corrector 15 adjusts two or more different high/low response characteristics and corrects camera shake. The photography operation determiner 32 determines the start and the end of photography operation from the detected value of the amount of movements. At first the response characteristic is beforehand established as "low", then the camera shake determiner 36 increase or decrease an evaluation period so as to cope with a change in an interval when photography operations are performed, by setting the response characteristic as "high" if the detected value of the amount of movements at the start of the photography operation exceeds a predetermined threshold, or by setting again the response characteristic as "low" if the detected value of the amount of movements in a predetermined evaluation period T from the end of the photography operation is under the predetermined threshold. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a camera shake correction apparatus, an imaging apparatus, a camera shake correction method, and a program.

  Video cameras, still cameras, and the like that are reduced in size and weight have a camera shake correction function to correct image shake caused by camera shake that occurs during shooting. However, it is known that in a video camera having a camera shake correction function, an unnatural image is recorded at the start of a shooting operation such as a pan operation or a tilt operation.

  This is because the camera shake correction is fully functioning at the start of the shooting operation, so that the intentional movement of the video camera at the start of the shooting operation is judged as a camera shake, and the movement of the video camera is cancelled. It is due to that. As a result, the image at the start of the shooting operation is recorded as an unnatural image that follows the shooting operation after a while from the start of the shooting operation.

  For this reason, in Patent Document 1 described below, camera shake is corrected so that an unnatural image does not stand out by shifting the reading position from the image sensor during camera shake and using an interpolated image by electronic zoom during shooting operation.

  In Patent Document 2 described below, camera shake is corrected so as to suppress unnatural image shooting by switching response characteristics of camera shake correction between camera shake and shooting operation.

  In the following Patent Document 3, a movement vector of an image is detected, and if the movement vector in a certain period is less than a predetermined threshold value, camera shake correction is turned off to suppress unnatural image shooting. Has been corrected.

JP-A-10-210350 Japanese Patent No. 3473105 JP 2008-60799 A

  However, the camera shake correction methods described in Patent Documents 1 and 2 require a certain determination period in order to determine the camera shake and the shooting operation. For this reason, the camera shake correction is completely functioning during the determination period, and if a photographing operation is performed during the determination period, the same problem as in the conventional case occurs.

  On the other hand, with the camera shake correction method described in Patent Document 3, the conventional problem is solved to some extent. However, if a sudden shooting operation is repeated at an interval shorter than a certain period, the camera shake correction is not turned off, and the camera operation is performed with the camera shake correction fully functioning. Problems arise. In addition, since camera shake correction is switched on / off, image shake due to camera shake is recorded when camera shake correction is off.

  The present invention has been made in view of the above problems, and its purpose is to provide a novel and improved camera shake correction apparatus, image pickup apparatus, camera shake correction method, which can suppress unnatural image recording associated with camera shake correction, And to provide a program.

  In order to solve the above-described problem, according to a first aspect of the present invention, a movement detection unit that detects a movement amount of an imaging device and a camera shake correction that corrects camera shake by adjusting two or more different high and low response characteristics. , A shooting operation determination unit that determines the start and end of the shooting operation from the detection value of the movement amount, a response characteristic is set to low in advance, and the detection value of the movement amount at the start of the shooting operation exceeds a predetermined threshold value A response characteristic setting unit configured to set the response characteristic to high again, and to set the response characteristic to low again when the detection value of the movement amount in the predetermined evaluation period is less than the predetermined threshold from the end of the photographing operation. There is provided a camera shake correction apparatus including an evaluation period setting unit that increases or decreases an evaluation period so as to correspond to an increase or decrease in an interval at which photographing operations are performed.

  According to such a configuration, when the response characteristic is set to low in advance and the detected value of the movement amount at the start of the photographing operation exceeds the predetermined threshold, the response characteristic is set to high. If the detected value of the movement amount in the predetermined evaluation period from the end of the photographing operation is less than the threshold value, the response characteristic is set to low again.

  As a result, the response characteristics are set to low in advance, so that at the start of a sudden shooting operation, unnatural image recording associated with camera shake correction is suppressed compared to when camera shake correction is fully functioning. can do. In the evaluation period after the end of the sudden photographing operation, the response characteristic is set to be high, so that the camera shake that occurs after the sudden photographing operation can be corrected appropriately. After the evaluation period, if a large amount of movement is not detected during the evaluation period, the response characteristic is set to low again, so at the start of the next sudden shooting operation, an unnatural image associated with camera shake correction will be displayed. Recording can be suppressed.

  Also, the evaluation period is set to be increased or decreased so as to correspond to the increase or decrease of the interval at which the photographing operations are performed. As a result, when the shooting operations are performed at short / long intervals, the evaluation period is set to be short / long, so that the probability that the response characteristic is set to low at the start of the next sudden shooting operation. Becomes high, and it becomes easy to suppress the recording of an unnatural image accompanying the camera shake correction.

  Further, the evaluation period setting unit may set the evaluation period from an interval between a predetermined number of photographing operations in the latest period or an interval between photographing operations within the most recent predetermined period.

  Further, the evaluation period setting unit may set the evaluation period to a predetermined upper limit value when the set evaluation period exceeds a predetermined upper limit value.

  Further, the response characteristic setting unit starts the predetermined evaluation period instead of setting the response characteristic low when the detected value of the movement amount in the predetermined evaluation period is less than the predetermined threshold from the end of the photographing operation. The response characteristic may be set so that the response characteristic gradually changes from high to low during the period from the end to the end.

  Further, according to the second embodiment of the present invention, an imaging device including the imaging device and the camera shake correction device according to the first embodiment is provided. In addition, according to the third embodiment of the present invention, a camera shake correction method applicable to the camera shake correction apparatus according to the first embodiment is provided. According to the fourth embodiment of the present invention, there is provided a program for causing a computer to execute the camera shake correction method according to the third embodiment.

  According to the present invention, it is possible to provide a camera shake correction device, an imaging device, a camera shake correction method, and a program capable of suppressing unnatural image recording associated with camera shake correction.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[1. Configuration of imaging device]
First, the configuration of an imaging apparatus according to an embodiment of the present invention will be described with reference to FIG. The imaging device is a device capable of recording a moving image, such as a video camera, a still camera, or a mobile phone. Hereinafter, a case where the imaging device is a video camera will be described as an example.

  As shown in FIG. 1, the video camera 10 includes a control unit 12, an imaging unit 14, a video codec 16, a display unit 18, an audio input unit 20, an audio codec 22, an audio output unit 24, a format processing unit 26, and a reproduction / recording unit. 28, an operation interface (I / F) 30, a shooting operation determination unit 32, a shooting operation information holding unit 34, and a camera shake determination unit 36.

  The control unit 12 includes a CPU, a ROM, a RAM, and the like, and performs processing necessary for the operation of the video camera 10 and controls each unit connected via the bus 40. The CPU reads a program or the like for performing a camera shake correction method, which will be described later, from the ROM, and develops and executes the program on the RAM.

  The imaging unit 14 includes an imaging lens, an imaging element, a lens driving mechanism, and the like, converts an imaging target into an electrical signal, and outputs the electrical signal to the video codec 16. The imaging unit 14 also performs a zoom operation of the imaging lens by driving the imaging lens in accordance with a zoom operation using the operation I / F 30. The camera shake correction unit 15 that forms a part of the image pickup unit 14 adjusts two or more different response characteristics, using electronic correction and / or optical correction, to correct camera shake. The camera shake correction unit 15 switches or adjusts the response characteristic of camera shake correction according to the response characteristic information notified from the camera shake determination unit 36.

The video codec 16 converts video information input from the imaging unit 14 into MPEG, AVC (Advanced Video).
Coding) and the like are encoded and output to the format processing unit 26. The video codec 16 also outputs the motion vector of the image obtained during the encoding process to the shooting operation determination unit 32 and the camera shake determination unit 36. The video codec 16 decodes the video data input from the format processing unit 26 into a video signal and outputs the video signal to the display unit 18. The display unit 18 is composed of a liquid crystal panel or the like, and outputs a video corresponding to the video signal input from the video codec 16.

The audio input unit 20 includes a microphone or the like, converts audio input from the subject to be captured into an audio signal, and outputs the audio signal to the audio codec 22 as audio information. The audio codec 22 converts audio information input from the audio input unit 20 into an AAC (Advanced Audio
Coding) and the like are encoded and output to the format processing unit 26. The audio codec 22 decodes the audio data input from the format processing unit 26 into an audio signal and outputs the audio signal to the audio output unit 24. The audio output unit 24 includes a speaker or the like, and outputs audio corresponding to the audio signal input from the audio codec 22.

  The format processing unit 26 receives video data and audio data from the video codec 16 and the audio codec 22 when recording a moving image. The format processing unit 26 creates predetermined moving image data from the video data and the audio data, and outputs it to the reproduction / recording unit 28.

  The playback / recording unit 28 writes the moving image data input from the format processing unit 26 to the recording medium, and outputs the moving image data read from the recording medium to the format processing unit 26. The operation I / F 30 includes operation buttons, operation keys, and the like, and outputs an operation signal corresponding to a user operation to the control unit 12 and the like. The operation I / F 30 includes a zoom lever for zoom operation, and outputs a zoom detection value that changes according to the operation of the zoom lever to the photographing operation determination unit 32.

  The shooting operation determination unit 32 calculates the movement amount of the video camera 10 by analyzing the motion vector input from the video codec 16 and analyzing the zoom detection value input from the operation I / F 30. The photographing operation determination unit 32 determines the state of the photographing operation from the movement amount. The shooting operation determination unit 32 creates shooting operation information indicating the status of the shooting operation including the start and end of the shooting operation from the movement amount, notifies the camera shake determination unit 36, and stores it in the shooting operation information holding unit 34. . Here, the shooting operation information is information indicating the status of shooting operations such as panning, tilting, dolly, and zooming performed on the video camera 10, and includes, for example, types of shooting operations / suddenness, start time, and end time. , Including spacing etc.

  The shooting operation information holding unit 34 is a storage medium that includes a memory or the like and sequentially stores shooting operation information input from the shooting operation determination unit 32.

  The camera shake determination unit 36 calculates the movement amount (camera shake amount) of the video camera 10 by analyzing the motion vector input from the video codec 16 and determines the movement state of the video camera 10 from the movement amount. The camera shake determination unit 36 sets camera shake correction response characteristics according to the shooting operation information notified from the shooting operation determination unit 32 or the movement state of the video camera 10 and notifies the image pickup unit 14 as response characteristic information.

  The camera shake determination unit 36 sets the response characteristic to low in advance, sets the response characteristic to “high” when the detection value of the movement amount at the start of the shooting operation exceeds a predetermined threshold, and ends the shooting operation. To a response characteristic setting unit that sets the response characteristic to “low” again when the detected value of the movement amount in the predetermined evaluation period T is less than the predetermined threshold value. In addition, the camera shake determination unit 36 also functions as an evaluation period setting unit that sets the evaluation period T to be increased or decreased so as to correspond to an increase or decrease in the interval at which the photographing operations are performed.

  Here, the amount of movement of the video camera 10 will be described with reference to FIG. As shown in FIG. 2, the optical axis direction of the lens is perpendicular to the Y axis and the Y axis, and the direction corresponding to the left and right direction of the video camera 10 is perpendicular to the X axis, the X axis, and the Y axis, and the vertical direction of the video camera 10 The corresponding direction is assumed to be the Z axis. According to the analysis of the motion vector, for example, the movement amount associated with the shooting operation for moving the video camera 10 in the X axis, Y axis, and Z axis directions, and the X axis, Y axis, and Z axis. The amount of movement associated with the shooting operation for rotating the video camera 10 is calculated.

  When creating the shooting operation information, the pan operation is determined from the rotational movement on the XY plane, and the tilt operation is determined from the rotational movement on the YZ plane. Further, the dolly operation is determined from the movement amount in the Y-axis direction, and the zoom operation is determined from the zoom detection value. When creating shooting operation information, in addition to pan, tilt, dolly, and zoom operations, it includes any combination of position, velocity, acceleration, direction, rotation, angular velocity, and angular acceleration with respect to the X, Y, and Z axes. A shooting operation involving a change in movement may be determined.

  Next, the configuration of a video camera 60 according to a modification of the embodiment of the present invention will be described with reference to FIG. As shown in FIG. 3, in the video camera 60, a sensor group 38 is added to the video camera 10 described above. The sensor group 38 includes an acceleration sensor, an angular velocity sensor, a gravity sensor, an azimuth sensor, and the like, and detects an acceleration value, an angular velocity value, a gravity value, an azimuth, and the like that change according to a photographing operation performed on the video camera 60. And output to the shooting operation determination unit 32 and the camera shake determination unit 36.

  In the video camera 60, the shooting operation determination unit 32 performs video analysis by analyzing detection values such as acceleration, angular velocity, gravity, and azimuth input from the sensor group 38, and analyzing zoom detection values input from the operation I / F 30. The movement amount of the camera 60 is calculated, and the state of the shooting operation is determined from the movement amount. In addition, the camera shake determination unit 36 calculates the movement amount of the video camera 60 by analyzing the detection values such as acceleration, angular velocity, gravity, and azimuth input from the sensor group 38, and determines the movement state of the video camera 60 from the movement amount. judge.

[2. Judgment method of shooting operation]
Next, a method for determining a shooting operation will be described with reference to FIGS. 4 and 5 schematically show time-series data of the moving amount and the cumulative moving amount of the video camera 10 at the time of camera shake and photographing operation. The movement amount of the video camera 10 is calculated from the analysis result of the video data or the detection value of the sensor group 38, and the cumulative movement amount is calculated as a cumulative value of the movement amount.

  Here, the movement amount of the video camera 10 includes the movement amount of the video camera 10 in the X-axis, Y-axis, and Z-axis directions, the rotational movement amount with respect to each axis, and any combination thereof. Hereinafter, the movement amount of the video camera 10 accompanying the pan operation and tilt operation will be described as an example, but other movement amounts can be described in the same manner.

  FIG. 4 shows a state ta1 during which a small camera shake occurs and a state ta2 during which a large camera shake occurs. Here, the magnitude of camera shake is classified according to whether or not the amount of movement of the video camera 10 due to camera shake exceeds a predetermined threshold value ± th1. In the periods ta1 and ta2, the movement amount varying in the + direction and the − direction from the reference line 0 is confirmed, but only a small cumulative movement amount less than a predetermined threshold ± th2 is confirmed. In addition, the movement amount exceeding the threshold ± th1 is not confirmed in the period ta1 in which the small camera shake occurs, but the movement amount exceeding the threshold ± th1 is confirmed in the period ta2 in which the large camera shake occurs.

  FIG. 5 shows a state during a period tb1 during which a gentle photographing operation is performed and a period tb2 during which a sudden photographing operation is performed. Here, the speed of the shooting operation is classified according to whether or not the movement of the video camera 10 by the shooting operation exceeds a predetermined threshold value ± th1. In periods tb1 and tb2, a movement amount that varies only in one direction from the reference line 0 (in the + direction in the example shown in FIG. 5) is confirmed, and a large cumulative movement amount exceeding a predetermined threshold + th2 is confirmed. In addition, during the period tb1 during which the gentle shooting operation is performed, a movement amount exceeding the threshold value ± th1 is not confirmed, but during the period tb2 during which the sudden shooting operation is performed, a movement amount exceeding the threshold value + th1 in the + direction. It is confirmed.

  For this reason, camera shake and photographing operation are determined from the tendency of the accumulated movement amount in the predetermined period t (= ta1, ta2, tb1, tb2), and the magnitude of the camera shake is determined from the movement amount tendency in the predetermined period t. It is discriminated or whether the photographing operation is slow or steep. Further, the start point and end point of the photographing operation are determined from a period in which the change tendency of the movement amount is less fluctuating and certain regularity is recognized. For example, the start time of the photographing operation is determined based on the amount of movement during the determination period tp to ta from the current time ta to a time tp that goes back a predetermined period t (or a period t ′ shorter than the predetermined period t), When the direction of the movement amount during the determination period is constant in either the + direction or the − direction, it is determined as the time point tp. Further, the end point of the shooting operation is determined as the time when the direction of the movement amount is reversed after the start of the shooting operation is determined or the time when the movement amount becomes zero.

  6 and 7 show an example of the shooting operation information indicating the determination result of the shooting operation. The shooting operation information includes information related to the type / slowness, start time, end time, and interval of the shooting operation, which is determined based on the movement amount and cumulative movement amount of the video camera 10 in a predetermined period t.

  In the example illustrated in FIGS. 6 and 7, a pan operation and a tilt operation are determined as the types of shooting operations. Here, the type of the shooting operation is determined from the moving direction specified from the video data or the analysis result of the detection value of the sensor group 38. For example, a movement on the XY plane is specified as a pan operation, and a movement on the YZ plane is specified as a tilt operation.

  Whether the shooting operation is slow or rapid is determined according to whether or not the amount of movement in a predetermined period t exceeds a predetermined threshold value ± th1. The start / end time of the shooting operation is determined from the change tendency of the movement amount in the predetermined period t. The interval between shooting operations is calculated as the difference between the end time of one shooting operation and the start time of the next shooting operation.

  Further, in the example illustrated in FIG. 6, all shooting operations are determined as sudden shooting operations. On the other hand, in the example illustrated in FIG. 7, some shooting operations (sixth shooting operations 1:10 to 1:13) are determined as sudden shooting operations, and other shooting operations are determined as slow shooting operations. Yes.

[3. Image stabilization method]
Next, the camera shake correction method according to the present embodiment will be described with reference to FIGS. FIG. 8 shows a flow showing the overall processing of the camera shake correction method. FIG. 9 is a schematic diagram for explaining the overall processing of the camera shake correction method. FIG. 10 shows a flow showing a partial process of the camera shake correction method.

  First, the overall processing of the camera shake correction method will be described with reference to FIGS. FIG. 9 shows a response characteristic switching state before and after a sudden photographing operation and a response characteristic switching state before and after a gentle photographing operation. When instructed to start recording a moving image via the operation I / F 30, the control unit 12 instructs each unit to start a recording operation.

  When the start of the recording operation is instructed, the camera shake determination unit 36 sets the response characteristic of camera shake correction to “low” (step S10), notifies the imaging unit 14 as response characteristic information, and the imaging unit 14 The response characteristic of the correction unit 15 is switched to “low”. Therefore, in the subsequent imaging processing, the imaging unit 14 outputs the video information of the imaging target to the video codec 16 while correcting the camera shake with the response characteristic “low” until the response characteristic is changed. Further, the camera shake determination unit 36 sets a predetermined evaluation period T described later to a predetermined upper limit value Tmax (S12).

  The imaging unit 14 outputs video information to be imaged to the video codec 16 while correcting camera shake, and the video codec 16 encodes the video information into video data and outputs the video data to the format processing unit 26. The audio input unit 20 outputs the audio information to be captured to the audio codec 22, and the audio codec 22 encodes the audio information into audio data and outputs it to the format processing unit 26. The format processing unit 26 creates predetermined moving image data from the video data and the audio data and outputs it to the reproduction / recording unit 28. The reproduction / recording unit 28 stores the moving image data in a recording medium.

  The video codec 16 outputs the motion vector of the image obtained during the encoding process to the shooting operation determination unit 32 and the camera shake determination unit 36. The operation I / F 30 outputs a zoom value that changes according to the operation of the zoom lever to the photographing operation determination unit 32. Then, the shooting operation determination unit 32 analyzes the motion vector and zoom value of the image, and calculates the movement amount, cumulative movement amount, and the like of the video camera 10.

  In the video camera 60 according to the modification shown in FIG. 2, the sensor group 38 outputs sensor values such as acceleration, angular velocity, gravity, and azimuth to the shooting operation determination unit 32 and the camera shake determination unit 36. The operation I / F 30 outputs a zoom value that changes according to the operation of the zoom lever to the photographing operation determination unit 32. Then, the shooting operation determination unit 32 analyzes the sensor value and the zoom value, and calculates the movement amount, cumulative movement amount, and the like of the video camera 60.

  The shooting operation determination unit 32 starts determining the shooting operation status from the movement amount, the accumulated movement amount, and the like (S14). In the determination of the shooting operation status, as described above, the camera shake and the shooting operation are determined from the accumulated movement amount in the predetermined period t, and the magnitude of the camera shake or the speed of the shooting operation is determined from the movement amount. Further, the start of the photographing operation is determined from a period in which the change tendency of the movement amount is less fluctuating and a certain regularity is recognized.

  The shooting operation determination unit 32 determines the start of the shooting operation until the start of the shooting operation is determined (S16). When the start of the shooting operation is determined in a predetermined period t immediately after the start of the shooting operation, the shooting operation determination unit 32 determines whether the started shooting operation is a sudden shooting operation (S18).

  Here, since the response characteristic is set to “low” in advance, at the start of a sudden shooting operation, an unnatural image recording accompanying the camera shake correction is performed as compared with the case where the camera shake correction is completely functioning. Can be suppressed. That is, even if it is determined that the intentional movement of the video camera 10 at the start of the photographing operation is a camera shake, the intentional movement is not completely canceled because the response characteristic of the camera shake correction is low. Accordingly, it is possible to suppress unnatural image recording that follows the shooting operation after a while from the start of the shooting operation.

  When it is determined that the shooting operation is abrupt, the shooting operation determination unit 32 notifies the camera shake determination unit 36 of the sudden start of the shooting operation. The camera shake determination unit 36 sets the response characteristic to “high” (S20), notifies the imaging unit 14 as response characteristic information, and the imaging unit 14 switches the response characteristic of the camera shake correction unit 15 to “high”. Therefore, in the subsequent imaging processing, the imaging unit 14 outputs the video information of the imaging target to the video codec 16 while correcting the camera shake with the response characteristic “high” until the response characteristic is changed.

  On the other hand, if it is determined that the shooting operation is not abrupt, the shooting operation determination unit 32 maintains the response characteristic setting at “low”. Therefore, the imaging unit 14 outputs the video information of the imaging target to the video codec 16 while correcting the camera shake with the response characteristic “low”.

  When the shooting operation is started regardless of whether the shooting operation is slow or slow, the shooting operation determination unit 32 continues to determine the shooting operation status in order to determine the end of the shooting operation. In the determination of the shooting operation status, the end of the shooting operation is determined from a period in which the change tendency of the movement amount is small and a certain regularity is recognized. When the shooting operation determination unit 32 determines the end of the shooting operation in a predetermined period t immediately after the shooting operation is ended (S22), the shooting operation information is set with the type / slowness, start time, and end time of the shooting operation as shooting operation information. The shooting operation information holding unit 34 sequentially outputs the shooting operation information (S24).

  When it is determined that the completed operation is an abrupt shooting operation (S26), the shooting operation determination unit 32 notifies the camera shake determination unit 36 of the end of the shooting operation. When notified of the end of the shooting operation, the camera shake determination unit 36 sets a predetermined evaluation period T (S28), and starts determining the movement state over the predetermined evaluation period T after the end of the shooting operation (S30). . The shooting operation determination unit 32 continues to determine the shooting operation status in order to determine the start of the next shooting operation.

  Here, the evaluation period T is set to be increased / decreased so as to correspond to an increase / decrease in the interval at which the photographing operations are performed. Accordingly, when the shooting operations are performed at short / long intervals, the evaluation period T is set to be short / long, so that the response characteristic is set to low at the start of the next sudden shooting operation. The probability increases, and it becomes easy to suppress the recording of an unnatural image accompanying the camera shake correction. In particular, even when a sudden shooting operation is repeated with a short interval, if the movement amount does not exceed the threshold during the evaluation period T set short, the response characteristic is set to “low”. At the start of the next sudden shooting operation, unnatural image recording can be suppressed. Details of the method for setting the evaluation period T will be described with reference to FIG.

  Here, in the evaluation period T after the end of the sudden photographing operation, the response characteristic is set to “high”, so that the camera shake that occurs after the sudden photographing operation can be corrected appropriately. This makes it easier for camera shake to occur immediately after the end of an abrupt shooting operation, so that camera shake can be corrected effectively.

  The camera shake determination unit 36 calculates the movement amount of the video camera 10 through the analysis of the motion vector of the image or the analysis of the sensor value, and determines whether a large movement amount is detected during the evaluation period T (S32, S34). ). When it is determined that a large amount of movement is not detected during the evaluation period T, the camera shake determination unit 36 sets the response characteristic to “low” (S36), notifies the imaging unit 14 as response characteristic information, and performs imaging. The unit 14 switches the response characteristic of the camera shake correction unit 15 to “low”. Therefore, in the subsequent imaging processing, the imaging unit 14 outputs the video information of the imaging target to the video codec 16 while correcting the camera shake with the response characteristic “low” until the response characteristic is changed.

  Here, after the end of the evaluation period T, if a large amount of movement is not detected in the evaluation period T, the response characteristic is set to “low”. Unnatural image recording can be suppressed. Further, since the response characteristic is set to “low” from the end of the evaluation period T to immediately after the start of the next sudden shooting operation (the period from the start of the shooting operation to the end of the predetermined period t). Compared with the case where the camera shake correction function is turned off between the shooting operations, the camera shake generated between the shooting operations can be appropriately corrected.

  On the other hand, when the camera shake determination unit 36 determines that a large amount of movement has occurred during the evaluation period T, the camera shake determination unit 36 maintains the response characteristic at “high”. Therefore, the imaging unit 14 outputs the video information of the imaging target to the video codec 16 while correcting the camera shake with the response characteristic “high”. In this case, the shooting operation determination unit 32 determines whether the next sudden shooting operation is started in the process of step S16.

  When the end of the next shooting operation is determined, the shooting operation determination unit 32 determines the previous shooting operation from the end time of the previous shooting operation and the start time of the next shooting operation, together with the type, slowness, and start time of the next shooting operation. The interval between the next shooting operation and the next shooting operation is calculated. Then, the shooting operation determination unit 32 outputs the shooting operation information as a part of the shooting operation information to the shooting operation information holding unit 34, and the shooting operation information holding unit 34 stores the shooting operation information.

  Next, a method for setting the predetermined evaluation period T will be described with reference to FIG. When the start of the recording operation is instructed, the camera shake determination unit 36 sets the predetermined evaluation period T to the predetermined upper limit value Tmax as described above (S12).

  When the camera shake determination unit 36 is notified of the end of the sudden shooting operation from the shooting operation determination unit 32 (S26), first, based on the shooting operation information stored in the shooting operation information holding unit 34, the camera shake determination unit 36 It is determined whether the five shooting operations correspond to shooting operations within the past two minutes, and the evaluation period T is temporarily set (S40). Here, if the condition is satisfied, the minimum value is temporarily set as the evaluation period T among the intervals between the shooting operations within the past two minutes (S42). If the condition is not satisfied, the latest five shootings are performed. Among the intervals between operations, the minimum value is provisionally set as the evaluation period T (S44).

  If shooting operation information related to the latest five shooting operations is not stored, shooting operation information related to less than five shooting operations is referred to. Further, specific determination conditions such as “the last five” shooting operations and “within the past two minutes” can be adjusted as appropriate. Further, the evaluation period T is provisionally set as the average value, the median value, the nth value in descending / ascending order, etc., instead of being provisionally set as a predetermined number or the minimum value of the intervals between the photographing operations for a predetermined period. Also good.

  Thereby, since the evaluation period T is set based on the interval between the predetermined number of shooting operations in the latest period or the interval between shooting operations in the latest predetermined period, the response characteristic at the start of the next sudden shooting operation Can be improved. The photographer may change the frequency of performing the photographing operation according to the condition of the subject. For this reason, it is effective to set the evaluation period T suitable for the tendency of the shooting operation by the photographer according to the interval between the most recent predetermined number of shooting operations or the interval between the shooting operations within the predetermined period.

  Next, the camera shake determination unit 36 determines whether or not the temporarily set evaluation period T exceeds a predetermined upper limit value Tmax, and sets the evaluation period T (S46). If the condition is satisfied, a predetermined upper limit value Tmax is set as the evaluation period T. If the condition is not satisfied, the temporarily set evaluation period T is set as the evaluation period T (S48). .

  As a result, the evaluation period T is set as a period equal to or less than the predetermined upper limit value Tmax, so that an evaluation period T that is longer than necessary can be prevented from being set. If the evaluation period T is longer than necessary, the probability that the response characteristic is set to “high” at the start of a sudden shooting operation increases, and it becomes impossible to suppress unnatural image recording associated with camera shake correction. There is a case. For this reason, it is effective to set the evaluation period T so as not to exceed the predetermined upper limit value Tmax. Note that the upper limit value Tmax of the evaluation period T is desirably set as a period obtained by adding a slight margin to a period in which the influence of camera shake that occurs after the end of a sudden photographing operation is eliminated.

[4. Specific example of image stabilization method]
Hereinafter, a specific example of the camera shake correction method will be described with reference to FIGS. 11 and 12. FIGS. 11 and 12 show the results when the camera shake correction method shown in FIGS. 8 to 10 is performed for the photographing operation shown in FIGS. 6 and 7. It is assumed that the upper limit value Tmax of the evaluation period T is set to 5 seconds and the initial value of the evaluation period T is set to the upper limit value Tmax.

  In the example shown in FIG. 11, the first photographing operation is started from time 0:10 with a response characteristic “low”. Since the first shooting operation is an abrupt panning operation, the response characteristic is set to “high” immediately after the start of the shooting operation, and the shooting operation ends with the response characteristic “high”. When the photographing operation ends, the evaluation period T starts from the end time 0:13. Here, since the evaluation period T is set to the upper limit value Tmax, the evaluation period T continues until time 0:18. Since a large amount of movement is not detected during the evaluation period T, the response characteristic is set to “low” as the evaluation period T ends.

  Next, the second photographing operation is started from time 0:20 with the response characteristic being low. The interval between the first and second shooting operations is calculated as 7 seconds from the end time 0:13 of the first shooting operation and the start time 0:20 of the second shooting operation. Since the second shooting operation is an abrupt tilt operation, the response characteristic is set to “high” immediately after the start of the shooting operation, and the shooting operation ends with the response characteristic “high”.

  When the photographing operation ends, the evaluation period T starts from the end time 0:28. Here, since the evaluation period T is set to the upper limit value Tmax, the evaluation period T continues until time 0:33. However, in the middle of the evaluation period T, at time 0:30, the third photographing operation with a large movement amount is started. Here, since the third photographing operation is an abrupt tilt operation, the camera shake is corrected in a state where the response characteristic is maintained at “high”. The interval between the second and third shooting operations is calculated as 2 seconds from the end time 0:28 of the second shooting operation and the start time 0:30 of the third shooting operation.

  When the shooting operation ends at time 0:35 with the response characteristic “high”, the evaluation period T starts after the shooting operation ends. Here, since the minimum interval of past photographing operations is 2 seconds and is not more than the upper limit value Tmax, the evaluation period T is set to 2 seconds, and the evaluation period T continues until time 0:37. Since a large amount of movement is not detected during the evaluation period T, the response characteristic is set to “low” as the evaluation period T ends. For the fourth to seventh photographing operations, similarly to the third photographing operation, an evaluation period T of 2 seconds from the end time of the photographing operation is secured, and the response characteristic becomes “low” as the evaluation period T ends. Is set.

  When the eighth shooting operation ends at 3:00, the latest five shooting operations include shooting operations (fourth and fifth shooting operations) other than the shooting operations performed within the past two minutes. Will be. For this reason, the evaluation period T is provisionally set to 25 seconds (interval between the fifth and sixth photographing operations), which is the minimum interval among the intervals between photographing operations performed within the past two minutes. Since it exceeds the value Tmax, it is set to 5 seconds.

  Further, when the tenth shooting operation ends at 5:00, the latest five shooting operations include shooting operations (sixth to eighth shooting operations) other than the shooting operations performed within the past two minutes. Will be. For this reason, the evaluation period T is provisionally set to 20 seconds (interval between the eighth and ninth photographing operations), which is the minimum interval among the intervals between photographing operations performed within the past two minutes. Since it exceeds the value Tmax, it is set to 5 seconds.

  Here, since the evaluation period T is set to the upper limit value Tmax, the evaluation period T continues until time 5:05. However, the eleventh photographing operation is started at time 5:02 in the middle of the evaluation period T. Here, since the eleventh photographing operation is a sudden tilt operation with a large movement amount, the camera shake is corrected in a state where the response characteristic is maintained at “high”. The interval between the tenth and eleventh shooting operations is calculated as 2 seconds from the end time 5:00 of the tenth shooting operation and the start time 5:02 of the eleventh shooting operation.

  When the photographing operation is completed at time 5:06 with the response characteristic “high”, the evaluation period T is started after the photographing operation is completed. Here, since the minimum interval of past photographing operations is 2 seconds and is not more than the upper limit value Tmax, the evaluation period T is set to 2 seconds, and the evaluation period T continues until time 5:08. Since a large amount of movement is not detected during the evaluation period T, the response characteristic is set to “low” as the evaluation period T ends. As for the twelfth to fifteenth photographing operations, similarly to the eleventh photographing operation, an evaluation period T of 2 seconds is secured from the end time of the photographing operation, and the response characteristic becomes “low” as the evaluation period T ends. Is set.

  In the example shown in FIG. 12, the first photographing operation is started from time 0:10 with a response characteristic “low”. Since the first to fifth shooting operations are slow panning or tilting operations, the response characteristic is maintained at “low” until immediately after the start time 1:10 of the sixth shooting operation.

  Since the sixth shooting operation is an abrupt panning operation, the response characteristic is set to “high” immediately after the start of the shooting operation, and the shooting operation ends with the response characteristic “high”. When the photographing operation ends, the evaluation period T starts from the end time 1:13. Here, since it is set to 2 seconds (interval between the second and third photographing operations), which is the minimum interval among the intervals between the latest five photographing operations, the evaluation period T continues until time 1:15. Since a large amount of movement is not detected during the evaluation period T, the response characteristic is set to “low” as the evaluation period T ends. Since the seventh to fifteenth shooting operations are all gentle shooting operations, the response characteristics are maintained at “low”.

  Therefore, in the example shown in FIG. 11, the response characteristic is set to “low” at the start of the first to fifteenth sudden shooting operations excluding the third and eleventh sudden shooting operations. In the example shown in FIG. 12, the response characteristic is set to “low” at the start of the sixth sudden photographing operation. As a result, unnatural image recording associated with camera shake correction can be suppressed.

  In the above embodiment, the response characteristic is set to “high” immediately after the start of the sudden shooting operation, and if no large movement amount is detected in the evaluation period T after the end of the shooting operation, after the end of the evaluation period T. The response characteristic is set to “low”. However, as a modification of the above embodiment, the response characteristic may be set to gradually decrease from “high” to “low” regardless of the amount of movement in the period from the start to the end of the evaluation period T. . As a result, even if a large amount of movement is detected during the evaluation period T, the response characteristics have deteriorated to some extent. For example, even when the third and eleventh sudden shooting operations in the example shown in FIG. It is possible to suppress unnatural recording of images to some extent.

[5. Summary]
As described above, according to the camera shake correction method according to the present embodiment, since the response characteristic is set to be low in advance, compared to the case where the camera shake correction is completely functioning at the start of a sudden shooting operation. Thus, it is possible to suppress unnatural image recording associated with camera shake correction. In the evaluation period T after the end of the sudden photographing operation, the response characteristic is set to be high, so that the camera shake that occurs after the sudden photographing operation can be corrected appropriately. After the evaluation period T ends, if a large amount of movement is not detected in the evaluation period T, the response characteristic is set to low again. Therefore, at the start of the next sudden shooting operation, an unnatural image associated with camera shake correction. Recording can be suppressed.

  Further, according to the camera shake correction method according to the present embodiment, the evaluation period T is set to be increased or decreased so as to correspond to the increase or decrease of the interval at which the photographing operations are performed. Accordingly, when the shooting operations are performed at short / long intervals, the evaluation period T is set to be short / long, so that the response characteristic is set to low at the start of the next sudden shooting operation. The probability is high, and it becomes easy to suppress the recording of an unnatural image accompanying the camera shake correction.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, this invention is not limited to the example which concerns. It is obvious for those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea described in the claims. It is understood that it belongs to.

1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. It is a figure for demonstrating the movement amount of a video camera. It is a block diagram which shows the structure of the modification of the video camera which concerns on one Embodiment of this invention. It is a schematic diagram which shows the time series data of the moving amount of a video camera at the time of camera shake, and an accumulated moving amount. It is a schematic diagram which shows the time series data of the moving amount of a video camera at the time of imaging | photography operation, and a cumulative moving amount. It is a figure which shows an example of the imaging operation information which shows the determination result of imaging operation. It is a figure which shows the other example of the imaging operation information which shows the determination result of imaging operation. It is a flowchart which shows the whole process of the camera-shake correction method. It is a schematic diagram for demonstrating the whole process of the camera-shake correction method. It is a flowchart which shows the partial process of a camera shake correction method. FIG. 9 is a diagram illustrating a result when the camera shake correction method illustrated in FIG. 8 is performed with respect to the photographing operation illustrated in FIG. 6. FIG. 9 is a diagram illustrating a result when the camera shake correction method illustrated in FIG. 8 is performed with respect to the photographing operation illustrated in FIG. 7.

DESCRIPTION OF SYMBOLS 10, 60 Video camera 15 Camera shake correction part 16 Image | video codec 32 Shooting operation determination part 36 Camera shake determination part 38 Sensor group

Claims (7)

A movement detection unit for detecting the movement amount of the imaging device;
A camera shake correction unit that corrects camera shake by adjusting two or more different high and low response characteristics;
A shooting operation determination unit that determines the start and end of a shooting operation from the detection value of the movement amount;
The response characteristic is set to a low value in advance, and when the detection value of the movement amount at the start of the shooting operation exceeds a predetermined threshold, the response characteristic is set to a high level and a predetermined value from the end of the shooting operation is set. A response characteristic setting unit that sets the response characteristic to low again when the detected value of the movement amount in the evaluation period is less than a predetermined threshold;
An evaluation period setting unit configured to increase or decrease the evaluation period so as to correspond to an increase or decrease in the interval at which the photographing operations are performed;
A camera shake correction device comprising:   The camera shake correction apparatus according to claim 1, wherein the evaluation period setting unit sets an evaluation period from an interval between a predetermined number of photographing operations in the latest period or an interval between photographing operations within the most recent predetermined period.   The camera shake correction apparatus according to claim 1, wherein the evaluation period setting unit sets the evaluation period to a predetermined upper limit value when the set evaluation period exceeds a predetermined upper limit value.   The response characteristic setting unit sets the predetermined evaluation instead of setting the response characteristic low when the detected value of the movement amount in the predetermined evaluation period from the end of the photographing operation is less than a predetermined threshold. The camera shake correction device according to claim 1, wherein the response characteristic is set so that the response characteristic gradually changes from high to low in a period from the start to the end of the period. An image sensor;
A movement detection unit for detecting the movement amount of the imaging device;
A camera shake correction unit that corrects camera shake by adjusting two or more different high and low response characteristics;
A shooting operation determination unit that determines the start and end of a shooting operation from the detection value of the movement amount;
The response characteristic is set to a low value in advance, and when the detection value of the movement amount at the start of the shooting operation exceeds a predetermined threshold, the response characteristic is set to a high level and a predetermined value from the end of the shooting operation is set. A response characteristic setting unit that sets the response characteristic to low again when the detected value of the movement amount in the evaluation period is less than a predetermined threshold;
An evaluation period setting unit configured to increase or decrease the evaluation period so as to correspond to an increase or decrease in the interval at which the photographing operations are performed;
An imaging apparatus comprising: A step of setting a response characteristic of camera shake correction to low in advance;
Determining the start of the shooting operation from the detected value of the movement amount of the imaging device;
When the detected value of the movement amount at the start of the photographing operation exceeds a predetermined threshold, setting the response characteristic to high;
Determining the end of the shooting operation from the detected value of the movement amount;
Increasing or decreasing a predetermined evaluation period so as to correspond to an increase or decrease in the interval between the shooting operations;
When the detected value of the movement amount in the evaluation period from the end of the photographing operation is less than a predetermined threshold, setting the response characteristic to low;
Including image stabilization. A step of setting a response characteristic of camera shake correction to low in advance;
Determining a start time of a shooting operation from a detection value of a moving amount of the imaging device;
When the detected value of the movement amount at the start of the photographing operation exceeds a predetermined threshold, setting the response characteristic to high;
Determining the end of the shooting operation from the detected value of the movement amount;
Increasing or decreasing a predetermined evaluation period so as to correspond to an increase or decrease in the interval between the shooting operations;
When the detected value of the movement amount in the evaluation period from the end of the photographing operation is less than a predetermined threshold, setting the response characteristic to low;
A program for causing a computer to execute a camera shake correction method.

Images