JP2014171110A - Image reproduction device, control method thereof, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To compensate for image blur at the time of image reproduction through removal of a panning component without incurring increases in memory size and cost.SOLUTION: A microcomputer 210 detects, with reference to shake data recorded in a recording medium 121, whether a panning operation has been performed at the time of capture of image data. If a panning operation is detected, the microcomputer 210 removes a panning component from the shake data, thereby generating panning component removed data. At the time of reproduction of the image data, if a panning operation has been detected, an image processing part 223 compensates for a shake of an imaging device in the image data in accordance with the panning component removed data. In accordance with whether a panning operation has been detected, the microcomputer changes an acquisition cycle for acquisition of the shake data from the recording medium.

Description

  The present invention relates to a video playback device that plays back video, a control method thereof, and a control program.

  In general, when an image is recorded in an imaging device such as a video camera, the recorded image (recorded image) may be blurred due to the shaking of the imaging device due to camera shake or the like. In order to correct blur in the recorded video during video playback, a motion vector representing the blur in the recorded video is detected, and a partial area in the video is selectively enlarged based on the motion vector. An imaging device is known (see Patent Document 1).

  On the other hand, there is known an imaging apparatus that detects blurring of an imaging apparatus with a sensor at the time of imaging, obtains blur data, and records the blur data in a memory together with video data. In this imaging apparatus, blurring of video is corrected using blur data during video playback (see Patent Document 2).

JP-A-7-143380 Japanese Patent Laid-Open No. 10-42233

  By the way, in the imaging devices described in Patent Documents 1 and 2, blurs caused by panning operation or tilting operation (hereinafter collectively referred to as panning operation) during imaging are also similar to those caused by camera shake. It will be corrected during playback.

  Therefore, when correcting a shake caused by camera shake (hereinafter referred to as a shake component), it is necessary to remove a shake component (hereinafter referred to as a panning component) due to a panning operation.

  On the other hand, it is known that the panning component increases or decreases in series at the start or end of the panning operation. Therefore, the panning component can be removed by acquiring blur data at predetermined time intervals and detecting whether or not the panning operation has been performed according to the size and direction of the panning component.

  However, if the time interval for detecting the panning component from the blur data is longer than the playback period of the video, the video blur correction is performed without removing the panning component.

  For example, in the vicinity of the start and end of the panning operation, the panning component is not properly removed, and image blur correction cannot be performed appropriately, giving the user a sense of incongruity.

  On the other hand, if the time interval for detecting the panning component is shortened to about the reproduction period of the video, the change in the panning component near the start and end of the panning operation can be acquired with higher accuracy. As a result, the accuracy of removing the panning component is improved, and appropriate image blur correction can be performed.

  However, since the presence / absence of the panning component is always determined here, the number of times and the amount of readout of blur data are increased. For this reason, an increase in memory size for storing blur data and a processing capability for processing blur data at high speed are required, which not only increases the cost of the video reproduction apparatus itself but also increases power consumption.

  Accordingly, an object of the present invention is to provide a video playback apparatus, a control method thereof, and a control program capable of correcting a blur of video during video playback by removing panning components without increasing the memory size. It is to provide.

  In order to achieve the above object, a video reproduction apparatus according to the present invention reproduces the video data as a video from a recording medium in which video data captured by the imaging apparatus is recorded together with shake data indicating the shake amount of the imaging apparatus. A video reproduction device, wherein a detection unit that detects whether or not a panning operation has been performed when the video data is imaged according to the shake data; and when the panning operation is detected by the detection unit, Removing means for removing panning components resulting from the panning operation from shake data to generate panning component removal data; and when reproducing the video data, if the panning operation is not detected by the detection means, Accordingly, a shake of the imaging device in the video data is corrected, and the detection means When the panning operation is detected, a shake correction unit that corrects a shake of the imaging device in the video data according to the panning component removal data, and the recording according to whether the panning operation is detected by the detection unit Changing means for changing an acquisition cycle for acquiring the shake data from a medium.

  A control method according to the present invention is a control method for a video reproduction device for reproducing the video data as video from a recording medium in which video data captured by the imaging device is recorded together with shake data indicating a shake amount of the imaging device. A detection step for detecting whether or not a panning operation has been performed when the video data is imaged according to the shake data; and when the panning operation is detected in the detection step, the panning operation is detected from the shake data. Removing the panning component due to the generation of the panning component removal data, and when reproducing the video data, when the panning operation is not detected in the detection step, the video data according to the shake data The shake of the imaging device is corrected, and the panning operation is performed in the detection step. Is detected from the recording medium according to whether or not the panning operation is detected by the detection step, and a shake correction step of correcting a shake of the imaging device in the video data according to the panning component removal data And a changing step for changing an acquisition cycle for acquiring shake data.

  The control program according to the present invention is a control program used in a video reproduction device that reproduces the video data as a video from a recording medium in which video data captured by the imaging device is recorded together with shake data indicating the shake amount of the imaging device. A detecting step for detecting whether or not a panning operation is performed when the video data is imaged according to the shake data, and the panning operation is detected in the detecting step. When removing the panning component resulting from the panning operation from the shake data and generating panning component removal data, and when playing back the video data, when the panning operation is not detected in the detection step In the video data according to the shake data A shake correction step for correcting shake of the image pickup device in the video data according to the panning component removal data when the shake of the image pickup device is corrected and the panning operation is detected in the detection step, and the detection step And a change step of changing an acquisition cycle for acquiring the shake data from the recording medium in accordance with whether or not the panning operation has been detected.

  According to the present invention, it is possible to remove a panning component without correcting an increase in memory size and cost, thereby correcting a shake of a video during video playback.

It is a block diagram which shows the structure about an example of an imaging device provided with the video reproduction apparatus by embodiment of this invention. It is a block diagram for demonstrating the structure of the video reproduction apparatus shown in FIG. 3 is a flowchart for explaining a reproduction process in the video reproduction apparatus shown in FIG. 2. It is a figure for demonstrating the time-sequential change of the shake data and panning component in the case of panning operation | movement (panning operation), (a) is a figure which shows an example of the time-sequential change of shake data, (b) is ( The figure which shows the data which are the result of having performed the filter process using the low-pass filter with the cutoff frequency which removes a camera shake component and passes a panning component with respect to the shake data shown to a), (c) shows to (b) It is a figure which shows an example of the acquisition timing at the time of acquiring shake data with a long period with respect to the data after a filter process. It is a figure which shows the state of the shake data in the elapsed time shown to Fig.4 (a). It is a flowchart for demonstrating the presence or absence of panning shown in FIG. FIGS. 3A and 3B are diagrams for explaining an example of panning removal in the video reproduction apparatus shown in FIG. 2, in which FIG. 3A is a diagram illustrating panning data after filter processing, and FIG. (C) is a figure which shows the data for the panning removal produced | generated in the acquisition timing shown in (b), (d) is the result of having removed the panning component using the data shown in (c) (E) is a diagram showing a state in which panning data is acquired in a short cycle, (f) is a diagram showing data for panning removal generated at the acquisition timing shown in (e), (g) ) Is a diagram showing the result of removing the panning component using the data shown in (f).

  Hereinafter, an example of a video reproduction apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of an example of an imaging apparatus including a video reproduction apparatus according to an embodiment of the present invention.

  The illustrated imaging device is, for example, a video camera, and includes a video recording device that records video and a video playback device 200 that plays back video.

  In FIG. 1, a video camera (hereinafter simply referred to as a camera) includes a lens unit (lens) 101, and a subject image (optical image) is formed on an image sensor 102 by the lens 101. The image sensor 102 includes, for example, a CCD or CMOS sensor, photoelectrically converts a subject image, and outputs an electrical signal (analog signal) corresponding to the subject image.

  The analog signal processing unit 103 performs a predetermined process on the electrical signal that is the output of the imaging element 102 to generate an analog imaging signal. The analog signal processing unit 103 includes, for example, a CDS (Co-related Double Sampling) circuit and an AGC (Automatic Gain Control) circuit.

  The camera signal processing unit 104 includes an A / D converter, and generates a digital video signal corresponding to an analog imaging signal that is an output of the analog signal processing unit 103. The encoding unit 105 encodes the digital video signal in, for example, the MPEG2 format and outputs an MPEG2 video stream.

  The angular velocity sensor 111 detects the angular velocity of shake (blur) applied to the camera. The shake (blur) data setting unit 112 calculates the shake amount (shake data) of the camera based on the detection output of the angular velocity sensor 111. The time code setting unit 114 sets a time code for video recording according to the time information acquired from the clock 113. The metadata setting unit 115 outputs shake data, a time code, camera-related data regarding the camera, and the like as metadata. The configuration related to the camera-related data is not shown in FIG.

  A recording / playback control unit (also called a drive unit) 121 receives an MPEG2 video stream (video data) from the encoding unit 105 and metadata from the metadata setting unit 115, and records these video data and metadata on the recording medium 122. To do.

  Note that the recording / playback control unit 121 reads video data and metadata from the recording medium 122 and plays back video when playing back video.

  Through the recording process described above, video data to which metadata including shake data is added is recorded on the recording medium 122.

  FIG. 2 is a block diagram for explaining the configuration of the video reproduction apparatus 200 shown in FIG.

  As described with reference to FIG. 1, video data to which metadata including shake data is added is recorded on the recording medium 122. However, when recording the metadata and the video data, as long as the camera 100 can acquire the shake data and the video data, they may be recorded in any way. For example, the metadata is recorded separately from the video data. It may be.

  When playing back video data recorded on the recording medium 122, the recording / playback control unit 121 acquires the video data and metadata from the recording medium 122. Then, the recording / playback control unit 121 sends video data and metadata to the decoding unit (decoding unit) 221.

  The decoding unit 221 obtains decoded data (digital video signal) by decoding the video data, and writes the digital video signal in the memory 222 together with the metadata. The memory 222 has a capacity for storing a decoded digital video signal and metadata for a predetermined time (for example, several seconds).

  If the digital video signal (and metadata) for several seconds is stored in the memory 222, the microcomputer 200 can temporally advance the digital video signal with respect to the reproduced image displayed on the display device 224 described later. Can be obtained.

  The image processing unit 223 reproduces a digital video signal as a reproduction image according to the cutout position and cutout size determined by the microcomputer 210. For example, the image processing unit 223 sequentially cuts out images of a predetermined cut-out size from a predetermined cut-out position for each of a plurality of unit images (field image or frame image) constituting the video indicated by the digital video signal. Then, the image processing unit 223 performs electronic zoom processing on the cut-out image and outputs it to the display device 224 as a display image. The display device 224 displays video by sequentially displaying the reproduced images.

  As illustrated, the microcomputer 210 includes a motion vector detection unit 201, a cutout position setting unit 202, a metadata reading unit 203, a shake data acquisition unit 204, a cutout size setting unit 205, a time code setting unit 206, and a read address setting unit. And the functions of the panning component acquisition unit 208 and the panning determination unit 209.

  The motion vector detection unit 201 detects the motion vector of the subject from the image indicated by the digital video signal stored in the memory 222. As will be described later, the motion vector is used as an index of image blur.

  The cutout position setting unit 202 determines the cutout position of the unit image so as to correct the image blur based on the motion vector detected by the motion vector detection unit 201, and sets the cutout position in the image processing unit 223. To do. The metadata reading unit 203 reads metadata from the memory 222 (here, metadata is added to the digital video signal).

  The metadata reading unit 203 reads metadata from a read address set by a read address setting unit 207 described later. Further, the metadata reading unit 203 reads the metadata at a designated cycle of one field cycle (hereinafter referred to as a short cycle) and four field cycles (hereinafter referred to as a long cycle). The short period (second period) is the same period as the video data reproduction period, and the long period may be an integer multiple of the short period.

  The shake data acquisition unit 204 acquires shake data from the metadata read by the metadata reading unit 203. The cutout size setting unit (enlargement rate setting unit) 205 determines the cutout size of the unit image based on the shake data and sets it in the image processing unit 223.

  The time code setting unit 206 acquires the time code of the current reproduction image (unit image being reproduced) from the metadata reading unit 203, and a time code after a predetermined time (time code after a predetermined time). ) To the read address setting unit 207.

  The read address setting unit 207 sets a read address corresponding to the time code after a predetermined time in the memory 222. As a result, the metadata reading unit 203 can acquire metadata of an image (unit image) reproduced after a predetermined time from the current reproduced image.

  FIG. 3 is a flowchart for explaining playback processing in the video playback apparatus 200 shown in FIG. Note that the processing of the flowchart shown in FIG.

  When the reproduction process is started, first, the shake data acquisition unit 204 receives a predetermined period of time from the reproduction start point of the digital video signal (hereinafter also referred to as video data) from the memory 222 via the metadata reading unit 203. The shake data is acquired (step S301: long cycle acquisition). Then, the shake data acquisition unit 204 sends the shake data to the panning component acquisition unit 208.

  In the following description, a shake generated by a panning operation or a tilting operation (hereinafter collectively referred to as a panning operation) is referred to as a panning component, and a shake caused by the shake is referred to as a shake component.

  The panning component acquisition unit 208 performs a filtering process for extracting a low frequency component that does not include a camera shake component from shake data, and extracts the low frequency component as a panning component (step S302). Subsequently, the panning determination unit 209 determines whether or not panning has occurred (that is, panning operation) according to the panning component, that is, the low frequency component (step S303). The panning determination unit 209 notifies the metadata reading unit 203 of a time code related to a frame in which panning has not occurred.

  If it is determined that panning has occurred (YES in step S303), the metadata reading unit 203 changes the metadata acquisition cycle to a short cycle. Then, the metadata reading unit 203 sets the time code notified from the panning determination unit 209 in the time code setting unit 206 as a time code for acquiring shake data.

  The shake data acquisition unit 204 acquires shake data for a predetermined period in a short cycle from the reproduction start point of the video data via the metadata reading unit 203 (step S304). Then, the shake data acquisition unit 204 sends the shake data to the panning component acquisition unit 208.

  The panning component acquisition unit 208 performs a filtering process for extracting a low frequency component that does not include a camera shake component from shake data, and extracts the low frequency component as a panning component (step S305). The panning determination unit 209 determines whether or not panning has occurred according to the panning component, that is, the low frequency component (step S306).

  If it is determined that panning has occurred (YES in step S306), the metadata reading unit 203 changes the next metadata acquisition cycle to a short cycle (step S307). Then, the metadata reading unit 203 uses the time code obtained by adding the time code notified from the panning determination unit 209 and the metadata acquisition cycle in step S304 as the time code for acquiring shake data as the time code setting unit 206. Set to.

  On the other hand, if it is determined in step S303 that panning does not occur (NO in step S303), the metadata reading unit 203 changes the next metadata acquisition cycle to a long cycle (step S308). Then, the metadata reading unit 203 sets the time code obtained by adding the time code notified from the panning determination unit 209 and the metadata acquisition cycle in step S304 to the time code setting unit 206 as a time code for acquiring shake data. . Subsequently, the processing proceeds to step S309 described later.

  If it is determined in step S306 that panning has not occurred (NO in step S306), the metadata reading unit 203 changes the next metadata acquisition cycle to a long cycle in step S308.

  After the process of step S307 or S308, the panning component acquisition unit 208 is a shake component (also referred to as a shake amount) that is data obtained by removing the panning component obtained in step S302 or S305 described above from shake data obtained from the metadata. Is obtained (step S309). Then, the panning component acquisition unit 208 sets the camera shake component as the camera shake data in the motion vector detection unit 201 and the cutout size setting unit 205.

  Subsequently, the cutout size setting unit 205 checks the amplitude of shake indicated by the camera shake data, and determines the cutout size so that the change of the cutout size per frame falls within a predetermined range. Further, the cutout position setting unit 202 sets the cutout position so as to correct the image blur based on the amount of the motion vector detected by the motion vector detection unit 201 and the camera shake data obtained by the shake data acquisition unit 204. It is determined (that is, moved) and set in the image processing unit 223 (step S310). Note that the cutout position is determined using a known method.

  Next, the image processing unit 223 cuts out an image having the set cut-out size from the cut-out position set for the unit image (step S311: image blur correction). Then, the image processing unit 223 outputs the clipped image to the display device 224. When it is determined in step S303 or S306 that panning has occurred, the cutout position setting unit 202 may stop the movement of the cutout position. As a result, the image blur correction is stopped during the panning operation.

  Next, panning presence / absence determination in the microcomputer 210 shown in FIG. 2 will be described. The panning component acquisition unit 208 stores a predetermined number of panning components (for example, five times) acquired in the long cycle and the short cycle in a temporary memory (not shown). A panning ingredient is used.

  FIG. 4 is a diagram for explaining time-series changes in shake data and panning components during a panning operation (panning operation). FIG. 4A is a diagram showing an example of time-series changes in shake data acquired by the shake data acquisition unit 204 shown in FIG. 2, and FIG. 4B is a panning component acquisition unit 208 shown in FIG. FIG. 5 is a diagram showing data obtained by performing filter processing using a low-pass filter at a cutoff frequency that removes camera shake components and allows panning components to pass through the shake data shown in FIG. FIG. 4C is a diagram illustrating an example of the acquisition timing when the shake data is acquired with a long period with respect to the data after the filter processing illustrated in FIG.

  FIG. 5 is a diagram showing the state of the shake data from the elapsed time t1 to t4 shown in FIG.

  As described above, the shake data is composed of the shake component caused by the shake of the user and the panning component generated when the user operates the camera 100 to pan. In general, the camera shake component is about 5 to 10 Hz, the panning component is 2 Hz or less, and have different frequency characteristics.

  From the elapsed time t1 to t4 shown in FIG. 4A, as shown in FIG. 5, only camera shake occurs at the elapsed time t1. Then, the panning operation is started at the elapsed time t2, and the panning operation is in the middle of the acceleration operation at the elapsed time t3. When the elapsed time t4 is reached, the panning operation is performed at a substantially constant speed. That is, in FIG. 4A, the elapsed time t2 to t4 is a period of a transient state where panning occurs.

  The panning component acquisition unit 208 performs a filtering process on the shake data shown in FIG. 4A with a low-pass filter (LPF) having a cutoff frequency that removes the camera shake component and passes the panning component. Thereby, the data shown in FIG. 4B is obtained.

  The panning determination unit 209 determines the presence / absence of panning according to the data after the filter processing illustrated in FIG. Here, as shown in FIG. 4 (c), when shake data is acquired in a long cycle, data after filter processing is obtained at the timing of every four fields at the acquisition timing. Is stored in the memory, and the panning determination unit 209 determines the presence / absence of panning based on the data after the filter processing (that is, the panning component).

  FIG. 6 is a flowchart for explaining the presence / absence determination of panning shown in FIG. Note that the panning presence / absence determination shown in FIG.

  When the panning determination process is started, the panning determination unit 209 obtains an absolute value (hereinafter referred to as Xcur) of the panning component obtained from the current panning component acquisition unit 208 (step S401). Then, the panning determination unit 209 determines whether the shake data acquisition cycle is a long cycle or a short cycle (step S402).

  When the shake data acquisition cycle is a long cycle (YES in step S402), the panning determination unit 209 compares the panning component absolute value Xcur with a preset first threshold value Xth1, and the first comparison result Is obtained (step S403). The first threshold value is a value indicating an angular displacement that can be determined that panning has occurred.

Subsequently, the panning determination unit 209 obtains the absolute value of the difference between the current absolute value Xcur and the absolute value of the panning component acquired once before (hereinafter referred to as Xprev1) according to the following equation (1). The absolute difference value A1.
A1 = | Xcur-Xprev1 | (1)

  The difference absolute value A1 is a value proportional to the slope of the function when Xcur and Xprev1 are interpolated with a linear function, and indicates the change per unit time of the panning component.

  Next, the panning determination unit 209 compares the absolute difference value A1 with the change threshold value Ath and obtains the comparison result (large / small result) (step S404). The change threshold value Ath indicates the amount of change in angular displacement per long cycle that can be determined to be panning.

  Subsequently, the panning determination unit 209 compares the first panning component absolute value Xprev1, the previous panning component absolute value Xprev2, and the third previous panning component absolute value Xprev3 with the first threshold value Xth1. To obtain second to fourth comparison results. Then, the panning determination unit 209 determines whether or not all of the first comparison result and the second to fourth comparison results are equal to or greater than the first threshold value Xth1 (step S405).

  If any of the first to fourth comparison results indicates that it is less than the first threshold value Xth1 (NO in step S405), the panning determination unit 209 determines the panning component absolute values Xprev1 to Xprev5 for the past five times. Are used to find the absolute difference values A2 to A5 according to the following formulas (2) to (5).

A2 = | Xprev1-Xprev2 | (2)
A3 = | Xprev2-Xprev3 | (3)
A4 = | Xprev3-Xprev4 | (4)
A5 = | Xprev4-Xprev5 | (5)

  The difference absolute values A2 to A5 are values proportional to the slope of the function when the panning component absolute values Xprev1 to Xprev5 are interpolated with a linear function. Then, the panning determination unit 209 determines whether or not all the difference absolute values A2 to A5 are smaller than the change threshold Ath (step S407).

  When all of absolute difference values (that is, slopes) A2 to A5 are smaller than change threshold value Ath (YES in step S407), panning determination unit 209 generates panning because there is almost no change in panning component over time and the amount of change is small. It is assumed that “no panning” is in effect (step S409).

  On the other hand, when at least one of the difference absolute values A2 to A5 is equal to or greater than the change threshold value Ath (NO in step S407), the panning determination unit 209 has a panning component but the panning component absolute value is the second threshold value Xth2. As described above, since it is less than the first threshold value Xth1 and the time change of the angular displacement is a constant amount, it is assumed that panning is in progress (step S410). Then, the panning determination unit 209 determines “no panning has occurred” as the panning determination result subsequent to the process of step S409 or S410 (step S411).

  In step S405, if all of the first to fourth comparison results indicate that the value is equal to or greater than the first threshold value Xth1 (YES in step S405), the panning determination unit 209 continues the size of panning. The absolute difference values A1 to A3 described above are obtained by assuming that a certain level has been exceeded. Then, the panning determination unit 209 determines whether or not all the difference absolute values A1 to A3 are smaller than the change threshold Ath (step S412).

  If any of the absolute difference values A1 to A3 is equal to or greater than the change threshold Ath (NO in step S412), the panning determination unit 209 determines that the panning operation is still in the process of starting and proceeds to the process of step S410.

  On the other hand, when all of the difference absolute values A1 to A3 are smaller than the change threshold value Ath (YES in step S412), the panning determination unit 209 has a large level of panning component from the past to the present (present) and the time change thereof. It is determined that there is little panning. Then, the panning determination unit 209 sets the panning state to “stable panning” (step S413). Subsequently, the panning determination unit 209 sets the panning determination result as “panning is occurring” (step S414).

  In step S402, if the shake data acquisition cycle is not a long cycle (NO in step S402), the panning determination unit 209 determines the absolute value of the current panning component acquired in the short cycle and the absolute value of the past five panning components. Is obtained (step S416). Then, the panning determination unit 209 determines whether or not the moving average value Xave is smaller than the second threshold value Xth2 (step S417).

  When moving average value Xave is smaller than second threshold value Xth2 (YES in step S417), panning determination unit 209 sets the panning determination result to “no panning has occurred” (step S418). On the other hand, when moving average value Xave is equal to or greater than second threshold value Xth2 (NO in step S417), panning determination unit 209 sets the panning determination result to “panning is occurring” (step S419).

  Subsequent to the processing in step S411, S414, S418, or S419 described above, the panning determination unit 209 refers to the current panning component absolute value Xcur as past information in the next panning determination. The data is stored in a temporary memory (not shown) (step S415). Then, the panning determination unit 209 sets the panning determination result obtained in the above process as the determination result in the above-described step S303 or S306 (step S420), and ends the panning determination process.

  As described above, in the above-described video reproduction device, at the time of video reproduction, images of a predetermined cut-out size are sequentially cut out from a predetermined cut-out position in the unit image for each of a plurality of unit images constituting the video. Then, the video reproduction apparatus moves the cutout position of the unit image so as to correct the blur of the video.

  If it is determined that panning is being performed in accordance with the shake of the camera 100, the period for acquiring shake data from the metadata is shortened, and the panning component detected in the transient state of the start and end of panning is removed. On the other hand, if it is determined that panning is not in progress, the shake data acquisition period is lengthened and the previous shake data is acquired in time, so that the occurrence of panning of the camera 100 is detected earlier.

  In the video playback device 200, as described above, when video data playback is started, the shake data acquisition cycle is set to a long cycle (first cycle), and the start of the panning operation is detected. Then, the acquisition cycle is changed from the long cycle to the short cycle (second cycle). Further, when it is detected that the panning operation is completed when the shake data acquisition cycle is a short cycle, the acquisition cycle is changed from a short cycle to a long cycle.

  FIG. 7 is a diagram for explaining an example of panning removal in the video reproduction apparatus 200 shown in FIG. FIG. 7A is a diagram showing the panning data after the filter processing, and FIG. 7B is a diagram showing a state in which the panning data is acquired in a long cycle. FIG. 7C shows the data for panning removal generated at the acquisition timing shown in FIG. 7B, and FIG. 7D uses the data shown in FIG. 7C. It is a figure which shows the result of having removed the panning component. Further, FIG. 7 (e) is a diagram showing a state in which panning data is acquired in a short cycle, and FIG. 7 (f) is data for panning removal generated at the acquisition timing shown in FIG. 7 (e). FIG. FIG. 7 (g) is a diagram showing the result of removing the panning component using the data shown in FIG. 7 (f).

  Now, it is assumed that the panning data filtered by the panning component acquisition unit 208 is data shown in FIG. Here, data (camera shake data) that is subject to image blur correction in image blur correction after the panning component is removed when the panning data acquisition cycle is a long cycle or a short cycle is shown. In FIG. 7, the time interval between time t2 and time t4 is the elapsed time in the occurrence of panning shown in FIG.

  When panning data is acquired in the long cycle shown in FIG. 7B and data (removal data) for removing panning components is generated based on the panning data, the data shown in FIG. 7C is obtained. It is done. In the long period, removal data is generated at a rough acquisition interval (sampling interval) with respect to the panning time change. Therefore, when the removal data is used to remove the panning component from the data shown in FIG. The result shown in FIG. 7D is obtained.

  It can be seen that the panning component cannot be removed properly in the panning generation transient state from the panning start time t2 to the time t4 until the panning is stabilized. When image blur correction is performed based on the result shown in FIG. 7D, the panning component that remains without being removed is erroneously determined to be camera shake, and even if there is no camera shake, the image blur correction is erroneously performed. Will be done.

  On the other hand, as shown in FIG. 7E, when the shake data is acquired again in a short cycle for the purpose of removing the panning component, the removal data becomes the data shown in FIG. When the panning component is removed using the removal data, the result shown in FIG. 7G is obtained.

  Compared with the result shown in FIG. 7 (d), the result shown in FIG. 7 (g) prevents the image blur correction from being erroneously determined during video reproduction because the intensity of the panning component that remains without being removed is not large. Thus, it is possible to surely correct the camera shake component to be suppressed.

  In the above-described embodiment, the presence or absence of panning is determined according to the absolute value of panning component, the time-series difference, and the moving average value, and the panning component at the time of panning is removed from the shake data. However, other methods may be used to detect panning and remove panning components. Furthermore, a known method can be used for image blur correction during video reproduction.

  As described above, in the embodiment of the present invention, the panning operation is detected according to the shake data acquired in the long cycle, and when the panning operation is detected, the shake data is acquired in the short cycle and the shake data is obtained. Since the panning component is removed and the video data is corrected according to the panning component removal data, the panning component is removed without increasing the memory size and increasing the cost. Image blur can be corrected.

  As is clear from the above description, in the example shown in FIG. 2, the microcomputer 210 functions as a detection unit, a removal unit, and a change unit, and the microcomputer 210 and the image processing unit 222 function as a shake correction unit.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a control method, and this control method may be executed by the video reproduction device. Further, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the video reproduction device. The control program is recorded on a computer-readable recording medium, for example.

  Each of the above control method and control program has at least a detection step, a removal step, a shake correction step, and a change step.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed.

DESCRIPTION OF SYMBOLS 201 Motion vector detection part 202 Cutout position setting part 203 Metadata read-out part 204 Shake data acquisition part 205 Cutout size setting part (enlargement ratio setting part)
206 Time Code Setting Unit 207 Read Address Setting Unit 208 Panning Component Acquisition Unit 209 Panning Determination Unit 210 Microcomputer

Claims (7)

A video playback device that plays back the video data as video from a recording medium in which video data captured by the imaging device is recorded together with shake data indicating a shake amount of the imaging device,
Detecting means for detecting whether a panning operation has been performed when the video data is imaged according to the shake data;
When the panning operation is detected by the detecting means, a removing means for removing panning components resulting from the panning operation from the shake data and generating panning component removal data;
When reproducing the video data, if the panning operation is not detected by the detection means, the shake of the imaging device in the video data is corrected according to the shake data, and the panning operation is detected by the detection means. A shake correction unit that corrects a shake of the imaging device in the video data according to the panning component removal data;
Changing means for changing an acquisition cycle for acquiring the shake data from the recording medium according to whether the panning operation is detected by the detecting means;
A video playback apparatus comprising:   2. The video according to claim 1, wherein the changing unit changes the acquisition cycle to a second cycle shorter than a predetermined first cycle when the panning operation is detected by the detection unit. Playback device. The second period is the same period as a reproduction period that is a period for reproducing the video data,
The video reproduction apparatus according to claim 2, wherein the first period is an integral multiple of the second period.   The changing unit sets the acquisition cycle to the first cycle when the reproduction of the video data is started, and sets the acquisition cycle to the first cycle when the detection unit detects the start of a panning operation. 4. The video reproduction apparatus according to claim 2, wherein the period is changed from the period to the second period.   When the acquisition period is the second period and the detection unit detects that the panning operation is completed, the change unit changes the acquisition period from the second period to the first period. The video reproduction apparatus according to claim 2, wherein the video reproduction apparatus is a video reproduction apparatus. A control method of a video playback device for playing back the video data as video from a recording medium in which video data captured by the imaging device is recorded together with shake data indicating a shake amount of the imaging device,
A detection step of detecting whether or not a panning operation has been performed when the video data is imaged according to the shake data;
When the panning operation is detected in the detection step, a removal step of removing panning components resulting from the panning operation from the shake data to generate panning component removal data;
When reproducing the video data, if the panning operation is not detected in the detection step, the shake of the imaging device in the video data is corrected according to the shake data, and the panning operation is detected in the detection step. When the shake correction step of correcting the shake of the imaging device in the video data according to the panning component removal data,
A change step of changing an acquisition cycle for acquiring the shake data from the recording medium according to whether or not the panning operation is detected by the detection step;
A control method characterized by comprising: A control program used in a video playback device that plays back the video data as video from a recording medium in which video data captured by the imaging device is recorded together with shake data indicating a shake amount of the imaging device,
In the computer provided in the video reproduction device,
A detection step of detecting whether or not a panning operation has been performed when the video data is imaged according to the shake data;
When the panning operation is detected in the detection step, a removal step of removing panning components resulting from the panning operation from the shake data to generate panning component removal data;
When reproducing the video data, if the panning operation is not detected in the detection step, the shake of the imaging device in the video data is corrected according to the shake data, and the panning operation is detected in the detection step. When the shake correction step of correcting the shake of the imaging device in the video data according to the panning component removal data,
A change step of changing an acquisition cycle for acquiring the shake data from the recording medium according to whether or not the panning operation is detected by the detection step;
A control program characterized by causing

Images