JP2007300581A - Moving image photographing apparatus and moving image photographing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively make shake correction function even in considerable shaking. <P>SOLUTION: A control unit 20 detects a moving amount within a video image picked up with respect to a time base direction and performs shake correction by vertically and horizontally moving a video cut position on the basis of the moving amount. At such a time, when the position of an area to be cut reaches an edge in a full angle of a view, namely, when there is no margin (blank) for camera shake correction, the angle of the view of the area to be cut is reduced and the margin is secured. Thus, a video image cut at an angle of a view resulting from the adjustment of the angle of the view to be cut in accordance with the cutting position is magnified into the display size of a display unit 25 and displayed. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving image shooting apparatus and a moving image shooting program for performing camera shake correction.

  Conventionally, a digital camera has a function of shooting a moving image in addition to a still image. There are roughly two types of camera shake correction in moving image shooting. The first camera shake correction (mechanical camera shake correction) is based on the output signal from the acceleration / gyro sensor that detects the vibration of the digital camera housing due to camera shake, and shakes the image sensor or the lens to shake the camera shake. It is a technology to cancel. The second camera shake correction (electronic camera shake correction) takes a picture while cutting out a part of the entire angle of view of the image obtained from the image sensor, and swings the cutout position in response to the camera shake. This is a recording technique (see, for example, Patent Document 1).

JP 2004-248171 A

  However, the mechanical camera shake correction described above has a problem that the lens becomes large and the cost increases. Further, the electronic camera shake correction described above has a problem that the size of the angle of view to be cut is constant, and the shake correction does not work effectively when the shake is large.

  Accordingly, an object of the present invention is to provide a moving image shooting apparatus and a moving image shooting program capable of effectively functioning blur correction even when blurring is large.

  In order to achieve the above object, the inventions of claims 1 and 8 detect the amount of motion in the video sequentially captured in the time axis direction, cut out the video at a predetermined angle of view from the captured video, When this clipped image is enlarged or reduced to a predetermined display size and sequentially displayed, blur correction is performed by changing the cut position of the image based on the detected amount of motion, and the clipped image is clipped. When the position of the area reaches an end within the entire angle of view or when it is predicted that the end of the area will be reached, the angle of view of the area to be cut out is reduced.

  Further, as a preferred aspect, for example, as in claim 2, in the moving image photographing apparatus according to claim 1, the cut angle of view adjustment means is configured such that the position of the region cut by the cut means is an end portion within the entire angle of view. If the angle of view has not been reached or if it is no longer expected to reach the end, the reduced angle of view may be enlarged.

  Further, as a preferred aspect, for example, as in claim 3, in the moving image photographing apparatus according to claim 2, when the cut angle of view is enlarged, the cut angle of view is gradually increased by a predetermined amount. It may be enlarged to return to the original angle of view.

  Further, as a preferable aspect, for example, as in claim 4, in the moving image photographing apparatus according to any one of claims 1 to 3, the cut angle adjustment unit is configured such that the position of the region cut by the cut unit is all. When the end of the angle of view is reached, the angle of view may be reduced so that the area can be accommodated within the entire angle of view without changing the position of the area.

  Moreover, as a preferable aspect, for example, as in claim 5, in the moving image photographing apparatus according to any one of claims 1 to 3, the cut angle adjustment unit includes an end of a region cut by the cut unit, The rate at which the angle of view is reduced may be increased as the distance between the end portions in the entire angle of view becomes closer.

  Further, as a preferred aspect, for example, as in claim 6, in the moving image photographing apparatus according to claim 1, the cut angle of view adjustment means includes a movement amount detected by the movement amount detection means exceeding a predetermined threshold value. If it is determined that the position of the area cut by the cutting means is predicted to reach the end within the entire angle of view, the angle of view of the area cut by the cutting means is reduced, and the movement If the amount falls below a predetermined threshold, it is determined that the position of the area cut by the cutting means is no longer expected to reach the end, and the angle of view of the area cut by the cutting means is enlarged. You may do it.

  Further, as a preferable aspect, for example, as in claim 7, in the moving image photographing apparatus according to any one of claims 1 to 6, the cutout position of the image changed by the blur correction unit is set by a predetermined amount. Cutout position adjusting means for gradually returning to the center within the entire angle of view may be provided.

  According to the present invention, the amount of motion in a video that is sequentially captured in the time axis direction is detected, and the video is cut out from the captured video at a predetermined angle of view. When the image is sequentially displayed enlarged or reduced in accordance with the amount of motion, blur correction is performed by changing the cut-out position of the video based on the detected amount of motion, and the position of the cut-out region is within the end of the entire angle of view. When it reaches the edge, or when it is predicted that the edge will be reached, the angle of view of the area to be cut out is reduced, so that even if the blur is large, the blur correction can function effectively. The advantage that it can be obtained.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. First embodiment A-1. Configuration of First Embodiment FIG. 1 is a block diagram showing a configuration of a digital camera according to a first embodiment of the present invention. In the figure, the image acquisition unit 10 includes a lens 11, a shutter 12, and an LPF 13. The lens 11 is a normal optical lens and includes a lens group in which aspherical lenses are stacked. The shutter 12 is a so-called mechanical shutter that is operated by a driver 14 driven by the control unit 20 when a shutter button is operated. Depending on the digital camera, a mechanical shutter may not be provided. In the case of a model equipped with a retractable lens structure and a mechanical zoom, these drivers are also controlled by the driver 14. The LPF 13 is a crystal low-pass filter and is mounted to prevent the occurrence of moire.

  Next, the analog signal processing unit 15 includes an image sensor (CCD, CMOS) 16, a sampling / signal amplification processing unit 17, and an A / D converter 18. The imaging sensor 16 forms a subject image (image), and converts the intensity of light of each color of RGB into a current value. The sampling / signal amplification processing unit 17 performs correlated double sampling processing and signal amplification processing for suppressing noise and color unevenness. The A / D converter 18 is also called an analog front end, and converts the sampled / amplified analog signal into a digital signal (converts each color of RGB and CMYG into 12-bit data and outputs it to the bus line).

  Next, the control unit (CPU) 20 controls the entire digital camera 1 (imaging device) according to a program stored in a program memory described later. In particular, in the first embodiment, when the margin (margin) for camera shake correction disappears, the margin is ensured by reducing the size of the cutout area of the image.

  The preview engine 22 is stored in the image buffer 26 immediately after detection of digital data input via the image acquisition unit 10 and the analog signal processing unit 15 in the recording mode (also referred to as recording mode or photographing mode) or shutter operation detection. Thinning processing is performed in order to display the digital data and the digital data stored in the image memory 31 on the display unit 25. The D / A converter 23 converts the digital data thinned out by the preview engine 22 and outputs it to the driver 24 at the subsequent stage.

  The driver 24 includes a buffer area for temporarily storing digital data displayed on the display unit 25 at the subsequent stage, and drives the display unit 25 based on a control signal input via the key operation unit 27 and the control unit 20. The display unit 25 includes a color TFT liquid crystal, an STN liquid crystal, or the like, and displays a preview image, image data after shooting, a setting menu, and the like.

  The image buffer 26 temporarily stores digital data immediately after photographing until it is input via the analog signal processing unit 15 or the digital signal processing unit 28 and passed to the digital signal processing unit 28. The key operation unit 27 includes a shutter button, a recording / playback mode selection slide switch, a menu button, a cross key (determined by pressing the center), and the like.

  The digital signal processing unit 28 performs white balance processing, color processing, gradation processing, contour emphasis, conversion from RGB format to YUV format, YUV format for digital data input via the analog signal processing unit 15. To JPEG format. The image compression / decompression processing unit 29 compresses and encodes the digital data input via the digital signal processing unit 28 into the JPEG format, generates a motion JPEG format movie file, or converts the motion JPEG format movie file to MPEG. It is converted into a moving image file of a format, or in a reproduction mode, a moving image file in JPEG format, motion JPEG format, or MPEG format is expanded.

  The program memory 30 stores various programs loaded on the control unit 20, EV values in the best shot function, color correction information, and the like. The image memory 31 stores image data temporarily stored in the image buffer 26, digital data converted into various file formats, moving image data, and the like. The card I / F 32 controls data exchange between the external recording medium 33 and the digital camera body.

  The external recording medium 33 is a detachable recording medium composed of a compact flash (registered trademark), a memory stick, an SD card, or the like. The external connection I / F 34 includes a USB connector slot and the like, is connected to a personal computer or the like, and is used for transferring photographed image data. The RAM 35 stores various parameters necessary for control of the control unit (CPU) 20, various parameters at the time of night scene shooting (gain (ISO sensitivity), aperture, shutter speed, threshold for image composition, weight, etc.), and the like. .

A-2. Operation of First Embodiment Next, the operation of the first embodiment described above will be described. 2 and 3 are flowcharts for explaining the operation of the digital camera according to the first embodiment. FIGS. 4 to 7 are schematic diagrams for explaining the case where there is no camera shake correction, camera shake correction & when shooting starts, during movie shooting, and when the size of the cutout area is corrected according to the first embodiment.

  When the moving image shooting mode is set, the analog signal processing unit 15 takes in an image (full angle of view) formed on the image sensor (CCD, CMOS) 16 and performs predetermined signal processing, as shown in FIG. Through display is performed on the display unit 25 (step S10). Before the start of photographing, the angle of view is 100%, that is, the pre-margin is zero. In this state, when the user performs a moving image shooting start operation (step S12), first, the image cutout region position is set (initialized) so that the center of the image cutout region is the center within the entire angle of view (step S12). Next, it is determined whether or not camera shake correction is set to the on state (step S14). If the camera shake correction is set to the off state, the process proceeds to a moving image shooting process in which normal camera shake correction is not performed (step S16).

  On the other hand, when the camera shake correction is set to the on state, the cut angle of view is set to 80%, that is, the pre-margin is set to 20% (step S18). Next, a plurality of frames from the analog signal processing unit 15 are buffered (step S20), and a motion vector is detected from the difference between the target frame and the adjacent frame (step S22). Next, an image cropping offset (adjusted values before and after the cropping position) is calculated according to the motion vector (step S24), and the calculated offset value is added to the current image cropping region position to obtain the image cropping region position. Update. Then, it is determined whether or not the entire clip region specified by the updated clip region position and the current clip angle is within the image data range (within the full angle of view) (step S26). In this case, it may be determined whether or not the end of the cutout region extends beyond the end within the entire angle of view, and the distance between the end of the cutout region and the end within the entire angle of view is determined in advance. It may be determined whether or not it is closer than the distance.

If the cutout region position is outside the image data range, the cutout region size (cutting view angle) is reduced (step S28). On the other hand, if the cutout region position is within the image data range, nothing is done next. move on. In this case, the size of the cutout area may be reduced as much as necessary to fit within the entire angle of view without changing the position of the cutout area, and the distance between the edge of the cutout area and the edge within the full angle of view is You may make it enlarge the ratio to reduce, so that it is closer.
Next, the image is cut out at the cut angle of view (step S30), the image is enlarged in the display area of the display unit 25 (step S34), and displayed on the display unit 25 (step S36).

  Then, the current image cutout region position is slightly returned to the center (step S37). This is to make the image cutout region position follow the camera panning operation intended by the photographer in addition to making the image cutout region follow the camera shake not intended by the photographer. The process of returning to the center may be performed by a certain amount for each frame, or as the distance between the edge of the image cutout region position and the edge within the entire angle of view is smaller, the amount to be returned at once is increased. It may be. In this way, when the camera shake is small, the moving image becomes smoother and it is possible to cope with a larger camera shake.

  Next, it is determined whether or not it is a buffer end (step S38). If it is not a buffer end, the next frame is selected (step S40). Next, it is determined whether or not any operation or photographing is finished (step S42). If photographing is continued, the process returns to step S22 to detect a motion vector, calculate a clipping offset, and cut based on the offset. The area size (cutting angle of view) is reduced, the image is cut out, and displayed on the display unit 25.

  On the other hand, if it is a buffer end, the cut out plural frame image is encoded and recorded in the image memory 31 (step S44). Thereafter, the process returns to step S18, and buffering of a plurality of frames, detection of motion vectors, calculation of a cut-off offset, reduction of a cut-out area size (cut-off view angle) based on the offset, cut-out of an image, and display on the display unit 25 are performed. .

  In step S42, if there is an operation for ending shooting, the recorded image is saved as a moving image file (step S46), and then the processing is ended.

  By the above-described operation, immediately after the start of moving image shooting, as shown in FIG. Using this clip region position as a starting point, the clip region is moved vertically and horizontally according to the amount of movement from the previous frame. Depending on the image, the cutout region position may reach the uppermost position, the lowermost position, the leftmost position, or the rightmost position. In this case, no more camera shake correction can be performed in that direction.

  Therefore, during moving image shooting, as shown in FIG. 6, the cutout area moves back and forth and to the left and right for camera shake correction. Then, when the cutout area is out of the image data range due to camera shake correction and there is no margin, the cutout area size (cutting angle of view) is reduced as shown in FIG. As a result, a margin can be ensured and camera shake correction can function effectively.

  In the above-described first embodiment, when there is no camera shake correction margin during video shooting with camera shake correction enabled, a margin is ensured by reducing the size of the cutout area, that is, the cutting angle of view. Camera shake correction can be made to function effectively even for moving images with large blurring.

B. Next, a modification of the first embodiment of the present invention will be described.
FIG. 8 is a flowchart (part) for explaining the operation of the modification of the first embodiment. In the modification of the first embodiment, as in the first embodiment described above, after detecting a motion vector from the difference between the target frame and an adjacent frame in step S22, the image clipping offset is calculated in step S24. Then, it is determined whether the amount of motion based on the motion vector exceeds a predetermined threshold (step S50). When the amount exceeds the predetermined threshold, the size of the cutout region is reduced (step S52). Is enlarged (step S54). The subsequent steps are the same as in the first embodiment described above.

  In the first embodiment described above and the modification of the first embodiment, if there is still a margin in the distance between the end of the cutout region position and the end within the entire angle of view, the cutout is performed when the blur is large. Predict that there is a high possibility that the area will protrude from the full angle of view, and reduce the size of the cutout area in advance, so even if the blurring becomes large, video shooting is performed with proper camera shake correction It becomes possible to do.

  Further, in the above-described modification of the first embodiment, when the blur is small, that is, when the position of the region to be cut out does not reach the end within the entire angle of view or when it is no longer expected to reach the end. In other words, by enlarging the size (view angle) of the cut-out area, it is possible to shoot a moving image while performing camera shake correction at an appropriate size according to the size of the shake.

  Also, in the modification of the first embodiment described above, when the angle of view that has been reduced is enlarged when blurring is small, it is gradually increased by a predetermined amount and returned to the original angle of view. Since the angle of view changes smoothly, smooth moving image shooting can be performed while performing appropriate camera shake correction.

  Further, in the first embodiment and the modification of the first embodiment described above, when the position of the cut-out area reaches the end within the full angle of view, the area is not changed without changing the position of the area. Since the angle of view is reduced to fit within the corner, it is possible to shoot a smoother moving image while performing appropriate camera shake correction.

  In the first embodiment described above and the modification of the first embodiment, the ratio of reducing the angle of view is increased as the distance between the edge of the region to be cut out and the edge within the entire angle of view is closer. Since it is increased, it becomes possible to shoot a moving image while performing camera shake correction at an appropriate size according to the size of the shake.

  Further, in the modification of the first embodiment described above, it is determined that when the amount of motion exceeds a predetermined threshold, it is predicted that the position of the region to be clipped will reach the end within the entire angle of view. The size of the cutout area is reduced, and if the amount of movement falls below the threshold, it is determined that the position of the cutout area is no longer predicted to reach the edge, and the cutout area is enlarged. Therefore, it becomes possible to shoot a moving image while performing camera shake correction at an appropriate size according to the size of the shake.

  Further, in the first embodiment and the modification of the first embodiment described above, the image clipping position is gradually returned to the central portion within the entire angle of view by a predetermined amount. In addition to making the image cutout region position follow, the image cutout region position can be made to follow the panning operation of the camera intended by the photographer.

C. Second Embodiment Next, a second embodiment of the present invention will be described. In the first embodiment described above, the size of the cutout area is changed during blur correction at the time of moving image shooting. However, in the second embodiment, the size of the cutout area is set according to the amount of motion acquired at the previous shooting. It is characterized by adjusting. Information such as whether the maximum movement amount and cumulative movement amount in each of the up / down / left / right directions and the cutout area position have reached the highest position, lowest position, leftmost position, rightmost position, etc. Based on this information, the cut-out area size (pretending angle of view) used for the next moving image shooting is determined. That is, the cutout area size is not changed during blur correction during moving image shooting.

  In the above-described second embodiment, information such as whether the maximum motion amount and cumulative motion amount acquired at the previous playback and the cutout region position have reached the uppermost position, the lowermost position, the leftmost position, and the rightmost position, respectively. Since the size of the cutout region is determined based on the above, the camera shake correction can be effectively functioned even for a moving image with a large amount of shake. In the first embodiment described above, the enlargement ratio displayed during moving image reproduction is changed, whereas in the second embodiment, the enlargement ratio when displayed on the display unit 25 is constant.

It is a block diagram which shows the structure of the digital camera by 1st Embodiment of this invention. 4 is a flowchart for explaining the operation of the digital camera according to the first embodiment. 4 is a flowchart for explaining the operation of the digital camera according to the first embodiment. In this 1st Embodiment, it is a schematic diagram for demonstrating the state at the time of camera shake correction absence. In this 1st Embodiment, it is a schematic diagram for demonstrating the state at the time of camera-shake correction | amendment & imaging | photography start. In this 1st Embodiment, it is a schematic diagram for demonstrating the state during video recording. In this 1st Embodiment, it is a schematic diagram for demonstrating the state at the time of the size correction | amendment of a cut-out area | region. It is a flowchart (part) for demonstrating operation | movement of the modification of this 1st Embodiment.

Explanation of symbols

1 Digital Camera 10 Image Acquisition Unit 11 Lens 12 Shutter 13 LPF
DESCRIPTION OF SYMBOLS 14 Driver 15 Analog signal processing part 16 Imaging sensor 17 Sampling / signal amplification processing part 18 A / D converter 19 Flash 20 Control part 22 Preview engine 23 D / A converter 24 Driver 25 Display part 26 Image buffer 27 Key operation part 28 Digital signal Processing unit 29 Image compression / decompression processing unit 30 Program memory 31 Image memory 32 Card I / F
33 External recording medium 34 I / F for external connection
35 RAM

Claims (8)

A video shooting device that displays an image captured by performing camera shake correction,
Imaging means for imaging video;
Video display means for sequentially displaying videos sequentially captured by the imaging means;
A motion amount detection means for detecting a motion amount in the video imaged by the imaging means with respect to the time axis direction;
When the video is displayed by the video display means, a cutting means for cutting the video at a predetermined angle of view from the video taken by the imaging means and expanding or reducing the cut video according to a predetermined display size. When,
Based on the amount of motion detected by the amount of motion detection means, a blur correction means for performing blur correction by changing the cutout position of the video by the cutout means,
Cutting angle adjustment for reducing the angle of view of the area cut by the cutting means when the position of the area cut by the cutting means reaches or is expected to reach the end within the entire angle of view. And a moving image photographing apparatus.   When the position of the region cut by the cutting unit does not reach the end within the entire view angle or when it is no longer predicted to reach the end, the cut view angle adjusting unit is configured to reduce the reduced view angle. The moving image photographing device according to claim 1, wherein the moving image photographing device is enlarged.   3. The moving image photographing apparatus according to claim 2, wherein when the reduced angle of view is enlarged, the cut angle of view is gradually enlarged by a predetermined amount to return to the original angle of view. .   When the position of the area cut out by the cutting means reaches the end within the full angle of view, the cut angle of view adjustment means is configured so that the area can be accommodated within the full angle of view without changing the position of the area. 4. A moving image photographing apparatus according to claim 1, wherein the angle of view is reduced.   The cut angle of view adjustment means increases the rate of reducing the angle of view as the distance between the end of the area cut by the cut means and the end within the full angle of view becomes closer. The moving image photographing apparatus according to claim 1.   The cut angle adjusting means adjusts the position of the region cut by the cut means to an end within the entire angle of view when the amount of movement detected by the movement amount detecting means exceeds a predetermined threshold. If the angle of view of the area cut out by the cutout means is reduced and the amount of motion falls below a predetermined threshold, the position of the area cut out by the cutout means is at the end. 2. The moving image photographing apparatus according to claim 1, wherein it is determined that it is no longer predicted to reach, and an angle of view of an area cut out by the cutting unit is enlarged.   4. A cut position adjusting means for gradually returning a cut position of an image changed by the blur correction means to a central portion within a full angle of view by a predetermined amount. Video shooting device. A video shooting program that displays images taken with camera shake correction,
On the computer,
An imaging function to capture images,
A video display function that sequentially displays video that is sequentially captured,
A motion amount detection function for detecting a motion amount in the captured image with respect to a time axis direction;
A cutting function for cutting the video at a predetermined angle of view from the captured video when displaying the video, and expanding or reducing the cut video according to a predetermined display size;
Based on the detected amount of motion, a blur correction function that performs blur correction by changing the cutout position of the video,
When the position of the cut-out area reaches or is expected to reach the end within the entire angle of view, a cut-off angle adjustment function that reduces the angle of view of the cut-out area is realized. A featured video shooting program.

Images