JP2012222485A - Imaging apparatus, imaging control method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of generating a natural moving image without depending on a frame rate of imaging of the moving image.SOLUTION: An imaging apparatus includes a CPU 13 and an image processing section 8. The CPU 13 calculates the number of frames required for covering a stop period from a frame rate in recording a moving image and a shutter speed in shooting a still image during the stop period in recording the moving image, when it is detected that a still image recording instruction is operated by a user in recording the moving image. The image processing section 8 generates a combined frame by combining a frame image of the moving image shot before and after the stop period with a reduced still image generated from the still image as an intermediate frame at a predetermined ratio, encodes a shot frame image, in which the generated intermediate frame and combined frame are stored in the stop period, into an MPEG image, and records the MPEG image in a recording medium such as a flash memory 16 or a memory card 20.

Description

  The present invention relates to an imaging apparatus, an imaging control method, and a program.

  In an imaging device such as a digital camera or a mobile phone having an imaging function, there is a technique for recording high-definition still image data at a desired timing by instructing recording of a still image during moving image shooting.

  In such a technique, there is a technique for complementarily generating a frame at the time of interruption from a recorded still image so that the interruption of moving image recording at the time of still image recording is not noticeable at the time of moving image reproduction (for example, Patent Document 1). reference).

  By the way, in recent years, with the speeding up of image processing and compression encoding processing, a technique for recording a moving image with a high frame rate at the time of recording has appeared. In this case, it is devised to apply the above technique. ing.

JP 2001-111934 A

  However, with the technique described in Patent Document 1, when trying to generate a complementary frame using recorded still image data, the number of frames to be generated is insufficient due to the high frame rate, and continuity cannot be maintained. There is a risk that a movie will be generated.

  Accordingly, the present invention provides an imaging apparatus, an imaging control method, and a program capable of generating a natural moving image without depending on the frame rate of moving image shooting in the technique of shooting a still image in the middle of moving image shooting. With the goal.

  According to the present invention, the imaging means, the first output means for driving the imaging means to output images sequentially taken at a predetermined frame rate, and the predetermined output when the images are sequentially taken at the predetermined frame rate. A second output means for temporarily stopping the image pickup operation at the frame rate, driving the image pickup means during the stop period, and outputting an image picked up by exposure at a predetermined shutter speed as still image data; Based on the predetermined frame rate and the predetermined shutter speed, calculation means for calculating the number of frames corresponding to the stop period, and images of the number of frames calculated by the calculation means are displayed before and after the stop period. An imaging apparatus comprising: a generation unit configured to generate a synthesized image and the still image data.

  According to the present invention, there is an advantage that a natural moving image can be generated without depending on the frame rate of moving image shooting in the technique of shooting a still image in the middle of moving image shooting.

1 is a block diagram illustrating a digital camera according to an embodiment of the present invention. 4 is a flowchart for explaining the operation of the digital camera 1 according to the present embodiment. It is a conceptual diagram for demonstrating the complementary process by the digital camera of this embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. Configuration of Embodiment FIG. 1 is a block diagram showing a digital camera according to an embodiment of the present invention. In the figure, a digital camera 1 includes an imaging lens 2, a lens driving unit 3, a diaphragm / shutter 4, a CCD 5, and a TG (Timing).
Generator) 6, unit circuit 7, image processing unit 8, speaker 9, microphone 10, audio processing unit 11, CPU 13, DRAM 14, memory 15, flash memory 16, display unit 17, key input unit 18, card I / F 19, and A memory card 20 is provided.

  The imaging lens 2 includes a focus lens, a zoom lens, and the like, and a lens driving unit 3 is connected thereto. The lens driving unit 3 includes a motor for driving the focus lens and the zoom lens constituting the imaging lens 2 in the optical axis direction, a focus motor driver for driving the focus motor and the zoom motor according to a control signal from the CPU 13, and a zoom motor driver. It is configured.

  The diaphragm / shutter 4 includes a drive circuit (not shown), and operates according to a control signal sent from the CPU 13 by the drive circuit. The diaphragm / shutter 4 controls the amount of light entering from the imaging lens 2. The CCD (imaging device) 5 converts the light of the subject projected through the imaging lens 2 and the aperture / shutter 4 into an electrical signal and outputs the electrical signal to the unit circuit 7. The CCD 5 is driven in accordance with a predetermined frequency timing signal generated by the TG 6.

The unit circuit 7 is a CDS (Correlated) that holds the imaging signal output from the CCD 5 by correlated double sampling.
Double Sampling) circuit, AGC (Automatic) for automatic gain adjustment of the imaged signal after sampling
(Gain Control) circuit and an A / D converter for converting an analog image pickup signal after the automatic gain adjustment into a digital signal. The unit circuit 7 is driven in accordance with a timing signal having a predetermined frequency generated by the TG 6. The imaging signal of the CCD 5 is sent to the image processing unit 8 as a digital signal through the unit circuit 7.

  The image processing unit 8 performs image processing (pixel interpolation processing, γ correction, luminance color difference signal generation, white balance processing, exposure correction processing, etc.) of image data sent from the unit circuit 7, and compression / decompression of image data ( For example, processing of JPEG format, M-JPEG format, or MPEG format compression / decompression), and moving image recording interruption at the time of still image recording are made not to be noticeable at the time of moving image reproduction, the frame at the time of interruption from the recorded still image Completion processing to generate completion is performed. The image processing unit 8 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

  In accordance with the control of the CPU 13, the audio processing unit 11 performs control such as D / A conversion of audio data and outputting from the speaker 9, and A / D conversion of audio input from the microphone 10. The speaker 9 outputs sound at the time of recorded moving image reproduction. The microphone 10 collects sound during video recording. The CPU 13 is a one-chip microcomputer that controls each part of the digital camera 1. In particular, in the present embodiment, when the CPU 13 captures a still image during moving image shooting, the image processing unit 8 generates a frame (plurality) for complementing the stop period of moving image recording caused by the still image shooting. Control such as.

  The DRAM 14 is used as a buffer memory for temporarily storing image data sent to the CPU 13 after being imaged by the CCD 5 and also as a working memory for the CPU 13. The memory 15 stores a program necessary for controlling each part of the digital camera 1 by the CPU 13 and data necessary for controlling each part. The CPU 13 performs processing according to this program. The flash memory 16 and the memory card 20 are recording media for storing image data captured by the CCD 5.

  The display unit 17 includes a color LCD and its driving circuit, and displays the subject imaged by the CCD 5 as a through image when in the imaging standby state, and from the flash memory 16 or the memory card 20 when reproducing the recorded image. The read and expanded recorded image is displayed. The key input unit 18 includes a plurality of operation keys such as a shutter switch, a zoom switch, a mode key, a SET key, and a cross key, and outputs an operation signal corresponding to the user's key operation to the CPU 13. A memory card 20 is detachably attached to the card I / F 19 via a card slot (not shown) of the digital camera 1 main body.

B. Operation of Embodiment Next, the operation of the above-described embodiment will be described.
FIG. 2 is a flowchart for explaining the operation of the digital camera 1 according to the present embodiment. Note that the flowchart shown in FIG. 2 is executed during moving image recording. First, the CPU 13 determines whether or not a still image recording instruction from the user has been detected during moving image recording (step S10). When recording a still image during moving image recording, the user presses a still image button, for example.

  If it is not detected that the user has operated the still image recording instruction (NO in step S10), the captured frame image is encoded into MPEG and stored in a recording medium such as the flash memory 16 or the memory card 20. Recording is performed (step S24), and the process returns to a main routine (moving image recording) (not shown).

  On the other hand, when it is detected that the user has operated a still image recording instruction during moving image recording (YES in step S10), a still image is displayed during the frame rate (FPS) during operation recording and the moving image recording stop period. The shutter speed at the time of shooting is acquired (step S12), and the frame rate is divided by the shutter speed to calculate the number of frames required to fill the stop period (step S14).

  Next, it is determined whether the calculated number of frames is an odd number or an even number (step S16). If the calculated number of frames is an odd number such as 1, 3, 5,... (Odd number in step S16), the captured still image is thinned out (reduced) to match the resolution of the frame image of the moving image. And generate an intermediate frame (step S18). Since the number of frames is an odd number, there is one frame located in the middle.

  On the other hand, when the calculated number of frames is an even number such as 2, 4, 6,... (Even number in step S16), the captured still image is thinned out so as to match the resolution of the frame image of the moving image. An intermediate frame is generated (step S20). Since the number of frames is an even number, there are two frames located in the middle.

  Next, in any case, the frame image of the moving image captured before and after the stop period and the above-described thinned still image (reduced still image generated as an intermediate frame) are combined at a predetermined ratio to stop A composite frame required to fill the period is generated (step S22). At this time, the predetermined ratio is determined for each image to be synthesized in accordance with the number of frames and the complementary position in the stop period. Then, the captured frame image containing the generated intermediate frame and composite frame in the stop period is encoded into MPEG and recorded on a recording medium such as the flash memory 16 or the memory card 20 (step S24). Return to the routine (video recording).

  FIG. 3 is a conceptual diagram for explaining the complementing process by the digital camera of the present embodiment. In the illustrated example, the moving image shooting conditions are 100 frames per second, that is, 10 msec / frame, 4 pixel addition (horizontal 2, vertical 2), 3M pixels, and the still image shooting conditions are 50 msec / frame, all Pixel addition, 12M pixels. That is, the frame rate is 100 fps, and the shutter speed is 5/100 seconds. Therefore, the number of frames required to fill the stop period is 100 / (5/100) = 5 frames.

  When the user instructs the still image recording during the still image recording, the moving image recording is interrupted during the still image recording. In the illustrated example, when a still image recording is instructed while a 10 msec moving image is being photographed as M1, M2,..., After moving image M3 (10 msec) is photographed, a transition is made to still image photographing. In this case, if it takes 50 msec to shoot the still image S1, moving image recording is interrupted during that time. After shooting the still image S1, moving image recording starts again, and a 10 msec moving image is shot as M4, M5,.

  In the recorded moving image, a stop period of 50 msec occurs when the still image S1 is captured. If this is left as it is, the moving image will be skipped during the stop period when the moving image is reproduced, and continuity cannot be maintained and unnaturalness will be conspicuous. Therefore, in the present embodiment, it is necessary to fill the stop period based on the frame rate “100 fps” at the time of motion recording and the shutter speed “5/100 seconds” when shooting a still image during the stop period of moving image recording. The number of frames “5” is calculated.

  Next, the number of frames required to fill the stop period is synthesized by combining the frame images of the moving images taken before and after the stop period and the still images taken during the stop period at a predetermined ratio. Generate an image. The predetermined ratio is determined for each image to be combined according to the number of frames and the complementary position in the stop period.

  In the illustrated example, composition (image) # 1 combines the frame image M3 and the reduced image of the still image S1 at a ratio of 2: 1. In composition (image) # 2, the frame image M3 and the reduced image of the still image S1 are combined at a ratio of 1: 2. Similarly, composition (image) # 3 combines the reduced image of the still image S1 and the frame image M4 at a ratio of 2: 1. In composition (image) # 4, the reduced image of the still image S1 and the frame image M4 are combined at a ratio of 1: 2. Thereafter, the synthesized moving image frame is encoded into MPEG and recorded in a recording medium such as the flash memory 16 or the memory card 20.

  According to the above-described embodiment, it is necessary to fill the stop period by synthesizing the frame image of the moving image captured before and after the stop period and the still image captured during the stop period at a predetermined ratio. By generating an image of the number of frames to be generated, a natural moving image can be generated without being influenced by the frame rate of moving image shooting.

  In the above-described embodiment, the frame image of the moving image captured before and after the stop period and the still image captured during the stop period are combined at a predetermined ratio. The transparency according to the weight may be set and α blending may be performed for each pixel.

As mentioned above, although several embodiment of this invention was described, this invention is not limited to these, The invention described in the claim, and its equal range are included.
Below, the invention described in the claims of the present application is appended.

(Appendix 1)
According to the first aspect of the present invention, the image pickup unit, the first output unit that drives the image pickup unit and outputs images sequentially picked up at a predetermined frame rate, and the image are sequentially picked up at the predetermined frame rate. Sometimes, the imaging operation at the predetermined frame rate is temporarily stopped, and during this stop period, the imaging means is driven, and an image captured by exposure at a predetermined shutter speed is output as still image data. Based on the output means, the calculation means for calculating the number of frames corresponding to the stop period based on the predetermined frame rate and the predetermined shutter speed, and the image having the number of frames calculated by the calculation means An imaging apparatus comprising: a generation unit configured to generate an image captured before and after a period and the still image data.

(Appendix 2)
According to the second aspect of the present invention, the generating unit weights the frame according to the number of frames calculated by the calculating unit and the complementary position in the stop period, and the images captured before and after the stop period and the still image The imaging apparatus according to appendix 1, wherein the imaging apparatus combines data.

(Appendix 3)
The invention described in Supplementary Note 3 is the imaging apparatus according to Supplementary Note 2, wherein the generation unit sets α according to the weighting and performs α blending for each pixel.

(Appendix 4)
The invention according to attachment 4 further includes resizing means for resizing the still image data in accordance with a size of an image output by the first output means, wherein the generating means is further calculated by the calculating means. 4. The imaging apparatus according to any one of appendices 1 to 3, wherein an intermediate frame in each frame having the same number of frames is a frame resized by the resizing unit.

(Appendix 5)
The invention according to appendix 5 is the imaging apparatus according to any one of appendices 1 to 4, wherein the calculation means uses the value obtained by dividing the predetermined frame rate by the shutter speed as the number of frames. is there.

(Appendix 6)
The invention according to appendix 6 includes a first output step of driving the imaging unit to output images sequentially captured at a predetermined frame rate in the imaging control method by the imaging apparatus including the imaging unit; When sequentially capturing images at a frame rate, the image capturing operation at the predetermined frame rate is temporarily stopped, and during the stop period, the image capturing unit is driven and exposed at a predetermined shutter speed to capture an image captured. A second output step for outputting as image data, a calculation step for calculating the number of frames corresponding to the stop period based on the predetermined frame rate and the predetermined shutter speed, and imaging before and after the stop period A generating step of generating an image having the calculated number of frames by combining the image and the still image data. It is an imaging control method according to claim.

(Appendix 7)
The invention according to appendix 7 is a first output function that causes a computer of an imaging apparatus including an imaging unit to drive the imaging unit and output images sequentially captured at a predetermined frame rate, at the predetermined frame rate. When the images are sequentially captured, the image capturing operation at the predetermined frame rate is temporarily stopped, and during the stop period, the image capturing unit is driven and exposed at a predetermined shutter speed as an image captured as still image data. A second output function to be output; a calculation function for calculating the number of frames corresponding to the stop period based on the predetermined frame rate and the predetermined shutter speed; images captured before and after the stop period; and A program for executing a generation function for generating an image of the calculated number of frames by combining still image data A.

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Imaging lens 3 Lens drive part 4 Shutter combined shutter 5 CCD
6 TG
7 Unit Circuit 8 Image Processing Unit 9 Speaker 10 Microphone 11 Audio Processing Unit 13 CPU
14 DRAM
15 Memory 16 Flash Memory 17 Display 18 Key Input 19 Card I / F
20 Memory card

Claims (7)

Imaging means;
First output means for driving the imaging means to output images sequentially taken at a predetermined frame rate;
When images are sequentially captured at the predetermined frame rate, the image capturing operation at the predetermined frame rate is temporarily stopped, and during the stop period, the image capturing unit is driven to perform exposure at a predetermined shutter speed for imaging. Second output means for outputting an image as still image data;
Calculation means for calculating the number of frames corresponding to the stop period based on the predetermined frame rate and the predetermined shutter speed;
An imaging apparatus comprising: generating means for generating an image having the number of frames calculated by the calculating means by combining the images taken before and after the stop period and the still image data.   The generation unit combines the still image data with images captured before and after the stop period, with weighting according to the number of frames calculated by the calculation unit and the supplementary position in the stop period. The imaging apparatus according to claim 1.   The imaging apparatus according to claim 2, wherein the generation unit sets α according to the weighting and performs α blending for each pixel. Resize means for resizing the still image data in accordance with the size of the image output by the first output means;
4. The method according to claim 1, wherein the generation unit further sets an intermediate frame in each frame of the number of frames calculated by the calculation unit as a frame resized by the resizing unit. 5. Imaging device.   5. The image pickup apparatus according to claim 1, wherein the calculation unit uses a value obtained by dividing the predetermined frame rate by the shutter speed as the number of frames. In an imaging control method by an imaging apparatus including an imaging unit,
A first output step of driving the imaging unit and outputting images sequentially captured at a predetermined frame rate;
When images are sequentially captured at the predetermined frame rate, the imaging operation at the predetermined frame rate is temporarily stopped, and during the stop period, the imaging unit is driven and exposed at a predetermined shutter speed for imaging. A second output step for outputting an image as still image data;
A calculation step of calculating the number of frames corresponding to the stop period based on the predetermined frame rate and the predetermined shutter speed;
An imaging control method comprising: generating an image of the calculated number of frames by combining the images captured before and after the stop period and the still image data. In the computer of the imaging device provided with the imaging unit,
A first output function for driving the imaging unit to output images sequentially captured at a predetermined frame rate;
When images are sequentially captured at the predetermined frame rate, the imaging operation at the predetermined frame rate is temporarily stopped, and during the stop period, the imaging unit is driven and exposed at a predetermined shutter speed for imaging. A second output function for outputting an image as still image data;
A calculation function for calculating the number of frames corresponding to the stop period based on the predetermined frame rate and the predetermined shutter speed;
A program for executing a generation function for generating an image of the calculated number of frames by combining images captured before and after the stop period and the still image data.

Images