JP2016058878A - Imaging apparatus, control method therefor, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for suppressing fast fluctuation of an imaging parameter in a portion where a frame is thinned, when recording a captured moving image while thinning the frame in the portion of the moving image.SOLUTION: An imaging apparatus includes: imaging means for capturing a moving image; control means for controlling an imaging parameter of the moving image on the basis of a state of a subject while capturing the moving image; and recording means for recording the moving image, the recording means being configured to perform the recording while thinning a part of captured frames regarding a portion that is captured for a predetermined period, in the moving image. During the predetermined period, the control means makes a followup speed of the imaging parameter relative to a state change of the subject lower than that during any period other than the predetermined period.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an imaging apparatus, a control method thereof, and a program.

  2. Description of the Related Art In moving image shooting, an imaging device capable of shooting a moving image in which the frame rate is partially changed during reproduction (hereinafter referred to as a variable frame rate moving image) within one moving image is known. Since the moving image shot in this manner is subject to slow motion or high speed temporarily, the user can be provided with a different pleasure from a normal moving image.

  With a variable frame rate moving image, the moving image is shot at a constant frame rate, and the recording is performed by partially decimating the frame during recording, thereby adding a slow motion or high speed effect to the moving image. For example, when a movie is shot, the movie is shot at a frame rate of 30 fps, and at the time of recording, the frame is partially thinned out by half (15 fps) and recorded to create a movie that is temporarily played back at twice the speed. can do. A similar moving image can be created even if frames are thinned out during playback without thinning out frames during recording.

  Even in the case of a variable frame rate moving image, it is expected that a moving image is shot with appropriate shooting parameters (exposure, focus, color (white balance), etc.) as in a normal moving image. However, in the case of a variable frame rate moving image, the follow-up speed of exposure, focus, and tint varies between a portion that is played back at normal speed and a portion that is added with the effect of slow motion or high speed. For example, consider a case where automatic exposure (AE), autofocus (AF), and auto white balance (AWB) are controlled at a constant follow-up speed during moving image shooting at 30 fps. In this case, it appears that the AE, AF and AWB control is performed at the double follow-up speed in the portion where the frame is thinned out (15 fps) during recording. Accordingly, the exposure, focus, and color change are accelerated only in that portion, resulting in a moving image that is difficult to see.

  Patent Document 1 is known as a technique that pays attention to the relationship between the recording frame rate, the reproduction frame rate, and the follow-up speed of the shooting parameter. The imaging apparatus of Patent Document 1 changes the tracking speed of AE and AWB according to the relationship between the recording frame rate and the reproduction frame rate.

JP 2012-156886 A

  However, Patent Document 1 does not consider recording a part of a captured moving picture with thinning out frames. Therefore, with the technique of Patent Document 1, it is not possible to suppress high-speed fluctuations in imaging parameters in a portion where frames are thinned out.

  The present invention has been made in view of such a situation, and suppresses high-speed fluctuations in shooting parameters in a portion where a frame is thinned when recording or reproducing a portion of the shot moving image by thinning the frame. It aims at providing the technology to do.

  In order to solve the above-described problems, the present invention provides a shooting unit for shooting a moving image, a control unit for controlling shooting parameters of the moving image based on a state of a subject during the shooting of the moving image, and recording the moving image. Recording means for recording a portion of the moving image taken during a predetermined period by thinning out a part of the photographed frame, and the control means includes the predetermined period Provides an imaging apparatus characterized by lowering the follow-up speed of the shooting parameter with respect to a change in the state of the subject than in a period other than the predetermined period.

  Similarly, in order to solve the above-described problem, the present invention provides a photographing unit for photographing a moving image, a control unit for controlling a photographing parameter of the moving image based on a state of a subject during the photographing of the moving image, and recording the moving image. A recording means for reproducing, and a reproducing means for thinning and reproducing a part of the photographed frame for a portion shot in a predetermined period of the moving image recorded by the recording means, and the control means comprises: In the predetermined period, an imaging apparatus is provided, in which a follow-up speed of the imaging parameter with respect to a change in the state of the subject is reduced as compared with a period other than the predetermined period.

  Other features of the present invention will become more apparent from the accompanying drawings and the following description of the preferred embodiments.

  According to the present invention, it is possible to suppress high-speed fluctuations in shooting parameters in a portion where a frame is thinned when recording or reproducing a portion of the shot moving image by thinning the frame.

5 is a flowchart showing a moving image shooting process executed by the imaging apparatus 200. FIG. 2 is a block diagram illustrating a configuration of an imaging apparatus 200. The figure which shows the specific example of a video recording process.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The technical scope of the present invention is determined by the claims, and is not limited by the following individual embodiments. In addition, not all combinations of features described in the embodiments are essential to the present invention.

[First Embodiment]
FIG. 2 is a block diagram illustrating a configuration of the imaging apparatus 200 according to the first embodiment. In FIG. 2, the system control unit 201 includes, for example, a CPU (MPU), a memory (DRAM, SRAM), and the like, and executes various processes (programs) to control each block of the imaging apparatus 200 or between each block. To control data transfer in the network. Further, the system control unit 201 controls each block of the imaging device 200 in accordance with an operation signal from the operation unit 202 that receives an operation from the user.

  The operation unit 202 includes a power button, a mode switch, a moving image shooting button, and the like. When the power button is operated by the user, a necessary voltage is supplied to each block of the imaging apparatus 200, and a startup process is performed under the control of the system control unit 201. When the mode switch is operated by the user, the operation mode of the system control unit 201 is switched to an arbitrary shooting mode. Examples of the arbitrary shooting mode include a still image shooting mode, a moving image shooting mode, and a frame rate variable mode described later.

  When a moving image shooting button included in the operation unit 202 is operated by the user, an operation signal is transmitted to the system control unit 201, and a moving image shooting process is executed. In the moving image shooting process, the system control unit 201 generates image data via the image processing unit 208 and the memory control unit 209 based on the signal read from the image sensor 207 and writes the image data in the recording medium 210. During moving image shooting, operations such as AF (autofocus) processing by the focus control unit 204, AE (automatic exposure) processing by the aperture control unit 206, and AWB (auto white balance) processing by the image processing unit 208 are performed.

  The focus lens 203 is a lens for adjusting the focus position. The focus control unit 204 controls driving of the focus lens 203. During moving image shooting, by moving the focus lens 203 to the in-focus position based on information from the signal read from the image sensor 207, the moving subject can be focused. It is also possible to perform pan-focus imaging by fixing the focus lens 203 at a predetermined position and reducing the aperture 205 by the aperture controller 206.

  The diaphragm 205 adjusts the amount of light taken into the image sensor 207. The aperture control unit 206 controls the aperture 205 so that the amount of light taken into the image sensor 207 is optimal exposure. In addition to optimal exposure, control is performed so that the exposure is arbitrarily set by the user, and control is performed so that the exposure is over (brighter) or under (darker) than the appropriate value according to the exposure correction value. You can also. During moving image shooting, by controlling the diaphragm 205 based on information from the signal read from the image sensor 207, the optimum exposure can be maintained even if the brightness of the subject changes.

  The image sensor 207 converts the optical image of the subject captured by the lens into an image signal using an image sensor such as a CCD sensor or a CMOS sensor, performs analog-digital conversion, and transmits the image signal to the image processing unit 208. During moving image shooting, the image sensor 207 continues to transmit image signals at a predetermined frame rate (for example, 30 fps).

  The image processing unit 208 performs predetermined pixel interpolation processing and color conversion processing on the data from the image sensor 207 or the data from the memory control unit 209 and transmits the processed data to the memory control unit 209. During moving image shooting, the image processing unit 208 can maintain the color of the subject in an appropriate state by continuously adjusting the white balance based on the setting parameters of the imaging apparatus 200 and the information on the subject. . The image processing unit 208 also has a face detection function, and performs face detection processing on data from the image sensor 207 or data from the memory control unit 209. The focus control unit 204, the aperture control unit 206, and the image processing unit 208 perform AF, AE, and AWB processes that are optimal for the face based on information such as the detected position and size of the face. In addition, during moving image shooting, the image processing unit 208 transmits an image signal from the image sensor 207 at a predetermined frame rate. Thus, the image processing unit 208 performs a predetermined encoding process on each frame and transmits the frame to the memory control unit 209. In addition, the image processing unit 208 can thin out a part of each frame, perform encoding processing using the remaining frames, and transmit the encoded frame to the memory control unit 209. By reproducing the moving image data created in this way at a constant frame rate, a high-speed effect can be applied to a portion where frames are thinned out. For example, the image sensor 207 transmits an image signal at a frame rate (shooting frame rate) of 30 fps, and the image processing unit 208 thins out a frame of a predetermined period in half (corresponds to 15 fps) and transmits the frame to the memory control unit 209. . When the moving image data created in this way is reproduced at a frame rate (reproduction frame rate) of 30 fps, only a predetermined period is reproduced at a double speed. Although the present embodiment has been described by taking an example of a configuration in which frames are thinned out during recording, a configuration in which frames are thinned out during reproduction without thinning out frames during recording may be used.

  A memory control unit 209 controls data writing to the image processing unit 208 and the recording medium 210. During moving image shooting, the memory control unit 209 transmits an image signal encoded at a predetermined frame rate from the image processing unit 208, so that the moving image data can be recorded by writing each frame to the recording medium 210.

  The recording medium 210 is a memory card, a hard disk, or the like that can be attached to and detached from the imaging apparatus 200.

  Next, with reference to FIG. 1, a moving image shooting process executed by the imaging apparatus 200 according to the first embodiment will be described. The processing of each step in the flowchart shown in FIG. 1 is realized by the system control unit 201 executing a predetermined program unless otherwise specified. When the moving image shooting button included in the operation unit 202 is operated by the user when the operation mode of the imaging apparatus 200 is the frame rate variable mode, the processing of this flowchart starts.

  In S100, the imaging apparatus 200 starts shooting a moving image at a predetermined frame rate (for example, 30 fps). The frame rate is automatically determined by the system control unit 201 according to the state of the subject immediately before the moving image shooting button is operated and the parameters such as the settings of the imaging apparatus 200.

  In S101, the imaging apparatus 200 determines whether or not the current timing is included in a frame thinning recording period (a period in which a part of a captured frame is thinned and recorded). In the variable frame rate mode, a moving image is shot at a constant frame rate, and at the time of recording, a frame is partially thinned out and recorded, thereby adding a slow motion or high speed effect to the moving image. If the current timing is not included in the frame skip recording period, the process proceeds to S102. If included, the process proceeds to S107.

  Note that the frame thinning recording period may be determined in advance (before shooting is started), or may be dynamically performed during moving image shooting. For example, when the moving image shooting button is operated, the imaging apparatus 200 can automatically determine the frame thinning recording period according to the state of the subject immediately before the operation and the parameters such as the settings of the imaging apparatus 200. Alternatively, the imaging apparatus 200 may dynamically determine the frame thinning recording period according to the state of the subject during moving image shooting.

  In S <b> 102, the imaging apparatus 200 performs normal AF, AE, and AWB processing using the focus control unit 204, the aperture control unit 206, and the image processing unit 208. This is the same as the AF, AE, and AWB processing performed in the normal moving image shooting mode (not the frame rate variable mode).

  The AF, AE, and AWB processes in S102 are performed based on the state of the subject. In addition, the imaging apparatus 200 may detect a face from the subject by the image processing unit 208, and may perform AF, AE, and AWB processing based on the face state if the detection is successful.

  In the present embodiment, the imaging parameters to be controlled are the focus position, the exposure value, and the white balance parameter, but are not limited thereto. Only some of these three shooting parameters may be controlled objects, or other types of shooting parameters may be controlled objects.

  In step S <b> 103, the imaging apparatus 200 performs a predetermined encoding process on the frame imaged by the imaging element 207. In S104, the imaging apparatus 200 records the frame encoded in S103 on the recording medium 210.

  In S105, the imaging apparatus 200 determines whether or not the normal recording period (period other than the frame thinning recording period) has ended. If the normal recording period has not ended, the process returns to S103, and the next frame is recorded. When the normal recording period ends, the process proceeds to S106.

  As described above, the determination of the frame thinning recording period may be performed in advance or may be performed dynamically during moving image shooting. When the frame thinning recording period is determined, at the same time, the remaining period is determined as the normal recording period. Therefore, when the frame thinning recording period and the normal recording period are determined in advance, in S105, the imaging apparatus 200 determines whether or not the predetermined normal recording period has ended. When the frame thinning recording period and the normal recording period are dynamically determined during moving image shooting, in S105, the imaging apparatus 200 determines whether to end the normal recording period according to the state of the subject. . Also, when the predetermined shooting time has elapsed, it is determined that the normal recording period has ended.

  In step S106, the imaging apparatus 200 determines whether to end moving image shooting. If the movie shooting is not finished, the process returns to S101. When the moving image shooting is finished, the processing of this flowchart is finished.

  Note that the movie shooting end condition is that the shooting time automatically determined by the system control unit 201 in advance according to the state of the subject immediately before the movie shooting button is operated and the parameters such as the settings of the imaging device 200 has elapsed. can do. Alternatively, the imaging apparatus 200 may dynamically determine whether to end moving image shooting according to the state of the subject during moving image shooting. Furthermore, the imaging apparatus 200 also ends the moving image shooting when the moving image shooting end button included in the operation unit 202 is operated by the user during moving image shooting.

  Next, the operation of the imaging apparatus 200 during the frame thinning recording period will be described. In step S <b> 107, the imaging apparatus 200 performs AF, AE, and AWB processing for the frame thinning recording period using the focus control unit 204, the aperture control unit 206, and the image processing unit 208. If normal AF, AE, and AWB processing similar to S102 is performed in the frame thinning recording period, the follow-up speed of the exposure, focus, and color during the frame thinning period changes, resulting in a movie that is difficult to see. For example, if an image is captured at a frame rate of 30 fps in S100 and each frame is thinned to 15 fps and recorded, AF, AE and AWB processes are performed at twice the follow-up speed compared to the case where no thinning is performed. It looks like. Therefore, in S107, the imaging apparatus 200 can create an easy-to-view video by reducing the tracking speed of AF, AE, and AWB processing.

  In S108, the imaging apparatus 200 determines whether to use the frame imaged by the imaging element 207. If a frame is used, the process proceeds to S109. If a frame is not used, the process proceeds to S111. Whether or not to use a frame is determined according to the thinning amount of the frame. For example, when creating a moving image that is played back at twice the speed, it is only necessary to record the frame by halving it, so that the determination is made to use the frame at a rate of once every two times. The frame thinning amount can be automatically determined in advance by the system control unit 201 in accordance with the state of the subject immediately before the moving image shooting button is operated and the parameters such as the settings of the imaging device 200. Alternatively, the imaging apparatus 200 may dynamically determine the thinning amount according to the state of the subject during moving image shooting.

  The AF, AE, and AWB processes in S108 are performed based on the state of the subject as in S102. In addition, the imaging apparatus 200 may detect a face from the subject by the image processing unit 208, and may perform AF, AE, and AWB processing based on the face state if the detection is successful. Further, the imaging apparatus 200 may perform control so as not to switch whether or not the face is used during the frame thinning recording period. For example, if a face is detected by the image processing unit 208 at the start of the frame thinning recording period, the imaging apparatus 200 starts AE, AF, and AWB processing based on the face state. When the face goes out of the angle of view in the middle of the frame thinning recording period and the face detection by the image processing unit 208 fails, the imaging apparatus 200 determines the AE, AF based on the previously detected face state. And continue the AWB process. By doing so, the exposure, focus, and color are largely prevented from fluctuating at the timing when the face goes out of the angle of view during the frame thinning recording period, and an easy-to-view video is created.

  In S109, the imaging apparatus 200 performs a predetermined encoding process on the frame imaged by the imaging element 207. In S110, the imaging apparatus 200 records the frame encoded in S109 on the recording medium 210.

  In S111, the imaging apparatus 200 determines whether the frame thinning recording period has ended. If the frame thinning recording period has not ended, the process returns to S108, and the same process is performed for the next frame. If the frame thinning recording period has ended, the process proceeds to S106. As described above, the frame thinning recording period may be determined in advance, or may be dynamically determined during moving image shooting.

  With the above processing, AF, AE, and AWB processing in which the tracking speed is reduced in S107 is performed in the frame thinning recording period. Accordingly, high-speed fluctuations in the shooting parameters are suppressed, and an easy-to-view moving image is created.

  A specific example of the moving image shooting process according to the first embodiment will be described with reference to FIG. FIG. 3 shows a case where a moving image having a playback speed of “normal speed → double speed → normal speed” is created. The imaging device 207 performs imaging for 6 seconds at a constant frame rate of 30 fps, and creates a total of 180 frames. The first 1 second (30 frames) is a normal recording period. Therefore, the imaging apparatus 200 performs AE, AF, and AWB tracking as usual, and records all 30 frames. The next 4 seconds (120 frames) is a frame thinning recording period. Therefore, the imaging apparatus 200 thins out the frame in half and records 60 frames out of 120 frames. During this time, the imaging apparatus 200 suppresses the tracking speed of AE, AF, and AWB. For example, the imaging apparatus 200 reduces the follow-up speed to a half of the normal speed according to the ratio of frames to be recorded (here, 1/2) among the captured frames. The last one second (30 frames) is a normal recording period. Therefore, the imaging apparatus 200 performs AE, AF, and AWB tracking as usual, and records all 30 frames.

  When the recorded moving image is played back at 30 fps, the first 1 second is played at the normal speed, the next 2 seconds is played at the double speed, and the last 1 second is played at the normal speed. The follow-up speeds of AE, AF, and AWB during the period that is reproduced at the double speed are the same follow-up speeds during reproduction as during the period that is reproduced at the normal speed. Therefore, when a series of moving images are reproduced, the following speeds of AE, AF, and AWB are constant, and an easy-to-view moving image can be created.

  In the above description, the rate at which the follow-up speed is reduced in the frame thinning-out recording period coincides with the proportion of frames that are recorded out of the captured frames. However, these two ratios do not have to completely match. In this case, the tracking speeds of AE, AF, and AWB are not completely constant at the time of reproduction, but the fluctuation is suppressed to some extent, so that the visibility of the moving image is still improved.

  Further, the imaging apparatus 200 may reduce the follow-up speed during the frame thinning recording period to zero. In other words, the imaging apparatus 200 may stop the AE, AF, and AWB processing (imaging parameter control) during the frame thinning recording period. In this case, the imaging apparatus 200 may perform pan-focus imaging by fixing the focus lens 203 at a predetermined position and reducing the aperture 205 by the aperture controller 206. By doing so, the in-focus state can be maintained as much as possible even if the subject moves during the frame thinning recording period.

  Although FIG. 3 shows a moving image that is partially played back at high speed, the same applies to the case of shooting a moving image that is partially played back in slow motion. In this case, the imaging apparatus 200 shoots a moving image at 60 fps, for example, and records the frames for a predetermined period in half (corresponding to 30 fps). Then, the imaging apparatus 200 reduces the follow-up speed of the AF, AE, and AWB processes for the period in which frames are thinned out and recorded. When the moving image obtained in this way is played back at 30 fps, it is played back at normal speed during the frame thinning-out period, and is played back in slow motion during the remaining period.

  As described above, according to the first embodiment, the imaging apparatus 200 reduces the tracking speed of the shooting parameter with respect to the change in the state of the subject during the frame thinning recording period. As a result, it is possible to suppress high-speed fluctuations in the shooting parameters in the portion where the frames are thinned out.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

  DESCRIPTION OF SYMBOLS 200 ... Imaging device 201 ... System control part 204 ... Focus control part 206 ... Aperture control part 207 ... Imaging element 208 ... Image processing part 209 ... Memory control part 210 ... Recording medium

Claims (12)

Shooting means for shooting videos,
Control means for controlling shooting parameters of the moving image based on the state of the subject during shooting of the moving image;
Recording means for recording the moving image, the recording means for thinning a part of the photographed frame and recording the portion taken in a predetermined period of the moving image,
With
The imaging device according to claim 1, wherein the control unit reduces the follow-up speed of the imaging parameter with respect to a change in the state of the subject during the predetermined period as compared with a period other than the predetermined period. Shooting means for shooting videos,
Control means for controlling shooting parameters of the moving image based on the state of the subject during shooting of the moving image;
Recording means for recording the moving image;
Reproduction means for thinning and reproducing a part of a photographed frame for a part photographed in a predetermined period of the moving picture recorded by the recording means;
With
The imaging device according to claim 1, wherein the control unit reduces the follow-up speed of the imaging parameter with respect to a change in the state of the subject during the predetermined period as compared with a period other than the predetermined period. The image pickup apparatus according to claim 1, wherein the control unit reduces the follow-up speed according to a ratio of frames to be recorded among the captured frames in the predetermined period. The imaging apparatus according to claim 1, wherein the control unit stops control of the imaging parameter during the predetermined period. The imaging apparatus according to claim 4, wherein the control unit controls the imaging unit to perform imaging with pan focus during the predetermined period. The imaging apparatus according to claim 1, wherein the shooting parameter includes at least one of a focus position, an exposure value, and a white balance parameter. Further comprising detection means for detecting a face from the subject,
The imaging apparatus according to any one of claims 1 to 6, wherein when the detection unit detects a face, the control unit controls the imaging parameter based on a state of the face. When the detection of the face by the detection unit fails during the predetermined period, the control unit controls the imaging parameter based on the face state detected before the failure. Item 8. The imaging device according to Item 7. The imaging apparatus according to any one of claims 1 to 8, further comprising a determining unit that determines the predetermined period based on a state of the subject before or during the shooting of the moving image. . A method for controlling an imaging apparatus,
A shooting process for shooting videos;
A control step of controlling shooting parameters of the moving image based on a state of a subject during shooting of the moving image;
A recording step of recording the moving image, wherein a portion of the moving image taken during a predetermined period is recorded by thinning out a part of the taken frame;
With
In the control step, in the predetermined period, the follow-up speed of the imaging parameter with respect to a change in the state of the subject is decreased as compared with a period other than the predetermined period. A method for controlling an imaging apparatus,
A shooting process for shooting videos;
A control step of controlling shooting parameters of the moving image based on a state of a subject during shooting of the moving image;
A recording step of recording the moving image;
A reproduction step of thinning out and reproducing a portion of a shot frame for a portion shot during a predetermined period of the moving image recorded in the recording step,
With
In the control step, in the predetermined period, the follow-up speed of the imaging parameter with respect to a change in the state of the subject is decreased as compared with a period other than the predetermined period.   The program for functioning a computer as each means of the imaging device of any one of Claims 1 thru | or 9.

Images