JP2011119934A - Image shooting device and image shooting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To record an appropriate still image even when one still image is being processed for recording and another still image can not be recorded. <P>SOLUTION: A video camera 100 that is one example of an image shooting device for shooting a still image while shooting a moving image, is equipped with: a shooting section 110 for shooting the still image while shooting the moving image; a recording section 120 for recording the still image shot by the shooting section 110 on a recording medium 150; an evaluation section 130 for determining an image evaluation value of a frame by evaluating the moving image in units of frames; and a recording control section 140 that halts recording of a first still image and controls the recording section 120 to record a second still image when an image evaluation value of a second frame corresponding to the second still image shot by the shooting section 110 within a predetermined period from the starting time of recording the first still image shot by the shooting section 110 via the recording section 120 is higher than an image evaluation value of a first frame corresponding to the first still image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image capturing device that captures both a moving image and a still image, and more particularly, to a technique for automatically capturing a suitable still image during moving image shooting.

  2. Description of the Related Art Conventionally, in a video camera, a model that can record a still image together with a moving image when a user presses a still image (still image) shooting button or the like when shooting a moving image (moving image) has been sold. Also, recent video cameras can generate digest (summary) moving images by analyzing attribute information (metadata) such as shooting information at the time of shooting and external input information.

  For example, in Patent Document 1, a moving image is recorded on a magnetic tape during normal shooting. For still image recording, the still image recording switch is operated, and the output signals from the auto exposure (AE) and auto focus (AF) circuits are in the optimum exposure and optimum focus states. If it is, record a still image.

Japanese Patent Laid-Open No. 11-187350

  However, in the above prior art, when a plurality of still images are captured in a short period of time, a part of the captured still images cannot be recorded on a recording medium, and an appropriate still image is stored. There is a problem that cannot be done.

  In order to record a still image on a recording medium, a corresponding processing time such as software processing of the device, data transfer, and writing to the recording medium is required. For example, when recording a high-quality still image with high resolution, it takes a time on the order of tens of milliseconds to complete recording on a recording medium such as a hard disk (HDD: Hard Disk Drive) and a flash memory, During shooting by the video camera, shooting processing and recording processing of a still image on a recording medium are executed in parallel.

  In addition, since the amount of information that can be simultaneously recorded by the recording unit is limited, a certain still image is temporarily held in a primary storage medium such as a RAM (Random Access Memory), or secondary recording from the primary storage medium. While recording on the medium, another still image cannot be recorded simultaneously. In this case, the recording unit completes the recording process of the first still image even when the second still image that can be determined to be more preferable than the first still image is shot while the first still image determined to be preferable is recorded. Until then, a new second still image cannot be recorded.

  Therefore, in view of the above problems, the present invention is capable of recording an appropriate still image even when a still image is being recorded and another still image cannot be recorded. An object is to provide a photographing apparatus and an image photographing method.

  In order to solve the above-described conventional problems, an image capturing apparatus according to the present invention is an image capturing apparatus that captures a still image while capturing a moving image, and captures the moving image while capturing the moving image. An image capturing unit that captures a still image, a recording unit that records the still image captured by the image capturing unit on a recording medium, and a priority of a frame to be recorded as a still image by evaluating the moving image for each frame. An evaluation unit for determining an image evaluation value indicating a degree, and a first image captured by the imaging unit within a predetermined period from the time when the recording unit started recording the first still image captured by the imaging unit. When the image evaluation value of the second frame corresponding to two still images is higher than the image evaluation value of the first frame corresponding to the first still image, the recording of the first still image is stopped, and the second Recording still images And a recording control unit for controlling the urchin the recording unit.

  As a result, when a more suitable frame is detected within a predetermined period after the start of recording of a suitable still image, the still image recording process that has started recording is stopped, and a more suitable frame is detected. Can be recorded as a still image.

  In addition, the recording control unit, when the second still image is captured by the imaging unit within the predetermined period, the image evaluation value of the first frame and the image evaluation value of the second frame When the image evaluation value of the second frame is higher than the image evaluation value of the first frame, the recording of the first still image is stopped and the second still image is recorded. The recording unit may be controlled.

  Thus, by comparing the evaluation values, it is possible to determine whether to continue or stop the still image recording process based on the comparison result.

  In addition, the recording control unit may perform the second period by the photographing unit within the period from when the recording unit starts recording the first still image until the recording is completed as the predetermined period. The comparison may be performed when a still image is taken.

  Accordingly, when a more preferable frame is detected during recording of a certain still image, a more preferable frame can be recorded as a still image by stopping the recording process of the still image being recorded.

  Further, the recording control unit captures the second still image by the imaging unit when the image evaluation value of the first frame is equal to or less than a predetermined first threshold value and within the predetermined period. If so, the comparison may be performed.

  As a result, if the evaluation value of the frame corresponding to the still image being recorded is larger than a predetermined threshold, even if the still image corresponding to the frame having the next highest evaluation value is captured, The recording process can be continued. That is, since a still image having an evaluation value larger than the predetermined threshold can be reliably recorded, for example, when a human smile is detected, the first detected still image can be reliably recorded.

  The first frame is a frame at a time when the first still image is captured by the imaging unit, and the recording control unit further includes a third frame before the first frame included in the moving image. When the image evaluation value of the frame is larger than a predetermined second threshold, the photographing unit is instructed to shoot a still image, and the photographing unit shoots the first still image according to an instruction from the recording control unit. The recording unit may record the first still image captured by the imaging unit on the recording medium.

  As a result, a predetermined delay time occurs from when the shooting instruction is performed based on the evaluation value of the third frame until the first still image is actually shot. Even if an object crosses or a camera shake of the photographer occurs, the possibility of recording an undesired still image can be reduced.

  Further, the evaluation unit further evaluates the moving image for each frame based on the camera work of the image photographing device, and is an evaluation value based on the camera work at the time of photographing, and is recorded as a still image. An operation evaluation value indicating a priority of the frame to be determined, and the recording control unit further selects the first still image when the operation evaluation value of the first frame is greater than a predetermined third threshold value. The recording unit may be controlled to record.

  Thereby, since evaluation is performed based on camera work of the image capturing apparatus such as camera shake or facing the ground, the possibility of recording an undesired still image can be further reduced.

  Further, the image photographing device further includes an attribute information generating unit that generates one or more types of attribute information indicating a shooting state of the moving image for each frame, and the evaluation unit is provided in advance for each type of the attribute information. The moving image is evaluated for each frame based on a first evaluation table in which a predetermined evaluation value is associated, thereby determining the image evaluation value and predetermined for each type of attribute information indicating the camera work. The motion evaluation value may be determined by evaluating the moving image for each frame based on a second evaluation table different from the first evaluation table in which the evaluated values are associated.

  Thereby, since the evaluation value is predetermined for every attribute information, a moving image can be easily evaluated for every frame based on the attribute information matched with the object frame. Furthermore, since evaluation is performed based on a plurality of evaluation tables, a frame to be recorded as a still image can be determined from different viewpoints, and a more suitable still image can be recorded.

  Further, the image photographing device further includes an attribute information generating unit that generates one or more types of attribute information indicating a shooting state of the moving image for each frame, and the evaluation unit is provided in advance for each type of the attribute information. You may determine the said image evaluation value by evaluating the said moving image for every flame | frame based on the 1st evaluation table which matched the defined evaluation value.

  Thereby, since the evaluation value is predetermined for every attribute information, a moving image can be easily evaluated for every frame based on the attribute information matched with the object frame.

  The image capturing method according to the present invention is an image capturing method for capturing a still image while capturing a moving image. The image capturing method is for capturing the moving image and capturing a still image while capturing the moving image. An image evaluation value indicating a priority of a frame to be recorded as a still image by evaluating the moving image for each frame, a recording step for recording the still image captured in the imaging step on a recording medium; An evaluation step for determining, wherein in the recording step, the first still image recorded in the imaging step is recorded in the imaging step within a predetermined period from the time when the recording unit starts. When the image evaluation value of the second frame corresponding to two still images is higher than the image evaluation value of the first frame corresponding to the first still image, the first still image is being recorded. As well as, recording the second still image.

  According to the image capturing device and the image capturing method of the present invention, even when a still image is being recorded and another still image cannot be recorded, an appropriate still image can be recorded. it can.

1 is an external view of a video camera that is an example of an image capturing device that captures moving images and still images according to Embodiment 1. FIG. 2 is a diagram showing an outline of a hardware configuration inside a video camera according to Embodiment 1. FIG. 3 is a block diagram illustrating an example of a functional configuration of the video camera according to Embodiment 1. FIG. 3 is a block diagram illustrating an example of a detailed functional configuration of the video camera according to Embodiment 1. FIG. 4 is a diagram illustrating a configuration of a moving image captured by the video camera according to Embodiment 1. FIG. 6 is a diagram showing an example of information for identifying a scene according to Embodiment 1. FIG. It is a figure which shows an example of the relationship between the attribute information which concerns on Embodiment 1, and evaluation. It is a figure which shows the result of the scene analysis part which concerns on Embodiment 1 assigning evaluation to each scene. 6 is a diagram illustrating a result of evaluating a moving image for each frame from the viewpoint of extracting a still image according to Embodiment 1. FIG. It is a figure which shows an example of the relationship between attribute information and evaluation from a viewpoint of camerawork evaluation which concerns on Embodiment 1. FIG. 4 is a flowchart illustrating processing in which a still image recording control unit according to Embodiment 1 extracts and records a still image. 6 is a diagram illustrating an example of an operation in which the still image recording control unit according to Embodiment 1 stops recording and records a second still image while recording the first still image. FIG. 10 is a flowchart illustrating processing in which a still image recording control unit according to Embodiment 2 extracts and records a still image. FIG. 11 is a diagram illustrating an example of an operation when a second still image is captured while the still image recording control unit according to the second embodiment is recording a first still image.

  Hereinafter, an embodiment of an image photographing apparatus according to the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
When the second still image is captured during recording of the first still image, the image capturing device according to Embodiment 1 uses the evaluation value of the first frame corresponding to the first still image and the second still image. The evaluation value of the corresponding second frame is compared, and if the evaluation value of the second frame is higher than the evaluation value of the first frame, the recording process of the first still image is stopped and the second still image It is characterized by starting the recording.

<Configuration of image capturing device>
First, the configuration of the image capturing apparatus according to Embodiment 1 will be described.

  FIG. 1 is an external view of a movie camera or video camera 100 that is an example of an image capturing device that captures moving images and still images. In Embodiment 1, the video camera 100 will be described as an example.

  FIG. 2 is a diagram showing an outline of a hardware configuration inside the video camera 100 of FIG.

  The video camera 100 includes a lens group 200, an image sensor 201, a video ADC (Analog Digital Converter) 202, a video signal conversion circuit 203, a CPU (Central Processing Unit) 204, a lens control module 206, and an attitude detection sensor. 207, an input button 208, a display 209, a speaker 210, an output I / F (Interface) 211, a compression / decompression circuit 212, a ROM (Read Only Memory) 213, a RAM 214, a memory card 215, and audio. An ADC (Analog Digital Converter) 216 and a microphone 217 are provided as components.

  The lens group 200 is a set of one or more lenses that adjust light incident from a subject in order to form a subject image on the image sensor 201. Specifically, the lens group 200 adjusts the focal length and zoom (moving image magnification) by changing the distance between a plurality of lenses having various characteristics. This adjustment may be made manually by the photographer of the video camera 100, or automatically by control from the CPU 204 or the like through a lens control module 206 described later.

  The image sensor 201 is a photoelectric conversion unit that converts light incident through the lens group 200 into an electrical signal. An image sensor such as a CCD (Charge Coupled Device) and a C-MOS (Complementary Metal Oxide Semiconductor) is used for the image sensor 201.

  The video ADC 202 converts an analog electrical signal output from the image sensor 201 into a digital electrical signal (digital signal). The digital signal is output to the video signal conversion circuit 203.

  The video signal conversion circuit 203 converts the digital signal output from the video ADC 202 into a moving image signal of a predetermined standard such as NTSC (National Television System Committee) or PAL (Phase Alternating Line).

  The CPU 204 is a control unit that controls the entire video camera 100. As the type of control, for example, there is lens control for controlling the incident light to the image sensor 201 by controlling the focal length and zoom of the lens via the lens control module 206. Further, there are input control for external input from the input button 208 and the posture detection sensor 207, operation control of the compression / decompression circuit 212, and the like. The CPU 204 executes these control algorithms with software or the like.

  The lens control module 206 detects the state of the lens group 200 and operates one or more lenses included in the lens group 200 based on the control from the CPU 204. The lens control module 206 includes a lens control motor 206a and a lens position sensor 206b.

  The lens position sensor 206b detects a distance or a positional relationship between a plurality of lenses constituting the lens group 200. Position information between the plurality of lenses detected by the lens position sensor 206b is transmitted to the CPU 204. The CPU 204 transmits a signal for properly arranging a plurality of lenses to the lens control motor 206a based on information from the lens position sensor 206b and information from other components such as the image sensor 201.

  The lens control motor 206 a drives a motor that operates the lens based on the control signal transmitted from the CPU 204. As a result, the distance between the plurality of lenses of the lens group 200 is changed, and the focal length and zoom of the lens can be adjusted. As a result, the lens control module 206 can collect the incident light that has passed through the lens group 200 on the image sensor 201 and form a subject image accurately.

  In addition to the above, the CPU 204 may detect a camera shake at the time of moving image shooting with the video camera 100 with a lens position sensor 206b or a posture detection sensor 207, which will be described later, and drive the lens control motor 206a. . Thereby, the CPU 204 can also execute an operation for preventing camera shake via the lens control module 206.

  The posture detection sensor 207 detects the posture state of the video camera 100. The posture detection sensor 207 includes an acceleration sensor 207a, an angular velocity sensor 207b, and an elevation angle / decline angle sensor 207c. With these various sensors, the CPU 204 detects in what state the video camera 100 is shooting. Note that these sensors are preferably capable of detecting in three axial directions (vertical direction (z axis), horizontal direction (x axis, y axis, etc.)) in order to detect the attitude of the video camera 100 in detail. .

  The input button 208 is one of input interfaces used by the photographer of the video camera 100. Accordingly, the photographer can transmit various requests to the video camera 100, such as the start and end of shooting, and the insertion of a marking into a moving image during video shooting.

  The display 209 is a display unit provided to view a moving image shot by the video camera 100 and to be used as a shooting finder or the like. As a result, the photographer can check the captured video on the spot. In addition to the above, the display 209 displays various information of the video camera 100, so that more detailed information such as shooting information and device information can be transmitted to the photographer.

  The speaker 210 is used for audio output when playing back a captured moving image. In addition, the speaker 210 can transmit a warning output from the video camera 100 to the photographer with sound.

  The output I / F 211 is used to output a moving image shot by the video camera 100 to an external device. Specifically, the output I / F 211 is a cable interface when the external device and the video camera 100 are connected by a cable. Alternatively, the output I / F 211 is a memory card interface or the like for recording a captured moving image on a memory card. As a result, the captured moving image can be viewed using an external display larger than the display 209 provided in the video camera 100.

  The compression / decompression circuit 212 is a circuit that performs an encoding process for changing a captured moving image or sound into a predetermined digital data format. Specifically, moving picture experts group (MPEG) or H.264 is used for captured moving image and audio data. An encoding process is performed in accordance with a standard such as H.264, and converted (compressed) into a predetermined digital data system. The compression / decompression circuit 212 is a circuit that performs a decoding process of decompressing moving image data in a predetermined digital data format and displaying it on the display 209 or the like when reproducing the captured data.

  The ROM 213 is an example of a memory that records software programs processed by the CPU 204 and various data for operating the programs.

  The RAM 214 is an example of a memory used as a memory area or the like used when executing a software program processed by the CPU 204. Further, the CPU 204 may use the RAM 214 in common with the compression / decompression circuit 212.

  The memory card 215 is an example of a recording medium that can be attached to and detached from the video camera 100 and is used for storing moving image data encoded by the compression / decompression circuit 212 and other captured still image data. SD card. Note that the memory card 215 can also record data other than moving image data and still image data. In addition, the video camera 100 may include an HDD instead of the memory card 215.

  The audio ADC 216 converts an audio analog electric signal acquired from the outside by the microphone 217 into a digital electric signal.

  The microphone 217 converts audio outside the video camera 100 into an analog electrical signal and outputs the analog electrical signal.

  As described above, the hardware configuration of the video camera 100 has been described with reference to FIG. 2, but the present invention is not limited to the above configuration. For example, the video ADC 202 and the video signal conversion circuit 203 can be realized as a single integrated circuit. Alternatively, a part of the software program executed by the CPU 204 can be separately realized as hardware by using an FPGA (Field Programmable Gate Array).

  Next, the functional configuration of the video camera 100 will be described with reference to FIGS. 3A and 3B. FIG. 3A is a block diagram illustrating an example of a functional configuration of the video camera 100.

  As shown in FIG. 3A, the video camera 100 includes a photographing unit 110, a recording unit 120, an evaluation unit 130, and a recording control unit 140.

  The imaging unit 110 captures a moving image and also captures a still image while capturing the moving image. Specifically, the imaging unit 110 captures a still image according to an instruction from the recording control unit 140 while capturing a moving image. The imaging unit 110 corresponds to the lens group 200, the image sensor 201, the video ADC 202, the video signal conversion circuit 203, and the like in FIG.

  The recording unit 120 records the still image captured by the imaging unit 110 on the recording medium 150. The recording unit 120 corresponds to an algorithm that is software processed by the CPU 204 of FIG.

  Note that the recording medium 150 may not be included in the video camera 100 as shown in FIG. 3A. When the video camera 100 includes the recording medium 150, the recording medium 150 corresponds to the RAM 214 and the memory card 215 in FIG.

  The evaluation unit 130 determines an image evaluation value indicating the priority of a frame to be recorded as a still image by evaluating the moving image captured by the imaging unit 110 for each frame. When the target frame is to be recorded as a still image, the image evaluation value takes a large value.

  As will be described later, one or more types of attribute information indicating the shooting state of the moving image are associated with the moving image, and the evaluation unit 130 evaluates in advance for each type of the attribute information. The moving image is evaluated for each frame based on the first evaluation table in which values are associated. The evaluation unit 130 corresponds to an algorithm that is software-processed by the CPU 204 in FIG.

  The evaluation unit 130 may further determine an evaluation value based on the camera work of the video camera 100 and an operation evaluation value indicating the priority of the frame to be recorded as a still image. Specifically, the evaluation unit 130 associates a predetermined evaluation value for each type of attribute information indicating the camera work of the video camera 100, based on a second evaluation table different from the first evaluation table. Is evaluated for each frame.

  The recording control unit 140 records the first still image captured by the imaging unit 110 from the time when the recording unit 120 starts, and corresponds to the second still image captured by the imaging unit 110 within a predetermined period. When the image evaluation value of two frames is higher than the image evaluation value of the first frame corresponding to the first still image, the recording unit 120 stops recording the first still image and records the second still image. To control. Note that the frame corresponding to the still image is, for example, a frame at the time when the still image was captured, that is, a frame at the moment when the shutter is dropped by the imaging unit 110.

  The predetermined period is specifically a period from the time when the recording of the first still image is started until the recording is completed. Although the period required for recording one still image varies depending on the size of the still image, the maximum value of the period can be estimated and the estimated period can be set in advance. The recording control unit 140 corresponds to an algorithm that is software processed by the CPU 204 of FIG.

  As will be described later, the recording control unit 140 instructs the photographing unit 110 to shoot a still image. Since the photographing unit 110 shoots a still image based on the instruction, there is a predetermined delay time from when the recording control unit 140 instructs the photographing to when the photographing unit 110 actually shoots a still image. Occurs. Therefore, for example, when the image evaluation value of the third frame prior to the first frame, which is the frame at the time when the first still image is captured, is larger than a predetermined threshold, the recording control unit 140 causes the imaging unit to Instructs to take a still image.

  Hereinafter, a more detailed functional configuration of the video camera 100 will be described with reference to FIG. 3B. FIG. 3B is a block diagram illustrating an example of a detailed functional configuration of the video camera 100.

  The video camera 100 includes, as functional components, a lens unit 300, an imaging unit 301, a video AD conversion unit 302, a signal processing unit 303, a video signal compression unit 304, an imaging control unit 305, and video analysis. Unit 306, lens control unit 307, posture detection unit 308, attribute information generation unit 309, scene analysis unit 310, audio analysis unit 312, audio signal compression unit 313, multiplexing unit 314, and primary storage Unit 315, secondary storage unit 316, display unit 318, audio AD conversion unit 321, microphone unit 322, external input unit 323, still image recording control unit 324, and still image signal expansion unit 325 Prepare.

  The lens unit 300 adjusts the focal length of the light incident from the subject and the zoom magnification (magnification magnification of the moving image). This adjustment is performed under the control of the lens control unit 307. The lens unit 300 corresponds to the lens group 200 in FIG.

  The imaging unit 301 converts the light transmitted through the lens unit 300 into an electrical signal. The imaging unit 301 outputs data in an arbitrary range on the imaging element as moving image data under the control of the imaging control unit 305. In addition to the moving image data, the imaging unit 301 also includes color space information of three primary colors, white coordinates, gain information of at least two of the three primary colors, color temperature information, Δuv (delta uv), and three primary colors or It is also possible to output information such as gamma information of the luminance signal. These pieces of information are output to the attribute information generation unit 309 via the imaging control unit 305. The imaging unit 301 corresponds to the imaging element 201 in FIG.

  The video AD conversion unit 302 performs analog / digital conversion on the electrical signal indicating the moving image data from the imaging unit 301 according to predetermined processing content. The video AD conversion unit 302 corresponds to the video ADC 202 in FIG.

  The signal processing unit 303 converts the digital signal output from the video AD conversion unit 302 into a moving image signal having a predetermined format. For example, the moving image signal in a predetermined format is a moving image signal in a format conforming to the number of horizontal lines, the number of scanning lines, and the frame rate specified by NTSC. The signal processing unit 303 corresponds to the video signal conversion circuit 203 in FIG.

  The video signal compression unit 304 performs a predetermined encoding process on the digital moving image signal output from the signal processing unit 303, thereby realizing data compression and the like. Specific examples include MPEG2, MPEG4, H.264. There are H.264 encoding schemes. The video signal compression unit 304 corresponds to the compression function of the compression / decompression circuit 212 of FIG.

  The imaging control unit 305 controls the operation of the imaging unit 301. Specifically, the imaging control unit 305 controls the imaging unit 301 such as an exposure amount at the time of shooting, a shooting speed, and sensitivity. These control information are also output to the attribute information generation unit 309. The imaging control unit 305 is one of control algorithms that are software processed by the CPU 204 of FIG.

  The video analysis unit 306 extracts the feature of the moving image from the captured moving image signal. In the present embodiment, the video analysis unit 306 detects color information (for example, a color distribution included in a moving image) and information such as white balance. In addition, when the moving image includes a human face, the video analysis unit 306 extracts the feature of the moving image by analyzing the moving image signal such as performing face detection. The color distribution can be detected by confirming color information included in the data forming the moving image signal. Further, face detection can be realized by using pattern matching or the like. The video analysis unit 306 is one of algorithms processed by the CPU 204 in FIG.

  The lens control unit 307 controls the operation of the lens unit 300. The lens control unit 307 includes a zoom control unit 307a, a focus control unit 307b, a camera shake correction control unit 307c, and the like. The lens control unit 307 controls the lens unit 300 and outputs control information related to the control of the lens unit 300 to the attribute information generation unit 309. The lens control unit 307 corresponds to the lens control module 206 in FIG.

  The posture detection unit 308 includes an acceleration sensor 308a, an angular velocity sensor 308b, and an elevation angle / decline angle sensor 308c, and detects the acceleration, angular velocity, elevation angle, depression angle, and the like of the video camera 100. The detected information is information used for the purpose of detecting the attitude of the video camera 100 and its change state. Note that it is desirable that acceleration and angular velocity can be detected in three directions, vertical (z-axis) and horizontal (two directions of x-axis and y-axis). The posture detection unit 308 corresponds to the posture detection sensor 207 in FIG.

  The microphone unit 322 converts ambient sounds into electrical signals and outputs them as audio signals. The microphone unit 322 corresponds to the microphone 217 in FIG.

  The audio AD conversion unit 321 converts the analog audio signal output from the microphone unit 322 into digital audio data. The audio AD conversion unit 321 corresponds to the audio ADC 216 in FIG.

  The voice analysis unit 312 extracts a characteristic sound from the voice data. Here, the characteristic sounds are, for example, a photographer's voice, pronunciation of a specific word, cheers, gunshots, and the like. The identification of these sounds is performed by using a method such as registering in advance the unique frequencies of these sounds (speech) and discriminating them based on the comparison result with the registered frequencies. Thereby, the voice analysis unit 312 can extract a characteristic sound. The voice analysis unit 312 also detects characteristics such as the input level of the sound captured by the microphone unit 322. The voice analysis unit 312 is one of the algorithms processed by the CPU 204 in FIG.

  The audio signal compression unit 313 converts the audio data output from the audio AD conversion unit 321 using a predetermined encoding algorithm. Examples of audio encoding algorithms include MP3 (MPEG Audio Layer-3) and AAC (Advanced Audio Coding). The audio signal compression unit 313 is one of the compression functions in the compression / decompression circuit 212 of FIG.

  The external input unit 323 is an interface that outputs various types of information received from the outside at the time of moving image shooting, for example, button input by a photographer, shooting index information received from the outside via communication, and the like. Note that the shooting index information is, for example, an identification number used for identifying each shooting such as a number for identifying a shooting scene and a number indicating the number of shooting times during movie shooting. The external input unit 323 corresponds to the input button 208 in FIG.

  The attribute information generation unit 309 generates shooting information at the time of shooting a moving image or a still image, external input information, and other information as attribute information (metadata). The following information can be considered as an example of attribute information.

-White balance information-Exposure information-Focal length information-Shooting speed information-Zoom magnification information-Sensitivity information-Color space information for the three primary colors-Gain information for at least two of the three primary colors-Color temperature information-Color distribution information-Face Detection information, face recognition information ・ Camera posture information (acceleration, angular velocity, elevation angle, depression angle, etc.)
・ Shooting time information (shooting start time, end time)
・ Shooting index information ・ User input information ・ Frame rate information ・ Sampling frequency information

  The attribute information includes new information characterizing the moving image scene calculated from the various information at the time of shooting (information calculated by combining various information at the time of shooting and analyzing the combined information, etc. ) Is also included. For example, the attribute information generation unit 309 can calculate camera work such as panning and tilting at the time of shooting by the video camera 100 from information on the camera posture (acceleration, angular velocity, elevation angle, depression angle, etc.). Further, the attribute information generation unit 309 can output the focal length and zoom magnification information as attribute information as it is.

  The attribute information generation unit 309 generates attribute information by extracting and calculating information useful for scene evaluation from various types of information at the time of shooting. Specifically, the attribute information generation unit 309 generates one or more types of attribute information as described above for each frame.

  The scene analysis unit 310 evaluates each of a plurality of scenes constituting the moving image based on the attribute information generated by the attribute information generation unit 309. That is, the scene analysis unit 310 assigns one or more corresponding attribute information for each scene, and evaluates the target scene based on the one or more assigned attribute information. Then, the scene analysis unit 310 selects a scene to be reproduced based on the evaluation result. The scene evaluation and selection method will be described in detail later. As will be described later, in the present embodiment, scene analysis unit 310 assigns one or more attribute information for each frame, and evaluates a moving image for each frame based on the assigned one or more attribute information. Then, the scene analysis unit 310 determines a frame to be recorded as a still image based on the evaluation result.

  The attribute information generation unit 309 and the scene analysis unit 310 are one of algorithms processed by the CPU 204 in FIG.

  The multiplexing unit 314 multiplexes and outputs the encoded moving image data from the video signal compression unit 304 and the encoded audio data from the audio signal compression unit 313. The multiplexing unit 314 may be software executed by the CPU 204 in FIG. 2, or may be processed by the compression / decompression circuit 212.

  The primary storage unit 315 temporarily holds the encoded moving image data output from the multiplexing unit 314, the data in which the encoded audio data is multiplexed, and the still image data from the still image recording control unit 324. It is memory. The primary storage unit 315 corresponds to the RAM 214 in FIG.

  The secondary storage unit 316 is a memory that holds data temporarily held by the primary storage unit 315 for a long period of time as necessary. The operation of recording data from the primary storage unit 315 to the secondary storage unit 316 will be described later. The secondary storage unit 316 corresponds to the memory card 215 or the like in FIG.

  The still image recording control unit 324 extracts a suitable screen as a still image from the captured moving image based on the information generated by the attribute information generation unit 309 or the evaluation result analyzed by the scene analysis unit 310, and performs primary storage. Control to record in the unit 315 and the secondary storage unit 316 is performed. A specific extraction method will be described later. The extracted still image is subjected to processing such as image compression as necessary, and is recorded in the primary storage unit 315.

  The still image signal decompressing unit 325 reads the still image data recorded in the secondary storage unit 316 and displays the read still image on the display unit 318. Note that the still image recorded in the secondary storage unit 316 is not necessarily subjected to processing such as image compression. When the still image recorded in the secondary storage unit 316 is compressed, the still image signal decompressing unit 325 reads the still image data, decompresses the read still image data, and outputs the decompressed data to the display unit 318. . The still image signal expansion unit 325 is one of the functions of the compression / expansion circuit 212 of FIG.

  The display unit 318 displays still images and moving images. The display unit 318 corresponds to the display 209 in FIG.

  Note that the imaging unit 110 illustrated in FIG. 3A corresponds to, for example, the lens unit 300, the imaging unit 301, the video AD conversion unit 302, and the signal processing unit 303 illustrated in FIG. 3B. The evaluation unit 130 corresponds to the attribute information generation unit 309 and the scene analysis unit 310, for example. The recording unit 120 and the recording control unit 140 correspond to, for example, the still image recording control unit 324. Note that the above correspondence is not limited to this.

  The hardware configuration in FIG. 2 and the functional block diagrams in FIGS. 3A and 3B are one mode for implementing the image capturing apparatus according to the present invention, and the present invention is not limited to this.

  The correspondence relationship between the hardware configuration in FIG. 2 and the functional configuration in FIGS. 3A and 3B is not limited to the above description. The above description is shown as an embodiment for implementing the image capturing apparatus according to the present invention, and the functional configuration and the hardware configuration may be different.

<Analysis of captured scene>
Next, scene analysis of a captured moving image will be described.

  FIG. 4 is a diagram illustrating a configuration of a moving image captured by the video camera 100. The unit of the moving image that is taken until the photographer instructs to start shooting and instructs the end of shooting or pause of shooting is referred to as “clip”. When the photographer repeats the start of shooting, the end of shooting, or the pause repeatedly, a plurality of clips are generated. In this case, as shown in FIG. 4, a clip number (Clip # 1 to #n) is assigned to each clip.

  One clip is composed of one or a plurality of “scenes”. A scene is a series of videos that are logically connected. The scene can be arbitrarily set by the scene analysis unit 310. For example, one clip may be set as one scene, that is, “1 clip” = “1 scene”. In addition, a “scene” may be set on the boundary that the screen changes greatly. In this case, when the video analysis unit 306 calculates a motion vector between frames and the magnitude (change) of “motion” is larger than a predetermined value, the scene switching portion may be used.

  In addition, the scene analysis unit 310 may segment the scene based on other shooting information and the like. For example, the scene may be divided by a button input from the photographer. In this case, the scene in the clip can be configured with the clear intention of the photographer. When a plurality of scenes are included in one clip, as shown in FIG. 4, scene numbers (Scene # 1 to #n) are assigned to each scene.

  A scene is composed of one or more “frames”. A frame is an individual still image constituting a moving image. As shown in FIG. 4, frame numbers (Frame # 1 to #n) are assigned to each frame.

  FIG. 5 is a diagram illustrating an example in which the scene analysis unit 310 divides the clip into a plurality of scenes. The scene analysis unit 310 divides the clip based on the attribute information and the like as described above. In FIG. 5, each scene is defined by a “start time” and an “end time”, but the start and end of the scene may be defined by a frame number or the like.

  The “representative time” shown in FIG. 5 indicates the time of a representative frame among a plurality of frames included in each scene. For example, the frame of the representative time is displayed when the scene is displayed as a thumbnail.

  FIG. 6 is a diagram illustrating a relationship example between the attribute information used when the scene analysis unit 310 evaluates each frame and the evaluation. The list shown in FIG. 6 is an example of a first evaluation table according to the present invention. For example, the clip-in (shooting start part) and the clip-out (pre-shooting end part) part can be considered as the introduction part and the important part of the movie, and the taken movie has a high logical meaning. Inferred. Therefore, in the example shown in FIG. 6, the evaluation value of “clip-in” “A” and “clip-out” “F” is 100.

  In addition, “zoom up” “D” and “zoom down” “G” as camera work at the time of shooting are also set to an evaluation value of 30 from the viewpoint of increasing the degree of attention to a specific subject. . As described above, the scene analysis unit 310 stores a numerical evaluation for each attribute information in advance.

  In the example of FIG. 6, the higher the evaluation value, the higher the evaluation (preferred). The scene analysis unit 310 evaluates each frame based on the relationship between the attribute information and the evaluation in FIG.

  When a plurality of pieces of attribute information are given to one frame, the evaluation (evaluation value) assigned to each piece of attribute information may be added. When a plurality of pieces of attribute information are given to one frame, the evaluation (evaluation value) of the attribute having the highest evaluation among the attribute information may be used as the evaluation value of the frame. Furthermore, if various attributes included in the frame are taken into consideration, the average value of the evaluation values of a plurality of attributes may be used as the evaluation.

  Note that the evaluation need not be performed only on the preferred frame. For example, camera shake at the time of shooting may result in a moving image that is difficult for the viewer of the moving image to see, so a frame with such attributes may be evaluated for a deduction (minus point). In the example of FIG. 6, the evaluation for “hand shake” “I” is set as −20, and the evaluation for “ground (vertical downward direction) shooting” “J” is set as −10.

  Note that the relationship between the attribute information and the evaluation in FIG. 6 is not limited to one. For example, the combination data of a plurality of attribute information and evaluation may be switched depending on a mode (for example, landscape shooting, person (portrait) shooting, still life shooting, etc.) taken by the photographer of the video camera 100. Also, a plurality of combination data may be provided in advance, and a plurality of data may be combined (addition of respective evaluation values at a certain ratio, etc.) depending on the shooting mode. In this case, the combination data of the attribute information and the evaluation can be dynamically changed by changing the composition ratio.

  Furthermore, the user may change the item of attribute information and the evaluation value using the external input unit 323 or the like. Thereby, since the evaluation with respect to the attribute which a user considers more important can be improved, the scene analysis part 310 can perform evaluation according to a user's intention more.

  The frame evaluation indicates the priority. That is, a frame with a high evaluation value is a frame that the user or the video camera 100 has determined to be important, and is a frame that the user desires to reproduce in preference to a frame with a low evaluation value.

  FIG. 7 is a diagram illustrating a result of assigning evaluation (priority) to each frame by the scene analysis unit 310. FIG. 7 shows time (frames) on the horizontal axis and evaluation (priority) of each frame on the vertical axis.

  A frame labeled “A” near time 0 in FIG. 7 has an attribute of “clip-in” since it is immediately after the start of shooting. According to FIG. 6, the attribute of “clip-in” has an evaluation (priority) of 100.

  A frame to which “B” is attached has an attribute of “specific voice extraction”. The attribute of “specific voice extraction” is assigned to the target frame by the scene analysis unit 310 based on the voice information obtained by the voice analysis unit 312 and the like. According to FIG. 6, the attribute of “specific voice extraction” has an evaluation (priority) 70.

  Frames labeled with “C” have an attribute of “photographed still (after camera work)”. In other words, the frame “C” indicates an attribute that means that the photographer shoots by moving the camera body of the video camera 100 such as pan and tilt, and then shoots still. It is conceivable that the frame “C” can be judged to have a high value as a moving image for a subject when shooting still. According to FIG. 6, the attribute of “shooting still after camera work” has an evaluation (priority) 40.

  Frames labeled with “D” have a “zoom-up” attribute. In other words, the “D” frame is a frame in which the video camera 100 is zoomed up. In FIG. 6, the “zoom-up” attribute has an evaluation (priority) 30.

  As shown in FIG. 6, the “zoom-down” attribute also has an evaluation (priority) 30. However, it is possible to vary the evaluation value between zoom-up and zoom-down. For example, by setting the zoom-up “D” higher than the zoom-down “G”, a frame that is shot with the zoom-up, that is, a frame in which the enlargement magnification of the moving image becomes large (the subject that is shot with an enlarged image) High evaluation (priority) can be assigned to a certain frame. On the contrary, a relatively low evaluation (priority) can be assigned to a frame in which the enlargement magnification of the moving image is small.

  Frames labeled with “E” have a “pan, tilt” attribute. In other words, the frame “E” is a frame in which the video camera 100 has performed shooting (camera work) with movement such as panning and tilting. In FIG. 6, the attribute of “pan, tilt” has an evaluation (priority) of 25.

  Frames labeled with “I” have a “hand shake” attribute. That is, the “I” scene indicates a frame that is shot with camera shake at the time of shooting. In FIG. 6, the “shake” attribute has an evaluation (priority) of −20. This is because the “I” frame may be a frame that is difficult to see, for example, the screen may be shaken when a moving image is viewed.

  A frame to which “J” is attached has an attribute of “shooting the ground (vertically downward)”. That is, the frame “J” is such that the photographer continues to photograph the ground, which is likely to occur when the photographer accidentally moves while holding the video camera 100 without stopping or temporarily stopping the photographing. Indicates a frame. In FIG. 6, the attribute of “ground (vertical downward direction) shooting” has evaluation (priority) −10.

  As described above, the scene analysis unit 310 assigns an evaluation (priority) for each frame. In the example of FIG. 7, the evaluation is assigned in units of frames. However, the scene analysis unit 310 may perform the above-described evaluation assignment in units of clips or scenes. In this case, the frame is assigned the same evaluation as that assigned to the clip or scene in which the frame is included. FIG. 8 shows an example of only an evaluation value as a result of evaluation assignment in units of frames.

  In addition, the scene analysis unit 310 performs evaluation assignment for camera work evaluation and calculates a camera work evaluation value in order to determine whether a still image recording control unit 324, which will be described later, has successfully captured a still image. The camerawork evaluation value is notified to the still image recording control unit 324 for each frame. For example, the attribute information and the evaluation are set so that the evaluation value becomes low in a frame that is not preferable as a still image to be recorded, such as setting a deduction point of -100 when a camera shake or ground photography is detected. FIG. 9 is a diagram illustrating an example of the relationship between the attribute information for calculating the camerawork evaluation value and the evaluation.

  The scene analysis unit 310 further extracts only preferable frames based on the evaluation assigned to each frame. Specifically, only frames with high evaluation are extracted. In the example of FIG. 7, only the frames in the portions (i) to (iii) are extracted.

  As will be described later, the scene analysis unit 310 according to the first embodiment evaluates moving images for each frame using the list shown in FIG. 6 and also uses the list shown in FIG. 9 for each frame. Evaluate moving images. That is, the scene analysis unit 310 uses the list shown in FIG. 6 to evaluate a moving image for each frame for the purpose of acquiring a suitable still image, and a camera for actually acquiring a still image. For the purpose of confirming the work, the moving image is evaluated for each frame using the list shown in FIG. The list shown in FIG. 9 is an example of a second evaluation table according to the present invention.

  Although an undesired still image is recorded due to a processing delay in the video camera 100 or the like, the possibility of an undesired still image being recorded is reduced by evaluating the camera work as described above. be able to.

  In the first place, the video camera 100 determines the acquisition of the still image based on the evaluation result of the moving image using the list shown in FIG. 6, and then the video camera 100 actually operates the shutter. Processing delays such as internal software processing time and communication between microcomputers occur. During this processing delay, an undesired still image may be recorded due to a person suddenly crossing between the photographing apparatus and the subject or due to camera shake of the photographer.

  Hereinafter, the automatic extraction and recording process of a still image will be specifically described. That is, the operations of the scene analysis unit 310 and the still image recording control unit 324 among the operations of the video camera 100 according to the first embodiment will be described.

<Automatic extraction and recording of still images>
The still image recording control unit 324 acquires the evaluation for each frame from the scene analysis unit 310, and determines whether to acquire a still image based on the evaluation. FIG. 10 is a flowchart illustrating processing in which the still image recording control unit 324 extracts and records a still image. Hereinafter, a still image extraction and recording process will be described with reference to FIG.

  As described above, in the video camera 100 according to Embodiment 1, the acquired moving image is evaluated for each frame by the attribute information generation unit 309 and the scene analysis unit 310 during shooting of the moving image. Specifically, the scene analysis unit 310 evaluates a moving image according to the evaluation list shown in FIG. 6 for acquiring a still image, thereby obtaining an image evaluation value (frame evaluation) for each frame of the moving image shown in FIG. Generate.

  Furthermore, the scene analysis unit 310 evaluates a moving image according to the evaluation list shown in FIG. 9 for camera work evaluation, thereby generating an operation evaluation value (camera work evaluation) for each frame of the moving image shown in FIG. .

  The still image recording control unit 324 acquires the evaluation value (image evaluation value) of the current frame among the evaluation results (frame evaluation) performed by the scene analysis unit 310 for each frame shown in FIG. 11 (step S101). The still image recording control unit 324 selects a frame whose evaluation value is larger than a predetermined threshold A (corresponding to the second threshold according to the present invention) based on the evaluation result for each frame evaluated by the scene analysis unit 310. Frame is determined (step S102). In the example of FIG. 11, for example, the still image recording control unit 324 determines the third frame (or the fourth frame) having an evaluation value exceeding a predetermined threshold A (for example, 50) as a still image acquisition target frame. To do.

  When the still image recording control unit 324 determines a still image acquisition target frame (Yes in step S102), the still image recording control unit 324 instructs the imaging control unit 305 to capture a still image, and the still image captured by the imaging unit 301 is signaled. It is acquired via the processing unit 303 and recorded as still image data in the primary storage unit 315 (step S103). In the example illustrated in FIG. 11, the still image recording control unit 324 determines the third frame (or the fourth frame) as a still image acquisition target frame, and outputs a shooting instruction to the imaging control unit 305. When receiving an imaging instruction, the imaging control unit 305 controls the imaging unit 301 to acquire a still image. The imaging unit 301 captures a still image based on control by the imaging control unit 305.

  Note that the still image recording control unit 324 and the imaging control unit 305 are realized by software that operates on the CPU 204, and other functions also operate on the CPU 204. For this reason, it takes a certain amount of processing time from when the still image recording control unit 324 captures a still image to when the imaging unit 301 actually acquires a still image based on control by the imaging control unit 305.

  That is, there is a delay time from the shooting instruction by the still image recording control unit 324 until the imaging unit 301 actually captures a still image. More specifically, as illustrated in FIG. 11, the still image recording control unit 324 determines the third frame as a still image acquisition target frame, but the first still image actually captured by the imaging unit 301 is determined. The corresponding first frame is a frame later than the third frame by a delay time. In other words, the third frame is a frame before the first frame corresponding to the first still image that is actually captured.

  The still image recording control unit 324 can acquire still image data to be extracted from either data output from the signal processing unit 303 or data output from the video signal compression unit 304. However, when the still image recording control unit 324 extracts still image data from the moving image data encoded by the video signal compression unit 304, a part of the data (data related to the still image) is deleted at the time of encoding. Therefore, it is preferable to extract still image data from the data output from the signal processing unit 303 before encoding.

  The still image recording control unit 324 obtains the camera work evaluation value (operation evaluation value) from the scene analysis unit 310 when the still image is actually taken, that is, the moment when the shutter is released, and the camera work evaluation value is a predetermined value. It is determined whether or not it is larger than a threshold value B (corresponding to a third threshold value according to the present invention) (step S104). When the camerawork evaluation value exceeds the predetermined threshold B (Yes in step S104), the still image recording control unit 324 determines that the acquired still image is an image obtained as a result of successful shooting. In the example shown in FIG. 11, when the first still image (or the second still image) is actually captured, that is, the first frame corresponding to the first still image (or the second frame corresponding to the second still image). Since the camerawork evaluation value of the image exceeds the predetermined threshold value B, the still image recording control unit 324 obtains the acquired first still image (or second still image) as an image obtained as a result of successful shooting. Is determined.

  When still image shooting fails due to camera shake or the like, the camerawork evaluation value falls below a predetermined threshold B (No in step S104), and the still image recording control unit 324 discards the acquired still image from the primary storage unit 315. (Step S109). The camerawork evaluation value is stored in the buffer memory together with the time information of each scene for a predetermined range, and the evaluation value corresponding to the time when the imaging unit 301 releases the shutter may be used. preferable.

  If the still image has been successfully captured and no still image is being processed (No in step S105), the still image recording control unit 324 temporarily stores the acquired still image in the primary storage unit 315. At the same time, transfer to the secondary storage unit 316 is started (step S108). When recording in the secondary storage unit 316, the evaluation value acquired in step S101 is also recorded.

  In the example illustrated in FIG. 11, the still image recording control unit 324 stores the first still image in the primary storage unit 315 and starts recording in the secondary storage unit 316. Here, the still image recording process refers to a first period in which the acquired still image is temporarily stored in the primary storage unit 315, and recording from the primary storage unit 315 to the secondary storage unit 316. It is a recording process in the second period.

  If another still image recording process is being performed (Yes in step S105), the evaluation value of the frame corresponding to the still image being recorded and the frame corresponding to the newly acquired still image, The evaluation value of the current frame is compared (step S106). In the example illustrated in FIG. 11, when the still image recording control unit 324 attempts to store the second still image in the primary storage unit 315, the secondary storage unit 316 is performing the first still image recording process. is there. Therefore, the still image recording control unit 324 compares the evaluation value of the first frame corresponding to the first still image being recorded with the evaluation value of the second frame corresponding to the newly acquired second still image.

  If the evaluation value of the first frame corresponding to the first still image being recorded is higher than the evaluation value of the second frame corresponding to the newly acquired second still image (No in step S106), the newly acquired second The still image is discarded (step S109), and the still image recording process of the first still image being recorded is continued.

  If the evaluation value of the second frame corresponding to the newly acquired second still image is higher than the evaluation value of the first frame corresponding to the first still image being recorded (Yes in step S106), the first still image being recorded The still image recording process of the image is stopped (step S107), and the still image recording process of the newly acquired second still image is started (step S108). In the example shown in FIG. 11, since the evaluation value of the second frame is higher than the evaluation value of the first frame, the still image recording control unit 324 stops the still image recording process of the first still image, and the second still image recording Start the process.

  As described above, with the configuration described in this embodiment, the video camera 100 can perform evaluation for each frame while capturing a moving image, and can automatically capture a still image corresponding to a suitable frame. .

  In video camera 100 according to the present embodiment, there is a delay time from when shooting is determined based on the frame evaluation result until when the shutter actually falls (until the still image is actually acquired). On the other hand, the video camera 100 according to the present embodiment can reduce the possibility that an undesired still image is shot by determining the evaluation value at the moment when the shutter is actually dropped.

  In addition, in video camera 100 according to the present embodiment, a predetermined period is required to record a captured still image in a primary storage unit such as a RAM and a secondary storage unit such as a memory card. On the other hand, in the video camera 100 according to the present embodiment, the evaluation value of the first still image being recorded is compared with the evaluation value of the newly acquired second still image, and the evaluation value of the second still image is compared. If is greater than the evaluation value of the first still image, recording of the first still image is stopped and recording of the second still image is performed.

  Thus, when a more preferable next still image is shot during the still image recording process, a more preferable next still image can be recorded. Accordingly, even when a still image is being recorded and another still image cannot be recorded, an appropriate still image is recorded.

(Embodiment 2)
The image capturing apparatus according to the second embodiment, when attempting to record a captured second still image, performs recording of the first still image being recorded when another first still image recording process is being performed. When the evaluation value is larger than a predetermined threshold, the first still image recording process is continued without recording the second still image. That is, when the evaluation value of the first still image being recorded is smaller than a predetermined threshold, the evaluation value of the first frame corresponding to the first still image and the evaluation value of the second frame corresponding to the second still image are obtained. Compare.

  The difference between the image capturing apparatus according to the second embodiment and the image capturing apparatus according to the first embodiment is the processing method of the still image recording control unit 324. Since other points such as the configuration of the image capturing apparatus according to the second embodiment are the same as those in the first embodiment, the description thereof is omitted.

  The still image recording control unit 324 acquires the evaluation for each frame from the scene analysis unit 310, and determines whether to acquire a still image based on the evaluation. FIG. 12 is a flowchart illustrating processing in which the still image recording control unit 324 extracts and records a still image. Hereinafter, the recording process of the image capturing apparatus according to the second embodiment will be described with reference to FIG.

  In the video camera according to the second embodiment, as in the first embodiment, the attribute information generation unit 309 and the scene analysis unit 310 obtain the evaluation results (frame evaluation and camera work evaluation) for each frame of the moving image shown in FIG. Generate.

  The still image recording control unit 324 acquires the evaluation value (image evaluation value) of the current frame among the evaluation results (frame evaluation) performed for each frame shown in FIG. 13 by the scene analysis unit 310 (step S101). The still image recording control unit 324 determines, as a still image acquisition target frame, a frame whose evaluation value is greater than the predetermined threshold A based on the evaluation result for each frame evaluated by the scene analysis unit 310 (step S102). In the example of FIG. 13, for example, the still image recording control unit 324 determines the third frame (or the fourth frame) having an evaluation value exceeding a predetermined threshold A (for example, 50) as a still image acquisition target frame. To do.

  When the still image recording control unit 324 determines a still image acquisition target frame (Yes in step S102), the still image recording control unit 324 instructs the imaging control unit 305 to capture a still image, and the still image captured by the imaging unit 301 is signaled. It is acquired via the processing unit 303 and recorded as still image data in the primary storage unit 315 (step S103). In the example illustrated in FIG. 13, the still image recording control unit 324 determines the third frame (or the fourth frame) as a still image acquisition target frame, and outputs a shooting instruction to the imaging control unit 305. When receiving an imaging instruction, the imaging control unit 305 controls the imaging unit 301 to acquire a still image. The imaging unit 301 captures a still image based on control by the imaging control unit 305. As in the first embodiment, a predetermined delay time is generated from the shooting instruction until actual shooting.

  The still image recording control unit 324 can acquire still image data to be extracted from either data output from the signal processing unit 303 or data output from the video signal compression unit 304. However, when the still image recording control unit 324 extracts still image data from the moving image data encoded by the video signal compression unit 304, a part of the data (data related to the still image) is deleted at the time of encoding. Therefore, it is preferable to extract still image data from the data output from the signal processing unit 303 before encoding.

  The still image recording control unit 324 obtains the camera work evaluation value (operation evaluation value) from the scene analysis unit 310 when the still image is actually taken, that is, the moment when the shutter is released, and the camera work evaluation value is a predetermined value. It is determined whether or not it is larger than the threshold B (step S104). When the camerawork evaluation value exceeds the predetermined threshold B (Yes in step S104), the still image recording control unit 324 determines that the acquired still image is an image obtained as a result of successful shooting. In the example shown in FIG. 13, when the first still image (or second still image) is actually captured, that is, the first frame corresponding to the first still image (or the second frame corresponding to the second still image). ) Exceeds the predetermined threshold B, the still image recording control unit 324 obtains the acquired first still image (or second still image) as a result of successful shooting. It is determined that

  When still image shooting fails due to camera shake or the like, the camerawork evaluation value falls below a predetermined threshold B (No in step S104), and the still image recording control unit 324 discards the acquired still image from the primary storage unit 315. (Step S109). The camerawork evaluation value is stored in the buffer memory together with the time information of each scene for a predetermined range, and the evaluation value corresponding to the time when the imaging unit 301 releases the shutter may be used. preferable.

  If the still image has been successfully captured and no still image is being processed (No in step S105), the still image recording control unit 324 temporarily stores the acquired still image in the primary storage unit 315. At the same time, recording to the secondary storage unit 316 is started (step S108). When recording in the secondary storage unit 316, the evaluation value acquired in step S101 is also recorded.

  When the still image recording process of another still image is being performed (Yes in step S105), and the evaluation value of the frame corresponding to the still image being recorded exceeds the predetermined threshold C (Yes in step S210). Then, the still image acquired later is discarded from the primary storage unit 315, and the still image recording process of the still image being recorded is continued. In the example illustrated in FIG. 13, since the evaluation value of the first frame corresponding to the first still image exceeds the threshold value C, the first still image is captured even though the second still image is captured by the imaging unit 301. Continue the recording process. As shown in FIG. 13, the threshold value C (corresponding to the first threshold value of the present invention) is larger than the threshold value A. That is, the threshold value C for determining whether to record or discard the second still image is larger than the threshold value A for determining shooting of the first still image (or the second still image).

  Thus, when the scene analysis unit 310 detects a smiling face of a person, or when it is detected that the photographer is shooting focusing on a specific target, the first detected scene is surely detected. Can be recorded.

  If the evaluation value of the frame corresponding to the still image being recorded is smaller than the threshold C (No in step S210), the still image recording control unit 324 newly acquires the evaluation value of the frame corresponding to the still image being recorded and The evaluation value of the frame corresponding to the still image is compared (step S106). If the evaluation value of the frame corresponding to the still image being recorded is higher than the evaluation value of the frame corresponding to the newly acquired still image (No in step S106), the newly acquired still image is discarded (step S109). Continue the process of recording the still image being recorded.

  If the evaluation value of the frame corresponding to the newly acquired still image is higher than the evaluation value of the frame corresponding to the still image being recorded (Yes in step S106), the still image recording process of the still image being recorded is stopped ( In step S107, the newly acquired still image recording process is started (step S108).

  As described above, with the configuration described in this embodiment, the video camera 100 according to this embodiment performs evaluation for each frame while shooting a moving image, and automatically captures a still image corresponding to a suitable frame. Can be done.

  In the video camera 100 according to the second embodiment, when recording a captured second still image, if another first still image recording process is being performed, the first still image being recorded is recorded. If the evaluation value is larger than the predetermined threshold C, the recording process of the first still image is continued without recording the second still image. Accordingly, the first still image having an evaluation value larger than the predetermined threshold C can be reliably recorded. For example, when the smile of a person is detected, the first detected still image is reliably recorded. be able to.

  Note that in the video camera 100 according to the second embodiment, when the second still image is to be recorded and the first still image recording process is being performed, the first still image evaluation value and the second still image are recorded. Although the comparison with the threshold value is performed, the evaluation value of the first still image may be compared with the second threshold value when the first still image is captured. When the evaluation value of the first still image is larger than the second threshold value, the still image recording control is performed for a period until the recording process of the first still image ends or for a predetermined period corresponding to the period. The unit 324 may stop outputting a new still image shooting instruction.

  As described above, the image capturing apparatus and the image capturing method according to the present invention have been described based on the embodiments. However, the present invention is not limited to these embodiments. Unless it deviates from the meaning of this invention, the form which carried out the various deformation | transformation which those skilled in the art can think to the said embodiment, and the form constructed | assembled combining the component in a different embodiment is also contained in the scope of the present invention. .

  For example, in each embodiment, the case where the video camera 100 is used as the image capturing apparatus according to the present invention has been described as an example. However, the present invention is not limited to this embodiment. For example, the present invention can be implemented in a digital still camera. The present invention can also be implemented in a surveillance camera such as a network camera.

  Furthermore, the present invention can realize the above embodiment as a method. For example, it can be realized as a control method in the video camera 100 or the like shown in the above-described embodiment or the like.

  Furthermore, in the first embodiment and the second embodiment, when the still image is completely recorded in the secondary storage unit 316, the display unit 318 may display an icon indicating that the shooting has been successful, or Alternatively, an image obtained by reducing a still image that has been successfully photographed may be displayed. Thus, the photographer can check the recorded still image while shooting the moving image.

  In each embodiment, the frame corresponding to the still image according to the present invention is, for example, the frame at the time when the still image was shot, that is, the frame at the moment when the shutter is dropped by the shooting unit 110. On the other hand, the frame at the moment when the recording control unit 140 decides to perform photographing, that is, a frame that is a predetermined delay time before the actually photographed still image may be used. Further, it may be another frame within the delay time.

  Specifically, as shown in FIGS. 10 and 11, in step S106, the still image recording control unit 324 compares the evaluation value of the first frame with the evaluation value of the second frame. The value may be compared with the evaluation value of the fourth frame.

  Further, as described above, the present invention can be realized not only as an image capturing apparatus and an image capturing method, but also as a program for causing a computer to execute the image capturing method of the present embodiment. Moreover, you may implement | achieve as recording media, such as computer-readable CD-ROM which records the said program. Furthermore, it may be realized as information, data, or a signal indicating the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet.

  In the present invention, some or all of the constituent elements constituting the image capturing apparatus may be configured by one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip. Specifically, the system LSI is a computer system including a microprocessor, a ROM, a RAM, and the like. .

  The image capturing apparatus and the image capturing method according to the present invention have an effect of recording an appropriate still image even when a certain still image is being recorded and another still image cannot be recorded. It can be used for a photographing apparatus such as a digital video camera.

DESCRIPTION OF SYMBOLS 100 Video camera 110 Image pick-up part 120 Recording part 130 Evaluation part 140 Recording control part 150 Recording medium 200 Lens group 201 Image pick-up element 202 Image | video ADC
203 Video signal conversion circuit 204 CPU
206 Lens control module 206a Lens control motor 206b Lens position sensor 207 Posture detection sensor 207a, 308a Acceleration sensor 207b, 308b Angular velocity sensor 207c, 308c Elevation angle / Depression angle sensor 208 Input button 209 Display 210 Speaker 211 Output I / F
212 Compression / decompression circuit 213 ROM
214 RAM
215 Memory card 216 Audio ADC
217 Microphone 300 Lens unit 301 Imaging unit 302 Video AD conversion unit 303 Signal processing unit 304 Video signal compression unit 305 Imaging control unit 306 Video analysis unit 307 Lens control unit 307a Zoom control unit 307b Focus control unit 307c Camera shake correction control unit 308 Posture detection Unit 309 attribute information generation unit 310 scene analysis unit 312 audio analysis unit 313 audio signal compression unit 314 multiplexing unit 315 primary storage unit 316 secondary storage unit 318 display unit 321 audio AD conversion unit 322 microphone unit 323 external input unit 324 still image Recording control unit 325 Still image signal expansion unit

Claims (9)

An image capturing device for capturing a still image while capturing a moving image,
A shooting unit that shoots the moving image and shoots a still image during shooting of the moving image;
A recording unit for recording a still image captured by the imaging unit on a recording medium;
An evaluation unit that determines an image evaluation value indicating a priority of a frame to be recorded as a still image by evaluating the moving image for each frame;
The image evaluation value of the second frame corresponding to the second still image captured by the imaging unit within a predetermined period from the time when the recording unit started recording the first still image captured by the imaging unit. Is higher than the image evaluation value of the first frame corresponding to the first still image, the recording of the first still image is stopped and the recording unit is controlled to record the second still image. An image capturing apparatus comprising a recording control unit. The recording control unit compares the image evaluation value of the first frame and the image evaluation value of the second frame when the second still image is shot by the shooting unit within the predetermined period. The recording unit is configured to stop recording the first still image and record the second still image when the image evaluation value of the second frame is higher than the image evaluation value of the first frame. The image photographing device according to claim 1. The recording control unit, as the predetermined period, includes the second still image by the photographing unit within a period from when the recording unit starts recording the first still image to when the recording is completed. The image capturing apparatus according to claim 2, wherein the comparison is performed when a photograph is taken. The recording control unit captures the second still image by the imaging unit when the image evaluation value of the first frame is equal to or less than a predetermined first threshold value and within the predetermined period. The image photographing device according to claim 2, wherein the comparison is performed in the case. The first frame is a frame at the time when the first still image was shot by the shooting unit,
The recording control unit further captures a still image in the imaging unit when an image evaluation value of a third frame before the first frame included in the moving image is greater than a predetermined second threshold value. Direct,
The photographing unit photographs the first still image according to an instruction from the recording control unit,
The image recording apparatus according to claim 1, wherein the recording unit records the first still image captured by the imaging unit on the recording medium. The evaluation unit further evaluates the moving image for each frame based on the camera work of the image photographing device, and is an evaluation value based on the camera work at the time of photographing, and should be recorded as a still image Determine the action evaluation value indicating the priority of the frame,
The recording control unit further controls the recording unit to record the first still image when the motion evaluation value of the first frame is larger than a predetermined third threshold value. Image shooting device. The image photographing device further includes:
An attribute information generating unit that generates one or more types of attribute information indicating a shooting state of the moving image for each frame;
The evaluation unit determines the image evaluation value by evaluating the moving image for each frame based on a first evaluation table in which an evaluation value predetermined for each type of the attribute information is associated. By evaluating the moving image for each frame based on a second evaluation table different from the first evaluation table, which associates a predetermined evaluation value for each type of attribute information indicating the camera work, The image photographing device according to claim 6, wherein an operation evaluation value is determined. The image photographing device further includes:
An attribute information generating unit that generates one or more types of attribute information indicating a shooting state of the moving image for each frame;
The evaluation unit determines the image evaluation value by evaluating the moving image for each frame based on a first evaluation table in which an evaluation value predetermined for each type of the attribute information is associated. The image photographing device according to any one of 1 to 6. An image capturing method for capturing a still image while capturing a moving image,
A shooting step of shooting the moving image and shooting a still image during shooting of the moving image;
A recording step of recording the still image shot in the shooting step on a recording medium;
An evaluation step for determining an image evaluation value indicating a priority of a frame to be recorded as a still image by evaluating the moving image for each frame;
In the recording step,
The image evaluation value of the second frame corresponding to the second still image photographed in the photographing step within a predetermined period from the time when the recording unit started recording the first still image photographed in the photographing step. Is higher than the image evaluation value of the first frame corresponding to the first still image, the recording of the first still image is stopped and the second still image is recorded.

Images