JP5655668B2 - Imaging apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an imaging device having a photographing function such as a video camera, and an image processing method and program in the imaging device, and more particularly to a technique for selecting a still image of a desired quality from still images and moving images.

  When searching for a moving image or a still image in which a specific subject is photographed, a visual search is not practical because the amount of moving images increases and the work becomes enormous. Therefore, conventionally, a technique for automatically searching for a moving image or the like in which a specific subject is photographed has been proposed. For example, Patent Document 1 discloses a method for searching for each object prior to searching for a moving image or a still image including a desired object (for example, a person, a face, a pet, etc.) from a large number of moving images and still images. An image information processing apparatus is disclosed in which the information is automatically added to a moving image (data) or a still image (data).

  This image information processing apparatus extracts object related information, which is information related to an object, from related content previously associated with a moving image or a still image for one object included in the moving image or the still image, and corresponds to the moving image Object-related information data corresponding to the extracted object-related information is stored in association with moving image data or still image data corresponding to a still image.

JP 2008-160408 A

  However, in the image information processing apparatus described in Patent Literature 1, for a recorded moving image or still image, an object included in these images is detected, object related information is extracted, and the extracted object related information is extracted. A desired image cannot be extracted unless analysis processing such as storing corresponding object-related information data is executed. In other words, prior to the search, analysis processing of the recorded moving image or the like is necessary, and a quick search or image extraction cannot be performed.

  The present invention has been made to solve such problems, and generates information about the quality of frame images or still images constituting a moving image before the image is captured and recorded. Accordingly, an object of the present invention is to provide an imaging apparatus, an image processing method, and a program capable of automatically selecting images quickly.

  The present invention also provides an imaging device and image processing capable of automatically acquiring a captured image having a certain quality by generating information for automatically selecting a captured image having a desired quality. An object is to provide a method and a program.

Imaging device according to the present invention includes an imaging unit for acquiring moving image, while shooting a moving picture, a selection information acquiring unit for acquiring information on the quality of a frame picture image constituting the front kidou image, the sorting information based on the acquisition unit has acquired the information, and the selection information generating unit that generates an information image selection information is for sorting according to the video to its quality, based on the image selection information the screened information generating unit has generated An image determination unit that determines whether each frame image constituting the moving image is appropriate as a target still image to be extracted, and a recording unit that records the image selection information together with the moving image, The determination unit indicates whether or not each frame image constituting the moving image is inappropriate as the target still image based on an imaging condition while the imaging unit acquires the moving image. Size Information is generated, and for each frame image constituting the moving image, second determination information indicating whether or not the frame image is suitable as the target still image is generated, and the recording unit includes the first and the second Second determination information or comprehensive determination information obtained from the first and second determination information is recorded together with the moving image as part or all of the image selection information .

In the present invention, since the selection information generation unit generates image selection information that is information related to the quality of the frame image or still image constituting the moving image that is the captured image, the user can extract the still image from the moving image. The desired quality image can be selected. Further, the image selection information is generated while the captured image is captured and before the captured captured image is recorded. That is, since it is generated before the compression processing of the moving image data, for example, if it is stored separately from the moving image data, it is possible to automatically select the compressed moving image data without decompression processing. Further, by using the first and second determination information, the user can narrow down frame images with higher quality.

Image processing method according to the present invention, during the shooting a video, and selection information acquisition step of acquiring information about the quality of the frame picture images constituting the moving image based on the information acquired by the selection information acquiring step The moving image is configured based on a selection information generating step for generating image selection information, which is information for selecting the moving image according to its quality, and the image selection information generated in the selection information generating step. An image determining step for determining whether each frame image is appropriate as a target still image to be extracted; and a recording step for recording the image selection information together with the moving image , wherein the image determining step includes the moving image First determination information indicating whether or not each frame image constituting the moving image is inappropriate as the target still image is generated based on an imaging condition while acquiring And a step of generating, for each frame image constituting the moving image, second determination information indicating whether or not the frame image is suitable as the target still image, and the recording step includes the first step And the second determination information, or the comprehensive determination information obtained from the first and second determination information, as a part or all of the image selection information, is recorded together with the moving image. .

Program according to the present invention, during the shooting a video, and selection information acquisition processing for acquiring information on the quality of the frame picture images constituting the moving image based on the information acquired by the selection information acquiring process, the A selection information generation process for generating image selection information, which is information for selecting a moving image according to its quality, and each frame constituting the moving image based on the image selection information generated in the selection information generation process A program for causing a computer to execute an image determination process for determining whether an image is appropriate as a target still image to be extracted, and a recording process for recording the image selection information together with the moving image , the image determination The processing is based on imaging conditions while acquiring the moving image, and for each frame image constituting the moving image, the frame image is set as the target still image. Processing for generating first determination information indicating whether or not the image is inappropriate, and generation of second determination information indicating whether or not each frame image constituting the moving image is suitable as the target still image The recording process includes the first and second determination information or the comprehensive determination information obtained from the first and second determination information as part or all of the image selection information. And a process of recording together .

  According to the present invention, an image pickup apparatus, an image processing method, and a program capable of quickly automatically selecting an image by generating information for automatically selecting the image before capturing and recording the image. Can be provided.

  In addition, according to the present invention, an imaging device capable of automatically acquiring a captured image having a certain quality by generating information for automatically selecting a captured image having a desired quality, An image processing method and program can be provided.

1 is a block diagram showing an overall configuration of an imaging apparatus (digital video camera) 1 according to Embodiment 1 of the present invention. It is a flowchart which shows the production | generation method of the moving image data with image selection information concerning Embodiment 1 of this invention. It is a block diagram which shows the imaging device concerning Embodiment 2 of this invention. It is a figure which shows the to-be-photographed object information contained in a certain frame image. FIG. 5 is a diagram showing subject information included in a frame image that is temporally one after the frame image shown in FIG. 4. It is a flowchart which shows the production | generation method of the moving image data with image selection information concerning Embodiment 2 of this invention. It is a block diagram which shows the imaging device concerning Embodiment 3 of this invention.

  A preferred embodiment of an imaging apparatus according to the present invention will be described below with reference to the drawings. Specific numerical values and the like shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

<Embodiment 1 of the present invention>>
In the present embodiment, image selection information that is information relating to the quality of a frame image or still image constituting a moving image that is a captured image is generated and added to the captured image. Here, by using this image selection information, it is typically possible to select a still image of a desired quality from the moving image data.

  The video data is composed of a large number of frame images, and all the frame images can be extracted as still images.For example, a frame image at a time when the zoom magnification is being changed or the shooting state is unstable is regarded as a still image. Not suitable for extraction. However, in order for the user to select this, it is necessary for the user to monitor the movement of the subject and adjust the selection timing while reproducing the moving image. Even if the user adjusts the timing and extracts a still image, a still image with a desired quality is not always obtained depending on the shooting conditions and the like.

  In this specification, “quality” refers not only to the beauty of image quality and sharpness depending on the shooting conditions, but also to whether the desired subject is included, the size / position / direction of the subject, the smile of the subject. It is assumed that “quality” also includes whether the subject included in the image, such as the degree, is reflected in the form desired by the user. That is, if the subject included in the image is captured in the form desired by the user, it can be evaluated that the quality is high.

  Further, for example, if a person's face or family face is registered and a frame image showing the registered person can be automatically selected or picked up, the still image extraction process by the user is facilitated. Further, based on the shooting conditions at the time of shooting, subject information of the shot image, etc., it is determined in advance whether each frame image is a frame image that is not suitable for still image extraction or whether it is a suitable frame image. If it exists, it can assist the user in extracting a still image.

  Therefore, in the first embodiment, information related to shooting conditions is acquired at the time of shooting a moving image, and image selection information is generated based on the acquired information and recorded together with moving image data. By recording the image selection information together with the moving image data, when extracting a still image from the moving image data, it is possible to execute an automatic selection process for removing a frame image having insufficient quality for extraction as a still image. Here, prior to describing the characteristic configuration of the imaging apparatus according to the present embodiment, first, the overall configuration of the imaging apparatus according to the present embodiment will be described in order to facilitate understanding of the present invention.

  FIG. 1 is a block diagram showing an overall configuration of an imaging apparatus (digital video camera) 1 according to an embodiment of the present invention. The imaging device 1 can capture moving images and still images.

  The present invention can be used in other electronic devices such as a digital still camera, a mobile phone, a PHS (Personal Handyphone System), and a PDA (Personal Digital Assistant) as long as it can capture a moving image. .

  The imaging apparatus 1 includes an imaging unit 100 including a zoom lens 101, a focus lens 102, a diaphragm 103, and an imaging element 104. The zoom lens 101 is moved along the optical axis LA by a zoom actuator (not shown). Similarly, the focus lens 102 moves along the optical axis LA by a focus actuator (not shown). The diaphragm 103 operates by being driven by a diaphragm actuator (not shown).

  Imaging using the imaging apparatus 1 is performed according to the following procedure. The image sensor 104 photoelectrically converts light that has passed through the zoom lens 101, the focus lens 102, and the aperture 103 to generate an analog image signal of the subject.

  After the analog image signal processing unit 105 amplifies the analog image signal, the A / D conversion unit 106 converts the amplified signal into digital image data. The image input controller 107 takes in the image digital data output from the A / D conversion unit 106 and stores it in the line memory 203 or the main memory 204 via the bus 200.

  When the imaging apparatus 1 captures a moving image, the digital image data output from the A / D conversion unit 106 is stored in the main memory 204 via the bus 200. Hereinafter, moving image digital image data is referred to as moving image data.

  When acquiring a still image, a still image frame (frame image) is acquired from the digital image data acquired by the image sensor 104 at an arbitrary timing, and frame data that is a part of the frame image is stored in the line memory 203. Then, after performing predetermined compression processing and the like to generate digital image data, the main memory 204 stores the digital image data.

  By processing as described above, when the imaging apparatus 1 captures a moving image and a still image at the same time, after the still image data is directly compressed by the line memory 203, the compressed still image data is stored in the main memory. Therefore, the amount of data stored in the main memory 204 can be reduced. As a result, the bus bandwidth of the main memory 204 can be ensured, and for example, compressed still image data such as still image frames acquired 60 frames per second can be written to the main memory 204.

  The digital signal processing unit 108 takes in the digital image data stored in the main memory 204 based on a command from the central control unit 400 via the bus 200, performs predetermined signal processing, and includes a luminance signal and a color difference signal. Generate data.

  The digital signal processing unit 108 also performs offset processing from sensitivity information at the time of imaging output from the imaging device 104 or the like, for example, AGC (Automatic Gain Control) gain or ISO (International Organization for Standardization) sensitivity. Various digital corrections such as white balance adjustment processing, gamma correction processing, RGB interpolation processing, noise reduction processing, contour correction processing, color tone correction processing, and light source type determination processing are performed.

  The multiplexing unit 109 multiplexes the digital image data stored in the main memory 204 to generate stream data.

  The compression / decompression processing unit 201 performs predetermined compression processing on the digital image data stored in the line memory 203 or the main memory 204 in accordance with an instruction from the central control unit 400 via the bus 200 to generate compressed data. Further, in accordance with a command from the central control unit 400, the compressed data stored in the card-type recording medium 302 or the like is subjected to decompression processing in a predetermined format to generate uncompressed data.

  In the imaging apparatus 1 according to the present embodiment, the compression method based on the JPEG standard is used for still image data, and the moving picture experts group (MPEG) 2 standard or AVC / H. A compression method conforming to the H.264 standard is adopted.

  The sound / image processing unit 202 performs predetermined image processing on the digital image data read from the main memory 204 in accordance with an instruction from the central control unit 400 via the bus 200. For example, image data for various processing such as a menu image and an OSD image is generated, and the image data is superimposed on the original image digital data read from the main memory 204 and output to the liquid crystal monitor 304. With this output, the image displayed on the liquid crystal monitor 304 is a combination of various image data.

  The flash ROM 205 stores various setting information related to the operation of the imaging apparatus 1, such as user setting information. In addition to the above-described processes, the main memory 204 is used as a calculation work area for the central control unit 100 and also as a temporary storage area for various data. The VRAM 206 is used as a temporary storage area for image data for display.

  The media control unit 207 controls data writing to and data reading from the card-type recording medium 302 through the card I / F 301 in accordance with instructions from the central control unit 400.

  The liquid crystal monitor 304, the speaker 305, the operation unit 306, and the input / output terminal 307 are connected to the input / output I / F 303. The liquid crystal monitor 304 displays various images such as image data and various menu images temporarily recorded in the main memory 204, for example. The speaker 305 outputs sound temporarily recorded in the main memory 204, for example. The operation unit 306 includes an operation key including a relay switch and a power switch (not shown), a cross key, a joystick, a touch panel superimposed on the liquid crystal monitor 304, and the like. Accept.

  The input / output terminal 307 is connected to a television monitor (not shown), a PC (Personal Computer), or the like.

  The central control unit 400 is configured by a semiconductor integrated circuit including a CPU (Central Processing Unit), a ROM storing various programs, a RAM as a work area, and the like. Overall operations such as image display processing, various data recording processing to the card-type recording medium 302, selection information extraction processing, selection information generation processing, image determination processing, and automatic image selection processing described later. Control.

<Detailed Configuration of Embodiment 1>
Next, the imaging apparatus according to the present embodiment will be described in detail. As described above, in this embodiment, when a still image is cut out from moving image data, image selection information is generated to enable automatic selection based on whether or not the still image has a desired quality. Record along with the captured image. By using this image selection information, it is possible to automatically select a captured image.

  In the present embodiment, for example, a case will be described in which it is possible to automatically select a frame image of a certain quality or more as a still image extraction candidate from one moving image file. Therefore, it goes without saying that the present invention can also be applied to a case of automatically selecting a moving image file including frame images of a certain quality or more. In the following description, a frame image (target still image) that can be extracted as a still image from moving image data is also referred to as a capture frame.

  In the present embodiment, the central control unit 400 includes a selection information acquisition unit 401, a selection information generation unit 402, an image determination unit 403, and an automatic image selection unit 404. The sorting information acquisition unit 401 acquires information related to the quality of the frame image constituting the moving image that is the captured image. Based on the information acquired by the selection information acquisition unit 401, the selection information generation unit 402 generates image selection information that is information for selecting an image according to its quality. The selection information generation unit 402 acquires imaging condition information as quality-related information acquired while the imaging unit 100 acquires a moving image from the selection information acquisition unit 401, and based on the acquired imaging condition information, Image selection information is generated for each frame image constituting the moving image.

  Here, the selection information acquisition unit 401 acquires, as a shooting condition, for example, at least one of focal length information, shutter speed, aperture information, and sensitivity information at the time of shooting for each frame image or at regular intervals. As the focus information, focal length information at the time of imaging is acquired from a zoom actuator (not shown) of the zoom lens 102 described above for each frame image or at a certain timing. As the shutter speed information, the exposure period of the image sensor included in the image sensor 104 is acquired. As the aperture information, the aperture size is acquired from the aperture actuator (not shown) of the aperture 103 described above. As the sensitivity information, a sensitivity value such as an AGC gain of the image sensor 104 is acquired. Note that the information is not limited to the above example as long as the information is suitable for still image extraction.

  The selection information generation unit 402 receives the above-described shooting condition information from the selection information acquisition unit 401, and generates the image selection information based on this. Here, the selection information generation unit 402 can simply use the imaging condition information as image selection information as it is. In order to automatically select images, it is necessary to evaluate the shooting condition information as described later. This may be performed by the selection information generation unit 402, the image determination unit 403, or the image. When the determination unit 403 is not provided, the image automatic selection unit 404 may perform the determination. In other words, the imaging condition information may be evaluated before the selection stage of the automatic image selection unit 404.

  As will be described later, if the selection information generation unit 402 generates image selection information before the compression / decompression processing unit 201 compresses the moving image data, for example, it can be compressed together with the moving image data. Also, if the video data is separate from the video data, the image selection information can be referred to without decompressing the compressed video data. For example, without automatically reproducing the compressed video data, the image can be automatically selected. Is possible.

  The selection information generation unit 402 obtains each evaluation value from the photographing condition information, which is used as image selection information. The image determination unit 403 and the image automatic selection unit 404 determine each frame image based on the evaluation value. You may determine whether it is unsuitable as a capture frame. In addition, the selection information generation unit 402 may obtain an evaluation value of the shooting condition information, determine whether or not it is inappropriate as a capture frame, and use the determination result as image selection information. In this case, the image determination unit 403 is unnecessary, and the processing in the image automatic selection unit 404 is accelerated.

  Based on the image selection information generated by the selection information generation unit 402, the image determination unit 403 determines whether each frame image constituting the moving image is appropriate as an extracted still image to be extracted. In the present embodiment, the image determination unit 403 generates first determination information indicating whether or not the frame image is inappropriate as an extracted still image for each frame image constituting the moving image based on the imaging conditions. As described above, since the selection information generation unit 402 generates image selection information for each frame image, the image determination unit 403 has a quality that the frame image can withstand extraction as a still image for each frame image. It is possible to determine whether or not it is present, that is, whether or not it is inappropriate as a capture frame, and the first determination information as a result can be added to the moving image data as image selection information.

  The first determination information can simply represent whether or not the frame image is inappropriate as a frame image to be extracted as a still image, for example, by a 1-bit flag of 1 or 0. When extracting a still image from moving image data, a frame image whose flag is 1 (unsuitable as a frame image to be extracted as a still image) can be automatically removed by the image automatic selection unit 404 in advance. The desired still image can be extracted from the moving image data from the frame image excluding the frame image that is not suitable for extraction as a still image.

  Here, in the present embodiment, the image selection information generated by the selection information generation unit 402 and the first determination information generated by the image determination unit 403 are sent from the central control unit 400 to the multiplexing unit 109. . The multiplexing unit 109 multiplexes the image selection information and the first determination information together with the moving image data compressed by the compression / decompression processing unit 201, sends the multiplexed information to the media control unit 207, and the card type via the card I / F 301. Recording on the recording medium 302 is performed.

  The image selection information including the first determination information is stored in a file (may be a hidden file) different from the moving image data, and the moving image data, the image selection information, May be tied together. Alternatively, the image data and the image selection information may be stored in the same file, and the moving image data and the image selection information may be linked through the moving image data header.

  At least the moving image data and the image selection information are stored so as to be temporally linked. For example, 00:01:00 to 00:03:00 may be recorded such that focus = focus, focal length = 5.2, F = 3.0, shutter speed = 1/60, and the like. Alternatively, the frames 120 to 140 may be recorded such that focus = chorus, focal length = 5.2, F = 3.0, and shutter speed = 1/60.

  The automatic image sorting unit 404 uses image sorting information to automatically remove a frame image that is inappropriate for extraction as a still image from each frame image constituting a moving image in response to a user request or the like. Execute the process. A frame image unsuitable for extraction refers to a value at which an evaluation value calculated from a shooting condition value or shooting condition information is determined that the frame image does not have a predetermined quality. is there. When the imaging condition value is a value determined not to have a predetermined quality, the frame image can be automatically set as a still image extraction target. Further, if the flag such as the first determination information is provided, the automatic sorting process can be performed according to the value of the flag, and the sorting process can be performed at a very high speed.

<Photographing condition information and evaluation>
As described above, the selection information acquisition unit 401 includes, for example, focal length information, shutter speed (image sensor exposure period) information, aperture (iris) information, and sensitivity (AGC gain or AGC gain) as shooting condition information. Any one or more of (ISO sensitivity) information and the like are acquired from the imaging unit 100 for each frame image or at regular intervals.

  If various parameters indicating shooting conditions such as focal length, shutter speed, aperture, and sensitivity are fluctuating, that is, if there is a change in the shooting environment such as zoom movement or illuminance change, focus, gain, white balance, etc. Since the state of the image is unstable, it is possible to determine that the frame image is not suitable for extraction as a still image.

Hereinafter, an example of a specific method for using the shooting condition information will be described.
Focal length:
When the focal length fluctuates by a certain value or more between frame images, that is, a predetermined threshold value or more, it can be determined that zooming or focusing is in progress and can be excluded from the capture frame.

shutter speed:
When the shutter speed (the exposure period of the image sensor) is longer than 1/60 or 1/50, the same image continues several frames. For example, when the shutter speed is 1/30, the image is updated every two frames. Therefore, in such a case, only one frame image can be extracted as a capture frame from consecutive images, and other frame images can be excluded from the capture frame. For example, in the case of NTSC 1/15 shutter, the image is updated every 4 frames, such as “Image 1, Image 1, Image 1, Image 1, Image 2, Image 2, Image 2, Image 2,. For example, only the first frame image of the four frames can be used as a capture frame, and other frame images can be excluded from the capture frame. Thereby, capture of the same scene (image) can be prevented.

Aperture:
When the iris (iris) fluctuates by more than a certain value between frames, changes in lens control such as zoom and focusing, and image processing such as gain adjustment and white balance change. It can be determined that there is a possibility of a change or zoom operation. Therefore, when the aperture (iris) fluctuates more than a certain value between frames, the frame image can be excluded from the capture frame.

sensitivity:
If the sensitivity (AGC gain or ISO sensitivity) fluctuates by more than a certain value between frames, changes in lens control such as focusing and aperture, and changes in image processing such as white balance occur, and the illuminance of the subject It can be determined that there is a change. Therefore, when there is a fluctuation of a certain value or more between frames, the frame image can be excluded from the capture frame.

  Note that the shooting condition information is not limited to these values, and any information can be used as long as the shooting conditions at the time of moving image shooting can be determined. That is, any value that can be used for automatic selection of still image extraction may be used. However, information such as the focal length, shutter speed, aperture, and sensitivity is the most preferable information to be acquired at the time of shooting, and it is difficult to acquire by analysis after moving image shooting, or an accurate value is obtained. It is something that cannot be done. Such information that cannot be obtained later by moving image analysis or the like, is difficult to obtain, or requires complicated analysis processing to be obtained in advance, is not suitable for still image extraction. Can be used very effectively when automatically selecting a frame image.

  In addition, since these shooting condition information is acquired for each frame image or at regular intervals, it is possible to calculate how much the focal length or the like changes between the frame images. Therefore, the selection information generation unit 402 may calculate the amount of change between the frame images and generate the amount of change as image selection information together with the shooting condition information or instead of the shooting condition information. By using the amount of change as the image selection information, the image determination unit 403 and the automatic image selection unit 404, as described above, for example, if the focal length has changed by a certain value or more, the frame image is in focus. Therefore, it can be instantly determined that the frame image is inappropriate as a capture frame.

  The image determination unit 403 determines whether or not each frame image is unsuitable for the capture frame based on the shooting condition information, and generates first determination information as a result. Here, the image determination unit 403 provides a reference for setting a threshold for each shooting condition information with respect to the shooting condition information value or the amount of change between the frame images, and any one of the shooting condition information can be set. When the standard is not satisfied, that is, for example, in the case of the focal length, shutter speed, aperture, sensitivity, etc., when the amount of change exceeds a predetermined threshold, it is indicated that the frame image is not suitable as a capture frame. 1 determination information can be generated.

  Further, if any two or more or all of the photographing condition information values exceed the threshold value, it may be determined that it is not suitable as a capture frame. Furthermore, the user may specify the quality of the still image that the user wants to extract, and determine that all the frame images that are lower than the quality are not suitable as capture frames. The image determination unit 403 determines the image in advance based on the shooting condition information, so that the user can extract the still image from the moving image data, and the automatic selection process by the image automatic selection unit 404 can be accelerated. it can.

  On the other hand, by not providing the image determination unit 403 or by recording the shooting condition information together with the first determination information as the image determination information, the image automatic selection unit 404 uses the shooting condition information for the image automatic selection process. can do. That is, the image automatic selection unit 404 determines whether each frame image constituting the moving image is not suitable for the capture frame by the same determination method as the image determination unit 403 based on the shooting condition information, and automatically selects it. In addition, for example, when the user wants to reduce about 30% of the frame images constituting the moving image from the extraction target, the image automatic selection unit 404 dynamically changes the threshold value of each shooting condition value, and captures it. It is possible to set so that the number of frame images that are determined to be unsuitable for 30% of all frame images. Accordingly, the user can automatically select a desired number of frame images with high quality under the shooting conditions.

  In the present embodiment, the image determination unit 403 is described as generating the first determination information. However, as described above, the selection information generation unit 402 has a determination function, and the image determination unit 403 may be omitted.

<Video data generation method with image selection information>
Next, a method for generating moving image data with image selection information (first determination information) in the present embodiment will be described. FIG. 2 is a flowchart showing a method for generating moving image data with image selection information according to the present embodiment. As shown in FIG. 2, when the user presses a recording button (not shown) of the operation unit 306, moving image capturing is started, and this processing is executed according to a program by the central control of the central control unit 400.

  As described above, the moving image data is temporarily stored in the main memory 204 via the imaging unit 100 and the analog image signal processing unit 105 (step S101).

  During this period, that is, from the start of shooting to the end of shooting, shooting condition information is acquired by the imaging unit 100, and shooting condition information is sent from the imaging unit 100 to the selection information acquisition unit 401 (step S102). In this exemplary embodiment, the selection information generation unit 402 receives the shooting condition information from the selection information acquisition unit 401, uses this as image selection information, and sends it to the image determination unit 403. The image determination unit 403 generates first determination information indicating whether or not each frame image is unsuitable for a capture frame from the value of the shooting condition information or the amount of change in the shooting condition value between the frame images (step S1). S103). The image determination unit 403 sends the image selection information including the first determination information to the multiplexing unit 109, and the multiplexing unit 109 multiplexes the image selection information together with the compressed moving image data (step S104). The moving image data with image selection information is recorded on the recording medium 302 (step S105).

<Effect of Embodiment 1>
As described above, in the first embodiment of the present invention, the shooting condition information and the amount of change between the frame images are obtained as image selection information (tag) by acquiring the shooting condition information for each frame image at the time of shooting. It can be generated and recorded with moving image data. Therefore, when extracting still images from the moving image data, it is possible to automatically select whether each frame image does not have enough quality to be extracted as a still image, that is, whether it is inappropriate as a capture frame, The time required for the user to extract a desired still image can be shortened.

  In particular, in the conventional technique, for a still image acquired at the time of shooting, focus, aperture, exposure, and shutter speed (= sensor exposure period) when the shutter is pressed or when the shutter is pressed halfway. However, it is not possible to make such adjustments for captured images during playback. Therefore, in order to specify a better scene in the capture at the time of reproduction, it is an essential condition that the shooting conditions at the time of shooting are good. However, accurate shooting condition information cannot be directly known from already recorded moving image data.

  For example, the focus can be determined by detecting the edge level of a specific area. The subject can also be detected by detecting again like the focus. However, it is difficult to individually extract photographing information that is mutually related such as aperture, exposure, shutter speed, gain, and the like, and the accuracy is also low. That is, although it is not impossible to detect shooting information from a reproduced moving image, detection accuracy cannot be obtained, and processing load (time and power) increases. Therefore, conventionally, when a still image is captured from moving image data, a method in which a user designates a capture scene is a general method.

  On the other hand, in this embodiment, shooting information is acquired at the time of shooting, and is recorded together with moving image data as image selection information. Therefore, if this image selection information is used when automatically selecting frame images or extracting still images, a higher quality frame image can be extracted from the recorded moving image data based on the shooting condition information. In other words, it is possible to select a frame image with an optimal shooting condition such as capture at the time of shooting.

  Further, since the image selection information can be acquired before the compression processing of the moving image data, the convenience of the compressed moving image is improved by performing the compression processing together with the moving image data.

  Furthermore, the image selection information is not compressed with the moving image data, or the compressed moving image data is stored in a manner that can be referred to without being expanded, so that the moving image data is expanded to refer to the image selection information. This eliminates the need for processing, so that the user can execute the automatic sorting process at a very high speed without reproducing the moving image data.

  Furthermore, since frame images that do not have enough quality to be extracted as still images can be excluded from the extraction target, face detection and face authentication of the subject included in each frame image of moving image data, which will be described below, are performed. In the identification process, if such a frame image is excluded in advance using the image selection information in the present embodiment, a frame image that is inferior in quality for processing the same scene or extracting a still image. The processing can be excluded from the processing target, and the processing speed can be improved.

<Modification of Embodiment 1>
By the way, in this embodiment, shooting condition information is acquired for each frame image constituting a moving image. Here, the addition of Exif (Exchangeable image file format) information to captured still image data using the stored information will be described. Normally, Exif information is added to a still image. In the Exif information, the camera model and shooting condition information are embedded in the image, and can be applied by a viewer, photo retouching software, or the like. For example, in Exif 2.2, a standard called Exif Print is incorporated, and optimization is automatically performed based on condition information at the time of shooting, thereby enabling print output in an accurate state. Depending on the printer, it is possible to perform contrast correction, dark / bright correction, white balance correction, and the like according to the shooting conditions.

  However, Exif information cannot be added to still images captured from recorded moving image data because there is no camera information in the moving image data. In contrast, in the present embodiment, shooting condition information is acquired when creating image selection information. Therefore, even if the shooting condition information is processed to generate image selection information, if the shooting condition information is also recorded, these are information prepared for each frame of the moving image data, and thus captured. A frame image (still image) also has imaging condition information. Therefore, using this makes it possible to add Exif information to a capture frame.

<Embodiment 2 of the present invention>>
Next, a second embodiment of the present invention will be described. In the first embodiment described above, the shooting condition information is used as the image selection information. However, in the present embodiment, a case where information on a subject included in the image is used as the image selection information will be described. To do.

  The configuration of the imaging apparatus according to the present embodiment is the same as that of the first embodiment shown in FIG. 1, but the selection information acquisition unit 411, the selection information generation unit 412, the image determination unit 413 in the central control unit 410, The automatic image selection unit 414 is different from the first embodiment in that each image automatic selection unit 414 functions based on subject information.

  In the embodiment, an imaging apparatus that generates image selection information for enabling automatic selection when extracting a still image from moving image data and records it together with the captured image will be described. That is, the case where the imaging apparatus according to the present embodiment can automatically select, as a still image extraction candidate, a frame image that satisfies a certain criterion from one moving image file will be described. Therefore, it can also be applied to the case of automatically selecting a moving image file that includes a certain number of frame images that satisfy a certain standard, or the case of automatically selecting a still image that satisfies a certain standard from a plurality of still images. Needless to say.

  FIG. 3 is a block diagram illustrating an imaging apparatus according to the second embodiment of the present invention. As illustrated in FIG. 3, the central control unit 410 includes a selection information generation unit 412, an image determination unit 413, an automatic image selection unit 414, and a registered image DB 415.

  The selection information acquisition unit 411 according to the present embodiment detects a subject included in the moving image data as information on the quality of the frame image constituting the moving image, and detects which of the registered images in the registered image DB 415 matches. To do. If there is a matching subject, one or more of the number, size, angle or direction, position on the screen, and smile level of the subject that matches the registered image is detected. This subject information is acquired while shooting a moving image.

  Here, “while shooting a moving image” in this specification means that moving image data (raw data) is written into the main memory 204 from the start of shooting, and is read and compressed by the compression / decompression processing unit 201. Means until before. When video data is compressed, information about the subject cannot be obtained unless decompression is performed. However, in the present embodiment, information about the subject that can be used to determine the quality of the frame image is compressed. Get it before As a result, even if the moving image is compressed, the information about the subject can be evaluated and the quality of the frame image can be judged, that is, the frame image can be automatically selected without decompressing the moving image data. Become.

  The selection information generation unit 412 generates image selection information for each frame image constituting the moving image based on the information about the subject detected by the selection information acquisition unit 411. Specifically, the selection information generation unit 412 detects at least one of the number, size, angle or direction of the subject that matches the registered image, the position on the screen, and the smile level, and ranks this detection result. Any one or more of the assigned priority and the priority set for the registered image matching the subject is acquired for each frame image or at regular intervals, and image selection information is generated based on the acquisition result. If this image selection information is also a separate file from the moving image data, even after the moving image data is compressed, only this image selection information can be read out and the image passover process can be performed.

  And in this Embodiment, it has registration image DB415. The registered image DB 415 is an image database in which registered images are registered. For example, images of the person, family, friends, and the like can be registered as registered images by ID (identification number).

  Then, the selection information acquisition unit 411 detects which of the registered images the subject included in the frame image matches. When the subject included in the frame image matches the registered image, one or more of the number, size, angle or direction of the subject matching the registered image, the position on the screen, and the smile level are detected. The selection information generation unit 412 acquires any one or more of the detection result, the priority ranking the detection result, and the priority set for the registered image with the matching subject for each frame image or at regular intervals. Then, image selection information is generated based on the acquisition result.

  Similar to the image determination unit 403, the image determination unit 413 determines whether each frame image constituting the moving image has a quality as a capture frame based on the image selection information generated by the selection information generation unit 412. . Here, the image determination unit 413 determines whether each frame image is suitable as a capture frame by using image selection information generated based on information on the subject. In the first embodiment, it is determined whether or not it is unsuitable as a capture frame. In the present embodiment, it is determined whether or not it is suitable.

  That is, in the present embodiment, information about the subject is used as the image selection information. Therefore, when it can be determined that a desired subject that the user wants to take is captured at a desired size and position, the frame image can be determined to be suitable as a capture frame. Therefore, the image determination unit 413 generates, for each frame image, second determination information (such as a flag) indicating whether or not the frame image is suitable for a capture frame based on such image selection information.

  The image selection information generated by the selection information generation unit 412 and the second determination information generated by the image determination unit 413 are sent to the multiplexing unit 109 and multiplexed together with the moving image data compressed by the compression / decompression processing unit 201. And recorded on the card-type recording medium 302 or the like.

  The automatic image selection unit 414 automatically selects only frame images that are preferable for still image extraction from the recorded moving image data. In the present embodiment, since the moving image data has image selection information including the second determination information, automatic selection is possible based on the image selection information. For example, when a user wants to perform still image extraction for a certain moving image data, prior to the still image extraction, the image automatic selection unit 414 reads image selection information from the moving image data, and for example, the second frame image includes, for example, a second image. Only the frame image determined to be suitable as the capture frame is automatically selected. Alternatively, when the user wants to automatically select 100 frames from, for example, 10 minutes of moving image data, 100 frame images determined to be optimal can be automatically selected with reference to the image selection information. For example, the user can further extract desired 10 frame images from the selected 100 frame images.

<Information about the subject>
Next, information regarding the subject will be described in detail.
Registered image:
The registered image registered in advance has an ID for each registered image. A priority may be set for this ID (identification information). When a registered image is detected by authentication processing such as face authentication at the time of shooting, the registered ID itself can be used as image selection information. In the present embodiment, it is assumed that the subject is detected for each frame image, but may be performed at predetermined intervals. In the present embodiment, the registration image DB 415 is described in advance. However, for example, an ID is automatically assigned to each detected subject in order, and the registration image DB is automatically generated. Also good. In this case, the priority may be selected in the order of the highest detection rate, or when a certain number of registered image DBs are generated, the user may set the priority for each registered image.

Number of detected objects:
The number of subjects detected in each frame image matches the registered image is recorded. In the present embodiment, it is assumed that only the number of subjects that match the registered image is counted. However, this may also be counted when an unregistered subject is detected. In this case, a registered subject may be distinguished from an unregistered subject.

Subject size:
When the subject detected in each frame image matches the registered image, the size of the screen, for example, the number of pixels is recorded.

Subject angle and direction:
If the subject detected in each frame image matches the registered image, the angle and direction are recorded. Note that, in general face detection processing, techniques for matching registered face information and objects included in moving image data and detecting the face direction are well known, and details thereof will not be described here. Specifically, features are extracted from the position and shape of organ information such as face outline, eyes, nose, mouth, and the like, and a matching image is searched. Then, if there is a certain level of coincidence, it is determined as detection. Regarding the orientation of the face of the subject, the direction is detected from the position and shape of organ information such as the outline of the face, eyes, nose and mouth by the same method.

Subject position:
When the subject detected in each frame image matches the registered image, the position in the screen is recorded.

Subject smile level:
When the subject detected in each frame image matches the registered image, the smile level is recorded.

Subject focus:
When the subject detected in each frame image matches the registered image, the edge level around the subject can be detected to determine whether the subject is in focus.

Subject motion vector:
When the subject detected in each frame image matches the registered image, the motion vector of the subject is detected, and each frame image is evaluated according to the magnitude of the motion.

Detection priority:
The priority of each detection result is recorded for each frame image. As the priority, a ranking of the above detection results can be used. That is, for example, the priority order is set in this order for the number of detected objects, the size, the angle / direction, the position, the smile level, the focus, and the motion vector.

  Each subject information is extracted for each frame image, and information such as size, position, priority, and motion vector is obtained for each subject. In this case, only necessary information is obtained according to the priority. Also good. For example, even when 100 subjects are included in the screen, it is not necessary to perform authentication processing such as a face frame, name, and smile level on all. For this reason, a certain number of processes are executed from the top using the priority. Information stored as image selection information can also be a certain number from the top.

<Subject information in video data>
Next, a method for tracking a subject between frame images will be described. 4 and 5 are diagrams showing Tables 1 and 2, respectively. A case will be described in which a detection result as shown in Table 1 shown in FIG. The term “subject” as used herein refers to a subject in which a match with a registered image registered in the image determination unit 413 has been detected. Numbers No. 1, No. 2, No. 3,..., No. 10 are assigned in order of the detection result of the subject. The registered image ID, size, angle / direction, position, smile level, and priority of the subject are linked to this number. Then, it is assumed that the detection result of the next frame image 2 is as shown in Table 2 shown in FIG.

  As described above, in Table 2, No. 4, No. 6, and No. 7 disappear in the frame image 2, and No. 8 and No. 9 are added. This indicates that the subjects No. 4, No. 6, and No. 7 are no longer on the screen, and the subjects No. 8 and No. 9 are newly added to the screen. Therefore, since No. 1, No. 2, No. 3, and No. 5 exist in both frame images, the subject is tracked. By always performing such tracking and information organization, it is possible to display a face detection frame, focus on the face, and determine the subject to be tracked. Note that such a follow-up operation is normally performed at the time of shooting, and thus such follow-up recording can be acquired as image selection information from the time of shooting to the time of recording of a moving image.

<Evaluation method of image selection information>
Next, a method of using or evaluating the subject information (tag) detected as described above will be described.

Matching with registered image The subject has been detected to match the registered image contained in the registered image DB 415. Each registered image has an ID. In this case, for example, a frame image having an ID designated by the user can be extracted as a capture frame. Here, the registered image is not limited to the face image of the user or the like, and an animal such as a pet can be used as the registered image. Moreover, it is also possible to set an object such as a user's own car as a registered image as necessary.

Evaluation of subject size When the size of the subject that matches the registered image ID specified by the user is greater than or equal to a certain value, the frame image is determined to be a frame image suitable as a capture frame.

Evaluation of subject angle and direction If the angle and direction of the subject that matches the registered image ID specified by the user are within a certain range, the frame image is determined to be a frame image suitable as a capture frame.

Smile Evaluation If the smile level of the subject that matches the registered image ID specified by the user is above a certain level, it is determined that the frame image is suitable as a capture frame. Further, for example, when there are a plurality of subjects that match the registered image ID designated by the user, such as a group photo, if the average smile level of each subject is equal to or higher than a certain level, the frame image Is determined as a frame image suitable as a capture frame. Note that even a group photo may be focused only on the smile level of a specific subject.

Evaluation of subject movement In continuous frame images, as shown in Tables 1 and 2 above, in the case of a subject that can be followed, various types of information can also be acquired continuously. Therefore, for example, attention is paid to position information, the movement of the subject is grasped, and only when the amount of movement is within a certain range, the frame image is determined as a frame image suitable as a capture frame.

  For example, when it is determined that at least one of these is a frame image suitable as a capture frame, the image determination unit 413 sets a value (for example, 1) indicating that it is suitable as a capture frame to the second determination information. Can be set to In this way, by adding the second determination information, the image automatic selection unit 414 can automatically determine by determining whether or not to immediately extract as a capture frame.

  Further, when the image determination unit 413 is not provided, the selection information generation unit 412 uses values such as the number of detected subjects, the size, the angle / direction, the position, the smile level, the focus, and the motion vector as the image selection information. Each frame image has such information as image selection information. In this case, the image automatic selection unit 414 appropriately evaluates each value included in the image selection information, and if any one of the frame images passes as a capture frame, the frame image is extracted as an extraction candidate. Alternatively, the evaluation result may be weighted according to the priority, etc., and comprehensively taken into consideration to evaluate whether it is suitable as a capture frame.

  By making a determination in advance by the image determination unit 413, automatic selection by the image automatic selection unit 414 can be executed at high speed. On the other hand, when the automatic image selection unit 414 performs the selection process based on the image selection information, for example, the user can change the priority order, and the number of frame images to be automatically selected is also the evaluation result. It can be changed dynamically according to

  An example of the automatic sorting method of the image automatic sorting unit 414 will be described. The image automatic selection unit 414 sets a score according to the number of detected subjects included in the image selection information. If the number of detected objects is 1, it is 10 points, and if it is 2, it is 30 points. The same applies to the size of the subject, 10 points if it is less than 5% of the screen, 20 points if it is 5 to 10% of the screen. The angle / direction is also set to 100 points if it can be determined that it is facing the front, 30 points if it is horizontal. If the smile level is calculated from 0 to 100%, the value is used as a score.

  Next, these scores are weighted according to the priority set for each detection result, and a total score is calculated for each frame image. Then, each total score is converted back to a full score of 100, and this is used as the evaluation standard. The result calculated so far may be used as the image selection information. The automatic image selection unit 414 uses the 100-point evaluation criteria, and determines that, for example, a frame image of 70 points or more is suitable as a capture frame. Alternatively, when the user instructs to extract 100 optimal frame images, 100 frame images can be automatically selected in descending order of the evaluation value.

  For example, when the size of the subject is below a certain level, when the motion vector of the subject is larger than a certain level, or when the edge level of the subject is smaller than a certain level, it is not suitable as a frame image to be extracted as a still image. For those to be judged, by setting these evaluation values to -1000 points, etc., if the score of the frame image is the above case, the value is set so as not to exceed 70 points and selected as a capture frame. It is also possible not to be done.

<Video data generation method with image selection information>
Next, a method for generating moving image data with image selection information (second determination information) in the present embodiment will be described. FIG. 6 is a flowchart showing a method for generating moving image data with image selection information according to the present embodiment. As shown in FIG. 6, when the user presses a recording button (not shown) on the operation unit 306, moving image capturing is started, and this processing is executed according to a program by the central control of the central control unit 400.

  As described above, the moving image data is temporarily stored in the main memory 204 via the analog image signal processing unit 105 (step S201).

  The selection information acquisition unit 411 reads out and analyzes the moving image data stored in the main memory 204 to obtain subject information (step S202). For example, if the subject face detection process includes a face detection unit, the selection information generation unit 412 may receive the processing result.

  The selection information generation unit 412 generates image selection information from subject information. When the selection information generation unit 412 acquires information such as the number of detected subjects, the size, the angle / direction, the position, the smile level, the focus, the motion vector, etc., the selection information generation unit 412 obtains, for example, 100 points based on the above evaluation criteria It converts into the evaluation value of a perfect score, and makes this image selection information (step S203).

  The image determination unit 413 generates second determination information indicating whether each frame image is suitable for a capture frame based on the image selection information received from the selection information generation unit 412 (step S204). For example, when an evaluation value with a maximum score of 100 of each frame image is received as image selection information, for example, a frame image with an evaluation value of 70 points or more is set to 1, a frame image with an evaluation value of 69 points or less is set to 0 or the like. The determination information is generated.

  The image determination unit 413 sends the image selection information including the second determination information to the multiplexing unit 109 and multiplexes the compressed image data together with the compressed moving image data (step S205), for example, as the moving image data with image selection information on the card type recording medium 302. Recording is performed (step S206).

<Effect of Embodiment 2>
As described above, in Embodiment 2 of the present invention, information about a subject is acquired from each frame image constituting a moving image, and based on this, image selection information is generated and attached to moving image data. Thereby, for example, only the frame image suitable as the capture frame can be extracted by the image automatic selection unit 414.

  It is also possible to extract only the frame image where the subject of the registration ID designated by the user has moved. That is, automatic capture can be performed without the user instructing the capture timing from the moving image data.

  Furthermore, by setting a priority for each subject information to be evaluated such as the number of detected subjects, size, angle / direction, position, smile level, focus, motion vector, etc., at the time of automatic capture, You can specify the maximum number of captures from one video data. In other words, it is possible to extract from a higher-level frame image based on the priority of the extracted capture frame.

  Furthermore, when playing back a movie, information on the face, person, animal, or object is obtained from the image selection information, and only the frame image of the specified subject is displayed or paused at the specified frame image. The user can be prompted to capture a still image. In addition, by searching for a designated subject, a related scene can be automatically reproduced and captured.

  In addition to still image capture, each moving image data has image selection information for each frame image constituting the moving image data. With reference to this, the moving image data itself can be evaluated, for example, It is also possible to extract a moving image file including the largest number of frame images having a designated registered image subject from a plurality of moving image files.

  Furthermore, in the present embodiment, the description has been made on the assumption that the image selection information is generated from the subject information for the moving image file, but the same can be done for the still image file. That is, for still images, if the subject information of each still image is extracted and the image selection information is added and recorded, the user can extract a desired still image without reproducing all the still images. It becomes possible.

  Furthermore, the subject information is information that cannot be detected unless it is decompressed after being compressed and recorded, whether it is moving image data or still image data. Therefore, the image selection information is recorded in a place where it can be read out without decompression processing, so that desired still images can be automatically selected without decompression processing such as reproduction of moving image data or still image data. Therefore, the automatic sorting process can be speeded up.

<Modification of Embodiment 2>
Here, a modification of the present embodiment will be described. Conventionally, still image capture processing is performed at a fixed bit rate regardless of the bit rate of the moving image file to be reproduced. For this reason, the image quality of the captured still image depends on the bit rate of the moving image file to be reproduced.

  On the other hand, in this modification, information regarding the bit rate of a moving image file to be reproduced is acquired as image selection information, and an optimal capture process is performed according to this information. Therefore, in the present embodiment, the media control unit 207 acquires information about the bit rate information of the moving image data stored in the recording medium such as the card type recording medium 302, and the optimal quantization is performed at the time of capture. Set the parameters. That is, when capturing, information related to the bit rate is acquired from the recorded moving image file, and a more suitable still image is recorded by setting an optimal quantization parameter.

  Alternatively, when the compression / decompression processing unit 201 compresses the moving image data, information regarding the bit rate is sent to the selection information generation unit 402, and the selection information generation unit 402 includes this in the image selection information and includes the multiplexing unit 109. And may be multiplexed with the compressed moving image data. Further, as described above, the image selection information may be stored as a file different from the compressed moving image file.

  In this way, by including information related to the bit rate at the time of video data compression in the image selection information, if the bit rate at the time of video compression is high, the quantization parameter at the time of JPEG (Joint Photographic Experts Group) encoding (JPEG encoding accuracy) If the bit rate at the time of moving image compression is low, the quantization parameter at the time of JPEG encoding can be controlled to be low. As a result, the image quality of the recorded still image is optimized, and the data amount of the still image file can be suppressed optimally.

  For example, when capturing after performing image enlargement such as capturing 640 × 360 at 1920 × 1080, the actual resolution (spatial frequency) is not 920 × 1080 but 640 × 360 unless super-resolution processing or the like is performed. Will remain. For this reason, it is possible to reduce the quantization parameter at the time of JPEG encoding, and optimization can also be performed depending on the recording moving image size.

  That is, JPEG encoding can be performed with high accuracy, but it is usually difficult for the user to recognize the difference in image quality. Therefore, encoding all still images at a high bit rate leads to an increase in the data amount of the still image file and a decrease in the number of recorded images. Therefore, in this modification, by including information on the bit rate and recording moving image size at the time of moving image recording in the image selection information, it is possible to control the quantization parameter at the time of optimal JPEG encoding, and the image quality of the recorded still image Improvements can be made.

<Third embodiment of the present invention>
Next, a third embodiment of the present invention will be described. In the above-described second embodiment, if the subject is captured at the user's desired position, size, etc., it is determined that the quality is good, and this is extracted. By the way, among the images that can be determined to be good in terms of subject as described above, some of the images that are not good in terms of shooting conditions as described in Embodiment 1 above. Can exist. In this case, such a frame image should be determined to be inappropriate as a capture frame. Therefore, in the third embodiment of the present invention, the first determination information based on the shooting condition information used in the first embodiment and the second determination information based on the information about the subject are used as a capture frame. Automatic selection of more reliable frame images.

  FIG. 7 is a block diagram illustrating the imaging apparatus according to the present embodiment. The central control unit 420 includes a selection information generation unit 421, an image determination unit 422, an image automatic selection unit 423, and a registered image DB 424. The selection information generation unit 421 includes a shooting condition information acquisition unit 431 and a subject information acquisition unit 432, and the image determination unit 422 includes a shooting condition information determination unit 441 and a subject information determination unit 442.

  The imaging condition information acquisition unit 431 is formed in the selection information generation unit 421, and functions in the same manner as the selection information generation unit 402 in the first embodiment. That is, the imaging condition information acquisition unit 431 acquires imaging condition information while the imaging unit 100 acquires a moving image, and the selection information generation unit 421 configures each moving image based on the acquired imaging condition information. For each frame image, first image selection information based on the shooting condition information is generated.

  In addition, the subject information acquisition unit 432 is formed in the selection information generation unit 421, but functions in the same manner as the selection information generation unit 412 in the second embodiment. That is, the subject information acquisition unit 432 acquires subject information included in each frame image constituting the photographed moving image, and the selection information generation unit 421 stores the acquired subject registered in the registered image DB 424. It is detected whether any of these matches. When the subject included in the frame image matches the registered image, the detection result of detecting any one or more of the number, size, angle or direction, position on the screen, and smile level of the subject matching the registered image Any one or more of the priority ranking of the detection result and the priority set for the registered image that matches the subject is acquired for each frame image or at a fixed interval, and subject information is obtained based on the acquisition result. 2nd image selection information based on is generated.

  The shooting condition information determination unit 441 of the image determination unit 422 determines the suitability of each frame image constituting the moving image as the extracted still image to be extracted based on the first image selection information generated by the selection information generation unit 421. . Then, the shooting condition information determination unit 441 generates first determination information indicating whether or not the frame image is inappropriate as the extracted still image for each frame image constituting the moving image based on the imaging condition. Since the selection information generation unit 421 generates the first image selection information for each frame image as described above, the imaging condition information determination unit 441 extracts the frame image as a still image for each frame image. It is possible to determine whether or not the quality can endure, and as a result, first determination information is generated.

  Similar to the image determination unit 413, the subject information determination unit 442 of the image determination unit 422 uses the second image selection information generated by the selection information generation unit 421 as the capture frame. It is determined whether or not it has quality. Here, the subject information determination unit 442 uses the second image selection information generated based on the information related to the subject, and determines whether each frame image is suitable as a capture frame. The subject information determination unit 442 generates, for each frame image, second determination information indicating whether or not the frame image is suitable for the capture frame based on the second image selection information.

  Here, the image determination unit 422 in the present embodiment excludes the frame image that is not suitable for capture by the first determination information generated by the subject information determination unit 442, and uses the subject information in the remaining frame images. Third determination information for determining that it is most preferable to capture a frame image determined to be suitable for capture based on the second determination information generated by the determination unit 442 is generated. The image determination unit 422 sends the first to third determination information to the multiplexing unit 109 together with the first and second image identification information.

  The multiplexing unit 109 multiplexes the first and second image identification information including the first to third determination information into the moving image data compressed by the compression / decompression processing unit 201, and the card type recording medium 302. Record in etc.

  The image automatic selection unit 423 automatically selects only frame images preferable for still image extraction in response to a user request, such as when a user extracts a still image from moving image data. In the present embodiment, the moving image data has image selection information including the first to third determination information, and therefore can be automatically selected based on the image selection information. For example, when the user wants to perform still image extraction for a certain moving image data, prior to the still image extraction, the image automatic selection unit 414 reads out image selection information from the moving image data, and among the all frame images, for example, With reference to the determination information of No. 3, only frame images determined to be optimal as capture frames are automatically selected.

  Here, for example, when the image determination unit 422 is not provided, the first image selection information generated based on the shooting condition information and the second image selection information generated based on the subject information are displayed together with the moving image data. Will be recorded. As described above, the format in which the image selection information is recorded, that is, whether the file is the same file as the moving image data or a separate file is not limited. However, if the first and second image selection information is saved from the compressed moving image data at a location where the moving image data can be read without being expanded, the image automatic selection unit 423 causes each frame of the moving image data to be read. The frame image can be automatically selected with reference to the image selection information corresponding to the image. In this case, since it is not necessary to perform decompression processing such as reproduction of moving image data, it is possible to perform frame image selection at an extremely high speed.

  The selection in the automatic image selection unit 423 is the same processing as the imaging condition information determination unit 441 and the subject information determination unit 442 of the image determination unit 422. In accordance with a predetermined reference value, based on the first image selection information, an unsuitable frame image is excluded as a capture frame. Further, based on the second image selection information, further capture frames are selected from the selected frame images. It is possible to select only frame images suitable for At this time, as described above, it is also possible to dynamically change the threshold value for each piece of evaluation data and to execute selection according to the user's request. It goes without saying that only the first image selection information or only the second image selection information may be used for image selection according to the user's request.

<Effect of Embodiment 3>
In the present embodiment, since it is possible to select a frame image suitable for capture from information regarding the shooting conditions and the subject, it is possible to perform a finer selection than in the first and second embodiments described above. Therefore, it is possible to extract only frame images in which the subject specified by the user is captured and which have good shooting conditions.

  In addition, since the first image selection information originally includes lens information and image processing information at the time of shooting that cannot be restored at the time of reproduction, only frame images with good shooting conditions can be extracted. By specifying the capture scene using the generated second image selection information, it is possible to significantly reduce the processing time in the reproduction process or the capture scene specification.

  In addition, by generating image selection information for moving image data, it is possible to add a tag called image selection information having temporal continuity in order to acquire shooting conditions and subject information for each frame or at regular intervals. As a result, it is possible to extract a frame image at the shooting condition indicated for capture, and to select a frame image optimal for capture while grasping the change and movement of the subject.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  For example, in the above-described embodiment, the hardware configuration has been described. However, the present invention is not limited to this, and arbitrary processing may be realized by causing a CPU (Central Processing Unit) to execute a computer program. Is possible. In this case, the computer program can be stored using various types of non-transitory computer readable media and supplied to the computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W and semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)) are included. The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

DESCRIPTION OF SYMBOLS 1 Image pick-up device 100 Image pick-up part 101 Zoom lens 102 Focus lens 103 Diaphragm 104 Image pick-up element 105 Analog image signal processing part 106 A / D conversion part 107 Image input controller 106 A / D conversion part 109 Multiplexing part 200 Bus 201 Compression / decompression process Section 203 Line memory 204 Main memory 206 SDRAM
301 Card I / F
302 Card Type Recording Medium 400 Central Control Unit 401 Selection Information Acquisition Unit 402 Selection Information Generation Unit 403 Image Determination Unit 404 Image Automatic Selection Unit 410 Central Control Unit 411 Selection Information Acquisition Unit 412 Selection Information Generation Unit 413 Image Determination Unit 414 Image Automatic Selection Part 415 Registration Image DB
420 Central Control Unit 421 Selection Information Generation Unit 422 Image Determination Unit 423 Image Automatic Selection Unit 424 Registered Image DB
431 Imaging condition information acquisition unit 432 Subject information acquisition unit 441 Imaging condition information determination unit 442 Subject information determination unit

Claims (3)

An imaging unit for acquiring a video ;
While photographing the moving, and sorting information acquisition unit that acquires information about the quality of a frame picture image constituting the front kidou image,
Based on the information acquired by the selection information acquisition unit, a selection information generation unit that generates image selection information that is information for selecting the moving image according to its quality;
Based on the image selection information generated by the selection information generation unit, an image determination unit that determines whether or not each frame image constituting the moving image is appropriate as a target still image to be extracted;
A recording unit for recording the image selection information together with the moving image,
The image determination unit indicates whether or not each frame image constituting the moving image is inappropriate as the target still image based on an imaging condition while the imaging unit acquires the moving image. Generating first determination information, generating second determination information indicating whether or not each frame image constituting the moving image is suitable as the target still image;
The recording unit records the first and second determination information or the comprehensive determination information obtained from the first and second determination information as a part or all of the image selection information together with the moving image. An imaging device. During shooting movies, and sorting information acquisition step of acquiring information about the quality of the frame picture images constituting the moving,
Based on the information acquired in the selection information acquisition step, a selection information generation step of generating image selection information that is information for selecting the moving image according to its quality;
Based on the image selection information generated in the selection information generation step, an image determination step for determining whether or not each frame image constituting the moving image is appropriate as a target still image to be extracted;
A recording step of recording the image selection information together with the moving image,
The image determination step includes
Generating first determination information indicating whether or not each frame image constituting the moving image is inappropriate as the target still image based on an imaging condition while acquiring the moving image; ,
Generating, for each frame image constituting the moving image, second determination information indicating whether or not those frame images are suitable as the target still image,
The recording step includes
Recording the first and second determination information or the comprehensive determination information obtained from the first and second determination information together with the moving image as part or all of the image selection information. Image processing method. During shooting movies, and sorting information acquisition processing for acquiring information on the quality of the frame picture images constituting the moving,
Based on the information acquired in the selection information acquisition process, a selection information generation process for generating image selection information that is information for selecting the moving image according to its quality;
Based on the image selection information generated in the selection information generation process, an image determination process for determining whether or not each frame image constituting the moving image is appropriate as a target still image to be extracted;
A program for causing a computer to execute a recording process for recording the image selection information together with the moving image,
The image determination process includes
Processing for generating first determination information indicating whether or not each frame image constituting the moving image is inappropriate as the target still image based on an imaging condition while acquiring the moving image; ,
Processing for generating second determination information indicating whether or not each frame image constituting the moving image is suitable as the target still image,
The recording process includes
Including a process of recording the first and second determination information or the comprehensive determination information obtained from the first and second determination information together with the moving image as part or all of the image selection information. program.

Images