JP4595832B2 - Imaging apparatus, program, and storage medium - Google Patents

Description

  The present invention relates to a still image generation technique for outputting, as a still image, a scene in which many people's faces are captured from continuously shot images such as moving images.

  As a technique for quickly detecting a person in a video from a search target image, an image showing the person to be detected is designated as a key image, and a hue histogram of the face area and clothing area of the key image is secured as search data and read. The search target image is subjected to region segmentation processing to ensure a hue histogram for each region, and a frame in which the hue historygram of the entire screen changes abruptly is scanned from two directions, similar to the hue histogram of the face region The first region having the hue histogram and the second region having a hue histogram similar to the hue histogram of the clothing region are obtained, and the first region and the second region are in a vertical positional relationship close to each other. Sometimes, there is a technique that uses this as a person to be detected (see, for example, Patent Document 1).

  In addition, the face image of the person to be photographed is extracted for each frame image of the captured video so that the photographer can easily print and enjoy the photographed image with a good expression. Smile evaluation and neatness evaluation are performed, and for each frame image, a total smile evaluation value and a total neatness evaluation value of each subject in the image are calculated, one of the frame images is analyzed, and the captured image If the scene is judged and the shooting is determined to be a casual scene, the priority evaluation value is set as a smile evaluation value. If the shooting is determined to be a formal scene shooting, the priority evaluation value is set as a neat evaluation value. Then, the order is set for each frame image in descending order of priority evaluation value. There is a digital camera that displays each frame image based on the order (see, for example, Patent Document 2).

  In addition, as a technology that simplifies the number setting at the time of print order using the face extraction technology, the face image is extracted by analyzing the captured image stored in the recording medium, and the number of the extracted face regions is counted. There is a technique for generating the number information and holding it on a recording medium to reduce the complexity of specifying the number of sheets when ordering a print (see Patent Document 3).

JP 2000-30033 A JP 2002-204099 A JP 2004-145563 A

However, in the technique described in Patent Document 1, a designated person can be recognized from a moving image frame, but the number of persons can be detected by recognizing the face of an unspecified person facing the camera. Since it is not possible, a scene in which a specified number (a plurality) of people are captured from continuously shot images such as moving images cannot be output as a still image.
Further, in the technique described in Patent Document 2, the priority evaluation value is set as a neat evaluation value, and each frame image is displayed on the basis of the rank after setting the rank for each frame image in descending order of the priority evaluation value. Even if there are an unspecified number of non-photographers in the video, it is possible to detect and display and print the frame images of those subjects with good faces, but without setting a priority evaluation value. It is impossible to detect a frame image in which everyone is facing the camera from within a video.
The technique described in Patent Document 3 can detect the face area and generate the number of people information. However, a scene in which a specified number (a plurality) of people are captured from continuously shot images such as moving images is captured as a still image. Or a still image obtained by combining frame images before and after the central image cannot be output.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and an imaging device , a program, and a storage medium that simply output a scene in which many people's faces are captured from continuously shot images such as moving images as still images The purpose is to provide.

In order to solve the above problems, the invention described in claim 1, and the layout information storage means for storing a plurality of information of the layout of the still images generated, an image selection means for selecting the central image, the layout information storage unit Layout selecting means for selecting a layout to be used for generating a still image from a plurality of layouts stored in the image, and an image selecting means from among images whose number of persons acquired by the number information acquiring means is a predetermined number of frames. Still image generation means for generating image data of a still image in which a selected center image and a moving image captured before and after the center image are arranged based on a layout selected by the layout selection means An imaging device is provided.
Thereby, a frame in which a predetermined number of people are captured from a moving image can be acquired, and a still image having a predetermined layout centered on an image desired by the user can be generated.

According to a second aspect of the present invention, there is provided the imaging apparatus according to the first aspect, wherein the number of persons is the number of persons whose faces are directed to the imaging apparatus.
As a result, a frame in which a predetermined number of persons are facing the image pickup apparatus among the images in each frame of the moving image can be displayed as a moving image. In addition, since only a frame having a predetermined number of persons is displayed as a moving image, the time required for reproducing the moving image can be reduced as a whole. In addition, everyone can enjoy watching the images of everyone facing the imaging device in the video.

According to a third aspect of the present invention, there is provided the imaging apparatus according to the first or second aspect, further comprising an output unit that outputs the image data generated by the still image generating unit.
As a result, the user generates a still image having a predetermined layout centered on a desired image from moving images in which a desired number of persons are captured, and outputs (records or transmits to a printer or an external processing device) )it can.

Further, in the invention according to claim 4, comprising a number setting means for setting a predetermined number, any one of claims 1 to 3, wherein the number set by the number setting means is a plurality of 1 The imaging device according to the item is provided.
As a result, when the user sets a plurality of numbers, for example, 5 as a predetermined number, the frame in which five persons are shown in the moving image is detected, and the desired image is centered on the detected frame image. It is possible to generate a still image having a predetermined layout. It is also possible to acquire a frame showing five persons from a moving image and display the moving image, or output (record, print, or send to an external processing device). For example, when the user sets 3 as the predetermined number, a frame in which three or more persons are captured in the moving image is detected, and a predetermined layout centered on a desired image from the detected frame images. Still images can be generated. In addition, it is possible to acquire and display a moving image in which three or more people are captured from the moving image.

According to the fifth aspect of the present invention, a computer having an image recording memory stores a plurality of information on the number of persons for each frame of a moving image to be reproduced and layout information of a generated still image. A function for selecting a center image, a function for selecting a layout to be used for generating a still image from among the plurality of stored layouts, and an image for which the acquired number of persons is a predetermined number of frames. the central image and other moving image selected from among, provides a program for executing a function for generating image data of a still image arranged on the basis of the selected layout.
Thereby, a frame in which a predetermined number of people are captured from a moving image can be acquired, and a still image having a predetermined layout centered on an image desired by the user can be generated .

According to a sixth aspect of the present invention, there is provided a computer-readable storage medium characterized by storing the program according to the fifth aspect.
By executing a program stored in the storage medium, a frame showing a predetermined number of people is obtained from a moving image, and a still image having a predetermined layout centered on an image desired by the user is generated from the frame. Can

  According to the present invention, it is possible to obtain the number of persons facing the image capturing apparatus among the images captured in each frame of the moving image, so that the designated number of persons are facing the image capturing apparatus. Can be displayed and output (recorded, printed, or sent to an external processing device).

  FIG. 1 is a diagram showing an external appearance of a digital camera as an example of an image pickup apparatus and a playback apparatus common to the embodiments of the present invention. Here, the front (FIG. 1A) and the back (FIG. 1) are mainly shown. The external appearance of (b)) is shown. The digital camera 100 has an imaging lens (lens group) 2 on the front side as shown in FIG. Further, as shown in FIG. 1B, the back of the digital camera 100 has a mode dial 3, a liquid crystal monitor screen 4, a cursor key 5, a SET key 6, a zoom button 7 (W button 7-1, T button 7- 2) and a menu key 10 or the like. A shutter key 8 and a power button 9 are provided on the upper surface. Although not shown in the figure, a USB terminal connection portion used when connecting to an external device such as a personal computer (hereinafter referred to as a personal computer) or a modem and a USB cable is provided on the side portion.

  FIG. 2 is a block diagram of the digital camera 100 shown in FIG. 1. The digital camera 100 includes a (still image) normal shooting mode, a macro shooting mode, a continuous shooting mode, a rapid shooting mode,... The camera has a shooting mode such as a shooting mode, a still image shooting mode, a moving image shooting mode, a continuous shooting image playback mode, and the like. Further, as a lower mode of the moving image playback mode, there is a still image output mode for generating a still image from a moving image (making it a still image) and outputting it. Note that the high-speed continuous shooting mode is a shooting mode in which, for example, 50 or more images are continuously shot per second, and the present invention is not only for generating still images from moving images, but also for images shot in the high-speed continuous shooting mode. The present invention can also be applied to a case where a still image is generated from an image selected from the above and images taken before and after that image.

  That is, the digital camera 100 includes a camera body 1 and a recording medium 30 that can be attached to and detached from the camera body 1. The camera body 1 is provided with an optical system 12 including a zoom lens and a focus lens and a CCD 13. .

  The CCD 13 is an image pickup means of the present invention, and is driven by a drive signal sent from the timing generator 17 to photoelectrically convert a subject light image and output it as an image pickup signal. The output signal of the CCD 13 is subjected to correlated double sampling and gain adjustment by the CDS circuit 14, and is converted to a digital signal by the A / D conversion circuit 15. The A / D converted image signal is input to the DSP block 16 and subjected to processing such as pedestal clamping, and then the luminance (Y) signal and the color difference (UV) signal are processed by the luminance / color difference matrix circuit in the DSP block 16. Is converted to The DSP block 16 also performs processing for improving image quality such as auto iris (AE), white balance (WB), and edge enhancement.

  The YUV data converted by the DSP block 16 is converted to a preset image size by the resolution conversion block 18, and then data for one frame is sequentially stored in the internal memory 24 (for example, the SD internal memory 24). The YUV data for one frame stored in the built-in memory 24 is sent to the display controller 21, where it is converted into a video signal, and then displayed on the LCD (liquid crystal display unit) 12 as a through image, that is, a through moving image. .

  One frame of YUV data stored in the built-in memory 24 at the time of shooting in the still image mode is encoded by the data compression / decompression block 19 after being compressed by the JPEG method, etc. It is recorded as still image data (still image file) on the recording medium 30 via the controller 20.

  The recording medium 30 is a moving image recording means of the present invention, and YUV data stored in the built-in memory 24 at the time of moving image shooting is sequentially sent to the data compression / decompression block 19, and a predetermined moving image recording method (for example, Motion-JPEG or MPEG). ), The data is compressed after being compressed by the codec, filed in the built-in memory 24, and then recorded on the recording medium 30 as a moving image file via the media controller 20. The frame rate for moving image shooting is determined by the timing signal generated by the timing generator 17, and the compression rate of the encoded data is determined by the digitization table value in the data compression / decompression block 19.

  The data compression / decompression block 19 decompresses the still image or moving image data read from the recording medium 30 during reproduction of the still image or moving image, and expands the still image data or moving image frame data in the built-in memory 24. The developed image is sent to the display controller 21 where it is converted into a video signal and then displayed on the LCD 22 as a reproduced image. The LCD 22 displays not only a through image and a reproduced image, but also a menu screen and a setting screen for selecting and setting various functions in the digital camera 100 as necessary.

  The sound processing block 25 converts sound input to the microphone 26 built in the camera body 1 into a digital signal at the time of moving image shooting, and sends it to the built-in memory 24 as audio data after data compression. The audio data sent to the built-in memory 24 is sequentially written to the recording medium 30 as stream data together with the frame data. The audio processing block 25 decodes the audio data sent from the built-in memory 24 at the time of moving image shooting, converts it into an analog audio signal, and then outputs the audio from the built-in speaker 27 built in the camera body 1.

  The key input block 28 includes the mode dial 3, the cursor key 5, the SET key 6, the zoom button 7 (W button 7-1, T button 7-2), the shutter key 8, the power button 9, and the menu key 10 described above. , A key processing unit (not shown) that generates an operation signal of the operated key and sends it to the CPU 23 when the key is operated.

  The mode dial 3 is used to select a shooting mode and a playback mode. The user operates the mode dial 3 to select (still image) normal shooting mode, macro shooting mode, continuous shooting mode, rapid shooting mode,..., Movie shooting mode,. A playback mode such as a video playback mode can be selected.

  The cursor key 5 is a key operated when pointing (designating) a menu or icon displayed on the liquid crystal monitor screen 4 with the cursor when setting a mode or selecting a menu. Can be moved to. In addition, when the number of people in one frame is specified in the still image output mode, which is a lower mode of the video playback mode, the screen stopped by moving the cursor key 5 leftward (>) is set to 1 in the traveling direction. You can advance frame by frame. Further, by operating the cursor key 5 leftward (>), the screen can be advanced one frame at a time in the reverse direction.

  The SET key 6 is a key that is pressed when selecting, setting, or confirming the item or image where the cursor is positioned by the cursor key 5. When the multi-person image extraction mode is selected in the still image output mode which is a lower mode of the reproduction mode, when the SET key 6 is pressed, the frame displayed at that time becomes the center image of the still image to be generated.

  The zoom button 7 is used for zoom operation. In the case of optical zoom, the angle of view actually changes corresponding to the operation of the zoom button 7 (W button 7-1 or T button 7-2), and the liquid crystal monitor screen 4 Displays a wide (wide angle) image or a tele (telephoto) image. In the case of the electronic zoom, the image from the optical system 12 is displayed by enlarging / reducing the image by digital zoom processing in response to the operation of the zoom button 7.

  The shutter key 8 performs a release operation at the time of shooting and has a two-step stroke. In the still image shooting mode, auto-focus (AF) and automatic exposure (AE) are performed by the first-stage operation (half-pressed state). ) And a shooting instruction signal for performing shooting processing in the second stage operation (fully pressed state). In addition to being used for recording start / end instructions in the moving image shooting mode, in the still image output mode, when the shutter key 8 is pressed halfway, the displayed moving image is stopped and displayed still, and when the shutter key 8 is fully pressed. It can be used as a moving image display stop / stop release key for releasing the stop and continuing the moving image display. When the shutter key 8 is fully pressed, the displayed moving image is stopped and displayed as a still image, and when the shutter key 8 is pressed halfway, the stop is canceled and used as a video display stop / stop cancel key for continuing the video display. Also good.

  The menu key 10 is a key operated when displaying a selection menu or icon that can be processed at that time.

  When the number of people is specified in the still image output mode, which is a lower mode of the moving image playback mode, the number of people detection block 31 detects a frame in which a large number of people are copied from the frames in the moving image, and sends a detection signal to the CPU 23. . Here, the large number of people is the number of people specified by the user in the still image output mode, and in the embodiment, two or more people can be selected (example of the number of people selected; 2, 3, 4, 5,... 10 people, 2 or more, 3 or more, 4 or more, 5 or more, ... 10 or more). A known method can also be used as a method for detecting the number of persons. For example, a technique for recognizing a person like the technique described in Patent Document 1 (Japanese Patent Laid-Open No. 2000-30033) described above, or a method for acquiring the number of persons information described in Patent Document 3 (Japanese Patent Laid-Open No. 2004-145563) Can be adopted as appropriate. A dedicated circuit may be provided for the multi-person detection process, or the CPU 23 may execute a multi-person detection program.

  The contour extraction block 32 extracts the contour of a person from the subject of interest (a subject including a focus area in the embodiment). A well-known method (for example, the contour extracting method described in Japanese Patent Application Laid-Open No. 07-274067 and Japanese Patent Application Laid-Open No. 2004-297274) can be appropriately employed as the method for extracting a person's contour. The contour extraction block 32 is not essential.

  Each block described above is controlled by the CPU 23, and programs and data required for the CPU 23 to control each block are stored in a program memory 29 which is a rewritable nonvolatile memory such as an EEPROM or a flash memory. The CPU 23 operates based on the program and the key input signal from the key input block 28, thereby functioning as the selection means, the number information acquisition means, the still image generation means, and the control means of the present invention. The program memory 29 also stores setting data related to the system and each function set by the user.

  The program memory 29 is also a layout information storage means of the present invention. The program memory 29 stores layout data used in the above-described still image output mode operation in its area 29a. The layout data is data indicating the design of a still image generated when a moving image is converted into a still image, that is, the number, size, arrangement, and the like of picked-up images to be reduced around the central image (see FIG. 6).

  FIG. 3 is an explanatory diagram of a display method of a multi-person image detected from a moving image, and is a display example when an image in which three people are captured as a multi-person image is detected from the moving image. In the moving image to be reproduced, 2 people (frame number i (1-1 to 1-i), 0 person (frame number j (2-1 to 1-j)), 3 people (frame number k (3-1) ~ 3-k), 2 persons (frame number l (4-1 to 4-l), 1 person (frame number m (5-1 to 5-i)), 4 persons (frame number n (6-1 to 1) -N) 3 people (number of frames p (7-1 to 7-p)), 0 people (number of frames q (8-1 to 1-q)) (However, in the case of 0 people, no person is shown in the frame).

  When detecting an image with three people facing the camera as a multi-person image from a moving image, since only less than three persons are shown in frames 1-1 to 2-j), the image is skipped without being displayed. Is done. When a frame 3-1 in which three people are shown is detected, the frame stops at that position, and an image of the detected frame 3-1 is displayed. Next, when the shutter key 8 is fully pressed, an image of k-1 frames 3-2 to 3-3k is displayed as a moving image. If the shutter key 8 is pressed halfway during the moving image display, it stops at the frame that was displayed when it was pressed, and the image of that frame is displayed statically. When the shutter key 8 is fully pressed, the moving image display is started. When the display of k-1 moving images is completed, since only less than three persons are shown in the frames 4-1 to 6-n, the image display is skipped, and the frame 7-1 in which three people are shown is detected. Then, it stops at that position, and the image of frame 3-1 is displayed as a still image. Next, when the shutter key 8 is fully pressed, the images of the p-1 frames 7-2 to 7-p are displayed as moving images. If the shutter key 8 is pressed halfway during the moving image display, it stops at the frame that was displayed when it was pressed, and the image of that frame is displayed statically. When the shutter key 8 is fully pressed, the moving image display is started. Note that when the shutter key 8 is fully pressed, the displayed moving image is stopped and displayed statically, and when the shutter key 8 is pressed halfway, the moving image display may be started.

  As described above, it is possible to save time for outputting a still image by skipping the display of images other than the designated number of people. In addition, when displaying a continuous frame in which the designated number of people is shown as a moving image, the frame may be moved at a predetermined time interval and displayed slowly. In this way, it is easy to select a desired image from consecutive frames.

  In addition, when “3 or more” is designated as the number of persons facing the camera in the example of FIG. 3, the moving image stops at the frames 3-1 and 6-1. In addition, an image of k + 1−1 frames 3-2 to 4-l following the frame 3-1 displayed by the cursor operation is displayed as a moving image. Similarly, the images of n + p−1 frames 6-2 to 7-p following the image displayed in frame 6-1 are displayed as moving images.

  In the description of FIG. 3 described above, the image display is skipped for an image in which a person other than the designated number of people is shown. However, the image may be displayed without being skipped. Alternatively, a continuous frame showing the designated number of people may be displayed one by one by a cursor operation without displaying a moving image. In this way, when there is a desired image, it can be surely selected.

  FIG. 4 is an explanatory diagram of a still image output method according to the present embodiment, and FIG. 4A shows a multi-person image detected from a moving image. In the illustrated example, the image 41 is an image first detected from the moving image as an image of the selected number of people. In FIG. 4B, an image 45 is an image selected by a cursor operation as a center image (highlight frame) of a layout in a generated still image. FIG. 4C shows images before and after the center image 45 (pickup images 44 and 46), which are picked up when a still image is output. FIG. 4D shows a generated still image, and a still image 49 arranged in a layout (upper left corner and upper right corner in the illustrated example) in which reduced images 47 and 48 of the pickup image are selected is displayed. .

  In the example of FIG. 4, the pickup image is one image before and after, but a plurality of images before and after can be picked up. The number of images to be picked up can be selected and set by the user at the start of the still image output mode. Further, the picked-up image is not limited to the image immediately before and immediately after the central image, and may be an image several frames away from the central image. The layout varies depending on the number of images to be picked up (see FIG. 6).

FIG. 5 is a flowchart showing the control operation by the CPU when outputting a still image according to the present embodiment. This flowchart is for explaining a program for causing the digital camera 100 to realize the multi-person image detection function and the still image output function of the present invention.
The following processing is basically explained by an example in which the CPU executes in accordance with a program stored in a program memory such as a flash memory in advance, but it is not necessary to store all functions in the program memory. You may make it implement | achieve by receiving a part or all through a network. Hereinafter, description will be given with reference to FIGS. In the flowchart shown in FIG. 5, the moving image playback mode will be described as an example. However, the scope of the present invention is not limited to the moving image playback mode. For example, continuous shooting images (high-speed continuous shooting) captured in the high-speed continuous shooting mode described above are used. It can also be applied to still image output during playback of (image).

  In FIG. 5, when the still image output mode is selected after selecting the moving image playback mode, the CPU 23 displays a list of representative images (usually reduced images of the first frame) of each moving image file recorded on the recording medium 30 on the LCD 22. The user is allowed to select a desired moving image file by a predetermined key operation (for example, an operation of the cursor key 5) (step S1).

  Next, the CPU 23 displays a window for inputting the number of people on the LCD 22, prompts the designation (selection input) of the number of people shown in one frame, examines a signal from the key input block 28, acquires the designated number of people, It is held (temporarily stored) in the built-in memory 24 (step S2). In the embodiment, two or more people can be designated. In the case of no designation (pressing the ENTER key), it is assumed that three or more people are designated. In the window for inputting the number of people (not shown), the number of people that can be specified (for example, 2, 3, 4, 5,..., 10, 2, 3 or more, 4 or more, 5 or more, The user can specify a desired number of persons among the displayed numbers by operating the cursor keys 5.

  The CPU 23 controls the media controller 20 to read out the frame image of the selected moving image file from the recording medium 30 and decompress the data with the data compression / decompression block 19 to reproduce the image data (step S3). The face area is extracted from the image data controlled and reproduced in step S3, the number of face areas is extracted and counted, and the number information of person images facing the camera in the frame (number information) ) Is acquired (step S4), and the acquired number information is compared with the number of persons specified in step S2. If the acquired number of persons is the specified number of persons, the process proceeds to step S6. The process returns to step S3 to reproduce the frame data (step S5).

  The CPU 23 develops the image data reproduced in step S3 as moving image frame data in the built-in memory 24, and then displays it on the LCD 22 via the display controller 21 (step S6). As a result, among the moving images recorded in the selected moving image file, the first frame in which the number of person images specified in step 2 is captured is displayed stationary.

  When the user fully presses the shutter key 8 (moving image display instruction), a frame in which a specified number of persons are shown facing the camera can be displayed as a moving image. Therefore, the CPU 23 checks the signal from the key input block 28. If the shutter key 8 has been fully pressed, the process proceeds to step S8. If not, the process waits for the shutter key 8 to be fully pressed (step S7).

  The CPU 23 controls the media controller 20 to read out the frame image of the selected moving image file from the recording medium 30 and decompress the data with the data compression / decompression block 19 to reproduce the image data (step S8). The face area is extracted from the image data controlled and reproduced in step S3, and the number information of the extracted face areas is counted to obtain the number information (step S9). The obtained number information and the step S2 If the acquired number is the specified number, the process proceeds to step S11. If not, the process returns to step S8 to reproduce the next moving image frame data (step S10).

  The CPU 23 sequentially displays the moving image frame data (developed in the built-in memory 24 in step S8) on the LCD 22 via the display controller 21 (step S11).

  When the user presses the shutter key 8 halfway, the moving image being played back can be stopped and displayed as it is (that is, still display), so the CPU 23 checks the signal from the key input block 28 and the shutter key 8 is halfway. If pushed, the process proceeds to step S13, and if not, the process returns to step S8 (step S12). As a result, each frame showing the designated number of people is sequentially displayed as a moving image until the shutter key 8 is pressed halfway after this frame. In this step, the moving image may be displayed in a slow manner, and a still image may be displayed one frame at a predetermined time interval.

  When the shutter key 8 is pressed halfway, the CPU 23 stops displaying the moving image, so that the still image of the frame is displayed on the LCD 22. Here, since the user can return the image to be displayed as a still image to the previous frame or advance to the next frame by operating the cursor key 5, the CPU 23 checks the signal from the key input block 28, and the cursor key 5 When operated, the screen is moved back one frame at a time according to the operation of the cursor key 5 to display still images, or to the next frame to display still images (step S13).

  The CPU 23 checks the signal from the key input block 28 and returns to step S3 if there is an instruction to resume moving image display (full depression of the shutter key 8), and if there is a center image confirmation operation (for example, depression of the SET key 6). The process proceeds to S15 (step S14).

  The CPU 23 displays on the LCD 22 a message that prompts the user to specify the number of images to be picked up as frames before and after the center image of the layout when outputting a still image (for example, “Please specify the number of images to be picked up”). When the user selects the number of images to be picked up, the number is acquired and held, and the process proceeds to step S16 (step S15).

  The CPU 23 extracts the layout data (FIG. 6) corresponding to the number of frame images before and after acquired in step S8 from the program memory 29 to generate a pickup image display frame (step S16), and the generated pickup image display frame is displayed. At the position indicated by the layout data, the image is displayed on the LCD 22 so as to be superimposed on the still image (center image) currently displayed (step S17).

  Next, the CPU 23 holds (stores) the frame image of the currently displayed image (center image) from the built-in memory 24 in a predetermined area of the built-in memory 24, and the previous and next frame images (pickup images) in step S15. The number of frame images acquired before and after is obtained (step S18), reduced to the size of the pickup image display frame generated in step S18 and held (stored) in a predetermined area of the built-in memory 24 (step S19), and then reduced. The picked-up image is displayed in a pick-up image display frame displayed on the LCD 22 via the display controller 22. Thereby, a composite image (still image) as shown in FIG. 4D is displayed (step S20). For the reduced image, a thumbnail or the like prepared in advance may be used without being created each time.

  When the user performs confirmation input (for example, presses the SET key 6), the process proceeds to step S24 assuming that a still image generation instruction is issued. When another key (for example, the cursor key 5) is pressed, a change (redo) instruction is issued. As a result, the process proceeds to step S22 (step S21).

  If there is a change instruction, the layout data (FIG. 6) corresponding to the number of frame images before and after acquired in step S14 is checked. If there is no corresponding layout data, the process returns to step S14 (step S22). If there is layout data to be used, the layout data is taken out from the program memory 29 to generate a pickup image display frame, and the process returns to step S17 (step S23).

  When there is a still image generation instruction, the CPU 23 combines the frame image (data) of the central image held in the built-in memory 24 with each reduced image held in the built-in memory 24 in step S19 and displays it on the LCD 22. Still images (data) having the same layout as the still images being generated are generated (step S24), and the specified number of persons held in step S2 and the still image data generated in step S24 are recorded in association with each other on the recording medium 30 (step S24). S25) If there is the next image data, the process returns to step S3, and if not, the series of processes is terminated (step S26). Note that the designated number of people in step S25 may be associated with the still image data generated in step S24 and transmitted to an external processing device such as a printer.

  Through the control operation shown in the flowchart of FIG. 5, the user generates and outputs a still image having a predetermined layout centered on a desired image from moving images in which a desired number of persons face the camera ( Recording or transmission to a printer or an external processing device), it is possible to reproduce and display or print an image taken by everyone facing the camera. Also, since the number of people information is also output at the time of recording a still image, it is possible to automatically print photographs for the number of people based on the number of people information at the time of printing. In addition, since only images with a desired number of persons facing the camera are displayed, the time required for the processing can be reduced as a whole.

  In the flowchart of FIG. 5, the still image obtained by combining the center image and the preceding and succeeding images is generated and output. However, steps S15 to S24 are omitted, and step S14 is changed to “CPU 23 from the key input block 28. The signal is checked, and if there is an instruction to resume moving image display (shutter key 8 is fully pressed), the process returns to step S3. If an output image is confirmed (for example, the SET key 6 is pressed), the next step (that is, step S25). It is also possible to proceed to “recording process”.

  Further, in the flowchart of FIG. 5, an example in which a still image is output from compressed moving image data has been described. However, frame data at a predetermined time interval is automatically selected from uncompressed moving image data, and the frame image is determined in advance. Still images arranged in the layout can also be output.

  In addition, the control operation shown in the flowchart of FIG. 5 includes the steps shown in FIG. 5 and the description thereof, assuming that “moving image” is “high-speed continuous shooting” and “moving image” is “continuous shooting image”. It can be applied as a control flow when a still image is generated from.

  6A and 6B are explanatory diagrams of frame layout examples and layout data. FIGS. 6A to 6E show frame layout examples, and FIG. 6F shows the data structure of layout data stored in the program memory. FIG.

  In the example of FIG. 4, a layout in which reduced images of pickup images are arranged at the upper left corner and the lower right corner when the number of pickup images is one each before and after is shown in FIG. 6A. FIG. 6B shows an example in which reduced images 71-1 and 71-2 of pickup images are arranged in the upper right corner and the upper left corner when there is one each, and FIG. 6B shows the case where the number of pickup images is one each before and after. In this example, reduced images 71-1 and 71-2 of pickup images are arranged in the upper right corner and the lower left corner. FIG. 6C shows the pickup images at the four corners when the number of pickup images is two each in the front and rear. FIG. 6D shows an example in which reduced images 71-1, 71-2, 72-1, and 72-2 are arranged. FIG. 6D shows a reduction of three pickup images on the left and right when the number of pickup images is three on the front and rear. Images 71-1, 71-2, 71-3, 72- , 72-2, 72-3, FIG. 6 (e) shows reduced images 71-1, 71-2 of picked-up images, three at the top and bottom when the number of picked-up images is three respectively. In this example, 71-3, 72-1, 72-2, and 72-3 are arranged. The pick-up image is a frame image whose timing is shifted before and after the photographing time of the central image. In the example shown in the drawing, the layout in the case where 1 to 3 pickup images are specified before and after the photographing time of the central image is shown. However, the number of pickup images is not limited to 1 to 3 before and after, but 4 It may be the above. The layout may be created and registered by the user.

  In FIG. 6E, the layout data 80 is the design of a still image generated when a moving image is converted into a still image, that is, the number 81 of pick-up images, the size 82 of a reduced image generated from the pick-up image, and the arrangement of the reduced images. Position (center coordinates of each reduced image) 83 and the like). The arrangement position of the reduced image may be the coordinates of the start point and the end point of each reduced image.

  In the flowchart of FIG. 5 and FIG. 6, the layout data is displayed at a position defined by a predetermined layout. However, the contour of the person is extracted from the subject portion including the focus area by the contour extraction block 32 (FIG. 2). The frame image display frame may be automatically extracted and displayed on the portion (background portion or the like) excluding the target subject portion so that the layout displays the image. In this case, when the outline of the person of the target subject portion is extracted, the number of frame image display frames specified for the portion excluding the person of the target subject portion on the screen based on the number of picked-up images selected and designated by the user is displayed. Automatic generation and display are performed, and the size of the frame image display frame is determined based on the area and size of the background portion on the left and right of the central subject person.

(Embodiment 2)
In the first embodiment, an example of outputting a still image in which a desired number of persons are directly captured from a moving image has been described. However, in the present embodiment, the personal information of each frame (person who is captured) at the time of playing or shooting a moving image. The number information) is detected and recorded in the recording medium 30 in association with each frame, and is used when outputting a still image. Assume that the digital camera 100 of the first embodiment is used as a camera used in the present embodiment. Hereinafter, description will be made based on FIGS. 1 to 4 and FIGS. 6 to 9.

FIG. 7 is an explanatory view of the multi-person still image position data according to the second embodiment, and FIG. 7A shows a person facing the camera in each frame of the moving image detected by the number detection block 31. It is a figure which shows an example of a number, and illustrates from 1 to k + 1 frames. FIG. 7B is an example of multi-person still image position data generated in this embodiment, and corresponds to each frame in FIG. 7A.
The multi-person still image position data 90 detects the number information of each frame (number information of the number of people in the image) detected by the number detection block 31 at the time of playback or shooting of the moving image and associates it with each frame at the time of recording the moving image. Remembered. In the example of FIG. 7B, the multi-person still image position data 90 corresponds to the frames 1 to k + 1 of FIG. 7A, and the number of people 2,..., 2, 0,. ..., 3, 2, ... are recorded. The digital camera 100 reads out and outputs (displays or prints) specified moving image data based on the multi-person still image position data 90 when the still image output mode is selected in the moving image playback mode and a large number of people are selected. can do.

FIG. 8 is a flowchart showing a control operation by the CPU when outputting a still image of a digital camera having a multi-person still image position data generation function.
This flowchart is for explaining a program for causing the digital camera 100 to realize the multi-person still image position data recording function (corresponding to Step T3 to Step T7) and the still image output function of the present invention.
The following processing is basically explained by an example in which the CPU executes in accordance with a program stored in a program memory such as a flash memory in advance, but it is not necessary to store all functions in the program memory. You may make it implement | achieve by receiving a part or all through a network. Hereinafter, description will be given based on FIGS. 1 to 5 and FIGS. In the flowchart shown in FIG. 8, the moving image playback mode will be described as an example. However, the scope of the present invention is not limited to the moving image playback mode. For example, continuous shooting images (high-speed continuous shooting) captured in the high-speed continuous shooting mode described above. It can also be applied to still image output during playback of (image).

  In FIG. 8, when the still image output mode is selected after selecting the moving image playback mode, the CPU 23 displays a list of representative images (usually reduced images of the first frame) of each moving image file recorded on the recording medium 30 on the LCD 22. Then, the user is made to select a desired moving image file by a predetermined key operation (for example, operation of the cursor key 5) (step T1).

  The CPU 23 checks the management area of the recording medium 30, determines the presence / absence of the number information (multi-person still image position data) associated with the moving image file selected in step T1, and associates the number information with the selected moving image file. If it has been recorded, the process proceeds to step T8 (step T2). That is, when there is multi-person still image position data recorded in association with the moving image file selected in step T1, the number of people detection from the moving image file and the multi-person still image position data recording process in the following steps T3 to T7 Therefore, the still image output operation after step T8 can be started immediately.

  The CPU 23 controls the media controller 20 to read out the frame image of the selected moving image file from the recording medium 30, decompress the data with the data compression / decompression block 19, and reproduce the image data (step T 3). The face area is extracted from the image data controlled and reproduced in step T3, the number of extracted face areas is counted, and the number information (number of persons) of the person images facing the camera in the frame. Information) is acquired (step T4), and the acquired number information is additionally stored in the multi-person still image position storage area secured in the internal memory 24 (step T5). The multi-person still image position storage area has a size corresponding to the number of person storage areas (see FIG. 7B) for the number of frames of the selected moving image as in the multi-person still image position data 90, and the detected number of people. Are sequentially stored in a one-to-one correspondence with the read frames.

  The CPU 23 repeats the operations in steps T3 to T5 until the moving image data of the moving image file selected in step T1 ends (that is, until all frames are read) (step T6). The number data stored in the multi-person still image position storage area is recorded on the recording medium 30 as the multi-person still image position data in association with the moving image file selected in step T1 among the moving image files recorded on the recording medium 30. (Step T7).

  When the multi-person still image position data already exists or when the recording of the multi-person still image position data is completed in step T7, the CPU 23 displays a window for inputting the number of persons on the LCD 22 and designates the number of persons in one frame. (Selection input) is prompted, the signal from the key input block 28 is examined, the designated number of persons is acquired, and held (temporarily stored) in the built-in memory 24 (step T8). In the embodiment, two or more people can be designated. In the case of no designation (depressing the ENTER key), it is assumed that three or more people are designated in this embodiment, but this is not reduced. In the window for inputting the number of people (not shown), the number of people that can be specified (for example, 2, 3, 4, 5,..., 10, 2, 3 or more, 4 or more, 5 or more, The user can specify a desired number of persons among the displayed numbers by operating the cursor keys 5.

  Next, the CPU 23 opens the moving image file selected in step T1 among the moving image files recorded on the recording medium 30 to make it readable (step T9), and then selects the moving image file selected in step T1. The number information (multi-person still image position data) of each frame recorded in the recording medium 30 in association with is read and stored in the multi-person still image position storage area secured in the built-in memory 24 (step T10). The number of people information stored in the multi-person still image position storage area is extracted for one frame and compared with the number of people specified in step T8. If the number of people matches, the process proceeds to step T12. This step is repeated to compare the number of persons (step T11).

  The CPU 23 controls the media controller 20 and, among the frame images of the moving image file opened in step T9, the same frame number as the frame number of the frame in which the number of persons shown in step T8 matches the number of persons in the frame. The numbered frame image is read from the recording medium 30, and the image data expanded by the data compression / decompression block 19 is expanded as frame data in the internal memory 24 and then displayed on the LCD 22 via the display controller 21 (step T12). As a result, among the moving images recorded in the selected moving image file, the first frame in which the number of person images specified in step 2 is captured is displayed as a still image.

  When the user fully presses the shutter key 8 (moving image display instruction), a frame in which a specified number of persons are shown facing the camera can be displayed as a moving image. Therefore, the CPU 23 checks the signal from the key input block 28. If the shutter key 8 has been fully pressed, the process proceeds to step T14. If not, the process waits for the shutter key 8 to be fully pressed (step T13).

  The CPU 23 takes out the number of people information stored in the multi-person still image position storage area and compares it with the number of people designated in step T8. If the number of people matches, the process proceeds to step T15. This step is repeated in order to compare the number of the frames of (step T14).

  The CPU 23 controls the media controller 20 to transmit a frame image having the same frame number as the frame number of the frame number that matches the designated number of persons from the recording medium 30 among the frame images of the moving image file opened in step T9. The image data read and decompressed by the data compression / decompression block 19 is expanded as frame data in the internal memory 24 and then displayed on the LCD 22 via the display controller 21 (step T15).

  When the user presses the shutter key 8 halfway, the moving image being played back can be stopped and displayed as it is (that is, still display), so the CPU 23 checks the signal from the key input block 28 and the shutter key 8 is halfway. If pressed, the process proceeds to step T17, and if not, the process returns to step T14 (step T16). As a result, each frame showing the designated number of people is sequentially displayed as a moving image until the shutter key 8 is pressed halfway after the next frame. In this step, the moving image may be displayed slowly, and further, one frame may be displayed statically at predetermined time intervals.

  When the shutter key 8 is pressed halfway, the CPU 23 stops displaying the moving image, so that the still image of the frame is displayed on the LCD 22. Here, the user can return the image to be statically displayed to the previous frame or advance to the next frame by operating the cursor key 5, so that the CPU 23 checks the signal from the key input block 28 and the cursor key 5 operates. If it is, the screen is moved back one frame at a time according to the operation of the cursor key 5 to display still images, or advanced to the next frame to display still images (step T17).

  The CPU 23 checks the signal from the key input block 28, and if there is an instruction to resume moving image display (full press of the shutter key 8), the CPU 23 returns to step T14, and if there is a center image confirmation operation (for example, pressing the SET key 6), the step. Proceed to T19 (step T18).

  The operation from step T19 to T29 is the same as the operation from step S15 to S25 in FIG. However, the step numbers from step S15 to S25 in FIG. 5 are read in correspondence with T20 to T30, respectively. After the recording process in step T29 is completed, if there is next image data, the process returns to step T12. If not, the series of processes is terminated (step T30).

  In addition to the same effects as in the first embodiment, the operation shown in the flowchart of FIG. 8 generates multi-person still image position data when the moving image file is not associated with multi-person still image position data. Since acquisition does not involve moving image display, it can be done in a short time, and once the multi-person still image position data is generated, the multi-person still image output operation can be started immediately without involving the number detection process. That is, since the multi-person still image output operation in this case does not involve the number-of-people detection process, the time is shorter than the still image output operation of the first embodiment.

  Further, the multi-person still image position data recording operation (steps T3 to T7) shown in the flowchart of FIG. 8 generates multi-person still image position data at the time of moving image shooting and records it on the recording medium 30 in association with the moving image. It can also be applied to cases. For example, when recording captured video data, the number of people in each frame of the video is detected, multi-person still image position data is generated in association with the number of detected frames in each video, and the multi-person still image position data is recorded. What is necessary is just to make it record on the recording medium 30 linked | related with a moving image file.

  In the flowchart of FIG. 8, a still image obtained by synthesizing the center image and the preceding and succeeding images is generated and output. However, steps T19 to T28 are omitted, and step T18 is changed to “CPU 23 reads from the key input block 28. The signal is checked, and if there is an instruction to resume moving image display (shutter key 8 is fully pressed), the process returns to step T14. Proceed to processing).

  In the flowchart of FIG. 8, an example in which a still image is output from compressed moving image data has been described. However, frame data at a predetermined time interval is automatically selected from uncompressed moving image data, and the frame image is determined in advance. Still images arranged in the layout can also be output.

  Further, the control operation shown in the flowchart of FIG. 8 includes the steps shown in FIG. 8 and the description thereof, assuming that “moving image” is “high-speed continuous shooting” and “moving image” is “continuous shooting image”. It can be applied as a control flow when a still image is generated from.

  In the first embodiment and the second embodiment, only the designated number or more than the designated number of people are shown facing the camera, or only the designated number or more than the designated number of people are shown. These images are displayed when the current frame is output (recorded on the recording medium 30 or transmitted to the external processing device), but these images are read from the recording medium 30 without being displayed at all. Of the frames (image data for one moving image frame), only frames that meet the conditions may be output.

In the first embodiment and the second embodiment, the number of people to be detected from the moving image is designated. However, the number of people set in advance may be designated as “multiple people” and used as the multi-person determination value at the time of detection. .

As mentioned above, although several Example of this invention was described, it cannot be overemphasized that this invention is not limited to said each Example, A various deformation | transformation implementation is possible. For example, the term camera can be applied to an electronic camera such as a digital camera, a mobile phone with a camera, an information device having an imaging function, and the like.
In addition, the number of current items is acquired when a moving image once captured is later played back, but may be acquired in real time during imaging.

It is a figure which shows an example of an external appearance structure of a digital camera. It is a block diagram of the digital camera shown in FIG. It is explanatory drawing of the display method of the multi-person image detected from a moving image. FIG. 3 is an explanatory diagram of a still image output method according to the first embodiment. 3 is a flowchart illustrating a control operation by a CPU when outputting a still image according to the first embodiment. It is explanatory drawing of the example of a frame layout, and layout data. It is explanatory drawing of multi-person still image position data. It is a flowchart which shows the control operation by CPU at the time of still image output of the digital camera provided with the multi-person still image position data generation function.

Explanation of symbols

13 CCD
19 Data compression / decompression block 21 Display controller 22 LCD
23 CPU 29 Program memory 30 Recording medium 31 Number detection block 32 Outline extraction blocks 54 and 56 Pickup image 55 Center image 61 Outline 90 Multi-person still image position data 100 Digital camera

Claims (6)

A moving image capturing means for capturing a moving image;
Moving image recording means for recording and storing the moving image captured by the moving image capturing means;
Image reproducing means for reproducing the moving image recorded in the moving image recording means;
Number-of-persons information acquisition means for acquiring the number of persons for each frame of the moving image reproduced by the image reproduction means;
Layout information storage means for storing a plurality of information on the layout of the generated still image;
Image selection means for selecting a central image;
Layout selection means for selecting a layout used for still image generation from a plurality of layouts stored in the layout information storage means;
Based on the layout selected by the layout selection unit, the central image selected by the image selection unit and the other moving images from among the images whose number of persons acquired by the number-of-persons information acquisition unit is a predetermined number of frames. Still image generating means for generating image data of the arranged still image;
An imaging apparatus characterized by the above.   The imaging apparatus according to claim 1, wherein the number of persons is the number of persons whose faces are directed to the imaging apparatus.   The imaging apparatus according to claim 1, further comprising an output unit that outputs the image data generated by the still image generation unit. Comprising a number-of-people setting means for setting the predetermined number;
The imaging apparatus according to claim 1, wherein the number set by the number-of-persons setting unit is a plurality of numbers. To a computer equipped with an image recording memory,
A function of playing back a moving image recorded in the image recording memory;
A function of acquiring the number of people for each frame of the video to be played;
A function for storing a plurality of information of the layout of the generated still image;
A function to select the center image,
A function for selecting a layout to be used for still image generation from the plurality of stored layouts;
A function of generating image data of a still image in which a central image selected from images whose number of acquired persons is a predetermined number of frames and another moving image are arranged based on a selected layout;
A program that executes A computer-readable storage medium storing the program according to claim 5.

Images