JP2011066828A - Imaging device, imaging method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow a user to recognize a field angle or composition of a still or moving image that is previously imaged when imaging the still or moving image. <P>SOLUTION: A display unit 111 is so configured that face region information detected from a first image data by a face detection unit 106 together with the image based on the imaged first image data is superimposed with the face region information of the image data which is stored in NVRAM 109 and which has been imaged prior to the first image so that the superimposed image data are displayed on a display 113. This results in easily imaging video images whose composition is similar to each other by adjusting the composition thereof. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program, and more particularly, to a technique suitable for use in intermittent video shooting.

  Conventionally, in an imaging device such as a digital camera or a digital video camera, when an image is shot intermittently, an angle of view and a composition between the images may be different. For example, there is a desire to sequentially shoot moving images and still images with the same composition for a plurality of people one by one in order to conduct an interview or the like. In such a case, if the positional relationship between the subject and the imaging device is not fixed, the angle of view and composition differ between videos.

  There is also a digital video camera that has a moving image shooting function and a still image shooting function and can switch between a moving image shooting mode and a still image shooting mode. When a still image is shot while shooting a moving image using such a digital video camera, the angle of view and composition during shooting of the moving image may differ from the angle of view and composition during shooting of the still image. In such a case, the angle of view and composition of the movie will be different before and after taking a still image, and immediately after shooting the movie and still image before shooting the still image when the movie is played back continuously. Discomfort occurs due to the difference between the video and the video.

  Thus, for example, Patent Document 1 discloses an imaging apparatus having a moving image shooting function and a still image shooting function that solves such a problem. According to this imaging apparatus, when a still image is shot during moving image shooting, the image data at the time when the moving image shooting mode is changed to the still image shooting mode is held. Then, the image data held when the moving image shooting is resumed is displayed on the display of the imaging apparatus as a semi-transparent image. Thus, the angle of view and the composition are adjusted with reference to the image transmitted when the shooting of the moving image is resumed, and the shift of the angle of view and the composition of the moving image is reduced before and after the shooting of the still image.

JP 2006-129424 A JP-A-10-232934 JP 2000-48184 A

  However, the technique disclosed in Patent Document 1 has the following problems. In other words, since the last shot video is translucent or contour extracted and superimposed on the video to be shot and displayed, it is affected by the fact that the original video to be shot is blurred over the entire screen when continuously played back. End up. Therefore, although the influence is small when the video to be translucent or contour extracted is similar to the video to be taken, there are the following problems when interviewing or the like as described above. That is, if a plurality of people are photographed one by one with the same composition, there is a problem that the faces of other people are superimposed and the original video cannot be obtained.

  The present invention has been made in view of the above-described problems, and it is an object of the present invention to enable a user to recognize the angle of view and composition of a previously captured still image or moving image when capturing a still image or moving image.

  An imaging device according to the present invention includes an imaging unit that captures an image of a subject and generates image data, a display unit that displays an image based on the image data generated by the imaging unit on a display unit, and the imaging unit A subject detection unit for detecting a subject region from the image data; and a holding unit for holding information on the subject region detected by the subject detection unit. The display unit includes a first image generated by the imaging unit. Along with the image based on the data, information on the subject area detected from the first image data by the subject detection means and the imaging means held in the holding means before the first image data. Information on the subject area of the generated second image data is displayed on the display unit.

  According to the present invention, the user can recognize a deviation from the angle of view or composition of a still image or a moving image that was previously captured. Thus, it is possible to shoot with matching the angle of view and composition by adjusting the angle of view and composition of the still image or moving image.

It is a block diagram which shows the structural example of the imaging device in embodiment. It is a figure which shows an example of the face information output from a face detection unit. It is a flowchart which shows an example of the process sequence in the case of image | photographing an image | video. It is a figure which shows the outline | summary of a detection of a face area | region. It is a figure which shows the image | video displayed on the display on which the rectangular figure was superimposed. It is a flowchart which shows an example of the process sequence of an imaging | photography assistance function. It is a figure which shows the outline | summary of a detection of a face area | region. It is a figure which shows an example of the rectangular figure displayed using a imaging | photography assistance function. It is a figure which shows an example of the rectangular figure displayed using a imaging | photography assistance function. It is a flowchart which shows an example of the process sequence in the case of image | photographing an image | video. It is a figure which shows the outline | summary of a detection of a face area | region. It is a figure which shows the image | video displayed on the display on which the rectangular figure was superimposed.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus 10 according to the present embodiment.
In FIG. 1, an imaging apparatus 10 according to the present embodiment is a compact digital camera, and is an electronic shutter type digital still camera. The system controller 100 controls the entire imaging apparatus 10 of the present embodiment, controls each unit to be described later, and records image data according to user settings and operations.

  The optical lens unit 101 is a unit having an aperture, zoom, and focus function. The optical control unit 102 is for controlling the optical lens unit 101 in accordance with an instruction from the system control controller 100. Specifically, control for changing the aperture, zoom, and focus of the optical lens unit 101 is performed.

  The image sensor 103 is for converting light (video) introduced through the optical lens unit 101 into an electrical signal. Specifically, a CCD image sensor (CCD image sensor) using a CCD or a CMOS image sensor (CMOS image sensor) using a CMOS is used. In the imaging apparatus 10 according to the present embodiment, an imaging element having 3072 horizontal pixels and 2304 vertical pixels is used. The imaging control unit 104 is for controlling the imaging device 103 in accordance with an instruction from the system control controller 100. Specifically, sensitivity adjustment (gain control) of the image sensor 103, an operation of reading data from the image sensor 103 according to the shutter time, and the like are performed.

  The recording medium control unit 105 functions as a recording unit and records data on the recording medium 112 in accordance with an instruction from the system controller 100. As a specific example of the recording medium 112, a card-type removable memory such as an SD memory card Compact Flash (registered trademark) card or a memory stick is used. However, any recording medium may be used. The recording medium 112 records image data (RAW data) output from the imaging control unit 104 and image data (JPEG data or EXIF data) encoded by the system controller 100 from the image data. .

  The face detection unit 106 functions as a subject detection unit and detects face area information from image data in accordance with an instruction from the system controller 100. The face detection unit 106 receives image data output from the imaging control unit 104 or image data encoded by the system control controller 100. Then, information on the position and size of the face included in the image data is output to the system controller 100. In the present embodiment, an example in which a human face is detected as a subject will be described. However, an animal or the like may be detected in addition to a person.

  In the present embodiment, the face detection unit 106 detects a human face area as a characteristic part included in an image. Here, various techniques are known as techniques for detecting a face area as a subject area. For example, there are a method using learning typified by a neural network and a support vector machine, and a method of extracting a part having physical shape features such as eyes and nose by template matching from an image region. In addition, there is a method of detecting an image feature amount such as a skin color or an eye shape and analyzing it using a statistical method (for example, see Patent Document 2 or Patent Document 3).

  FIG. 2 is a diagram illustrating an example of face area information output from the face detection unit 106. In the example shown in FIG. 2, the face detection unit 106 outputs position and size information for each face from one video.

  Returning to the description of FIG. 1, the ROM 107 is a memory for storing a program for controlling the imaging apparatus 10 of the present embodiment and data used by the program. When the image pickup apparatus 10 is turned on, the system controller 100 reads a program from the ROM 107 and starts controlling the image pickup apparatus 10. The RAM 108 is a rewritable memory for using a program for controlling the imaging apparatus 10 of the present embodiment as a work area. Further, it is also used as a buffer for temporarily storing image data output from the imaging control unit 104.

  The NVRAM 109 functions as a holding unit, and is rewritable to hold setting values designated by the user in the UI unit 110 described later and parameters that the imaging apparatus 10 of the present embodiment needs to hold over the power cycle. It is a non-volatile memory. In the imaging apparatus 10 of the present embodiment, at least one piece of information on the face area detected by the face detection unit 106 is held. The UI unit 110 is for transmitting a user instruction to the imaging apparatus 10 of the present embodiment. The UI unit 110 includes a menu button for instructing the display unit 111 to display a menu and a button for operating the menu. Furthermore, in the present embodiment, an auxiliary button for enabling the shooting assist function is provided to superimpose and display an image for assisting shooting on the display 113 of the display unit 111 during shooting. This photographing assist function will be described later. The imaging apparatus 10 is also provided with a shooting button (not shown) for instructing shooting.

  The display unit 111 functions as a display unit, and is a unit for displaying an image in accordance with an instruction from the system controller 100. The display unit 111 includes a thin display 113 such as a liquid crystal display (LCD) that is a display unit, and a display driver unit that controls the display 113. In the imaging device 10 of the present embodiment, a liquid crystal display device having 320 pixels in width and 240 pixels in length is used.

  The display unit 111 displays a menu screen for performing various settings of the imaging apparatus 10 of the present embodiment and an image based on the image data output from the imaging control unit 104. Thus, the display unit 111 is used as a finder for determining composition. Further, the display unit 111 displays a captured image for confirming the captured image, and also displays an image based on the image data stored in the recording medium 112. Thus, the display unit 111 is also used as an image viewer. In addition, it is possible to superimpose and display an arbitrary graphics image on each image. In the imaging device 10 of the present embodiment, the number of pixels of the graphics image to be superimposed is the same as the number of pixels of the display unit 111.

  Next, the photographing operation in the present embodiment will be described with reference to the flowcharts of FIGS. FIG. 3 is a flowchart illustrating an example of a processing procedure when shooting a video in the present embodiment. 3 is performed under the control of the system controller 100.

  First, in step S <b> 301, an image input from the optical lens unit 101 is converted into an electrical signal by the imaging device 103, and the imaging control unit 104 outputs this electrical signal to the system controller 100 as image data. Next, in step S302, the system controller 100 outputs the image data acquired in step S301 to the display unit 111, and the display unit 111 displays the image for composition determination (preview) on the display 113. For example, when image data as shown in FIG. 4A is output from the imaging control unit 104 in step S301, the video is adjusted to the resolution (320 × 240) of the display unit 111 and displayed. .

  Next, in step S303, the face detection unit 106 determines whether a face is included in the image data acquired in step S301. If the result of this determination is that a face is not included, the processing of the next step S304 is not performed, and the routine proceeds to step S305.

  On the other hand, if the result of determination in step S303 is that a face is included, in step S304, the face detection unit 106 notifies the system controller 100 of face area information in the format shown in FIG. For example, for the image data shown in FIG. 4A, as shown in FIG. 4B, face area information 402 for the detected face area 401 is output. Then, the display unit 111 superimposes the acquired face area information on the video displayed in step S302.

  In the present embodiment, a rectangular figure (face frame) representing a face area is represented in green. For example, when the face area information 402 shown in FIG. 4B is acquired for the image data shown in FIG. 4A, the display unit 111 displays the position of the face area and the face area information 402 as shown in FIG. The size is converted in accordance with the resolution of the display unit 111. Then, a rectangular figure 501 as shown in FIG. 5 is created, and the created rectangular figure 501 is superimposed on the image data and displayed on the display 113 of the display unit 111.

  Next, in step S305, the system controller 100 determines whether or not the photographing assist function is enabled. In the present embodiment, the photographing assistance function can be validated by pressing the auxiliary button of the UI unit 110 by the user or selecting the photographing assistance function from the menu displayed by the menu button. If the result of this determination is that the shooting assist function is disabled, step S306 is skipped and processing proceeds to step S307.

  On the other hand, if the result of determination in step S305 is that the shooting assist function is valid, the display unit 111 displays information for shooting assist on the display 113 of the display unit 111 in step S306. Details of the processing in step S305 will be described later in the description of the flowchart of FIG.

  Next, in step S307, the system controller 100 confirms whether there is a shooting instruction. Specifically, it is determined whether or not a shooting button (not shown) has been pressed by the user, or whether or not a shooting instruction has been given after completion of the self-timer. If the result of this determination is that a shooting instruction has been given, the process proceeds to step S308. If no shooting instruction has been given, the process returns to step S301, and the composition determination (preview) video is updated.

  Next, in step S308, the system controller 100 converts the image data at the time of shooting instruction acquired via the imaging control unit 104 into the EXIF file format. Then, the recording medium control unit 105 records the EXIF file format image data on the recording medium 112.

  Next, in step S309, the face detection unit 106 determines whether it is determined in step S303 that a face is included. If the face is not detected in step S303 as a result of this determination, the process in the next step S310 is not performed and the process is terminated as it is. On the other hand, if the face is detected in step S303 as a result of the determination in step S309, the system controller 100 records the face area information 402 shown in FIG. 4B in the NVRAM 109 in step S310. In this manner, the NVRAM 109 holds at least the latest face area information among the face area information detected from the captured video.

  Next, the processing in step S306 in FIG. 3 will be described in detail with reference to the flowchart in FIG. FIG. 6 is a flowchart illustrating an example of a processing procedure for performing face frame display by the shooting assist function in step S306 of FIG. The flowchart in FIG. 6 constitutes a part of the flowchart in FIG. 3 and will be described with reference to each step in FIG. In the description of FIG. 6, it is assumed that, for example, the video shown in FIG. Accordingly, when a face is detected, it is assumed that face area information 702 for the detected face area 701 is recorded in the NVRAM 109 as shown in FIG.

  First, in step S601, the system controller 100 acquires face area information recorded in the NVRAM 109, and determines in step S602 whether face area information has been acquired. If the face information is not recorded in the NVRAM 109 and the face area information cannot be acquired as a result of this determination, the process is terminated as it is. On the other hand, if it is determined in step S602 that face area information has been acquired, the process proceeds to step S603. When a plurality of pieces of face area information are recorded in the NVRAM 109, the latest face area information (hereinafter referred to as past face area information) is acquired. For example, when face area information 702 shown in FIG. 7B is recorded in the NVRAM 109 as past face information, the face area information is acquired, and the process proceeds to step S603. In addition, when a plurality of face area information is recorded in the NVRAM 109, the system controller 100 functions as a selection unit so that which face area information is to be acquired can be selected according to a user operation. Good.

  In subsequent steps S603 to S605, from the relationship between the current face area information of the first image data detected in step S303 of FIG. 3 and the past face area information of the second image data acquired in step S601. Then, assist face frame processing is determined.

  First, in step S603, the system controller 100 determines whether face detection is successful in step S303 of FIG. That is, it is determined whether or not the image data output from the imaging control unit 104 to the system controller 100 includes face area information (current face area information). If the result of this determination is that current face area information is not included, processing proceeds to step S606, where assist face frame processing is performed. Details of this process will be described later.

  On the other hand, if the result of determination in step S603 is that current face area information is included, processing proceeds to step S604. Then, the system controller 100 functions as a comparison unit, compares the face size in the past face area information with the face size in the current face area information, and approximates the face size using the following equation (1). It is determined whether or not.

  In this embodiment, as shown in Equation 1, the face size is first normalized by the image size, and the difference between the normalized face size width and height is both equal to or less than a predetermined value (± 10% or less). In this case, it is determined that they are approximate. If the face size is not approximated as a result of the determination in step S604, the process proceeds to step S607, and assist face frame processing is performed. Details of this process will be described later.

  On the other hand, if the result of determination in step S604 is that the face size is approximate, in step S605, the face position in the past face area information and the face position in the current face area information are compared, and the following number 2 is used to determine whether the face positions are close to each other.

  In this embodiment, as shown in Equation 2, the face position is first normalized by the image size, and the differences between the normalized vertical and horizontal positions of the face are both equal to or less than a predetermined value (± 5% or less). In some cases, it is determined that they are close to each other. If the face position is not close as a result of the determination in step S605, the process proceeds to step S608, and if the face position is close, the process proceeds to step S609.

  Step S606 is an assisting face frame display process when only the past face area information exists and the current face area information does not exist. In this process, the display unit 111 superimposes only past face area information on the video displayed in step S302. At this time, by superimposing the past face area information in a format different from that in step S304, the user is made aware that the video superimposed in step S606 is information for assisting. In the present embodiment, a rectangular figure representing a face area is superimposed in gray.

  For example, when the face area information 702 shown in FIG. 7B is acquired as the past face area information with respect to the image data shown in FIG. 7A, the display unit 111 is as shown in FIG. Then, the position and size of the face area are converted into a position and size on the graphics image to be superimposed. Then, a rectangular figure 801 as shown in FIG. 8 is created and superimposed on the image data and displayed on the display 113 of the display unit 111.

  On the other hand, step S607 is a face frame display update process for assisting when the face size in the past face area information and the face size in the current face area information are not approximate. In this process, the image superimposed in step S304 is updated, and past face area information is also superimposed on the image in step S302 in addition to the current face area information. At this time, by superimposing the past face area information in a format different from that in step S304, the user is made aware that the video superimposed in step S607 is information for assisting. In the present embodiment, a rectangular figure representing a face area of past face area information is superimposed in gray.

  For example, when the face area information 402 shown in FIG. 4B is acquired as the current face information, and the face area information 702 shown in FIG. The position and size of the face area are converted according to the resolution of the display unit 111. Further, as shown in FIG. 9A, the current face area information is displayed as a green rectangular figure 901 and the past face area information is created as a gray rectangular figure 902 so that the display modes are different from each other. . Then, the created gray rectangular figure 902 is superimposed on the image data and displayed on the display 113 of the display unit 111.

  Step S608 is an assisting face frame display update process when the face size in the past face area information and the face size in the current face area information are close to each other but are not close to each other. In this process, the image superimposed in step S304 is updated, and past face area information is also superimposed on the image in step S302 in addition to the current face area information. At this time, by superimposing past face area information in a format different from that in step S304, the user is made aware that the video superimposed in step S608 is information for assistance. In addition, the current face frame (rectangular figure) is updated to a format different from that in step S304 to notify the user that the face size is approximate. In the present embodiment, the rectangular figure representing the face area of the past face area information is displayed in gray, and the rectangular figure representing the face area of the current face area information is blinked in green and displayed.

  For example, when the user who is going to shoot the video shown in FIG. 4A presses the zoom button, zooming is performed until the face size in the past face area information and the face size in the current face area information are approximated. Suppose you have gone. In this case, the display unit 111 converts the position and size of each face area according to the resolution of the display unit 111. Then, as shown in FIG. 9B, a green blinking rectangular figure 903 and a gray rectangular figure 904 are created and displayed on the display 113 of the display unit 111 while being superimposed on the image data.

  Further, step S609 is an assisting face frame process in the case where the face size in the past face area information and the face size in the current face area information are close and close to each other. In this process, the image superimposed in step S304 is updated, and the current face frame is updated to a format different from that in steps S304 and S608. Thus, the user can recognize that the past face frame and the current face frame are close in size and close to each other. In the present embodiment, a rectangular figure representing the face area of the current face area information is superimposed so as to blink in red. At this time, the rectangular figure representing the face area of the past face area information is not overlapped because the position is close and the size is approximate.

  For example, when the user who is going to shoot the video shown in FIG. 4A presses the zoom button and changes the zoom or composition, the value is close to the position and size of the face area in the past face area information. It shall be like that. In this case, the display unit 111 converts the position and size of the face area of the current face area information according to the resolution of the display unit 111. Then, as shown in FIG. 9C, a red blinking rectangular figure 905 is created, and the created red blinking rectangular figure 905 is superimposed on the image data and displayed on the display 113 of the display unit 111.

  As shown in FIG. 4, the processes in steps S301 to S307 and the process in FIG. 6 are repeated until the user gives a shooting instruction. That is, the video displayed on the display 113 of the display unit 111 is updated as needed. Accordingly, the processing of steps S606 to S609 allows the user to adjust the composition by comparing the past face area information and the current face area information while viewing the display 113. Accordingly, for example, since the composition as shown in FIG. 9A can be adjusted to the composition as shown in FIG. 9C, an image with a similar composition can be easily taken.

(Second Embodiment)
In the present embodiment, a digital video camera capable of shooting moving images and still images will be described as an example of an imaging apparatus, and the description will be made focusing on differences from the first embodiment. Note that the internal configuration of the imaging apparatus according to the present embodiment is basically the same as that shown in FIG. In the present embodiment, the system controller 100 is the same in that the image data output from the imaging control unit 104 is converted into still image data such as JPEG data or EXIF data. However, in the present embodiment, continuous image data is further converted into MPEG2 data, Motion JPEG data, H.264, or the like. It is converted into moving image data such as H.264 data. Then, the recording medium control unit 105 records these still image data and moving image data on the recording medium 112. In addition, the imaging apparatus 10 includes a moving image shooting start / stop button (not shown) for instructing start or stop of moving image shooting processing, a shooting button for instructing shooting of a still image, and the like.

  In the present embodiment, at least three pieces of face area information detected by the face detection unit 106 are recorded in the NVRAM 109. Here, at least three are because it is necessary to hold face area information for three frames of the first frame, the last frame, and the last frame in which a face is detected for a series of moving images (one take). . Thereby, face area information can be held for a moving image currently being shot and for at least one previously shot moving image. Note that the resolution of the image sensor 103 in this embodiment is 1920 pixels wide and 1080 pixels vertical, and similarly, the display unit 111 uses a liquid crystal display device having 480 pixels wide and 270 pixels vertical.

  Next, the photographing operation in the present embodiment will be described with reference to the flowchart of FIG. FIG. 10 is a flowchart illustrating an example of a processing procedure when shooting a video in the present embodiment. Here, each process shown in FIG. 10 is performed under the control of the system controller 100. Steps S1001 to S1006 are the same as steps S301 to S306 in FIG. Details of the processing in step S1006 will be described later. In this embodiment, when image data as shown in FIG. 11A is output from the imaging control unit 104, face area information 1102 for the detected face area 1101 as shown in FIG. 11B is acquired. An example will be described. As a result, as shown in FIG. 12, a rectangular figure 1201 is superimposed and displayed on the image.

  Next, in step S1007, the system controller 100 determines the current shooting situation. Specifically, there are four states: stop, immediately after shooting, shooting, and immediately after shooting stop, based on the operation status of the imaging apparatus 10 of this embodiment and the operation status of the moving image shooting start / stop button of the UI unit 110. Judge either one. As a result of this determination, if the camera is stopped, the process returns to step S1001. If it is during shooting (recording), the process proceeds to step S1010. If it is immediately after shooting is stopped, the process proceeds to step S1009.

  In step S1008, the system controller 100 records the face area information of the face detected from the image data acquired in step S1001 in the NVRAM 109 as face area information of the first frame of the moving image currently being shot. If no face is detected from the image data, information indicating that the face is not included in the first frame of the currently captured moving image is recorded in the NVRAM 109. Then, the process proceeds to step S1010.

  On the other hand, in step S1009, similar to step S1008, the face area information detected from the image data acquired in step S1001 is recorded in the NVRAM 109 as the face area information of the last frame of the movie currently being shot. . If no face is detected from the image data, information indicating that the face is not included in the last frame of the currently captured moving image is recorded in the NVRAM 109. Then, the process proceeds to step S1010.

  Next, in step S1010, the face detection unit 106 determines whether it is determined in step S1003 that a face is included. If the result of this determination is that a face has not been detected in step S1003, step S1011 is skipped and processing proceeds to step S1012. On the other hand, as a result of the determination in step S1010, if a face is detected in step S1003, the system controller 100 records the face area information in the NVRAM 109 as face area information of the final detection frame in step S1011.

  Next, in step S1012, the system controller 100 converts the image data acquired via the imaging control unit 104 into a predetermined file format, and the recording medium control unit 105 stores the converted image data in the recording medium 112. Record. MPEG2 and H.264 In a file format such as H.264, a moving image file is generated with reference to a plurality of frames. In the case of such a file format, the acquired image data is temporarily stored in the RAM 108 and recorded on the recording medium 112 every time desired file format data is configured. Then, the process returns to step S1001.

  Next, the process of step S1006 in FIG. 10 will be described in detail. As described above, step S1006 in FIG. 10 is processed according to the flowchart shown in FIG. In the present embodiment, differences from the first embodiment will be described.

  In step S601, face area information recorded in the NVRAM 109 is acquired. As described above, the NVRAM 109 records face area information of the first frame, the last frame, and the last frame in which the face was detected with respect to at least one moving image previously shot. Note that face area information may be held as invalid depending on the captured moving image.

  In the present embodiment, the face area information for photographing assistance can be selected in advance by the menu from the first frame, the last frame, or the frame in which the face is detected last. Thus, in step S601, face area information is acquired in accordance with the user's selection. If it is determined in step S602 that the face area information has been acquired, the process proceeds to step S603. If the face area information is retained as invalid, the process in the flowchart of FIG. 6 ends. Note that the processing in steps S603 to S609 is the same as that in the first embodiment.

  As shown in FIG. 10, the processes in steps S1001 to S1007 and the process in FIG. 6 are repeated regardless of the shooting state. That is, the video displayed on the display 113 of the display unit 111 is updated as needed. Accordingly, the processing of steps S606 to S609 allows the user to adjust the composition by comparing the past face area information and the current face area information while viewing the display 113. Images with similar composition can be easily taken.

  For example, when shooting is stopped once while moving image shooting is being performed, and moving image shooting is resumed after shooting a still image, the composition may differ between the still image and the moving image because the shooting purpose is different. In particular, when the aspect ratio of a moving image is 16: 9, while the aspect ratio of a still image is 4: 3, the composition is likely to be different.

  Conventionally, when the composition is changed between a moving image before shooting a still image (first moving image) and a moving image after shooting a still image (second moving image), the last frame of the first moving image and the first frame The composition differs between the first frame of the second video. For this reason, if it reproduces | regenerates continuously, it will become an unnatural discontinuous thing. However, in this embodiment, when determining the composition of the second moving image, by selecting the last frame of the first moving image in step S601, the user can select the first moving image when resuming the shooting of the moving image. Differences in composition from the final frame can be recognized. Thereby, discontinuity with the first moving image can be eliminated, and shooting of the second moving image can be started with a composition so that there is no sense of incongruity.

(Other embodiments according to the present invention)
In the above-described embodiment, the face area information is superimposed and displayed as a rectangular figure (face frame). However, the present invention is not limited to this, and the face area information is converted to the user by using text, icons, other figures, and the like. You may make it recognize. In the above-described embodiment, when the face size in the past face area information and the face size in the current face area information are not approximate, the color of the rectangular figure to be superimposed is changed and superimposed. Is not limited to this. You may make it make a user recognize using a text, an icon, another figure, etc. Furthermore, when the face size in the past face information is close to the face size in the current face information, the rectangular figure is blinked, but the present invention is not limited to this. You may make it possible for a user to recognize using a text, an icon, another figure, etc.

  In the first embodiment, the face area information is recorded in the NVRAM 109 in step S310, but the image data itself may be recorded in the NVRAM 109 instead. In this case, in step S601 in FIG. 6, face area information is acquired from the recorded image data by the face detection unit 106, and the subsequent processing is performed. Information specifying the image data such as a file name may be stored in the NVRAM 109 instead of the image data. In this case, in step S601 in FIG. 6, image data is acquired from the recording medium 112 based on this information, face area information is acquired from the image data by the face detection unit 106, and the subsequent processing is performed.

  In the second embodiment, the face area information is recorded in the NVRAM 109 in steps S1008, S1009, and S1011. However, instead of the face area information, the target frame of the video may be recorded as image data such as a JPEG file. In this case, in step S601 in FIG. 6, face area information is acquired from the recorded image data by the face detection unit 106, and the subsequent processing is performed. Note that information specifying a target frame such as a frame number and a time code may be recorded in the NVRAM 109 instead of the image data. In this case, in step S601 of FIG. 6, the image data of the frame is acquired from the recording medium 112 based on this information, the face area information is acquired from the image data by the face detection unit 106, and the subsequent processing is performed.

  Further, as described above, when image data is acquired in step S601 of FIG. 6 and face area information is acquired from the image data, a threshold value serving as a reference for determining the face area for the face detection unit 106 May be settable. In this case, if the face detection unit 106 fails to acquire the face area in step S601 in FIG. 6, the threshold value may be changed and the face area may be detected and acquired again.

  In the above-described embodiment, when the face size is not approximated in step S604 in FIG. 6, the rectangular figure (face frame) is displayed in a different color, but the zoom magnification is changed by the optical control unit 102, and the face is displayed. The size may be automatically matched. In this case, if the face size matches successfully, the process proceeds to step S605, and if unsuccessful, the process proceeds to S607.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

100 system controller, 106 face detection unit, 109 NVRAM, 111 display unit, 113 display

Claims (16)

Imaging means for imaging a subject and generating image data;
Display means for displaying an image based on the image data generated by the imaging means on a display unit;
Subject detection means for detecting a subject area from the image data generated by the imaging means;
Holding means for holding information on the subject area detected by the subject detection means,
The display means is held in the holding means together with information on the subject area detected from the first image data by the subject detection means, together with an image based on the first image data generated by the imaging means. An image pickup apparatus for displaying information on a subject area of second image data generated by the image pickup means before the first image data on the display unit.   The imaging apparatus according to claim 1, wherein the subject detection unit detects a human face area as the subject area.   The imaging apparatus according to claim 1, wherein the subject detection unit detects a size and a position of the subject region.   The display means displays information on the subject area of the first image data and information on the subject area of the second image data in different display modes. The imaging apparatus of Claim 1. A comparison means for comparing the subject area of the first image data with the subject area of the second image data;
The display means notifies that the subject areas match when the difference between the size and position of the subject area is equal to or less than a predetermined value as a result of the comparison by the comparison means. The imaging device described in 1.   When the difference between the size and position of the subject area is equal to or less than a predetermined value as a result of the comparison by the comparison means, the display means includes information on the subject area of the first image data and the second image data. 6. The image pickup apparatus according to claim 5, wherein information on the subject area is displayed in one display form to notify that the subject areas match.   As a result of the comparison by the comparison means, when the size of the subject area exceeds a predetermined value, the size of the subject area of the first image data becomes the size of the subject area of the second image data. The imaging apparatus according to claim 5, further comprising a control unit that controls a zoom magnification of the imaging unit.   8. The image data according to claim 1, wherein the second image data is image data in which a subject area is detected last by the subject detection means before the first image data. The imaging device described in 1. Selecting means for selecting information on the subject area to be displayed as information on the subject area of the second image data from the information on the subject area held in the holding means;
The display means displays on the display unit information on the subject area of the first image data and information on the subject area selected by the selection means together with an image based on the first image data. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized.   The imaging apparatus according to claim 1, wherein the second image data is still image data.   The second image data is moving image data, and the selection unit includes a first frame of the moving image data held in the holding unit, a frame in which a subject area is detected by the subject detection unit, and a last frame. The imaging apparatus according to claim 9, wherein information is selected from information on a subject area. The holding unit also holds the second image data generated by the imaging unit,
When the holding unit does not hold information on the subject area of the second image data, the subject detection unit changes the threshold value serving as a reference for detecting the subject area and is held by the holding unit. The imaging device according to claim 1, wherein a subject area is detected from the second image data.   13. The display unit according to claim 1, wherein the display unit displays the subject region information of the first image data and the subject region information of the second image data in a rectangular figure. The imaging device according to item.   The imaging apparatus according to claim 1, further comprising a recording unit that records image data generated by the imaging unit on a recording medium. An imaging step of imaging a subject and generating image data;
A display step of displaying an image based on the image data generated in the imaging step on a display unit;
A subject detection step of detecting a subject region from the image data generated in the imaging step;
A holding step of holding information on the subject area detected in the subject detection step in a holding unit,
In the display step, information on the subject area detected from the first image data in the subject detection step and the image based on the first image data generated in the imaging step, and the holding unit hold the information. And displaying information on the subject area of the second image data generated in the imaging step before the first image data on the display unit. An imaging step of imaging a subject and generating image data;
A display step of displaying an image based on the image data generated in the imaging step on a display unit;
A subject detection step of detecting a subject region from the image data generated in the imaging step;
Causing the computer to execute a holding step of holding information on the subject area detected in the subject detection step in a holding unit;
In the display step, information on the subject area detected from the first image data in the subject detection step and the image based on the first image data generated in the imaging step, and the holding unit hold the information. And a program for causing the computer to display information on the subject area of the second image data generated in the imaging step before the first image data on the display unit.

Images