JP2013201662A - Image reproduction device and imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of accurately detecting a person's facial expression even when the size of the person with respect to a field is small.SOLUTION: An image reproduction device comprises: a first display unit and a second display unit; a subject detection unit that detects a main subject from each of a plurality of captured images obtained by capturing a field consecutively in time series; and a control unit that extracts a partial image including an image area of the main subject detected from the captured image, and sequentially displays the captured image on the first display unit, and the partial image on the second display unit.

Description

  The present invention relates to an image reproduction device and an imaging device.

  Conventionally, a technique for imaging a person with a desired facial expression such as a smile has been developed.

  For example, each time a smiling face of a person is detected, the image is taken and the image is generated and stored.If a person who is blinking at the time of image capture is detected, the image of the face portion of the person who is blinking is enlarged and the blink is detected There is a technique for determining whether or not to save an image (see Patent Document 1).

JP 2010-62715 A

  However, in the conventional technology, when a person is imaged against a background of a mountain or the like, the size of the person with respect to the scene is relatively small. It may be detected.

  SUMMARY OF THE INVENTION In view of the above-described problems of the prior art, an object of the present invention is to provide a technique that can detect a human facial expression with high accuracy even when the size of the person relative to the scene is small. .

  In order to solve the above-described problem, an aspect of an image reproduction device illustrating the present invention includes a first display unit, a second display unit, and a plurality of captured images obtained by capturing an image of the scene continuously in time series. The main subject is detected from the subject detection unit, and the partial image including the image area of the main subject detected from the captured image is extracted, and the captured image is sequentially displayed on the first display unit and the partial image is sequentially displayed on the second display unit. A control unit.

  The control unit may display the partial image on the second display unit when the size of the main subject in the captured image is equal to or smaller than a predetermined size.

  The first display unit further includes a first touch panel that receives an operation from the user, and the control unit displays a partial image on the second display unit when the size of the main subject is equal to or smaller than a predetermined size. The button may be displayed on the first display unit, and the partial image may be displayed on the second display unit in accordance with the display button operation of the first touch panel by the user.

  The control unit may turn on the power of the second display unit when displaying the partial image.

  In addition, when the subject detection unit detects a plurality of main subjects, the control unit may extract partial images for each main subject and display the partial images of the plurality of main subjects on the second display unit.

  In addition, the storage unit stores reference data indicating the priority order of the plurality of main subjects, and the control unit performs the second display of the partial images of the plurality of main subjects in a size corresponding to the priority order based on the reference data. May be displayed on the screen.

  The second display unit further includes a second touch panel that receives an operation from the user, and the control unit determines the size of the partial image displayed on the second display unit according to the operation of the second touch panel by the user. It may be changed.

  Further, the control unit determines whether or not the color components of the image area of the main subject in the partial image and the background portion other than the main subject are similar. If the color components are similar, the color component of the background portion is determined as the main subject. The partial image may be displayed on the second display unit by changing the color component to a different color component.

  One aspect of an imaging apparatus that exemplifies the present invention includes an imaging unit that captures an image of a scene continuously in time series and generates a plurality of captured images, and the image reproduction apparatus of the present invention.

  In addition, a facial expression detection unit that detects the facial expression of the main subject is included, and the control unit determines whether the detected facial expression of the main subject is a desired facial expression and displays a partial image according to the determination result. Good.

  In addition, when the detected expression of the main subject is a desired expression, the control unit differs from the color component of the image area of the main subject in the image area of the background portion other than the main subject in the partial image of the main subject. The partial image may be displayed on the second display unit by changing to the color component.

  The facial expression detection unit may calculate the degree to which the facial expression of the main subject is similar to the desired facial expression, and the control unit may display the partial image of the main subject on the second display unit with a size corresponding to the degree. .

  Further, the captured image is an image for composition confirmation, and the control unit may cause the imaging unit to capture the main imaging when the expression of all or a predetermined number or more of the main subjects becomes a desired expression.

  According to the present invention, even if the size of the person relative to the object scene is small, the expression of the person can be detected with high accuracy.

The block diagram which shows the structure of the digital camera of one Embodiment The figure which shows the external appearance at the time of storing the 2nd monitor 17 of the digital camera of one Embodiment in a main body. The figure which shows the external appearance at the time of extending the 2nd monitor 17 of the digital camera of one Embodiment. The flowchart which shows the imaging operation by the digital camera of one Embodiment The figure which shows an example of the object scene of the imaging target in one embodiment The figure which shows the example of a through image display in dual display mode Diagram showing an example of dialog display Diagram showing an example of split display Figure showing another example of split display The flowchart which shows the reproduction | regeneration processing by the digital camera of other embodiment The figure which shows the modification of a split display Diagram showing an example of enlarged display

<< One Embodiment >>
FIG. 1 is a block diagram showing a configuration of a digital camera according to an embodiment of the present invention. FIG. 2 shows the appearance of the digital camera of this embodiment when viewed from (a) the front, (b) viewed from directly above, and (c) viewed from the back.

  The digital camera of this embodiment includes an imaging optical system 11, an imaging device 12, an AFE 13, an image processing unit 14, a memory 15, a first monitor 16, a second monitor 17, a recording I / F 18, a CPU 20, an operation member 24, and a release button. 25, a touch panel 26, and a bus. The image processing unit 14, the memory 15, the first monitor 16, the second monitor 17, the recording I / F 18, and the CPU 20 are respectively connected via a bus so as to be able to transmit information. The operation member 24, the release button 25, and the touch panel 26 are each connected to the CPU 20.

  The imaging optical system 11 includes a plurality of lens groups including a zoom lens and a focusing lens. The lens position of the imaging optical system 11 is adjusted in the optical axis direction by a lens driving unit (not shown). For simplicity, the imaging optical system 11 is illustrated as a single lens in FIG.

  The image pickup device 12 is a device that picks up an image of an object scene formed by a light beam that has passed through the image pickup optical system 11. The output of the image sensor 12 is input to the AFE 13. Note that the image sensor 12 of the present embodiment may be a progressive scan type solid-state image sensor (CCD or the like) or an XY address type solid-state image sensor (CMOS or the like).

  A plurality of light receiving elements are arranged in a matrix on the light receiving surface of the image sensor 12. In each light receiving element of the image sensor 12, red (R), green (G), and blue (B) color filters are arranged according to a known Bayer array. Therefore, each light receiving element of the imaging element 12 outputs an image signal corresponding to each color by color separation in the color filter. Thereby, the image sensor 12 can acquire a color image.

  Here, in the photographing mode of the digital camera, the image pickup device 12 picks up a recorded image (main image) in response to a full pressing operation of the release button 25. Further, the imaging element 12 in the shooting mode captures a composition confirmation image (through image) at a predetermined time interval even during shooting standby. The through image data is output from the image sensor 12 by thinning-out readout. The through image data is used for image display on the first monitor 16 and the second monitor 17 and various arithmetic processes by the CPU 20.

  The AFE 13 is an analog front end circuit that performs analog signal processing on the output of the image sensor 12. The AFE 13 performs correlated double sampling, image signal gain adjustment, and image signal A / D conversion. The output of the AFE 13 is connected to the image processing unit 14. In the present embodiment, the imaging device 12 and the AFE 13 constitute an imaging unit.

  The image processing unit 14 performs various types of image processing (for example, color interpolation processing, gradation conversion processing, contour enhancement processing, white balance adjustment) on the digital image signal for one frame.

  The memory 15 stores image data captured by the digital camera, a control program executed by the CPU 20, and the like. The memory 15 temporarily stores image data in a pre-process and post-process of image processing by the image processing unit 14. The memory 15 can be a non-volatile semiconductor memory or the like.

  The first monitor 16 and the second monitor 17 are monitors such as a liquid crystal monitor, and display various images according to instructions from the CPU 20. In the present embodiment, as shown in FIGS. 2 and 3, the first monitor 16 is installed on the rear surface of the digital camera housing, and the second monitor 17 is installed on the side surface of the housing. It is installed on a hinge part 27 that is rotatable in the axial direction. Thereby, as shown in FIG. 2, when the second monitor 17 is stored in the digital camera body (hereinafter referred to as single display mode), the first monitor 16 is arranged behind the second monitor 17. The second monitor 17 operates as a main monitor. On the other hand, as shown in FIG. 3, when the second monitor 17 is expanded (hereinafter referred to as dual display mode), the main image and the like are divided and displayed on the two monitors, the first monitor 16 and the second monitor 17. In the dual display mode, the first monitor 16 operates as a main monitor.

  In this embodiment, switching between the single display mode and the dual display mode is performed when the CPU 20 detects the open / closed state of the second monitor 17. Accordingly, during imaging standby in the shooting mode, the CPU 20 displays a through image as a moving image on the second monitor 17 in the single display mode and on the first monitor 16 in the dual display mode.

  Further, the CPU 20 superimposes and displays various information necessary for photographing on the through image on the second monitor 17 (single display mode) or the first monitor 16 (dual display mode). Further, the CPU 20 may display a GUI (Graphical User Interface) format button that allows input of various setting items on the second monitor 17 (single display mode) or the first monitor 16 (dual display mode). . However, in the dual display mode, the CPU 20 may display the through image as a moving image on the first monitor 16 and display the above-mentioned various information and GUI buttons on the second monitor 17.

  Further, the size of the display range of the first monitor 16 and the second monitor 17 of the present embodiment in the X-axis direction (horizontal scanning direction) and the Y-axis direction (vertical scanning direction) is as shown in FIG. , Xm pixels and Ym pixels, respectively. Note that the display ranges of the first monitor 16 and the second monitor 17 may be different from each other.

  A connector for connecting the storage medium 19 is formed in the recording I / F 18. The recording I / F 18 writes / reads data to / from the storage medium 19 connected to the connector. The storage medium 19 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is shown as an example of the storage medium 19.

  The CPU 20 is a processor that comprehensively controls each unit of the digital camera. By executing the control program, the CPU 20 performs known AF control by a contrast detection method, automatic exposure (AE) calculation, auto white balance calculation, and the like based on the data of the through image. Further, the CPU 20 performs various arithmetic processes in the shooting mode and also performs a display process on the setting screen. Furthermore, the CPU 20 of this embodiment operates as a face detection unit 21, an image size adjustment unit 22, and a facial expression determination unit 23 by executing the control program.

  As a subject detection unit, the face detection unit 21 performs a known face detection process on image data of a through image, a still image, or a moving image, and detects the face of a person who is a main subject in the object scene. For example, the face detection unit 21 extracts feature points such as end points of eyebrows, eyes, nose, and lips from an image by a known feature point extraction process, and based on these feature points, an image region ( Hereinafter, a face area) is detected. Alternatively, the face detection unit 21 obtains a correlation between face image data or the like prepared in advance and image data to be determined, and determines and detects a human face region when the correlation coefficient exceeds a certain threshold. Also good.

  As will be described later, the image size adjustment unit 22 displays the face image (partial image) of the image area including the face area of the person detected by the face detection unit 21 on the first monitor 16 and the second monitor 17. Adjust the face image size. The image size adjusting unit 22 can also change the size of the face image displayed on the first monitor 16 or the second monitor 17 in accordance with an operation instruction on the operation member 24 or the touch panel 26 by the user.

  The facial expression detection unit 23 applies a known facial expression detection algorithm to the face image of the person whose face has been detected by the face detection unit 21. Is detected and it is determined whether or not a desired facial expression such as a smile is obtained.

  The operation member 24 includes, for example, a command dial, a cross-shaped cursor key, a determination button, and the like (FIGS. 2 and 3). The operation member 24 receives various inputs of the digital camera from the user. For example, the operation member 24 is used for an operation for switching the operation mode of a digital camera, an input operation on a setting screen, and the like.

  The release button 25 receives from the user an instruction input for starting an AF operation before imaging by a half-press operation and an instruction input for starting an imaging operation by a full-press operation.

  The touch panel 26 detects the position of a stylus (or fingertip or the like) in contact with the panel surface, and outputs the detected position information to the CPU 20 to accept an instruction input from the user. The configuration of the touch panel 26 may be either a capacitance type or a pressure sensitive type. The touch panel 26 of the present embodiment is configured by a transparent panel having the same shape as the first monitor 16 and the second monitor 17, and is stacked on the entire surface of the first monitor 16 and the second monitor 17. The user inputs an instruction to the CPU 20 via the touch panel 26 while visually recognizing the GUI button displayed on the first monitor 16 or the second monitor 17.

  Next, an image capturing operation by the digital camera of the present embodiment will be described with reference to the flowchart of FIG. Note that the digital camera of this embodiment has a still image mode and a moving image mode having a single shooting mode and a continuous shooting mode as shooting modes. In the following description, it is assumed that the still image mode of the single shooting mode is preset as the shooting mode. However, the other modes operate in the same manner, and a description thereof is omitted. Further, in the present embodiment, a case will be described in which a scene including four persons 31a to 31d as a main subject as shown in FIG. 5 is captured by a digital camera, but the number of persons other than that (including one person) is also described. This also works in the same way.

  The CPU 20 receives a power-on instruction by turning on the power button on the operation member 24 by the user, and turns on the digital camera. In the present embodiment, as shown in FIG. 3, the second monitor 17 is assumed to be in a dual display mode expanded in advance. However, when the power is turned on, the second monitor 17 is not turned on. Thereby, the power consumption of the digital camera can be suppressed. After power-on, the CPU 20 initializes the digital camera and starts processing from step S101.

  Step S101: The CPU 20 causes the image sensor 12 to start capturing a through image 30 (captured image) of the object scene including the persons 31a to 31d shown in FIG. 5 at a frame rate such as 30 fps or 60 fps, for example. The CPU 20 acquires the through image 30 processed by the AFE 13 and the image processing unit 14 and displays it on the first monitor 16. The face detection unit 21 of the CPU 20 performs a known face detection process on the through image 30 to detect the faces of the persons 31a to 31d in the object scene. For example, as illustrated in FIG. 6, the CPU 20 superimposes and displays face frames corresponding to the face areas of the persons 31 a to 31 d on the first monitor 16. The face frame may not be displayed.

  Step S102: The CPU 20 compares the size of the face area of each of the persons 31a to 31d whose faces are detected in step S101 with the size of the through image 30, and determines the face images of the persons 31a to 31d according to the comparison result. 2 It is determined whether or not to enlarge and display on the monitor 17.

  Specifically, for example, the CPU 20 of the present embodiment obtains the sizes Xfa to Xfd in the X-axis direction of the face regions of the persons 31a to 31d on the first monitor 16, and the size of the first monitor 16 in the X-axis direction. The ratio with Xm is calculated. When the ratio of all the persons 31a to 31d is larger than a predetermined ratio (predetermined size) represented by a value such as 0.2 or 1 / (number of persons + 1), the CPU 20 determines each of the persons 31a to 31d. Is determined to be detectable based on the through image 30, and the process proceeds to step S106 (NO side).

  On the other hand, the CPU 20 determines that it is difficult to detect the facial expressions of the persons 31 a to 31 d based on the through image 30 when any ratio is equal to or less than the predetermined ratio. The CPU 20 enlarges and displays the faces of the persons 31a to 31d on the second monitor 17, and proceeds to step S103 (YES side) in order to detect the facial expressions of the persons 31a to 31d.

  Here, the predetermined ratio is preferably set as appropriate according to the number of persons 31, the size of the through image 30, the first monitor 16, or the second monitor 17.

  If any of the ratios is equal to or smaller than the predetermined ratio, the CPU 20 inquires whether or not to enlarge the faces of the persons 31a to 31d to the user as shown in FIG. 7 (display button). May be displayed on the first monitor 16. When the CPU 20 accepts an enlarged display instruction based on the operation of the touch panel 26 by the user, the CPU 20 proceeds to step S103 (YES side). When the enlarged display instruction is not accepted, the CPU 20 proceeds to step S106 (NO side). You may make it do. However, it is preferable that the dialog 35 is displayed on the first monitor 16 so as not to reach at least each of the persons 31a to 31d. Further, instead of the dialog 35, the digital camera may have a face enlargement display button as the operation member 24.

  Step S103: The CPU 20 turns on the power of the second monitor 17 in order to enlarge and divide and display the faces of the persons 31a to 31d. When the power is turned on, the CPU 20 may cause the first monitor 16 to display a message that “the face of each person is displayed on the second monitor”.

  Step S104: The CPU 20 extracts the image data of the image area including the face areas of the persons 31a to 31d as the face image from the through image 30, and temporarily stores it in the memory 15 or the like. Note that the CPU 20 preferably extracts face images so that the faces of the persons 31a to 31d do not overlap each other.

  Step S105: As shown in FIG. 8, the image size adjustment unit 22 of the CPU 20 divides the face images of the persons 31a to 31d within the display range of the second monitor 17, for example, at the maximum and substantially the same size. The size of each face image is adjusted so that it is displayed. The adjustment by the image size adjustment unit 22 is preferably performed based on the number and size of face images, the size of the display range of the second monitor 17, and the like. The CPU 20 divides and displays the face image in a size determined for each of the persons 31a to 31d by the image size adjustment unit 22 as shown in FIG. In FIG. 8, the face frame is superimposed and displayed on the face images of the persons 31 a to 31 d enlarged and displayed on the second monitor 17, but the face frame may not be superimposed. A part of the upper body and the like may also be displayed together with the faces of the persons 31a to 31d.

  Step S106: The facial expression detection unit 23 of the CPU 20 detects a facial expression by applying a known facial expression detection algorithm to the facial images of the persons 31a to 31d.

  Step S107: The CPU 20 determines whether or not the facial expressions of all the persons 31a to 31d detected in step S106 are desired facial expressions. When all the persons 31a to 31d have a desired facial expression, the CPU 20 proceeds to step S109 (YES side). On the other hand, when all the persons 31a to 31d do not have the desired facial expressions, the CPU 20 proceeds to step S108 (NO side).

  Note that the desired facial expression in this embodiment is a smile. Further, for example, the CPU 20 may change the color of the face frame of the person 31 whose facial expression detected by the facial expression detection unit 23 is a smile, or as shown in FIG. The color of the image area of the background portion other than the face area of a certain person 31 may be changed. Alternatively, the CPU 20 causes the image size adjusting unit 22 to display the face images of the persons 31a to 31d according to the degree of smile obtained by the facial expression detection unit 23 (for example, a value based on the value of the mouth corner or the degree of lowering the corners of the eyes). May be changed and displayed on the second monitor 17 as shown in FIG. Here, FIG. 9B shows an example in which the person 31d with the smallest smile level is enlarged and displayed to the maximum.

  Step S <b> 108: The CPU 20 determines whether or not an instruction to capture the main image by the user fully pressing the release button 25 has been received. When receiving an imaging instruction, the CPU 20 proceeds to step S109 (YES side). On the other hand, when the CPU 20 does not accept the imaging instruction, the CPU 20 proceeds to step S101 (NO side). The CPU 20 acquires the next through image and performs the processes of steps S101 to S107. As a result, even when all of the persons 31a to 31d do not necessarily have a desired facial expression, the main image can be reliably acquired.

  When the CPU 20 accepts a half-press operation of the release button 25 by the user, the AF control, AE calculation, auto white balance calculation, etc. based on the contrast information in the face images of the persons 31a to 31d in the through image 30. May be performed, and the process may proceed to step S101 (NO side).

  Step S109: The CPU 20 images the object scene in the still image mode of the single shooting mode with respect to the image sensor 12. If this imaging operation is based on the determination result in step S107, the operation of the release button 25 by the user is omitted, so that camera shake can be prevented. When the imaging mode is the moving image mode, it is preferable that the CPU 20 performs the same processing as steps S <b> 101 to S <b> 107 on a moving image frame that is a captured image during imaging of the moving image.

  Step S110: The image processing unit 14 performs various types of image processing on the digital image signal output from the AFE 13. The CPU 20 records the processed image data as a still image in the storage medium 19 or the like. The CPU 20 ends a series of processes.

  As described above, in this embodiment, even when the face of the person 31 to be imaged is smaller than the object scene, the CPU 20 enlarges and displays the face on the second monitor 17 to display the facial expression of the person. Can be detected with high accuracy, and an image can be taken with a desired facial expression.

Further, by enlarging and displaying the face images of the persons 31a to 31d on the second monitor 17, it is possible to easily visually recognize whether or not all the faces are in focus.
<< Other embodiments >>
A digital camera according to another embodiment of the present invention is the same as the digital camera according to one embodiment shown in FIG. Therefore, the same reference numerals are given to the components constituting the digital camera of the present embodiment, and detailed description thereof is omitted.

  The difference between the digital camera according to the present embodiment and that according to the first embodiment is the reproduction processing of a moving image captured as the main image. In other words, when a moving image is reproduced, if the captured person is smaller than the object scene, the face image of the person is enlarged and reproduced and displayed on the second monitor 17.

  Thus, a moving image playback process by the digital camera of the present embodiment will be described with reference to the flowchart of FIG. In the present embodiment, a case where a moving image in which an object scene including four persons 31a to 31d is captured as a main subject as shown in FIG. 5 is described. However, the number of persons other than that (including one person) is also described. ) Also works in the same way.

  The CPU 20 receives a power-on instruction by turning on the power button on the operation member 24 by the user, and turns on the digital camera. In the present embodiment, as shown in FIG. 3, the second monitor 17 is assumed to be in a dual display mode expanded in advance. However, when the power is turned on, the second monitor 17 is not turned on. After the power is turned on, the CPU 20 initializes the digital camera, and accepts and sets a moving image playback mode setting instruction according to the operation of the operation member 24 by the user. And CPU20 will start the process from step S201, if the reproduction | regeneration instruction | indication by a user is received via the operation member 24. FIG.

  Step S201: The CPU 20 reads moving image data stored in the storage medium 19 or the like. The CPU 20 displays the first frame 30 of the composition shown in FIG. The face detection unit 21 of the CPU 20 performs a known face detection process on the frame 30 to detect the faces of the persons 31 a to 31 d in the object scene. It is preferable that the CPU 20 does not display a face frame as shown in FIG. 6 during reproduction.

  Step S202: The CPU 20 compares the size of the face area of each of the persons 31a to 31d whose faces are detected in step S201 with the size of the frame 30, as in step S102 of the embodiment, and according to the comparison result. Then, it is determined whether or not the face images of the persons 31a to 31d are enlarged and displayed on the second monitor 17.

  For example, the CPU 20 of the present embodiment obtains the sizes Xfa to Xfd in the X-axis direction of the face areas of the persons 31a to 31d on the first monitor 16, and the ratio to the size Xm of the first monitor 16 in the X-axis direction. Is calculated. When the ratio of all the persons 31a to 31d is larger than a predetermined ratio represented by a value such as 0.2 or 1 / (number of persons + 1), for example, the CPU 20 displays the expressions of the persons 31a to 31d in the frame 30. Based on this, it is determined that detection is possible, and the process proceeds to step S206 (NO side).

  On the other hand, the CPU 20 determines that it is difficult to detect the facial expressions of the persons 31 a to 31 d based on the frame 30 when any ratio is equal to or less than the predetermined ratio. The CPU 20 enlarges and displays the faces of the persons 31a to 31d on the second monitor 17, and proceeds to step S203 (YES side) to visually recognize the facial expressions of the persons 31a to 31d.

  If any of the ratios is equal to or less than the predetermined ratio, the CPU 20 displays a dialog 35 for inquiring whether or not to enlarge the faces of the persons 31a to 31d to the user as shown in FIG. 16 may be displayed. When receiving an enlarged display instruction based on the operation of the touch panel 26 by the user, the CPU 20 proceeds to step S203 (YES side). When not received an enlarged display instruction, the CPU 20 proceeds to step S206 (NO side). You may make it do. However, it is preferable that the dialog 35 is displayed on the first monitor 16 so as not to reach at least each of the persons 31a to 31d. Further, instead of the dialog 35, the digital camera may have a face enlargement display button as the operation member 24.

  Step S203: The CPU 20 turns on the power of the second monitor 17 in order to enlarge and divide and display the faces of the persons 31a to 31d.

  Step S204: The CPU 20 extracts the face images of the persons 31a to 31d from the frame 30 and temporarily stores them in the memory 15 or the like, as in step S104 of one embodiment.

  Step S205: The image size adjustment unit 22 of the CPU 20 determines that the face images of the persons 31a to 31d are, for example, the largest and substantially the same size within the display range of the second monitor 17, as in step S105 of the embodiment. The size of each face image is adjusted so that it is divided and displayed. The CPU 20 divides and displays the face image in a size determined for each of the persons 31a to 31d by the image size adjustment unit 22 as shown in FIG. In the present embodiment, it is preferable not to superimpose the face frame on the face images of the persons 31a to 31d enlarged and displayed on the second monitor 17.

  The CPU 20 may detect a facial expression by applying a known facial expression detection algorithm to the facial expression of each of the persons 31 a to 31 d in the facial expression detection unit 23. In this case, for example, the CPU 20 superimposes and displays a colored face frame corresponding to the facial region of the person 31 whose facial expression detected by the facial expression detection unit 23 is a smile, as shown in FIG. In addition, it is preferable to change the color of the background image area other than the face area of the person 31 who is smiling. Alternatively, the CPU 20 causes the image size adjustment unit 22 to change the size of the face images of the persons 31a to 31d in accordance with the degree of smile obtained by the facial expression detection unit 23, as shown in FIG. 9B. 2 may be displayed on the monitor 17.

  Step S <b> 206: The CPU 20 determines whether or not a moving image reproduction end instruction from the user has been received via the operation member 24. CPU20 complete | finishes a series of reproduction | regeneration processes, when the reproduction | regeneration termination instruction | indication is received (YES side). On the other hand, when the CPU 20 has not received the reproduction end instruction, the CPU 20 proceeds to step S201 (NO side). CPU20 acquires the following flame | frame and performs the process of step S201-step S205.

As described above, in the present embodiment, even when the captured face of the person 31 is smaller than the object scene, the CPU 20 enlarges and displays the face of the person on the second monitor 17 in a divided manner. It can be detected with high accuracy.
<< Additional items of embodiment >>
(1) In the above embodiment, an example in which the CPU 20 implements each process of the face detection unit 21, the image size adjustment unit 22, and the expression detection unit 23 by software has been described. However, each of these processes is performed using an ASIC. May be realized in hardware.

  (2) The image reproducing device of the present invention is not limited to the example of the digital camera of the above embodiment. For example, the computer may function as the image reproducing apparatus of the present invention by causing the computer to read an image and causing the computer to execute an image reproduction processing program.

  (3) In the above embodiment, the power supply of the second monitor 17 is not turned on at the start of imaging and the start of reproduction. However, the present invention is not limited to this, and the power supply may be turned on.

  (4) In the above embodiment, a person is used as the main subject. However, the present invention is not limited to this, and pets such as dogs and cats may be used as the main subject.

  (5) In the above embodiment, in imaging and reproduction in the dual display mode, a through image or a moving image obtained by imaging the object scene is displayed on the first monitor 16 of the main monitor, and persons 31a to 31d are displayed on the second monitor 17 of the sub monitor. However, the present invention is not limited to this. For example, in the dual display mode, the second monitor 17 may be operated as a main monitor and the first monitor 16 may be operated as a sub monitor. That is, a through image or a moving image in which the scene is captured may be displayed on the second monitor 17, and the face images of the persons 31 a to 31 d may be enlarged and displayed on the first monitor 16. In this case, it is preferable that the power source of the first monitor 16 is not turned on until the face image of the person 31 is enlarged and divided and displayed.

  (6) In the above embodiment, the face image extracted from the through image or the frame is enlarged and displayed as it is on the second monitor 17, but the present invention is not limited to this. For example, as shown in FIG. 11, for example, when a person 31a to 31d is imaged against a brown building wall or the like, and the person 31c is tanned, the person 31c is detected by the second monitor 17. However, since the color of the face of the person 31c and the background are similar, it is difficult to visually recognize the face of the person 31c and its expression.

  In order to avoid this, in step S105 or step S205, the CPU 20 extracts color components (RGB, YCrCb, etc.) between the face area and the background image area in the face image of the person 31c. The CPU 20 determines whether the colors are similar to each other based on the color components of the extracted face area and the background image area. When the colors are similar, the CPU 20 changes the color component of the background portion to a color component different from the color component of the face region of the person 31c, for example, a color component such as an opposite color, red or pink, as shown in FIG. The face image of the person 31c is enlarged and displayed on the second monitor 17. Thereby, even if a person's face and a background have the same color, a person's face and expression can be easily visually recognized.

  (7) In the above embodiment, the face images of the persons 31a to 31d are divided and displayed in the same size on the second monitor 17 or displayed in a size corresponding to the degree of smile. It is not limited to this. For example, data (for example, feature points such as eyebrow, eye, nose, and lip end points extracted by a known feature point extraction process) indicating characteristics of the faces of the persons 31a to 31d in advance in the memory 15 (storage unit), The subject information (reference data) such as the priority order is stored, and the CPU 20 refers to the priority order of the subject information, and causes the image size adjustment unit 22 to size the face images of the persons 31a to 31d according to the priority order. And may be displayed on the second monitor 17.

  In addition, the CPU 20 may cause the image size adjustment unit 22 to change the size of the face images of the persons 31 a to 31 d and display the divided images on the second monitor 17 in accordance with the operation of the touch panel 26 by the user. For example, as shown in FIG. 12A, the user moves his / her finger as indicated by an arrow 40 on the touch panel 26 on the face image of the person 31a. Based on the control signal from the touch panel 26 according to the operation, the CPU 20 enlarges the face image of the person 31a in the image size adjusting unit 22 and the face images of the persons 31b to 31d as shown in FIG. Is changed so as to be smaller and displayed on the second monitor 17.

  (8) In the above embodiment, the faces of the persons 31a to 31d are not overlapped with each other. However, the present invention is not limited to this, and can also be applied to the case where the faces of the persons overlap. In addition, when the face detection unit 20 detects that the faces of the person 31 are overlapping, the CPU 20 displays on the first monitor 16 and the second monitor 17 “the faces are overlapping. Please line up so as not to overlap. Or the like may be displayed to instruct the user.

  (9) In the above embodiment, in step S107, the CPU 20 determines whether or not all of the persons 31a to 31d have a desired facial expression, but the present invention is not limited to this, and a predetermined number of people That's all. The predetermined number of persons is preferably determined as appropriate according to the number of persons 31 and the like.

  (10) In the other embodiments described above, the face detection unit 21 detects the persons 31a to 31d that are main subjects. However, the present invention is not limited to this, and the captured persons 31a to 31d and areas corresponding thereto May be added to the image data as metadata, and the CPU 20 may read and acquire the metadata of the information and use it instead of the detection result by the face detection unit 21.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

DESCRIPTION OF SYMBOLS 11 ... Imaging optical system, 12 ... Imaging element, 13 ... AFE, 14 ... Image processing part, 15 ... Memory, 16 ... 1st monitor, 17 ... 2nd monitor, 18 ... Recording I / F, 19 ... Storage medium, 20 ... CPU, 21 ... Face detection unit, 22 ... Image size adjustment unit, 23 ... Expression detection unit, 24 ... Operation member, 25 ... Release button, 26 ... Touch panel, 27 ... Hinge unit

Claims (13)

A first display section and a second display section;
A subject detection unit for detecting a main subject from each of a plurality of captured images obtained by capturing a scene continuously in time series;
A control unit that extracts a partial image including an image area of the main subject detected from the captured image, and sequentially displays the captured image on the first display unit and the partial image on the second display unit;
An image reproducing apparatus comprising: The image reproduction apparatus according to claim 1,
The control unit displays the partial image on the second display unit when a size of the main subject in the captured image is a predetermined size or less. The image reproduction apparatus according to claim 1 or 2,
The first display unit further includes a first touch panel that receives an operation from a user,
When the size of the main subject is equal to or less than the predetermined size, the control unit displays a display button for displaying the partial image on the second display unit on the first display unit, and the user performs the first button 1. The image reproduction device according to claim 1, wherein the partial image is displayed on the second display unit in response to the operation of the display button on the touch panel. The image reproduction apparatus according to any one of claims 1 to 3,
The image reproduction apparatus according to claim 1, wherein the control unit turns on the power of the second display unit when the partial image is displayed. The image reproduction apparatus according to any one of claims 1 to 4,
When the subject detection unit detects a plurality of main subjects, the control unit extracts the partial images for each main subject and displays the partial images of the plurality of main subjects on the second display unit. An image reproducing apparatus characterized by the above. The image reproduction apparatus according to claim 5, wherein
A storage unit for storing reference data indicating the priority order of the plurality of main subjects;
The control unit displays the partial images of the plurality of main subjects on the second display unit in a size corresponding to the priority order based on the reference data. The image reproduction apparatus according to any one of claims 1 to 6,
The second display unit further includes a second touch panel that receives an operation from the user,
The control unit changes the size of the partial image displayed on the second display unit according to an operation of the second touch panel by the user. The image reproduction device according to any one of claims 1 to 7,
The control unit determines whether or not the color components of the image areas of the main subject and the background portion other than the main subject in the partial image are similar, and when the color components are similar, the color of the background portion An image reproducing apparatus, wherein a component is changed to a color component different from a color component of the main subject and the partial image is displayed on the second display unit. An imaging unit that continuously images in time series and generates a plurality of captured images;
An image reproduction device according to any one of claims 1 to 8,
An imaging apparatus comprising: The imaging device according to claim 9,
An expression detection unit for detecting the expression of the main subject;
The control unit determines whether or not the detected facial expression of the main subject is a desired facial expression, and displays the partial image according to the determination result. The imaging device according to claim 10.
When the detected expression of the main subject is a desired expression, the control unit converts the color component of the image area of the background portion other than the main subject of the partial image of the main subject to the color of the image area of the main subject. An image pickup apparatus, wherein the partial image is displayed on the second display unit by changing to a color component different from the component. The imaging device according to claim 10.
The facial expression detection unit calculates a degree of similarity of the facial expression of the main subject to a desired facial expression;
The control unit displays the partial image of the main subject on the second display unit with a size corresponding to the degree. The imaging apparatus according to any one of claims 10 to 12,
The captured image is a composition confirmation image,
The control unit causes the imaging unit to capture a main image when all or a predetermined number or more of the main subject's facial expressions become the desired facial expression.

Images