JP2013012982A - Imaging apparatus and image reproduction apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow for detailed check of the faces of plural persons on a display without changing a photographing composition and a photographing field angle which are desired by a user.SOLUTION: While the taken image of first to sixth persons is displayed as a through image on a display, if a user desires detailed check of the states of the faces of the second to fifth persons, the user touches the faces of the second to fifth persons on a display screen with a finger. An imaging apparatus extracts from the taken image the face images of the second to fifth persons, expands the sizes of the respective extracted face images, and displays the expanded extracted face images (FACE' and FACE') with the taken image on the display.

Description

  The present invention relates to an imaging apparatus such as a digital camera and an image reproduction apparatus that reproduces an image.

  Many of recent imaging apparatuses include a display unit, and an image in a shooting area is displayed on the display unit as a moving image. A user as a photographer can check the photographing composition and the state of the subject group by looking at the display screen of the display unit.

FIG. 14 shows a state of the display screen in a state where the persons HM _{1 to} HM ₆ are stored in the imaging region. The hatched area in FIG. 14 represents the housing of the display unit. For example, a user of a photographer, wish detailed confirmation of the state of the face of the person HM ₂ and HM _5, it is necessary to perform a zoom-in operation while adjusting as necessary the shooting direction.

  In the interphone system, when a face of a person is detected from a captured image of a slave unit, a method of enlarging and displaying the face area on the master unit is disclosed (see Patent Document 1 below).

JP 2011-35644 A

As described above, when the user as a photographer wants to confirm the face states of the persons HM ₂ and HM ₅ in detail, if the photographing direction is changed or the photographing angle of view is changed by a zoom-in operation, the user's desired photographing composition is obtained. In addition, the shooting angle of view cannot be maintained. For example, if the shooting angle of view decreases with a zoom-in operation, one of the persons (for example, HM ₁ and HM ₆ ) is framed against the user's intention to place the persons HM ₁ to HM ₆ in the shooting area. May be out.

Such an event can also occur in the image reproducing apparatus. That is, when the input image including the image signals of the persons HM _{1 to} HM ₆ as shown in FIG. 14 is reproduced, the user as the viewer may want to confirm the face states of the persons HM ₂ and HM ₅ in detail. . If the zoom operation and the scroll operation are performed, only the person HM ₂ or HM ₅ can be enlarged and reproduced. However, the execution of the zoom operation and the scroll operation can be a burden on the user, and if the zoom operation or the scroll operation is performed, the person HM The entire input image including _{1 to} HM ₆ cannot be seen.

  Therefore, the present invention requires an imaging device that enables detailed state confirmation of a plurality of target objects without requiring a change in shooting composition and shooting angle of view, and a zoom operation and scroll operation during playback. An object of the present invention is to provide an image reproduction device that enables detailed state confirmation of a plurality of target objects.

  An imaging apparatus according to the present invention includes an imaging unit that outputs an image signal representing a captured image of a subject group, a display unit that displays an image based on the image signal, and images of a plurality of target objects included in the subject group. An extraction unit that extracts each target object as a target object image from the photographed image, and enlarges the image size of each of the extracted target object images, and displays each enlarged target object image together with the photographed image in the display unit And a display control unit for displaying the display.

  Thereby, the user can confirm the states of a plurality of target objects in detail on the display unit at the same time without requiring the change of the shooting composition and the shooting angle of view.

  Specifically, for example, a target object setting unit that sets the plurality of target objects based on an operation performed on the user interface may be further provided in the imaging apparatus.

  Alternatively, for example, each target object is a specific type of object, and a target object setting unit that sets the plurality of target objects by detecting the specific type of object from the captured image based on the image signal includes the imaging device. May be further provided.

  Further, for example, a thumbnail generation unit that generates a thumbnail image of the captured image using a plurality of target object images extracted from the captured image may be further provided in the imaging apparatus.

  The enlarged target object image is presumed to be an image portion having a high degree of attention of the user as a photographer. By generating a thumbnail image using an image portion having a high degree of attention of the user, it becomes easier for the user to imagine the situation at the time of shooting from the thumbnail image during reproduction, and as a result, it becomes easier to find a desired image.

  For example, in the imaging apparatus, the target object may include a human face.

  An image reproduction apparatus according to the present invention, in an image reproduction apparatus that receives an image signal of an input image containing images of a plurality of target objects, a display unit that displays an image based on the image signals, and images of the plurality of target objects For each target object as an object image from the input image, the image size of each extracted target object image is enlarged, and each of the enlarged target object images is displayed together with the input image. And a display control unit to be displayed on the unit.

  Accordingly, the user can confirm the states of a plurality of target objects in detail on the display unit at the same time without requiring a zoom operation or a scroll operation.

  Specifically, for example, a target object setting unit that sets the plurality of target objects based on an operation performed on the user interface may be further provided in the image reproduction apparatus.

  Alternatively, for example, each target object is a specific type of object, and a target object setting unit that sets the plurality of target objects by detecting the specific type of object from the input image based on the image signal You may provide further in an apparatus.

  For example, in the image reproduction apparatus, the target object may include a human face.

  According to the present invention, an imaging device that enables detailed state confirmation of a plurality of target objects without needing to change a shooting composition and a shooting angle of view, and a zoom operation and a scroll operation during reproduction are required. It is possible to provide an image reproduction device that enables detailed state confirmation of a plurality of target objects without any problems.

1 is a schematic overall block diagram of an imaging apparatus according to an embodiment of the present invention. It is an internal block diagram of the imaging part of FIG. It is a block diagram of the site | part especially related to operation | movement of 1st Example of this invention. It is a figure which shows the frame image which concerns on 1st Example of this invention. It is a figure which concerns on 1st Example of this invention and shows the mode of a display part, and the mode of touch panel operation. FIG. 6 is a diagram illustrating a display image and a display unit on which the display image is shown according to the first embodiment of the present invention. It is a block diagram of the site | part especially related to operation | movement of 2nd Example of this invention. It is a figure which shows the frame image which concerns on 2nd Example of this invention. It is a figure which shows the display part which concerns on 2nd Example of this invention and the display image and the display image were shown. It is a figure which shows the thumbnail production | generation part which concerns on 3rd Example of this invention. It is a figure which shows the thumbnail image which concerns on 3rd Example of this invention. It is a figure which shows the mode of the display part in thumbnail display mode. 1 is a schematic overall block diagram of an image reproduction apparatus according to an embodiment of the present invention. It is a figure which concerns on a prior art and shows the mode of the display part of an imaging device.

  Hereinafter, an example of an embodiment of the present invention will be specifically described with reference to the drawings. In each of the drawings to be referred to, the same part is denoted by the same reference numeral, and redundant description regarding the same part is omitted in principle. In this specification, for simplification of description, a symbol or reference that refers to information, signal, physical quantity, state quantity, member, or the like is written to indicate information, signal, physical quantity, state quantity or Names of members and the like may be omitted or abbreviated.

  FIG. 1 is a schematic overall block diagram of an imaging apparatus 1 according to an embodiment of the present invention. The imaging device 1 is a digital video camera that can capture and record still images and moving images. However, the imaging apparatus 1 may be a digital still camera that can capture and record only still images. The imaging device 1 may be mounted on a mobile terminal such as a mobile phone.

  The imaging device 1 includes an imaging unit 11, an AFE (Analog Front End) 12, a main control unit 13, an internal memory 14, a display unit 15, a recording medium 16, and an operation unit 17. The display unit 15 may be interpreted as being provided in an external device (not shown) of the imaging device 1.

  The imaging unit 11 captures a subject using an imaging element. FIG. 2 is an internal configuration diagram of the imaging unit 11. The imaging unit 11 includes an optical system 35, a diaphragm 32, an imaging device (solid-state imaging device) 33 including a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and the optical system 35 and the diaphragm 32. And a driver 34 for drive control. The optical system 35 is formed of a plurality of lenses including a zoom lens 30 for adjusting the angle of view of the imaging unit 11 and a focus lens 31 for focusing. The zoom lens 30 and the focus lens 31 are movable in the optical axis direction. The optical axis refers to the optical axis in the imaging unit 11 (the optical axis in the imaging device 1). Based on the control signal from the main control unit 13, the positions of the zoom lens 30 and the focus lens 31 in the optical system 35 and the opening of the diaphragm 32 (that is, the diaphragm value) are controlled.

  The image sensor 33 is formed by arranging a plurality of light receiving pixels in the horizontal and vertical directions. Each light receiving pixel of the image sensor 33 photoelectrically converts an optical image of a subject incident through the optical system 35 and the diaphragm 32, and outputs an electric signal obtained by the photoelectric conversion to an AFE 12 (Analog Front End).

  The AFE 12 amplifies the analog signal output from the imaging unit 11 (imaging device 33), converts the amplified analog signal into a digital signal, and outputs the digital signal to the main control unit 13. The amplification degree of signal amplification in the AFE 12 is controlled by the main control unit 13. The main control unit 13 performs necessary image processing on the image represented by the output signal of the AFE 12, and generates a video signal for the image after image processing. An image represented by the output signal itself of the AFE 12 or an image obtained by performing predetermined image processing on the image represented by the output signal itself of the AFE 12 is referred to as a captured image. The main control unit 13 includes a display control unit that controls the display content of the display unit 15, and performs control necessary for display on the display unit 15.

  The internal memory 14 is formed by SDRAM (Synchronous Dynamic Random Access Memory) or the like, and temporarily stores various data generated in the imaging device 1. The display unit 15 is a display device having a display screen such as a liquid crystal display panel, and displays a captured image, an image recorded on the recording medium 16, and the like under the control of the main control unit 13. In this specification, when it is simply referred to as a display and a display screen, they refer to a display and a display screen in the display unit 15. A touch panel 19 is provided on the display unit 15, and the user can give various instructions to the imaging apparatus 1 by touching the display screen of the display unit 15 with an operating tool (such as a finger or a touch pen).

  The recording medium 16 is a non-volatile memory such as a card-like semiconductor memory or a magnetic disk, and records a video signal or the like of a captured image under the control of the main control unit 13. The operation unit 17 includes a shutter button 20 that receives a still image shooting instruction, a recording button 21 that receives a moving image shooting start instruction or a shooting end instruction, and the like, and receives various operations from the outside. The content of the operation on the operation unit 17 is transmitted to the main control unit 13. The operation unit 17 and the touch panel 19 can be referred to as a user interface that receives an arbitrary instruction and operation from the user. Hereinafter, the operation unit 17 or the touch panel 19 or these are collectively referred to as a user interface. The shutter button 20 and the recording button 21 may be buttons on the touch panel 19.

  An operation by touching the display screen with an operating body is called a touch panel operation. The contact position between the operating tool and the display screen is called a touch position. When the operating body is touching the display screen, the touch panel 19 outputs touch panel operation information including touch position information indicating the touched position (ie, touch position) to the main control unit 13 in real time. The operating body is, for example, a user's finger or a pen. In the following, it is assumed that the operating tool is a user's finger. In order to distinguish an operation on a button or the like provided on the operation unit 17 from a touch panel operation, it is particularly referred to as a button operation.

  The operation mode of the imaging apparatus 1 includes a shooting mode in which an image (still image or moving image) can be shot and recorded, and an image (still image or moving image) recorded on the recording medium 16 is reproduced and displayed on the display unit 15. Playback mode to be included. Transition between the modes is performed according to the operation on the operation unit 17.

  In the shooting mode, a subject is periodically shot at a predetermined frame period, and shot images of the subject are sequentially acquired. A video signal representing an image is also called an image signal. Each of the output signal of the image sensor 33, the output signal of the AFE 12, and the video signal (luminance signal and color difference signal) generated by the main control unit 13 is a kind of image signal. In the following description, it is assumed that a plurality of persons are included as a plurality of subjects in the imaging region (field of view) of the imaging apparatus 1, and the plurality of subjects are also referred to as a subject group.

  As examples for explaining a more specific configuration and operation of the imaging apparatus 1 described above, first to sixth examples are shown below. As long as there is no contradiction, the matters described in one embodiment among the first to sixth embodiments can be applied to other embodiments.

<< First Example >>
A first embodiment will be described. In the first embodiment and the second embodiment to be described later, the characteristic operation of the imaging apparatus 1 in the shooting mode will be described. FIG. 3 is a block diagram of a portion particularly related to the operation of the first embodiment. The display control unit 51 and the face processing unit 52 can be provided in the main control unit 13 of FIG. The face processing unit 52 includes a face detection unit 53, a face image extraction unit 54, and a target face setting unit 55.

In the shooting mode, image signals representing captured images of the subject group are sequentially output from the imaging unit 11 at a predetermined frame period, and the output image signals of the imaging unit 11 are sent to the display control unit 51 and the face processing unit 52 via the AFE 12. Given. One photographed image (still image) expressed by an image signal for one frame period is particularly called a frame image. The display control unit 51 causes the display unit 15 to sequentially update and display frame images obtained sequentially in the frame period. The image FI in FIG. 4A is an example of one frame image. Here, it is assumed that six persons HM _{1 to} HM ₆ are included in the subject group at an arbitrary timing including the shooting time of the frame image FI.

The face detection unit 53 detects a human face, which is a specific type of object, from the subject group on the frame image FI based on the image signal of the frame image FI. Therefore, the face detection unit 53 detects the faces of the persons HM _{1 to} HM ₆ on the frame image FI. The face detection unit 53 can use a known face detection method based on an image signal.

The face image extraction unit 54 can extract an image of an arbitrary face on the frame image FI as a face image from the frame image FI using the face detection result by the face detection unit 53. As shown in FIG. 4 (b), an image of the face of the person _HM 1 _{~HM 6,} referred to as respectively the face image FACE ₁ ~FACE _6. Each of the face images FACE _{1 to} FACE ₆ is a part of the frame image FI, and the face image FACE _i is an image including an image signal of the face of the person HM _i (i is an integer). In FIG. 4B, the images within the six broken-line rectangular frames are the face images FACE _{1 to} FACE ₆ . In the example shown in FIG. 4 (b), although the outer shape of the face image FACE _i is in a rectangular outer shape of the face image FACE _i may not be rectangular, the contour along the contour of the face of the person HM _i The face image FACE _i may be provided.

The face processing unit 52 outputs face position / size information indicating the position and size of the face images FACE _{1 to} FACE ₆ on the frame image FI to the display control unit 51, and the display control unit 51 displays the face position / size. A face frame based on the height information is displayed on the display unit 15. That is, when the display control unit 51 displays the frame image FI on the display unit 15, as shown in FIG. 5A, _six face frames corresponding to the outer peripheries of the face images FACE _{1 to} FACE ₆ (FIG. 5A ) Are displayed on the display unit 15 by superimposing the FF _{1 to} FF ₆ on the frame image FI. In FIG. 5A, the hatched area represents the housing of the display unit 15 (the same applies to FIGS. 5B and 6B described later).

The main control unit 13 performs AF control for focusing on all or part of the faces of the persons HM _{1 to} HM ₆ based on the image signals of all or part of the face images FACE _{1 to} FACE ₆ , and the person HM. _{1 ~HM} AE control to optimize the exposure of all or part of the face _6, and, like AWB control all or to optimize some of the white balance of a person's face _HM 1 ~HM ₆ Can be made.

The target face setting unit 55 sets _one of the faces of the persons HM _{1 to} HM ₆ as the target face and sets the face image of the target face as the target face image based on the touch panel operation information. The target face setting unit may be referred to as a target face selection unit, and the setting of the target face or the target face image can be said to be the selection of the target face or the target face image (the same applies to other examples described later). For example, as shown in FIG. 5B, in the state where the frame image FI including the face images FACE _{1 to} FACE ₆ is displayed on the display screen, the display portions of the face frames FF ₂ and FF ₅ on the display screen are displayed by the user. Is touched with a finger, touch positions corresponding to the display positions of the face frames FF ₂ and FF ₅ are included in the touch panel operation information, and the target face setting unit 55 selects the face frames FF ₂ and FF ₂ based on the touch panel operation information. the face of the person HM ₂ and HM ₅ corresponding to FF ₅ is set to the target face (in other words, selected as target face) face image fACE ₂ and fACE ₅ corresponding to the face frame FF ₂ and FF ₅ with target Set to face image.

The operation of touching the face frame FF ₂ on the display screen with a finger and the operation of touching the face frame FF ₅ on the display screen with a finger may be performed simultaneously or at different timings. good. The user can also specify the face frames FF ₂ and FF ₅ by button operation. Even when the face frames FF ₂ and FF ₅ are specified by button operation, the target face setting unit 55 causes the person HM ₂ and FF ₂ and FF _{5 to} be specified. The face of HM ₅ is set as the target face, and the face images FACE ₂ and FACE ₅ are set as the target face images. Further, on the display screen, even if any part of the person HM ₂ (e.g. body portion of the person HM ₂₎ is touched by a finger, or any part of the person HM ₂ by a button operation (e.g. a person HM even if the body portion ₂₎ is specified, the face of the person HM ₂ is and facial image fACE ₂ is set to the target face may be set to the target face image. The same applies to other persons. The main control unit 13 refers to the position of the face of the person HM ₂ detected by the face detection unit 53, and uses a known method using contour extraction, area division, edge extraction, or the like, to detect the body of the person HM ₂ . The position, size, and shape of the area or the whole image can be detected, and whether or not a part of the person HM ₂ is touched with a finger or a part of the person HM ₂ is designated by a button operation using the detection result. (Same for other persons).

  When the target face image is set, the image signal of the target face image is sent from the face image extraction unit 54 to the display control unit 51. Upon receiving the image signal of the target face image, the display control unit 51 enlarges the image size of the target face image by performing image size enlargement processing on the target face image, and displays the enlarged target face image together with the frame image FI. Display on the unit 15. The target face image before the image size enlargement by the image size enlargement process is particularly called an original target face image, and the target face image after the image size enlargement by the image size enlargement process is called an enlargement target face image.

As an example, let us consider a case where the faces of the persons HM ₂ and HM ₅ are set as target faces. In this case, the display control unit 51 generates enlarged target face images FACE ₂ ′ and FACE ₅ ′ by performing an image size enlargement process on the original target face images FACE ₂ and FACE ₅ . Then, the display control unit 51 generates the display image FI ′ (see FIG. 6A) by superimposing the enlargement target face images FACE ₂ ′ and FACE ₅ ′ on the frame image FI, and displays the display image FI. 'Is displayed on the display unit 15. The face frame of each face may be superimposed on the display image FI ′. FIG. 6B shows a state of the display screen on which the display image FI ′ is displayed.

At this time, for example, the display control unit 51 displays the target face images FACE ₂ ′ and FACE ₅ ′ at the positions where the faces of the persons HM ₂ and HM ₅ exist on the frame image FI. Based on the position information, the display positions of the enlargement target face images FACE ₂ ′ and FACE ₅ ′ can be determined. Target face position information representing the position of each target face image on the frame image FI is sent from the face processing unit 52 to the display control unit 51. In the image size enlargement process, the display control unit 51 increases the number of pixels of the original target face image FACE _{2 in} the horizontal and vertical directions by k ₂ times, for example, by known resolution conversion, thereby expanding the target face image FACE ₂ ′. And the enlargement target face image FACE ₅ ′ is generated by multiplying the number of pixels of the original target face image FACE ₅ by k _{5 in} each of the horizontal and vertical directions. k ₂ and k ₅ are both coefficients larger than 1. The display control unit 51 may make the values of the coefficients k ₂ and k ₅ coincide with each other or may differ from each other. A predetermined value larger than 1 may be substituted for the coefficients k ₂ and k ₅ . Alternatively, for example, the display control unit 51 may determine the values of the coefficients k ₂ and k ₅ so that the number of pixels in the horizontal or vertical direction of the enlargement target face images FACE ₂ ′ and FACE ₅ ′ matches a predetermined value. .

As described above, frame images are sequentially acquired at a predetermined frame period. Therefore, each operation described above is performed on a frame image sequence that is a collection of frame images arranged in time series. The i-th frame image FI is particularly represented by the symbol FI [i]. If the user designates the face frames FF ₂ and FF ₅ when the frame image FI [i] is displayed on the display unit 15, the display control unit 51 displays the original extracted from the frame image FI [i + j]. to generate an enlarged target face image fACE ₂ 'and fACE _5' by performing image size enlargement process on the target face image fACE ₂ and fACE _5, an enlarged target face image fACE ₂ 'and fACE _5' frame image FI [i + j] Can be displayed on the display unit 15 (j is an integer of 1 or more).

The face processing unit 52 can use a known tracking process for tracking the position of each person's face on the frame image sequence based on the image signal of the frame image sequence. Can be discriminated as to which face of the persons HM _{1 to} HM ₆ , and face detection and face image extraction can be continuously executed in response to the position change of each face on the frame image sequence. .

  According to the first embodiment, it is possible to simultaneously enlarge and display a plurality of faces to which the user pays attention without changing the user's desired composition and shooting angle of view, and the user pays attention within a limited screen size. It becomes possible to confirm the state of a plurality of faces to be confirmed in detail on the display screen at the same time.

<< Second Example >>
A second embodiment will be described. In the first embodiment, the target face and the target face image are set according to the operation content by the user. However, in the second embodiment, the imaging device 1 automatically performs the target face and the target face regardless of the user operation. Set the image. FIG. 7 is a block diagram of a portion particularly related to the operation of the second embodiment. In the second embodiment, the target face setting unit 55 is deleted from the face processing unit 52.

An image 330 in FIG. 8 is an example of the frame image FI. Here, at an arbitrary timing including the shooting time of the frame image FI, three persons HM ₂ , HM ₃ and HM ₅ are included in the subject group, and persons other than the persons HM ₂ , HM ₃ and HM ₅ are It is assumed that it is not included in the subject group. Then, as described in the first embodiment, the faces of the persons HM ₂ , HM ₃ and HM ₅ are detected from the frame image 330 by the face detection unit 53, and the face image FACE ₂ from the frame image 330 by the face image extraction unit 54. , FACE ₃ and FACE ₅ are extracted.

  The face detection unit 53 according to the second embodiment includes the function of the target face setting unit, and sets the face detected by itself as the target face. At this time, the face detection unit 53 may unconditionally set all the faces detected by itself as target faces.

Alternatively, the face detection unit 53 may set the target face only for faces whose face size on the frame image 330 is equal to or smaller than a predetermined reference size. For example, when the number of pixels of the face image FACE ₃ on the frame image 330 exceeds a predetermined reference value, the face of the person HM ₃ is not set as the target face, and only when the number of pixels is equal to or less than the reference value The face of HM ₃ may be set as the target face (the same applies to the persons HM ₂ and HM ₅ ).

  Alternatively, a plurality of persons are set in advance in the face detection unit 53 as a plurality of registered persons, and a face image of each registered person is set in the face detection unit 53 as a registered face image. Of the person's faces, only the face of the registered person may be set as the target face. In this case, the face detection unit 53 determines which person corresponds to each face on the frame image 330 based on a known face recognition process using similarity evaluation between the registered face image and the face image on the frame image 330. It can be determined whether or not the registered person matches.

Now, it is assumed that only the faces of the persons HM ₂ and HM ₅ are set as the target faces by the face detection unit 53. Then, the face detection unit 53 sets the face images FACE ₂ and FACE ₅ as target face images. The operation of the imaging apparatus 1 after setting the target face image is the same as that described in the first embodiment, and the matters described in the first embodiment are also applied to the second embodiment.

That is, the display control unit 51 performs the image size enlargement process on the original target face images FACE ₂ and FACE ₅ extracted from the frame image 330 to generate the enlargement target face images FACE ₂ ′ and FACE ₅ ′, and the enlargement target The display image 330 ′ (see FIG. 9A) is generated by superimposing the face images FACE ₂ ′ and FACE ₅ ′ on the frame image 330. Then, the display control unit 51 displays the display image 330 ′ on the display unit 15 (see FIG. 9B). A face frame of each face may be superimposed on the display image 330 ′.

  According to the second embodiment, the same effect as that of the first embodiment can be obtained. Furthermore, in the second embodiment, the user's convenience can be improved because the operation of designating the target face becomes unnecessary.

<< Third Example >>
A third embodiment will be described. In the third embodiment, a thumbnail generation technique applicable to the first or second embodiment will be described.

  When the user presses the shutter button 20 in the shooting mode, the main control unit 13 causes the recording medium 16 to record a frame image obtained immediately after that as a target still image. At this time, the thumbnail generation unit 61 in the main control unit 13 (see FIG. 10) generates a thumbnail image corresponding to the target still image. Then, the main control unit 13 associates the thumbnail image with the target still image and records the target still image on the recording medium 16 together with the thumbnail image. For example, the main control unit 13 creates one image file in the recording medium 16, and stores the image signal of the target still image and the image signal of the thumbnail image in the main body area and the header area of the image file, respectively. A thumbnail image can be associated with the target still image.

  Normally, the thumbnail generation unit 61 generates a reduced image of the target still image obtained by simply reducing the image size of the target still image as a thumbnail image. However, as in the first or second embodiment (see FIG. 6B or 9B), the display image FI ′ or 330 ′ is displayed or the display image FI ′ or 330 ′. When the user presses the shutter button 20 after displaying the above, the thumbnail generating unit 61 generates a thumbnail image from a plurality of target face images extracted from the target still image.

For example, as in the first or second embodiment, consider a case where only the faces of the persons HM ₂ and HM ₅ are set as target faces, and the shutter button 20 is pressed in a state where this setting is made. In this case, the face image extraction unit 54 in FIG. 3 or 7 extracts the face images FACE ₂ and FACE ₅ from the target still image, and the thumbnail generation unit 61 extracts the face images FACE ₂ and FACE extracted from the target still image. By combining ₅ , a thumbnail image 360 as shown in FIG. 11 can be generated. In the thumbnail image 360, the face images FACE ₂ and FACE ₅ are arranged in a horizontal direction without overlapping each other. However, the direction in which the face images FACE ₂ and FACE ₅ are arranged may be other than the horizontal direction. A part of the image FACE _{2 and} a part of the face image FACE ₅ may overlap each other.

  When a plurality of thumbnail images including the thumbnail images 360 are recorded on the recording medium 16 together with a plurality of target still images, the operation mode of the imaging apparatus 1 is switched to a thumbnail display mode which is a kind of playback mode. Is displayed. In the thumbnail display mode, the display control unit 51 sets a plurality of thumbnail display areas in the entire display area of the display screen, and displays a plurality of thumbnail images simultaneously by displaying one thumbnail image in one thumbnail display area. Let In FIG. 12, four thumbnail images 370, 360, 371 and 372 are displayed simultaneously. The user can select any one of the thumbnail images 370, 360, 371 and 372 via the user interface. When this selection is made, the target still image corresponding to the selected thumbnail image is displayed on the display unit 15. Viewing and editing by the user.

  The target face image enlarged and displayed in the shooting mode is estimated to be an image portion having a high degree of attention of the user as a photographer. By generating a thumbnail image using an image part with a high degree of attention of the user and displaying it in the thumbnail display mode, the user can easily imagine the situation at the time of shooting from the thumbnail image, and as a result, a desired target still image can be obtained. It becomes easy to find.

<< 4th Example >>
A fourth embodiment will be described. In the fourth embodiment, another thumbnail generation technique applicable to the first or second embodiment will be described.

  In the shooting mode, the main control unit 13 causes the recording medium 16 to record a frame image sequence from the time when the recording button 21 is pressed for the i-th time to the time when the recording button 21 is pressed for the (i + 1) th time as the target moving image. Can do. At this time, the thumbnail generation unit 61 generates a thumbnail image corresponding to the target moving image. Then, the main control unit 13 records the target moving image together with the thumbnail image on the recording medium 16 after associating the thumbnail image with the target moving image. For example, the main control unit 13 creates one image file in the recording medium 16, and stores the image signal of the target moving image and the image signal of the thumbnail image in the main body area and the header area of the image file, respectively. The thumbnail image can be associated with the target moving image.

  Normally, the thumbnail generation unit 61 extracts any frame image forming the target moving image as a representative frame image, and a reduced image of the representative frame image obtained by simply reducing the image size of the representative frame image. Generate as a thumbnail image. However, as in the first or second embodiment (see FIG. 6B or 9B), the target face is set and the display image FI ′ or 330 ′ is displayed during shooting of the target moving image. In this case, the thumbnail generation unit 61 generates a thumbnail image from a plurality of target face images extracted from the target moving image.

For example, consider a case where the target moving image includes the frame image FI or 330, and only the faces of the persons HM ₂ and HM ₅ are set as the target faces as in the first or second embodiment during the shooting of the target moving image. . In this case, the face image extraction unit 54 of FIG. 3 or 7 extracts the face images FACE ₂ and FACE ₅ from the extraction source frame image, and the thumbnail generation unit 61 extracts the face image FACE ₂ extracted from the extraction source frame image. And FACE ₅ can be combined to generate a thumbnail image 360 as shown in FIG. The extraction source frame image is the frame image FI or 330. Alternatively, the extraction source frame image is any frame image that forms the target moving image, and is a frame image obtained after the faces of the persons HM ₂ and HM ₅ are set as target faces. The operation of the imaging apparatus 1 after the generation of the thumbnail image is the same as that shown in the third embodiment.

  The target face image enlarged and displayed in the shooting mode is estimated to be an image portion having a high degree of attention of the user as a photographer. By generating a thumbnail image using an image part with a high degree of attention of the user and displaying it in the thumbnail display mode, the user can easily imagine the situation at the time of shooting from the thumbnail image, and as a result, a desired target moving image can be obtained. It becomes easy to find.

<< 5th Example >>
A fifth embodiment will be described. In a fifth example and a sixth example, which will be described later, a characteristic operation of the imaging apparatus 1 in the reproduction mode will be described. In the fifth embodiment, the technique described in the first embodiment is realized in the reproduction mode. The block diagram of the part particularly related to the operation of the fifth embodiment is the same as that of the first embodiment (that is, FIG. 3).

  When the user performs a predetermined operation on the user interface in the playback mode, the image signal of the frame image FI recorded on the recording medium 16 is supplied to the display control unit 51 and the face processing unit 52 as an image signal of the input image, and displayed. The control unit 51 causes the display unit 15 to display an input image as the frame image FI.

The operations of the face processing unit 52 including the operations of the face detection unit 53, the face image extraction unit 54, and the target face setting unit 55 are the same as those described in the first embodiment. Also in the fifth embodiment, as in the first embodiment, the faces of the persons HM ₁ to HM ₆ are present on the frame image FI, and only the faces of the persons HM ₂ and HM ₅ are set as target faces. Consider the case. Then, in the playback mode, the display control unit 51 generates the display image FI ′ (see FIG. 6A) by superimposing the enlargement target face images FACE ₂ ′ and FACE ₅ ′ on the frame image FI, The display image FI ′ is displayed on the display unit 15 (see FIG. 6B).

The same applies to the case where the image signal of the input image supplied to the display control unit 51 and the face processing unit 52 is an image signal of a frame image sequence (that is, an image signal of a moving image). In this case, the i-th frame image FI forming the input image corresponds to the frame image FI [i] described in the first embodiment, and the technique described in the first embodiment is applied to the fifth embodiment. Can do. That is, when the face frames FF ₂ and FF ₅ are specified by the user while the frame image FI [i] is displayed on the display unit 15 in the playback mode, the display control unit 51 displays the frame image FI [i + j]. The enlargement target face images FACE ₂ ′ and FACE ₅ ′ are generated by performing image size enlargement processing on the original target face images FACE ₂ and FACE ₅ extracted from the image, and the enlargement target face images FACE ₂ ′ and FACE ₅ ′ are framed. A display image obtained by superimposing on the image FI [i + j] can be displayed on the display unit 15 (j is an integer of 1 or more).

  According to the fifth embodiment, a plurality of faces that the user is interested in can be enlarged and displayed at the same time without requiring a zoom operation or a scroll operation at the time of reproduction. It becomes possible to confirm in detail on the display screen the states of a plurality of faces of interest at the same time.

<< Sixth Example >>
A sixth embodiment will be described. In the sixth embodiment, the technique described in the second embodiment is realized in the reproduction mode. The block diagram of the part particularly related to the operation of the sixth embodiment is the same as that of the second embodiment (that is, FIG. 7).

  When the user performs a predetermined operation on the user interface in the reproduction mode, the image signal of the frame image 330 (see FIG. 8) recorded on the recording medium 16 is displayed as the image signal of the input image, and the display control unit 51 and the face processing unit 52. The display control unit 51 causes the display unit 15 to display an input image as the frame image 330.

The operations of the face processing unit 52 including the operations of the face detection unit 53 and the face image extraction unit 54 are the same as those described in the second embodiment. Also in the sixth embodiment, as in the second embodiment, the faces of the persons HM ₂ , HM ₃ and HM ₅ are present on the frame image 330, and only the faces of the persons HM ₂ and HM ₅ are targeted. Consider the case where the face is set. Then, in the playback mode, the display control unit 51 generates the display image 330 ′ (see FIG. 9A) by superimposing the enlargement target face images FACE ₂ ′ and FACE ₅ ′ on the frame image 330, The display image 330 ′ is displayed on the display unit 15 (see FIG. 9B).

  According to the sixth embodiment, the same effect as that of the fifth embodiment can be obtained. Furthermore, in the sixth embodiment, the convenience for the user can be improved because the operation of designating the target face becomes unnecessary.

Note that when the frame image FI or 330 is recorded on the recording medium 16 in the photographing mode, the thumbnail image associated with the frame image FI or 330 is also recorded on the recording medium 16. The thumbnail generation technique described in the third or fourth embodiment is a technique for generating a thumbnail image to be associated with the frame image FI or 330 during the shooting mode. Alternatively, it can be applied to the sixth embodiment. That is, for example, when the frame image FI or 330 is displayed on the display unit 15 in the reproducing mode, if a human face HM ₂ and HM ₅ is set to the target face, described in the third or fourth embodiment The thumbnail image 360 may be generated according to the method, and the thumbnail image that is recorded on the recording medium 16 and should be associated with the frame image FI or 330 may be replaced with the thumbnail image 360.

<< Deformation, etc. >>
The embodiment of the present invention can be appropriately modified in various ways within the scope of the technical idea shown in the claims. The above embodiment is merely an example of the embodiment of the present invention, and the meaning of the term of the present invention or each constituent element is not limited to that described in the above embodiment. The specific numerical values shown in the above description are merely examples, and as a matter of course, they can be changed to various numerical values. As annotations applicable to the above-described embodiment, notes 1 to 5 are described below. The contents described in each comment can be arbitrarily combined as long as there is no contradiction.

[Note 1]
In the reproduction mode, the imaging device 1 functions as an image reproduction device that reproduces an image, and the imaging device 1 in the reproduction mode can also be called an image reproduction device. Therefore, the operation of the imaging apparatus 1 in the reproduction mode described in the fifth or sixth embodiment can be realized in the image reproduction apparatus 101 other than the imaging apparatus 1. FIG. 13 is an internal configuration diagram of the image playback device 101. The image reproduction device 101 includes at least the display unit 15, the display control unit 51, and the face processing unit 52, and may further include any part in the imaging device 1 necessary for the operation in the reproduction mode. The image playback device 101 can be a mobile terminal, a game machine, a personal computer, a mobile phone, an electronic book reader, an electronic dictionary, or the like.

[Note 2]
In the above-described embodiment, the case where the number of target face images is two is illustrated, but the number of target face images may be three or more.

[Note 3]
The imaging device 1 or the image reproduction device 101 can be configured by hardware or a combination of hardware and software. When the imaging apparatus 1 or the image playback apparatus 101 is configured using software, a block diagram of a part realized by software represents a functional block diagram of the part. A function realized using software may be described as a program, and the function may be realized by executing the program on a program execution device (for example, a computer).

[Note 4]
In the above-described embodiment, the specific type of object to be detected from the frame image is exemplified as a human face. However, the specific type of object is an object other than the human face (for example, an animal other than a person). Face, flower, robot). The imaging device 1 and the image reproduction device 101 may be formed so that the user can specify a specific type of object to be detected from the frame image via the user interface.

[Note 5]
The face detection unit 53 is an example of a specific type object detection unit that detects the face of a person as a specific type of object based on an image signal of a frame image that should be called a captured image or an input image. The target face setting unit 55 is an example of a target object setting unit that can also be called a target object selection unit, and sets or sets a plurality of target faces as a plurality of target objects based on operations performed on the user interface. select. However, in the second or sixth embodiment, the face detection unit 53 functions as a target object setting unit or also functions as a target object setting unit, and is based on an image signal of a frame image that should be called a captured image or an input image. A plurality of target faces as a plurality of target objects are set or selected. The target face image is extracted as a target object image from the captured image or the input image by the face image extraction unit 54 which is an example of the extraction unit.

DESCRIPTION OF SYMBOLS 1 Imaging device 11 Imaging part 15 Display part 51 Display control part 52 Face processing part 53 Face detection part 54 Face image extraction part 55 Target face setting part 61 Thumbnail generation part

Claims (9)

An imaging unit that outputs an image signal representing a captured image of the subject group;
A display unit for displaying an image based on the image signal;
An extraction unit that extracts images of a plurality of target objects included in the subject group from the captured image as target object images for each target object;
An image pickup apparatus comprising: a display control unit configured to enlarge an image size of each extracted target object image and display the enlarged each target object image on the display unit together with the captured image. The imaging apparatus according to claim 1, further comprising a target object setting unit configured to set the plurality of target objects based on an operation performed on a user interface. Each target object is a specific type of object,
2. The imaging apparatus according to claim 1, further comprising a target object setting unit configured to set the plurality of target objects by detecting the specific type of object from the captured image based on the image signal. Imaging device. The imaging according to any one of claims 1 to 3, further comprising a thumbnail generation unit that generates a thumbnail image of the captured image using a plurality of target object images extracted from the captured image. apparatus. The imaging apparatus according to claim 1, wherein the target object includes a human face. In an image reproduction apparatus that receives an image signal of an input image that includes images of a plurality of target objects,
A display unit for displaying an image based on the image signal;
An extraction unit that extracts images of the plurality of target objects from the input image as target object images for each target object;
An image reproduction apparatus comprising: a display control unit configured to enlarge an image size of each extracted target object image and display the enlarged target object image on the display unit together with the input image. The image reproducing apparatus according to claim 6, further comprising a target object setting unit configured to set the plurality of target objects based on an operation performed on a user interface. Each target object is a specific type of object,
7. The image reproducing apparatus according to claim 6, further comprising a target object setting unit configured to set the plurality of target objects by detecting the specific type of object from the input image based on the image signal. The image reproducing device described. The image reproduction device according to claim 6, wherein the target object includes a human face.

Images