JP2012032728A - Image display device and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device and its method capable of displaying images in consideration of relationships of subjects appeared in the images.SOLUTION: Plural pieces of face information included in images are selected and relationships of plural subjects are judged from the face information (S103). The predetermined number of images are chosen while including the images which include the relationships judged from plural images based on the judged relationships (S105-S108) and the chosen images are displayed as a slide show or an album (S113, S114).

Description

  The present invention relates to an image display device and an image display method, and more particularly, to an image display device and an image display method that determine the relationship between a plurality of subjects in an image and perform optimal display.

  In many cases, multiple images are displayed in a slide show or album format to enjoy the images. In this case, it is often a problem to determine the order in which images are displayed in the slide show and the arrangement of the positions and sizes of the images to be pasted on the album.

  Accordingly, various proposals have been made for the arrangement of a plurality of images. For example, Patent Document 1 discloses a slide show reproduction method in which a group scene is generated by determining a size from a representative image of a group having a large number of face images, generating a group scene, and displaying the group scene at the end of the slide show. Yes.

JP 2007-249371 A

  In this way, proposals for arranging a plurality of images have been made. However, if a plurality of images are arranged simply because there are many characters, the images are dry and cannot be enjoyed. However, if human relationships such as parent-child relationships and relationships between the appearing subjects such as relationships between pets and owners are known, each image can be viewed with familiarity. The prior art does not suggest any arrangement in consideration of the relationship between the appearing subjects.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image display device and an image display method that enable image display in consideration of the relationship between subjects appearing in an image.

  In order to achieve the above object, an image display device according to a first invention includes a face information extraction unit that extracts a plurality of face information included in an image, and a relationship determination that determines a relationship between a plurality of subjects from the face information. And a display control unit that controls a display mode of an image including the plurality of face information based on the relationship determined by the relationship determination unit, and the display mode determined by the display control unit And a display unit for displaying an image.

The image display device according to a second invention is the image display device according to the first invention, wherein the plurality of faces are at least a child's face or an animal's face, and the relationship determination unit The relationship between a face and a person with another face is determined.
In the image display device according to a third aspect of the present invention, in the second aspect, the relationship determination unit specifies a guardian or owner of the child or the animal.

In the image display device according to a fourth aspect of the present invention, in the first aspect, the display control unit controls the slide show, the order of the album, or the arrangement of the images on the page.
In the image display device according to a fifth aspect of the present invention, in the first aspect, the display control unit selects a relationship image in which the relationship between a plurality of subjects is known, and displays the relationship image at least at the beginning and at the end.

In the image display device according to a sixth aspect based on the first aspect, the relationship determination unit is configured to determine the relationship between the subjects based on whether or not the face person extracted by the face information extraction unit faces each other. Determine.
In the image display apparatus according to a seventh aspect based on the first aspect, the face information extraction unit determines a position of a person's hand of the extracted face, and the relationship determination unit includes the face information extraction. The relationship between the subject is determined based on the face position extracted by the unit and the hand position.
In the image display device according to an eighth aspect based on the first aspect, the relationship determination unit is configured to determine the relationship between the subjects based on the positions and sizes of a plurality of faces extracted by the face information extraction unit. Determine.

  An image display method according to a ninth invention extracts a plurality of face information included in an image, determines a relationship between a plurality of subjects from the face information, and based on the determined relationship, the plurality of the plurality of face information. Display an image containing face information.

  ADVANTAGE OF THE INVENTION According to this invention, the image display apparatus and the image display method which enabled the image display which considered the relationship of the subject which appears in an image can be provided.

It is a block diagram which shows the whole structure mainly having the electric system of the camera concerning one Embodiment of this invention. It is a figure which shows the example of the image | photographed image in the camera concerning one Embodiment of this invention. It is a figure which shows the other example of the image image | photographed in the camera concerning one Embodiment of this invention. It is a figure which shows the relationship of the image | photographed image in the camera concerning one Embodiment of this invention. It is a figure which shows the example of the slide show of the image | photographed image in the camera concerning one Embodiment of this invention. It is a figure which shows the example of the album of the image | photographed image in the camera concerning one Embodiment of this invention. It is a figure which shows the example which displays based on the relationship between a pet and a human in the camera concerning one Embodiment of this invention. It is a figure which shows the example of a shoulder wheel image in the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the display control of the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the relationship determination of the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the slide show execution of the camera concerning one Embodiment of this invention. 6 is a flowchart illustrating an operation of an album page layout of a camera according to an embodiment of the present invention. It is a figure which shows the example of the page layout of the album of the camera concerning one Embodiment of this invention.

  Hereinafter, a preferred embodiment will be described using a camera to which the present invention is applied according to the drawings. A camera according to a preferred embodiment of the present invention is a digital camera, includes an imaging unit, converts a subject image into image data by the imaging unit, and converts the subject image into a main body based on the converted image data. Live view is displayed on the display unit placed on the back of the camera. The photographer determines the composition and the photo opportunity by observing the live view display. During the release operation, image data is recorded on the recording medium. The image data recorded on the recording medium can be reproduced and displayed on the display unit when the reproduction mode is selected. In the playback mode, an album mode or a slide show can be performed. In this case, a relationship between subjects is determined, and a plurality of images are appropriately selected and displayed according to the determination result.

  The configuration of the camera 10 according to an embodiment of the present invention will be described using the block diagram shown in FIG. The camera 10 includes a control unit 1, an imaging unit 2, a face detection unit 2a, a child determination unit 2b, a pet determination unit 2c, a recording unit 6, a clock unit 7, a display unit 8, an operation unit 9, a flash memory 11, and the like. The

  The imaging unit 2 includes an exposure control unit such as a photographic lens and a shutter, an imaging device, an imaging device driving and readout circuit, and the like. The subject image formed by the photographic lens is converted into image data by the imaging device and is output. . In the present specification, the image data is not limited to the image signal output from the image sensor, but is recorded in the image data processed by the image processing unit 5 in the control unit 1 and the recording unit 6. Also used for image data.

  The face detection unit 2a receives the image data image-processed by the image processing unit 5 in the control unit 1, and if the image includes a face part of a person or an animal or if it is included Detects the position and the orientation of the face, and outputs the detection result to the control unit 1. Further, when a plurality of facial features are registered, it is also determined whether or not the facial features match any one of the registered facial features. Further, not only the face but also the position of the hand is detected. The position of the face or hand is detected by matching with data of face parts and hand parts prepared in advance.

  When the face detection unit 2a detects a face, the child determination unit 2b determines whether the face is a child. Whether the child is a child is determined based on the height, clothes, and the like using the image data of the subject image. When the face detection unit 2a detects a face, the pet determination unit 2c determines whether the face is a pet such as a dog or a cat. Whether the pet is a pet is determined from the shape of the face, limbs, torso, etc. using the image data of the subject image.

  The control unit 1 includes a CPU (Central Processing Unit) and peripheral hardware circuits, and performs an entire sequence of the camera 10 according to a program stored in a nonvolatile storage unit such as a flash memory 11. Control. The control unit 1 includes a face size / expression unit 1a, a relationship determination unit 1b, a display control unit 1c, and an image processing unit 5.

  The face size / expression unit 1a detects the size and expression of the face detected by the face detection unit 2a, and becomes an accent when creating a slide show or album based on the detection result, as will be described later. Like, select an image.

  Based on the detection / determination results of the face detection unit 2a, the child determination unit 2b, the pet determination unit 2c, and the face size / expression unit 1a, the relationship determination unit 1b The relationship between children (and animals such as pets) is determined. For example, it is determined whether or not there is a parent-child relationship or a subordinate / subordinate relationship such as a pet and owner. This determination is made based on the positional relationship between a plurality of faces, the direction in which the faces are facing, the positional relationship between the face and the hand, and the like. For example, when a person is facing each other, when it is determined that the person is hugging from the position of the face or hand, or when it is determined that the state of the shoulder wheel is based on the size and position of the plurality of faces, between the plurality of subjects Is determined to be related.

  The display control unit 1c performs display control on the display unit 8 such as live view display, REC view display during shooting, playback display, and a menu screen. In the live view display, the image data acquired by the imaging unit 2 and subjected to image processing for live view display by the image processing unit 5 is displayed on the display unit 8. In the reproduction display, the image data recorded in the recording unit 6 is read and displayed on the display unit 8 for reproduction. Further, in the reproduction display by the album reproduction or the slide show, the display form relating to the images to be displayed from among a plurality of images, the order of the images, the display position, and the like is controlled based on the relationship determined by the relationship determination unit 1b.

  The image processing unit 5 performs digital amplification (digital gain adjustment processing), color correction, gamma (γ) correction, contrast correction, monochrome / color mode processing, live view display processing, and the like of image data output from the imaging unit 2. Various image processing is performed. In addition, image compression processing is performed to record the captured image, and image expansion processing is also performed to reproduce and display the recorded captured image.

  The operation unit 9 is various operation members such as a release button, a power button, a playback button, a simple album button, a menu button, a cross button, and an OK button arranged on the exterior of the camera 10, and the operation unit 9 performs these operations. An operation state of a switch or the like interlocked with the member is determined, and the determination result is output to the control unit 1.

  The display unit 8 is connected to the control unit 1 and has a monitor such as a liquid crystal or an organic EL disposed on the back surface of the main body. As described above, according to control by the display control unit 1c, live view display and A REC view display at the time of shooting, a display of a recorded image recorded in the recording unit 4, and a control screen such as a menu screen are displayed. The display unit 8 is provided with a touch panel. When the user touches the monitor screen, information such as a touch position is output to the control unit 1.

  The recording unit 6 includes a recording medium that is detachable from the camera body or a built-in recording medium. The recording unit 6 records image data of still images and moving images output from the imaging unit 2 and subjected to image processing by the image processing unit 5 together with accompanying information. The clock unit 7 has a timekeeping function and outputs date / time information. At the time of shooting, this date / time information is recorded on the recording medium of the recording unit 6 together with the image data.

  The flash memory 11 is an electrically rewritable nonvolatile memory, and stores a program for controlling the entire camera 10 as described above. In addition to the program, the flash memory 12 stores various data such as adjustment values.

  Next, an image display form according to this embodiment will be described with reference to FIGS. FIG. 2 shows an example of four images recorded in the recording unit 6. The image 21 shown in FIG. 2 is a single image of the adult A, the image 22 is an image of the child B in close contact with the adult A, and the images 23 and 24 are single images of the child B. Since the images 21, 23, and 24 are images in which the adult A and the child B are individually shown, the relationship between them is unknown. However, it can be determined that there is a relationship between the adult A and the child B based on the image 22.

  When only the images 21, 23, and 24 are displayed as a slide show or album, the relationship between the adult A and the child B is not known and the display is abrupt. Therefore, in the present embodiment, in the case of a slide show, the relationship is such that the adult A and the child B are displayed by displaying the image 22 at the first or last, sometimes both timings, and the two are in close contact with each other. In other words, it can be seen that there is a parent-child relationship. Further, in the case of album display, by displaying the image 22 together with the images 21, 23, 24, it is possible to obtain a screen that eliminates a sudden feeling.

  FIG. 3 shows an example of another image recorded in the recording unit 6. The image 26 shown in FIG. 3 is an image of the owner P and the pet (cat) Q in close contact, and the images 27 and 28 are images of the pet Q. When only the images 27 and 28 are displayed in a slide show or an album, there is a sense of abruptness, except for viewers who like pets, and they are not very interesting images. However, when the image is displayed together with the image 26 showing the owner P together with the pet Q, the relationship between the pet Q and the owner P can be understood, and the abrupt feeling disappears and the image becomes interesting. Therefore, in the present embodiment, not only the images 27 and 28 in which the pet Q is shown alone are displayed, but also an image 26 that shows the relationship with the owner P of the pet Q is displayed in a slide show or album. .

  FIG. 4 is a diagram illustrating an example of a relationship between subjects in an image. In this example, image No. For 1 to 6, the images are arranged in terms of whether the relationship can be determined, the type of face, the size of the face, whether it is dependent or dependent, and the display order. Here, the type of face is determined by the face detection unit 2a, the child determination unit 2b, and the pet determination unit 2c, and it is determined whether it is male, female, or child, a pet such as a cat or dog, etc. If it does not belong to the type, it is determined as a landscape image.

  The face size is determined by the face size / display unit 1a based on the ratio of the face size compared to the screen size. For example, when the face size is 1/5 or less, the face size is determined as “1”. As the ratio increases, “2” and “3” are determined.

  The relationship determination including subordinate or subordinate is determined by the relationship determination unit 1b. The relationship determination includes two or more subjects in the screen, and whether the subject is dependent or dependent is determined based on the face position, the hand position, or the like. In the relationship determination column of FIG. 4, whether or not the image can be determined is recorded. The relationship determination will be described later in detail using the flowchart shown in FIG.

  The display order indicates the order in which the slide show is displayed, and is determined based on items recorded in each column from the relationship determination to the subordinate. In the example shown in FIG. 2 is displayed. 3 → No. 4 → No. 1 → No. Are displayed in the order of 2. Here, first, an image No. 1 in which a parent and a child are shown. By displaying 2, the relationship between the images to be displayed is shown, and then a picture of child B (No. 3) is displayed. Then, the photograph (No. 4) whose face size is larger than this photograph is displayed, then the photograph of the male A (No. 1) is displayed, and finally the photograph of the parent and child is displayed again. Yes. By adopting such a display order, the relationship between the subjects appearing in the slide show can be understood, and the image can be changed while the whole is unified, and the slide show can be enjoyed.

  The parent-child photograph (image No. 2) may be the last one only. In this case, it is possible to make a slide show with a sense of expectation regarding the relationship between the child B and the man A. Also, the smile levels may be arranged together, and the smile levels may be taken into account when determining the display order.

  Next, an example of displaying an image using the relationship as shown in FIG. 4 will be described with reference to FIGS. FIG. 5 shows an example of a slide show when the subject is a pet. In this example, first, an image 31 showing a pet cat is displayed, and then an image 32 and an image 33 are displayed in this order. Finally, an image 34 in which the owner is holding the cat is displayed. In this image 34, a dear mark 34a is displayed to indicate that the owner takes care of the pet.

  When the photos of the pets continue, the viewer is interested in where the cat is. Finally, a picture (image 34) showing the pet owner is displayed, so that the relationship between the pet and the owner can be understood. As a result, there will be a change and the slide show will be organized as a whole. Of course, as shown in the example of the display order shown in FIG. 4, the image 34 in which the pet and the owner are displayed together may be displayed not only at the end but also at the beginning.

  FIG. 6 shows an example of a family photo album. In the upper left of the album screen, an image 36 of a photograph showing a parent and child is arranged, and in the upper right, an image 37 of a photograph showing a child is arranged. In addition, an image 38 of a photograph showing a parent is arranged in the lower left of the screen, and an image 39 of a photograph showing a child is arranged in the lower right. Only the images 37 and 39 of the child alone and the image 38 of the parent alone have no sense of unity and the relationship between the two is unknown. However, by arranging the image 36 of the photograph showing the parent and child together, It becomes an understandable album. In addition, the image 37 of the child who is the main character is displayed in a large size.

  FIG. 7 shows an example of a single still image display or a single image display during a slide show. FIG. 7A shows a still image 41 in which the owner 41a and the pet cat 41b are shown. If the relationship between the subject of the image 41 is determined and the relationship between the cat 41b and its owner 41b is found, as a display method on the display unit 8 of the camera 10, first, as shown in FIG. The entire image 41 is shown, and then the cat 41b is cut out and enlarged as shown in FIG. 7C. By knowing the relationship between the two, it is possible to display from the whole to the part. As a display method, conversely, the cat 41b may be enlarged and then displayed as a whole. In this case, the display gives a sense of expectation as to who the owner of the cat 41b is.

  In this embodiment, the relationship between subjects is determined from a plurality of subjects in the image. As a case where this relationship is clear, as shown in the image 22 of FIG. 2, when the positions of the faces of an adult and a child are close, as shown in FIG. 7A, there is a pet face under the human face. There is a case. Further, as shown in FIG. 8, there are cases where the face of the child is above the face of the adult, such as an image when the adult is shouldering the child.

  As described above, in this embodiment, when an image is analyzed and there are two or more subjects in the image, the relationship between the two is determined, and the display is switched using the determination result. When a plurality of images are displayed in a slide show, a photo showing the relationship between the appearing subjects or a single photo is sequentially displayed in the order in which the images are displayed. In the case of album display, a picture that shows the relationship between the appearing subjects or a picture that appears alone is selected as appropriate and arranged on the screen.

  Next, the operation in the present embodiment will be described with reference to the flowcharts shown in FIGS. These flows are executed by the CPU in the control unit 1 according to a program stored in the flash memory 11. The flow of display control starts when the playback button, which is one of the operation units 9, is operated and the playback mode starts.

  If the display control flow shown in FIG. 9 is entered, it is first determined whether or not the album is a simple album (S101). The simple album is a mode that is displayed in the slide show shown in FIG. 5 or the album page layout shown in FIG. 6, and is selected by operating a simple album button that is one of the operation units 9 or by specifying on the menu screen. Can do. In this step, it is determined whether or not a simple album button has been operated or a simple album has been designated on the menu screen.

  If the result of determination in step S101 is a simple album, next, a search range is designated (S102). The user can specify the search range when displaying with a simple album. For example, when the range is specified by the shooting date, the specified number is the most recently shot, and the shot images are classified The classification can be specified. Therefore, in this step, the range specified by the user is read. If there is no designation, the search range is a predetermined number of pictures taken recently.

  When the search range is designated, the relationship is next determined (S103). Here, based on the plurality of images within the designated range, as described with reference to FIGS. 2 to 8, it is determined whether or not there is a relationship between the plurality of subjects in the image. Detailed operation of this relationship determination will be described later with reference to FIG.

  Once the determination of the relationship is determined, it is next determined whether or not there is a relationship determination image (S104). In step S103, since it is determined whether or not there is a relationship for each image, in this step, it is determined whether or not there is an image determined to have a relationship.

  If the result of determination in step S104 is that there is a related image, then a relevance determination image is selected and the image is designated IM1 (S105). Here, for example, the image 22 in FIG. 2, the image 26 in FIG. 2. An image in which the relationship among a plurality of subjects is known, such as an image 34 in FIG. 5 and an image 36 in FIG. 6, is selected, and the image is referred to as IM1.

  Next, the subject B single image is selected, and the image is set to IM2 (S106). Here, an image in which the subject B appearing in the related image IM1 selected in step S104 is shown alone is selected. The subject B is a subject that plays a leading role in a plurality of images, such as a child or a pet.

  If the subject B (IM2) is selected, then an accent image is selected and the image is designated IM3 (S107). Here, the accent image is an image with a large expression of the face of the subject B or an image having a facial expression such as a smile, and is selected by the face size / expression unit 1a.

  If an accent image is selected, then the subject A single image is selected and the image is designated IM4 (S108). Here, an image in which the subject A appearing alone in the related image IM1 selected in step S104 is selected is selected. In step S105 to step S108, four images are selected, but by looking at the four images in order, an image that feels the streak of starting and rolling is selected. That is, first, an image in which the subject A and the subject B are combined appears, then the main subject B, then the subject B up or smile, and finally the subject A appears.

  On the other hand, if the result of determination in step S104 is that there is no relationship determination image, next, the four latest images are selected, and these are designated as IM1 to IM4 in chronological order (S111). Since there is no related image, the latest image in order of shooting date is selected.

  If an image is selected in step S111 or S108, it is next determined whether or not to perform a slide show (S112). Simple albums include a slideshow and album page layout. In this step, it is determined whether or not the user has selected a slide show on the menu screen or the like.

  If the result of determination in step S112 is that a slide show has been selected, the slide show is executed (S113). Here, IM1 to IM4 selected in steps S105 to S108 or S111 are sequentially displayed in a slide show. Detailed operation of this slide show execution will be described later with reference to FIG.

  If the result of determination in step S112 is not a slide show, display using an album layout is performed (S114). Here, the display is performed by the album layout as shown in FIG. Detailed operation of this album page layout will be described later with reference to FIG.

  If the result of determination in step S101 is not a simple album, next, a file list display is performed (S121). If the album is not a simple album, the display is based on the normal playback mode. Therefore, in this step, the captured images recorded in the recording unit 6 are displayed as a list of thumbnail images.

  If the file list is displayed, it is next determined whether or not the file is selected (S122). From the images displayed in a list, an image that the user desires to enlarge is selected by operating the cross button or the like, and confirmed by operating the OK button or the like. In this step, it is determined whether or not these operations have been performed.

  If the result of determination in step S122 is that file selection has been made, the selected file is then played back (S123). Here, the image data of the selected image is read from the recording unit 6 and enlarged and displayed on the display unit 8.

  Once the selected file has been played, it is next determined whether or not to return (S124). When the user wants to end the currently reproduced image, the user operates the return button. In this step, it is determined whether or not the return button has been operated. If the result of this determination is that the return button has not been operated, a standby state is entered.

  If the result of determination in step S124 is a return, or if the result of determination in step S122 is that file selection has not been made, or if a slide show in step S113 is executed, or if an album page layout in step S114 is performed, Then, it is determined whether or not the process is finished (S125). When the playback display is ended, the playback button is ended again by operating the playback button. Here, it is determined whether or not the playback button has been operated. If the result of this determination is not end, the flow returns to step S101. On the other hand, if it has ended, the playback mode is ended.

  Next, the relationship determination operation in step S103 will be described with reference to the flowchart shown in FIG. When entering the relationship determination flow, first, it is determined whether or not a face of a predetermined size or larger is detected at the top of the screen (S201). Here, it is determined whether or not the position of the face detected by the face detection unit 2a is at the top of the screen, and if it is at the top, the size of the face determined by the face size / expression unit 1a. Is larger than a predetermined value, for example, whether there is a face size as large as a portrait.

  If the result of determination in step S201 is that a face larger than a predetermined size is detected at the top of the screen, it is determined whether or not both hands are under the face (S202). Here, the position of the hand is detected by the face detection unit 2a and the like, and the determination is made based on the detection result.

  If the result of determination in step S202 is that both hands are present under the face, it is next determined whether or not there is a face near the upper hand (S203). Here, it is determined whether there is another face different from the face detected in step S201 near the upper hand in both hands detected in step S202.

  If the result of determination in step S202 is that there are no hands under the face, it is next determined whether or not the face orientation is centered (S205). Here, when a face of a predetermined size or larger is detected at the upper part of the screen, the face detection unit 2a determines whether or not the face direction of the person faces the center of the screen.

  If the result of determination in step S205 is that the face orientation is centered, it is next determined whether or not there is a second face that is substantially the same size as the face facing (S206). Here, it is determined based on the detection result of the face detection unit 2a whether or not there is another face in the direction of the center face detected in step S205.

  If the result of determination in step S206 is that the second face is present, it is next determined whether or not the orientation of the second face is toward the center (S207). Here, it is determined based on the detection result of the face detection unit 2a whether or not the orientation of the second face detected in step S206 is the center of the screen.

  If the result of determination in step S207 is that the direction of the second face is centered, it is determined whether or not the distance between the faces is equal to or less than one face (S208). Here, the determination is based on the distance between the face detected in step S201 and the second face detected in step S206.

  If the result of determination in step S208 is that the distance between faces is not less than one face, it is next determined whether or not the outer side of the screen is raised (S209). Here, the shoulder of a person with at least one of the faces is detected, and it is determined whether or not the shoulder outside the screen is raised.

  If the result of determination in step S201 is that a face larger than a predetermined size has not been detected at the top of the screen, it is next determined whether or not it is a shoulder wheel image (S204). The shoulder wheel image is an image as shown in FIG. In determining whether the image is a shoulder car image, it is determined whether or not the first and second faces exist on the screen, and there is a small face above the face larger than a predetermined value. The image that satisfies the condition is the shoulder wheel image.

  If the result of determination in step S203, S208, S209, or S204 is Yes, the first detected face is the first face (S212), the first face is the subject A, and the second face is the first face. A subject B is set (S213), and it is determined that there is a determination image (S214). When the determination in step S203 is Yes, it is an image in which an adult is holding a baby or a child, or an owner is holding a pet, and both have a dependency / subordinate relationship. In this case, the face detected in step S201 is the first face (subject A), and the face detected in step S203 is the second face (subject B).

  If the result of determination in step S208 is Yes, the image is such that two people or a person and a pet face each other in the center of the screen, and both have a dependency / subordinate relationship. . In this case, the face detected in step S201 is the first face, and the face detected in step S206 is the second face.

  If the result of the determination in step S209 is Yes, the distance between the faces is somewhat apart, but the two are facing each other, so there is a dependency / subordinate relationship between the two. . In this case, the face detected in step S201 is the first face, and the face detected in step S206 is the second face.

  Further, when the result of determination in step S204 is Yes, since the shoulder wheel image is as shown in FIG. 8, both have a dependency / subordinate relationship. In this case, the face detected in step S201 is set as the first face, and the face detected in step S204 is set as the second face.

  On the other hand, if the result of determination in step S203, S205, S206, S207, S209, or S204 is No, it is determined that there is no dependency / subordinate relationship. It performs (S215).

  If it is determined that there is a determination image in step S214, or if it is determined that there is no determination image in step S215, the process returns to the original flow. When returning to the original flow, the determination in step S104 is performed based on the determination results in steps S214 and S215.

  Next, the slide show execution operation in step S113 will be described using the flowchart shown in FIG. If the slideshow execution flow is entered, it is first determined whether or not there is a relationship determination image (S301). Here, based on the result of the relationship determination in step S103, the determination is made in the same manner as in step S104.

  If the result of determination in step S301 is that there is a relationship determination image, then IM1 is displayed (S302). Here, the image of IM1 selected in step S105 is displayed on the display unit 8.

  When the image of IM1 is displayed, the face of the subject B in IM1 is next close up (S303). Here, as described with reference to FIGS. 7B and 7C, the subject B (cat as a pet) is displayed in close-up on the display unit 8.

  Once the subject B is closed up, next, IM2 to IM4 are displayed at predetermined time intervals (S304). Here, the images (IM2 to IM4) selected in steps S106 to S108 are sequentially displayed on the display unit 8. The predetermined time is approximately the same as the time interval in a normal slide show.

  Subsequently, IM1 is displayed again with a decoration mark (S305). Here, for example, as shown in FIG. 5, decoration marks such as a dear mark 34 a indicating the relationship between the subjects A and B are superimposed on the image (IM1) and displayed on the display unit 8.

  If the result of determination in step S301 is that there is no relationship determination image, IM1 to IM4 are displayed at predetermined time intervals (S306). If there is no relationship determination image, in step S111, the four latest images are selected and set to IM1 to IM4 in the order of shooting date. In this step, the selected images of IM1 to IM4 are sequentially displayed on the display unit 8 at predetermined time intervals.

  When IM1 is displayed with a decoration mark in step S305, or when IM1 to IM4 are displayed at predetermined time intervals in step S306, the process returns to the original flow.

  Next, the operation of the album page layout in step S114 will be described using the flowchart shown in FIG. If the album page layout flow is entered, template candidates are first displayed (S401).

  For example, as shown in FIG. 13, the template candidate indicates the position and size of a frame for displaying a plurality of images. In the template shown in FIG. 13, four images can be pasted, with P1 on the upper left, P2 on the upper right, P3 on the lower left, and P4 on the lower right. One image is monotonous on one screen, but the screen becomes complicated when too many images are displayed. Therefore, in the template shown in FIG. 13, four images are displayed on one screen. The number of display images on one screen is not limited to four, and may be three or five.

  Once the template candidates are displayed, it is next determined whether or not to change the template (S402). When the user desires to change the template candidate displayed in step S401, the user operates with the cross button or the like, so in this step, it is determined whether or not the change operation has been performed.

  If the result of determination in step S402 is that a template change operation has been performed, template candidates are switched in accordance with the change operation (S403). Here, the switching process is performed based on a plurality of templates stored in advance. When the template candidate is switched, the original flow is restored.

  If the result of determination in step S402 is that template change has not been made, then IM2 (main role) is displayed in the largest frame (S411). In the example of the template shown in FIG. 13, the image of IM2 selected in step S106 is displayed at the position P2.

  Next, it is determined whether or not the person of IM1 is small (S412). IM1 is an image showing the relationship between the leading role (children, pets, etc.) and other people. For this reason, it is desirable to handle it as an auxiliary image for explanation. Further, since the upper left and lower right of the screen are positions corresponding to the beginning or end of the horizontally written document, related images are displayed. Therefore, IM1 is arranged at either the upper left position or the upper right position. In this case, IM1 is set at any position according to the size of the person of IM1.

  If the result of determination in step S412 is that the person of IM1 is small, IM1 is displayed on the larger of the upper right and lower left frames (S413). On the other hand, when the person of IM1 is large, IM1 is displayed on the smaller one of the upper right and lower left (S414). When the person of IM1 is small, a large frame is selected so that the person looks large. On the other hand, when the person of IM1 is large, the small frame is selected because it is sufficiently visible.

  If IM1 is displayed in step S413 or S414, then IM4 is displayed in the smallest remaining frame (S415). Since IM4 is often the owner of a pet or the parent of a child and is not the main character, a small frame is selected and displayed.

  Once IM4 is displayed, IM3 is then displayed in the remaining frame (S416). When IM3 is displayed, the original flow is restored.

  As described above, in the present embodiment, the relationship between a plurality of subjects included in an image is determined using face information, and how to display a plurality of images is determined based on the determination result. Yes. In the present embodiment, an image including a plurality of persons is displayed as the first or last image, a representative image, a large image, a small image, or the like. For this reason, it is possible to perform image display in consideration of the relationship between subjects appearing in the image.

  Further, by extracting and displaying images having a relationship as shown in the present embodiment, the images can be used effectively. For example, you may get along by taking a picture of your family and showing it to an overseas person. In that case, photographs of children and pets and their guardians evoke emotions common to all countries, review the background that is often ignored, such as when and where it was taken, and various imaginations. Has an effect to evoke. By extracting an image as shown in this embodiment, the image can be used effectively, and the image can be further enjoyed.

  In the present embodiment, when the simple album mode is selected, the slide show and the album display are performed. However, only one of them may be displayed. Moreover, in determining the relationship, the determination was made in three ways: holding (S201 to S203 Yes), facing each other (S205 to S209), and carrying a shoulder wheel (S204). Of course, the present invention is not limited to this, and other relationships may be determined as being related.

  In the present embodiment, an example in which the present invention is applied to a digital camera having an imaging unit has been described. However, the present invention is not limited to this, and the present invention can also be applied to a dedicated reproduction / display device for a recorded image and a personal computer, which has a recording unit that records the captured image, omitting the imaging unit.

  Furthermore, in the present embodiment, the digital camera is used as the photographing device. However, the camera may be a digital single-lens reflex camera or a compact digital camera, and may be used for moving images such as video cameras and movie cameras. It may be a camera, or may be a camera built in a mobile phone, a personal digital assistant (PDA), a game machine, or the like.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 1 ... Control part, 1a ... Face size and expression part, 1b ... Relationship determination part, 1c ... Display control part, 2 ... Imaging part, 2a ... Face detection part, 2b: Child determination unit, 2c: Pet determination unit, 5 ... Image processing unit, 6 ... Recording unit, 7 ... Clock unit, 8 ... Display unit, 9 ... Operation Part, 10 ... camera, 11 ... flash memory, 21-24 ... image, 26-28 ... image, 31-34 ... image, 34a ... dear mark, 36-39 ..Image, 41 ... Image, 41a ... Owner, 41b ... Cat

Claims (9)

A face information extraction unit that extracts a plurality of pieces of face information included in an image;
A relationship determination unit for determining a relationship between a plurality of subjects from the face information;
A display control unit that controls a display mode of an image including the plurality of face information based on the relationship determined by the relationship determination unit;
A display unit for displaying the image according to the display mode determined by the display control unit;
An image display device comprising: The plurality of faces are at least a child's face or an animal's face,
The image display apparatus according to claim 1, wherein the relationship determination unit determines a relationship between the face of the child or the animal and a person having another face.   The image display device according to claim 2, wherein the relationship determination unit specifies a guardian or owner of the child or the animal.   The image display apparatus according to claim 1, wherein the display control unit controls a slide show, an album order, or an arrangement of images on a page.   The image display apparatus according to claim 1, wherein the display control unit selects a related image that shows a relationship between a plurality of subjects, and displays the related image at least at the beginning and the end.   The image display apparatus according to claim 1, wherein the relationship determination unit determines the relationship between the subjects based on whether or not the face person extracted by the face information extraction unit faces each other. The face information extraction unit determines the position of the hand of the person of the extracted face,
The relationship determination unit determines a relationship between the subject based on the face position extracted by the face information extraction unit and the hand position;
The image display apparatus according to claim 1.   The image display apparatus according to claim 1, wherein the relationship determination unit determines the relationship between the subjects based on positions and sizes of a plurality of faces extracted by the face information extraction unit. Extract multiple face information contained in the image,
From the face information, determine the relationship between multiple subjects,
Based on the determined relationship, an image including the plurality of face information is displayed.
An image display method characterized by the above.