JP2006079457A - Electronic album display system, electronic album display method, electronic album display program, image classification device, image classification method and image classification program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically select and display an image desired by an appreciator of the image with high accuracy. <P>SOLUTION: This electronic album display system displaying a plurality of images included in an electronic album has: an image storage part storing the plurality of images; a person recognition part recognizing a plurality of persons included in the image; an in-image intimacy degree calculation part calculating an in-image intimacy degree that is an intimacy degree between the persons included in the image as higher value as a distance between the images each showing each the person is small on the basis of a recognition result; an in-album intimacy degree calculation part calculating an in-album intimacy degree that is an intimacy degree between the persons included in the image of at least a part of the plurality of images on the basis of the in-image intimacy degree; an image selection part selecting the image including the person whose in-album intimacy degree with the appreciator is a predetermined reference value or above; and an image display part displaying the selected image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an electronic album display system, an electronic album display method, an electronic album display program, an image classification device, an image classification method, and an image classification program. In particular, the present invention relates to an electronic album display system, an electronic album display method, an electronic album display program for displaying a plurality of images included in an electronic album, an image classification apparatus, an image classification method, and an image for classifying a plurality of images. Regarding the classification program.

Conventionally, as a display method for displaying a plurality of images, a display method has been proposed in which each of the plurality of images is classified into a plurality of groups based on related information of the images and a representative image selected from each group is displayed. (For example, refer to Patent Document 1).
JP2004-13575

  In a display method for displaying a plurality of images, it is not easy to automatically select a desired image by the viewer when only a part of the images is selected and displayed instead of displaying all the images. Absent. For example, when a representative image representing the group is selected and displayed from each of the groups into which the images are classified as in the display method described in Patent Document 1, the general contents of the plurality of images are efficiently grasped efficiently. Although there is an effect of being able to do so, it is difficult to accurately select an image desired by the viewer.

  Therefore, an object of the present invention is to provide an electronic album display system, an electronic album display method, an electronic album display program, an image classification device, an image classification method, and an image classification program that can solve the above-described problems. This object is achieved by a combination of features described in the independent claims. The dependent claims define further advantageous specific examples of the present invention.

  In order to solve the above problem, in the first embodiment of the present invention, an electronic album display system for displaying a plurality of images included in an electronic album, an image storage unit storing a plurality of images, For each of the plurality of images, based on the recognition result by the person recognition unit that recognizes the plurality of persons included in the image and the person recognition unit for each of the plurality of images, the parent between the persons included in the image An image intimacy calculating unit that calculates an in-image intimacy as a density as a higher value as a distance between images indicating each of the persons is smaller, and an in-image parent calculated for each of a plurality of images Based on the density, the in-album intimacy calculating unit that calculates in-album intimacy between each of the persons included in at least some of the images. The image selection unit for selecting an image including a person whose album intimacy with the viewer is equal to or higher than a predetermined reference value, and the image selected by the image selection unit are displayed. An image display unit.

  For each of the plurality of images, the in-image intimacy calculation unit indicates the in-image intimacy between the persons included in the image, and the smaller the distance between the face images included in the image indicating each of the persons, It may be calculated as a higher value. If the distance between images indicating each person included in the image is equal to or greater than a predetermined reference value for each of the plurality of images, the image intimacy calculation unit The inner familiarity may be calculated as a predetermined standard value.

  The image storage unit stores an image including the owner of the electronic album, and the image selection unit determines the closeness in the album between the owner and the viewer when the owner and the viewer are different. An image may be selected based on. The image selection unit may select an image including a person whose in-album intimacy with the owner is lower than in-album intimacy between the owner and the viewer. The electronic album display system is configured to capture an image from a person included in at least a part of a plurality of images based on an image capturing unit that captures a face image of a viewer and the face image captured by the image capturing unit. A viewer identification unit for identifying the person who is the viewer, and the image selection unit has an album intimacy between the viewer and the person identified by the viewer identification unit greater than or equal to a predetermined reference value You may select the image in which the person who is is.

  Further, in the second embodiment of the present invention, there is provided an electronic album display method for displaying a plurality of images included in an electronic album, wherein each of the plurality of images recognizes a plurality of persons included in the image. Based on the recognition results in the recognition stage and the person recognition stage for each of the plurality of images, the intimacy in the image, which is the intimacy between the persons included in the image, is determined between the images indicating each of the persons. Included in at least some of the plurality of images based on the in-image intimacy calculation stage calculated as a higher value as the distance is smaller and the in-image intimacy calculated for each of the plurality of images In-album intimacy calculation stage for calculating in-album intimacy, which is the intimacy between each person to be viewed, and in-album intimacy between viewers among multiple images Comprising an image selection step of selecting an image included is a predetermined reference value or more persons, and an image display step of displaying the image selected in the image selection step.

  According to a third aspect of the present invention, there is provided an electronic album display program for causing a computer to function as an electronic album display system for displaying a plurality of images included in an electronic album, the computer storing a plurality of images. An image storage unit, a person recognition unit for recognizing a plurality of persons included in the image, and a recognition result by the person recognition unit for each of the plurality of images. An in-image intimacy calculation unit that calculates an in-image intimacy that is an intimacy between included persons as a smaller value as the distance between images indicating each of the persons is smaller, and each of the plurality of images The intimacy between each of the persons included in at least some of the images based on the intimacy within the image calculated for An in-album intimacy calculation unit that calculates in-album intimacy and an image that includes a person whose album intimacy with the viewer is greater than or equal to a predetermined reference value among a plurality of images And an image display unit that displays an image selected by the image selection unit.

  According to the fourth aspect of the present invention, there is provided an image classification device that classifies a plurality of images, and includes an image storage unit that stores a plurality of images and each of the plurality of images. Based on the recognition result by the person recognition unit for recognizing a plurality of persons and the person recognition unit for each of the plurality of images, the intimacy in the image that is the intimacy between the persons included in the image is determined. Based on the in-image intimacy calculated for each of the plurality of images, the in-image intimacy calculation unit calculated as a higher value as the distance between the images indicating each is smaller, Intra-album intimacy calculation unit for calculating intimacy in album, which is an intimacy between each person included in at least some images, and in-album intimacy calculated by in-album intimacy calculation unit Based on the time, and an image classifying unit for classifying each of the plurality of images into a plurality of groups.

  For each of the plurality of images, the in-image intimacy calculation unit indicates the in-image intimacy between the persons included in the image, and the smaller the distance between the face images included in the image indicating each of the persons, It may be calculated as a higher value. The image classification unit classifies each of the plurality of images into a group for each range of intimacy within the album for one person included in at least some of the plurality of images of the person included in the image. Also good.

  Further, in the fifth aspect of the present invention, there is provided an image classification method for classifying a plurality of images, for each of the plurality of images, a person recognition stage for recognizing a plurality of persons included in the image, Based on the recognition result at the person recognition stage for each of the images, the intimacy within the image, which is the intimacy between the persons included in the image, becomes smaller as the distance between the images indicating the respective persons becomes smaller. Intra-image intimacy calculation stage calculated as a high value, and between each person included in at least some of the images based on the in-image intimacy calculated for each of the plurality of images In-album intimacy calculation stage for calculating in-album intimacy that is the intimacy in And an image classification step for classifying into a plurality of groups, respectively.

  According to a sixth aspect of the present invention, there is provided an image classification program that causes a computer to function as an image classification device that classifies a plurality of images, the computer including an image storage unit that stores a plurality of images, and a plurality of images A person recognition unit for recognizing a plurality of persons included in the image and a closeness between the persons included in the image based on a recognition result by the person recognition unit for each of the plurality of images. The in-image intimacy calculating unit calculates the in-image intimacy in the image as a higher value as the distance between the images representing the persons is smaller, and the in-image intimacy calculated for each of the plurality of images. In-album parent that calculates an in-album intimacy between each of the persons included in at least some of the images based on A degree calculating unit, on the basis of the album in intimacy degree calculated by the closeness calculation unit in the album, to function as an image classification apparatus and an image classifying unit for classifying each of the plurality of images into a plurality of groups.

  The above summary of the invention does not enumerate all the necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

  According to the present invention, an image desired by an image viewer can be automatically selected and displayed with high accuracy.

  Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the scope of claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

  FIG. 1 shows an example of an electronic album display system 10 according to the first embodiment of the present invention. The electronic album display system 10 includes an electronic album playback unit 20 and an image display unit 30, displays a plurality of images included in the electronic album, and allows a viewer 40 to appreciate the images. For example, the electronic album display system 10 may sequentially display the plurality of images as a so-called slide show while switching at predetermined time intervals.

  The electronic album playback unit 20 stores a plurality of images included in the electronic album. Here, the image may be either a still image or a moving image. Then, the electronic album playback unit 20 selects an image to be displayed from the plurality of images, plays back the selected image as an image signal, and outputs the image signal to the image display unit 30. The electronic album playback unit 20 may be, for example, a video playback device such as a DVD player that can play back still images or moving images. The image display unit 30 may be, for example, a television receiver, and receives and displays an image signal indicating an image selected by the electronic album playback unit 20.

The electronic album display system 10 according to the first embodiment of the present invention does not display all the images included in the electronic album, but is included in the images when only some images are selected and displayed. An object of the present invention is to automatically select and display an image that the viewer 40 desires to view automatically and accurately by selecting an image according to the intimacy between persons calculated based on the distance between the persons. And
In addition, the electronic album display system 10 according to the first exemplary embodiment of the present invention provides closeness between the owner 45 and the viewer 40 when the owner 45 of the electronic album and the viewer 40 are different. The purpose is to automatically select and display an image with high accuracy that the owner 45 desires the viewer 40 to view by selecting an image based on the image.

  FIG. 2 is a block diagram illustrating an example of a functional configuration of the electronic album display system 10 according to the first embodiment of the present invention. The electronic album playback unit 20 includes an image storage unit 200, a face image database 210, a person recognition unit 220, an in-image closeness calculation unit 230, an in-album closeness calculation unit 240, an imaging unit 250, a viewer identification unit 260, an image selection A unit 270 and an image processing unit 280.

  The image storage unit 200 stores a plurality of images included in the electronic album. The face image database 210 stores face images of a plurality of persons. For each of a plurality of images stored in the image storage unit 200, the person recognition unit 220 determines a plurality of persons included in the image based on the face images of the plurality of persons stored in the face image database 210. recognize. Then, the person recognizing unit 220 outputs the recognition result of the person in each image to the in-image closeness calculating unit 230, the in-album closeness calculating unit 240, and the image selecting unit 270.

  The in-image closeness calculation unit 230 calculates the distance between images indicating each of the persons included in the image based on the recognition result by the person recognition unit 220 for each of the plurality of images included in the electronic album. . Then, the in-image closeness calculation unit 230 calculates, for each of the plurality of images, the in-image closeness that is the closeness between the persons included in the image based on the distance between the images indicating the respective persons. To calculate. Specifically, the in-image familiarity calculation unit 230 calculates the in-image familiarity between persons included in the image as a higher value as the distance between images representing the persons is smaller. . Then, the in-image closeness calculation unit 230 outputs the in-image closeness between each person in each calculated image to the in-album closeness calculation unit 240. The in-album familiarity calculation unit 240 is based on the in-image familiarity calculated by the in-image familiarity calculation unit 230 for each of a plurality of images included in the electronic album, and at least a part of the plurality of images. The in-album intimacy, which is the intimacy between each person included in the image, is calculated. Then, the in-album familiarity calculation unit 240 outputs the calculated in-album familiarity to the image selection unit 270.

  The imaging unit 250 is a digital camera provided in the electronic album playback unit 20, for example, and captures the face image of the viewer 40. Then, the imaging unit 250 outputs the captured face image of the viewer 40 to the viewer identification unit 260. The viewer identification unit 260 is stored in the image storage unit 200 based on the face image of the viewer 40 captured by the imaging unit 250 and the face images of a plurality of persons stored in the face image database 210. The person who is the viewer 40 is identified from the persons included in at least some of the plurality of images. Then, the viewer identification unit 260 outputs the identification result of the person who is the viewer 40 to the image selection unit 270.

  The image selection unit 270 receives, from the person recognition unit 220, recognition results of a plurality of persons included in each of the plurality of images stored in the image storage unit 200. The image selection unit 270 also calculates an in-album familiarity between each person included in at least some of the plurality of images stored in the image storage unit 200. Receive from 240. Then, the image selection unit 270 is based on the recognition result of the person and the closeness in the album between the persons, and among the plurality of images stored in the image storage unit 200, the image selection unit 270 An image including a person whose intimacy within the album is equal to or greater than a predetermined reference value is selected. Here, based on the identification result received from the viewer identification unit 260, the image selection unit 270 recognizes which of the persons included in each of the plurality of images the viewer 40 is. It's okay. Then, the image selection unit 270 outputs the selected image to the image processing unit 280. Further, the image selection unit 270 can select an image based not only on the intimacy within the album but also on tag information (for example, tag information in an exchangeable image file format) attached to each image. In addition, when a plurality of images are selected, the reproduction order of the plurality of images may be determined. For example, the image selection unit 270 further selects an image in which the shooting date and time and the shooting point are included in a range designated by the viewer 40 or the like, among images whose intimacy within the album is equal to or higher than a reference value. Good. Further, for example, when selecting a plurality of images, the image selection unit 270 changes the reproduction order of the plurality of images in the order of closeness in closeness, the order of imaging date / time, or the order in which the imaging points are close to each other. Etc. may be determined. Furthermore, the image selection unit 270 may output the information indicating the determined reproduction order of the plurality of images in a file or the like. Accordingly, even in a device that does not have the image selection unit 270, such as a digital still camera, the plurality of images can be reproduced as a slide show according to the reproduction order.

  The image processing unit 280 performs image processing on the image selected by the image selection unit 270. Here, the image processing may be, for example, JPEG (Joint Photographic Experts Group) expansion processing, conversion processing to a video signal such as NTSC or PAL, and the like. Then, the image processing unit 280 outputs the image subjected to the image processing to the image display unit 30. In addition, when a plurality of images are selected by the image selection unit 270 and the reproduction order of the plurality of images is determined, the image processing unit 280 displays the plurality of images that have been subjected to image processing, The images may be sequentially output to the image display unit 30 according to the reproduction order. Then, the image display unit 30 displays the image received from the image processing unit 280 and allows the viewer 40 to appreciate it.

  When two persons are very intimate, an image is often captured with the two persons in close proximity. On the other hand, when the two persons are not very close, images are not often captured in the state where the two persons are close to each other. Thus, it can be said that the distance between persons when an image is captured is related to the intimacy between the persons. Here, according to the electronic album display system 10 according to the first embodiment of the present invention, based on the distance between the persons included in the image, the intimacy within the image that is the closeness between the persons included in the image Can be calculated, and based on the in-image intimacy in each of the images included in the electronic album, in-album intimacy that is the intimacy between persons in the entire image in the electronic album can be calculated . Then, it is possible to select and display an image including a person whose intimacy within the album with the viewer 40 is equal to or higher than a reference value. That is, the viewer 40 automatically selects an image including a person who is close to him / herself without inputting information on the closeness with the person included in the image into the electronic album display system 10. Can be displayed.

  Further, according to the electronic album display system 10, a face image of the viewer 40 is captured, and which person is the viewer 40 among a plurality of persons included in each of the images in the electronic album is recognized. be able to. Thus, the viewer 40 can easily view a desired image without registering his / her own information or performing a login procedure with respect to the electronic album display system 10.

  FIG. 3 shows an example of the image storage unit 200 according to the first embodiment of the present invention. The image storage unit 200 stores, for each of a plurality of images included in the electronic album, an image identifier for identifying the image and image data indicating the image in association with each other. Here, the image identifier may be a value determined to be unique for each image when the image storage unit 200 stores the image. In this figure, the image identifier is shown as a numerical value. Alternatively, the image identifier may be a character string such as a file name assigned to a file indicating image data.

  FIG. 4 shows an example of the face image database 210 according to the first embodiment of the present invention. The face image database 210 stores, for each of a plurality of persons, a person identifier for identifying the person and a face image of the person in association with each other. Here, for example, when a face image is stored in the face image database 210, the person identifier may be a value determined to be unique for each face image. The face image database 210 may receive and store face images from the person recognition unit 220. Specifically, when the person recognition unit 220 recognizes a person included in an image stored in the image storage unit 200, the face image of the person extracted from the image is stored in the face image database 210. If no similarity level higher than a predetermined reference value is shown for any face image, a face image of a person extracted from the image may be stored in the face image database 210 as a new face image. . Thereby, the face image database 210 can store the face images of a person included in at least some of the plurality of images stored in the image storage unit 200.

  FIG. 5 shows an example of a person recognition result by the person recognition unit 220 according to the first embodiment of the present invention. For example, the person recognizing unit 220 performs known image processing such as contour extraction processing and color distribution analysis processing on each of a plurality of images stored in the image storage unit 200, so that a plurality of images included in the image are included. Detect the subject. Then, the person recognizing unit 220 detects a characteristic amount specific to the face of the person from the detected image showing the subject, and when the subject is a person, the person's face is extracted from the image showing the subject. The partial image which shows is detected. Then, the person recognizing unit 220 compares the partial image indicating the detected human face with each of the plurality of face images stored in the face image database 210 shown in FIG. Each of a plurality of persons included in each stored image is identified. Then, as shown in FIG. 5, the person recognizing unit 220 detects a plurality of person identifiers indicating a plurality of persons included in the image for each of the plurality of images, and the image intimacy calculating unit 230, in the album This is output to the intimacy calculation unit 240 and the image selection unit 270. In addition, the person recognizing unit 220 uses, for each of the plurality of images, position information indicating the position in the image of the partial image indicating the person detected in the image or the face of the person, the in-image closeness calculating unit 230. Output to. Here, the position information may include, for example, the coordinate values of the pixels at the upper left corner and the lower right corner of the partial image indicating the person or the face of the person in each image.

  FIG. 6 shows an example of the calculation result of the in-image familiarity by the in-image familiarity calculation unit 230 according to the first embodiment of the present invention. The in-image closeness calculating unit 230 calculates an in-image closeness that is a closeness between persons included in the image for each of the plurality of images stored in the image storage unit 200. Specifically, the image intimacy calculation unit 230 calculates the image intimacy between persons included in the image for each of the plurality of images as the distance between the images indicating the persons decreases. It may be calculated as a higher value. For example, the in-image closeness calculation unit 230 calculates the in-image closeness so that the in-image closeness between persons included in each image is inversely proportional to the distance between images representing the respective persons. It's okay. In addition, the image intimacy calculation unit 230 calculates, for each of the plurality of images, the image intimacy between the persons included in the image, and the distance between the face images included in the image indicating each of the persons. A smaller value may be calculated as a higher value. The image intimacy calculation unit 230 receives a plurality of distances between images indicating each person or a distance between face images included in an image indicating each person from the person recognition unit 220. You may calculate based on the positional information which shows the position in the said image, and the area information which shows the magnitude | size in the said image of the partial image which shows the person in each of the image or the said person's face.

  When two people's faces are close to each other and an image is captured, the two people are often very close. On the other hand, when the two persons are not very close, images are not often captured with the faces of the two persons close to each other. Here, according to the electronic album display system 10 according to the first embodiment of the present invention, the intimacy within the image can be calculated based on the distance between the face images of the persons included in the image. The in-album familiarity that is the intimacy between persons in the entire image included in the album can be calculated with high accuracy.

  Further, the in-image familiarity calculation unit 230 has a distance between images indicating each of the persons included in the image for each of the plurality of images stored in the image storage unit 200 equal to or greater than a predetermined reference value. In this case, the closeness in the image between the persons may be calculated as a predetermined standard value. Here, the predetermined reference value and the predetermined standard value may be values specified in advance by the viewer 40 or the owner 45, for example.

  Although the distance between two persons and the intimacy between the two persons are related, at any distance, the closer the distance, the more intimate is not always. For example, there may be cases where there is no difference in the intimacy between the two persons when the two persons are 10 meters apart and when the two persons are 15 meters away. Here, according to the electronic album display system 10 according to the first embodiment of the present invention, when the distance between persons is equal to or greater than a predetermined reference value, the intimacy within the image between the persons Can be calculated as a predetermined standard closeness, so that the closeness within the album, which is the closeness between persons in the entire image included in the electronic album, can be calculated with high accuracy.

  FIG. 7 shows an example of the calculation result of the in-album familiarity by the in-album familiarity calculation unit 240 according to the first embodiment of the present invention. The in-album familiarity calculation unit 240 calculates the plurality of images based on the in-image familiarity between the respective persons included in the image calculated for each of the plurality of images stored in the image storage unit 200. The in-album intimacy, which is the intimacy between each person included in at least some of the images, is calculated. For example, as shown in the figure, the in-album familiarity calculation unit 240 calculates the in-album intimacy between the person whose person identifier is 0 and the person whose person identifier is 1 as 9.8. . Here, the in-album familiarity calculation unit 240 calculates the in-album intimacy between the first person and the second person included in at least some of the plurality of images. Among them, the higher the number of images including both the first person and the second person, the higher the calculation may be. For example, the in-album familiarity calculation unit 240 calculates the in-album intimacy between the first person and the second person, among the plurality of images, the first person and the second person. May be calculated by adding up the intimacy in the image between the first person and the second person calculated for each of the images including both of the images.

  The more frequently each of the two persons is included in the same image, the closer the relationship between the two persons is. Therefore, by calculating the intimacy in the album by adding the intimacy in the image, the intimacy between persons included in the images in the electronic album can be calculated with high accuracy.

  Then, the image selection unit 270 receives the viewer based on the person identifier indicating the person who is the viewer 40 received from the viewer identification unit 260 and the calculation result of the familiarity in the album as shown in the figure. The person identifier of the person whose in-album familiarity with 40 is more than a predetermined reference value is detected. Then, the image selection unit 270 selects an image including the detected person identifier based on the recognition result of the person included in each of the plurality of images stored in the image storage unit 200 by the person recognition unit 220. .

  The in-album familiarity calculation unit 240 also stores a plurality of images stored in the image storage unit 200 when the image storage unit 200 stores the imaging date and time of the image in association with each image. Based on the imaging date and time in each of the above, the temporal change in the familiarity in the album may be further calculated. Specifically, the in-album familiarity calculation unit 240 calculates, for each predetermined unit period such as one year or one month, an image calculated for each of a plurality of images captured within the unit period. Based on the inner familiarity, an in-album familiarity between each of the persons included in at least some of the plurality of images is calculated. Then, the in-album familiarity calculation unit 240 calculates a temporal change in the in-album intimacy between each person, calculated for each unit period. Here, the time change is calculated, for example, as a rate of change in intimacy within the album calculated for each of adjacent unit periods, specifically, “10% increase”, “50% decrease”, and the like. Good.

  Then, the image selection unit 270 displays a recent time change in the in-album familiarity with the viewer 40 among persons whose in-album intimacy with the viewer 40 is equal to or higher than a predetermined reference value. Alternatively, an image including other persons may be selected except for persons who show a reduction rate equal to or higher than a predetermined threshold. Here, the latest time change is the slowest plurality of units of a predetermined number by the viewer 40 or the like among a plurality of unit periods for which the intimacy in album is calculated by the intimacy in album calculation unit 240. It may be a time change calculated in the period. As a result, when the viewer 40 and another person have been intimate before but are no longer intimate, the image including the other person is not selected, and the intimacy is still present. Since an image including a person can be preferentially selected, an image desired by the viewer 40 to be viewed can be accurately selected and displayed.

  As described above, the electronic album playback unit 20 may describe information indicating the playback order of a plurality of images determined by the image selection unit 270 in a file. In each of a plurality of stored images, the recognition result of the person included in the image, the intimacy within the image between the persons included in the image, or each of the persons included in the image and the viewer 40 May be stored in the image file indicating the image as tag information based on the Exif format or the like. Thereby, when the process of recognizing a person included in the image or calculating the familiarity is executed again for each image, the time required for the process can be shortened.

  In addition, the electronic album display system 10 may perform each of the processes in the electronic album playback unit 20 described in FIGS. 1 to 7 in a plurality of different devices. Specifically, the electronic album display system 10 performs a person recognition process by the person recognition unit 220 in a digital still camera, an image intimacy calculation process by the image intimacy calculation unit 230, and an album intimacy. The calculation process of the familiarity in the album by the calculation unit 240 may be performed in a personal computer (PC). In this case, the digital still camera displays the person recognition result of the captured image by the person recognition unit 220 together with imaging information such as GPS information, color space information, exposure time, presence / absence of flash, shutter speed, etc. The tag information may be recorded on a recording medium in association with the image. The PC may read the information recorded on the recording medium to calculate the intimacy within the image and the intimacy within the album, select an image to be displayed, and determine the reproduction order of the images. . Then, the PC records the selected image and information indicating the reproduction order of the image on the recording medium and inputs the information to the digital still camera, thereby causing the digital still camera to select the selected image according to the reproduction order. Can be reproduced sequentially. The digital still camera may calculate not only the person recognition process but also the intimacy in the image and the intimacy in the album. In this case, the digital still camera may record the calculated in-image intimacy or in-album intimacy in association with the image as tag information.

  FIG. 8 is a flowchart illustrating an example of a process flow in the electronic album display system 10 according to the first embodiment of the present invention. In this example, the image storage unit 200 stores an image that is different from the viewer 40 and includes the owner 45 of the electronic album. First, for each of a plurality of images stored in the image storage unit 200, the person recognition unit 220 converts a plurality of persons included in the image into face images of a plurality of persons stored in the face image database 210. Based on the recognition (S1000). Subsequently, the image intimacy calculating unit 230 calculates the image intimacy, which is the intimacy between persons included in the image, based on the recognition result by the person recognizing unit 220 for each of the plurality of images. A smaller value is calculated as the distance between images representing each person is smaller (S1010). Subsequently, the in-album familiarity calculation unit 240 determines, based on the in-image intimacy calculated for each of the plurality of images, between each of the persons included in at least some of the plurality of images. The in-album intimacy which is the intimacy at is calculated (S1020).

  Subsequently, the image capturing unit 250 captures the face image of the viewer 40, and the viewer identification unit 260 includes the captured face image of the viewer 40 and the faces of a plurality of persons stored in the face image database 210. Based on the images, the person who is the viewer 40 is identified from the persons included in at least some of the plurality of images (S1030). Here, the electronic album display system 10 determines whether or not the viewer identifying unit 260 has identified the person who is the viewer 40 (S1040). When the person who is the viewer 40 is identified (S1040: Yes), the imaging unit 250 further captures the face image of the owner 45, and the viewer identification unit 260 further captures the owner of the imaged image. Based on the 45 face images and the face images of the plurality of persons stored in the face image database 210, the person who is the owner 45 from the persons included in at least some of the plurality of images Is identified (S1050). Here, the electronic album display system 10 determines whether or not the viewer identifying unit 260 has identified the person who is the owner 45 (S1060).

  When the person who is the owner 45 is identified (S1060: Yes), the image selection unit 270 is displayed by the image display unit 30 from the plurality of images stored in the image storage unit 200. A power image is selected (S1070). For example, the image selection unit 270 selects an image including a person whose album intimacy with the viewer 40 is greater than or equal to a predetermined reference value among a plurality of images stored in the image storage unit 200. You may choose. Here, the predetermined reference value may be, for example, a value designated by the viewer 40, or a standard value that is commonly determined for any viewer 40. Alternatively, it may be a value calculated according to the number of images desired to be viewed and designated by the viewer 40. In addition, for example, the image selection unit 270 selects an image including a person whose closeness with the viewer 40 is within a predetermined range from among a plurality of images stored in the image storage unit 200. You can do it. Here, the predetermined range may be a range input in advance by the viewer 40, for example.

  Further, the image selection unit 270 may select an image to be displayed based on the familiarity in the album between the owner 45 and the viewer 40. Specifically, the image selection unit 270 selects an image including a person whose intimacy in the album with the owner 45 is lower than the intimacy in the album between the owner 45 and the viewer 40. Good. Alternatively, the image selection unit 270 may select an image including only a person whose intimacy in the album with the owner 45 is lower than the intimacy in the album between the owner 45 and the viewer 40. Here, the image selection unit 270 recognizes which person is the owner 45 among the persons included in each of the plurality of images based on the identification result received from the viewer identification unit 260. Good. Subsequently, the electronic album display system 10 determines whether an image to be displayed has been selected (S1080). When an image is selected (S1080: Yes), the image display unit 30 displays the selected image (S1090).

  For example, in the case where the owner 45 is a person who is different from the viewer 40 as in the case where the owner 45 has a friend view the electronic album, the owner 45 views all the images included in the electronic album. It may not always be possible to show to the person 40. For example, when the viewer 40 is not a person who is very close to himself / herself, the owner 45 shows the viewer 40 an image including a person who is very close to himself / herself, in other words, a private image. Sometimes you do n’t want to. That is, the image that the owner 45 may view to the viewer 40 may change depending on the familiarity between the owner 45 and the viewer 40. Here, according to the electronic album display system 10 according to the first embodiment of the present invention, the electronic album is selected such that an image including a person whose degree of intimacy is lower than that between the owner 45 and the viewer 40 is selected. The image to be displayed can be automatically selected based on the familiarity between the owner 45 and the viewer 40 calculated from the included images. Thereby, the owner 45 can allow the viewer 40 to view a desired image without manually selecting an image that may be shown to the viewer 40 from among many included in the electronic album. .

  FIG. 9 is a block diagram showing an example of a functional configuration of the image classification device 50 according to the second embodiment of the present invention. The image classification device 50 includes an image storage unit 200, a face image database 210, a person recognition unit 220, an in-image familiarity calculation unit 230, an in-album familiarity calculation unit 240, a person selection unit 500, an image classification unit 510, and a classification storage. Part 520. The image classification device 50 classifies a plurality of images stored in the image storage unit 200 and stores them in the classification storage unit 520. Of the members provided in the image classification device 50 shown in this figure, members having the same reference numerals as those provided in the electronic album display system 10 shown in FIG. 2 correspond to those described in FIGS. Since it has substantially the same function and configuration as the member, the description is omitted except for differences.

  When classifying each of a plurality of images into a plurality of groups, the image classification device 50 according to the second embodiment of the present invention is based on the distance between the persons included in the image, and the closeness between the persons And classifying images based on the calculated closeness between persons.

  The person selection unit 500 selects one person from each of the persons included in at least some of the plurality of images stored in the image storage unit 200. Specifically, the person selecting unit 500 selects one face image from a plurality of face images stored in the face image database 210, and associates the face image database 210 with the one face image. Extract the stored person identifier. Then, the person selection unit 500 outputs information indicating the selected one person, that is, the extracted person identifier, to the image classification unit 510. The image classification unit 510 receives a recognition result of a plurality of persons included in each of the plurality of images stored in the image storage unit 200 from the person recognition unit 220. The image classification unit 510 also calculates an in-album familiarity between each of the persons included in at least some of the plurality of images stored in the image storage unit 200. Receive from 240. Then, the image classification unit 510 classifies each of the plurality of images stored in the image storage unit 200 into a plurality of groups based on the recognition result of the person and the closeness in the album between the persons. Specifically, the image classification unit 510 classifies each of the plurality of images into groups for each range in the album familiarity with respect to the person selected by the person selection unit 500 of the person included in the image. Then, the image classification unit 510 causes the classified storage unit 520 to store each of the classified images. The classification storage unit 520 stores each of the images stored in the image storage unit 200 according to the classification result by the image classification unit 510.

  According to the image classification device 50 according to the second embodiment of the present invention, based on the distance between the persons included in the image, the in-image closeness that is the closeness between the persons included in the image is calculated. In addition, the in-album familiarity that is the intimacy between persons in the entire image included in the electronic album can be calculated based on the in-image closeness in each of the images included in the electronic album. A plurality of images can be classified into a plurality of groups based on the calculated in-album familiarity. This makes it more practical based on the content of the image and the importance to the person viewing the image compared to the conventional case where the image is classified based only on the image information such as the imaging date and time and the imaging location. Image classification can be performed.

  Further, according to the image classification device 50, a plurality of images can be classified into groups for each range of intimacy within the album for the person selected by the person selection unit 500 such as the owner of the electronic album. As a result, an image having a higher degree of importance for the owner, such as an image including a person with a high degree of intimacy with respect to the owner, is extracted and processed into a DVD-Video or the like. The work of deleting and organizing electronic albums can be performed efficiently.

  FIG. 10 shows an example of the classification storage unit 520 according to the second embodiment of the present invention. The classification storage unit 520 classifies and stores each of the plurality of images into a plurality of groups (600, 610,...) According to the classification by the image classification unit 510. Specifically, the classification storage unit 520 stores information indicating the range of closeness in the album in association with each group, and the parent in the album with the person selected by the person selection unit 500 An image including a person whose density is included in the range is classified into the group and stored. Further, the classification storage unit 520 stores a plurality of pieces of information indicating images in association with each group. Here, the information indicating the image may include, for example, an image identifier for identifying the image and image data indicating the image.

  Note that the classification storage unit 520 may classify one image so as to overlap each of a plurality of groups. The classification storage unit 520 also recognizes the recognition result of the person included in the image, the intimacy within the image between the persons included in the image, or the album between the persons included in the image. Intimacy or the like may be stored in the image file indicating the image as tag information based on the Exif format or the like. Thereby, when the process of recognizing a person included in the image or calculating the familiarity is executed again for each image, the time required for the process can be shortened.

  FIG. 11 is a flowchart showing an example of the flow of processing in the image classification device 50 according to the second embodiment of the present invention. First, for each of a plurality of images stored in the image storage unit 200, the person recognition unit 220 converts a plurality of persons included in the image into face images of a plurality of persons stored in the face image database 210. Based on the recognition (S1100). Subsequently, the image intimacy calculating unit 230 calculates the image intimacy, which is the intimacy between persons included in the image, based on the recognition result by the person recognizing unit 220 for each of the plurality of images. A smaller value is calculated as the distance between images representing each person is smaller (S1010). In addition, the image intimacy calculation unit 230 calculates, for each of the plurality of images, the image intimacy between the persons included in the image, and the distance between the face images included in the image indicating each of the persons. A smaller value may be calculated as a higher value. Thereby, the familiarity between persons in the image can be accurately calculated.

  Subsequently, the in-album familiarity calculation unit 240 determines, based on the in-image intimacy calculated for each of the plurality of images, between each of the persons included in at least some of the plurality of images. The in-album intimacy that is the intimacy at is calculated (S1120). Subsequently, the person selection unit 500 selects one person from each of the persons included in at least some of the plurality of images stored in the image storage unit 200 (S1130). Here, the person selection unit 500, for example, when a plurality of face images stored in the face image database 210 are displayed in a list on a display device provided in the image classification device 50, the user selects A person corresponding to the selected face image may be selected. Further, for example, the person selection unit 500 is based on a user's face image captured by a digital camera or the like provided in the image classification device 50 and each of the face images stored in the face image database 210. A person who is a user may be detected from among persons included in at least some of the plurality of images, and the detected person may be selected. Subsequently, the image classification unit 510 classifies each of the plurality of images stored in the image storage unit 200 into a plurality of groups based on the recognition result of the person and the closeness in the album between the persons. Then, it is stored in the classification storage unit 520 (S1140).

  FIG. 12 is a block diagram showing an example of a hardware configuration of a computer 1500 according to the embodiment of the present invention. A computer 1500 according to an embodiment of the present invention includes a CPU peripheral unit having a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 connected to each other by a host controller 1582, and a host controller 1582 by an input / output controller 1584. I / O unit having a communication interface 1530, a hard disk drive 1540, and a CD-ROM drive 1560 connected to the I / O controller, and a legacy input having a ROM 1510, a flexible disk drive 1550, and an I / O chip 1570 connected to the I / O controller 1584. And an output unit.

  The host controller 1582 connects the RAM 1520 to the CPU 1505 and the graphic controller 1575 that access the RAM 1520 at a high transfer rate. The CPU 1505 operates based on programs stored in the ROM 1510 and the RAM 1520 and controls each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 and the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the communication interface 1530, the hard disk drive 1540, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The communication interface 1530 communicates with other devices via a network. The hard disk drive 1540 stores programs and data used by the CPU 1505 in the computer 1500. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program executed when the computer 1500 is started up, a program depending on the hardware of the computer 1500, and the like. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 via the RAM 1520. The input / output chip 1570 connects various input / output devices via a flexible disk drive 1550 and, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

  The electronic album display program or the image classification program provided to the hard disk drive 1540 via the RAM 1520 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The electronic album display program or the image classification program is read from the recording medium, installed in the hard disk drive 1540 in the computer 1500 via the RAM 1520, and executed by the CPU 1505. The electronic album display program or the image classification program installed and executed on the computer 1500 works on the CPU 1505 or the like to cause the computer 1500 to perform the electronic album display system 10, the electronic album playback unit 20 described with reference to FIGS. It functions as the image classification device 50.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1590 and the CD-ROM 1595, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a recording medium, and the program may be provided to the computer 1500 via the network.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

1 is a diagram illustrating an example of an electronic album display system 10 according to a first embodiment of the present invention. It is a block diagram which shows an example of a function structure of the electronic album display system 10 which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the image storage part 200 which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the face image database 210 which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the recognition result of the person by the person recognition part 220 which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the calculation result of the in-image closeness calculation part 230 by the in-image closeness calculation part 230 which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the calculation result of the familiarity in an album by the familiarity calculation part 240 in an album which concerns on the 1st Embodiment of this invention. It is a flowchart which shows an example of the flow of a process in the electronic album display system 10 which concerns on the 1st Embodiment of this invention. It is a block diagram which shows an example of a function structure of the image classification apparatus 50 which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the classification | category storage part 520 which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows an example of the flow of a process in the image classification apparatus 50 which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows an example of the hardware constitutions of the computer 1500 concerning embodiment of this invention.

Explanation of symbols

10 electronic album display system, 20 electronic album playback unit, 30 image display unit, 40 viewer, 45 owner, 50 image classification device, 200 image storage unit, 210 face image database, 220 person recognition unit, 230 closeness in image Calculation unit, 240 Intimacy in album calculation unit, 250 imaging unit, 260 viewer identification unit, 270 image selection unit, 280 image processing unit, 500 person selection unit, 510 image classification unit, 520 classification storage unit

Claims (13)

An electronic album display system for displaying a plurality of images included in an electronic album,
An image storage unit storing the plurality of images;
For each of the plurality of images, a person recognition unit that recognizes a plurality of persons included in the image;
Based on the recognition result by the person recognition unit for each of the plurality of images, the intimacy within the image, which is the intimacy between the persons included in the image, is determined by the distance between the images indicating each of the persons. The smaller the smaller, the in-image familiarity calculating unit that calculates as a higher value,
Based on the in-image intimacy calculated for each of the plurality of images, an in-album intimacy that is an intimacy between each of the persons included in at least some of the plurality of images. A closeness calculation unit in the album to be calculated;
An image selection unit that selects an image including a person whose intimacy in the album with a viewer is equal to or higher than a predetermined reference value among the plurality of images;
An electronic album display system comprising: an image display unit that displays an image selected by the image selection unit.   For each of the plurality of images, the in-image intimacy calculation unit calculates the in-image intimacy between persons included in the image, and the distance between face images included in the images indicating the persons. The electronic album display system according to claim 1, wherein a smaller value is calculated as a smaller value.   When the distance between images indicating each of the persons included in the image is greater than or equal to a predetermined reference value for each of the plurality of images, the intra-image closeness calculating unit The electronic album display system according to claim 1, wherein the intimacy in the image is calculated as a predetermined standard value. The image storage unit stores an image including an owner of the electronic album,
The electronic album according to claim 1, wherein the image selection unit selects an image based on an intimacy in the album between the owner and the viewer when the owner and the viewer are different. Display system.   The image selection unit selects an image including a person whose intimacy in the album with the owner is lower than the intimacy in the album between the owner and the viewer. The electronic album display system described. An imaging unit for imaging the viewer's face image;
A viewer identifying unit for identifying a person who is the viewer from a person included in at least some of the plurality of images based on a face image captured by the imaging unit;
The image selecting unit selects an image including a person whose in-album intimacy with the person who is the viewer identified by the viewer identifying unit is greater than or equal to a predetermined reference value. The electronic album display system according to 1. An electronic album display method for displaying a plurality of images included in an electronic album,
For each of the plurality of images, a person recognition stage for recognizing a plurality of persons included in the image;
Based on the recognition result in the person recognition stage for each of the plurality of images, the intimacy within the image, which is the intimacy between persons included in the image, is determined by the distance between images indicating each of the persons. The smaller the smaller, the in-image intimacy calculation stage to calculate as a higher value,
Based on the in-image intimacy calculated for each of the plurality of images, an in-album intimacy that is an intimacy between each of the persons included in at least some of the plurality of images. In-album intimacy calculation stage to calculate,
An image selection step of selecting an image including a person whose intimacy in the album with the viewer is equal to or higher than a predetermined reference value among the plurality of images;
An electronic album display method comprising: an image display step of displaying the image selected in the image selection step. An electronic album display program for causing a computer to function as an electronic album display system for displaying a plurality of images included in an electronic album,
The computer
An image storage unit storing the plurality of images;
For each of the plurality of images, a person recognition unit that recognizes a plurality of persons included in the image;
Based on the recognition result by the person recognition unit for each of the plurality of images, the intimacy within the image, which is the intimacy between the persons included in the image, is determined by the distance between the images indicating each of the persons. The smaller the smaller, the in-image familiarity calculating unit that calculates as a higher value,
Based on the in-image intimacy calculated for each of the plurality of images, an in-album intimacy that is an intimacy between each of the persons included in at least some of the plurality of images. Intimacy calculation part in album to calculate,
An image selection unit that selects an image including a person whose intimacy in the album with a viewer is equal to or higher than a predetermined reference value among the plurality of images;
An electronic album display program that functions as an electronic album display system, comprising: an image display unit that displays an image selected by the image selection unit. An image classification device for classifying a plurality of images,
An image storage unit storing the plurality of images;
For each of the plurality of images, a person recognition unit that recognizes a plurality of persons included in the image;
Based on the recognition result by the person recognition unit for each of the plurality of images, the intimacy within the image, which is the intimacy between the persons included in the image, is determined by the distance between the images indicating each of the persons. The smaller the smaller, the in-image familiarity calculating unit that calculates as a higher value,
Based on the in-image intimacy calculated for each of the plurality of images, an in-album intimacy that is an intimacy between each of the persons included in at least some of the plurality of images. Intimacy calculation part in album to calculate,
An image classification device comprising: an image classification unit that classifies each of the plurality of images into a plurality of groups based on the in-album familiarity calculated by the in-album familiarity calculation unit. For each of the plurality of images, the in-image intimacy calculation unit calculates the in-image intimacy between persons included in the image, and the distance between face images included in the images indicating the persons. The image classification device according to claim 9, wherein a smaller value is calculated as a higher value. The image classification unit is configured to convert each of the plurality of images for each range of intimacy in the album with respect to one person included in at least a part of the plurality of images of a person included in the image. The image classification device according to claim 9, wherein the image classification device is classified into groups. An image classification method for classifying a plurality of images,
For each of the plurality of images, a person recognition stage for recognizing a plurality of persons included in the image;
Based on the recognition result in the person recognition stage for each of the plurality of images, the intimacy within the image, which is the intimacy between persons included in the image, is determined by the distance between images indicating each of the persons. The smaller the smaller, the in-image intimacy calculation stage to calculate as a higher value,
Based on the in-image intimacy calculated for each of the plurality of images, an in-album intimacy that is an intimacy between each of the persons included in at least some of the plurality of images. In-album intimacy calculation stage to calculate,
An image classification method comprising: an image classification step of classifying each of the plurality of images into a plurality of groups based on the in-album familiarity calculated in the in-album familiarity calculation step. An image classification program that causes a computer to function as an image classification device that classifies a plurality of images,
The computer,
An image storage unit storing the plurality of images;
For each of the plurality of images, a person recognition unit that recognizes a plurality of persons included in the image;
Based on the recognition result by the person recognition unit for each of the plurality of images, the intimacy within the image, which is the intimacy between the persons included in the image, is determined by the distance between the images indicating each of the persons. The smaller the smaller, the in-image familiarity calculating unit that calculates as a higher value,
Based on the in-image intimacy calculated for each of the plurality of images, an in-album intimacy that is an intimacy between each of the persons included in at least some of the plurality of images. Intimacy calculation part in album to calculate,
An image classification program that functions as an image classification device, comprising: an image classification unit that classifies each of the plurality of images into a plurality of groups based on the in-album familiarity calculated by the in-album familiarity calculation unit.

Images