JP2013196417A - Image display device, image display method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device for supporting image selection for recommending an image which is suitable for the sensitivity of a person who selects an image.SOLUTION: A control part 7 of an image display device 1 displays a taste information setting screen 701 at a display part 17, and a user is made to designate his or her desired human face, its priority order, and its expression from among the images of read image data 60, and they are stored in taste information 70. The control part 7 calculates the featured values of face images included in the whole images of the image data 60, and the desired human face stored in the taste information 70 is identified, and a degree of recommendation D of each image is calculated from the presence/absence of the desired human face, similarity E with the desired expression, and the value of the priority order about each image, and the image having the value of the degree of recommendation D which is larger than a threshold TH is displayed at the display part 17 as a recommended image 90.

Description

  The present invention relates to an image display device that supports image selection.

  As digital camera performance improves and recording media such as memory cards for storing photographs and the like increase in price and capacity, the tendency to capture and collect many images is increasing. As a result, it is a burden to pick up a favorite image according to the purpose such as printing from many images stored in the recording medium.

  For this reason, a method for selecting a good image and a method for selecting a good image have been proposed (for example, Patent Document 1 and Patent Document 2).

  Patent Document 1 is an invention relating to an imaging apparatus, in which a feature amount of a favorite face is registered in advance, a plurality of images are captured, and a registered facial feature is selected from the plurality of captured images. An imaging apparatus that determines and records an image including a subject close to the amount has been proposed.

  Patent Document 2 discloses a technique for quickly searching and displaying a plurality of images, and proposes a method of selecting an image based on the smile level of the face of the subject located at the center of the screen.

JP 2006-115406 A JP 2009-205322 A

However, the method of Patent Document 1 is a technique provided in an imaging apparatus, and there is a problem that face images other than registered favorite face images are not recorded.
The method of Patent Document 2 selects an image with a high degree of smile based on the assumption that a smile image is good. However, the expression that is judged to be good differs depending on the person who chooses, and the smile is not necessarily good. .

FIG. 14 is a diagram illustrating an example of selecting an image from a plurality of images.
As shown in FIG. 14A, the operation of selecting an image that seems to be good from the images 1 to 10 (731 to 740) in which a plurality of subjects are photographed is burdened unexpectedly.
FIG. 14B shows an example in the case where smile is selected as a determination criterion, and images (image 8, image 9, and image 10) in which all subjects are smiling are selected.

However, the criteria for selection are not limited to smiles, and may vary depending on the user and may select an image in which a specific person is shown.
FIGS. 14C and 14D are examples in which a specific facial expression of a specific person is used as a criterion. That is, it is desirable that a specific person (A, B, C) is captured, and an image close to the condition that the person A is smiling, the person B is surprised, and the person C is a true face is, for example, Images 6, 7, and 10 are different from the images selected in FIG.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an image display device that supports image selection that can recommend an image that matches the sensitivity of a person who selects an image. To do.

  In order to achieve the above-described object, the first invention includes a first storage unit that stores a plurality of image data, a desired person's face, the priority order of the desired person, and the desired person's priority. Face designating means for designating a desired facial expression and storing it in the second storage unit together with the facial feature value of the desired person, and all the faces included in the image data stored in the first storage unit A feature value calculating means for calculating a feature value and a facial expression, a facial feature value included in the image data calculated by the feature value calculating means, and the desired person stored in the second storage unit The feature value of the face is collated to identify whether or not the face is the desired person, and the face included in the image data by the identifying means is the face of the desired person The facial expression included in the image data Facial expression distance calculating means for calculating a difference in desired facial expression; presence or absence of the face of the desired person for each of the image data; difference in facial expression calculated by the facial expression distance calculating means; and the priority An image display comprising: a recommendation unit that calculates a recommendation level of the image data based on the ranking and recommends the image data as recommended image data when the recommendation level is equal to or higher than a predetermined value. Device.

  According to the first invention, it is possible to specify a person desired by the user and a desired facial expression of the person's face by the face designating means, and to the first storage unit by the recommendation means. Image data including a face close to a desired expression of a desired person can be extracted from a plurality of stored image data, and an image suitable for the user's preference can be recommended. .

  In addition, it is possible to designate a plurality of people with priority by the face designating means, and by calculating the recommendation degree based on the priority order by the recommending means, It is possible to extract image data including many faces close to a desired facial expression, and it is possible to recommend an image that matches the user's preference.

  A second invention is an image display method for extracting and displaying recommended image data from a plurality of image data stored in a first storage unit, wherein the computer has a desired human face and the desired image A face designation step of designating the priority order of the person and the facial expression desired by the desired person, and storing it in the second storage unit together with the feature value of the face of the desired person, and storing in the first storage unit A feature value calculating step for calculating feature values and facial expressions of all the faces included in the image data, a face feature value included in the image data calculated by the feature value calculating step, and the second storage A face value included in the image data by the identification step of collating the feature value of the face of the desired person stored in the section and identifying whether the face is the face of the desired person Is identified as the face of the desired person A facial expression distance calculation step for calculating a difference between the facial expression included in the image data and the desired facial expression, the presence or absence of the desired human face, and the facial expression for each of the image data. The degree of recommendation of the image data is calculated based on the difference in facial expression calculated by the distance calculation step and the priority order, and the image data is recommended as recommended image data when the recommendation degree is equal to or greater than a predetermined value. An image display method characterized by executing a recommendation step.

  According to the second invention, it is possible to designate the person desired by the user and the desired facial expression of the person's face with priority by the face designation step, and priority by the recommendation step. It is possible to extract image data containing many faces close to a desired facial expression of a person with high ranking and recommend an image that meets the user's preference.

  According to a third aspect of the invention, the computer is configured to designate the first storage unit that stores a plurality of image data, the face of the desired person, the priority order of the desired person, and the desired facial expression of the desired person. , Face designation means for storing the desired person's facial feature value in a second storage unit, and all facial feature values and facial expressions included in the image data stored in the first storage unit. A feature value calculating means to calculate; a face feature value included in the image data calculated by the feature value calculating means; and a desired person face feature value stored in the second storage unit; And the identification means for identifying whether or not the face is the desired person, and when the face included in the image data is identified by the identification means as the face of the desired person , Facial expressions included in the image data, and the desired expression Expression distance calculation means for calculating a difference in emotion, for each image data, the presence or absence of the face of the desired person, the difference in expression calculated by the expression distance calculation means, and the priority order A program for calculating a recommendation level of the image data and causing the image display device to function as a recommendation unit that recommends the image data as recommended image data when the recommendation level is equal to or higher than a predetermined value. .

  According to the third invention, it is possible to extract image data including a lot of faces close to a desired facial expression of a person with high priority, and to recommend an image that matches the user's preference.

  According to the present invention, it is possible to provide an image display device that supports image selection that can recommend an image that matches the sensitivity of the person who selects the image.

FIG. 2 is a block diagram illustrating a hardware configuration example of the image display device 1. 4 is a flowchart showing a flow of processing of the image display device 1. The flowchart which shows the process of step 102 in detail. The figure which shows the example 701 of preference information setting screens. The flowchart which shows the process of step 103 and step 104 in detail. The figure which shows the characteristic value 73 of a to-be-photographed object. The figure which shows the feature value 65 about the face of all the images. The figure which shows the image list 71 of a to-be-photographed object. The flowchart which shows the process of step 105 in detail. The flowchart which shows the process of step 507 in detail. The figure which shows the distance table 80 of a facial expression. The figure explaining the selection process of a recommendation image. The figure which shows the image list display example 801 after recommendation image determination. The figure explaining the conventional image selection.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of an image display device and the like according to the invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, the same reference numerals are given to components having substantially the same functional configuration, and redundant description will be omitted.

FIG. 1 is a block diagram illustrating a hardware configuration example of the image display device 1.
The image display device 1 may be, for example, a computer such as a personal computer, or may be an image print system capable of printing digital camera image data recorded on a recording medium brought in by a user. May be.

  In the image display device 1, a control unit 7, a storage unit 9, a media input / output unit 11, a communication control unit 13, an input unit 15, a display unit 17, a peripheral device I / F unit 19, and the like are connected via a bus 21. .

The control unit 7 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.
The CPU calls and executes a program stored in the ROM, the storage unit 9 and the like in a work memory area on the RAM, drives and controls each device connected via the bus 21, and realizes processing performed by the computer.
The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like.
The RAM is a volatile memory, and temporarily stores a program, data, and the like loaded from the storage unit 9, ROM, storage medium, and the like, and includes a work area used by the control unit 7 to perform various processes.

The storage unit 9 is an HDD (hard disk drive), and stores a program executed by the control unit 7, data necessary for program execution, an OS (Operating System), and the like. Regarding programs, a control program corresponding to the OS, application programs, files, and the like are stored.
These program codes are read by the control unit 7 as necessary, transferred to the RAM, and executed as various means by the CPU.
The program and data for executing the image display apparatus 1 according to the present embodiment of the present invention are stored in the storage unit 9 and read and executed by the control unit 1 as necessary.

The media input / output unit 11 is a drive device that inputs / outputs data of a recording medium. For example, a floppy (registered trademark) disk drive, a CD drive (-ROM, -R, -RW, etc.), a DVD drive (-ROM). , -R, -RW, etc.) and a media input / output device such as an MO drive.
The communication control unit 13 includes a communication control device, a communication port, and the like, and is a communication interface that mediates communication via the network 4 and controls communication with other computers.

The input unit 15 inputs data and includes, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad. An operation instruction, an operation instruction, data input, and the like can be performed on the computer via the input unit 15.
The display unit 17 includes a CRT (Cathode Ray Tube) monitor, a display device such as a liquid crystal panel, and a logic circuit (such as a video adapter) for realizing a video function in cooperation with the display device.
The peripheral device I / F unit 19 is a USB (Universal Serial Bus) port or the like for connecting peripheral devices.
The bus 21 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

FIG. 2 is a flowchart showing the flow of processing of the present embodiment according to the present invention.
First, the control unit 7 of the image display device 1 reads the image data 60 and stores it in the storage unit 9 (step 101). The image data 60 is composed of a plurality of images as shown in FIG. 14A, for example, and is read from a memory card or the like via the media input / output unit 11 of the image display device 1, for example. Or you may make it read from the other computer apparatus via the communication control part 13 of the image display apparatus 1 via networks, such as the internet.

Next, the control part 7 makes a user set preference information (step 102). The preference information is a person desired when the user selects an image, its priority, and a desired facial expression of the person's face. Step 102 will be described in detail later.
Next, the control unit 7 analyzes the face image (face image) included in each image with respect to the image data 60 stored in the storage unit 9 in step 101 (step 103).
Step 103 will be described in detail later.

Next, the control unit 7 identifies each face image based on the analysis result of the face image in step 103, and identifies the face set as the preference information in step 102 from each image of the image data 60 (step 104).
Step 104 will be described in detail later.

Next, the control unit 7 extracts, as a recommended image, an image including a face image close to the face image of the user preference information set by the user in Step 102 from the plurality of images of the image data 60 (Step 105). The method for determining the recommended image in step 105 will be described in detail later.
The control unit 7 displays the recommended image determined in step 105 on the display unit 17 (step 106), and ends the process.

  Next, the processing of step 102 (setting preference information) in FIG. 1 will be described. FIG. 3 is a flowchart showing the processing flow of step 102, and FIG. 4 is an example of a preference information setting screen 701.

First, the control unit 7 displays the image data 60 stored in the storage unit 9 in step 101 of FIG. 1 on the display unit 17 so that the face portion can be selected (step 201).
As shown in FIG. 4B, for example, a thumbnail of the image data 60 is displayed on the image data display unit 703 of the preference information setting screen 701.

In the example shown in FIG. 4B, the image data display unit 703 displays images 1 to 6 (731 to 736) as thumbnail images so that faces included in the images 1 to 6 can be selected.
In order to display the face part in a selectable manner, it is necessary to extract the face position and area. The extraction of the face position and area is described in a known technique, for example, Japanese Patent Laid-Open No. 2005-165447. Can be used.

  As shown in the image data display unit 703 in FIG. 4B, the control unit 7 displays each image 1 to 6 (731 to 736) face area extracted by a known method, for example, by enclosing it with a rectangle, By placing the cursor 711 on the rectangle and clicking, the face in the face area can be selected.

Next, the control unit 7 causes the user to select a desired subject with a priority order (step 202).
For example, as shown in FIG. 4A, it is assumed that the user desires an image in which people A, B, and C are shown. Furthermore, it is assumed that the user desires the priority order of the people in the order of people A, B, and C.

The control unit 7 may cause the user to select the face of the desired person by placing the cursor 711 in the order of priority, or to select the face area where the cursor 711 is placed on the left side of the preference information setting screen 701. You may make it drag to the position of the desired priority of the object part of the preference information data 71 in the part.
Through the above-described processing for selecting a desired face, desired subjects are selected and displayed as preference information data 71 on the preference information setting screen 701 in order of priority.

Next, the control unit 7 causes the user to select a desired facial expression for the selected subject (step 203).
As shown in FIG. 4, the user selects a desired facial expression for desired persons A, B, and C. As shown in FIG. 4A, for example, the user displays a smiling expression for the person A, a surprised expression for the person B, and a facial expression selected from the image data display unit 703 for the person C. Assume that it is desired.

As shown in FIG. 4B, the control unit 7 displays a desired facial expression in the preference information data 71 portion of the preference information setting screen 701 so as to be selectable. For example, it is possible to select from among “fun”, “sad”, “surprise”, “anger”, “true face”, and “selected facial expression”.
The user only has to place the cursor on the facial expression to be selected and click.

  By selecting the desired facial expression, for example, as shown in FIG. 4B, the “fun” facial expression is given to the person A having the priority “1” and the “surprise” is given to the person B having the priority “2”. “Selected facial expression” is selected as the preference information data 71 for the person C whose priority is “3”.

The control unit displays a “registration” button 709, an “other photo” button 705, and the like in addition to the preference information setting screen 701.
By selecting the “Register” button 709, preference information data 71 including priority order, subject, and desired facial expression is stored as preference information 70 in the storage unit 9.
Further, the “other photo” button 705 is selected by the user when there is no desired person or desired expression in the image data display unit 703, so that the control unit 7 stores the other items stored in the storage unit 9. Similarly to the image 1 to image 6 (731 to 736), the thumbnail image of the image data 60 is displayed so that the face area can be selected.

  Through the processing in steps 201 to 203 (step 102 in FIG. 1) described above, the control unit 7 selects a plurality of desired human faces with priorities, and further selects the desired person for each desired person. Can be selected. Thereby, it is possible to extract an image according to the user's preference.

  Next, the processing of step 103 (face image analysis of image data) in FIG. 1 will be described. FIG. 5A is a flowchart showing the flow of processing in step 103, and FIGS. 6 and 7 are diagrams showing examples of feature values as face image analysis results.

First, the control unit 7 analyzes the face image of the subject selected in Step 102, calculates a feature value, and stores it as the feature value 73 of the target subject in the preference information 70 of the storage unit 9 (Step 301).
That is, the feature value is extracted for the face image of the subject (A, B, C) selected by the user in the preference information data 71 stored as the preference information 70 in the storage unit 9.

  The feature value of the face image is, for example, information such as face area coordinates, eye position coordinates, nose position coordinates, mouth position coordinates, gender, age, smile accuracy, and the like, for example, JP 2009-294925 A In addition to JP 2005-165447 A and JP 2007-336124 A, Face. A publicly known method such as API (Application Program Interface) of face recognition technology publicly disclosed by Com may be used. For example, the following method may be used.

  First, a face area is detected from a face image by a face area detection unit, and face feature information is extracted by a face feature extraction unit. In addition, personal facial feature information for a wide range of age groups by gender is created in advance and stored in the facial feature holding unit together with age and gender information, and the facial feature information and facial features extracted by the facial feature extraction unit The similarity is obtained by collating with the personal face feature information in the holding unit. What is necessary is just to discriminate | determine the age and sex of the said person from the information of the obtained similarity and the age and sex attached to it.

  The accuracy and expression of smiles can be quantified based on how the mouth bends, how the mouth corners rise, the size of the eyes, the degree of wrinkles, and the like. By comparing with a preset threshold value based on the quantified value, it can be determined how much the subject image is laughing.

FIG. 6 is a diagram showing the feature value 73 of the face image of the subject selected in step 102.
By analyzing the face image of the subject (A, B, C) in step 301, for example, for each subject (A, B, C), for example, face area coordinates, eye position coordinates, nose position coordinates, mouth position coordinates Feature values such as the estimated age, smile accuracy (Smiling), and gender are extracted and stored in the preference information 70 of the storage unit 9 as feature values.

  Further, for example, the smile accuracy (Smiling) of the subject subjects 1 to 3 (A, B, C) is 43%, 85%, 50%, etc., respectively, and the higher the smile accuracy, the more smiling the subject. Show.

Here, the “desired facial expression” selected by the user in the preference information selection process (step 102 in FIG. 2) is stored as the facial expression (mod) value of the feature value 73 of the target subject.
FIG. 6C is a table showing the correspondence between the desired facial expression options and the value of the expression of the feature value 73 of the target subject. For example, “fun” corresponds to “happy”, “surprise” corresponds to “surprised”, and “selected facial expression” corresponds to “feature value of selected image”.

  As shown in FIG. 6A, the expression (mod) value of the subject 1 (A) is “happy” because the preference information data 71 is selected as “happy”, and the value of the subject 2 (B). The value of the expression (mod) is “surprise” because “surprise” is selected in the preference information data 71, and the value of the expression (mod) of the subject 2 (C) is “selected” in the preference information data 71. Since “expression” is selected, it is “feature value of selected image”.

Next, the control unit 7 analyzes all face images included in each image stored in the image data 60 of the storage unit 9, extracts feature values of each face image, and features values about the faces of all the images. 65 is stored in the storage unit 9 (step 302).
The analysis method of the face image is the same as in step 301.

FIG. 7 is a diagram illustrating an example of the feature value 65 for the faces of all images.
As shown in FIG. 7, the feature values 65 for the faces of all images include an image ID, a face ID, a gender, a facial expression (mod), a smile accuracy (smiling), a face area coordinate, an eye position coordinate, Nose position coordinates, mouth position coordinates, estimated age, etc.

For example, since there are five face images in image 1 with image ID 1, face IDs, Img1-1, Img1-2,..., Img1-5 are assigned to each face. For the ID, the extracted feature value is stored.
Through the above steps 301 and 302 (step 103 in FIG. 2), the feature value 73 of the target subject selected by the user and the feature value 65 of the face of all images are extracted and stored in the storage unit 9.

  Next, the processing of step 104 (personal identification) in FIG. 1 will be described. FIG. 5B is a flowchart showing the flow of processing in step 104, and FIG. 8 is a diagram showing a subject image list 75 which is a result of individual identification.

  The control unit 7 selects the feature values of all face images (face IDs: Img1-1, Img1-2,...) Of all the images (image IDs 1, 2,...) Extracted in step 103 and the user. The target feature value 73 of the target subject is collated, a face corresponding to the target subject is extracted from all the face images, and is stored in the storage unit 9 as a subject image list 75 (step 401).

  As a method for extracting face information from image data and collating with face information registered in advance, Non-Patent Document 1 (“Face Recognition System Using Face Image”, Communication Technique PRMU97-50,) 1997, for example. June, Yamaguchi, Fukui, Maeda) and Japanese Patent Application Laid-Open No. 2003-141542 may be used.

FIG. 8 is a diagram illustrating an example of the subject image list 75.
As shown in FIG. 8, the target subject 1 (person A) selected by the user is identified as Img2-2 of image 2, Img5-1 of image 5, Img6-1 of image 6, Subject 2 (person B) is Img1-2 in image 1, Img6-4 in image 6, and target subject 3 (person C) is Img1-1 in image 1, Img2-3 in image 2, and Img6 in image 6. 2 is included.

  The other data shown in FIG. 8, that is, for example, face area coordinates, eye / nose / mouth position coordinates, age, gender, mood, smiley accuracy, This is the face information value corresponding to the feature value 65 for the faces of all the images calculated in step 302 in FIG.

  Next, the processing of step 105 (determination of recommended images) in FIG. 1 will be described. FIGS. 9 and 10 are flowcharts showing the flow of processing in step 105, and FIGS. 11 and 12 are diagrams for explaining a recommended image determination method.

  In the recommended image determination process (step 105), for all the images 1, 2,... Of the image data 60, an image including more face images of desired facial expressions of a plurality of target subjects with priority is recommended. This is a process for determining an image.

First, the control unit 7 initializes a variable n of the image ID to 1 (step 501). Further, the control unit 7 initializes the value of the recommendation degree D (n) used for selecting the recommended image for the image of the image IDn to 0 (step 502).
Further, the control unit 7 initializes the variable m indicating the ID of the target subject selected by the user (step 503).

Next, the control unit 7 determines whether or not the target subject m exists in the image n (step 504). If the target subject m exists in the image n (Yes in step 504), the temporary variable G is set to 1 (step 505), and if the target subject m does not exist in the image n (No in step 504), the temporary variable G is temporarily set. The variable G is set to 0 (step 506).
For example, since the target subject 1 does not exist in the image 1, G = 0, and since the target subject 2 and the target subject 3 exist, G = 1.

After the target subject m exists in the image n and G = 1 is set in step 505, the control unit 7 is included in the facial expression of the target subject m included in the image n and the target subject characteristic value 73 of the preference information 70. The similarity E of the expression (mod) is calculated (step 507).
As an example of facial expression similarity calculation processing (step 507), FIG. 10 shows a flowchart showing a detailed processing flow.

The method of calculating the similarity E of facial expressions in step 507 uses the desired facial expression as the “feature value of the selected image” as the preference information 70, or other facial expressions (“fun”, “sad”, “surprise”, “anger” ”Or“ true face ”).
First, the control unit 7 determines whether or not the “mod (expression)” value of the feature value 73 of the target subject is the “feature value of the selected image” (step 601).
In the example of the target subject characteristic value 73 shown in FIG. 6, the “mod” value of the target subject 3 is the “feature value of the selected image”, and other target subjects 1 and 2 have other facial expressions. It is.

If step 601 is No, the facial expression similarity E is determined according to the facial expression similarity table 80 of FIG. 11 (step 602).
As shown in FIG. 11, the expression similarity degree table 80 sets the expression similarity in the preference information 70 and the value of 0 to 5 as the expression similarity E according to the value of the mood (expression) of the face image included in the image n. Given.

  For example, if the expression setting of the preference information 70 is “fun” and the mod value of the face image included in the image n is “happy”, the expression similarity E = 5, and the expression setting of the preference information 70 is “fun”. Thus, if the value of the mood of the face image included in the image n is “nuetal”, the expression similarity E = 3.

On the other hand, when step 601 is Yes, for example, the similarity E of facial expressions is calculated based on the value of smile accuracy (Smiling) (step 603).
FIG. 12A shows an example of an expression for calculating the expression similarity E in step 603.
A value of 5 to 0 is calculated based on the difference between the value of Smile of the target subject m and the value of Smile of the face image included in the image n by the equation of FIG. The greater the difference, the closer the facial expression similarity E is to zero.

Next, the control unit 7 determines the weight W from the priority order value of the target subject m in the preference information 70 (step 508). This is set so that the recommendation degree D (n) increases as the target subject m having a higher priority is included in the image n. For example, the weight W is calculated according to the equation shown in FIG. Is calculated.
According to the calculation formula of the weight W shown in FIG. 12B, the weight W of the target subject 1 with the first priority is 3, the weight of the target subject 2 with the second priority is 2, and the priority is 3 The target subject 3 has a weight of 1.

Next, the control unit 7 calculates a value of G × E × W from the temporary variable G obtained in steps 505 and 506, the similarity E of the facial expression obtained in step 507, and the weight W obtained in step 508. Is the value of the temporary variable G (step 509).
Further, the control unit 7 adds the value of the temporary variable G obtained in step 509 to the recommendation degree D (n) of the image n to obtain the recommendation degree D (n) (step 510).

  Next, the control unit 7 increments the variable m of the target subject by 1 (step 511), and when the value of the variable m is equal to or less than the value of the target subject number M (Yes in step 512), the processing from step 504 to step 512 is performed. repeat. As a result, a value indicating which target subject of the M target subjects is included in the image n and how close the facial expression is to the facial expression information of the preference information 70 is the recommendation degree D (n). Calculated.

When step 512 is No (when the calculation of the recommendation level D (n) for the M target subjects is completed), the value of the recommendation level D (n) is compared with the value of a predetermined threshold TH (step 513).
When the recommendation degree D (n) ≧ the threshold value TH (Yes in Step 513), the image n is held as a recommended image (Step 514).

  After the processing of step 514 and when the recommendation level D (n) <threshold value TH (No in step 513), the value of n is incremented by 1 to perform the next image n + 1 (step 515), When the value of n is less than or equal to the total number N of images included in the image data 60 (Yes in step 516), the process returns to step 502 and the processes in steps 502 to 516 are repeated. When the processing is completed for the total number N of images (No in step 516), the processing in step 105 is terminated.

FIG. 12C illustrates a recommended image determination example 89.
For the image ID (n = 1 to 6,...), A temporary variable G is obtained from the presence / absence (1 or 0) of the subject m (m = 1 to 3), the similarity E of the facial expression, and the weight W. The sum of the values of temporary variables G with m = 1 to 3 is shown as the value of D (n).

For example, in the case of the images 3, 4, and 5, the target subjects 1 to 3 are not included, and as a result, the recommendation degree D (n) is 0.
Further, for example, in the case of the image 2, the target subjects 1 and 3 are included, and in the case of the target subject 1, the expression similarity E is “fun” − “happy” in the expression similarity table 80 of FIG. 11. When E = 5 and the target subject 3, the Smile value (50%) of the feature value 73 of the target subject shown in FIG. 6 and the target subject shown in FIG. E = 4.7 based on the Milling value (44%) of the image 2 of 3, and D (2) = 19.7 by the weight W.
Similarly, when the value of D (6) is calculated for the image 6 as well, D (6) = 29.65.

  Here, if the threshold value TH = 15, the image 2 and the image 6 become the recommended images 90.

Finally, the control unit 7 displays the recommended image 90 on the display unit 17 in step 106 in FIG.
FIG. 13 is a diagram illustrating an example of a display screen of the recommended image 90 on the display unit 17.
In the example of FIG. 13, all images are displayed in order from the image 1, and the recommended image is marked with a “★” mark to indicate that it is a recommended image.

  In the example of FIG. 13, a specific mark is added to the recommended image 90 to indicate that it is a recommended image. However, the method of presenting the recommended image 90 is not limited thereto. For example, only the recommended image 90 may be separately displayed on the display unit 17.

As described above, according to the present invention, instead of simply recommending a smiling face as a good image, an image including a lot of face information close to the specified facial expression of the person specified as the preference information 70 based on the user's preference. Can be extracted as a recommended image, and a more detailed recommended image can be extracted.
In addition, by assigning priorities to the designation of people in the preference information 70, it is possible to extract, as a recommended image, an image including face information close to a designated facial expression of a designated person with a high priority.

  As mentioned above, although embodiment of this invention was described referring an accompanying drawing, the technical scope of this invention is not influenced by embodiment mentioned above. It is obvious for those skilled in the art that various modifications or modifications can be conceived within the scope of the technical idea described in the claims, and these are naturally within the technical scope of the present invention. It is understood that it belongs.

  For example, in the present embodiment, the value of smile similarity (Smiling) or the value of mod (the types of facial expressions such as “fun” and “sad”) are used as a method of calculating the similarity E of facial expressions. Although the value of the similarity E is calculated according to the equation of a) or the table of FIG. 11, the value of the equation of FIG. 12A or the table of FIG. 11 is not limited to this, and other equations Other values may be used.

DESCRIPTION OF SYMBOLS 1 ......... Image display apparatus 7 ......... Control part 9 ......... Storage part 17 ......... Display part 60 ......... Image data 70 ......... Preference information

Claims (3)

A first storage unit for storing a plurality of image data;
A face designating unit for designating a desired person's face, a priority order of the desired person and a desired facial expression of the desired person, and storing them in a second storage unit together with a feature value of the desired person's face; ,
Feature value calculating means for calculating feature values and facial expressions of all faces included in the image data stored in the first storage unit;
The feature value of the face included in the image data calculated by the feature value calculation means is collated with the feature value of the face of the desired person stored in the second storage unit, and the desired person Identification means for identifying whether or not the face is,
When the identification means identifies that the face included in the image data is the face of the desired person, the facial expression included in the image data and an expression for calculating the difference between the desired expression A distance calculating means;
For each of the image data, the degree of recommendation of the image data is calculated based on the presence or absence of the face of the desired person, the difference in facial expression calculated by the facial expression distance calculation means, and the priority order, Recommending means for recommending the image data as recommended image data when the degree of recommendation is a predetermined value or more;
An image display device comprising: An image display method for extracting and displaying recommended image data from a plurality of image data stored in a first storage unit,
Computer
A face designating step of causing a desired person's face, a priority order of the desired person and an expression desired by the desired person to be designated, and storing them in a second storage unit together with a feature value of the desired person's face;
A feature value calculating step of calculating feature values and facial expressions of all the faces included in the image data stored in the first storage unit;
The feature value of the face included in the image data calculated by the feature value calculation step is collated with the feature value of the desired person's face stored in the second storage unit, and the desired person An identification step for identifying whether the face is
When the identification step identifies that the face included in the image data is the face of the desired person, an expression for calculating the difference between the facial expression included in the image data and the desired expression A distance calculation step;
For each of the image data, based on the presence / absence of the face of the desired person, the difference in facial expression calculated by the facial expression distance calculation step, and the priority, the recommendation degree of the image data is calculated, A recommendation step of recommending the image data as recommended image data when the recommendation level is a predetermined value or more;
The image display method characterized by performing. Computer
A first storage unit for storing a plurality of image data;
A face designating unit for designating a desired person's face, a priority order of the desired person and a desired facial expression of the desired person, and storing them in a second storage unit together with a feature value of the desired person's face; ,
Feature value calculating means for calculating feature values and facial expressions of all faces included in the image data stored in the first storage unit;
The feature value of the face included in the image data calculated by the feature value calculation means is collated with the feature value of the face of the desired person stored in the second storage unit, and the desired person Identification means for identifying whether or not the face is,
When the identification means identifies that the face included in the image data is the face of the desired person, the facial expression included in the image data and an expression for calculating the difference between the desired expression A distance calculating means;
For each of the image data, the degree of recommendation of the image data is calculated based on the presence or absence of the face of the desired person, the difference in facial expression calculated by the facial expression distance calculation means, and the priority order, Recommending means for recommending the image data as recommended image data when the degree of recommendation is a predetermined value or more;
A program for causing an image display apparatus to function as an image display apparatus.

Images