JP2012078461A - Image display device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device and program which is capable of flexibly switching a display image in accordance with user's emotional change.SOLUTION: The image display device displays images selected from a plurality of images at intervals of a preliminarily determined time, on a display unit by switching and includes: photographing means which photographs a range including a face position of a viewer viewing an image displayed on the display unit; expression recognition means which recognizes a face expression of the viewer on the basis of photographic data of the viewer obtained by the photographing means; setting means which sets a preference index showing viewer's degree of preference for the image displayed on the display unit, on the basis of a recognition result of the expression recognition means; preference degree storage means which stores the preference index set by the setting means in association with the image; and control means which selects an image to be displayed on the display unit, on the basis of preference indexes stored in the preference degree storage means.

Description

  The present invention relates to an image display device that sequentially switches and displays digital images, and a program.

  There is a digital photo frame (digital photo frame) in which digital images are sequentially switched and displayed. In this digital photo frame, when a plurality of image data is input to a memory or a storage device, these images can be displayed on the display of the digital photo frame in order or at random at predetermined intervals.

  However, if the images are displayed randomly or in a predetermined order, the photos that the user wants to see are not always displayed at the timing at which they want to see them. Rather, there is a problem in that the user's feelings may be impaired as a result of displaying a photograph that does not match the user's mood.

  Conventionally, as a technique for switching a plurality of image data in accordance with a viewer, there is an electronic signboard used in a store or a public place (for example, Patent Document 1). This digital signage system is equipped with a camera, and by photographing the face of the person watching the display and determining the gender and age, advertisement images of products targeted for the determined category are displayed on this display. There has been disclosed a technique for switching to another image when a viewer makes a boring expression while displaying or displaying an advertisement video.

JP 2005-267611 A

  However, the conventional image selection method switches according to the average preference of an unspecified number of people belonging to the same category in terms of age and gender, and therefore, a limited number of users like digital photo frames repeatedly browse In such a case, there is a problem that it is not possible to switch flexibly according to changes in the emotions of each user.

  An object of the present invention is to provide an image display device and a program capable of switching an image to be displayed flexibly according to a change in user's emotion.

In order to achieve the above object, the invention described in claim 1
In an image display device for switching an image selected from a plurality of images at predetermined time intervals to be displayed on a display unit,
Photographing means for photographing a range including a face position of a viewer who browses an image displayed on the display unit;
A facial expression recognition means for recognizing the facial expression of the viewer based on the photographed data of the viewer by the photographing means;
Setting means for setting a likability index indicating the likability of the viewer with respect to the image displayed on the display unit based on the recognition result by the facial expression recognition means;
Favorability storage means for storing the likability index set by the setting means in association with the image;
Control means for selecting an image to be displayed on the display unit based on the preference index stored in the preference storage means;
It has.

The invention according to claim 2 is the image display device according to claim 1,
The setting means includes the likability index for the image displayed before switching the image displayed on the display unit, and the facial expression recognized by the facial expression recognition unit before and after switching the image displayed on the display unit. Based on the direction of change, the preference index for the image displayed after the switching is set.

The invention according to claim 3 is the image display device according to claim 2,
The setting means includes the likability index for the image displayed before switching the image displayed on the display unit, and the facial expression recognized by the facial expression recognition unit before and after switching the image displayed on the display unit. Based on the degree of change, the preference index for the display image after the switching is set.

According to a fourth aspect of the present invention, in the image display device according to the first aspect,
The setting means is based on a comparison result between the degree of facial expression change recognized by the facial expression recognition means before and after the switching of the image displayed on the display unit, and the difference in the favorable index with respect to the images before and after the switching. The preference index for the display image after switching is changed.

The invention according to claim 5 is the image display device according to claim 1,
The setting means changes the likability index for the display image after the switching based on the facial expression change direction recognized by the facial expression recognition means before and after switching the image displayed on the display unit. It is said.

Invention of Claim 6 is an image display apparatus as described in any one of Claims 1-5,
The control means does not select the image to be displayed on the display unit when the likability indicated by the likability index associated with one image is lower than a predetermined level. .

The invention according to claim 7 is the image display device according to any one of claims 1 to 5,
The control means sets a selection range of the favorableness index based on the type of facial expression first recognized by the facial expression recognition means, and displays an image associated with the favorableness index within the selection range. It is characterized by being selected as an image to be displayed on the screen.

The invention according to claim 8 is the image display device according to claim 7,
The control means is characterized in that the selection range of the favorableness index is changed based on a cumulative change of the facial expression recognized by the facial expression recognition means.

Invention of Claim 9 is an image display apparatus as described in any one of Claims 1-8,
Browser information storage means for storing information used when identifying the registered viewer's face;
A viewer identifying means for identifying the viewer based on the photographed data of the viewer by the photographing means and the information stored in the viewer information storage means;
With
The setting means sets the preference index for the image displayed on the display unit for each viewer identified by the viewer identification means based on the recognition result by the facial expression recognition means,
The likability storage unit is characterized in that the likability index set by the setting unit is associated with image data of the image and is stored for each viewer identified by the viewer identification unit.

The invention according to claim 10 is:
A display unit that displays an image, and a photographing unit that captures a face position of a viewer who browses the image displayed on the display unit, and selected from a plurality of images at predetermined time intervals A computer used in an image display device that switches an image to be displayed on the display unit,
A facial expression recognition means for recognizing the facial expression of the viewer based on the photographed data of the viewer by the photographing means;
Setting means for setting a likability index indicating the likability of the viewer with respect to the image displayed on the display unit based on the recognition result by the facial expression recognition means;
Favorability storage means for storing the likability index set by the setting means in association with the image;
Control means for selecting an image to be displayed on the display unit based on the preference index stored in the preference storage means;
It is a program to make it function as.

  According to the present invention, in an image display device and a program for switching an image selected from a plurality of images at predetermined time intervals and displaying them on a display unit, a viewer of the display unit is photographed by a photographing unit, and a facial expression Setting means for setting a favorable index for the image displayed on the display unit based on the facial expression of the viewer recognized by the recognition unit, and an image to be displayed on the display unit based on the favorable index And a control means for selecting. Therefore, there is an effect that the image to be displayed can be switched flexibly according to the change of the user's emotion.

1 is an external view of a digital photo frame as an embodiment of the present invention. It is a block diagram which shows the internal structure of a digital photo frame. It is a figure explaining the favorable parameter of each image memorize | stored in the non-volatile memory. It is a flowchart which shows the procedure of the display control process performed when displaying an image sequentially on the display part of an image display apparatus. It is a flowchart which shows the procedure of the image selection process called and performed in a display control process. It is a flowchart which shows the example of the procedure of the favorableness setting process called and performed in a display control process. It is a figure explaining the favorable parameter of each image memorize | stored in the non-volatile memory in 2nd Embodiment. It is a flowchart which shows the procedure of the display control process in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is an external view of a digital photo frame as an image display apparatus according to the first embodiment.

  The digital photo frame 1 (hereinafter referred to as DPF 1) surrounds the display unit 2 and the side surface of the display unit 2 on the front surface as shown in the front view of FIG. A frame 3 and a camera 6 as a photographing means provided on the upper front side of the frame 3 are provided. Further, as shown in the side view of FIG. 1B and a slightly rear view, a box 4 is provided on the back surface of the frame 3 in contact with the frame 3. , An operation unit 8 including operation buttons 8a to 8d, a media insertion unit 9 for inserting a removable medium M, a power input terminal 11, a network connection terminal 10 for connecting to an external network, and the like. . In addition, a leg 5 and a speaker 7 are provided on the rear surface of the box 4.

  The display unit 2 is, for example, a liquid crystal display using a glossy liquid crystal panel. Alternatively, other display methods such as an organic EL (Electro-Luminescent) display may be used.

The camera 6 is for continuously photographing a user who browses an image displayed on the display unit 2 from the front of the DPF 1, and corresponds to the face position at the time of general browsing by the user. It is possible to photograph over the front angle α and the distances L _{0 to} L ₁ .

  The speakers 7 are respectively provided on the back side of the box 4 so as to be divided into left and right, and are constituted by a left speaker 7a and a right speaker 7b that output stereo sound. Alternatively, a single speaker may be provided so that only monaural sound can be output. From the speaker 7, it is possible to play, for example, a preset BGM for relaxing the user or music stored in the storage unit by the operation command from the operation unit 8. ing. Alternatively, when the user's facial expression photographed by the camera 6 is particularly steep, a BGM for automatically relaxing the user may be played.

  The operation buttons 8a to 8d provided on the operation unit 8 are, for example, a power switch, a menu switch, and a switch for switching to the next image or redisplaying the previous image. In addition, a switch for adjusting the volume of the speaker 7, an input changeover switch, and the like may be provided independently.

  The media insertion unit 9 is a slot for inserting an SD memory card, for example. Alternatively, various removable media such as a miniSD card, a microSD card, an MMC (multimedia card), and a USB flash memory can be attached and detached. The network connection terminal 10 is a terminal to which a LAN cable such as 100Base-TX can be connected.

  FIG. 2 is a block diagram showing the internal configuration of the DPF 1.

  Inside the box 4 of the DPF 1, there are a control unit 16 that performs control over the entire operation of the DPF 1, a power supply unit 14, a storage unit 15, a media I / F unit 12, an input / output control unit 13, and the like. Is provided. Further, the display unit 2, the camera 6, the speaker 7, and the operation unit 8 are further connected to the control unit 16.

  The power supply unit 14 supplies necessary power to each unit of the DPF 1 in addition to the control unit 16 based on power transmitted from an external power source connected to the power input terminal 11 via a power cable. Moreover, this power supply part 14 is good also as providing the storage battery which can supply electric power independently, for example, when the electric power sent from the outside interrupts temporarily.

  The storage unit 15 is a storage device that stores image data to be displayed on the display unit 2 and audio data output from the speaker 7. The storage unit 15 is, for example, a flash memory, a nonvolatile memory such as an SSD (Solid State Drive), or an HDD (Hard Disk Drive). The image data may include not only still image data (for example, JPEG) but also moving images (for example, MPEG).

  Alternatively, the display image and audio can be acquired from the removable medium M or an external network. The media I / F unit 12 is an interface that connects the removable medium M inserted into the media insertion unit 9 and the control unit 16. The input / output control unit 13 controls input / output of information exchanged with an external network via a LAN cable connected to the network connection terminal 10. The input / output control unit 13 may be configured to perform input / output control of information performed using wireless communication such as a wireless LAN or Bluetooth (registered trademark).

  In addition to a CPU (Central Processing Unit) 161 (expression recognition means, setting means, control means, viewer identification means) that executes control processing and calculations, the control unit 16 includes a RAM (Random Access Memory) 162, a ROM ( A read only memory (163) and an EEPROM (electrically erasable and programmable read only memory) 164 (favorability storage means, viewer information storage means) are provided.

  The RAM 162 provides a working memory space for the CPU 161 and stores temporary data. The ROM 163 stores various control programs 161a executed by the CPU 161 and initial setting data. The EEPROM 164 stores a preference table in which the preference parameters of each image set in the display control program executed by the CPU 161 are arranged. This likability table is overwritten and updated as necessary.

  FIG. 3 shows an example of the favorableness table stored in the EEPROM 164.

  This preference table 164a is stored in the removable medium M inserted in the storage unit 15 or the media insertion unit 9, or is listed as an image that can be acquired from an external network and can be displayed on the display unit 2. Each image data ID is stored in association with a viewer's preference parameter for these images. As this image ID, a unique number may be assigned in the DPF 1 or an image file name may be used as it is. The preference parameter is indicated by an integer of 1 to 10 in the DPF 1 of the present embodiment. Alternatively, positive and negative values such as −5 to +5 may be used, and the number of steps can be arbitrarily set. The favorable parameter of newly added image data that has never been viewed by the user may be “5” at a time, for example, or “6” for a warm color image, In the system image, it may be classified and set as “5”.

  In addition, the control unit 16 decodes encoded moving image data and audio data (for example, MP3) out of digital data stored in the storage unit 15 or the like into a data signal that can be output from the display unit 2 or the speaker 7. Is possible.

  Next, an image display control operation in the DPF 1 of the first embodiment will be described.

  FIG. 4 is a flowchart illustrating a procedure of display control processing executed by the CPU 161 of the control unit 16 in the DPF 1 of the first embodiment.

  This display control process is a process that is automatically started when the user turns on the power of the DPF 1, for example. When the display control process is started, first, the CPU 161 sends a signal to the camera 6 to start a camera photographing operation (step S101).

  The CPU 161 recognizes the user's facial expression based on the shooting data of the camera 6 and determines the initial facial expression at the start of the display control process (step S104). As the facial expression recognition technique, any conventionally known technique can be used. For example, by using FACS (Facial Action Coding System), an AU (Action Unit) that is a basic operation unit of facial expression is detected based on image data taken at a timing when a predetermined time has elapsed after switching display images. Then, the detected AU combinations are classified into facial expressions such as emotions and dislikes. In addition, based on the video recording data of the camera before and after switching the display image, for example, by recognizing the dynamic change of facial expression and the change of line of sight accompanying the change of emotion using, for example, a facial expression score, the image can be viewed It is also possible to identify more detailed emotions, such as classifying spontaneous smiles by smiles and smiles by external factors, and the direction and degree of the change. Further, it may be configured such that emotion changes can be read by reading a lip based on moving image data for a specific keyword such as a word indicating tongue or disgust.

  Next, the CPU 161 executes an image selection process to be described later based on the determined initial facial expression (step S105). The CPU 161 displays the image selected in this image selection process on the display unit 2 (step S106). At this time, when switching the display from the previous display image to the newly selected image, the CPU 161 switches the image after a predetermined time has elapsed since the start of displaying the previous image.

  Subsequently, the CPU 161 recognizes the change in the facial expression of the user before and after the switching of the image data based on the shooting data of the camera 6. Then, the CPU 161 quantifies the amount of change in facial expression indicating how positive the user's facial expression has changed (step S107). In this processing, the CPU 161 quantifies the degree of facial expression showing positive emotions such as fun, joyful, happy, gentle, and peaceful in the facial expression identification processing performed using the FACS or the like. This numerical value (facial expression parameter) is not particularly limited, but can be set, for example, in the same manner as the setting standard for the number of steps or step interval of the favorable parameter for each image. Then, a difference value between the user's facial expression parameter for the previous display image and the user's facial expression parameter for the current display image is acquired as a facial expression change amount.

  Then, based on the acquired facial expression change value, the CPU 161 performs a preference setting process described later, and sets the value of the user's preference parameter for the displayed image (step S108). The CPU 161 overwrites and updates the original favorable parameter value stored in the EEPROM 164 with the newly set favorable parameter value (step S109).

  CPU161 returns to the process of step S105 after progress of predetermined time, and repeats the process of step S105-S109. This display control process is ended when, for example, the user presses the power switch or sets an end timer to turn off the power, or presses the menu switch to execute another command.

  Next, the image selection process called in the display control process will be described.

  FIG. 5 is a flowchart showing the control procedure of the image selection process.

  When the image selection process is called, the CPU 161 acquires initial facial expression data and determines which type of facial expression it is. When it is determined that the initial facial expression is a pleasant facial expression (“YES” in step S201), the CPU 161 first sets the initial favorable parameter of the first selection target image to “9” (step S202). Then, in order to keep the user's feeling in a good state, the CPU 161 determines the image to be displayed on the display unit 2 next time from the range of the favorable parameter “5” to “10”, in particular, the image with the favorable parameter “9”. The image is selected by setting a weight such that is preferentially selected for each favorable parameter (step S203). Then, the process of the CPU 161 returns to the display control process every time an image is selected.

  When it is determined that the initial facial expression is an angry facial expression (“NO” in step S201, “YES” in step S211), the CPU 161 first sets the initial favorable parameter of the first selection target image to “5”. Alternatively, “6” is set (step S212). Then, the CPU 161 sets the range of the favorable parameter of the image to be selected for the second and subsequent times (for example, the favorable parameter “5” to “7” in the initial stage), and increases and decreases the favorable parameter every time. While selecting the image, the image is selected (step S213). Further, the CPU 161 gradually increases the range of the favorable parameter of the image to be selected as the facial expression parameter increases while confirming the change in the facial expression of the user. Then, the process of the CPU 161 returns to the display control process every time an image is selected.

  When it is determined that the initial facial expression is a sad expression (“NO” in steps S201 and S211, “YES” in step S221), the CPU 161 first sets the initial favorable sensitivity parameter of the first selection target image to “6”. "Or" 7 "is set (step S222). Then, the CPU 161 preferentially selects the image of the favorable parameter that matches the facial expression parameter while checking the change in the facial expression of the user every time. Further, the CPU 161 slowly increases the range of the favorable parameter of the image to be selected as the facial expression parameter increases (step S223). Then, the process of the CPU 161 returns to the display control process every time an image is selected.

  If it is determined that the initial facial expression is not one of the above, for example, if it is a normal facial expression without any special emotional expression or no facial expression (“NO” in steps S201, S211 and S221) ") First, the CPU 161 sets the initial favorable parameter of the first selection target image to" 7 "(step S232). Then, in order to improve the user's mood, the CPU 161 selects an image as the next display image from the range of “5” to “9” in which the favorable parameter is centered on the favorable parameter “7”. An image is selected after weighting is performed for each favorable parameter. Further, the CPU 161 changes this weight so that an image with a large favorable parameter is gradually selected preferentially as the number of times the image selection process is called increases (step S233). Then, the process of the CPU 161 returns to the display control process every time an image is selected.

  Note that the above image selection process is continuously repeated based on the initial facial expression determined at the start of the display control process. For example, when the facial expression changes greatly from the initial facial expression, The facial expression may be reset, or the data of the initial facial expression may be valid for a predetermined time regardless of the contents of determination.

  Next, the favorableness setting process that is called and executed in the display control process will be described.

  FIG. 6 is a flowchart showing various examples of the preference setting process.

  FIG. 6A shows an example 1 of the favorableness setting process. When this favorable sensitivity setting process is called, the CPU 161 sets the value of the expression change amount as a positive sensitivity parameter of the currently displayed image to a value obtained by adding to the positive sensitivity parameter of the previously displayed image. The data in the table 164a is updated. Then, the process returns to the display control process.

  FIG. 6B is an example 2 of the preference setting process. When this preference setting process is called, the CPU 161 first determines whether or not the value of the expression change amount before and after switching to the current display image is positive (step S311). When it is determined that the expression change amount is positive, that is, when it is determined that the user's mood has been improved, the process proceeds to step S312 and the CPU 161 displays the image of the favored parameter of the currently displayed image previously displayed. Is set to a value larger by 1 than the favorable sensitivity parameter (step S312). Then, the process of the CPU 161 returns to the display control process. On the other hand, when it is determined that the facial expression change amount is not positive, that is, when it is determined that the user's mood has not been improved, the process proceeds to step S313, and the CPU 161 sets the favorability parameter of the currently displayed image to the previous value. The value is set to a value smaller by 1 than the favorability parameter of the displayed image. Then, the process of the CPU 161 returns to the display control process.

  FIG. 6C illustrates an example 3 of the favorable sensitivity setting process. When this preference setting process is called, the CPU 161 determines that the difference between the user's preference parameter for the current display image and the preference parameter for the previously displayed image is a switch from the previous display image to the current display image. It is determined whether or not the expression change amount before and after is larger (step S321). If it is determined that the difference in the favorable parameter is larger than the expression change amount, the process branches to “YES”, and the CPU 161 sets the positive parameter of the display image to a value obtained by subtracting 1 from the positive parameter of the previous image. (Step S322). Then, the process of the CPU 161 returns to the display control process. On the other hand, if it is determined that the difference in the favorable parameter is smaller than the expression change amount, the process branches to “NO”, and the CPU 161 sets the positive parameter of the display image to the positive parameter of the previous image. Is set to the added value (step S323). Then, the process of the CPU 161 returns to the display control process. Here, when the original favorable parameter of the display image is larger than the value of the positive parameter that increases by the above setting, or when it is smaller than the value of the favorable parameter that decreases by the above setting, the display image It is good also as not changing the favorable parameter with respect to to the calculated setting value.

  FIG. 6D is an example 4 of the favorable sensitivity setting process. When this preference setting process is called, the CPU 161 determines whether or not the value of the facial expression change amount before and after switching to the current display image is positive (step S331). When it is determined that the expression change amount is positive, that is, when it is determined that the user's mood has been improved, the process proceeds to step S332, and the CPU 161 increments the value of the favorable sensitivity parameter of the currently displayed image by one. Let Then, the process of the CPU 161 returns to the display control process. On the other hand, when it is determined that the expression change amount is not positive, that is, when it is determined that the user's mood has not been improved, the CPU 161 decreases the favorable parameter of the currently displayed image by 1 (step S333). . Then, the process of the CPU 161 returns to the display control process.

  FIG. 6E is a fifth example of the favorable sensitivity setting process. When this preference setting process is called, the CPU 161 determines that the difference between the user's preference parameter for the current display image and the user's preference parameter for the previously displayed image is changed from the previous display image to the current display image. It is determined whether or not the facial expression change amount before and after the switching is greater than the value (step S341). If it is determined that the difference in the favorable parameter is larger than the expression change amount, the process branches to “YES”, and the CPU 161 decreases the value of the positive parameter of the currently displayed image by 1 (step S342). Then, the process of the CPU 161 returns to the display control process. On the other hand, if it is determined that the difference in the favorable parameter is smaller than the expression change amount, the process branches to “NO”, and the CPU 161 increases the value of the positive parameter of the currently displayed image by 1 (step). S343). Then, the process of the CPU 161 returns to the display control process.

  As described above, according to the DPF 1 of the first embodiment, the face of the user viewing the image displayed on the display unit 2 is photographed by the camera 6 provided on the surface of the DPF 1, and the photographed image data is included in the photographed image data. The user's facial expression can be recognized based on the recognition result, and the user's preference for the display image can be set based on the recognition result. Then, the parameter indicating the set likability can be stored in the EEPROM 164 and used as a selection criterion for an image to be displayed for the user. A suitable image can be selectively displayed on the display unit 2.

  In addition, the user's browsing is performed because the responsiveness parameter for the display image after the switching is set every time based on the favorability parameter for the display image before the switching of the image depending on whether or not the facial expression has changed positively by the switching of the image. An image suitable for the user's mood can be selectively displayed on the display unit 2 by quickly reflecting the relative favorable sensitivity corresponding to the mood of the time.

  In addition, with the preference parameter for the display image before switching the image as a reference, by setting the preference parameter for the display image after switching based on the amount of change in facial expression before and after the image switching, the previous display is performed. In this case, even if the image is preferable, it is possible to control the display by changing the favorable parameter as occasion demands.

  Also, comparing the amount of change in facial expression before and after switching the displayed image with the difference in favorable parameter set in the image before and after switching, the amount of facial expression change is more than the difference in positive parameter If it is larger, change the favorability parameter for the display image after switching so as to increase the difference in the favorability parameter, and if the difference in the favorability parameter is larger than the amount of expression change, the favorability parameter Because it is possible to change the preference for the display image after image switching so as to reduce the difference between the images, by setting a preference parameter that appropriately reflects the amount of change in the relative expression on the display image, An image more suitable for the user's mood can be selected.

  In addition, since the favorable parameter set in the image after switching is increased / decreased based on the direction of the expression change before and after switching the display image, it is easier for the image to be changed in a direction that improves the user's mood. High likability parameters can be set.

  Further, since it is possible to exclude an image having a low likability parameter from selection of a display image, the image can be viewed comfortably without making the user feel uncomfortable. In particular, an image given by another person can be displayed on the display unit 2 by selecting a preferable image for the user without taking time to organize data.

  Also, it is suitable for maintaining a pleasant mood by determining the user's facial expression at the start of image display and setting the range of favorable parameters of the image to be displayed afterwards based on each emotional expression It is possible to improve the user's mood by appropriately selecting the image, the image suitable for refreshing the sad feeling and the image suitable for refreshing, or the image suitable for changing the mood to relax.

  In addition, the range of likability parameters for selecting display images is changed sequentially as each user's facial expression changes from the initial facial expression, so that an image that fits the user's mood can be displayed smoothly. You can continue.

[Second Embodiment]
Next, a DPF as an image display device according to the second embodiment will be described. In the DPF according to the second embodiment, a plurality of users can select and display an image according to each favorable parameter and expression for each image. The configuration of the DPF 1 in the second embodiment is the same as the configuration of the DPF 1 in the first embodiment shown in FIG. 1 and FIG.

  FIG. 7 is a diagram for explaining the favorable parameter of each image stored in the EEPROM 164 in the DPF 1 of the second embodiment.

  As shown in FIGS. 7A and 7B, the EEPROM 164 stores two tables. As shown in FIG. 7A, one table is a likability table 164b that stores likability parameters for each image (1001) of each of a plurality of users (P01 to P03). The other table is a user registration table 164c, and as shown in FIG. 7B, a user ID for identifying a plurality of users is associated with data indicating the characteristics of face data corresponding to each user. Saved.

  FIG. 8 is a flowchart showing a procedure of display control processing executed by the CPU 161 in the DPF 1 of the second embodiment. The same processing as the display control processing in the DPF 1 of the first embodiment shown in FIG.

  In the display control processing in the second embodiment, after starting camera shooting (step S101), the CPU 161 performs identification processing of a user who views the display image of the DPF 1 based on shooting data from the camera 6 (step S102). . The CPU 161 extracts the facial characteristic parameters included in the photographic data, compares them with the facial feature data of each user stored in the user registration table 164c, and extracts the characteristic parameters extracted from the registered users. Identifies users with matching parameters.

  When the user is identified by the processing in step S102, the CPU 161 designates a column corresponding to the user's favourability parameter from the favourability table 164b based on the identified user ID of the user, and sets the necessary preference. Sensitivity parameter data is acquired (step S103). Then, the process of the CPU 161 proceeds to step S104.

  Further, when the favorable parameter for the display image is set in the favorable sensitivity setting process in step S108, the CPU 161 updates the value of the favorable parameter for the display image in the column corresponding to the user in the favorableness table 164b ( Step S109a). Then, the process of the CPU 161 returns to step S105.

  As described above, according to the DPF 1 of the second embodiment, the faces of a plurality of users can be identified based on the captured images of the camera 6. In addition, since the preference parameter can be set for each user in the EEPROM 164, even when a plurality of users use the DPF 1, an image is selected for each recognized user and displayed on the display unit 2. Since it can be displayed, an image unpleasant for each user can be eliminated, and an appropriate image suitable for the mood can be selected and displayed.

  The present invention is not limited to the above-described embodiment, and various modifications can be made. For example, in the above embodiment, the scalar value is set as the favorable parameter, but the present invention is not limited to this. For example, by setting multiple parameters such as pleasure value and relaxation value, and distinguishing positive enjoyment and pleasure from stability such as calmness and relaxation based on 2D and 3D vector values The selection may be controlled more finely. Further, the preference parameter is not limited to a numerical value, and for example, an alphabet may be used.

  In the above-described embodiment, the user's visible light image is captured by the camera 6. However, for example, the user's heat generation condition may be added to the expression determination material using the infrared captured image.

  In the above embodiment, one image is selected and displayed each time. However, a plurality of images may be combined and displayed.

  Further, in the above embodiment, an image whose likability is lowered to 4 or less will not be displayed thereafter. For example, for an image that has not been displayed for a predetermined time interval or more, only when the user's facial expression is very enjoyable. May be displayed again. Or it is good also as a structure which can delete the image with very low favorable sensitivity from the memory | storage part 15. FIG.

  In the above embodiment, the value of the likability parameter is set based on the relative change in the user's facial expression. However, the likability parameter may be set based on the absolute value of the facial expression parameter.

  Further, the display image selection method shown in the above embodiment is an example, and the method for setting the range of the favorable parameter of the image to be selected each time and the weighting method within this range can be arbitrarily set. .

  In the above embodiment, four patterns of fun, anger, sadness, no expression and others are set as the classification of the initial facial expression. However, the classification is not limited to this classification pattern. More classification patterns may be provided, and conversely, display images may be selected based on a certain standard without classification.

  In the second embodiment, a plurality of users are registered in the user registration table 164c, and an individual likability table 164b is set for each user. Registration may be set by an input from the operation unit 8, or may be automatically registered for a user who has been detected a predetermined number of times. Conversely, registration settings for users who are not detected for a certain period or longer may be deleted. In addition, when it is determined that the viewer of the display unit 2 is not registered in the user registration table 164c, for example, the average value of the likability parameter that is set and the amount of change in the facial expression of the viewer The display image selection control is performed based on this, and the data of the likability table may not be updated. Alternatively, the display image selection control is not performed from the beginning, and the display image selection is completely random. It is also good to do.

  In the description of the above embodiment, an example in which the ROM 163 is used as a computer-readable medium for the program according to the present invention is disclosed, but the present invention is not limited to this example. As other computer-readable media, a non-volatile memory such as a flash memory and a portable recording medium such as a CD-ROM can be applied. A carrier wave is also applied to the present invention as a medium for providing the program data according to the present invention via a communication line.

  In the above-described embodiment, a digital photo frame has been described as an example of the image display device. However, the present invention is not limited to this. For example, you may apply to portable apparatuses, such as a mobile phone, PDA (Personal Digital Assistant), and a smart phone. Alternatively, it can be used as an image display function of a PC or video device to which a camera and a display are connected. In addition, details such as the specific configuration, arrangement, and numerical values shown in the above embodiment can be changed as appropriate without departing from the spirit of the invention.

1 Digital Photo Frame (DPF)
2 Display unit 3 Frame 4 Box 5 Leg 6 Camera 7 Speaker 7a Left speaker 7b Right speaker 8 Operation unit 8a-8d Operation button 9 Media insertion unit 10 Network connection terminal 11 Power input terminal 12 Media I / F unit 13 Input / output Control unit 14 Power supply unit 15 Storage unit 16 Control unit 161 CPU
161a program 162 RAM
163 ROM
164 EEPROM
164a, 164b Favorability table 164c User registration table M Removable media

Claims (10)

In an image display device for switching an image selected from a plurality of images at predetermined time intervals to be displayed on a display unit,
Photographing means for photographing a range including a face position of a viewer who browses an image displayed on the display unit;
A facial expression recognition means for recognizing the facial expression of the viewer based on the photographed data of the viewer by the photographing means;
Setting means for setting a likability index indicating the likability of the viewer with respect to the image displayed on the display unit based on the recognition result by the facial expression recognition means;
Favorability storage means for storing the likability index set by the setting means in association with the image;
Control means for selecting an image to be displayed on the display unit based on the preference index stored in the preference storage means;
An image display device comprising: The setting means includes the likability index for the image displayed before switching the image displayed on the display unit, and the facial expression recognized by the facial expression recognition unit before and after switching the image displayed on the display unit. The image display apparatus according to claim 1, wherein the preference index for the image displayed after the switching is set based on a change direction. The setting means includes the likability index for the image displayed before switching the image displayed on the display unit, and the facial expression recognized by the facial expression recognition unit before and after switching the image displayed on the display unit. The image display apparatus according to claim 2, wherein the preference index for the display image after switching is set based on the degree of change. The setting means is based on a comparison result between the degree of facial expression change recognized by the facial expression recognition means before and after the switching of the image displayed on the display unit, and the difference in the favorable index with respect to the images before and after the switching. The image display apparatus according to claim 1, wherein the preference index for the display image after switching is changed. The setting means changes the likability index for the display image after the switching based on the facial expression change direction recognized by the facial expression recognition means before and after switching the image displayed on the display unit. The image display device according to claim 1. The control means does not select the image as an image to be displayed on the display unit when the likability indicated by the likability index associated with one image is lower than a predetermined level. The image display apparatus as described in any one of Claims 1-5. The control means sets a selection range of the favorableness index based on the type of facial expression first recognized by the facial expression recognition means, and displays an image associated with the favorableness index within the selection range. The image display device according to claim 1, wherein the image display device is selected as an image to be displayed on the screen. The image display apparatus according to claim 7, wherein the control unit changes the selection range of the favorableness index based on a cumulative change of the facial expression recognized by the facial expression recognition unit. Browser information storage means for storing information used when identifying the registered viewer's face;
A viewer identifying means for identifying the viewer based on the photographed data of the viewer by the photographing means and the information stored in the viewer information storage means;
With
The setting means sets the preference index for the image displayed on the display unit for each viewer identified by the viewer identification means based on the recognition result by the facial expression recognition means,
The likability storage unit stores the favorability index set by the setting unit in association with image data of the image for each viewer identified by the viewer identification unit. The image display apparatus as described in any one of 1-8. A display unit that displays an image, and a photographing unit that captures a face position of a viewer who browses the image displayed on the display unit, and selected from a plurality of images at predetermined time intervals A computer used in an image display device that switches an image to be displayed on the display unit,
A facial expression recognition means for recognizing the facial expression of the viewer based on the photographed data of the viewer by the photographing means;
Setting means for setting a likability index indicating the likability of the viewer with respect to the image displayed on the display unit based on the recognition result by the facial expression recognition means;
Favorability storage means for storing the likability index set by the setting means in association with the image;
Control means for selecting an image to be displayed on the display unit based on the preference index stored in the preference storage means;
Program to function as.

Images