JP2010067104A - Digital photo-frame, information processing system, control method, program, and information storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a digital photo-frame, an information processing system, a control method, a program and an information storage medium for achieving an interface environment which is highly convenient for a user. <P>SOLUTION: A digital photo-frame includes: a display part for displaying an image; a display control part for performing the display control of the display part; and a detection information acquisition part for acquiring detection information detected by a sensor. The display part is configured of a touch panel display. The display control part performs display control to change the display configurations of an operation icon for a touch panel according to the obtained detection information. The display configurations of the operation icon for the touch panel are changed according to at least one of a position relation between a user and the display part, the visualizing state of the user, the utterance state of the user, the recognition result of the number of persons of the user, and the person identification result of the user. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a digital photo frame, an information processing system, a control method, a program, an information storage medium, and the like.

  In recent years, a digital photo frame has been spotlighted as a device that can easily reproduce an image taken by a digital camera such as a digital still camera. This digital photo frame is a device in which a portion for putting a photo on a photo stand is replaced with a liquid crystal display, and performs reproduction processing of digital image data (electrophotography) read via a memory card or a communication device. .

  As a conventional technique of a digital photo frame, for example, there is a technique disclosed in Patent Document 1. In this prior art, a telephone line connection device is provided in a digital photo stand, which is a digital photo frame, to form a transmission path between the photo stand and a wired or wireless telephone line.

  However, conventional digital photo frames simply reproduce a series of images stored in the memory, and do not have an appropriate interface for a user to select a display image, change a display method, or the like. It was. Therefore, the reproduced image is monotonous and an interface environment that is easy to use cannot be provided to the user.

JP 2000-324473 A

  According to some aspects of the present invention, it is possible to provide a digital photo frame, an information processing system, a control method, a program, and an information storage medium that can realize an interface environment that is highly convenient for the user.

  The present invention includes a display unit that displays an image, a display control unit that performs display control of the display unit, and a detection information acquisition unit that acquires detection information detected by a sensor, and the display unit is a touch panel display. The display control unit relates to a digital photo frame that performs display control for changing a display mode of the operation icon for the touch panel according to the acquired detection information. The present invention also relates to a program that causes a computer to function as each of the above-described units, or a computer-readable information storage medium that stores the program.

  According to the present invention, the display unit of the digital photo frame is configured by a touch panel display. As a result, a highly convenient digital photo frame interface environment can be provided to the user. According to the present invention, the detection information detected by the sensor is acquired, and display control is performed in which the display mode of the operation icon for the touch panel changes according to the acquired detection information. In this way, it is possible to provide the user with a suitable interface environment by touching the touch panel operation icon, and to provide a digital photo frame having an unprecedented type of interface.

  Further, in the present invention, based on the acquired detection information, determination of the positional relationship between the user and the display unit, determination of the user's visual status with respect to the display unit, determination of the user's utterance state, recognition of the number of users And a user state determination unit that performs at least one of user authentication of the user, the display control unit, the positional relationship between the user and the display unit, the user's visual recognition state, the user's utterance state, Display control for changing the display mode of the operation icons for the touch panel may be performed in accordance with at least one of the recognition result of the number of people and the result of user authentication of the user.

  According to the present invention, detection information detected by a sensor is acquired, and a user state is determined or user recognition is performed based on the acquired detection information. And the display mode of the operation icon for touch panels changes according to the positional relationship with the user, the user's visual recognition state, the user's utterance state, the number of users, or the user authentication result. In this way, the operation icon for the touch panel reflecting the positional relationship with the user, the user's visual state and speech state, the number of users, the person authentication result, etc. is displayed on the display unit of the digital photo frame, An unprecedented type of interface environment can be provided.

  In the present invention, the user state determination unit determines a distance between the user and the display unit as the positional relationship between the user and the display unit, and the display control unit includes the user and the display unit. Display control of the operation icon for the touch panel may be performed according to the distance between the two.

  In this way, the display mode of the touch panel operation icon changes according to the distance from the user, and an interface environment reflecting the distance from the user can be provided.

  In the present invention, the display control unit may turn on the display of the operation icon for the touch panel when the distance between the user and the display unit is within a predetermined distance.

  In this way, when the user approaches, the display of the touch panel operation icon is turned on, so that both viewing of the content image and the interface environment using the touch panel operation icon can be achieved.

  Further, in the present invention, the display control unit, when it is determined that the distance between the user and the display unit is larger than a predetermined distance, the first to Nth (N is an integer of 2 or more) content images. Is displayed on the display unit, and when the Kth (1 ≦ K ≦ N) content image among the first to Nth content images is displayed, the user and the display unit When it is determined that the distance between them is within a predetermined distance, control for displaying a touch panel operation icon corresponding to the Kth content image may be performed.

  In this way, when the user approaches when the K-th content image among the first to N-th content images is displayed, an operation icon for a touch panel suitable for the K-th content image is displayed. become able to.

  In the present invention, the sensor is an imaging sensor that images a user, and the user state determination unit detects a user's face area based on imaging information from the imaging sensor, and the detected face area The distance between the user and the display unit may be determined based on the size of the display.

  In this way, it is possible to determine the distance from the user only by detecting the size of the face area, and to ensure that the user's line of sight is directed toward the display unit.

  In the present invention, the sensor may be an imaging sensor that captures an image of the user, and the user state determination unit may determine a distance between the user and the display unit by an automatic focusing process for the user.

  In this way, it is possible to determine the distance to the user and the like using the existing automatic focusing.

  In the present invention, the sensor may be an ultrasonic sensor, and the user state determination unit may determine the distance between the user and the display unit using the ultrasonic sensor.

  In this way, it is possible to determine the distance to the user and the like using the ultrasonic sensor.

  Further, in the present invention, the user state determination unit determines whether the user is gazing at the display unit as the user's visual recognition state, and the display control unit is gazed at the display unit by the user. Display control of the operation icon for the touch panel may be performed depending on whether or not it is performed.

  In this way, the display mode of the touch panel operation icon changes depending on whether or not the user is gazing, and an interface environment reflecting the gazing state of the user can be provided.

  In the present invention, the display control unit may turn on the display of the touch panel operation icon when it is determined that the user is gazing at the display unit.

  In this way, when the user gazes at the display unit, the display of the operation icon for the touch panel is turned on, and thus the on / off control of the display of the operation icon for the touch panel reflecting the user's gaze state can be realized. .

  In the present invention, the display control unit may perform control to display a touch panel operation icon corresponding to the content image that the user is gazing at.

  In this way, it is possible to display a touch panel operation icon suitable for the content image being watched by the user.

  In the present invention, when it is determined that the user is not gazing at the display unit, the display control unit displays the first to Nth (N is an integer of 2 or more) content images on the display unit. The slide display control is performed, and it is determined that the user is gazing at the display unit when the Kth (1 ≦ K ≦ N) content image among the first to Nth content images is displayed. In such a case, control for displaying a touch panel operation icon corresponding to the Kth content image may be performed.

  In this way, when the user gazes at the Kth content image among the first to Nth content images, a touch panel operation icon suitable for the Kth content image can be displayed.

  In the present invention, the user state determination unit may determine a user's utterance state by voice recognition, and the display control unit may perform display control of a touch panel operation icon according to the result of the voice recognition. .

  In this way, it is possible to realize display control of the operation icon for the touch panel reflecting the voice recognition result of the user's utterance state.

  Further, in the present invention, the display control unit is configured such that a word constituting the user's voice extracted by the voice recognition matches or is similar to a keyword set as a tag in the content image displayed on the display unit. In addition, control for displaying a touch panel operation icon corresponding to a matching or similar keyword may be performed.

  In this way, it becomes possible to display a touch panel operation icon corresponding to a word spoken by the user, and an intuitive interface environment for the user can be provided.

  In the present invention, the user state determination unit recognizes the number of detected users, and the display control unit determines that the number of detected users is one and a plurality of users. The display control may be performed so that the display mode of the operation icon for the touch panel displayed on the display unit is different depending on the case.

  In this way, the display mode of the operation icons for the touch panel changes according to the detected number of users, and an interface environment considering the number of users can be provided.

  In the present invention, the display control unit is configured to display a touch panel operation icon for a personal mode corresponding to one detected user when it is determined that the number of detected users is one. May be performed.

  In this way, when the user is alone, the touch panel operation icon for the personal mode corresponding to the user is displayed, so that the touch panel operation icon corresponding to the user can be displayed.

  Further, in the present invention, when it is determined that the number of detected users is plural, the display control unit displays an operation icon for a touch panel for a sharing mode corresponding to the detected plural users. Control may be performed.

  In this way, when there are a plurality of users, the touch panel operation icons for the sharing mode corresponding to the plurality of users are displayed, so that an interface environment suitable for a case where there are a plurality of users is provided. it can.

  In the present invention, when the detected user is determined to be the first user, the display control unit performs control to display a touch panel operation icon for the first user, and is detected. When it is determined that the user is the second user, control for displaying a touch panel operation icon for the second user may be performed.

  In this way, since the touch panel operation icon corresponding to each user is displayed, a suitable interface environment corresponding to each user can be provided.

  The present invention further includes a registration processing unit for performing user registration processing, wherein the user state determination unit performs person authentication as to whether or not the detected user is a registered user, and the display control unit is detected. If it is determined that the registered user is a registered user, control for displaying a personal mode touch panel operation icon corresponding to the registered user may be performed.

  In this way, when the detected user is a registered user, it becomes possible to display a touch panel operation icon for personal mode reflecting the user's registration process, which is suitable for each registered user. An interface environment can be provided.

  In the present invention, the user state determination unit performs age recognition of the detected user, and the display control unit performs display control of the operation icon for the touch panel based on the detected age recognition result of the user. Also good.

  In this way, the touch panel operation icon reflecting the recognition result of the user's age is displayed on the display unit, and a suitable interface environment according to the user's age can be provided.

  In the present invention, when the display control unit determines that the user's age group is the predetermined age group based on the detected age recognition result of the user, the operation for the touch panel for the predetermined age group is performed. You may perform control which displays an icon.

  If it does in this way, the operation icon for touchscreens for predetermined age groups suitable for the user can be displayed to each user of the predetermined age group.

  Further, the present invention may include a display mode switching unit that switches a display mode of the touch panel operation icon according to at least one of the detected number of users and the person authentication result.

  In this way, it is possible to change the display mode of the operation icons for the touch panel according to the number of users and the result of person authentication by a simple process of switching the display mode.

  In the present invention, when the image displayed on the display unit is a first content image, the display control unit displays a touch panel operation icon associated with the first content image, When the image displayed on the display unit is the second content image, control for displaying a touch panel operation icon associated with the second content image may be performed.

  In this way, since the touch panel operation icon associated with each content image is displayed when each content image is displayed, a suitable interface environment corresponding to the content of the content image can be provided.

  According to the present invention, the display control unit turns off the display of the touch panel operation icon when the user does not perform an operation after a predetermined time has elapsed after the display of the touch panel operation icon is turned on. It may be.

In this way, it is possible to prevent a situation in which the touch panel operation icon hinders viewing of the content image or the like. ;
Further, the present invention may include an information update unit that writes the user's operation history for the touch panel operation icon displayed on the display unit into the personal information storage unit to update the user's personal information.

  In this way, the user's operation history for the touch panel operation icon is written in the personal information storage unit as the user's personal information, so that display control reflecting the operation history can be performed.

  The present invention also includes a display instruction unit for instructing display of an image displayed on the display unit of the digital photo frame, and a detection information acquisition unit for acquiring detection information detected by the sensor, wherein the display unit is a touch panel display. The display instruction unit is related to an information processing system that issues a display instruction to change the display mode of the touch panel operation icon in accordance with the acquired detection information.

  The present invention is also a method for controlling a digital tough photo frame, wherein display control of a display unit is performed, detection information detected by a sensor is acquired, and the display unit is configured by a touch panel display. The present invention relates to a control method for performing display control for changing a display mode of a touch panel operation icon displayed on the display unit according to detection information.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. Configuration FIG. 1A shows an example of a digital photo frame 300 (digital photo player, image reproduction apparatus) of this embodiment. FIG. 1A illustrates an example of a so-called photo stand type digital photo frame. The digital photo frame 300 is installed by a user in an arbitrary place such as a house. Then, reproduction processing (image reproduction, sound reproduction) of content information such as digital image data and sound data is executed. The digital photo frame 300 can automatically play back content information (media information) such as an image without an explicit playback instruction from the user, for example. For example, a photo slideshow is automatically executed, or an image is automatically reproduced.

  Although FIG. 1A shows an example of a photo stand type digital photo frame, for example, as shown in FIG. As this wall-mounted digital photo frame, for example, electronic paper realized by an electrophoretic display or the like can be used. The digital photo frame 300 may be provided with a content information reproduction instruction button or the like, or a reproduction instruction may be performed using a remote controller.

  The digital photo frame 300 can include a memory card interface such as an SD card, for example. Alternatively, a wireless communication interface such as a wireless LAN or Bluetooth, or a wired communication interface such as USB can be provided. For example, when a user saves content information in a memory card and attaches it to the memory card interface of the digital photo frame 300, the digital photo frame 300 executes automatic reproduction (slideshow or the like) of the content information saved in the memory card. . Alternatively, when the digital photo frame 300 receives content information from the outside through wireless communication or wired communication, the digital photo frame 300 executes a reproduction process (automatic reproduction process) of the content information. For example, when a portable electronic device such as a digital camera or a mobile phone possessed by a user has a wireless function such as Bluetooth, content information is transferred from the portable electronic device to the digital photo frame 300 using this wireless function. Forward. Then, the digital photo frame 300 executes a reproduction process of the transferred content information.

  FIG. 2 shows a configuration example of the digital photo frame 300. The digital photo frame 300 includes a processing unit 302, a storage unit 320, a communication unit 338, a display unit 340, a sensor 350, and an operation unit 360. Various modifications such as omitting some of these components (for example, a communication unit, an operation unit, and a sensor) and adding other components (for example, a speaker) are possible.

  The processing unit 302 performs various control processes and arithmetic processes. For example, each part of the digital photo frame 300 is controlled or overall control is performed. The function of the processing unit 302 can be realized by hardware such as various processors (CPU and the like), ASIC (gate array and the like), a program stored in the information storage medium 330, and the like.

  The storage unit 320 serves as a work area for the processing unit 302, the communication unit 338, and the like, and its functions can be realized by a memory such as a RAM, an HDD (hard disk drive), or the like. The storage unit 320 includes a content information storage unit 322 that stores content information such as images and sounds, a detection information storage unit 324 that stores acquired detection information, a user registration information storage unit 325 that stores user registration information, and a specific information. User state storage unit 326 for storing the user status and the result of user recognition processing, a switching flag storage unit 328 for storing a display mode switching flag, and a personal information storage unit (sensitivity model storage unit for storing user personal information) 332, a user shared information storage unit (shared sensibility model storage unit) 334 that stores information shared (shared) among a plurality of users.

  The information storage medium 330 (a computer-readable medium) stores programs, data, and the like, and its function can be realized by a memory card, an optical disk, or the like. The processing unit 302 performs various processes of the present embodiment based on a program (data) stored in the information storage medium 330. That is, in the information storage medium 330, a program for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment (a program for causing the computer to execute processing of each unit). Is memorized.

  The communication unit 338 (communication interface) exchanges information with an external device (for example, a server or a portable electronic device) by wireless or wired communication, and functions as a communication ASIC or communication. This can be realized by hardware such as a processor or communication firmware.

  The display unit 340 is for displaying an image as content information, and can be realized by, for example, a liquid crystal display, a display using a light emitting element such as an organic EL, an electrophoretic display, or the like.

  In the present embodiment, the display unit 340 includes a touch panel display (touch screen). For example, a touch panel display is a device that combines a display such as a liquid crystal panel and an input device such as a touch pad. By touching operation icons on the screen with a finger or a dedicated pen (stylus), operation information is displayed. It is a device that can input. As the touch panel method, various methods such as a resistive film method (4-wire type, 5-wire type), a capacitance method, an electromagnetic induction method, a surface acoustic wave method (ultrasonic wave method), an infrared scanning method, and the like can be adopted. In the resistance film method, for example, a voltage is applied to one resistance film of two opposing resistance films (metal thin film constituting a transparent electrode), and a voltage generated in the other resistance film according to the touched position. To detect the touch position. In the capacitive method, a change in capacitance between a fingertip or the like and a conductive film is detected to detect a touch position, and electrostatic coupling that occurs when a finger or the like approaches the glass surface. Is used. As the electrostatic capacity method, there are a surface type and a projection type. The electromagnetic induction method is a method that detects the touch position using the principle of electromagnetic induction, and the surface acoustic wave method is a method that uses ultrasonic surface acoustic waves.

  A sensor 350 (a user detection sensor, a user recognition sensor, a motion sensor, or the like) is a device that outputs detection information based on a detection result. In the present embodiment, the detection information from the sensor 350 is used to determine, for example, the positional relationship between the user (person) and the display unit 340 (display screen, digital photo frame), the user's visual status with respect to the display unit 340, the user Utterance state (speech content, conversation content, voice state), recognition of the number of users, or user authentication (for example, age recognition). The detection information may be used for determination processing and recognition processing other than these.

  As the sensor 350, for example, a human sensor such as a pyroelectric sensor can be used. The pyroelectric sensor is a sensor that receives infrared rays generated by a person or the like, converts the infrared rays into heat, and converts the heat into electric charges by the pyroelectric effect of the element. By using this pyroelectric sensor, it is detected whether there is a user (person) in the detection range (detection area), the movement of the user in the detection range, the number of users in the detection range, etc. it can.

  As the sensor 350, for example, an image sensor such as a CCD or a CMOS sensor can be used. An imaging sensor (image sensor) is an optical sensor that converts one-dimensional or two-dimensional optical information into a time-series electrical signal. By using this image sensor, it is possible to detect whether or not there is a user in the detection range, user movements in the detection range, the number of users in the detection range, and the like. Further, user authentication (age recognition, etc.) can be realized by face image recognition using an image sensor. Further, the positional relationship such as the distance between the user and the display unit 340 and the angle of the user's line of sight with respect to the display unit 340 can be detected by face detection using an imaging sensor. Alternatively, it is possible to detect a user's visual state such as whether or not the display unit 340 is in the user's visual field range and whether or not the user is gazing at the display unit 340. Alternatively, whether or not the user is approaching can also be detected.

  As the sensor 350, for example, a distance sensor such as an ultrasonic sensor can be used. The ultrasonic distance sensor is a sensor that measures the distance from the time by emitting an ultrasonic pulse and receiving the ultrasonic pulse reflected by a person or the like.

  Further, as the sensor 350, various sensors such as a motion sensor that detects the user's operation state (movement of a hand or body) can be used.

  The sensor 350 may be the sensor device itself, or may be a sensor device including a control unit, a communication unit, and the like in addition to the sensor device. The detection information may be primary information obtained directly from the sensor, or may be secondary information obtained by processing (information processing) the primary information.

  The sensor 350 may be directly attached to the digital photo frame 300, or a home sensor or the like may be used as the sensor 350. When the sensor 350 is attached to the digital photo frame 300, the sensor 350 can be attached to, for example, a frame portion of the digital photo frame 300 as shown in FIG. Alternatively, the sensor 350 and the digital photo frame 300 may be connected using a wired cable or the like.

  The operation unit 360 is used by the user to input various types of information, and can be realized by, for example, an operation button or a remote control device. The user can perform user registration or registration of playback content (favorite image) desired by the user using the operation unit 360 or the touch panel display. For example, the user can input user registration information using the operation unit 360 or the touch panel display.

  The processing unit 302 includes a detection information acquisition unit 304, a user state determination unit 306, a registration processing unit 314, a display mode switching unit 316, an information update unit 317, and a display control unit 318. Note that various modifications may be made such as omitting some of these components (for example, the user state determination unit, the display mode switching unit, the information updating unit, etc.) or adding other components.

  The detection information acquisition unit 304 performs processing for acquiring detection information detected by the sensor 350. For example, when the sensor 350 detects the user state and the like and outputs detection information (imaging information), the detection information acquisition unit 304 takes in the detection information. The captured detection information is stored in the detection information storage unit 324 of the storage unit 320. When an external sensor such as a home sensor is used as the sensor 350, the communication unit 338 receives the detection information, and the detection information acquisition unit 304 acquires the received detection information.

  The user state determination unit 306 (user recognition unit) determines the user state and the like based on the detection information acquired by the detection information acquisition unit 304. For example, based on the acquired detection information, the determination of the positional relationship between the user (person) and the display unit 340, the determination of the user's visual status with respect to the display unit 340, the determination of the user's utterance state (utterance content), Recognize the number of users and perform user authentication.

  Here, the positional relationship between the user and the display unit 340 is that the user exists within the distance between the user and the display unit 340, the user's line-of-sight direction with respect to the display unit 340, and the detection range set for the display unit 340. Or not. The positional relationship determination unit 307 determines the positional relationship between the user and the display unit 340. For example, as a positional relationship between the user and the display unit 340, a distance (distance information, distance parameter) between the user and the display unit 340 is determined.

  The visual recognition state is a state of the user's visual field range, a gaze state, and the like. Specifically, whether or not the display unit 340 is in the user's visual field range (view volume), and the user watches the display unit 340. Or not. The visual recognition state determination unit 308 determines the visual recognition state of the user. For example, it is determined whether the user is gazing at the display unit 340 as the user's viewing state.

  For example, it is assumed that an imaging sensor for imaging the user is provided as the sensor 350. In this case, the user state determination unit 306 (positional relationship determination unit) detects the user's face area (rectangular frame area) based on the imaging information from the imaging sensor. Based on the detected size of the face area, the distance between the user and the display unit 340 is determined (estimated). Also, the user state determination unit 306 sets a measurement area that includes the detected face area and is larger in size than the face area. That is, a measurement area that overlaps the face area is set. Then, the time during which the face area exists in the measurement area is measured, and based on the measured time, it is determined whether or not the user is gazing at the display unit 340. For example, when the presence time in the measurement area is a predetermined time or more, it is determined that the user is gazing.

  Alternatively, the user state determination unit 306 may determine the distance between the user and the display unit 340 by an automatic focusing process (autofocus function) for the user. For example, when the active method is adopted, a device for emitting infrared rays or ultrasonic waves is provided on the digital photo frame 300 or the like, and an infrared or ultrasonic light receiving sensor is provided as the sensor 350. And what is necessary is just to detect the distance with a user, etc. by detecting the reflected light from a user with a light receiving sensor. When the passive method is adopted, an image sensor is provided as the sensor 350, and the distance to the user or the like may be detected by performing image processing on the captured image using a phase difference detection method or a contrast detection method. .

  The utterance state of the user is a word constituting the voice uttered by the user, a voice state (tone, frequency, size), an utterance speed, or the like. Specifically, the user's speech state can be determined (analyzed) by voice recognition. The utterance state determination unit (voice determination unit) 309 determines the utterance state of the user.

  The number of users can be recognized (identified) using, for example, a pyroelectric sensor (human sensor) that is the sensor 350, or can be recognized using an imaging sensor that is the sensor 350. Recognition of the number of users is performed by the user number recognition unit 310.

  User authentication of a user can be realized, for example, by face recognition processing using an image sensor. The user authentication of the user is performed by the person authentication unit 311. Note that human authentication may be performed using a retina that is a capillary blood vessel pattern in the retina of an eye or an iris that is a radial pattern in a black eye.

  It should be noted that age recognition may be performed as user personal authentication. For example, face patterns for each age group such as a face pattern for adults and a face pattern for children are prepared, and age recognition is performed by collating these face patterns. Alternatively, the user's wrinkle is recognized by face recognition processing, and the user's age is determined based on the number of recognized wrinkles and the like.

  Information on the user's positional relationship, viewing state, speech state, number of people recognition result, and person authentication result is stored in the user state storage unit (recognition result storage unit) 326. For example, when the number of users within the detection range is recognized, the number of users is stored in the user state storage unit 326. For example, when the user's face recognition process is performed by the imaging sensor, the result of image recognition (for example, feature point information such as the user's face image) is stored in the user state storage unit 326. In this case, these state information and recognition result information are stored in association with the user registration information, for example.

  The registration processing unit 314 performs user registration processing. For example, user registration information setting processing is performed. Specifically, when the user inputs user registration information using the operation unit 360 or the touch panel display on the user registration screen or the like, the input user registration information is stored in the user registration information storage unit 325 in association with the user. To do. Here, the user registration information can include, for example, a user ID, password, customization information for an image displayed on the display unit 340, an operation icon for the touch panel, and the like. The display unit 340 displays a content image reflecting the user registration information and a touch panel operation icon.

  The display mode switching unit 316 performs display mode switching processing. For example, the operation mode for the touch panel and the display mode of the content image are switched according to at least one of the detected number of users and the person authentication result. Specifically, the display mode of the touch panel operation icon and the content image is switched to the personal mode, the sharing mode, and the normal mode based on the number of users and the person authentication result. Here, the personal mode is a mode in which display control for a personal mode associated with one user is performed, and a touch panel operation icon and a content image for the user are displayed. The sharing mode is a mode in which display control for a sharing mode associated with a plurality of users is performed. For example, a shared touch panel operation icon and a content image are displayed to a plurality of users. The normal mode is a mode in which display control is performed for the normal mode, for example, a mode for displaying general (default) touch panel operation icons and content images. For example, when the mode is not the personal mode or the sharing mode, the normal mode is used. Set to Note that the display mode switching unit 316 may switch the display mode according to a user state such as a positional relationship between the user and the display unit 340 and a user's viewing state.

  The display mode is switched using a switching flag stored in the switching flag storage unit 328. Specifically, when the number of users is recognized by the user state determination unit 306 or person authentication is performed, a display mode switching flag is set according to the result, and the set switching flag is displayed as a switching flag storage unit. Stored in 328. Further, when the user state determination unit 306 determines a user state such as a positional relationship, a visual recognition state, and an utterance state, a display mode switching flag is set according to the user state, and the set switching flag is stored in the switching flag storage. Stored in the unit 328.

  A tag is assigned to the content image stored in the content information storage unit 322. Specifically, each content image is provided with a keyword tag corresponding to the content, various display mode tags such as a personal mode tag and a shared mode tag. For example, the display control of the operation icon for the touch panel can be realized by comparing the word extracted from the voice uttered by the user with the tag of the content image. Alternatively, by using a tag attached to each content image, display control corresponding to various display modes is performed at the time of switching the display mode, and the content image corresponding to the display control is stored in the content information storage unit 322. Can be displayed on the display unit 340.

  The information update unit 317 updates information in the personal information (sensitivity model) storage unit 332 and the user shared information (shared sensitivity model) storage unit 334. For example, the user's operation history for the touch panel operation icon displayed on the display unit 340 is written in the personal information storage unit 332 to update the user's personal information and the like.

  Here, the operation history is a history of various operations performed by the user, for example, a history of a user's image browsing operation, image skip operation, or operation to return to the previous image. Specifically, it is a history of touch operations of touch panel operation icons corresponding to these operations.

  The display control unit 318 performs display control of the display unit 340. For example, based on the content image information stored in the content information storage unit 322, control for displaying an image on the display unit 340 (such as a writing process to a drawing buffer) is performed.

  Specifically, in the present embodiment, the display unit 340 is configured by a touch panel display. The display control unit 318 performs display control to change the display mode of the touch panel operation icon according to the detection information from the sensor 350 (sensor other than the touch panel sensor) acquired by the detection information acquisition unit 304. Specifically, based on the acquired detection information, determination of the positional relationship between the user and the display unit 340, determination of the user's visual status with respect to the display unit 340, determination of the user's utterance state, recognition of the number of users, And at least one of user authentication is performed. The display control unit 318 uses the touch panel for at least one of the positional relationship between the user and the display unit 340, the user's viewing state, the user's utterance state, the user's number of people recognition result, and the user's person authentication result. Display control for changing the display mode of the operation icon is performed. For example, changing the displayed image of the touch panel operation icon itself, changing the display timing, display time, character size, character font, color, transparency, brightness, contrast, size, or display method of the touch panel operation icon Let

  In addition, you may perform control which changes the display mode of a content image according to the acquired detection information. For example, the displayed content image itself is changed, or the display timing, display time, or display method of the content image is changed.

  The touch panel operation icons (operation menus) are icons (images represented by pictures and characters) that are touched by the user on the touch panel display, and each operation icon corresponds to various operations. For example, it is associated with various operations such as display of detailed information, display of the previous image, display of the next image, display of various information related to the content, storage of the image, destruction of the image, input of information, and the like. When the user touches the operation icon, an operation associated with the operation icon is executed.

  For example, it is assumed that the user state determination unit 306 determines the distance between the user and the display unit 340 as the positional relationship between the user and the display unit 340. In this case, the display control unit 318 performs display control of the touch panel operation icon according to the distance between the user and the display unit 340. For example, when the distance between the user and the display unit 340 is within a predetermined distance (when the user approaches), the display of the touch panel operation icon is turned on. Then, after the display of the operation icon for the touch panel is turned on, if no user operation is performed after a predetermined time has passed (when the operation icon is not touched), the operation icon for the touch panel is displayed. Turn off the display.

  In addition, when it is determined that the distance between the user and the display unit 340 is greater than the predetermined distance (when it is determined that the user is not approaching), the display control unit 318 first to Nth (N Is displayed as a slide on the display unit 340 (displayed sequentially). For example, the first to Nth content images are content images of different genres or different categories. On the other hand, when the Kth (1 ≦ K ≦ N) content image among the first to Nth content images is displayed, it is determined that the distance between the user and the display unit 340 is within a predetermined distance. If it is determined (when it is determined that the user has approached), a touch panel operation icon corresponding to the Kth content image is displayed.

  Note that the color, brightness, transparency, contrast, size, or the like of the touch panel operation icon may be changed in accordance with the distance between the user and the display unit 340. The display control unit 318 does not need to change the display mode of the touch panel operation icon based on the distance itself, and may change the display mode based on a parameter equivalent to the distance (for example, the size of the face area). . The display control unit 318 turns off the display of the display unit 340 when it is determined that the user does not exist within the detection range. For example, the power consumption is saved by switching from the normal mode to the power saving mode.

  Further, it is assumed that the user state determination unit 306 determines whether the user is gazing at the display unit 340 as the user's visual recognition state. In this case, the display control unit 318 performs display control of the touch panel operation icon depending on whether or not the user is watching the display unit 340. For example, when it is determined that the user is gazing at the display unit 340, the display of the operation icon for the touch panel is turned on. Specifically, a touch panel operation icon corresponding to the content image being watched by the user is displayed. For example, when the user gazes at the first content image, the first touch panel operation icon associated with the first content image is displayed, and when the user gazes at the second content image, the second content image is displayed. The second touch panel operation icon associated with the content image is displayed. For example, when the user does not perform an operation even after a predetermined time has elapsed, the display of the touch panel operation icon is turned off.

  In addition, when it is determined that the user is not gazing at the display unit, the display control unit 318 slides the first to Nth content images on the display unit 340. On the other hand, if it is determined that the user is gazing at the display unit 340 (Kth content image) when the Kth content image among the first to Nth content images is being displayed, A touch panel operation icon corresponding to the K content image is displayed.

  Further, it is assumed that the user state determination unit 306 determines the user's utterance state by voice recognition. In this case, the display control unit 318 performs display control of the touch panel operation icon according to the result of the speech recognition. Specifically, a word (word) constituting a user's voice extracted by voice recognition is a keyword (either one or a plurality of keywords) set as a tag in the content image displayed on the display unit 340. It is judged whether or not it matches. If they match or are similar, a touch panel operation icon corresponding to the matching or similar keyword is displayed. For example, when the spoken word matches or is similar to the first keyword, the first touch panel operation icon corresponding to the first keyword is displayed and matches or is similar to the second keyword. In this case, a second touch panel operation icon corresponding to the second keyword is displayed.

  Further, it is assumed that the user state determination unit 306 recognizes (identifies) the number of users. In this case, the display control unit 318 displays the touch panel operation icon displayed on the display unit 340 when the number of detected users is determined to be one and when it is determined that there are a plurality of users. Display control is performed to change the display mode. Specifically, when it is determined that the number of detected users is one, a touch panel operation icon for personal mode corresponding to (associated with) the detected one user is displayed. For example, an operation icon for the personal mode (private mode) is displayed based on the user registration information registered for the one user. On the other hand, when it is determined that there are a plurality of detected users, a touch panel operation icon for the sharing mode corresponding to (associated with) the detected plurality of users is displayed. For example, based on user registration information (group registration information) registered for the plurality of users, a display operation is performed in the sharing mode (group mode, family mode), and an operation icon for the sharing mode is displayed.

  Further, when it is determined that the detected user is the first user, the display control unit 318 displays a touch panel operation icon for the first user, and the detected user is the second user. If it is determined that there is a touch panel operation icon for the second user, control is performed. For example, the registration processing unit 314 performs user registration processing, and the user state determination unit 306 performs person authentication to determine whether or not the detected user is a registered user. Then, when it is determined that the detected user is a registered user, the display control unit 318 performs control to display a personal mode touch panel operation icon corresponding to the registered user. That is, according to the user registration information stored in the user registration information storage unit 325, an operation icon for the personal mode corresponding to the user is displayed.

  When the user state determination unit 306 recognizes the user's age, the display control unit 318 performs display control of the operation icon for the touch panel based on the user's age recognition result (age group recognition result). Specifically, when it is determined that the user's age group is a predetermined age group based on the detected age recognition result of the user, a touch panel operation icon for the predetermined age group is displayed. For example, when it is determined that the user's age group is a first age group (for example, a child's age group), an operation icon for the first age group is displayed, and the user's age group is the second age group. When it is determined that it is an age group (for example, an elderly person or an adult age group), an operation icon for the second age group is displayed. Or you may change the content of a content image, etc. based on an age recognition result (age group).

  In addition, when the image displayed on the display unit 340 is the first content image, the display control unit 318 performs control to display the touch panel operation icon associated with the first content image. On the other hand, when the image displayed on the display unit 340 is the second content image, control is performed to display a touch panel operation icon associated with the second content image. That is, an operation icon corresponding to the displayed content image is displayed.

2. In this embodiment, the display mode of the operation icons for the touch panel is changed according to the detection information acquired by the sensor 350. Specifically, the operation icon is changed according to the positional relationship between the user and the display unit 340.

  For example, in FIGS. 3A and 3B, the operation icon is changed in accordance with the distance (positional relationship in a broad sense) between the user and the display unit 340 (digital photo frame). Specifically, in FIG. 3A, the user UA is viewing a photo that is a content image through a slide show of the digital photo frame 300. In FIG. 3A, the distance between the user UA and the display unit 340 is long, and is greater than or equal to a predetermined distance. In this case, the operation icon is turned off.

  On the other hand, in FIG. 3B, the distance between the user UA and the display unit 340 is close, and is smaller than the predetermined distance. That is, the user UA is approaching to see the display content of the display unit 340 or to perform an operation. In this case, operation icons (touch panel operation menu) for the touch panel shown in A1 to A3 are displayed. That is, an operation icon is displayed so as to be superimposed on the content image.

  Here, the operation icon A1 is an icon for instructing the display of the previous content image in the slide display or the like, and the operation icon A2 is an icon for instructing the display of the next content image. The icon A3 is an icon for instructing display of detailed information of the displayed content image.

  By displaying such an operation icon, the user can perform an operation associated with the operation icon by touching the operation icon on the touch panel display. For example, during the slide display of a photo, it is possible to view the previous photo by touching the operation icon A1, or to fast-forward and view the next photo by touching the operation icon A2. Further, by touching the operation icon A3, it becomes possible to confirm the situation and recording (for example, tag information set at the time of photographing such as the photographing time) of the photograph.

  Many digital photo frames so far have only a function of automatically displaying photographs and the like, and the displayed images are monotonous and scarcely changed. Moreover, it did not have a highly convenient interface for the user.

  In this regard, in the digital photo frame of the present embodiment, the display unit is configured by a touch panel display. Therefore, the user can perform an operation desired by simply touching an operation icon on the screen, and an interface environment that is highly convenient for the user can be provided.

  Further, for example, in a navigation device or the like, there is no problem even if the operation icon is always displayed. However, since the main focus of the digital photo frame is to appreciate a photograph or the like, if the operation icon is always displayed, the viewing of the photograph or the like is hindered.

  In this regard, in the present embodiment, the display mode of the operation icon changes based on the detection information from the sensor. For example, when the user approaches, the display of the operation icon is turned on. When a predetermined condition such as the user not operating for a certain time is satisfied, the operation icon display is turned off. Accordingly, the user can execute a desired operation by viewing a photograph or the like in the state of FIG. 3A and approaching the digital photo frame and touching an operation icon when an operation is desired. Therefore, according to this embodiment, it is possible to achieve both viewing of a photograph or the like in a state where no operation icon is displayed and provision of a simple interface using a touch panel.

  Various display modes of operation icons can be assumed. For example, FIGS. 4A and 4B show examples of other display modes.

  In FIG. 4A, product advertisement information of a camera is displayed as a content image. In this case, when the user UA approaches the digital photo frame 300, a touch panel operation icon corresponding to the content image is displayed as shown in B1 to B4 of FIG. For example, when the user UA touches the operation icons B1 and B2, the specifications of the camera as a product and word-of-mouth information are displayed. Further, the user UA can touch the operation icon B3 to put a check mark on the product, or touch the operation icon B4 to purchase the product via the Internet or the like.

  Thus, according to the present embodiment, it is possible to display the touch panel operation icons corresponding to the respective content images. For example, when the user approaches when the first content image is displayed, the first touch panel operation icon corresponding to the first content image is displayed, and when the user approaches when the second content image is displayed. A second touch panel operation icon corresponding to the second content image is displayed. 3A and 3B, operation icons for facilitating the viewing of photos can be displayed, as shown in FIGS. 4A and 4B. In the product advertisement content, operation icons that facilitate the collection and purchase of product information can be displayed. Therefore, it is possible to provide a user with an appropriate interface environment according to each content and realize various information presentations using a digital photo frame.

  In order to display an operation icon corresponding to each content image, a tag for instructing the corresponding operation icon may be associated with each content image. That is, the content information storage unit 322 in FIG. 2 stores the instruction information of the corresponding operation icon as a tag in association with each content image. The operation information instruction information tag is set by the user as user registration information, set by the content provider as content information, or automatically set according to a predetermined algorithm.

  When the display mode of the operation icon or the like is switched according to the distance to the user, this distance may be a linear distance between the user and the display unit 340, or a distance in the depth direction (distance in the Z direction). It may be. In addition, the distance in this case includes a parameter mathematically equivalent to the distance (for example, the area of a face area described later), and may be a parameter that changes according to a change in distance, for example. The positional relationship with the user is not limited to the distance, and may be, for example, an angle formed by the user's line-of-sight direction and the display screen of the display unit 340.

  3A to 4B, the case where the display mode of the operation icons for the touch panel is changed according to the positional relationship between the user and the display unit 340 has been described. However, according to the positional relationship with the user. The display mode of the content image may be changed. For example, the detail level of the content image, the number of screen divisions, and the character size are changed according to the positional relationship with the user. Specifically, as the distance from the user approaches, the level of detail of the content image is increased, the number of screen divisions is increased, and the character size is decreased. In this case, the level of detail, the number of screen divisions, and the character size may be switched in two stages or in three or more stages. Or you may change the display mode of a content image by displaying a related image according to the positional relationship with a user, or turning ON / OFF the display of a display part.

  If the display mode of the content image is changed in accordance with the positional relationship with the user in this way, for example, more detailed information can be provided to a user who has approached the content image with interest. Thus, it is possible to display an appropriate content image according to the distance to the user.

  Next, an example of a method for detecting the distance (positional relationship) between the user and the display unit 340 will be described with reference to FIGS. 5A and 5B, an image sensor (camera) such as a CCD or CMOS sensor is used as the sensor 350. Based on the imaging information from the imaging sensor, a face area FAR that is a rectangular frame area is detected, and the size of the detected face area FAR is obtained. Based on the obtained size, the distance between the user and the display unit 340 is determined.

  For example, in FIG. 5A, since the size of the face area FAR is small (below a predetermined size), it is determined that the distance from the user is long. Therefore, in this case, the display of the touch panel operation icon is turned off as shown in FIGS. On the other hand, in FIG. 5B, since the size of the face area FAR is large (because it is larger than the predetermined size), it is determined that the distance to the user is short. Accordingly, in this case, the display of the touch panel operation icon is turned on as shown in FIGS.

  Here, various methods can be considered as a method for detecting a face region. For example, in order to perform face detection, it is necessary to cut out a face region by distinguishing a place where the face is from another object in a captured image captured by an image sensor. The face is composed of eyes, nose, mouth and the like, and these shapes and positional relationships have almost common characteristics, although there are individual differences. Therefore, using this common feature, the face is identified from other objects and cut out from the screen. Clues for this include skin color, facial movement, shape, size, and the like. In the case of using skin color, a method of converting RGB data into HSV data composed of hue, luminance, and saturation and extracting a human skin hue is adopted.

  Or you may produce the average face pattern produced | generated from the face pattern of many people as a face template. Then, this face template is scanned on the screen of the captured image to obtain a correlation with the captured image, and an area having the highest correlation value is detected as a face area.

  In order to improve the detection accuracy, a plurality of face templates may be prepared as dictionary data, and a face area may be detected using the plurality of face templates. Alternatively, the face area may be detected in consideration of the characteristics of the eyes, nose, mouth, the positional relationship between these, and the contrast in the face. Alternatively, the face area can be detected by statistical pattern recognition using a neural network model.

  5A and 5B, not only the distance between the user and the display unit 340 can be detected based on the size of the face area FAR, but also whether or not the user is looking at the display unit 340 at the same time. There is an advantage that it can be detected. That is, when the user's line of sight is not directed toward the display unit 340, the correlation value with the face template is low, and the face area FAR is not detected. Therefore, the detection of the face area FAR is equivalent to the fact that the user's line of sight is directed toward the display unit 340 and the display unit 340 is within the visual field range of the user. In this state, if the size of the face area FAR is detected and the display mode of the operation icon or the like is changed as shown in FIGS. 3A to 4B, the content image on the display unit 340 is displayed. Appropriate operation icons can be displayed for the viewing user. As a result, the digital photo frame 300 that has never existed can be provided.

  Note that the user detection method is not limited to the method shown in FIGS. 5 (A) and 5 (B). For example, the user may be detected by effectively utilizing an autofocus function mounted on a general camera or a camera of a mobile phone. By using this autofocus function, it is possible to determine whether or not a user (person) exists in front of the digital photo frame 300 and the positional relationship such as the distance between the user and the display unit 340.

  For example, when there is no one in the room, the focus of autofocus is almost fixed. However, when the user crosses the display unit 340 of the digital photo frame 300, the autofocus function works, It can be determined whether or not the user exists. In addition, when the user is viewing the display unit 340 of the digital photo frame 300, the autofocus is activated in response to the user, and the user and the display unit 340 are automatically focused on the user. Can detect the approximate distance.

  As an autofocus method, there are an active method and a passive method. In the active method, infrared rays or ultrasonic waves are emitted to measure the distance from a subject such as a user. That is, the distance to the subject is measured by measuring the time until the reflected wave returns. This active method has an advantage that the focus can be easily adjusted even in a dark place.

  On the other hand, in the passive method, the luminance information of the subject is received by an imaging sensor such as a CCD sensor, and the distance (focus position) from the subject is detected by electrical processing. That is, the distance is measured using the photographed image. The passive method includes a phase difference detection method for detecting a lateral shift of a luminance signal, a contrast detection method for detecting a contrast of the luminance signal, and the like.

  Next, a specific processing example of the display control of the operation icon according to the positional relationship with the user will be described using the flowchart of FIG.

  First, a pyroelectric sensor, which is one of the sensors, is used to determine whether or not a user (person) exists within the detection range (step S1). That is, a pyroelectric sensor is used to roughly detect whether or not a user exists around the digital photo frame. Thus, by efficiently using the pyroelectric sensor and the image sensor as the sensors, the user state can be efficiently detected.

  If it is determined that there is no user (step S2), the display on the display unit is turned off and the power saving mode is set (step S3). On the other hand, if it is determined that there is a user, the display of the display unit is turned on and the content image is displayed (step S4). By doing so, it is possible to prevent a situation where an image is displayed on the display unit and power is wasted even when there is no user near the digital photo frame.

  Next, it is determined whether or not the display unit is within the visual field range of the user by face detection using an imaging sensor (camera) that is one of the sensors (step S5). If it is determined that it is within the visual field range (step S6), the distance between the user and the display unit (display screen) is determined from the size of the face area (frame area) (step S7). For example, when a face area is detected as shown in FIGS. 5A and 5B, it is determined that the display unit is within the visual field range of the user. Then, the size of the face area is detected, and the distance between the user and the display unit is determined (estimated).

  When it is determined that the distance between the user and the display unit is within a predetermined distance (step S8), the display of the touch panel operation icon is turned on (step S9). At this time, for example, the detailed display mode of the content image may be turned on. Then, after the display of the touch panel operation icon is turned on in this way, when a predetermined time has passed without any user operation (step S10), the display of the touch panel operation icon is turned off (step S11). . As described above, as described with reference to FIGS. 3A to 4B, the on / off control of the display of the operation icon for the touch panel according to the positional relationship such as the distance can be realized.

3. Display control of operation icon according to visual recognition state In this embodiment, the display mode of the operation icon for touch panels is changed according to the visual recognition state of the user. Specifically, the display mode is changed according to whether or not the user is gazing at the display unit 340, whether or not the display unit 340 is within the visual field range of the user, and the like.

  For example, in FIGS. 7A and 7B, the display mode of the operation icon is changed depending on whether or not the user UA is gazing at the display unit 340. Specifically, in FIG. 7A, the line of sight of the user UA does not face the direction of the display unit 340, and the display unit 340 is not watched. In this case, only the content image is displayed and the operation icon is turned off.

  On the other hand, in FIG. 7B, the user is watching the display unit 340. For example, the display unit 340 is within the visual field range of the user UA for a predetermined time or more. In this case, the operation icon display is turned on as indicated by C1 to C4. Specifically, an operation icon for providing product information of a camera as a content or purchasing a product is displayed superimposed on a content image of the camera.

  For example, when the user is not gazing at the content image of the camera as shown in FIG. 7A, it is considered that the user is not so interested in purchasing the camera. On the other hand, when the user is gazing as shown in FIG. 7B, he / she is interested in the camera and is willing to purchase. Therefore, in this case, the operation icons as shown in C1 to C4 are displayed to provide an appropriate interface environment for the user to check the camera information or purchase. By doing so, an interface environment that is highly convenient for the user can be realized.

  FIG. 8 shows another example in which the display mode of the operation icon is changed according to the visual recognition state.

  As shown in FIG. 8, when it is determined that the user is not gazing at the display unit 340 and the gazing mode is set to off, content images IM1 to IM5 (first to Nth contents in a broad sense) Image) slide display. That is, the content images IM1 to IM5 are displayed sequentially every predetermined time. Here, the content contents (themes) of the content images IM1 to IM5 are different. Specifically, the content images IM1, IM2, IM3, IM4, and IM5 are contents representing news, weather, stock prices, landscape photos, and animal photos, respectively.

  Thus, the content images IM1 to IM5 are slide-displayed. For example, when the weather content image IM2 (K-th content image in a broad sense) is displayed, the user gazes at the display unit 340, and the gaze mode is turned on. Suppose. In this case, the content image IM21A in which the operation icon corresponding to the weather content as indicated by D1 and D2 is turned on is displayed. When the user touches the operation icon for the detailed display instruction shown in D1, the content image IM21B showing the detailed information of today's weather is displayed. On the other hand, when the user touches the operation icon indicated by D2, the next content image IM22A indicating information on the probability of today's precipitation is displayed.

  Then, in this content image IM22A, when the user touches the operation icon D3, a content image IM22B showing detailed information on today's precipitation probability is displayed. On the other hand, when the operation icon D4 is touched, the next content image IM23A showing pollen information is displayed.

  Then, in the content image IM23A, when the user touches the operation icon D5, the content image IM23B showing the detailed information of the pollen is displayed. On the other hand, when the operation icon D6 is touched, the display returns to the display of the content image IM21A showing today's weather information.

  Further, in the detailed display content images IM21B, IM22B, and IM23B, when the operation icons for simple display instructions shown in D7, D9, and D11 are touched, the display returns to the display of the simple display content images IM21A, IM22A, and IM23A. On the other hand, in the content images IM21B, IM22B, IM23B, when the operation icon for the next content display instruction shown in D8, D10, D12 is touched, the next content images IM22B, IM23B, IM21B are displayed.

  According to the display control method of FIG. 8, when content images IM1 to IM5 are slide-displayed, if a content image in which the user is interested is watched, an operation icon corresponding to the content image is displayed. Then, when the user touches the displayed operation icon, a detailed image of the content image and a related image are displayed. For example, the content image IM21B, which is a detailed image of IM21A, is displayed by touching the operation icon D1 on the content image IM21A. Further, by touching the operation icon D2, a content image IM22A that is a related image of IM21A is displayed. Therefore, the content image according to the user's interest can be efficiently displayed, and the detailed image and the related image can be displayed when the user touches the operation icon. As a result, the display mode of the content image can be changed in various ways, and an unprecedented type of digital photo frame can be provided.

  In FIG. 8, when it is determined that the distance between the user and the display unit 340 is larger than the predetermined distance (when it is determined that the user is not approaching), the content images IM1 to IM5 are displayed in a slide manner, and the content is displayed. When it is determined that the distance between the user and the display unit 340 is within a predetermined distance when the image IM2 (Kth content image) is displayed (when it is determined that the user is approaching) The display of the touch panel operation icon may be turned on.

  7A to 8 illustrate the case where the display mode of the operation icon for the touch panel is changed according to the user's visual state, but the display mode of the content image is changed according to the user's visual state. Also good. For example, the level of detail of the content image, the number of screen divisions, and the character size are changed according to the user's visual recognition state. Specifically, when the user gazes at the content image, the detail level of the content image is increased, the number of screen divisions is increased, or the character size is reduced. In this case, the level of detail, the number of screen divisions, and the character size may be switched in two stages or in three or more stages. Or you may change the display mode of a content image by displaying a related image according to a user's visual recognition state, or turning on and off the display of a display part. In this way, when a visual state such as a user's gaze is detected, the display mode of the display image changes accordingly. Accordingly, it is possible to increase the variety of display images provided to the user and to efficiently transmit information, and to provide a digital photo frame of an unprecedented type.

  Next, an example of a method for detecting a user's gaze state will be described with reference to FIG. In FIG. 9A, an image sensor is used as the sensor 350. Then, as described with reference to FIGS. 5A and 5B, the user's face area FAR is detected based on the imaging information from the imaging sensor.

  Next, a measurement area SAR corresponding to the detected face area FAR is set. This measurement area SAR is an area that includes the face area FAR and is larger in size than the face area FAR. This measurement area SAR can be set, for example, by oversizing the face area FAR. Then, the time during which the face area FAR exists in the measurement area SAR is measured, and it is determined whether the user is gazing at the display unit 340 based on the measured time. For example, when the face area FAR is located in the measurement area SAR for a certain time or more, it is determined that the user is gazing at the display unit 340. Then, the display mode of the operation icon is changed as shown in FIGS.

  According to the detection method of FIG. 9A, both the detection of the distance to the user and the detection of the gaze state by the method of FIG. 5A and FIG. 5B can be realized using the imaging sensor. become. Thereby, the number of parts of the sensor can be reduced and efficient processing can be performed.

  Note that the gaze state detection method is not limited to the method shown in FIG. For example, in FIG. 9B, the gaze state is detected by irradiating infrared light and detecting the user's red eyes. Specifically, the infrared light irradiated by the infrared device 354 is reflected by the half mirror 353 and reaches the eyes of the user. The user's red-eye state is captured and detected by a camera 352 provided with an imaging sensor, thereby detecting the position of the pupil of the user's eyes, detecting the user's gaze direction, and determining whether the user is in the gaze state. Determine whether. In FIG. 9C, the position of the pupil is detected from the brightness and darkness of the image area around the eyes of the user's face image captured by the two cameras 356 and 357 (stereo cameras). Then, the user's line-of-sight direction is detected from the detected center position of the pupil and the center position of the eyeball, and it is determined whether or not the user is in a gaze state. As described above, various methods can be adopted as a method for detecting the user's gaze state.

  Moreover, although the case where the distance to the user and the user's gaze state are detected and the display mode of the operation icon and the content image is changed has been described above, the present embodiment is not limited to this. For example, the display state of the operation icon or the content image may be changed by detecting the degree of approach of the user within the detection range. For example, the rate of change of the size of the face area FAR in FIG. When the rate of change in size is equal to or greater than a certain value, it is determined that the degree of approach of the user is high, and the same as in FIGS. 3 (A) to 4 (B) and FIGS. 7 (A) to 8. The display mode of the operation icon and the content image is changed by the technique. In this way, operation icons and content images in various display modes can be displayed for a user who is approaching with interest in the digital photo frame.

  Further, whether or not the user is within the detection range is detected by the sensor 350, and when it is determined that the user exists, the display of the display unit 340 may be switched from off to on. For example, the image is displayed on the display unit 340 by switching from the power saving mode to the normal mode. In addition, when the display of the display unit 340 is on and the user leaves the detection range and it is determined that no user exists, the display of the display unit 340 is switched from on to off. In this way, it is possible to prevent a situation where an image is displayed on the digital photo frame when the user is not nearby and power is consumed wastefully.

  Further, when it is detected that the user is gazing at the display unit 340, the display mode of the operation icon or the content image may be changed on the condition that one gazing is detected, or a plurality of times The display mode of the operation icon and the content image may be changed on condition that gaze is detected. Specifically, the number of gazes, which is the number of times the user gazes the display unit 340 within a predetermined time, is counted and recorded. Then, the display mode is not changed until the number of gazes within a predetermined time exceeds a predetermined number of times (for example, 2 to 5 times) serving as a threshold value. On the other hand, when the number of gazes within a predetermined time exceeds the predetermined number, the display mode is changed to, for example, the gaze mode, and the display mode is changed.

  Next, a specific processing example of operation icon display control according to the user's visual recognition state will be described using the flowchart of FIG. Steps S21 to S25 in FIG. 10 are the same as steps S1 to S5 in FIG.

  If it is determined in step S26 in FIG. 10 that the display unit is within the visual field range of the user, it is determined whether or not the user is gazing at the display unit by gaze detection using an imaging sensor or the like. (Step S27). If it is determined that the user is gazing, the gazing mode is turned on as described with reference to FIG. 7B and the touch panel operation icon is turned on (step S29). At this time, for example, an image for a gaze mode (detailed image, related image, etc.) may be displayed as a content image. Then, after the display of the touch panel operation icon is turned on in this way, when a predetermined time has passed without any user operation (step S30), the display of the touch panel operation icon is turned off (step S31). . As described above, as described with reference to FIGS. 7A to 8, it is possible to realize the on / off control of the display of the operation icons for the touch panel according to the user's visual state such as gaze.

  Next, details of the gaze state detection process described with reference to FIG. 9A will be described with reference to the flowchart of FIG.

  First, a face area (frame area) is detected by face detection using an imaging sensor (camera) (step S41). That is, the face area is detected by the method described with reference to FIGS. Next, a measurement area that includes the detected face area and is larger than the face area is set (step S42). That is, as shown in FIG. 9A, a measurement area oversized from the face area is set. Then, the time during which the face area exists in the measurement area is measured using a timer (step S43). That is, after setting the measurement area, timer measurement is started and the time during which the face area is located within the measurement area is measured. Then, it is determined whether or not a predetermined time or more has elapsed (step S44), and when it has elapsed, the gaze mode is turned on (step S45). As a result, the display of the operation icon for the touch panel is turned on, or a gaze image is displayed. On the other hand, if it has not elapsed, the gaze mode is kept off (step S46). By doing so, it becomes possible to detect the user's gaze state and set the gaze mode accordingly.

4). Operation Icon Display Control According to Speech State In this embodiment, the display mode of the touch panel operation icon is changed according to the user's speech state (conversation state). Specifically, the words constituting the speech uttered by the user are extracted, the extracted words are matched with the keywords of the content image (similarity determination), and the operation icon is displayed.

  For example, in FIG. 12A, the user UA sees a photo that is a content image by a slide show. Then, looking at the picture in FIG. 12 (A), he says, “It ’s a nice scenery. Where is the place?” Then, the speech of the user UA is analyzed by voice recognition, and words such as “scenery”, “location”, and “where” are extracted.

  On the other hand, various keywords corresponding to the content are set as tags in the content image of the photograph displayed in FIG. Then, it is determined whether or not words such as “scenery”, “location”, and “where is it” that constitute the user's voice extracted by voice recognition match (similar) the keyword set as the tag of the content image Is done. In the case of FIG. 12A, the word “location” matches the keyword “location” set in the content image. Accordingly, in FIG. 12B, a touch panel operation icon corresponding to the matched keyword “place” is displayed. Specifically, an operation icon “shooting location: to map” is displayed as indicated by E1. Then, when the user UA touches the operation icon E1, a map of the photo shooting location is displayed on the digital photo frame 300. As a result, the user UA can know the shooting location of the photograph. Also, if it is a picture taken by itself, it becomes possible to remember the place where the picture was taken.

  In FIG. 13 (A), the user UA sees the photos displayed by the slide show and says, "It's a nice scenery. What is this mountain?" Then, the speech of the user UA is analyzed by voice recognition, and words such as “scenery”, “mountain”, and “something” are extracted. In this case, since the extracted word “mountain” matches the keyword “mountain” in the content image, the touch panel corresponding to the keyword “mountain” as shown in FIG. An operation icon is displayed. Specifically, an operation icon “To mountain information” is displayed as indicated by E2. Then, when the user UA touches the operation icon E2, information on the mountains shown in the photograph is displayed on the digital photo frame 300. As a result, the user UA can know the mountain information shown in the photograph in detail.

  Thus, in this embodiment, the user's utterance state is analyzed, and the operation icons for the touch panel corresponding to the user's utterance state are displayed. Therefore, the user simply speaks what he / she feels, and an operation icon corresponding to the content of the utterance is displayed. By operating the operation icon, the user can easily know information he / she wants to know. In this case, since the operation icon corresponding to the word that the user uttered unconsciously is displayed, there is an advantage that the user can intuitively reach the information he / she wants to know.

  In FIGS. 12A to 13B, words uttered by the user are extracted and operation icons corresponding to the extracted words are displayed. However, the present embodiment is not limited to this. For example, the user's emotion or the like may be inferred from the user's voice state (tone, size, speed), and an operation icon corresponding to the emotion or the like may be displayed.

  The voice recognition process is a process in which a voice language spoken by a user (human) is analyzed by a computer and extracted as text data. The speech recognition process is realized by acoustic analysis, a recognition decoder, an acoustic model, a dictionary, and a language model. In acoustic analysis, a user's voice is converted into feature amount information by signal processing such as Fourier analysis. The recognition decoder outputs text data based on the feature amount information. Specifically, sound is converted into text data by comprehensively determining acoustic information and language information. The determination process in the recognition decoder is realized by a statistical method such as a hidden Markov model or a dynamic time expansion / contraction method. A dictionary is a data of words (words) to be recognized, and associates phoneme strings with words. The language model is a data of probabilities for words in the dictionary. Specifically, the appearance probability and connection probability of each word are converted into data. By using such an acoustic analysis, recognition decoder, acoustic model, dictionary, and language model, the user's voice as shown in FIGS. 12 (A) to 13 (B) is properly recognized and matched with the keyword. Judgment can be realized.

  Next, a specific processing example of operation icon display control according to the user's utterance state will be described with reference to the flowchart of FIG.

  First, the user's speech is recognized by speech (step S51). Various known methods can be employed for this voice recognition. And the word of the voice which the user uttered is extracted (step S52). In this case, in order to realize proper word extraction, it is desirable to perform a user's voice learning process.

  Next, a keyword set as a tag in the content image displayed when the user speaks is acquired (step S53). Specifically, the keyword tag associated with the content image is acquired in the content information storage unit 322 of FIG. Then, a keyword matching the word extracted in step S52 is searched (step S54). If a keyword matching the word is searched, a touch panel operation icon corresponding to the keyword is displayed (step S56). If there is no completely matching keyword, an operation icon corresponding to the keyword similar to the word may be displayed.

5). In this embodiment, the display mode of the operation icons for the touch panel is changed according to the recognition result of the number of users. Specifically, the display mode of the operation icon is different depending on whether it is determined that the number of users is one or more than one.

  For example, in FIG. 15A, only one user UA is detected by the sensor 350. In this case, an operation icon for personal mode corresponding to one detected user UA is displayed. Specifically, in FIG. 15A, a content image of an unorganized photo is displayed. The user UA touches the operation icons shown in G1, G2, G3, and G4, and selects one of the content images for personal mode, shared (family) mode, normal use, or visitor, respectively.・ Classify and touch the G5 operation icon. When discarding the content image, the G6 operation icon is touched.

  On the other hand, in FIG. 15B, a plurality of users UA, UB, and UC are detected by the sensor 350. In this case, operation icons for a sharing mode (group mode) corresponding to a plurality of users UA, UB, and UC are displayed. Then, for example, by touching the operation icon of G7, another family photo on the shooting date of the photo is displayed. Further, by touching the G8 operation icon, a family photo or the like similar to the photo is searched and displayed by a similar image search or the like. In this way, when a family gathers, it is possible to display and enjoy shared photos on the digital photo frame 300, and to provide an interface environment using a touch panel suitable for such a situation.

  Moreover, in this embodiment, the display mode of the operation icon for touch panels can be changed according to the result of user authentication of the user. Specifically, when it is determined by personal authentication that the user is a registered user, an operation icon for the registered user's personal mode is displayed. The display mode of the operation icon is changed based on the user's age recognition result.

  FIG. 16A shows an example of an input screen for user registration information. The user inputs user registration information using the operation unit or the touch panel display while viewing the user registration screen displayed on the display unit 340 of the digital photo frame 300 as shown in FIG. Thereby, user registration processing is realized. The user registration information may be set using a PC (personal computer) or the like, and the set user registration information may be transferred to the digital photo frame 300 directly or via a home server or the like.

  In FIG. 16A, the user ID, password, user hobbies / preferences, and the like are input as the user registration information. For example, in FIG. 16A, it is input that the user's hobbies / preferences are investments. In addition to hobbies and preferences, user registration includes user favorite information, screen mode settings, number of screen divisions, gaze control on / off, display time, slide display method, character font, character size, language, etc. You may enable it to input as information.

  After inputting such user registration information, the recognition result of the user's face image is registered as shown in FIG. Specifically, face detection is performed and a face area FAR is detected. Next, image recognition processing is performed on the user image shown in the face area FAR, and feature point data of the user face image is extracted. Then, the extracted feature point data is registered as a face image recognition result in association with, for example, the user registration information described with reference to FIG. In this way, the registration processing unit 314 implements user registration processing.

  Next, as shown in FIG. 17A, when an adult user UB approaches the digital photo frame 300, the person authentication whether the detected user UB is a registered user by a sensor 350 such as an image sensor is detected. Is done. The detected user UB is determined to be a registered user because the user registration as shown in FIGS. 16A and 16B is performed. Therefore, as shown by F1 and F2 in FIG. 17A, operation icons for personal mode corresponding to the detected user UB are displayed.

  Specifically, the hobby / preference of the user UB is set as investment as shown in FIG. Therefore, in F1 and F2 of FIG. 17A, operation icons related to investment are displayed as operation icons for the personal mode of the user UB. For example, when the user UB touches the operation icon F1, the company information related to the digital camera, which is the content, is displayed on the digital photo frame 300. Further, when the operation icon F2 is touched, a share transition of the digital camera by manufacturer is displayed. Thereby, the user UB who is interested in investment can obtain useful information related to the content image.

  On the other hand, in FIG. 17B, the user UC as a child is approaching the digital photo frame 300, and person authentication is performed to determine whether or not the user UC detected by the sensor 350 is a registered user. This user UC is set to have a hobby / preference as a machine on the user registration screen. Therefore, as shown in F3 and F4 of FIG. 17B, operation icons for personal mode corresponding to the detected hobbies and preferences of the user UC are displayed. For example, when the user UC touches the operation icon F <b> 3, information about the camera mechanism is displayed on the digital photo frame 300. When the F4 operation icon is touched, information about the history of the camera is displayed.

  As described above, if the display mode of the operation icon is changed according to whether or not the user is a registered user, the operation icon corresponding to the user registration customized for the user can be displayed to the registered user.

  In FIG. 17A and FIG. 17B, user age recognition processing is also performed based on detection information from a sensor 350 such as an imaging sensor. In FIG. 17B, it is determined by age recognition processing using the sensor 350 that the age group of the user UC is a child (predetermined age group in a broad sense). In this case, an operation icon for children (in a broad sense, an operation icon for a predetermined age group) is displayed. Specifically, a kana is assigned to a kanji or the like whose operation icon is difficult. It should be noted that difficult kanji characters for operation icons may be displayed in hiragana, displayed using only hiragana, or may use colors and fonts preferred by children. When it is determined that the user's age group is an elderly person (predetermined age group), an operation icon for the elderly (operation icon for the predetermined age group) may be displayed. Specifically, large and easy-to-read characters are used, bold characters are used, and colors and fonts preferred by the elderly are used.

  As described above, if the display mode of the operation icon is changed according to the age recognition result, the operation icon can be displayed in an appropriate display mode according to the age group of the user.

  Various processes can be considered as the user's age recognition process. For example, an average face pattern generated from a large number of child face patterns is created as a child face template, and an average face pattern generated from a large number of adult (or elderly) face patterns is created as an adult face template. Then, the child template is scanned on the screen of the captured image, the correlation with the captured image is obtained, the child is detected based on the correlation value, and the adult template is scanned on the screen of the captured image to capture the image. A correlation with an image is obtained, and an adult is detected based on the correlation value. Or, if the number of wrinkles is specified by face image recognition, if the number of wrinkles is small, it is determined that the child is a child, if the number of wrinkles is large, it is determined that the child is an adult, and the number of wrinkles is larger You may decide that you are an old man.

  Next, a specific processing example of display control of operation icons according to the number of users and the result of person authentication will be described.

  FIG. 18 is a flowchart of the user registration process. First, a user registration screen as shown in FIG. 16A is displayed (step S61). As a result, the user can input user registration information. Then, it is determined whether or not the input of the user registration information has been completed (step S62). If it has been completed, as described with reference to FIG. 16B, an image sensor (camera) is used to detect the face area of the user. Detection is performed (step S63). Then, the face image of the user is recognized using the imaging sensor (step S64), and the recognized face image is stored in association with the user registration information (step S65).

  FIG. 19 is a flowchart illustrating a detailed processing example of display control of an operation icon based on a user recognition result. Steps S71 to S74 in FIG. 19 are the same as steps S1 to S4 in FIG.

  After the content image is displayed in step S74 of FIG. 19, the number of users (persons) is recognized using a pyroelectric sensor or an imaging sensor (step S75). If the number of persons is one (step S76), the user is authenticated by face image recognition using an image sensor (step S77). When the person authentication is successful (step S78), the personal mode touch panel operation icon is displayed as shown in FIG. 15A using the user registration information of the authenticated user (step S79). At this time, a content image for personal mode may be displayed. On the other hand, if the person authentication fails, a touch panel operation icon in the normal mode (default setting) is displayed (step S80).

  If it is determined in step S76 that there are a plurality of users, the user is authenticated by face image recognition using an image sensor (step S81). When the person authentication is successful (step S82), as shown in FIG. 15B, the operation icons for the touch panel in the shared mode of the plurality of authenticated users are displayed (step S83). At this time, a content image for the sharing mode may be displayed. On the other hand, if the person authentication fails, a touch panel operation icon in the normal mode (default setting) is displayed (step S84).

  By doing as described above, it is possible to display an appropriate touch panel operation icon according to the recognition result of the number of users and the result of person authentication.

6). Construction of Kansei Model Now, in order to perform display control that reflects the user's preference and behavioral tendency, it is effective to construct the user's Kansei model. In order to construct such a sensitivity model, it is desirable to record various states of the user when the display mode is the personal mode or the sharing mode.

  For example, in FIG. 20, it is assumed that the display mode switching unit 316 switches the display mode to the personal mode. In the personal mode, when the user performs various operations by touching the touch panel operation icon, the operation history is written in the personal information storage unit 332 as personal information. For example, the user operation history information is written in association with the content image displayed in the personal mode.

  And based on the user's personal information stored in this way, the user's sensitivity model (personal model) is updated. In this way, it becomes possible to construct a sensitivity model for each user without the user being aware of it. Then, by using the sensitivity model updated in this way, and then performing display control of the operation icons for the touch panel, it is possible to realize the display control of the operation icons suitable for the user's sensitivity, and an unprecedented type of digital Photo frames can be provided. For example, it is possible to realize display control in which an operation icon frequently selected by the user in the personal mode is preferentially displayed on the display unit 340, or the user can easily select the operation icon.

  Alternatively, in FIG. 20, it is assumed that the display mode is switched to the shared mode by the display mode switching unit 316. In this case, the information update unit 317 writes operation histories for a plurality of users in the user shared information storage unit 334 as shared information of the plurality of users. For example, the operation history information is written in association with the content image displayed in the sharing mode.

  And based on the user share information memorize | stored in this way, the share sensitivity model of a some user is updated. In this way, it is possible to construct a shared sensibility model in which operations performed in the sharing mode of a plurality of users are reflected. As a result, usage that is characteristically seen only in the sharing mode is reflected, and an unprecedented type of digital photo frame can be provided. For example, an operation icon that is selected only when two users are together can be preferentially displayed on the display unit 340, or display control that facilitates selection of the operation icon can be realized.

  Note that a probability model can be used as the sensitivity model, and an example of such a probability model is a hidden Markov model. In this hidden Markov model (HMM), the system assumes that the parameter is an unknown Markov process and estimates the unknown parameter from observable information. That is, the HMM has a probability that a certain system has an internal state that transitions according to a Markov process and estimates the state transition inside the system from the appearance probability distribution of the symbol when it is considered that a symbol corresponding to each state is output stochastically. It is a model. Only a series of symbols can be observed from the outside, and it is called “hidden” because the internal state transition cannot be observed directly.

7). Modified Example Next, a modified example of the present embodiment will be described. FIG. 21 shows a first modification of the present embodiment. In the system of the first modification, a home server 200 (information processing system in a broad sense) is provided. The home server 200 includes a processing unit 202, a storage unit 220, a communication unit 238, and an operation unit 260. Various modifications may be made such as omitting some of these components or adding other components. In addition, about the component similar to FIG. 2, the same code | symbol is attached | subjected and the description is abbreviate | omitted.

  The processing unit 202 performs various types of processing such as management processing, and can be realized by a processor such as a CPU or an ASIC. The storage unit 220 is a work area for the processing unit 202 and the communication unit 238, and can be realized by, for example, a RAM or an HDD. The communication unit 238 is for performing wired or wireless communication with the digital photo frame 300 or the external server 600, and can be realized by a communication ASIC or a communication processor. The operation unit 260 is for the server administrator to input various information.

  In FIG. 21, the digital photo frame 300 and the home server 200 are communicatively connected via a network such as a wireless LAN. In FIG. 21, home sensors installed at various places in the house are used as the sensors 250. Then, the detection information acquisition unit 204 of the home server 200 acquires detection information from the sensor 250 that is a home sensor, and the acquired detection information is stored in the detection information storage unit 224.

  The detection information acquired by the home server 200 is transferred to the digital photo frame 300 by the transfer unit 205 via the communication units 238 and 338. Then, the detection information acquisition unit 304 of the digital photo frame 300 acquires the transferred detection information, and the acquired detection information is stored in the detection information storage unit 324.

  The user state determination unit 306 determines a positional relationship with the user, a user's visual recognition state, and an utterance state based on the acquired detection information. Further, based on the acquired detection information, recognition processing such as recognition of the number of users and user authentication of the user is performed. The display mode switching unit 316 switches the display mode of operation icons and content images according to the positional relationship, the visual recognition state, the speech state, the number of people recognition result, the person authentication result, and the like. The information update unit 317 updates information in the personal information storage unit 332 and the user shared information storage unit 334 based on the operation history and the like.

  The display control unit 318 performs control to display operation icons and content images on the display unit 340. For example, control is performed to display operation icons and content images according to the positional relationship, visual recognition state, speech state, number of people recognition result, person authentication result, and the like.

  The content information such as the content image is downloaded from the content information storage unit 222 of the home server 200 to the content information storage unit 322 of the digital photo frame 300. Alternatively, it may be downloaded from the content information storage unit 622 of the external server 600. A program for realizing the processing unit 302 of the digital photo frame 300 may also be downloaded from the external server 600 or the home server 200 to the digital photo frame 300.

  FIG. 22 shows an installation example of a home sensor that is the sensor 250. In FIG. 22, cameras 251, 252, 253, and 254 having imaging sensors are installed as home sensors at, for example, four corners of a room ceiling. Then, by photographing the user 10 and the digital photo frame 300 using these cameras 251 to 254, the positional relationship between the user 10 and the display unit 340 of the digital photo frame 300, and the visual state of the user 10 with respect to the display unit 340 Detection information for determining the above is acquired. In addition, identification of the number of users 10 and person authentication of the users 10 are performed. The acquired detection information is transferred from the home server 200 to the digital photo frame 300, and the digital photo frame 300 determines the user state or performs user recognition based on the detection information, and the user state and the user recognition result. An operation icon or a content image corresponding to is displayed on the display unit 340. The home sensor is not limited to the imaging sensors of the cameras 251 to 254, and various sensors such as a pyroelectric sensor and an ultrasonic sensor can be used.

  If the 1st modification of FIG. 21 is used, for example, the home sensor provided for home security etc. will be used effectively, detection information will be acquired, a user state may be judged based on this detection information, User recognition is performed, and operation icons and content images corresponding to the user state and the user recognition result can be displayed on the display unit 340 of the digital photo frame 300.

  FIG. 23 shows a second modification of the present embodiment. In FIG. 23, the processing unit 202 of the home server 200 includes a user status determination unit 206, a registration processing unit 214, a display mode switching unit 216, and an information update unit 217 in addition to the detection information acquisition unit 204 and transfer unit 205. Also included is a display instruction unit 218 that instructs the digital photo frame 300 to display. In addition to the content information storage unit 222 and the detection information storage unit 224, the storage unit 220 of the home server 200 includes a user registration information storage unit 225, a user state storage unit 226, a switching flag storage unit 228, personal information (sensitivity model). A storage unit 332 and a user shared information (shared sensitivity model) storage unit 334 are included.

  In FIG. 23, based on the detection information from the home sensor, which is the sensor 250, the user state determination unit 206 of the home server 200 determines the positional relationship, the visual recognition state, the utterance state, etc. Perform authentication. The registration processing unit 214 of the home server 200 also performs user registration processing.

  The display mode switching unit 216 performs processing for switching the display mode according to the user state and the user recognition result. For example, the display mode is switched according to the positional relationship with the user, the visual recognition state, and the like. Alternatively, the display mode is switched according to the number of users, the result of person authentication, and the like.

  The display instruction unit 218 issues a display instruction to change the display mode of the touch panel operation icon and the content image in accordance with the detection information acquired by the sensor 250. Specifically, a display that changes the display mode of operation icons and content images displayed on the display unit 340 according to at least one of the positional relationship between the user and the display unit 340, the user's visual recognition state, and the speech state. Give instructions. Alternatively, a display instruction for changing the display mode of the operation icon or the content image is performed according to at least one of the detected number of users and the person authentication result.

  The display control unit 318 of the digital photo frame 300 performs display control of the display unit 340 in accordance with an instruction from the display instruction unit 218. As a result, the display mode of the operation icon and the content image on the display unit 340 changes according to the positional relationship with the user, the user's visual state and speech state, the number of users, the person authentication result, and the like.

  According to the second modified example of FIG. 23, the home server 200 performs a user state determination process, a user recognition process, and a display mode switching process, so that the processing load on the digital photo frame 300 can be reduced. Therefore, even when the processing capability of the processing unit 302 (CPU) of the digital photo frame 300 is low, the processing of this embodiment can be realized. Note that these processes may be realized by distributed processing of the home server 200 and the digital photo frame 300.

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term (distance, gaze state, etc.) described together with a different term (positional relationship, viewing state, etc.) in a broader sense or the same meaning at least once in the specification or drawing, It can be replaced by that different term. This embodiment also describes the configuration and operation of digital photo frames, information processing systems, operation icon display control methods, user state determination methods, user recognition methods, positional relationship detection methods, visual state detection methods, and the like. However, the present invention is not limited to the above, and various modifications can be made.

1A and 1B are diagrams illustrating examples of digital photo frames. 2 is a configuration example of a digital photo frame according to the present embodiment. FIG. 3A and FIG. 3B are explanatory diagrams of a display control method for operation icons according to the positional relationship with the user. FIG. 4A and FIG. 4B are also explanatory diagrams of a display control method for operation icons in accordance with the positional relationship with the user. 5A and 5B are explanatory diagrams of a method for detecting the distance between the user and the display unit. The flowchart of the display control process of the operation icon according to the positional relationship with a user. FIG. 7A and FIG. 7B are explanatory diagrams of a display control method of operation icons according to the user's visual recognition state. Explanatory drawing of the display control method of the operation icon according to a user's visual recognition state. FIGS. 9A to 9C are explanatory diagrams of a method for detecting a user's gaze state. The flowchart of the display control process of the operation icon according to a user's visual recognition state. The flowchart for demonstrating the detection process of a gaze state. FIGS. 12A and 12B are explanatory diagrams of a display control method of operation icons according to the user's utterance state. FIGS. 13A and 13B are explanatory diagrams of a display control method of operation icons according to the user's utterance state. The flowchart of the display control process of the operation icon according to a user's utterance state. FIG. 15A and FIG. 15B are explanatory diagrams of operation icon display control methods according to the recognition result of the number of users. 16A and 16B are explanatory diagrams of user registration processing. FIG. 17A and FIG. 17B are explanatory diagrams of a display control method of operation icons according to a user person authentication result. The flowchart for demonstrating a user registration process. The flowchart of the display control process of the operation icon according to the number of users and a person authentication result. Explanatory drawing of the update method of personal information or user shared information. The 1st modification of this embodiment. Home sensor installation example. The 2nd modification of this embodiment.

Explanation of symbols

200 server, 202 processing unit, 204 detection information acquisition unit, 205 transfer unit,
206 user status determination unit, 214 registration processing unit, 216 display mode switching unit,
217 information update unit, 218 display instruction unit, 220 storage unit,
222 content information storage unit, 224 detection information storage unit,
225 user registration information storage unit, 226 user status storage unit,
228 switching flag storage unit, 238 communication unit,
250 sensors (home sensors), 260 operation units,
300 digital photo frame, 302 processing unit, 304 detection information acquisition unit,
306 user state determination unit, 307 positional relationship determination unit, 308 visual recognition state determination unit,
309 Utterance state determination unit, 310 User number recognition unit, 311 Person authentication unit,
316 display mode switching unit, 317 information updating unit, 318 display control unit,
320 storage unit, 322 content information storage unit, 324 detection information storage unit,
325 user registration information storage unit, 326 user state storage unit,
328 switching flag storage unit, 330 information storage medium,
338 communication unit, 340 display unit, 350 sensor, 360 operation unit

Claims (29)

A display for displaying images;
A display control unit that performs display control of the display unit;
A detection information acquisition unit for acquiring detection information detected by the sensor,
The display unit includes a touch panel display,
The display control unit
A digital photo frame characterized by performing display control for changing a display mode of a touch panel operation icon in accordance with the acquired detection information. In claim 1,
Based on the acquired detection information, determination of the positional relationship between the user and the display unit, determination of the user's visual status with respect to the display unit, determination of the user's utterance state, recognition of the number of users, and the user's person Including a user state determination unit that performs at least one of authentication,
The display control unit
Display of touch panel operation icons in accordance with at least one of the positional relationship between the user and the display unit, the user's visual recognition state, the user's speech state, the number of users' recognition result, and the user's person authentication result A digital photo frame characterized by performing display control for changing a mode. In claim 2,
The user status determination unit
As the positional relationship between the user and the display unit, the distance between the user and the display unit is determined,
The display control unit
A digital photo frame which performs display control of an operation icon for a touch panel in accordance with a distance between a user and the display unit. In claim 3,
The display control unit
A digital photo frame that turns on display of an operation icon for a touch panel when a distance between a user and the display unit is within a predetermined distance. In claim 3 or 4,
The display control unit
When it is determined that the distance between the user and the display unit is greater than a predetermined distance, control is performed to slide-display the first to Nth (N is an integer of 2 or more) content images on the display unit. When the Kth (1 ≦ K ≦ N) content image among the first to Nth content images is displayed, it is determined that the distance between the user and the display unit is within a predetermined distance. If so, the digital photo frame is controlled to display a touch panel operation icon corresponding to the Kth content image. In any of claims 3 to 5,
The sensor is an imaging sensor that images a user;
The user status determination unit
A digital device that detects a user's face area based on imaging information from the imaging sensor and determines a distance between the user and the display unit based on the detected size of the face area. Photo frame. In any of claims 3 to 5,
The sensor is an imaging sensor that images a user;
The user status determination unit
A digital photo frame, wherein a distance between a user and the display unit is determined by an automatic focusing process for the user. In any of claims 3 to 5,
The sensor is an ultrasonic sensor;
The user status determination unit
A digital photo frame, wherein a distance between a user and the display unit is determined by the ultrasonic sensor. In any of claims 2 to 8,
The user status determination unit
As the user's visual state, it is determined whether or not the user is gazing at the display unit,
The display control unit
A digital photo frame, wherein display control of an operation icon for a touch panel is performed according to whether or not a user is gazing at the display unit. In claim 9,
The display control unit
A digital photo frame that turns on display of a touch panel operation icon when it is determined that a user is gazing at the display unit. In claim 9 or 10,
The display control unit
A digital photo frame that performs control to display an operation icon for a touch panel corresponding to a content image that a user is gazing at. In any of claims 9 to 11,
The display control unit
When it is determined that the user is not gazing at the display unit, the first to Nth (N is an integer of 2 or more) content images are controlled to be slid on the display unit. When it is determined that the user is gazing at the display unit when the Kth (1 ≦ K ≦ N) content image among the Nth content images is displayed, the Kth content image A digital photo frame that performs control to display an operation icon for a touch panel corresponding to. In any of claims 2 to 12,
The user status determination unit
The user's utterance state is judged by voice recognition,
The display control unit
A digital photo frame which performs display control of an operation icon for a touch panel according to the result of the voice recognition. In claim 13,
The display control unit
A touch panel corresponding to a matched or similar keyword when a word constituting the user's voice extracted by the voice recognition matches or is similar to a keyword set as a tag in the content image displayed on the display unit A digital photo frame characterized by performing control to display an operation icon. In any of claims 2 to 14,
The user status determination unit
Recognize the number of detected users,
The display control unit
Display control is performed so that the display mode of the operation icons for the touch panel displayed on the display unit is different depending on whether the number of detected users is determined to be one or more than one. A digital photo frame characterized by In claim 15,
The display control unit
A digital photo frame characterized in that, when it is determined that the number of detected users is one, control for displaying an operation icon for a personal mode touch panel corresponding to the detected one user is performed. In claim 15 or 16,
The display control unit
When the detected number of users is determined to be plural, the digital photo is controlled to display an operation icon for a shared mode touch panel corresponding to the detected plural users. flame. In claim 15,
The display control unit
When it is determined that the detected user is the first user, control for displaying a touch panel operation icon for the first user is performed, and the detected user is determined to be the second user. If so, a digital photo frame that performs control to display a touch panel operation icon for a second user. In any of claims 2 to 18,
Including a registration processing unit for performing user registration processing;
The user status determination unit
Performs person authentication to determine whether the detected user is a registered user,
The display control unit
A digital photo frame, wherein when it is determined that a detected user is a registered user, control for displaying an operation icon for a personal mode touch panel corresponding to the registered user is performed. In any of claims 2 to 19,
The user status determination unit
Recognize the age of the detected user,
The display control unit
A digital photo frame, wherein display control of an operation icon for a touch panel is performed based on a detected age recognition result of a user. In claim 20,
The display control unit
Based on the detected age recognition result of the user, when it is determined that the user's age group is a predetermined age group, control is performed to display the operation icon for the touch panel for the predetermined age group. Digital photo frame. Any one of claims 2 to 21
A digital photo frame comprising a display mode switching unit for switching a display mode of a touch panel operation icon in accordance with at least one of the detected number of users and a person authentication result. In any one of Claims 1 thru | or 22.
The display control unit
When the image displayed on the display unit is the first content image, a touch panel operation icon associated with the first content image is displayed, and the image displayed on the display unit is the second content image. In the case of a content image, the digital photo frame is controlled to display a touch panel operation icon associated with the second content image. In any one of Claims 1 thru | or 23.
The display control unit
A digital photo frame that turns off the display of a touch panel operation icon when a user operation is not performed after a predetermined time has elapsed after the display of the touch panel operation icon is turned on. In any one of Claims 1 thru | or 24.
A digital photo frame comprising: an information updating unit for updating a user's personal information by writing a user's operation history for the touch panel operation icon displayed on the display unit into a personal information storage unit. A display instruction unit for instructing display of an image displayed on the display unit of the digital photo frame;
A detection information acquisition unit for acquiring detection information detected by the sensor,
The display unit includes a touch panel display,
The display instruction unit
An information processing system that performs a display instruction to change a display mode of an operation icon for a touch panel displayed on the display unit in accordance with the acquired detection information. A digital tough photo frame control method comprising:
Perform display control of the display,
While obtaining the detection information detected by the sensor,
The display unit includes a touch panel display,
A control method for performing display control for changing a display mode of a touch panel operation icon in accordance with the acquired detection information. A display control unit that performs display control of a display unit on which an image is displayed;
As a detection information acquisition unit that acquires detection information detected by the sensor,
Make the computer work,
The display unit includes a touch panel display,
The display control unit
The program which performs the display control which changes the display mode of the operation icon for touchscreens according to the acquired said detection information.   A computer-readable information storage medium, wherein the program according to claim 28 is stored.

Images