JP4355823B2 - Information processing device for facial expressions - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention determines the psychology of the person from the facial expression and gesture of the person image input from the digital camera, and based on the determination result, the pet moving image reflecting the psychology of the person image is input to the screen of the display device. The present invention relates to an information processing apparatus such as a facial expression used as a personal authentication engine, a condition determination engine of a home care system, or a pet robot, which is displayed as an image feedback signal.
[0002]
[Prior art]
Mobile phones and personal computers with digital cameras are well known in the art, and these have a network connection function and an interactive function.
[0003]
[Problems to be solved by the invention]
In a conventional mobile phone with a digital camera or personal computer, the psychology of the person image taken with the digital camera is analyzed in real time from the expression of the person image, and the animated pet moving image reflecting the person's psychology is displayed as a feedback image. No one has the function of giving an emotional stimulus to the person in front of the digital camera by the feedback pet image and causing the person to induce the next facial expression. Currently, it is possible to quantify human facial expressions by extracting the movement of each part. When the emotional psychology of a person in front of a digital camera is determined and an expressive animated pet moving image that reflects this emotional psychology is displayed on the screen as a feedback signal of that person's facial expression, A person causes various reactions specific to that person. This reaction can be used as various judgment data. For example, if the facial expression of a person taken with a digital camera was “suffering”, and the emotional stimulus was applied to create a joyful expression through a pet image on the screen, the “suffering” facial expression would not disappear, It turns out that there is a problem with the health of the person.
The present invention displays an expressive pet moving image on the screen as a feedback image of the image of the person in front of the digital camera, and uses the reaction of the person in front of the digital camera to the pet moving image to authenticate the person. The purpose of this is to enable the determination.
[0004]
[Means for Solving the Problems]
To achieve the above object, the present invention provides a CPU, a storage device connected to the CPU, a three-dimensional image processing device connected to the CPU, a digital camera connected to the three-dimensional image processing device, and the CPU. A display database that records past facial expression data of a specific person in the storage device, and expresses a person or animal displayed as a moving image on the screen of the display device as an animated moving image A facial expression database in which various facial expressions of the pet moving image recorded are recorded, and an information processing program, and the information processing program extracts and quantifies a human facial expression taken by the digital camera; the motion extraction means compares the facial expression of the past database and quantified expressions, and history difference detecting means for outputting a difference, the history difference test Determining means for determining the psychology of the input person image based on the difference signal of the means, emotion generating means for determining the facial expression of the pet moving image based on the determination result of the determining means, and emotion determined by the emotion generating means based on the output of the signal, it is constituted by a display control means for displaying the pet moving image corresponding to the emotion signal on a screen of the display device as a feedback signal of the input person image from the facial expression database, the history difference detection Means for collating the facial expression of the human image inputted in response to the pet moving image fed back to the screen and the facial expression data in the past database, and the determining means is based on the result of the collation Thus, it is determined that the person of the inputted person image is specified.
In the present invention, the motion extraction means extracts and quantifies the person's gesture in addition to the person's facial expression, and the past data relating to the past gesture of the specific person is recorded in the past database, and the history difference The detection means compares the gesture data quantified by the motion detection means with the data of the past database, and outputs the difference.
According to the present invention, a speaker is connected to the CPU via an audio device so that a message is emitted from the speaker together with the video of the pet moving image.
According to the present invention, the emotion generating means outputs a signal indicating emotions such as anger, fear, joy and sadness based on the determination result of the determining means.
According to the present invention, a microphone is connected to the CPU via a voice processing device, voice print extraction means is provided in the information processing program, voice print data of a specific person is recorded in the past database, and the history difference detection means is The voiceprint extraction signal extracted by the voiceprint extraction means and the voiceprint data of the past database are compared and the difference is output, and the psychology of the input person image is determined based on the output result. .
[0005]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.
Reference numeral 2 denotes an information processing apparatus for facial expressions used in a system for performing information processing such as human facial expressions, voices, and actions, and has a communication function that can be connected to a network such as the Internet. The information processing apparatus 2 has all of these functions so that it can be used as a mobile phone or various portable information terminals. The information processing device 2 includes a display device 4 using liquid crystal, a digital camera 6, a stereo microphone system including left and right microphones 8a and 8b, and left and right speakers 10a and 10b. A stereo speaker system is installed. Reference numeral 12 denotes a storage device built in the information processing apparatus 2, and the storage device 12 includes an information processing program, a database necessary for executing the program, and a pet video created by animation of a virtual person or animal. A pet moving image facial expression database for displaying the image 14 on the screen of the display device 4 is stored.
[0006]
Reference numeral 16 denotes a CPU built in the information processing apparatus 2, to which a storage device 12, a three-dimensional image processing device 18, a three-dimensional sound processing device 20, a display device 4, and an acoustic device 22 are connected. The information processing program includes motion extraction means 24, voiceprint extraction means 26, history difference detection means 28, emotion extraction means 30, psychological extraction means 32, determination means 34, emotion generation means 36, display control means 38, and sound control means 40. It consists of and. The motion extraction means 24 analyzes the output image of the digital camera 6 using the three-dimensional image processing device 18 and extracts the motion of a person taken by the digital camera 6. This movement includes human facial expressions and gestures. Considering the facial expression of a person, the face can be seen as a moving body. Exercise is done by muscles, and human expressions can be described as the work of various muscles. These muscles open and close the eyes, nostrils, clefts, etc., and raise the facial and mandibular muscles that move the auricles to close the mouth, push the mandible forward, and move horizontally Masticatory muscles that move the hyoid bone to assist swallowing, the hyoid muscle that performs powerful actions such as opening the mouth by lifting the mandible, the eye muscle that freely moves the eyeball, and the pupil depending on the surrounding light and darkness Pupil sphincter and pupil dilator muscle that adjusts the amount of light in the eyeball by focusing, and tongue muscle that works to change the shape of the tongue and extend it out of the mouth are known.
[0007]
These muscles are structured such that when a group of muscles starts to move according to a certain kind of emotion, certain muscles or specific muscles contract simultaneously through peripheral nerves. The motion extraction means 24 extracts and quantifies changes in the facial expression of the person based on these muscle movements. Also, the movement of the person's limbs and the whole is extracted and quantified. In the past database 42 stored in the storage device 12, past facial expression, gesture data, and voiceprint data of the owner of the information processing device 2 are quantified and recorded. The history difference detection means 28 compares the extraction signal of the motion extraction means 24, the voiceprint extraction signal of the voiceprint extraction means 26 with the corresponding facial expression and voiceprint data in the past database 42, and detects the difference. For example, when facial expression data with the mouth closed is stored in the past database 42, and the facial expression that the movement extracting means 24 opens the mouth, the history difference detecting means 28 uses the data with the mouth closed and the mouth closed. And the difference, that is, the state that the mouth is open is detected. The emotion pattern database 44 stored in the storage device 12 records facial expressions and voiceprint patterns for various human emotions. The emotion extraction means 30 determines which emotion pattern in the pattern database 44 the signal from the history difference detection means 28 belongs to, and from the facial expression, gesture and voiceprint of the person caught by the digital camera 6 and the microphones 8a and 8b, The emotion of the person is extracted. The psychological extraction means 32 extracts the psychology of the person in front of the digital camera 6 from the extraction signal of the emotion extraction means 30, that is, emotions such as surprise, fear, anger, disgust, and sadness.
[0008]
Based on the extraction signal of the psychological extraction unit 32, the determination unit 34 determines, according to the purpose of the program, what action should be taken for the person in front of the digital camera 6 through the pet moving image 14 and sound. The emotion generating means 36 generates emotions, that is, emotional signals such as anger, fear, joy and sadness based on the determination result of the determination means 34. It is stored in the storage device 12. The facial expression database 46 stores a database of pet moving images 14 of various facial expressions corresponding to various emotion output signals from the emotion generating means 36. The voice database 48 stores voice data corresponding to various emotions. When an emotion signal is input from the emotion generator 36, the display controller 38 displays the pet moving image 14 on the screen of the display device 4 based on the animation data in the facial expression database 46, and displays the digital camera 6 in front of the digital camera 6. A visually emotional stimulus is given to the person 50 through the movement and expression of the pet moving image 14. Similarly, when an emotion signal is input from the emotion generating unit 36, the voice control unit 40 sends voice to the speakers 10 a and 10 b through the acoustic device 22 based on the data in the voice database 48, thereby giving an emotional stimulus to the person 50. . This operation is repeated.
[0009]
Therefore, the facial expression, gesture, and voice of the person in front of the digital camera inputted for the second time are given emotional stimulation by the intention display of the pet moving image on the screen, and the digital camera 6 is used as an image in response thereto. And input to each means after the microphones 8a and 8b.
As is clear from the above description, the series of data processing systems from the digital camera 6 and the microphones 8a and 8b of the apparatus 2 to the input unit to the output unit for displaying the pet moving image 14 constitute a closed loop, and the history Data processing is repeated so that the difference output of the difference detection means 28 becomes zero. During the repeated data processing, the past database 42 stores movement data such as facial expressions, gestures, and voiceprint data caught by the digital camera 6 and the microphones 8a and 8b, and a computer built in the information processing apparatus 2. Will have a learning function for the person who owns the device 2. The judging means 34 makes a judgment according to the purpose of the program based on the short-term and long-term changes in the facial expression, gesture, and tone of the person 50.
[0010]
Next, an embodiment will be described in which the present apparatus 2 is constituted by a mobile phone and this mobile phone is used as a person authentication engine.
A description will be given of how the apparatus identifies a user when the user 50 is wearing sunglasses, for example. The mobile phone stores an information processing program for personal authentication in advance in the storage device 12. Since mobile phones are usually not equipped with stereo speakers or stereo microphones, they are equipped in advance. When the person 50 who is a user takes a picture with the digital camera 6, the motion extraction means 24 extracts sunglasses. When facial expression data with sunglasses is not recorded in the past database 42, the history difference detection means 28 outputs a quantified history difference signal.
[0011]
The emotion generating means 36 outputs an emotion signal for giving a stimulus for the user to return to a normal facial expression and outputs a message “let's take off sunglasses”. Due to the output of the emotion generating means 36, the pet moving image 4 appears on the screen, and the user's facial expression is different from the usual ones. The message “No” is issued, and emotional stimulation is given to the person 50 in front of the digital camera 6, that is, the user. In accordance with the pet moving image 14 on the screen and the message from the speaker, the user removes the sunglasses from the face and takes this state with the camera 6. This image is sent to the motion extraction means 24. The history difference detection means 28 collates with the facial expression data not wearing glasses in the past database 42, and outputs zero when there is no difference. The determination means 34 determines that the person in front of the digital camera 6 is a specific person registered in the past database when the history difference detection means 28 outputs zero.
[0012]
When the user clicks the authentication button on the screen, an authentication message is displayed on the screen. When the user changes the hairstyle or the face is scratched, the voice of the pet moving image 14 and the speakers 10a and 10b gives the user a message that is different from the usual expression. In response to this message, the user sends images with various facial expressions, so that the determination unit 34 can authenticate the user as a specific person with high accuracy. In addition to the method used as the person authentication engine, the present apparatus can be widely applied to a condition determination engine and the like in a home care system.
[0013]
【The invention's effect】
Since the present invention is configured as described above, the person in front of the camera can be specified with high accuracy, and person authentication and the like can be performed with high accuracy.
[Brief description of the drawings]
FIG. 1 is a block diagram of the present invention.
FIG. 2 is a block explanatory diagram of the present invention.
FIG. 3 is an explanatory diagram of the present invention.
[Explanation of symbols]
2 Information processing device 4 Display device 6 Digital camera 8a Microphone 8b Microphone 10a Speaker 10b Speaker 12 Storage device 14 Pet moving image 16 CPU
18 Three-dimensional image processing device 20 Three-dimensional sound processing device 22 Acoustic device 24 Motion extraction means 26 Voiceprint extraction means 28 History difference detection means 30 Emotion extraction means 32 Psychological extraction means 34 Determination means 36 Emotion generation means 38 Display control means 40 Voice control Means 42 Past database 44 Pattern database 46 Facial expression database 48 Voice database 50 Person

Claims (5)

A CPU, a storage device connected to the CPU, a three-dimensional image processing device connected to the CPU, a digital camera connected to the three-dimensional image processing device, and a display device connected to the CPU; In addition, a past database that records past facial expression data of a specific person, and various facial expressions of a pet moving image that expresses a person or animal displayed as a moving image in the form of animation on the screen of the display device are recorded. a facial expression database is, an information processing program is stored, said information processing program, a motion extractor to quantify extracting facial expressions of a person taken by the digital camera, a facial expression that the motion extracting means is quantified A history difference detecting means for comparing the facial expression of the past database and outputting the difference, and an input person based on a difference signal of the history difference detecting means A determination means for determining the psychology of an image; an emotion generation means for determining an expression of the pet moving image based on a determination result of the determination means; and the expression database based on an output of an emotion signal determined by the emotion generation means consists pet moving image corresponding to the emotion signal from at a display control means for displaying on the screen of the display device as a feedback signal of the input figures, the history difference detecting means, is fed back to the screen pet The facial expression of the person image input in response to the moving image is collated with the facial expression data in the past database, and the determination means determines the person of the input human image based on the result of the collation An information processing apparatus for facial expressions, etc., characterized in that a determination for identifying a face is performed. The motion extraction means extracts and quantifies the person's gesture in addition to the facial expression of the person, and gesture data relating to past gestures of a specific person is recorded in the past database, and the history difference detection means includes the history difference detection means, The information processing apparatus for facial expression or the like according to claim 1, wherein the data of the gesture quantified by the motion detection means is compared with the data of the past database and the difference is output. A facial expression according to claim 1 or claim 2, wherein a speaker is connected to the CPU via an audio device so that a message is issued from the speaker together with the video of the pet moving image. Information processing device. The claims 1 to 3 are characterized in that the emotion generating means outputs a signal indicating emotions such as anger, fear, joy and sadness based on the determination result of the determining means. An information processing apparatus such as the facial expression described in any one of the above. A microphone is connected to the CPU via a voice processing device, voice print extraction means is provided in the information processing program, voice print data of a specific person is recorded in the past database, and the history difference detection means is the voice print extraction means The voiceprint extraction signal extracted from the voice database is compared with the voiceprint data of the past database, and the difference is output, and the psychology of the input person image is determined based on the output result. An information processing apparatus for facial expressions or the like according to any one of claims 1 to 4.

Images