WO2007149123A2 - Method and apparatus for providing user profiling based on facial recognition - Google Patents

Method and apparatus for providing user profiling based on facial recognition Download PDF

Info

Publication number
WO2007149123A2
WO2007149123A2 PCT/US2006/048168 US2006048168W WO2007149123A2 WO 2007149123 A2 WO2007149123 A2 WO 2007149123A2 US 2006048168 W US2006048168 W US 2006048168W WO 2007149123 A2 WO2007149123 A2 WO 2007149123A2
Authority
WO
WIPO (PCT)
Prior art keywords
user
data
facial feature
facial
face
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2006/048168
Other languages
French (fr)
Other versions
WO2007149123A3 (en
Inventor
Glen P. Goffin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Arris Technology Inc
Original Assignee
General Instrument Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by General Instrument Corp filed Critical General Instrument Corp
Priority to JP2008547380A priority Critical patent/JP2009521186A/en
Publication of WO2007149123A2 publication Critical patent/WO2007149123A2/en
Anticipated expiration legal-status Critical
Publication of WO2007149123A3 publication Critical patent/WO2007149123A3/en
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/31—Indexing; Data structures therefor; Storage structures
    • G06F16/316—Indexing structures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working

Definitions

  • the present disclosure relates to user profiling, recognition, and authentication.
  • it relates to user profiling, recognition, and authentication using videophone systems or image capturing devices.
  • Audiovisual conferencing capabilities are generally implemented using computer based systems, such as in personal computers (“PCs") or videophones.
  • Some videophones and other videoconferencing systems offer the capability of storing user preferences.
  • user preferences in videophones and other electronic devices are set up such that the preferences set by the last user are the preferences being utilized by the videophone or electronic device.
  • these systems typically require substantial interaction by the user. Such interaction may be burdensome and time- consuming.
  • Face representation data is captured with an imaging device.
  • the imaging device focuses on the face of the user to capture the face representation data.
  • a determination is made as to whether a facial feature database includes user facial feature data that matches the face representation data.
  • User preference data is loaded on a memory module of the electrical device when the face representation data matches user facial feature data in the facial feature database.
  • a new user profile is added to the user profile database when the face representation data does not match user facial feature data in the facial feature database.
  • a user profiling system that includes a facial recognition module, a facial feature database, a user profiling module, and a user profiling database.
  • the facial recognition module receives face representation data, the face representation data being captured by an imaging device.
  • the imaging device focuses on the face of the user to capture the face representation data.
  • the facial feature database stores a plurality of user records, each of the plurality of user records storing face representation data.
  • each of the plurality of user records may correspond to each of a plurality of users of an electrical device.
  • the user profiling module loads user preference data on a memory module of the electrical device.
  • the user preference data is loaded on the electrical device when the face representation data matches user facial feature data in the facial feature database.
  • the user profiling module creates a new user profile when the face representation data does not match user facial feature data in the facial feature database.
  • the user profiling database stores a plurality of user profiles and corresponding user preference data, the user profiles corresponding to each of the plurality of users of the electrical device.
  • Figure 1 illustrates a videophone imaging a human face.
  • Figure 2 illustrates components and peripheral devices of a facial recognition and profiling unit.
  • Figure 3 illustrates a flowchart for a process for facial recognition and user profiling based facial recognition.
  • Figures 4A-4C illustrate examples of electronic devices that may be coupled with the facial recognition and profiling unit.
  • Figure 5 illustrates a personal data assistant interacting with the facial recognition and profiling unit over a computer network.
  • Figure 6 illustrates a block diagram of a facial recognition and profiling system.
  • a method and apparatus for automated facial recognition and user profiling is disclosed.
  • the system and method may be applied to one or more electrical systems that provide the option of setting up customized preferences.
  • These systems may be personal computers, telephones, videophones, automated teller machines, personal data assistants, media players, and others.
  • Figure 1 illustrates a videophone imaging a human face.
  • a videophone 104 utilizing a camera 110 and a facial recognition and profiling unit 100 may be configured to capture the users face, facial expressions, and other facial characteristics that may uniquely identify the user.
  • the facial recognition and profiling unit 100 receives a captured image from the camera 110, and saves the data representing the user's face.
  • the camera 110, and the facial recognition and profiling unit 100 are housed within the videophone 104.
  • the camera 110, and the facial recognition and profiling unit 100 are housed in separate housings the videophone 104.
  • the videophone 102 captures the face of the user only when the user is in a videoconference communicating with other videophone users.
  • video recognition and profiling are performed without disturbing the user's videoconferencing session.
  • the recognition and profiling are processes that are transparently carried out with respect to the user.
  • the facial recognition and profiling unit 100 may generate user preference and setting based on the user actions.
  • the videophone 102 captures the face of the user when the user is operating the videophone 102, and not necessarily during a videoconference. As such, the facial recognition and profiling unit 100 collects user action and behavior data to corresponding to any interaction between the user the videophone 102.
  • the user may set the volume at a certain level. This action is recorded by the facial recognition and profiling unit 100 and associated with the user's profile. Then, when the user returns to make another videoconference call, the user's face is recognized by the facial recognition and profiling unit 100, and the volume is automatically set to the level at which the user set it on the previous conference call.
  • both the near- end caller and the far-end caller is recognized by the facial recognition and profiling unit 100.
  • the near-end user may be a user that has been recognized in the past by the facial recognition and profiling unit 100.
  • the facial recognition and profiling unit 100 searches for the far-end caller profile and load the near-end user preferences with respect to communication with the far-end user.
  • the far-end caller preferences and data may also be load for quick retrieval or access by the facial recognition and profiling unit 100.
  • the facial recognition and profiling unit 100 may be configured to load any number of user profiles that may be parties of a conference call. The profiles, data and other associated information to the users participating in the conference call may or may not be available to other users in the conference call, depending on security settings, etc.
  • the outgoing videophone call log may be recorded for each user.
  • the contact information for the parties in communication with each user is automatically saved.
  • the contact information for all of the contacted parties in the call log may be automatically loaded.
  • the facial recognition and profiling unit 100 stores user profiles for multiple users. Thus, if a second user engages in a video conference call at the same videophone 100, the videophone 100 may recognize the second user's face, and immediately load the contact list pertinent to the second user. As such, by performing facial recognition and automatically generating user profiles, minimal user interaction is required.
  • FIG. 2 illustrates components and peripheral devices of a facial recognition and profiling unit.
  • the facial recognition and profiling unit 100 may include a facial features database 102, a user profile database 104, a facial recognition module 106, a user maintenance module 108, a processor 112, and a random access memory 114.
  • the facial features database 102 may store facial feature data for each user in the user profile database 104. In one embodiment, each user has multiple associated facial features. In another embodiment, each user has a facial feature image stored in the facial features database 102.
  • the facial recognition module 106 includes logic to store the facial features associated with each user. In one embodiment, the logic includes a comparison of the facial features of a user with the facial features captured by the camera 110. If a threshold of similarity is surpassed by a predefined number of facial features, then the captured face is authenticated as belonging to the user associated with the facial features deemed similar to the captured face.
  • the facial recognition module 106 includes logic that operates based template matching algorithms. Pre- established templates for each may be configured as part of the recognition module 106 and a comparison be made to determined the difference percentage.
  • a new user, and associated facial features and characteristics may be added if the user is not recognized as an existing user. In one embodiment, if a threshold of similarity is not surpassed by a predefined number of facial features, then the captured face is added as a new user with the newly captured facial characteristics. In another embodiment, if a threshold of similarity is surpassed by at least one facial feature, then the captured face is added as a new user with the newly captured facial characteristics.
  • the facial recognition module 106 stores images for five facial features of the user (e.g. eyes, nose, mouth, and chin) in the facial features database 102.
  • the facial recognition module 106 stores measurements of each of the facial features of a user.
  • the facial recognition module 106 stores blueprints of each of the facial features of a user.
  • the facial recognition module 106 stores a single image of the user's face.
  • the facial recognition module 106 stores new facial feature data if the user is a new user.
  • One or more pre-existing facial recognition schemes may be used to perform facial recognition.
  • the user profile database 104 may store user preferences, alternative identification codes, pre-defined commands, and other user- specific data.
  • the user maintenance module 108 includes logic to perform user profiling. In one embodiment, the maintenance module includes logic to extract a user profile based on a user identifier. The user identifier may be, for example, the user facial features stored in the facial features database 102. In another embodiment, the maintenance module 108 includes logic to save user settings under the user's profile. In another embodiment, the maintenance module 108 includes logic to interpret user operations as a user preference and save the user preference under the user's profile. In another embodiment, the maintenance module 108 includes logic to interpret user operations as a user preference and save the user preference under the user's profile. In yet another embodiment, the maintenance module 108 includes logic to add a new user if the user is not associated with an existing user profile.
  • the facial recognition and profiling unit 100 may be connected to one or more peripheral devices for input and output.
  • a camera 110 is coupled with the facial recognition and profiling unit through a communications bus 116.
  • the camera 110 captures the face of a person and generates an image of the user's face.
  • the camera 110 streams a captured data to the facial recognition module 104 without any presorting or pre-processing the images captured.
  • the camera 110 is configured to only transmit to the facial recognition module 106 images that resemble a human face.
  • a keypad 120, a microphone 118, a display 122 and a speaker 124 is connected to the facial recognition and profiling unit 100 via the communications bus 116.
  • Various other input and output devices may be in communication with the facial recognition and profiling unit 100. The inputs form various input devices may be utilized to monitor and learn user behavior and preferences.
  • the facial recognition and profiling unit 100 is separated into two components in two separate housings.
  • the facial recognition module 106 and the facial features database 102 is housed in a first housing.
  • the user profile database 104 and the user maintenance module 108 may be housed in the second housing.
  • facial recognition entails receiving a captured image of a user's face, for example through the camera 110, and verifying that the provided image corresponds to an authorized user by searching the provided image in the facial features database 102. If the user is not recognized, the user is added as a new user based on the captured faced characteristics. The determination of whether the facial features in the captured image correspond to facial features of an existing user in the facial features database 102 is performed by the facial recognition module 106. As previously stated, the facial recognition module 106 may include operating logic for comparing the captured user's face with the facial feature data representing an authorized user's faces stored in facial features database 102.
  • the facial features database 102 includes a relational database that includes facial feature data for each of the users profiled in the user profile database 104.
  • the facial features database 102 may be a read only memory (ROM) lookup table for storing data representative of an authorized user's face.
  • ROM read only memory
  • user profiling may be performed by a user maintenance module 108.
  • the user profile database 104 is a read-only memory in which user preferences, pre-configured function commands, associated permissions, etc. are stored. For example, settings such as preview inset turned on/off, user interface preferences, ring-tone preferences, call history logs, phonebook and contact lists, buddy list records, preferred icons, preferred emoticons, chat-room history logs, email addresses, schedules, etc.
  • the user maintenance module 108 retrieves and stores data on the user profile database 104 to update the pre-configured commands, preferences, etc.
  • the user maintenance module 108 includes operating logic to determine user actions that are included in the user profile.
  • the facial recognition and profiling unit 100 includes a computer processor 112, which exchanges data with the facial recognition module 106 and the user maintenance module 108.
  • the computer processor 112 executes operations such as comparing incoming images through the facial recognition module 106, and requesting user preferences, profile and other data associated with an existing user through the user maintenance module 108.
  • FIG. 3 illustrates a flowchart for a process for facial recognition and user profiling based facial recognition.
  • the process is performed by the facial recognition and profiling unit 100.
  • Process 300 starts at process block 304 wherein the camera 110 captures an image of the user's face.
  • the user's face has been captured by facial recognition module 106 which is configured to discard any incoming images that are not recognized as a human face shape.
  • the camera 100 only captures the image of the user's face if the camera 110 detects an object in the camera's 110 vicinity.
  • the camera 110 is configured to detect if a shape similar to a face is being focused by the camera 110.
  • the camera 110 forwards all the captured data to the facial recognition module 106 wherein the determination of whether a face is being detected is made.
  • the process 300 then continues to process block 306.
  • process block 306 data representing the image of the scanned face is compared against the facial feature data stored in the facial features database 102 according to logic configured in the facial recognition module 106. As such, at decision process block 306 a determination is made whether the data representing the image of the scanned face matches facial feature data representing stored the facial feature database 102. The process 300 then continues to process block 308.
  • process block 308 if the data representing the image of the scanned face matches data representing an image of at least one reference facial feature stored the facial feature database 102 user preferences are loaded on the electrical device. In one embodiment, a determination is made as to whether or not there are user preferences pre-set and stored in the user profiled database 102. If there are user preferences already in place, then the user profile and corresponding preferences are loaded on the electrical device. In another embodiment, if there are no pre-established user preferences, the user subsequent requests, actions, commands and input are collected in order to generate and maintain the user profile. In one embodiment, user preferences are automatically generated. Facial expressions, actions, commands, etc., corresponding to recognized user faces are automatically collected and stored in a user profile database. The data stored for each user may include call history logs, user data, user contact information, and other information learned while the user is using the videophone. User profiles may be generated without the need for user interaction. The process 300 then continues to process block 310.
  • the user is added as a new user to the user profile database 104.
  • Facial features data representing the user's face are added to the facial feature database 102.
  • the user profile database 104 includes a new record that may be keyed based on the user's face or facial features. Thus, every time a new user is added, a new record with associated facial features and preferences is created. Multiple users may access the system and establish a user account based on user-specific facial features.
  • FIGs 4A, 4B, 4C and 4D illustrate examples of electronic devices that may be coupled with the facial recognition and profiling unit 100.
  • the facial recognition and profiling unit 100 is incorporated into the electronic device such that the components are in the same housing.
  • the facial recognition and profiling unit 100 is provided in a separate housing from the electronic device.
  • FIG. 4A illustrates a personal computer 402 interacting with the facial recognition and profiling unit 100.
  • the personal computer 402 may be operated depending on different configurations established by the facial recognition and profiling unit 100.
  • the personal computer includes a camera 110 that feeds an image of the captured face or facial features of each user of the personal computer.
  • a user profile may be generated and stored based on a user's face or facial features.
  • the facial recognition and profiling unit 100 will retrieve user preferences and load them for interaction with the recognized . user. For example, font size, wallpaper image, preferred Internet download folder, etc., be loaded and provided by the personal computer 402 once a user is recognized and preference parameters are loaded.
  • FIG. 4B illustrates an automated teller machine 404 interacting with the facial recognition and profiling unit 100.
  • the automated teller machine 404 may be operated depending on different configurations established by the facial recognition and profiling unit 100.
  • the automated teller machine 404 includes a camera 110 that feeds an image of the captured face or facial features of each user of the automated teller machine 404.
  • a user profile may be generated and stored based on a user's face or facial features. As the user interacts with the automated teller machine 404 the new settings, preferences, and other user-specific data are learned, generated and stored by the facial recognition and profiling unit 100.
  • the facial recognition and profiling unit 100 may retrieve user preferences and load them for interaction with the recognized user. For example, display font size, voice activation, frequently used menu items, etc., is loaded and provided by the automated teller machine 404 once a user is recognized and preference parameters are loaded.
  • FIG. 4C illustrates a television unit 406 interacting with the facial recognition and profiling unit 100.
  • the television unit 406 may be operated depending on different configurations established by the facial recognition and profiling unit 100.
  • the television unit 406 includes a camera 110 that feeds an image of the captured face or facial features of each user of the television unit 406.
  • a user profile is generated and stored based on a user's face or facial features.
  • the new settings, preferences, and other user-specific data are learned, generated and stored by the facial recognition and profiling unit 100.
  • the facial recognition and profiling unit 100 may retrieve user preferences and load them for interaction with the recognized user. For example, favorite channels, sound preference, color, contrast, preferred volume level, etc., may be loaded and provided by the television unit 406 once a user is recognized and preference parameters are loaded.
  • FIG. 4D illustrates a personal data assistant 408 interacting with the facial recognition and profiling unit 100.
  • the personal data assistant 408 may be operated depending on different configurations established by the facial recognition and profiling unit 100.
  • the personal data assistant 408 includes a camera 110 that feeds an image of the captured face or facial features of each user of the personal data assistant 408.
  • a user profile may be generated and stored based on a user's face or facial features.
  • the facial recognition and profiling unit 100 may retrieve user preferences and load them for interaction with the recognized user. For example, font size, wallpaper image, and preferred Internet download folder may be loaded and provided by the personal data assistant 408 once a user is recognized and preference parameters are loaded.
  • FIG. 5 illustrates a personal data assistant 502 interacting with the facial recognition and profiling unit over a computer network.
  • the facial recognition and profiling unit 100 is located at a server 504.
  • the facial recognition and profiling unit 100 communicates with the server 504 through a network 210 such as a Local Area Network ("LAN”), a Wide Area Network ("WAN”), the Internet, cable, satellite, etc.
  • the personal data assistant 502 may have incorporated an imaging device such as a camera 110.
  • the camera 100 is connected to the personal data assistant but it is not integrated under the same housing.
  • the personal data assistant 502 may communicate with the facial recognition and profiling unit 100 to provide user facial features, user operations, and other data as discussed above.
  • the facial recognition and profiling unit 100 stores user profiles, recognize new and existing user facial features, and exchange other data with the personal data assistant 502.
  • FIG. 6 illustrates a block diagram of a facial recognition and profiling system 600.
  • the facial recognition and profiling system 600 may be employed to automatically generate users profiles and settings based on user actions, commands, order of accessing information, etc., utilizing facial recognition to distinguish among users.
  • facial recognition and profiling system 600 is implemented using a general- purpose computer or any other hardware equivalents.
  • the facial recognition and profiling system 600 comprises processor (CPU) 112, memory 114, e.g., random access memory (RAM) and/or read only memory (ROM), facial recognition module 106, and various input/output devices 602, (e.g., storage devices, including but not limited to, a tape drive, a floppy drive, a hard disk drive or a compact disk drive, a receiver, a transmitter, a speaker, a display, an image capturing sensor, e.g., those used in a digital still camera or digital video camera, a clock, an output port, a user input device (such as a keyboard, a keypad, a mouse, and the like, or a microphone for capturing speech commands)).
  • processor CPU
  • memory 114 e.g., random access memory (RAM) and/or read only memory (ROM)
  • ROM read only memory
  • facial recognition module 106 e.g., storage devices, including but not limited to, a tape drive, a floppy drive, a hard
  • the facial recognition module 106 may be implemented as one or more physical devices that are coupled to the processor 112 through a communication channel.
  • the facial recognition module 106 may be represented by one or more software applications (or even a combination of software and hardware, e.g., using application specific integrated circuits (ASIC)), where the software is loaded from a storage medium, (e.g., a magnetic or optical drive or diskette) and operated by the processor 112 in the memory 114 of the facial recognition and profiling system 600.
  • ASIC application specific integrated circuits
  • the facial recognition module 106 (including associated data structures) of the present invention may be stored on a computer readable medium, e.g., RAM memory, magnetic or optical drive or diskette and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Collating Specific Patterns (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Processing Or Creating Images (AREA)

Abstract

A method and system of providing user profiling for an electrical device is disclosed. Face representation data is captured with an imaging device. The imaging device focuses on the face of the user to capture the face representation data. A determination is made as to whether a facial feature database includes user facial feature data that matches the face representation data. User preference data is loaded on a memory module of the electrical device when the face representation data matches user facial feature data in the facial feature database. A new user profile is added to the user profile database when the face representation data does not match user facial feature data in the facial feature database.

Description

METHOD AND APPARATUS FOR PROVIDING USER PROFILING BASED ON FACIAL RECOGNITION
BY GLEN P. GOFFIN
BACKGROUND
[0001] 1. Field of the Disclosure
[0002] The present disclosure relates to user profiling, recognition, and authentication. In particular, it relates to user profiling, recognition, and authentication using videophone systems or image capturing devices.
[0003] 2. General Background
[0004] Audiovisual conferencing capabilities are generally implemented using computer based systems, such as in personal computers ("PCs") or videophones. Some videophones and other videoconferencing systems offer the capability of storing user preferences. Generally, user preferences in videophones and other electronic devices are set up such that the preferences set by the last user are the preferences being utilized by the videophone or electronic device. In addition, these systems typically require substantial interaction by the user. Such interaction may be burdensome and time- consuming.
[0005] Furthermore, images captured by cameras in videophones are simply transmitted over a videoconferencing network to the destination videophone. As such, user facial expressions and features are not recorded for any other purpose than for transmission to the other videoconferencing parties. Finally, current videophones and other electrical devices only permit setting up user preferences for a single user. SUMMARY
[0006] A method and system of providing user profiling for an electrical device is disclosed. Face representation data is captured with an imaging device. The imaging device focuses on the face of the user to capture the face representation data. A determination is made as to whether a facial feature database includes user facial feature data that matches the face representation data. User preference data is loaded on a memory module of the electrical device when the face representation data matches user facial feature data in the facial feature database. A new user profile is added to the user profile database when the face representation data does not match user facial feature data in the facial feature database.
[0007] A user profiling system that includes a facial recognition module, a facial feature database, a user profiling module, and a user profiling database. The facial recognition module receives face representation data, the face representation data being captured by an imaging device. The imaging device focuses on the face of the user to capture the face representation data. The facial feature database stores a plurality of user records, each of the plurality of user records storing face representation data. In addition, each of the plurality of user records may correspond to each of a plurality of users of an electrical device. The user profiling module loads user preference data on a memory module of the electrical device. The user preference data is loaded on the electrical device when the face representation data matches user facial feature data in the facial feature database. The user profiling module creates a new user profile when the face representation data does not match user facial feature data in the facial feature database. Finally, the user profiling database stores a plurality of user profiles and corresponding user preference data, the user profiles corresponding to each of the plurality of users of the electrical device. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] By way of example, reference will now be made to the accompanying drawings.
[0009] Figure 1 illustrates a videophone imaging a human face.
[0010] Figure 2 illustrates components and peripheral devices of a facial recognition and profiling unit.
[0011] Figure 3 illustrates a flowchart for a process for facial recognition and user profiling based facial recognition.
[0012] Figures 4A-4C illustrate examples of electronic devices that may be coupled with the facial recognition and profiling unit.
[0013] Figure 5 illustrates a personal data assistant interacting with the facial recognition and profiling unit over a computer network.
[0014] Figure 6 illustrates a block diagram of a facial recognition and profiling system.
DETAILED DESCRIPTION
[0015] A method and apparatus for automated facial recognition and user profiling is disclosed. The system and method may be applied to one or more electrical systems that provide the option of setting up customized preferences. These systems may be personal computers, telephones, videophones, automated teller machines, personal data assistants, media players, and others.
[0016] Electrical systems do not generally store and manage settings and user-specific information or multiple users. Rather, current systems provide user interfaces with limited interfacing capabilities. The method and apparatus disclosed herein automatically maintain preferences and settings for multiple users based on facial recognition. Unlike current systems which are cumbersome to operate and maintain, the system and method disclosed herein automatically generate users preferences, and settings based on user actions, commands, order of accessing information, etc. Once a facial recognition module recognizes a returning user's face, a user-profiling module may collect user specific actions generate and learn user preferences for the returning user. If the user is not recognized by the facial recognition module, a new profile may be created and settings, attributes, preferences, etc., may be stored as part of the new user's profile.
[0017] Figure 1 illustrates a videophone imaging a human face. A videophone 104 utilizing a camera 110 and a facial recognition and profiling unit 100 may be configured to capture the users face, facial expressions, and other facial characteristics that may uniquely identify the user. The facial recognition and profiling unit 100 receives a captured image from the camera 110, and saves the data representing the user's face. In one embodiment, the camera 110, and the facial recognition and profiling unit 100 are housed within the videophone 104. In another embodiment, the camera 110, and the facial recognition and profiling unit 100 are housed in separate housings the videophone 104.
[0018] In one. example, the videophone 102 captures the face of the user only when the user is in a videoconference communicating with other videophone users. Thus, video recognition and profiling are performed without disturbing the user's videoconferencing session. Thus, the recognition and profiling are processes that are transparently carried out with respect to the user. While the user is on a videoconference, the facial recognition and profiling unit 100 may generate user preference and setting based on the user actions. In another embodiment, the videophone 102 captures the face of the user when the user is operating the videophone 102, and not necessarily during a videoconference. As such, the facial recognition and profiling unit 100 collects user action and behavior data to corresponding to any interaction between the user the videophone 102.
[0019] For example, during a videoconference call the user may set the volume at a certain level. This action is recorded by the facial recognition and profiling unit 100 and associated with the user's profile. Then, when the user returns to make another videoconference call, the user's face is recognized by the facial recognition and profiling unit 100, and the volume is automatically set to the level at which the user set it on the previous conference call.
[0020] In another example, during a videoconference call, both the near- end caller and the far-end caller is recognized by the facial recognition and profiling unit 100. The near-end user may be a user that has been recognized in the past by the facial recognition and profiling unit 100. When the near-end user receives a call from an far-end caller, the facial recognition and profiling unit 100 searches for the far-end caller profile and load the near-end user preferences with respect to communication with the far-end user. In addition, the far-end caller preferences and data may also be load for quick retrieval or access by the facial recognition and profiling unit 100. The facial recognition and profiling unit 100 may be configured to load any number of user profiles that may be parties of a conference call. The profiles, data and other associated information to the users participating in the conference call may or may not be available to other users in the conference call, depending on security settings, etc.
[0021] In yet another example, the outgoing videophone call log may be recorded for each user. The contact information for the parties in communication with each user is automatically saved. When the user returns to engage in another video conference call, the contact information for all of the contacted parties in the call log may be automatically loaded. In one embodiment, the facial recognition and profiling unit 100 stores user profiles for multiple users. Thus, if a second user engages in a video conference call at the same videophone 100, the videophone 100 may recognize the second user's face, and immediately load the contact list pertinent to the second user. As such, by performing facial recognition and automatically generating user profiles, minimal user interaction is required.
[0022] Figure 2 illustrates components and peripheral devices of a facial recognition and profiling unit. The facial recognition and profiling unit 100 may include a facial features database 102, a user profile database 104, a facial recognition module 106, a user maintenance module 108, a processor 112, and a random access memory 114.
[0023] The facial features database 102 may store facial feature data for each user in the user profile database 104. In one embodiment, each user has multiple associated facial features. In another embodiment, each user has a facial feature image stored in the facial features database 102. The facial recognition module 106 includes logic to store the facial features associated with each user. In one embodiment, the logic includes a comparison of the facial features of a user with the facial features captured by the camera 110. If a threshold of similarity is surpassed by a predefined number of facial features, then the captured face is authenticated as belonging to the user associated with the facial features deemed similar to the captured face. In another embodiment, if a threshold of similarity is surpassed by at least one facial feature, then the captured face is authenticated as being the user associated with the facial feature deemed similar to the facial features in the user's face. In another embodiment, the facial recognition module 106 includes logic that operates based template matching algorithms. Pre- established templates for each may be configured as part of the recognition module 106 and a comparison be made to determined the difference percentage.
[0024] A new user, and associated facial features and characteristics may be added if the user is not recognized as an existing user. In one embodiment, if a threshold of similarity is not surpassed by a predefined number of facial features, then the captured face is added as a new user with the newly captured facial characteristics. In another embodiment, if a threshold of similarity is surpassed by at least one facial feature, then the captured face is added as a new user with the newly captured facial characteristics.
[0025] In one example, the facial recognition module 106 stores images for five facial features of the user (e.g. eyes, nose, mouth, and chin) in the facial features database 102. In another example, the facial recognition module 106 stores measurements of each of the facial features of a user. In yet another example, the facial recognition module 106 stores blueprints of each of the facial features of a user. In another example, the facial recognition module 106 stores a single image of the user's face. In another example, the facial recognition module 106 stores new facial feature data if the user is a new user. One or more pre-existing facial recognition schemes may be used to perform facial recognition.
[0026] The user profile database 104 may store user preferences, alternative identification codes, pre-defined commands, and other user- specific data. The user maintenance module 108 includes logic to perform user profiling. In one embodiment, the maintenance module includes logic to extract a user profile based on a user identifier. The user identifier may be, for example, the user facial features stored in the facial features database 102. In another embodiment, the maintenance module 108 includes logic to save user settings under the user's profile. In another embodiment, the maintenance module 108 includes logic to interpret user operations as a user preference and save the user preference under the user's profile. In another embodiment, the maintenance module 108 includes logic to interpret user operations as a user preference and save the user preference under the user's profile. In yet another embodiment, the maintenance module 108 includes logic to add a new user if the user is not associated with an existing user profile.
[0027] The facial recognition and profiling unit 100 may be connected to one or more peripheral devices for input and output. For example, a camera 110 is coupled with the facial recognition and profiling unit through a communications bus 116. The camera 110 captures the face of a person and generates an image of the user's face. In one embodiment, the camera 110 streams a captured data to the facial recognition module 104 without any presorting or pre-processing the images captured. In another embodiment, the camera 110 is configured to only transmit to the facial recognition module 106 images that resemble a human face. In another example, a keypad 120, a microphone 118, a display 122 and a speaker 124 is connected to the facial recognition and profiling unit 100 via the communications bus 116. Various other input and output devices may be in communication with the facial recognition and profiling unit 100. The inputs form various input devices may be utilized to monitor and learn user behavior and preferences.
[0028] In one embodiment, the facial recognition and profiling unit 100 is separated into two components in two separate housings. The facial recognition module 106 and the facial features database 102 is housed in a first housing. The user profile database 104 and the user maintenance module 108 may be housed in the second housing.
[0029] In one embodiment, facial recognition entails receiving a captured image of a user's face, for example through the camera 110, and verifying that the provided image corresponds to an authorized user by searching the provided image in the facial features database 102. If the user is not recognized, the user is added as a new user based on the captured faced characteristics. The determination of whether the facial features in the captured image correspond to facial features of an existing user in the facial features database 102 is performed by the facial recognition module 106. As previously stated, the facial recognition module 106 may include operating logic for comparing the captured user's face with the facial feature data representing an authorized user's faces stored in facial features database 102. In one embodiment, the facial features database 102 includes a relational database that includes facial feature data for each of the users profiled in the user profile database 104. In another embodiment, the facial features database 102 may be a read only memory (ROM) lookup table for storing data representative of an authorized user's face.
[0030] Furthermore, user profiling may be performed by a user maintenance module 108. In another embodiment, the user profile database 104 is a read-only memory in which user preferences, pre-configured function commands, associated permissions, etc. are stored. For example, settings such as preview inset turned on/off, user interface preferences, ring-tone preferences, call history logs, phonebook and contact lists, buddy list records, preferred icons, preferred emoticons, chat-room history logs, email addresses, schedules, etc. The user maintenance module 108 retrieves and stores data on the user profile database 104 to update the pre-configured commands, preferences, etc. As stated above, the user maintenance module 108 includes operating logic to determine user actions that are included in the user profile.
[0031] In addition, the facial recognition and profiling unit 100 includes a computer processor 112, which exchanges data with the facial recognition module 106 and the user maintenance module 108. The computer processor 112 executes operations such as comparing incoming images through the facial recognition module 106, and requesting user preferences, profile and other data associated with an existing user through the user maintenance module 108.
[0032] Figure 3 illustrates a flowchart for a process for facial recognition and user profiling based facial recognition. In one embodiment, the process is performed by the facial recognition and profiling unit 100. Process 300 starts at process block 304 wherein the camera 110 captures an image of the user's face. In one embodiment, at process block 304, the user's face has been captured by facial recognition module 106 which is configured to discard any incoming images that are not recognized as a human face shape. In one embodiment, the camera 100 only captures the image of the user's face if the camera 110 detects an object in the camera's 110 vicinity. In one embodiment, the camera 110 is configured to detect if a shape similar to a face is being focused by the camera 110. In another embodiment, the camera 110 forwards all the captured data to the facial recognition module 106 wherein the determination of whether a face is being detected is made. The process 300 then continues to process block 306.
[0033] At process block 306, data representing the image of the scanned face is compared against the facial feature data stored in the facial features database 102 according to logic configured in the facial recognition module 106. As such, at decision process block 306 a determination is made whether the data representing the image of the scanned face matches facial feature data representing stored the facial feature database 102. The process 300 then continues to process block 308.
[0034] At process block 308, if the data representing the image of the scanned face matches data representing an image of at least one reference facial feature stored the facial feature database 102 user preferences are loaded on the electrical device. In one embodiment, a determination is made as to whether or not there are user preferences pre-set and stored in the user profiled database 102. If there are user preferences already in place, then the user profile and corresponding preferences are loaded on the electrical device. In another embodiment, if there are no pre-established user preferences, the user subsequent requests, actions, commands and input are collected in order to generate and maintain the user profile. In one embodiment, user preferences are automatically generated. Facial expressions, actions, commands, etc., corresponding to recognized user faces are automatically collected and stored in a user profile database. The data stored for each user may include call history logs, user data, user contact information, and other information learned while the user is using the videophone. User profiles may be generated without the need for user interaction. The process 300 then continues to process block 310.
[0035] At process block 310, if the data representing the image of the scanned face does not match data representing an image of at least one reference facial feature stored the facial feature database 102 the user is added as a new user to the user profile database 104. Facial features data representing the user's face are added to the facial feature database 102. In addition, the user profile database 104 includes a new record that may be keyed based on the user's face or facial features. Thus, every time a new user is added, a new record with associated facial features and preferences is created. Multiple users may access the system and establish a user account based on user-specific facial features. [0036] Figures 4A, 4B, 4C and 4D illustrate examples of electronic devices that may be coupled with the facial recognition and profiling unit 100. In one embodiment, the facial recognition and profiling unit 100 is incorporated into the electronic device such that the components are in the same housing. In another embodiment, the facial recognition and profiling unit 100 is provided in a separate housing from the electronic device.
[0037] Figure 4A illustrates a personal computer 402 interacting with the facial recognition and profiling unit 100. The personal computer 402 may be operated depending on different configurations established by the facial recognition and profiling unit 100. In one embodiment, the personal computer includes a camera 110 that feeds an image of the captured face or facial features of each user of the personal computer. As explained above, a user profile may be generated and stored based on a user's face or facial features. As the user interacts with the personal computer 402, the new settings, preferences, and other user-specific data are learned, generated and stored by the facial recognition and profiling unit 100. In future interactions with the personal computer 402, the facial recognition and profiling unit 100 will retrieve user preferences and load them for interaction with the recognized . user. For example, font size, wallpaper image, preferred Internet download folder, etc., be loaded and provided by the personal computer 402 once a user is recognized and preference parameters are loaded.
[0038] Figure 4B illustrates an automated teller machine 404 interacting with the facial recognition and profiling unit 100. The automated teller machine 404 may be operated depending on different configurations established by the facial recognition and profiling unit 100. In one embodiment, the automated teller machine 404 includes a camera 110 that feeds an image of the captured face or facial features of each user of the automated teller machine 404. As explained above, a user profile may be generated and stored based on a user's face or facial features. As the user interacts with the automated teller machine 404 the new settings, preferences, and other user-specific data are learned, generated and stored by the facial recognition and profiling unit 100. In future interactions with the automated teller machine 404, the facial recognition and profiling unit 100 may retrieve user preferences and load them for interaction with the recognized user. For example, display font size, voice activation, frequently used menu items, etc., is loaded and provided by the automated teller machine 404 once a user is recognized and preference parameters are loaded.
[0039] Figure 4C illustrates a television unit 406 interacting with the facial recognition and profiling unit 100. The television unit 406 may be operated depending on different configurations established by the facial recognition and profiling unit 100. In one embodiment, the television unit 406 includes a camera 110 that feeds an image of the captured face or facial features of each user of the television unit 406. As explained above, a user profile is generated and stored based on a user's face or facial features. As the user interacts with the television unit 406, the new settings, preferences, and other user-specific data are learned, generated and stored by the facial recognition and profiling unit 100. In future interactions with the television unit 406, the facial recognition and profiling unit 100 may retrieve user preferences and load them for interaction with the recognized user. For example, favorite channels, sound preference, color, contrast, preferred volume level, etc., may be loaded and provided by the television unit 406 once a user is recognized and preference parameters are loaded.
[0040] Figure 4D illustrates a personal data assistant 408 interacting with the facial recognition and profiling unit 100. The personal data assistant 408 may be operated depending on different configurations established by the facial recognition and profiling unit 100. In one embodiment, the personal data assistant 408 includes a camera 110 that feeds an image of the captured face or facial features of each user of the personal data assistant 408. As explained above, a user profile may be generated and stored based on a user's face or facial features. As the user interacts with the personal data assistant 408 the new settings, preferences, and other user-specific data are learned, generated and stored by the facial recognition and profiling unit 100. In future interactions with the personal data assistant 408, the facial recognition and profiling unit 100 may retrieve user preferences and load them for interaction with the recognized user. For example, font size, wallpaper image, and preferred Internet download folder may be loaded and provided by the personal data assistant 408 once a user is recognized and preference parameters are loaded.
[0041] Figure 5 illustrates a personal data assistant 502 interacting with the facial recognition and profiling unit over a computer network. In one embodiment, the facial recognition and profiling unit 100 is located at a server 504. The facial recognition and profiling unit 100 communicates with the server 504 through a network 210 such as a Local Area Network ("LAN"), a Wide Area Network ("WAN"), the Internet, cable, satellite, etc. The personal data assistant 502 may have incorporated an imaging device such as a camera 110. In another embodiment, the camera 100 is connected to the personal data assistant but it is not integrated under the same housing.
[0042] The personal data assistant 502 may communicate with the facial recognition and profiling unit 100 to provide user facial features, user operations, and other data as discussed above. In addition, the facial recognition and profiling unit 100 stores user profiles, recognize new and existing user facial features, and exchange other data with the personal data assistant 502.
[0043] Figure 6 illustrates a block diagram of a facial recognition and profiling system 600. Specifically, the facial recognition and profiling system 600 may be employed to automatically generate users profiles and settings based on user actions, commands, order of accessing information, etc., utilizing facial recognition to distinguish among users. In one embodiment, facial recognition and profiling system 600 is implemented using a general- purpose computer or any other hardware equivalents.
[0044] Thus, the facial recognition and profiling system 600 comprises processor (CPU) 112, memory 114, e.g., random access memory (RAM) and/or read only memory (ROM), facial recognition module 106, and various input/output devices 602, (e.g., storage devices, including but not limited to, a tape drive, a floppy drive, a hard disk drive or a compact disk drive, a receiver, a transmitter, a speaker, a display, an image capturing sensor, e.g., those used in a digital still camera or digital video camera, a clock, an output port, a user input device (such as a keyboard, a keypad, a mouse, and the like, or a microphone for capturing speech commands)).
[0045] It should be understood that the facial recognition module 106 may be implemented as one or more physical devices that are coupled to the processor 112 through a communication channel. Alternatively, the facial recognition module 106 may be represented by one or more software applications (or even a combination of software and hardware, e.g., using application specific integrated circuits (ASIC)), where the software is loaded from a storage medium, (e.g., a magnetic or optical drive or diskette) and operated by the processor 112 in the memory 114 of the facial recognition and profiling system 600. As such, the facial recognition module 106 (including associated data structures) of the present invention may be stored on a computer readable medium, e.g., RAM memory, magnetic or optical drive or diskette and the like.
[0046] Although certain illustrative embodiments and methods have been disclosed herein, it will be apparent form the foregoing disclosure to those skilled in the art that variations and modifications of such embodiments and methods may be made without departing from the true spirit and scope of the art disclosed. Many other examples of the art disclosed exist, each differing from others in matters of detail only. Accordingly, it is intended that the art disclosed shall be limited only to the extent required by the appended claims and the rules and principles of applicable law.

Claims

WHAT IS CLAIMED IS:
1. A method of providing user profiling for an electrical device, comprising: capturing face representation data with an imaging device, wherein the imaging device focuses on the face of the user to capture the face representation data; determining whether a facial feature database includes user facial feature data that matches the face representation data; loading user preference data on the electrical device when the face representation data matches user facial feature data in the facial feature database; and adding a new user profile to the user profile database when the face representation data does not match user facial feature data in the facial feature database.
2. The method of claim 1 , further comprising storing new user preference data in the new user profile based on user interaction with the electrical device.
3. The method of claim 1 , further comprising storing new user history data in the new user profile based on user interaction with the electrical device.
4. The method of claim 1 , further comprising locating in the user profile database an existing user profile corresponding to the matching user facial feature data.
5. The method of claim 1 , wherein loading user preference data on the electrical device comprises loading user existing facial feature data existing on a memory module of electrical device.
6. The method of claim 1 , wherein determining whether the facial feature database includes user facial feature data that matches the face representation data is performed by a facial recognition module in the electrical device.
7. The method of claim 1 , wherein the user preference data and the history data is stored in the user profile database.
8. The method of claim 1, wherein the new user profile added to the user profile database is uniquely identifiable based on the face representation data.
9. The method of claim 1 , wherein the user preference data includes sound preference, color preferences, or video preferences.
10. A user profiling system, comprising: a facial recognition module that receives face representation data, the face representation data being captured by an imaging device, wherein the imaging device focuses on the face of the user to capture the face representation data; a facial feature database that stores a plurality of user records, each of the plurality of user records storing face representation data, wherein each of the plurality of user records corresponds to each of a plurality of users of an electrical device; a user profiling module that loads user preference data on the electrical device, the user preference data being loaded on the memory module of the electrical device when the face representation data matches user facial feature data in the facial feature database, wherein the user profiling module creates a new user profile when the face representation data does not match user facial feature data in the facial feature database; and a user profiling database that stores a plurality of user profiles and corresponding user preference data, the user profiles corresponding to each of the plurality of users of the electrical device.
PCT/US2006/048168 2005-12-20 2006-12-18 Method and apparatus for providing user profiling based on facial recognition Ceased WO2007149123A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2008547380A JP2009521186A (en) 2005-12-20 2006-12-18 Method and apparatus for providing user profiling based on facial recognition

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/312,220 2005-12-20
US11/312,220 US20070140532A1 (en) 2005-12-20 2005-12-20 Method and apparatus for providing user profiling based on facial recognition

Publications (2)

Publication Number Publication Date
WO2007149123A2 true WO2007149123A2 (en) 2007-12-27
WO2007149123A3 WO2007149123A3 (en) 2008-09-25

Family

ID=38173536

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2006/048168 Ceased WO2007149123A2 (en) 2005-12-20 2006-12-18 Method and apparatus for providing user profiling based on facial recognition

Country Status (5)

Country Link
US (1) US20070140532A1 (en)
JP (1) JP2009521186A (en)
KR (1) KR20080079685A (en)
CN (1) CN101427262A (en)
WO (1) WO2007149123A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109451334A (en) * 2018-11-22 2019-03-08 青岛聚看云科技有限公司 User, which draws a portrait, generates processing method, device and electronic equipment

Families Citing this family (78)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NO319858B1 (en) * 2003-09-26 2005-09-26 Tandberg Telecom As Identification procedure
US8125509B2 (en) * 2006-01-24 2012-02-28 Lifesize Communications, Inc. Facial recognition for a videoconference
TWI338253B (en) * 2007-06-20 2011-03-01 Quanta Comp Inc Remote control system and method for providing application program thereof
US8144944B2 (en) * 2007-08-14 2012-03-27 Olympus Corporation Image sharing system and method
US8943425B2 (en) * 2007-10-30 2015-01-27 Google Technology Holdings LLC Method and apparatus for context-aware delivery of informational content on ambient displays
US9986293B2 (en) * 2007-11-21 2018-05-29 Qualcomm Incorporated Device access control
CN103369391B (en) * 2007-11-21 2016-12-28 高通股份有限公司 The method and system of electronic equipment is controlled based on media preferences
CN101465961B (en) * 2007-12-19 2013-10-30 神基科技股份有限公司 Camera device and method for controlling shutter by feature image recognition
US20110292181A1 (en) * 2008-04-16 2011-12-01 Canesta, Inc. Methods and systems using three-dimensional sensing for user interaction with applications
US8430750B2 (en) * 2008-05-22 2013-04-30 Broadcom Corporation Video gaming device with image identification
KR101559772B1 (en) * 2008-10-16 2015-10-13 엘지전자 주식회사 Mobile terminal and Method for controlling in thereof
US8111247B2 (en) * 2009-03-27 2012-02-07 Sony Ericsson Mobile Communications Ab System and method for changing touch screen functionality
WO2011101848A1 (en) * 2010-02-18 2011-08-25 United Parents Online Ltd. Methods and systems for managing virtual identities
US20110029618A1 (en) * 2009-08-02 2011-02-03 Hanan Lavy Methods and systems for managing virtual identities in the internet
US9077951B2 (en) 2009-07-09 2015-07-07 Sony Corporation Television program selection system, recommendation method and recording method
CN102035930A (en) * 2009-09-29 2011-04-27 宏达国际电子股份有限公司 Image application operation method and system
JP2011107899A (en) * 2009-11-16 2011-06-02 Sony Corp Information processor, method and program for changing setting
KR101673032B1 (en) * 2010-01-25 2016-11-04 엘지전자 주식회사 Video communication method and digital television thereof
US20110257985A1 (en) * 2010-04-14 2011-10-20 Boris Goldstein Method and System for Facial Recognition Applications including Avatar Support
JP5713580B2 (en) 2010-04-28 2015-05-07 キヤノン株式会社 COMMUNICATION DEVICE, ITS CONTROL METHOD, AND PROGRAM
EP2570981A4 (en) * 2010-05-28 2014-04-09 Rakuten Inc CONTENT OUTPUT DEVICE, CONTENT OUTPUT METHOD, CONTENT OUTPUT PROGRAM, AND RECORDING MEDIUM WITH CONTENT OUTPUT PROGRAM
US9113190B2 (en) 2010-06-04 2015-08-18 Microsoft Technology Licensing, Llc Controlling power levels of electronic devices through user interaction
US9319625B2 (en) * 2010-06-25 2016-04-19 Sony Corporation Content transfer system and communication terminal
KR101172227B1 (en) * 2010-11-18 2012-08-07 현대자동차주식회사 System of management for entrance and exit using driver's face certification within vehicle and thereof method
CN103201731B (en) * 2010-12-02 2016-01-20 英派尔科技开发有限公司 Augmented reality system
US8462191B2 (en) 2010-12-06 2013-06-11 Cisco Technology, Inc. Automatic suppression of images of a video feed in a video call or videoconferencing system
US20130305383A1 (en) * 2010-12-22 2013-11-14 Intel Corporation System and method to protect user privacy in multimedia uploaded to internet sites
US20120226981A1 (en) * 2011-03-02 2012-09-06 Microsoft Corporation Controlling electronic devices in a multimedia system through a natural user interface
CN103098079A (en) * 2011-04-11 2013-05-08 英特尔公司 Personalized program selection system and method
US8760395B2 (en) 2011-05-31 2014-06-24 Microsoft Corporation Gesture recognition techniques
US8836530B1 (en) 2011-06-21 2014-09-16 Google Inc. Proximity wakeup
US8811685B1 (en) * 2011-06-21 2014-08-19 Google Inc. Proximity wakeup
US20130046637A1 (en) * 2011-08-19 2013-02-21 Firethorn Mobile, Inc. System and method for interactive promotion of products and services
JP5869805B2 (en) * 2011-08-25 2016-02-24 キヤノン株式会社 Information processing apparatus, imaging apparatus, control method therefor, program, and storage medium
US9338396B2 (en) * 2011-09-09 2016-05-10 Cisco Technology, Inc. System and method for affinity based switching
US20130097416A1 (en) * 2011-10-18 2013-04-18 Google Inc. Dynamic profile switching
US8635637B2 (en) 2011-12-02 2014-01-21 Microsoft Corporation User interface presenting an animated avatar performing a media reaction
US8838647B2 (en) * 2011-12-06 2014-09-16 International Business Machines Corporation Automatic multi-user profile management for media content selection
US9100685B2 (en) 2011-12-09 2015-08-04 Microsoft Technology Licensing, Llc Determining audience state or interest using passive sensor data
US8898687B2 (en) 2012-04-04 2014-11-25 Microsoft Corporation Controlling a media program based on a media reaction
CA2775700C (en) 2012-05-04 2013-07-23 Microsoft Corporation Determining a future portion of a currently presented media program
DE102012209190A1 (en) * 2012-05-31 2013-12-05 Siemens Aktiengesellschaft Method for detecting information of at least one object arranged on a patient support device in a medical imaging device and a medical imaging device for carrying out the method
CN102760215B (en) * 2012-06-27 2016-05-04 北京奇虎科技有限公司 A kind of method and system of carrying out user interface release based on image recognition
CN102779179B (en) * 2012-06-29 2018-05-11 华为终端（东莞）有限公司 A method and terminal for information association
CN102957743A (en) * 2012-10-18 2013-03-06 北京天宇朗通通信设备股份有限公司 Data pushing method and device
KR101990536B1 (en) 2012-11-02 2019-06-18 삼성전자주식회사 Method for providing information and Electronic apparatus thereof
CN103873941A (en) * 2012-12-17 2014-06-18 联想(北京)有限公司 Display method and electronic equipment
CN103076879A (en) * 2012-12-28 2013-05-01 中兴通讯股份有限公司 Multimedia interaction method and device based on face information, and terminal
US9134792B2 (en) * 2013-01-14 2015-09-15 Qualcomm Incorporated Leveraging physical handshaking in head mounted displays
CA2902983A1 (en) * 2013-03-06 2014-09-12 Arthur J. Zito, Jr. Multi-media presentation system
JP6097632B2 (en) * 2013-05-10 2017-03-15 キヤノン株式会社 IMAGING DEVICE, ITS CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM
KR20150008769A (en) * 2013-07-15 2015-01-23 삼성전자주식회사 Image display apparatus, and method for operating the same
CN103491437A (en) * 2013-07-30 2014-01-01 深圳市睿立南方科技有限公司 TV set-top box and TV program management method thereof
US20150082330A1 (en) * 2013-09-18 2015-03-19 Qualcomm Incorporated Real-time channel program recommendation on a display device
KR20150043795A (en) * 2013-10-15 2015-04-23 삼성전자주식회사 Image processing apparatus and control method thereof
US10121060B2 (en) * 2014-02-13 2018-11-06 Oath Inc. Automatic group formation and group detection through media recognition
KR20150137499A (en) * 2014-05-29 2015-12-09 엘지전자 주식회사 Video display device and operating method thereof
CN105743850B (en) * 2014-12-10 2020-04-10 深圳市智莱科技股份有限公司 Method and device for obtaining user authentication information when delivering articles by using express box
US9945573B2 (en) * 2015-01-23 2018-04-17 Samah Mobarak Balkhair Air conditioner system with air treatment integration
US10521012B2 (en) * 2015-03-13 2019-12-31 Apple Inc. Method for automatically identifying at least one user of an eye tracking device and eye tracking device
CN106557928A (en) * 2015-09-23 2017-04-05 腾讯科技（深圳）有限公司 A kind of information processing method and terminal
CN105979364A (en) * 2015-12-01 2016-09-28 乐视致新电子科技（天津）有限公司 Smart television user data discrimination method and device
US9934397B2 (en) 2015-12-15 2018-04-03 International Business Machines Corporation Controlling privacy in a face recognition application
US10440488B2 (en) * 2016-06-27 2019-10-08 International Business Machines Corporation Intelligent audio control
CN105938552B (en) * 2016-06-29 2020-04-24 北京旷视科技有限公司 Face recognition method and device for automatically updating base map
US10706843B1 (en) * 2017-03-09 2020-07-07 Amazon Technologies, Inc. Contact resolution for communications systems
US10070275B1 (en) 2017-09-29 2018-09-04 Motorola Solutions, Inc. Device and method for deploying a plurality of mobile devices
US10460330B1 (en) 2018-08-09 2019-10-29 Capital One Services, Llc Intelligent face identification
US11502862B2 (en) * 2018-09-05 2022-11-15 Avaya Inc. Custom communication actions based on environment analysis
US11079911B2 (en) * 2018-12-26 2021-08-03 Synaptics Incorporated Enrollment-free offline device personalization
CN109710780B (en) * 2018-12-28 2022-03-15 上海依图网络科技有限公司 Archiving method and device
US11443554B2 (en) * 2019-08-06 2022-09-13 Verizon Patent And Licensing Inc. Determining and presenting user emotion
CN110929077A (en) * 2019-10-17 2020-03-27 北京海益同展信息科技有限公司 Methods, apparatus, devices, electronic devices, and computer-readable media for animal documentation
US12554814B2 (en) * 2020-06-11 2026-02-17 Telefonaktiebolaget Lm Ericsson (Publ) Authorization using an optical sensor
US11985411B2 (en) * 2022-06-24 2024-05-14 Dell Products L.P. Detection of image sensor shutter state
US12445574B2 (en) * 2022-10-27 2025-10-14 Hewlett-Packard Development Company, L.P. Identification of persons of interest in meetings
US20250379862A1 (en) * 2024-06-10 2025-12-11 Charter Communications Operating, Llc Secure authentication of digital humans
US20260204094A1 (en) * 2025-01-16 2026-07-16 Dell Products Lp System and method for automatically adjusting on-screen display settings based on detected facial patterns of a user by a visual sensor

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5404393A (en) * 1991-10-03 1995-04-04 Viscorp Method and apparatus for interactive television through use of menu windows
US5497430A (en) * 1994-11-07 1996-03-05 Physical Optics Corporation Method and apparatus for image recognition using invariant feature signals
US5945988A (en) * 1996-06-06 1999-08-31 Intel Corporation Method and apparatus for automatically determining and dynamically updating user preferences in an entertainment system
US6119096A (en) * 1997-07-31 2000-09-12 Eyeticket Corporation System and method for aircraft passenger check-in and boarding using iris recognition
US6160903A (en) * 1998-04-24 2000-12-12 Dew Engineering And Development Limited Method of providing secure user access
US7130454B1 (en) * 1998-07-20 2006-10-31 Viisage Technology, Inc. Real-time facial recognition and verification system
US6940545B1 (en) * 2000-02-28 2005-09-06 Eastman Kodak Company Face detecting camera and method
US6947922B1 (en) * 2000-06-16 2005-09-20 Xerox Corporation Recommender system and method for generating implicit ratings based on user interactions with handheld devices
US6963659B2 (en) * 2000-09-15 2005-11-08 Facekey Corp. Fingerprint verification system utilizing a facial image-based heuristic search method
US6922488B2 (en) * 2001-02-16 2005-07-26 International Business Machines Corporation Method and system for providing application launch by identifying a user via a digital camera, utilizing an edge detection algorithm
TWI282941B (en) * 2001-03-15 2007-06-21 Toshiba Corp Entrance management apparatus and entrance management method by using face features identification
US8218829B2 (en) * 2001-08-20 2012-07-10 Polycom, Inc. System and method for using biometrics technology in conferencing
US20030046557A1 (en) * 2001-09-06 2003-03-06 Miller Keith F. Multipurpose networked data communications system and distributed user control interface therefor
US8064650B2 (en) * 2002-07-10 2011-11-22 Hewlett-Packard Development Company, L.P. File management of digital images using the names of people identified in the images
US7088220B2 (en) * 2003-06-20 2006-08-08 Motorola, Inc. Method and apparatus using biometric sensors for controlling access to a wireless communication device
US20050108406A1 (en) * 2003-11-07 2005-05-19 Dynalab Inc. System and method for dynamically generating a customized menu page

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109451334A (en) * 2018-11-22 2019-03-08 青岛聚看云科技有限公司 User, which draws a portrait, generates processing method, device and electronic equipment

Also Published As

Publication number Publication date
JP2009521186A (en) 2009-05-28
US20070140532A1 (en) 2007-06-21
WO2007149123A3 (en) 2008-09-25
KR20080079685A (en) 2008-09-01
CN101427262A (en) 2009-05-06

Similar Documents

Publication Publication Date Title
US20070140532A1 (en) Method and apparatus for providing user profiling based on facial recognition
US8487976B2 (en) Participant authentication for a videoconference
US9414013B2 (en) Displaying participant information in a videoconference
US7847815B2 (en) Interaction based on facial recognition of conference participants
US8125509B2 (en) Facial recognition for a videoconference
US10904483B2 (en) System and methods for automatic call initiation based on biometric data
US8120638B2 (en) Speech to text conversion in a videoconference
US10069830B2 (en) Communication system, communication method, and computer-readable recording medium
US20100085415A1 (en) Displaying dynamic caller identity during point-to-point and multipoint audio/videoconference
US20130318234A1 (en) Advanced Availability Detection
US20050044143A1 (en) Instant messenger presence and identity management
US8948361B2 (en) Mitigating spam and identifying callers in video calls
CN106233267B (en) Access management system, communication system, recording medium, and access management method
US7519202B2 (en) System and method for secure bio-print and access methods
US9171184B2 (en) Transmission terminal, transmission system and recording medium
CN113225521A (en) Video conference control method and device and electronic equipment
KR100617677B1 (en) How to handle instant messages on your mobile device
CN111556277B (en) Method, device and equipment for processing identifiers of participants in video conference and storage medium
US10893139B1 (en) Processing interaction requests with user specific data on a shared device
KR20240052387A (en) Feature registration system for facial recognition
US12321663B2 (en) Leveraging visual data to enhance audio reception
US12615336B2 (en) Multi-profile-based inauthenticity identification
US20230308498A1 (en) Method and device for modifying a multimedia content according to the attention of a user
HK1136923A (en) Displaying dynamic caller identity during point-to-point and multipoint audio/videoconference

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200680048147.1

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 06851531

Country of ref document: EP

Kind code of ref document: A2

WWE Wipo information: entry into national phase

Ref document number: 2008547380

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 1020087017516

Country of ref document: KR

122 Ep: pct application non-entry in european phase

Ref document number: 06851531

Country of ref document: EP

Kind code of ref document: A2