WO2012128861A1

WO2012128861A1 - Using face recognition to direct communications

Info

Publication number: WO2012128861A1
Application number: PCT/US2012/024816
Authority: WO
Inventors: Juana E. Nakfour; Ashish N. DORE
Original assignee: Motorola Mobility LLC
Current assignee: Motorola Mobility LLC
Priority date: 2011-03-24
Filing date: 2012-02-13
Publication date: 2012-09-27
Anticipated expiration: 2013-09-24
Also published as: US20120242840A1

Abstract

Disclosed are methods for using face-recognition software to enable a communicative act. For example, a user (102) points the camera on his device (104) at a friend (106). Software analyzes the image produced by the camera, detects (302) the friend's face (202), and recognizes the friend. The recognized face is then associated (308) with a profile, and an address for the friend is retrieved (310) from the profile. The address can be used (312) in communicating with the friend. The image containing the face may be retrieved (300) by the user's device (104) from a remote source (108). Aspects of the present invention are directed toward any type of communicative act. The communicative act need not be directed toward the friend whose face was recognized. For example, when that friend's profile is retrieved, it can be searched for a reference to another person, and the communicative act is then directed to that other person.

Description

USING FACE RECOGNITION TO DIRECT COMMUNICATIONS

FIELD OF THE INVENTION

[0001] The present invention is related generally to computing devices and, more particularly, to communications among such devices.

BACKGROUND OF THE INVENTION

[0002] As personal communications devices (e.g., cell phones) are developed to support greater and greater functionality, people are using them to do much more than talk. As is well known, these devices now usually allow their users to create media files (e.g., by taking a picture or by recording a video using a camera on the device) and to download media files from remote servers (via a web interface supported by the device). People want to use their devices to share these media files with their friends.

[0003] However, the development of new functionality on these devices has far outpaced the development of friendly interfaces that allow a user to comfortably control his device's new capabilities. Sometimes, the design of a user interface is constrained by the small size of the device and by the sheer number of functions that the device supports. For whatever reason, a user often finds that the control interface for a new function of his device is at best cumbersome and sometimes confusing. For example, if the user wishes to send a media file stored on his device to a friend standing next to him, he may first have to navigate through a list of contact and then pull up a separate media-sharing menu. This sort of complication limits the utility and attractiveness of the new features.

BRIEF SUMMARY

[0004] The above considerations, and others, are addressed by the present invention, which can be understood by referring to the specification, drawings, and claims. According to aspects of the present invention, face-recognition software is used for enabling a communicative act. For example, a user points the camera on his device at a friend. Software (on the device or accessed remotely) analyzes the image produced by the camera, detects the friend's face, and recognizes the friend. The recognized face is then associated with a profile (e.g., in a contacts list on the user's device), and an address for the friend is retrieved from the profile. The address can be used in communicating with the friend, all without the user ever having to explicitly manipulate his contacts list.

[0005] The image containing the face may be captured by a camera local to the device, as in the example above, or it may be retrieved by the user's device from a remote source. For example, the face may be detected in a movie clip downloaded from a media server.

[0006] Aspects of the present invention are directed toward any type of communicative act. As a few examples, the user may send an e-mail to the friend, post a file on the friend's social networking site, or establish a live communications link with the friend. The image with the friend's face may be sent to the friend, but the invention does not require that.

[0007] The communicative act need not be directed toward the friend whose face was recognized. For example, when that friend's profile is retrieved, it can be searched for a reference to another person, e.g., the friend's mother, and the communicative act is then directed to that other person.

[0008] In some embodiments, a user interface presents the image (whether captured locally or retrieved from a remote source) to the user. If the image contains a plurality of faces, the user can choose which face to recognize.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0009] While the appended claims set forth the features of the present invention with particularity, the invention, together with its objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which:

[0010] Figure 1 is an overview of a representational environment in which aspects of the present invention may be practiced; [0011] Figure 2 is a generalized schematic of a device embodying aspects of the present invention; and

[0012] Figure 3 is a flowchart of a method for using face recognition to direct communications .

DETAILED DESCRIPTION

[0013] Turning to the drawings, wherein like reference numerals refer to like elements, the invention is illustrated as being implemented in a suitable environment. The following description is based on embodiments of the invention and should not be taken as limiting the invention with regard to alternative embodiments that are not explicitly described herein.

[0014] In the communications environment 100 of Figure 1, a user 102 wishes to use his personal communications device 104 to communicate with a friend 106. For example, the user 102 may wish to send his friend 106 a photograph he just took using a camera on his device 104, or the user 102 may wish to share a music video that he just downloaded to his device 104 from a remote server 108.

[0015] To direct his communications to his friend 106, the user 102 may use traditional methods such as pulling up a list of contacts on his personal communications device 104, searching through the list of contacts until he sees the contact profile of his friend 106, and then retrieving an e-mail address of the friend 106 from her stored profile. As an example of one aspect of the present invention, the user 102 may instead point a camera on his device 104 at his friend 106 (assuming, of course, that she is within camera range), capture an image that includes her face, and then use facial-recognition software that associates the face in the captured image with contact information for the friend 106. (The contact information itself may be stored in a list of contacts on the device 104 as in the previous art.) This method of using facial recognition to address communications can be easier to use and more intuitive that previously known addressing methods. [0016] Figure 2 shows a representative personal communications device 104 (e.g., a mobile telephone, personal digital assistant, tablet computer, or personal computer) that incorporates an embodiment of the present invention. Figure 2 shows the device 104 as a smart phone presenting its main display screen 200 to its user 102. The main display 200 is of high resolution and is as large as can be comfortably accommodated in the device 104. The device 104 may have a second and possibly a third display screen for presenting status messages. These screens are generally smaller than the main display screen 200, and they can be safely ignored for the remainder of the present discussion. In the example of Figure 2, the main display 200 shows an image either captured by a camera (not shown but well known in the art) on the other side of the device 104 or an image downloaded from a remote server 108.

[0017] A typical user interface of the personal communications device 104 includes, in addition to the main display 200, a keypad and other user-input devices. The keypad may be physical or virtual, involving virtual keys displayed on a touch screen 200.

[0018] Figure 2 illustrates some of the more important internal components of the personal communications device 104. The network interface 204 sends and receives media presentations, related information, and download requests. The processor 206 controls the operations of the device 104 and, in particular, supports aspects of the present invention as illustrated in Figure 3, discussed below. The processor 206 uses the memory 208 in its operations. Specific uses of these components by specific devices are discussed as appropriate below.

[0019] The flowchart of Figure 3 generally illustrates aspects of the present invention. In some embodiments and in some scenarios of use, some of the steps of Figure 3 are optional and may be performed in an order different from the order shown in Figure 3.

[0020] The method of Figures 3 begins in step 300 when the personal communications device 104 receives an image. The image may be captured by a camera on the device 104, or it may be downloaded to the device 104 from a remote server 108. The image may be a still image, a live image, or a video.

[0021] In step 302, facial-detection software is applied to the image, and at least one face is detected in the image. Methods of facial detection are well known in the art, and different known methods may be appropriate in different embodiments.

[0022] In one embodiment of steps 300 and 302, the camera on the personal communications device 104 faces the front of the device 104. Software monitors the image captured by the camera and tries to recognize any faces. Often, the face of the user 102 is detected as the user 102 views the display 200. The software detects this face but, recognizing it to be the face of its user 102, the software ignores it. When, however, a different face is detected (e.g., the user 102 turns the device 104 toward his friend 106, and the camera captures an image of her face), the utility proceeds with the remainder of the method of Figure 3.

[0023] Step 304 may be applied when, in step 302, more than one face is detected in the image. Step 304 selects one face in order to proceed. The selection may be automatic if, for example, one face predominates (e.g., one face covers more of the image than any other face, is in better focus, or is in a central position). In other embodiments, the user 102 may be presented with the image on the screen 200 of his personal communications device 104. The user 102 then chooses one face. In the example of Figure 2, the user 102 has maneuvered the dotted box 202 to select the face of the woman on the right of the image rather than the face of the man on the left.

[0024] (In some embodiments, more than one face can be selected and used for communications: Steps 306 through 312, below, are then performed for each selected face, that is, the communicative act of step 312 is performed with multiple recipients.)

[0025] Well known facial-recognition software is used in step 306 to analyze the selected face. That is, parametric information is derived from the facial image such as distance between the eyes, hair color, cheek-bone prominence, and the like. In some embodiments, this facial-recognition step 306 may be performed on a remote server 108. This allows the use of more computationally intensive methods than could be comfortably performed by the personal communications device 104.

[0026] The output of the facial-recognition software is used in step 308 to associate the detected face with a stored profile. For example, the user 102 may have previously taken pictures of his friends, analyzed each picture with the facial-recognition software, and stored the output parameters of the recognition as part of each friend's contact information stored on the personal communications device 104. In other embodiments, the association of the face with a profile is performed partly or wholly on a remote server 108. The stored profile itself may reside on this remote server 108.

[0027] Some facial-recognition software provides to its user a confidence score for the recognition task. In some embodiments, this confidence score can be presented to the user 102 for further consideration. If, for example, the software has only a low level of confidence that its recognition is correct, then the user 102 can be queried to see if he wishes to continue. The user 102 may decide to take a clearer picture and then re-run the method of steps 300 through 308.

[0028] Once the stored profile that is associated with the detected face has been identified in step 308, contact information is retrieved from that profile in step 310. The particular type of contact information retrieved depends in part upon the nature of the communicative act that the user 102 wishes to perform in step 312. If, for example, the user 102 wishes to send an e-mail to his friend 106, then her e-mail address is retrieved from the stored profile in step 310, and the e-mail is sent in step 312.

[0029] As another example, the user 102 may wish to post some information to his friend's social-networking site. Then, the address of that site is retrieved from the profile in step 310. [0030] The user 102 may also use the above method as a dialer to make a telephone call from his personal communications device 104 to a telephone registered to his friend 106.

[0031] In the above examples, the contact information is mostly static. In some situations, however, the contact information may be dynamic, as in some social- networking situations. Thus, the user 102 can use these methods to query if the user 106 is currently participating in an on-line game or in some other social milieu, and, if so, the user 102 can retrieve the dynamic information necessary to join the game (assuming, of course, that this information is available).

[0032] The user 102 can also join a local peer-to-peer network with his friend 106 and share content over that network with her. In this case, the friend's Bluetooth ad hoc, Wi-Fi ad hoc, or Wi-Fi Direct networking information is retrieved from the stored profile.

[0033] Other communicative acts are contemplated for step 312. The possibilities are only limited by the availability of contact information accessible in step 310.

[0034] Note that in one of the examples of Figure 1, the user 102 wishes to send an photographic image to his friend 106. In some situations, that image is the same as the image used in steps 300 and 302. For example, when the user 102 is with his friends, he can take a picture of them with his personal communications device 104, then view the image, and, using the above methods, send it to those friends who are in the image. However, there is in general no necessity for the image used in steps 300 and 302 to be used in the communicative act of step 312.

[0035] In the above example, the communication act of step 312 was performed with the person whose profile is accessed in step 310. That is not always the case, however. Instead, the profile information from step 310 can be used to retrieve other contact information. Consider, for example, the case where the user 102 wishes to tell the mother of his friend 106 that she is well. He can retrieve the stored profile of his friend 106 using the above methods and, if her profile includes contact information for her mother, retrieve that contact information and send an e-mail to the mother.

[0036] In view of the many possible embodiments to which the principles of the present invention may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the invention. For example, the communicative act may be used for public-safety purposes rather than for social interaction. Therefore, the invention as described herein contemplates all such embodiments as may come within the scope of the following claims and equivalents thereof.

Claims

CLAIMS We claim:

1. On a personal communications device (104), a method for communicating, the method comprising:

receiving (300), at the personal communications device (104), an image; detecting (302), by the personal communications device (104), a face in the image;

analyzing (306), by the personal communications device (104), the detected face;

based, at least in part, on the analyzing, associating (308) the detected face with a stored profile;

retrieving (310), at the personal communications device (104), contact information from the associated profile; and

using (312), by the personal communications device (104), the contact information to perform a communicative act.

2. The method of claim 1 wherein the image is selected from the group consisting of: a live image, a still image, and a video image.

3. The method of claim 1 wherein the profile is stored on a server remote from the personal communications device.

4. The method of claim 1 wherein performing a communicative act comprises an element selected from the group consisting of: sending an e-mail, sharing a media file, posting to a social-networking site, establishing a private network connection, transferring a current context, and joining an on-line game.

5. The method of claim 1 wherein the communicative act is performed in relation to a person who is a subject of the associated profile.

6. The method of claim 1 wherein the communicative act is performed in relation to a person associated with a person who is a subject of the associated profile.

7. The method of claim 1 further comprising:

detecting, by the personal communications device, a plurality of faces in the image;

presenting, by the personal communications device to a user of the personal communications device, the plurality of detected faces; and

receiving, by the personal communications device from the user of the personal communications device, a selection of one of the detected faces.

8. A personal communications device (104) configured for communicating, the personal communications device (104) comprising:

a camera configured for capturing an image;

a transceiver (204); and

a processor (206) operatively connected to the camera and to the transceiver (204), the processor (206) configured for:

receiving (300), from the camera, the image;

detecting (302) a face in the image;

analyzing (306) the detected face;

retrieving (310) contact information from the associated profile; and

using (312) the contact information to perform, via the transceiver (204), a communicative act.

9. The personal communications device of claim 8 wherein the profile is stored on a server remote from the personal communications device.

10. The personal communications device of claim 8 further comprising a user interface, wherein the processor is operatively connected to the user interface, the processor further configured for:

detecting a plurality of faces in the image;

presenting, via the user interface to a user of the personal communications device, the plurality of detected faces; and

receiving, via the user interface from the user of the personal communications device, a selection of one of the detected faces.