JP4998202B2 - Mobile communication terminal - Google Patents

Description

  The present invention relates to a mobile communication terminal, and more particularly to a mobile communication terminal having an address book.

  In recent years, a face recognition technique for recognizing a person's face by a computer and determining who the face is becoming widely used. Patent Document 1 (Japanese Patent Application Laid-Open No. 2007-157022) describes an entrance management system that manages the entrance of a person by controlling the opening of a door using a mobile phone terminal. In the document, a mobile phone terminal possessed by a user generates image data obtained by capturing an image including the user with a camera, generates feature data from the generated image data, and stores the generated feature data in a storage unit Is described. Further, in this document, the door control device receives wireless data including feature data output from the mobile phone terminal, transmits shooting instruction information to the interlocking camera, and receives image data transmitted from the interlocking camera. Describes a technique for generating feature data from received image data, determining whether the feature data and the feature data received from the mobile phone terminal match, and opening the door if they match.

  Patent Document 2 (Japanese Patent Laid-Open No. 63-85799) registers in advance a means for capturing and inputting an image of the face of an operator who operates the terminal, a means for extracting facial features from the input image, and A workstation is described which includes means for comparing individual facial features by comparing the facial feature information and the facial features extracted from the input image.

  However, in Patent Document 1 and Patent Document 2, the face recognition technique is only used as a means for personal authentication, and is not used for other functions.

Incidentally, in recent years, mobile communication terminals such as mobile phone terminals have an address book for storing address information such as telephone numbers and mail addresses. The address book stores various information such as names and address information of acquaintances of the user of the mobile communication terminal. In Patent Document 3 (Japanese Patent Laid-Open No. 2002-92013), the face image data of a destination is stored in a telephone directory memory in association with destination identification data in a mobile phone, and the syllabary of the name of the registrant to be displayed is stored. A technique is described in which face image data is displayed in order and the corresponding destination identification data is selected by selecting the face image data when searching for the destination. Thereby, compared with the case where the character data is displayed and the destination is selected, the face image data that is easy to recognize is displayed, and the search to the destination is facilitated.
JP 2007-157022 A Japanese Patent Laid-Open No. 63-85979 JP 2002-92013 A

  The mobile communication terminal is always carried by the user, and information stored in the address book of the mobile communication terminal can be a useful information source for the user. Conventionally, however, the address book information has only been referred to when a user makes a call or sends an e-mail, and has not been fully utilized otherwise.

  For example, when a user accidentally encounters an acquaintance after a long time away from home, the name of the acquaintance may not be remembered. If you do not remember the name of an acquaintance, you may not be able to have a smooth conversation and may cause troubles such as discomfort to the other party. In such a case, the information of the acquaintance may be listed in the address book, and if you look at the address book, you can remember the name of the acquaintance and have a smooth conversation and avoid trouble. . However, the information in the address book cannot be viewed sequentially in front of the person's eyes, and the conversation may be forced to continue without remembering the name of the acquaintance. In the technique described in Patent Document 3, the face image data is only displayed so that the user can easily recognize the destination when selecting the destination. It is necessary to look at the information sequentially.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to enable a user to easily use information in an address book stored in a mobile communication terminal.

According to the present invention,
A mobile communication terminal having an address book for storing name and address information of a plurality of persons,
A storage unit that stores each person's name and address information stored in the address book in association with face feature point data for performing face recognition of the person;
A face image data input unit for inputting face image data;
A generating unit that generates face feature point data based on the face image data;
The face feature point data generated by the generating unit is compared with the face feature point data stored in the storage unit, and when there is matching face feature point data, the face feature point data is detected. And
A notification unit for notifying a user of at least a part of the name and the address information associated with the face feature point data when the matching unit detects matching face feature point data;
Is provided.

According to the present invention,
A computer for controlling a portable communication terminal having an address book for storing names and address information of a plurality of persons;
Storage means for storing each person's name and address information stored in the address book in association with face feature point data for performing face recognition of the person's face;
Face image data input means for inputting face image data;
Generating means for generating face feature point data based on the face image data;
The face feature point data generated by the generating means is checked against the face feature point data stored in the storage means, and if there is matching face feature point data, the face feature point data is detected. means,
A notification means for notifying a user of at least part of the name and the address information associated with the face feature point data when matching face feature point data is detected by the matching means;
A program is provided that functions as:

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, a user can utilize easily the information of the address book memorize | stored in the mobile communication terminal.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

  In the following embodiment, a case where the mobile communication terminal is a mobile phone terminal will be described as an example. The mobile communication terminal can be, for example, a PDA (Personal Digital Assistants) or a PHS (Personal Handyphone System) in addition to the mobile phone terminal.

(First embodiment)
In recent years, many mobile phone terminals with a built-in camera have become widespread. Therefore, it is possible to take a picture of a person in front of the person using a camera built in the mobile phone terminal. In this embodiment, an example in which a mobile phone terminal has a phone book (address book) and a camera will be described.

  In the present embodiment, the name and address information of each person stored in advance in the telephone directory is stored in association with the face feature point data of each person. For example, when the user cannot remember the name of a person who has been accidentally encountered, if the user acquires face image data with the built-in camera while holding the camera of the mobile phone terminal toward the person, the mobile phone terminal Based on the data, face feature point data is generated. Next, the mobile phone terminal compares the generated face feature point data with the face feature point data stored in advance, thereby reading the name and address information of the corresponding phone book and displaying the name and address information on the display. Or notify by voice. As a result, even when the user cannot remember the name of the person he / she meets, the user can grasp the name of the person and the like without listening to the person himself / herself only by acquiring the face image data of the person.

FIG. 1 is a block diagram showing a configuration of a mobile phone terminal according to the present embodiment.
The cellular phone terminal 100 includes an audio output unit (notification unit) 10, a display unit (notification unit) 20, a ROM 30, a work memory 40, a control unit 50, a camera (face image data input unit) 60, and an operation unit 70. . The ROM 30 includes a telephone directory data part (storage part) 32. The working memory 40 can be constituted by a RAM, and stores temporarily required data.

  The audio output unit 10 outputs various information as audio. The display unit 20 displays various information. The operation unit 70 receives an operation from the user. The camera 60 inputs face image data. Here, the face image data can be digitally converted data received by a light receiving element such as a CCD (Charged Coupled Device) when the lens is directed to a human face.

  The phone book data unit 32 stores a plurality of person names and address information in association with face feature point data of each person's face. As the face feature point data, various known techniques can be used. For example, position information of eyes, eyebrows, mouths, and the like can be used.

FIG. 2 is a diagram illustrating a data table that is an internal configuration of the telephone directory data unit 32.
The phone book data section 32 includes a memory number column, a name column, a phone book data column, and a face feature point data column. In the memory number column, a number uniquely assigned to each phone book data is stored. The name column stores the name and name of the target person of the address information. Address information such as a telephone number, a mail address, and an address is stored in the telephone book data column. In the face feature point data column, face feature point data generated based on the face image data of the target person of each address information is stored. The face feature point data need only be stored in association with any one of the memory number, name, and phone book data in the phone book data unit 32, and may not be physically stored in the same memory.

  Returning to FIG. 1, the control unit 50 includes a CPU 51, a face feature point data generation unit (generation unit) 52, and a collation unit 53. The CPU 51 controls the entire mobile phone terminal 100. The face feature point data generation unit 52 generates face feature point data from the face image data input from the camera 60. The face feature point data generated by the face feature point data generation unit 52 is temporarily stored in the work memory 40.

  The collation unit 53 collates the facial feature point data generated by the facial feature point data generation unit 52 with the facial feature point data stored in the telephone directory data unit 32, and when there is matching facial feature point data, The face feature point data is detected. For example, the collation unit 53 estimates the face direction based on position information such as eyes, eyebrows, and mouth in the face feature point data, and determines the degree of matching of the face feature point data according to the face direction. You can also.

  When the matching unit 53 detects matching face feature point data, the display unit 20 displays name and address information associated with the face feature point data. The voice output unit 10 may output the name and address information by voice. Here, only a part of the name and address information may be notified, or the whole may be notified.

FIG. 3 is a flowchart showing a processing procedure of the mobile phone terminal 100 according to the present embodiment.
When the user wants to search the name and address information of the person from the phone book based on the face image data of the person in front of the user, the user operates the operation unit 70 to perform face feature point data matching processing. Instruct. When the user gives an instruction for face feature point data collation processing (YES in S100), the camera is activated (S102). Note that the processes in steps S100 and S102 may be performed in the reverse order, and the facial feature point data collation process may be instructed while the camera is activated. For example, the user of the mobile phone terminal 100 may point the camera 60 at the target person with the camera 60 activated, and press the button for face feature point data matching processing instead of taking a picture.

  When the camera 60 inputs face image data, the face feature point data generation unit 52 generates face feature point data based on the face image data (S104). If the face image cannot be recognized, an error notification can be performed. If the face image can be recognized, a notification indicating that processing is in progress can be performed. The face feature point data generated by the face feature point data generation unit 52 is stored in the work memory 40.

  The collation unit 53 collates the facial feature point data of the subject stored in the work memory 40 with the facial feature point data stored in the telephone directory data unit 32 (S106). If there is matching face feature point data (YES in S108), the collation unit 53 reads information such as the name of the registrant and telephone book data associated with the matched face feature point data. The display unit 20 or the audio output unit 10 notifies the user of the information read by the collation unit 53 (S110). For example, when an earphone is connected to the mobile phone terminal 100, a voice notification can also be performed.

FIG. 4 is a diagram illustrating an example of information displayed on the display unit 20.
For example, the face feature point data generated in step S104 of FIG. 3 matches the face feature point data “data 001” of the memory number “001” stored in the telephone directory data section 32 described with reference to FIG. Suppose that In this case, the collation unit 53 receives from the phone book data unit 32 the name “Ichiro Suzuki” associated with the matched face feature point data “data 001” and the phone number “XXX-YYYY-1111” of the phone book data, The mail address “11@x.mail.com” and the like are read out. The display unit 20 displays these pieces of information.

  On the other hand, if there is no matching facial feature point data in step S108 of FIG. 3 (NO in S108), the collation unit 53 determines that there is no matching facial feature point data via the audio output unit 10 or the display unit 20. In addition to notifying the user, an inquiry is made as to whether or not to register the data of the target person in the telephone directory data section 32 (S112). If there is a registration instruction from the user (YES in S112), registration in the telephone directory is performed (S114). Thus, by registering new face feature point data in the telephone directory data part 32, it is possible to make the search from the next time easier. On the other hand, if there is no registration instruction from the user in step S112 (NO in S112), the process is terminated as it is.

  As described above, according to the mobile phone terminal of the present embodiment, the name and address information of the phone book (address book) basically provided in the mobile communication terminal, the facial feature point data of each person's face, It is possible to make it possible for the user to easily use the information in the phone book simply by storing the information in association with each other.

  The function of storing human face image data as telephone directory information is installed in many mobile phone terminals in recent years, as described in Patent Document 3, for example. However, conventionally, there has been no function for searching for a person whose name or the like is not found by searching the phone book using face feature point data generated based on face image data as a search key. According to the mobile phone terminal 100 in the present embodiment, for example, even when a user accidentally encounters an acquaintance after a long time away from home or the like and cannot remember the name of the acquaintance, the name of the person can be obtained in real time with a simple operation. And address information can be notified. That is, the user simply acquires the person's face image data using the camera built in the mobile phone terminal 100, and searches for the corresponding person's name and address information from the phone book using the face feature point data as a key. Since it is possible, the name of the other party can be easily grasped. Thereby, it becomes possible to avoid the trouble when the user cannot remember the name of the acquaintance. Further, even in the case of a disease such as dementia in which memory ability is reduced and the face cannot be remembered, the name of the person can be grasped.

  As described above, according to the mobile phone terminal 100 in the present embodiment, the mobile phone that is usually carried around is the part of memorizing the face of a person that has conventionally only been memorized or memorized. This can be easily performed using a mobile communication terminal such as a terminal. Moreover, it can be usefully used as a care function for a person with symptoms such as dementia and decreased memory ability.

  Furthermore, according to the mobile phone terminal 100 in the present embodiment, the facial feature point data generated based on the facial image data is registered instead of registering the facial image data itself in the telephone directory. There is no need to hold such photographic data.

  Further, according to the mobile phone terminal 100 in the present embodiment, only the face feature point data needs to be stored, so the amount of data can be reduced compared to the face image data, and even in a mobile phone terminal with a small storage capacity. The facial feature point data of a large number of people can be stored. As a result, it is not necessary to store data for searching for a person in a server or the like, and it is not necessary to access the server or the like via a network at the time of search, and there is no extra communication fee. However, in the present invention, the function of registering face image data in the conventional telephone directory is not excluded.

  Furthermore, since it is only necessary to acquire face feature point data instead of acquiring face image data, it is not necessary to take a photograph. Therefore, even when using a mobile communication terminal that produces a shutter sound when taking a picture, it is possible to search for a partner without making a shutter sound.

(Second Embodiment)
In recent years, some mobile phone terminals have short-range communication functions such as infrared rays and Bluetooth. Therefore, even if a camera is not built in the mobile phone terminal, it is possible to transfer a photograph taken by another camera to the mobile phone terminal on the spot by short-range communication.

  The present embodiment is different from the first embodiment in that the face image data is acquired by an external camera instead of the camera built in the mobile phone terminal, and the face image data is acquired by short-range communication or the like. . When acquiring the face image data of the other party using a camera built in the mobile phone terminal, it is necessary to point the mobile phone terminal at the other party. Therefore, the other party may feel uncomfortable or suspicious. In this embodiment, for example, a camera is built in glasses, and the face image data of the other party acquired by the camera is transferred to the mobile phone terminal by short-range communication, etc. Perform the verification process. Thereby, the name and address information of the other party can be grasped in a natural state.

FIG. 5 is a block diagram showing the configuration of mobile phone terminal 100 and camera 200 in the present embodiment.
In the present embodiment, the mobile phone terminal 100 may or may not have the camera 60 described with reference to FIG. Hereinafter, a configuration without the camera 60 is shown as an example. The cellular phone terminal 100 is different from the example shown in FIG. 1 in that it does not have the camera 60 but has the communication unit 80, but the other configuration is the same. The camera 200 also has a communication unit 210. The communication unit 210 and the communication unit 80 perform data transmission / reception by short-range communication such as infrared rays and Bluetooth, or wired communication.

  The mobile phone terminal 100 may have a camera 60 as in the configuration shown in FIG. In this case, the mobile phone terminal 100 can be configured to include a selection unit that selects which of the face image data input from the built-in camera 60 and the face image data input from the camera 200 is used. In this way, depending on the situation, it is possible to set which of the camera 60 built in the mobile phone terminal 100 or the image data input from the external camera 200 is used, and more naturally the collation process It can be performed.

  Also in this embodiment, the same effect as that of the first embodiment can be obtained.

(Third embodiment)
In the present embodiment, instead of notifying the user of the address information collated by the collation unit 53 as it is, the detected address information is notified to the user in the form of a quiz. This is different from the second embodiment.

  In the present embodiment, the telephone directory data unit 32 can store a memo or the like in addition to the information described with reference to FIG. FIG. 6 is a diagram showing a data table that is an internal configuration of the telephone directory data section in the present embodiment. The phone book data part 32 further includes a memo field in addition to the configuration shown in FIG.

  In the present embodiment, processes similar to those in the first embodiment are performed up to step S108 described with reference to FIG. 3 in the first embodiment. If there is matching face feature point data in step S108 (YES in S108), the collation unit 53 provides information such as the name of the registrant, the phone book data, and notes associated with the matched face feature point data. Is read. The display unit 20 or the voice output unit 10 does not notify the user of the information read out by the collation unit 53 as it is, but only the preset information such as only the information in the memo column, only the address, etc. to the user. Notice.

  The notification to the user may be performed in a quiz format using an application such as a game installed in the mobile phone terminal 100, for example. As a result, the user can be reminded of a person's name and the like in a game sense, and can also be used for brain training. FIG. 7 is a diagram illustrating an example of information displayed on the display unit 20 in the present embodiment. Here, only the information in the memo column of the telephone directory data portion 32 is displayed, and a “detail” button 22 is displayed along with it. When the user selects the “details” button 22, the name of the phone book data section 32, the phone number of the phone book data, the mail address, and the like are displayed as shown in FIG. 4.

  Also in this embodiment, the same effect as the first embodiment and the second embodiment can be obtained. Further, the user can perform brain training.

  In FIG. 1 and FIG. 5, the configuration of parts not related to the essence of the present invention is omitted. Each component of the mobile phone terminal 100 shown in FIG. 1 is not a hardware unit configuration but a functional unit block. Each component of the cellular phone terminal 100 is centered on an arbitrary computer CPU, memory, a program that realizes the components shown in the figure loaded in the memory, a storage unit such as a hard disk for storing the program, and a network connection interface. It is realized by any combination of hardware and software. It will be understood by those skilled in the art that there are various modifications to the implementation method and apparatus.

  As mentioned above, although embodiment of this invention was described with reference to drawings, these are the illustrations of this invention, Various structures other than the above are also employable.

  For example, by registering face feature point data in the phone book data section 32, face recognition technology is used to collate people in photos stored in the mobile phone terminal 100, and organize the photo data. It can also be used for the function to perform. For example, face feature point data is generated based on the photo data stored in the mobile phone terminal 100, and the face feature point data is compared with the face feature point data registered in the phone book data unit 32 to match. If there is face feature point data to be processed, processing such as renaming the file name of the photo data based on information such as the name associated with the face feature point data, creating a folder for each person, and distributing them It can be performed.

It is a block diagram which shows an example of a structure of the mobile telephone terminal in embodiment of this invention. It is a figure which shows an example of the data table which is an internal structure of a telephone directory data part. It is a flowchart which shows the process sequence of the mobile telephone terminal in embodiment of this invention. It is a figure which shows an example of the information displayed on a display part. It is a block diagram which shows the other example of a structure of the mobile telephone terminal in embodiment of this invention. It is a figure which shows the other example of the data table which is an internal structure of a telephone directory data part. It is a figure which shows the other example of the information displayed on a display part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Audio | voice output part 20 Display part 22 Detailed button 32 Telephone directory data part 40 Work memory 50 Control part 52 Face feature point data generation part 53 Collation part 60 Camera 70 Operation part 80 Communication part 100 Cellular phone terminal 200 Camera 210 Communication part

Claims (5)

A mobile communication terminal having an address book for storing name and address information of a plurality of persons,
Each person's name , address information stored in the address book , and each person's name, phone number, and e-mail address, and different memo information , which is information related to the person, A storage unit for storing in association with face feature point data for performing human face recognition;
A face image data input unit for inputting face image data;
A generating unit that generates face feature point data based on the face image data;
The face feature point data generated by the generating unit is compared with the face feature point data stored in the storage unit, and when there is matching face feature point data, the face feature point data is detected. And
When matching face feature point data is detected by the matching unit, the memo information associated with the face feature point data is notified to the user, and then the face feature point is received in accordance with an instruction from the user. A notification unit for notifying a user of at least a part of the name and the address information associated with data;
Including mobile communication terminals. The mobile communication terminal according to claim 1,
  A mobile communication terminal in which the memo information includes information indicating a relationship between a user of the mobile communication terminal and each of the plurality of persons. The mobile communication terminal according to claim 1 or 2,
The face image data input unit is a built-in camera, and the face image data is input from the camera. The mobile communication terminal according to claim 1 or 2,
The face image data input unit is a mobile communication terminal that inputs the face image data input from an external camera by short-range wireless communication or wired communication. A computer for controlling a portable communication terminal having an address book for storing names and address information of a plurality of persons;
Each person's name , address information stored in the address book , and each person's name, phone number, and e-mail address, and different memo information , which is information related to the person, Storage means for storing in association with face feature point data for performing human face recognition;
Face image data input means for inputting face image data;
Generating means for generating face feature point data based on the face image data;
The face feature point data generated by the generating means is checked against the face feature point data stored in the storage means, and if there is matching face feature point data, the face feature point data is detected. means,
When matching face feature point data is detected by the matching unit, the memo information associated with the face feature point data is notified to the user, and then the face feature point is received in accordance with an instruction from the user. Notification means for notifying a user of at least a part of the name and the address information associated with data
Program to function as.

Images