JP2009135616A - Communication terminal, information processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a communication terminal capable of improving accuracy of facial recognition in identifying a person by recognizing the face of the person included in an image; an information processing method; and a program. <P>SOLUTION: This communication terminal 20 includes: communication means 200, 220 and 270 communicating with the outside; a communication history storage means 210 storing communication history information by relating information of a person of a communication counterpart thereto; and an image data storage means 210 storing data of an image. The communication terminal is also provided with: a facial recognition means 260 identifying a person by recognizing the face of the person included in an image based on data of the image; and a recognition result correction means 265 correcting the recognition result based on communication history information stored in the communication history storage means. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a mobile communication terminal such as a mobile phone that can communicate with the outside, a personal computer with a communication function, a communication terminal such as a PDA, an information processing method, and a program.

2. Description of the Related Art Conventionally, a mobile phone capable of performing face recognition processing for identifying a person by recognizing a person's face included in the image based on image data captured by a camera is known (for example, Patent Document 1). ~ 3). The result of this face recognition can be used for authentication as to whether or not the user operating the mobile phone is an authorized user registered in advance.
JP 2006-093934 A JP 2006-319550 A JP 2007-047877 A

  However, when a person is identified by recognizing the face of the person included in the image as in the conventional mobile phone, the face depends on various factors such as the size, orientation, and background of the face in the image to be recognized. There is a problem that the accuracy of recognition is lowered.

  The present invention has been made in view of the above problems, and a purpose of the present invention is to provide a communication terminal capable of improving the accuracy of face recognition when identifying a person by recognizing the face of the person included in the image, and information processing It is to provide a method and program.

A communication terminal according to the present invention includes a communication unit that communicates with the outside, a communication history storage unit that stores communication history information in association with information about a person of a communication partner, and an image data storage unit that stores image data. Recognizing the person's face included in the image based on the image data and identifying the person, and the recognition result based on the communication history information stored in the communication history storage unit. Recognition result correcting means for correcting.
Here, the communication with the outside includes communication of a telephone for performing a telephone call by voice or a call by voice and an image with a communication partner and mail communication for exchanging mail with the communication partner.
In this communication terminal, there is a high possibility that the communication partner communicating with the communication means and the person included in the face recognition target image overlap. That is, the person included in the image that is recognized and processed by the face recognition means is highly likely to be the person of the communication partner of the communication history information in the communication history storage means. Accordingly, by correcting the recognition result of the face recognition unit based on the communication history information stored in the communication history storage unit, the person specified by the recognition result is included in the true image included in the face recognition target image. The probability of being a person can be increased. Therefore, the accuracy of face recognition when identifying a person by recognizing the face of the person included in the image can be improved.

The communication terminal further includes face recognition reference data storage means for storing a plurality of face recognition reference data used for face recognition for each of a plurality of persons, wherein the face recognition means includes the face of the person included in the image. The person included in the image is extracted by comparing the extracted face feature data with the plurality of face recognition reference data stored in the face recognition reference data storage means. May be specified.
In this communication terminal, the facial feature data that characterizes the face of the person included in the image is compared with a plurality of face recognition reference data stored in advance in the face recognition reference data storage means, thereby determining the person's face included in the image. The person can be identified by efficiently and accurately recognizing the face.

In the communication terminal, the image data storage unit may store the recognition result corrected by the recognition result correction unit and the image data in association with each other.
In this communication terminal, the person information, which is the recognition result corrected by the recognition result correcting means, and the corresponding image data are stored in association with each other, whereby the image data is classified by the person information. Can be saved and searched.

Further, in the communication terminal, the recognition result correcting means determines a closeness with the person of the communication partner based on the communication history information, and corrects the recognition result based on the closeness. May be.
In this communication terminal, since the intimacy between the user of the communication terminal and the person of the communication partner tends to change according to the content of communication with each communication partner included in the communication history information, the communication history Based on the information, it is possible to determine the familiarity with the person of the communication partner. And the higher the familiarity obtained by this determination, the higher the probability that the person of the communication partner is included in the image that is recognized by the face recognition means. Therefore, the accuracy of face recognition can be further improved by correcting the recognition result based on the familiarity obtained in the determination.

Further, in the communication terminal, the communication history information used by the recognition result correcting means is based on the communication frequency stored for each person of the communication partner and the date and time when communication with the communication partner is performed. It may be at least one piece of information on the elapsed time until the date and time and the type of communication with the communication partner.
In this communication terminal, the probability that a person of the communication partner is included in the image recognized and processed by the face recognition unit varies depending on at least one of the communication frequency, the elapsed time, and the type of communication with the communication partner. There are many. Therefore, the accuracy of face recognition can be further improved by correcting the recognition result based on at least one information of the communication frequency, the elapsed time, and the communication type.
Here, the recognition result correcting means determines that the probability that the person is included in the image is higher as the communication frequency with the person of the communication partner is higher, and corrects the recognition result. May be. As the frequency of communication with the person at the other end of the communication increases, the intimacy between the person and the user of the communication terminal often increases, so it is determined that the probability that the person is included in the image is high By correcting the result, the accuracy of face recognition can be further increased.
Further, the recognition result correcting means determines that the probability that the person is included in the image is higher as the elapsed time for communication with the person of the communication partner is shorter, and corrects the recognition result. It may be. The shorter the elapsed time for communication with the person at the other end of communication, the greater the intimacy between the person and the user of the communication terminal, so the probability that the person is included in the image is high. Then, the accuracy of face recognition can be further improved by correcting the recognition result.
In addition, the recognition result correcting means is configured to send a call to the person, receive a call from the person, send a mail to the person, The recognition result may be corrected by determining that there is a high probability that the person is included in the image in the order of reception of mail. The intimacy between the person and the user of the communication terminal increases in the order of making a call to the person at the other end of the communication, receiving a call from the person, sending mail to the person, and receiving mail from the person. There are many cases. For this reason, it is possible to further improve the accuracy of face recognition by determining that the probability that the person is included in the image is high and correcting the recognition result.

Further, in the communication terminal, the recognition result presents a plurality of candidates for a person included in the image, and the recognition result correcting means is configured to determine the recognition result based on the communication history information. The candidate to be included may be corrected.
In this communication terminal, by presenting a plurality of candidates as recognition results for the person included in the image, the person who is included in the image is finally identified while improving the accuracy of the recognition result as a whole. It is possible for the user of the communication terminal to judge and select when doing so.
Here, a display unit that displays a plurality of candidates of the recognition result, and one of the plurality of candidates displayed on the display unit is selected as the specific person included in the image Selecting means, and the image data storage means may store the information of the person selected by the selection means and the image data in association with each other. In this case, the selection work by the user who selects any one of the plurality of candidates as a specific person is facilitated, and the person stored in association with the image data in the image data storage means is the image. The probability of being a person included in can be increased.

The communication terminal may further include an imaging unit that captures an image, and the face recognition target image may be an image captured by the imaging unit.
In this communication terminal, the person who captures an image with the imaging means is often a communication partner communicating with the communication terminal, and is often intimate with the user of the communication terminal. Therefore, there is a higher possibility that the person of the communication partner by the communication terminal and the person included in the image that is captured by the imaging unit and is the target of face recognition will overlap. Therefore, it is possible to further improve the accuracy of face recognition when identifying a person by recognizing the face of the person included in the image.

An information processing method according to the present invention is an information processing method in a communication terminal that stores communication history information in association with information of a person of a communication partner, and includes a person's face included in the image based on image data And identifying the person, and correcting the recognition result based on the communication history information.
In the information processing method, face feature data characterizing a person's face included in the image is extracted, and the extracted face feature data and a plurality of face recognition reference data used for face recognition for each of a plurality of persons. A person included in the image may be specified by comparison.
The information processing method may further include a step of storing the previously corrected recognition result and the image data in association with each other.
Further, in the information processing method, closeness with the person of the communication partner may be determined based on the communication history information, and the recognition result may be corrected based on the closeness.
In the information processing method, the communication history information includes a communication frequency stored for each person of the communication partner, an elapsed time from a date and time when communication with the communication partner is performed, to a current date and time, And at least one piece of information of the type of communication with the communication partner.
In the information processing method, the recognition result may be corrected by determining that the probability that the person is included in the image is higher as the communication frequency with the person of the communication partner is higher.
In the information processing method, the recognition result may be corrected by determining that the probability that the person is included in the image is higher as the elapsed time for communication with the person of the communication partner is shorter. Good.
Further, in the information processing method, regarding the type of communication with the person of the communication partner, outgoing calls to the person, incoming calls from the person, transmission of mail to the person, mail from the person The recognition result may be corrected by determining that the probability that the person is included in the image is high in the order of reception.
Further, in the information processing method, the recognition result presents a plurality of candidates for a person included in the image, and the candidates included in the recognition result are determined based on the communication history information. It may be corrected.
Further, in the information processing method, a step of displaying a plurality of candidates of the recognition result, and a step of storing information on a person selected from the plurality of candidates and the image data in association with each other. Furthermore, you may have.
In the information processing method, the face recognition target image may be an image captured by an imaging unit.

A program according to the present invention is a program for executing information processing in a communication terminal that stores communication history information in association with information on a person of a communication partner using a computer of the communication terminal. Recognizing the face of a person included in the image based on the data to identify the person, correcting the recognition result based on the communication history information, and using the computer.
In the program, face feature data characterizing a person's face included in the image is extracted, and the extracted face feature data is compared with a plurality of face recognition reference data used for face recognition for each of a plurality of persons. Thus, information processing for specifying a person included in the image may be executed using the computer.
Further, in the program, information processing for storing a previously corrected recognition result and the image data in association with each other may be executed using the computer.
Further, in the program, a closeness with the person of the communication partner is determined based on the communication history information, and information processing for correcting the recognition result based on the closeness is executed using the computer. It may be a thing.
In the program, the communication history information includes a communication frequency stored for each person of the communication partner, an elapsed time from the date and time when communication with the communication partner is performed, and the current date and time, and The information may be at least one type of communication with the communication partner.
Further, in the program, the information processing for correcting the recognition result by determining that the probability that the person is included in the image is higher as the communication frequency with the person of the communication partner is higher. May be used.
Further, in the program, information processing for correcting the recognition result by determining that the probability that the person is included in the image is higher as the elapsed time for communication with the person of the communication partner is shorter, It may be executed using the computer.
Further, in the program, regarding the type of communication with the person of the communication partner, making a call to the person, receiving a call from the person, sending mail to the person, receiving mail from the person In this order, information processing for determining the probability that the person is included in the image is high and correcting the recognition result may be executed using the computer.
Further, in the program, the recognition result presents a plurality of candidates for the person included in the image, and corrects the candidate included in the recognition result based on the communication history information. Information processing may be executed using the computer.
Further, in the program, the step of displaying a plurality of candidates of the recognition result, and the information processing for storing the information of the person selected from the plurality of candidates and the image data in association with each other, It may be executed by using.
In the program, the face recognition target image may be an image captured by an imaging unit.

  According to the present invention, by correcting the recognition result of the face recognition unit based on the communication history information, the probability that the person specified by the recognition result is a true person included in the face recognition target image is calculated. Therefore, it is possible to improve the accuracy of face recognition when the face of a person included in an image is recognized and the person is specified.

Hereinafter, an embodiment in which the present invention is applied to a data sharing system using a mobile phone as a communication terminal will be described.
FIG. 1 is a schematic configuration diagram showing an example of a data sharing system using a mobile phone according to an embodiment of the present invention. This data sharing system shares image data of each user in a virtual community to which a plurality of users constructed via a communication network including the mobile phone communication network 10 belongs. It is configured using mobile phones 20 and 21 of users belonging to the community and a server 30 provided on the mobile phone communication network 10 so as to be able to communicate with the mobile phones 20 and 21. Each of the mobile phones 20 and 21 can communicate with a telephone such as a mobile phone via a wireless communication line established between the base stations 101 of the mobile phone communication network 10. it can. Each of the mobile phones 20 and 21 can perform communication such as mail with another personal computer or a communication terminal such as a PDA via an external communication network such as the Internet. The server 30 may be provided on another communication network such as the Internet 11 connected to the mobile phone communication network 10 via a gateway.

  The virtual community is composed of a plurality of users (members) having a specific relationship such as a relationship having the same hobby or a family / relative relationship, as in the case of the community in the real space. Corresponds to the group. In this virtual community, for example, an administrator who manages and operates the community transmits necessary registration information such as the name of the community, his / her e-mail address, telephone number, and name to the server 30 for predetermined authentication. By receiving, it can be constructed on a communication network. The administrator is set with an administrator ID and password for managing the community on the communication network. A user (member) to whom the community belongs is registered by transmitting necessary registration information such as his / her mail address, telephone number, and name to the server 30 and receiving predetermined authentication. This user registration may be performed when a new community is constructed, or may be performed at an arbitrary timing thereafter.

  FIG. 2 is a block diagram showing functional components of the mobile phone 20 constituting the data sharing system for sharing image data on the virtual community. The mobile phone 21 can be configured in the same manner. The cellular phone 20 has an image analysis function (face recognition function) for analyzing image data to be transmitted that is transmitted to the server 30 and uploaded. The cellular phone 20 includes a telephone communication unit (communication unit) 200, a data storage unit (data storage unit) 210, and a data transmission unit (data transmission unit, data communication unit) for performing telephone communication with other cellular phones. Communication means) 220, reservation setting section (reservation setting means) 230, image related information input section (image related information input means) 240, image related information addition section (image related information addition means) 250, and image analysis section. (Face authentication unit) 260, recognition result correction unit (recognition result correction unit) 265, data reception unit (data reception unit, communication unit) 270, imaging unit (imaging unit) 280, display unit (display unit) 285 And a selection information input unit (selection means) 290. The functions of these units are such that a program built in the mobile phone 20 or a program downloaded by the user is read into a computer including a CPU, and the program and various hardware resources of the mobile phone 20 cooperate. Is realized. This program can be transferred in a state where a recording medium such as a semiconductor memory such as a ROM, an optical disk (CD-ROM, CD-R, etc.), a magnetic disk (FD, HD, etc.), a magnetic tape, etc. is recorded. It can also be transferred by communication via a network. The mobile phone 20 includes a CPU, a RAM, a ROM, a baseband processing unit, a wireless communication module, a display using a liquid crystal panel, a key operation unit, a microphone, a speaker, a camera, a memory card, and the like as hardware resources.

  The wireless communication module includes, for example, a synthesizer, a frequency converter, a high frequency amplifier, and the like, and executes high frequency signal processing for wireless communication with the base station 101 of the mobile phone communication network 10. The baseband processing unit executes digital processing for performing telephone and data transmission / reception communication with an external mobile phone or server. The baseband processing unit and the wireless communication module are connected via a D / A converter or an A / D converter. An analog audio signal output from the microphone is converted into a digital signal by a sound signal processing unit (not shown) and sent to a baseband processing unit or the like. The speaker receives an analog signal converted from a digital signal by the sound signal processing unit, and outputs a voice during a call, or outputs a ringtone for a mail, a ringing tone for a telephone, music, or the like. The speaker may be configured by separately providing a receiver speaker (receiver) for listening to voice during a call and an external output speaker for outputting ringtones, music, etc. You may comprise with one speaker so that a speaker and an external output speaker may be combined. The key operation unit includes a plurality of keys such as a data input key (ten key, * key, # key), a call start key, an end key, a scroll key, and a multi-function key. The key operation unit 115 is used when the user instructs to start or end a call, select a menu, switch a screen, etc., or input information.

  The telephone communication unit 200 is configured using a wireless communication module, a baseband processing unit, and the like, and when a call is received from another mobile phone or the like, a call is made to another mobile phone or the like. In addition, a telephone (call) communication process is executed on a call line established with another mobile phone via the mobile phone communication network 10.

  The data storage unit 210 is configured using a RAM, a ROM, a memory card, or the like, and stores image data acquired by the user by being acquired by the imaging unit 280 or the like or downloaded from a server. In addition, the data storage unit 210 includes a communication history storage unit that stores communication history information in association with information on a person of a communication partner of a telephone or e-mail, and a plurality of face recognition standards used for face recognition for each of a plurality of persons. It also functions as a face recognition reference data storage means for storing data. The communication history is stored, for example, by associating the name of the communication partner (person) or the identification information of the communication for incoming / outgoing calls or mail transmission / reception with the date / time data of the communication. Further, the communication history information may be stored while leaving the latest predetermined number (for example, 100) of communication histories so that the data storage area does not become empty.

  FIG. 3 is an explanatory diagram showing an example of image data and its additional information stored in the data storage unit 210 as the image data storage means. In the illustrated example, image data (main body), basic information related to the image data, and image related information (acquisition related information, transmission control related information, tag information, and image analysis result) are stored in association with each other. As basic information, a file name, a file type (JPEG, PNG, etc.) and a file size are stored. As the acquisition related information, for example, position information of the mobile phone when the corresponding image data is acquired and date / time information of acquisition (photographing) are stored. As the transmission control related information, an upload flag (transmission designation information) for designating upload (transmission) of the corresponding image data to the server 30 and a deletion flag (designation for whether to delete after transmission) ( Deletion presence / absence designation information) is stored. As the tag information, tag information input by the image related information input unit 240 is stored. Further, as the image analysis result, the recognition result obtained by the face recognition process in the image analysis unit 260 obtained in (for example, person identification information such as an ID number uniquely assigned to each person) or the recognition thereof A corrected recognition result (person identification information) obtained by correcting the result based on the communication history information is stored.

  The data transmission unit 220 is configured using a wireless communication module or the like, and transmits mail data to a mail server on the communication network via the mobile phone communication network 10 or a content server such as a WEB server on the communication network. Or send an acquisition request to. In addition, the data transmission unit 220 transmits the image data to be transmitted whose upload flag is “1” to the server 30 at a preset transmission reservation time.

  The transmission reservation time is preferably set in a traffic quiet time period in which communication in the cellular phone communication network (communication network) 10 is quiet. For example, if a midnight band with few calls and data communication by a mobile phone is set as a traffic off time, traffic during the daytime of the mobile phone communication network (communication network) 10 can be reduced. Also, if you set the midnight hours when you don't use your mobile phone as a busy hour, you can use other services and applications on your mobile phone by uploading that data even when you upload image data with a large data size. The situation that it becomes impossible to occur does not occur. Furthermore, when a cheap communication charge (hour charge, data amount charge) is set for the traffic off-hours such as the midnight, the communication charge for uploading the image data can be reduced.

  The reservation setting unit 230 includes a key operation unit and the like, and a transmission reservation time for transmitting image data to the server 30 is set by a user operation. The reservation setting unit 230 may be configured to automatically set the transmission reservation time based on time designation information received from the server 30 using a wireless communication module or the like.

The image-related information input unit 240 includes a key operation unit and the like. Tag information related to the image data stored in the data storage unit 210, for example, image data, is operated as required by the user. Tag information (image related information) for identifying the contents of the image is input.
The image related information adding unit 250 includes a CPU or the like, and adds tag information to image data to be transmitted when tag information is input by the image related information input unit 240.

  When inputting the tag information, the ranking data obtained by associating a plurality of tag information used in the past with the usage frequency is received from the server 30 and the ranking data acquired from the server 30 is displayed. It may be. The user can select from the displayed ranking data and input desired tag information. The ranking data receiving unit (ranking data receiving unit) is configured using the wireless communication module or the like, and the ranking data display unit (ranking data display unit) is configured using a display such as a liquid crystal panel.

  The image analysis unit 260 includes a CPU, a recognition engine, and the like, reads and analyzes the image data stored in the data storage unit 210, specifies the type of the image, and outputs it as an analysis result. In particular, the image analysis unit 260 functions as a face recognition unit that identifies a person by recognizing the face of the person included in the image based on the image data. For example, when the image includes a face image of a person, the face image is analyzed to identify the person, and identification information (person identification information) of the person is output as a recognition result (analysis result). Various known image recognition techniques can be employed as the image analysis method for face recognition. For example, a reference image database for image recognition in which face recognition reference data serving as a reference for a plurality of persons who may be recognized is stored in association with corresponding person information is created and registered in advance. A person or the like may be specified by performing image analysis of the face recognition using a registered reference image database. In the reference image database, for example, face recognition reference data for a plurality of pre-registered persons (for example, about 100 to 200 persons) who are acquaintances of mobile phone users are persons whose names and identification information are used. Associated with information. The reference image database may be linked to a data table of a telephone book (address book) using a person's name or identification information as a key. Further, in the reference image database, the user of the mobile phone may be able to appropriately register the face recognition reference data and the person information based on the person image data acquired by photographing with the imaging unit 280 or the like.

  The image analysis unit 260 identifies a predetermined number of candidates set in advance for the persons included in the image as the recognition result, and determines the accuracy of face recognition for each of the candidates. You may make it output the recognition result which shows the index value (score) to represent with a candidate's image, a name, etc. When a plurality of persons are recognized as subjects for one image data, identification information of the plurality of persons is output as a recognition result (analysis result). The person identification information obtained as a recognition result by the image analysis unit 260 may be added to the image data to be transmitted separately from the manually input tag information, as shown in FIG. It may be added as a part.

  The recognition result correction unit 265 is configured by a CPU or the like, and is obtained by the image analysis unit 260 based on communication history information of telephone call / call and mail transmission / reception stored in the data storage unit 210. Correct the recognition result. Here, for the correction of the recognition result, only the communication history information of a predetermined number of people (for example, the top 20 people) is set in advance from the top when the communication partners are arranged in the order of the largest number of communications. Good. A specific example of processing in the recognition result correction unit 265 will be described later.

  The data receiving unit 270 is configured using the wireless communication module or the like, and receives mail data from a mail server on the communication network via the mobile phone communication network 10 or content such as a WEB server on the communication network. Data of desired content is transmitted from the server. Further, the data receiving unit 270 receives additional image data transmitted from the server 30 when image data is added to the server 30 for the virtual community to which the user of the mobile phone 20 belongs. Note that the additional image data may be controlled to be received when a preset reception reservation time comes.

  Further, the reception reservation time is preferably set in a traffic quiet time period in which communication in the cellular phone communication network (communication network) 10 is quiet, as in the case of the transmission reservation time described above. For example, if a midnight band with few calls and data communication by a mobile phone is set as a traffic off time, traffic during the daytime of the mobile phone communication network (communication network) 10 can be reduced. In addition, if you set the midnight hours when your mobile phone is not used as a busy hour, you can use other services and applications on your mobile phone by downloading the data even if you download large image data. The situation that it becomes impossible to occur does not occur. In addition, since the data download is completed in the middle of the night, the image can be quickly displayed based on the downloaded image data, and the operability for the user is improved. Furthermore, when a cheap communication charge (time charge, data amount charge) is set for the traffic off-hours such as the midnight, the communication charge for downloading the image data can be reduced.

  The imaging unit 280 is configured by an imaging device (camera) such as a CCD or a CMOS, and a user performs a predetermined photographing operation to photograph a surrounding person or landscape and save it in the data storage unit 210. Can do.

  The display unit 285 includes an LCD (Liquid Crystal Display), an organic EL (Organic Electroluminescence), an LED (Light-Emitting Diode) array, and the like. The display unit 285 displays an image acquired by the imaging unit 280 or the like, or a mail to be transmitted / received. Various kinds of information can be displayed. Further, the display unit 285 can display the recognition result obtained by the image analysis unit 260 and the recognition result corrected by the recognition result correction unit 265.

  The selection information input unit 290 includes a key operation unit and the like. When the recognition result corrected by the recognition result correction unit 265 presents a plurality of candidates, a plurality of selection information input units 290 are displayed on the display unit 285. This is used when the user selects one of the candidates as a specific person included in the image. The information (person identification information) of the person selected by the selection information input unit 290 is stored in the data storage unit 210 in association with the corresponding image data (see FIG. 3).

Next, a specific example of the recognition process correction process in the recognition result correction unit 265 will be described. In this example, an “overall score S” expressed by the following equation (1) is defined as an index value that represents the accuracy of human face recognition after correction processing in numerical form.
Overall score S = Wf × Sf + Wc × Sc (1)

Here, the meaning of each variable in the above equation (1) is as follows. Note that the set values of the weight values Wf and Wc in the equation (1) may be changed by the user operating the key operation unit on the setting screen displayed on the display unit 285.
Sf: score (face recognition score) of determination result by face recognition technology,
Wf: weight value for the determination result by the face recognition technology,
Sc: A score (communication history score) obtained by quantifying the communication history of calls / mails,
Wc: Weighting value for call / mail communication history.

The communication history score Sc is an index value representing the intimacy between the user of the mobile phone 20 and the person of the communication partner, and can be calculated by, for example, the following calculation method. First, the history of communication with a specific person (call origination, call arrival, mail transmission, mail reception) is represented by {(k1, t1), (k2, t2), ..., (kn, tn). }. Here, the meaning of each variable is as follows.
ki: Type of communication (outgoing call, incoming call, sending mail, receiving mail),
ti: Elapsed time = current date / time-date / time of communication.

If the communication history variable is used, the communication history score Sc can be calculated, for example, by the following equation (2).
Sc = SUM {i = 1 to n} · {Wk (ki) × Wt (ti)} (2)

Here, the variables Wk (ki) and Wt (ti) in the above equation (2) are respectively weighted values related to the type of communication and the elapsed time, and are set as follows, for example.
Wk (k) = 1.0 (k = outgoing call),
0.8 (k = incoming call),
0.6 (k = send email),
0.4 (k = reception of mail).
Wt (t) = 1.0 (t <1 day),
0.5 (1 day ≤ t <3 days),
0.25 (3 days ≦ t <7 days),
0.1 (7 days ≦ t <30 days).

  The weighted values Wk (ki) and Wt (ti) correspond to the intimacy between the user of the mobile phone and the communication partner, and the intimacy is numerically expressed. In the case of the above setting example, the order of priority in which the weighting value for the type of communication is large is: outgoing call> incoming call> email transmission> mail reception. It is getting bigger. The set values of these weighted values Wk (ki) and Wt (ti) may be changed by the user operating the key operation unit on the setting screen displayed on the display unit 285.

Table 1 is a specific example of the score Sf of the determination result by the face recognition technology for the two persons A and B included in the image and the communication history between the persons A and B. Table 2 is for the persons A and B. It is a specific example of calculation of the total score S of face recognition. In Table 2, the weighting value Wf in the equation (1) for calculating the total score S and the weighting value Wc for the call / mail communication history are set as follows.
Wf = 1.0,
Wc = 5.0

顔認識スコアＳｆ
通信履歴
人物Ａ
８０点
無し
人物Ｂ
７５点
８時間前：電話の発信１件
２日前： 電話の発信１件、
電話の着信１件
５日前： メールの送信１件

Face recognition score Sf
Communication history person A
Person B without 80 points
75 points 8 hours ago: 1 phone call 2 days ago: 1 phone call
1 incoming call 5 days ago: 1 outgoing email

顔認識スコア
Ｓｆ
通信履歴スコア
Ｓｃ
総合スコア
Ｓ
人物Ａ
８０点
０点
１．０×８０＋
５．０×０＝
８０点
人物Ｂ
７５点
Ｗｋ（電話の発信）×Ｗｔ（８時間）＋
Ｗｋ（電話の発信）×Ｗｔ（２日）＋
Ｗｋ（電話の着信）×Ｗｔ（２日）＋
Ｗｋ（メールの送信）×Ｗｔ（５日）＝
１．０×１．０＋
１．０×０．５＋
０．８×０．５＋
０．６×０．２５＝
２．０５点
１．０×７５＋
５．０×２．０５＝
８５．２５点

Face recognition score Sf
Communication history score Sc
Overall score S
Person A
80 points 0 points 1.0 x 80+
5.0 × 0 =
80 person B
75 points Wk (call outgoing) x Wt (8 hours) +
Wk (outgoing phone call) x Wt (2 days) +
Wk (incoming call) x Wt (2 days) +
Wk (send mail) x Wt (5 days) =
1.0 × 1.0 +
1.0 × 0.5 +
0.8 x 0.5+
0.6 × 0.25 =
2.05 points 1.0 x 75+
5.0 × 2.05 =
85.25 points

  FIG. 4 is a flowchart illustrating an example of processing from capturing an image including a person to storing image data. First, the user of the mobile phone 20 operates the key operation unit, takes a picture with the imaging unit 280, and temporarily saves the data of the taken image in the data storage unit 210 (S1, S2). Next, the image analysis unit 260 executes face recognition image analysis processing on the captured image, and information on a predetermined number of candidates determined to have a high probability of being a person included in the image (for example, , Name and person identification information) are stored in the data storage unit 210 as a recognition result (S3, S4). For example, as shown in the frame of FIG. 5, information on the top nine candidates with high face recognition scores Sf is stored in the data storage unit 210 as recognition results. Note that the three persons outside the frame in FIG. 5 are persons who are out of the nine candidates, and are the persons with the highest face recognition score Sf next to the nine candidates. The persons A and B in the figure correspond to the persons A and B shown in the above displays 1 and 2. Among the nine candidates, the face recognition score Sf of the person A is the worst. Here, when face recognition processing is actually performed on an image including a person's face, as shown in FIG. 5, one of the top nine candidates having a high face recognition score Sf is included in the image. The probability of being 92%. On the other hand, when focusing on one person with the highest face recognition score Sf, it was 30%.

  Next, the recognition result is corrected based on the communication history information of the mobile phone 20 (S5). For example, in the case of the persons A and B shown in the displays 1 and 2, as a result of correction based on the communication history information, the value of the total score S of the person B (85.25 points) is the value of the total score of the person A (80 As shown in FIG. 6, the recognition result is corrected so that the person A is removed from the nine candidates and the person B is put in the nine candidates instead. The By correcting this recognition result, the probability that any of the nine candidates shown in FIG. 6 is a person included in the image has increased to 94%. Then, the nine persons including the person B are displayed on the display unit 285 as a final candidate list (S6).

  Next, when the user who has viewed the display on the display unit 285 operates the key operation unit so as to select a person actually included in the image from nine candidates, the person of the selected person The identification information is automatically stored in association with the corresponding image data in the data storage unit 210 (S7, S8).

  As described above, according to the present embodiment, the face recognition process (image analysis process) is performed on the image acquired by the imaging by the imaging unit 280, and the communication history information stored in the data storage unit 210 is used. By correcting the recognition result of the face recognition processing, it is possible to increase the probability that the person specified by the recognition result is a true person included in the face recognition target image. The accuracy of face recognition when identifying a person by recognizing a face can be improved.

  Further, according to the present embodiment, face feature data characterizing a human face is extracted from the image data, and the face feature data and a plurality of face recognition reference data stored in advance in the reference image database of the data storage unit 210 are extracted. , The person's face included in the image can be efficiently and accurately recognized to identify the person.

  Further, according to the present embodiment, the person identification information, which is the recognition result corrected by the recognition result correcting unit 265, and the corresponding image data are stored in association with each other, whereby the image data is stored as a person. You can sort and save or search by information.

  Further, according to this embodiment, the familiarity with the person of the communication partner, the communication frequency with the communication partner, the elapsed time from the date and time when the communication with the communication partner was performed to the current date and time, and the communication By using the type of communication with the other party and correcting the face recognition score Sf of the recognition result based on the communication history score Sc corresponding to the familiarity obtained by the determination, the accuracy of face recognition Can be further increased.

  In addition, according to the present embodiment, by presenting a plurality of candidates as recognition results for the person included in the image, the accuracy of the recognition result as a whole can be improved, and the person included in the image can be selected. The user of the mobile phone can judge and select when finally specifying. In particular, since the user of the mobile phone can operate the key operation unit while confirming with the display unit 285, the user can easily select one of a plurality of candidates as a specific person. In addition, the accuracy of the person identification information stored in association with the image data in the data storage unit 210 can be improved.

  Although the preferred embodiments of the present invention have been described above, various modifications can be made to the disclosed embodiments within the scope of the technical matters described in the claims.

  For example, in the above embodiment, the image data may be still image data or moving image data. When uploading image data of a moving image to a server and storing it in a shareable manner, for example, image analysis (face recognition) is performed on a specific image frame of the moving image. The image frame for image analysis may be specified by the user, or the image analysis unit may scan a plurality of image frames and automatically select an image frame suitable for image analysis.

  Moreover, although the case where the communication terminal is a mobile phone has been described in the above embodiment, the present invention is not limited to other communication such as a PHS, a car phone, a personal digital assistant (PDA) having a communication function, a digital camera having a communication function, etc. It can be applied to the case of a terminal, and the same effect can be obtained.

1 is a schematic configuration diagram showing an example of a data sharing system using a mobile phone according to an embodiment of the present invention. The block diagram which shows the functional component of the mobile telephone which comprises a data sharing system. Explanatory drawing which shows an example of the image data memorize | stored in the data storage part of a mobile telephone, and its additional information. The flowchart which shows an example of a process after image | photographing the image containing a person until it preserve | saves image data. Explanatory drawing of the recognition result of face recognition. Explanatory drawing of the recognition result corrected based on communication history information.

Explanation of symbols

10 mobile phone communication network 20, 21 mobile phone 30 server

Claims (13)

A communication terminal comprising a communication means for communicating with the outside, a communication history storage means for storing communication history information in association with information on a person of a communication partner, and an image data storage means for storing image data. And
Face recognition means for recognizing a person's face included in the image based on the image data and identifying the person;
Recognition result correction means for correcting the recognition result based on communication history information stored in the communication history storage means;
A communication terminal comprising: The communication terminal according to claim 1, wherein
Face recognition reference data storage means for storing a plurality of face recognition reference data used for face recognition for each of a plurality of persons;
The face recognition means extracts face feature data characterizing a human face included in the image, and the extracted face feature data and the plurality of face recognition standards stored in the face recognition reference data storage means A communication terminal characterized by identifying a person included in the image by comparing with data. In the communication terminal according to claim 1 or 2,
The image data storage means stores the recognition result corrected by the recognition result correction means and the image data in association with each other. In the communication terminal according to any one of claims 1 to 3,
The communication terminal characterized in that the recognition result correcting means determines a closeness with the person of the communication partner based on the communication history information and corrects the recognition result based on the closeness. In the communication terminal in any one of Claims 1 thru | or 4,
The communication history information used by the recognition result correcting means includes a communication frequency stored for each person of the communication partner, an elapsed time from the date and time when communication with the communication partner is performed, and the current date and time, and A communication terminal comprising at least one piece of information of a type of communication with the communication partner. The communication terminal according to claim 5, wherein
The communication result correcting means determines that the probability that the person is included in the image is higher as the communication frequency with the person of the communication partner is higher, and corrects the recognition result. Terminal. The communication terminal according to claim 5, wherein
The recognition result correcting means determines that the probability that the person is included in the image is higher as the elapsed time for communication with the person of the communication partner is shorter, and corrects the recognition result. Communication terminal. The communication terminal according to claim 5, wherein
The recognition result correcting means is configured to send a call to the person, receive a call from the person, send a mail to the person, send a mail to the person, A communication terminal that corrects the recognition result by determining that the probability that the person is included in the image is high in the order of reception. In the communication terminal according to any one of claims 1 to 8,
The recognition result presents a plurality of candidates for a person included in the image,
The said recognition result correction means corrects the said candidate included in the said recognition result based on the said communication history information, The communication terminal characterized by the above-mentioned. The communication terminal according to claim 9, wherein
Display means for displaying a plurality of candidates for the recognition result displayed on the display means;
Selecting means for selecting any one of the plurality of candidates as a specific person included in the image; and
The communication terminal characterized in that the image data storage means stores information on the person selected by the selection means and the image data in association with each other. In the communication terminal in any one of Claims 1 thru | or 10,
It further comprises imaging means for taking an image,
The communication terminal, wherein the face recognition target image is an image taken by the imaging means. An information processing method in a communication terminal that stores communication history information in association with information on a person of a communication partner,
Recognizing a person's face included in the image based on image data and identifying the person;
Correcting the recognition result based on the communication history information. A program for executing information processing in a communication terminal that stores communication history information in association with information on a person of a communication partner using a computer of the communication terminal,
Recognizing a person's face included in the image based on image data and identifying the person;
Correcting the recognition result based on the communication history information,
A program that is executed using the computer.

Images