WO2007069361A1

WO2007069361A1 - Information processing terminal

Info

Publication number: WO2007069361A1
Application number: PCT/JP2006/314521
Authority: WO
Inventors: Ryouta Yoshida; Takaaki Nishi; Tetsurou Sugimoto; Tomoko Obama; Hideaki Matsuo; Junichi Oshiro
Original assignee: Matsushita Electric Industrial Co., Ltd.
Priority date: 2005-12-16
Filing date: 2006-07-21
Publication date: 2007-06-21

Abstract

An information processing terminal that can assist the user to remember his/her past conversation contents or message body contents without forcing the user to spend time and effort. The information processing terminal comprises a conversation device (10) for entering feeling identification information composed of at least a voice, a feeling identification device (20) for identifying a feeling on the basis of the feeling identification information entered into the conversation device (10) and a display section (50) for displaying information on the feeling identified by the feeling identification device (20).

Description

Specification

Information processing terminal

Technical field

The present invention relates to an information processing terminal capable of specifying the emotion of the creator of the email and the emotion of the speaker during the call from the character data described in the email and the voice during the call. About.

Background art

[0002] Recent mobile phones, PDAs (Personal Digital Assistance), and other mobile devices have a history of incoming calls (at least when the call arrives and if the caller's phone number and caller can be identified, the caller's name and , The items that are also configured in chronological order according to the incoming time) and outgoing call history (at least when the phone call time and the destination phone number and destination can be specified) And call history such as items arranged in chronological order according to the call origination time) and mail reception history (at least the time when the mail was received, the sender's email address and sender can be specified) In this case, the name of the sender, the email body, and the items that are configured in a time-sequential manner according to the reception time are sent in the email transmission history (at least the email transmission time and the email address of the recipient. If you can identify the address and destination, you can display the email history such as the destination name and the body of the email, arranged in chronological order according to the sending time) (For example, Patent Document 1).

Patent Document 1: Japanese Patent Laid-Open No. 11-275209

Disclosure of the invention

Problems to be solved by the invention

[0003] However, it is difficult for a mobile terminal user to recall past call contents and mail text contents simply by looking at the call history and mail history displayed on a conventional mobile terminal. Of course, it is difficult to remember even what feelings you were communicating with the call or email.

[0004] In order to be able to recall the contents of past calls, some conventional mobile terminals have a call sound. Some voices are saved, but it takes a lot of time to play back the saved call voice, or if you do not play the call voice until the end, you can remember the contents of the call. It takes a long time to remember the contents of past calls. In addition, to remember the contents of the mail text, the mail history is appended to each item that makes up the mail history, so select one item that you want to remember the mail text content and select the mail text. Is displayed, but the mobile terminal user is forced to perform a selection operation, and the mobile terminal user must reread the mail text. I will let you four.

[0005] The present invention has been made in view of the above circumstances, and can support recalling the contents of past calls and the contents of e-mail text without forcing the user to spend time and effort. An object is to provide a processing terminal.

Means for solving the problem

[0006] An information processing terminal according to the present invention includes an input means for inputting emotion specifying information composed of at least character data or voice, and an emotion specifying an emotion based on the emotion specifying information input by the input means. And a display unit for displaying information related to the emotion specified by the emotion specifying unit.

[0007] According to this configuration, it is possible to support the terminal user to remember the contents of communication using the past information processing terminal.

[0008] Further, the information processing terminal of the present invention includes mail transmission / reception means for transmitting or receiving an electronic mail, and the input means is character data described in the electronic mail transmitted or received by the mail transmission / reception means. The emotion specifying means specifies an emotion based on the character data input by the input means, and the display means transmits or receives by the mail transmitting / receiving means specified by the emotion specifying means. Including information about emotions corresponding to selected e-mails.

[0009] According to this configuration, it is possible to support the terminal user to remember the contents of the past mail text.

[0010] Further, in the information processing terminal of the present invention, at least a transmission time or at least for each e-mail in the order in which the display unit transmits or receives e-mails by the mail transmission / reception unit. Includes those that display information about the reception time, destination or source, and the emotion.

[0011] Further, the information processing terminal of the present invention includes the information processing terminal in which the display unit displays, for each date, information related to the emotion corresponding to the email transmitted or received on the date by the mail transmission / reception unit. .

[0012] In addition, in the information processing terminal of the present invention, for each target person registered in the telephone directory function, the display means also receives an e-mail transmitted to the target person by the mail transmitting / receiving means or the target person power. Including information on the emotions corresponding to the received e-mail.

[0013] According to this configuration, in conjunction with various PIM applications that execute functions such as an email transmission / reception history function, a schedule management function, or a telephone function, the mobile phone user's emotions and the mobile phone are used to By notifying mobile phone users of the feelings of other people who have communicated with users, mobile phone users can more easily remember past communications and provide an environment.

[0014] Further, in the information processing terminal according to the present invention, any one of the information related to the emotion specified by the display unit for each of the plurality of e-mails transmitted or received by the mail transmitting / receiving unit. Including things that display.

[0015] According to this configuration, the terminal user can confirm at a glance the emotion specified for each of a plurality of emails.

[0016] Further, the information processing terminal according to the present invention includes a calling means for making a call, wherein the input means inputs a call voice by the calling means, and the emotion specifying means inputs by the input means. Emotions are specified based on the voice of the call, and the display means specifies information related to emotions about the called party, the caller, or both the called party and the caller who have made a call by the calling means specified by the emotion specifying means. Including things to display.

[0017] According to this configuration, it is possible to assist the terminal user in recalling past call contents.

[0018] Further, in the information processing terminal according to the present invention, at least for each call, the display means makes a call or an incoming call in the order in which the display means makes or receives a call. Including those that display information about followers and the emotions.

[0019] Further, the information processing terminal according to the present invention is such that the display means displays, for each date, information related to the emotion corresponding to a call made or received by the call means on the date. Including.

[0020] Further, in the information processing terminal of the present invention, the display means displays, for each target person registered in the telephone directory function, information on the emotion about the target person who has called by the calling means. Including things to do.

[0021] According to this configuration, in conjunction with various PIM applications that execute functions such as an email transmission / reception history function, a schedule management function, or a telephone function, the mobile phone user's emotions and the mobile phone can be used to By notifying mobile phone users of the feelings of other people who have communicated with users, mobile phone users can more easily remember past communications and provide an environment.

[0022] Further, in the information processing terminal of the present invention, the display means displays at least one of the information related to the emotion specified for each of a plurality of calls transmitted or received by the call means. Including things to do.

[0023] According to this configuration, the terminal user can confirm at a glance the emotions specified for each of a plurality of calls.

[0024] Further, an information processing terminal of the present invention includes a call storage unit that stores a call voice by the call unit, and a call playback unit that plays back the call voice stored in the call storage unit. The call reproduction means reproduces the call voice specifying the information related to the emotion displayed by the display means among the call voices stored in the call storage means.

[0025] According to this configuration, when the terminal user wants to remember the contents of a past call, in addition to displaying the emotion symbolizing the contents specified in advance, the call voice is recorded and the recording is performed. Play the data. As a result, the mobile phone user can surely remember the contents.

[0026] Further, in the information processing terminal according to the present invention, the call storage unit stores a portion in which the emotion specified by the emotion specifying unit is reflected in the call voice by the call unit. Then, the call reproduction means reproduces at least the part of the call stored in the call storage means.

[0027] According to this configuration, it is possible to effectively reproduce the data portion that can remind the terminal user of the contents of past conversations in the recorded data that has been recorded. As a result, mobile phone users can minimize the time and effort required to recall the contents of past calls.

[0028] Further, the information processing terminal according to the present invention includes the information processing terminal in which the call reproduction means reproduces the location with a predetermined time before the start time of the location as a reproduction start time.

[0029] According to this configuration, the sound that has been the main cause of identifying the emotion of the recorded voice by starting playback of the recorded voice for a predetermined time before the voice portion in which the emotion of the recorded voice is reflected is started. Never miss a voice.

The invention's effect

[0030] According to the information processing terminal of the present invention, it is possible to support recalling the contents of past calls and the contents of e-mail texts without forcing the user to spend time and effort. Brief Description of Drawings

FIG. 1 is a configuration diagram of a mobile phone according to a first embodiment of the present invention.

[Figure 2] Affection information factors that are composed of affection, joy, anger, sorrow, and neutrality, and the degree of emotion for each factor

FIG. 3 shows information stored in the emotion identification device in the mobile phone according to the first embodiment of the present invention.

FIG. 4 is a processing flow for displaying a call history by the mobile phone according to the first embodiment of the present invention.

FIG. 5 is a display example of a call history by the mobile phone according to the first embodiment of the present invention. Figure 5 (a) shows the outgoing call history. Figure 5 (b) shows the incoming call history. Fig. 5 (c) shows an example of a feeling mark displayed in the outgoing call history or incoming call history.

FIG. 6: Display example of a schedule book by the mobile phone according to the first embodiment of the present invention

FIG. 7 shows an example of a telephone directory display by the mobile phone according to the first embodiment of the present invention.

[FIG. 8] Sorting example for each emotion by the mobile phone according to the first embodiment of the present invention.

FIG. 9 is another configuration diagram of the mobile phone according to the first embodiment of the present invention. FIG. 10 shows another example of the information list stored in the emotion identification device in the mobile phone according to the first embodiment of the present invention.

[FIG. 11] Processing flow for displaying the mail reception history by the mobile phone of the first embodiment of the present invention. [FIG.

FIG. 13 shows information stored in the emotion identification device in the mobile phone according to the second embodiment of the present invention.

FIG. 14 is a processing flow for displaying the call history of the mobile phone according to the second embodiment of the present invention.

FIG. 15 is a display example of call history by the mobile phone according to the second embodiment of the present invention.

FIG. 16: Display example of schedule book by mobile phone according to the second embodiment of the present invention

FIG. 17 is a display example of a telephone directory by the mobile phone according to the first embodiment of the present invention.

FIG. 18 is a processing flow for recording voice of a call by the mobile phone according to the second embodiment of the present invention (Example 1).

FIG. 19 is a processing flow for recording a call voice by the mobile phone according to the second embodiment of the present invention (Example 2).

Explanation of symbols

10 Telephone equipment

101 Communication Department

102 Audio signal output section

103 Call speaker

104 Microphone for calls

105 Voice memory

20, 70 Emotion identification device

201, 701 Emotion estimation unit

202, 702 Emotion accumulation part

203, 703 Emotion identification part

204, 704 Emotion information storage

30 PIM applications 301 Call history application

302 Scheduler application

303 Phonebook application

304 Mail history application

40, 80 Display control unit

50 Display

60 Communication equipment

601 Mail sending / receiving part

602 Character information output section

90 Playback control section

100 speakers

BEST MODE FOR CARRYING OUT THE INVENTION

[0033] Hereinafter, a form of an information processing terminal that implements the present invention will be described in detail by taking a mobile phone as an example.

[0034] (First embodiment)

FIG. 1 shows a configuration diagram of the mobile phone according to the first embodiment of the present invention. The mobile phone according to the first embodiment of the present invention includes a call device 10, an emotion identification device 20, a PIM (Personal Information Manager) application group 30, a display control unit 40, and a display unit 50. It should be noted that a configuration having various functions included in a mobile communication terminal such as a recent mobile phone or PDA (Personal Digital assistant) may be used.

The call device 10 includes a communication unit 101, an audio signal output unit 102, a call speaker 103, and a call microphone 104. The communication unit 101 performs mobile wireless communication with a mobile phone base station, and realizes a voice call by transmitting and receiving a voice signal between the mobile phone user of the present invention and another telephone user. The audio signal output unit 102 receives an audio signal received from another telephone through the communication unit 101, an audio signal of the mobile phone user picked up by the call microphone 104, or both of the audio signals. Output to 20. Also, the audio signal output unit 102 outputs the voice received by other telephone power to the call speaker 103, and outputs the call voice of the mobile phone user collected by the call microphone 104 to the communication unit 101. To do.

The emotion identification device 20 includes an emotion estimation unit 201, an emotion accumulation unit 202, an emotion identification unit 203, and an emotion information storage unit 204. Emotion estimation unit 201 emits the voice from the volume of the voice, the waveform of the voice, the pitch of the voice, or the phoneme included as information in the received voice signal and the transmitted voice signal input from communication device 10 Estimate the emotions of other telephone users and those of the mobile phone users (such an emotion estimation method is proposed in, for example, International Publication No. WO00Z62279). For example, the emotion estimation unit 201 indicates the degree of emotion expressed by values of 0, 1, and 2 for each factor of emotion information composed of love, joy, anger, sadness, and neutral (normal) shown in FIG. (0: No emotion, 1: Emotional weakness, 2: Emotional strong), from the beginning of the call to the end of the call, for each received voice signal and transmitted voice signal, continuously estimated at predetermined time intervals, and each of the estimated Are sequentially output to the emotion storage unit 202. The emotion estimation unit 201 does not necessarily have to estimate both the emotion of the other telephone user and the emotion of the mobile phone user. The emotion of either the other telephone user or the mobile phone user is not necessarily estimated in advance. If it is set to estimate the voice, either the incoming voice signal or the outgoing voice signal may be input from the voice signal output unit 102 of the communication device 10.

[0037] Emotion accumulation section 202 accumulates the numerical value for each factor input from emotion estimation section 201 in correspondence with the input time or order for each received voice signal and transmitted voice signal. The emotion accumulation unit 202 receives a numerical value for each of a series of factors from the emotion estimation unit 201 (the series here refers to the input start force of the numerical value from the emotion estimation unit 201 to the emotion accumulation unit 202, which is the end of input. When the call start power in a call by the call device 10 is input, the value for each factor in the series is input as a set of data (hereinafter, a set of data is referred to as one call). (Referred to as minute data). If the emotion accumulation unit 202 accumulates the numerical value for each factor input from the emotion estimation unit 201 for each received voice signal and transmitted voice signal, the data for one call is received voice data. One for each signal and transmitted voice signal.

[0038] The emotion identification unit 203 reads the data for one call from the emotion accumulation unit 202, analyzes the numerical value for each factor, identifies one characteristic factor from the read numerical value, One of them The emotion represented by the factor is output to the emotion information storage unit 204. When identifying one characteristic emotion, the emotion identification unit 203 uses the factor with the most powerful numerical value in the data for one call as the characteristic emotion, so that the content that is strongly impressed during the call is displayed. It is possible to identify feelings with emphasis. Another method is to identify emotions by placing emphasis on the content of the entire call, with the characteristic that the most powerful factor is the sum of the call start power in the data for one call and the end of the call. By selecting the factor that has the most powerful value immediately before the end of the call in the data for one call as the characteristic emotion, the emotion can be identified with emphasis on the reverberation of the conversation. . In addition, when the emotion identifying unit 203 identifies one characteristic factor from each of the data for one call for each received voice signal and transmitted voice signal, the one factor is used for the received voice signal and the transmitted voice signal. Both forces, which are specified, are also output.

[0039] Emotion information storage unit 204 inputs an emotion represented by one factor from emotion identification unit 203, and further, information related to a call made by communication unit 101 of call device 10 (the call is transmitted) One of the incoming calls, call start time 'call end time, and other telephone identification information (for example, the telephone number of the called party) are input from the communication unit 101. The emotion information storage unit 204 stores the emotion input from the emotion specifying unit 203 and various information input from the communication unit 101 in association with each other (the emotion specifying unit 203 stores 1 for each received voice signal and transmitted voice signal). If one characteristic factor is identified from each of the data for the call, the force that is the signal source of either the received voice signal or the transmitted voice signal is memorized. FIG. 3 shows a list of information stored in the emotion identification device in the mobile phone according to the first embodiment of the present invention. In FIG. 3, the emotion information storage unit 204 stores the emotion identified from the received voice signal (i.e., the other party's emotion) and the emotion identified from the transmitted voice signal (i.e. The emotion of the mobile phone user of the invention is memorized. If it is set in advance to estimate the feeling of either one of the other phone users or the mobile phone user (data for one call c), Emotions are not memorized (in Fig. 3, it is shown that they are not memorized as “one”).

[0040] The PIM application group 30 manages personal information and sends the personal information to mobile phone users. It is composed of multiple application cards for using information. Examples of the PIM application include a call history application 301 for displaying a call history, a scheduler application 302 for supporting schedule management by a mobile phone user, or a phone book application 303 for registering various personal information. Can be mentioned.

[0041] The display control unit 40 activates and executes one of the applications in the PIM application group, extracts data necessary for processing of the executed application from the emotion information storage unit 2004, and displays the data in the display unit 50. Display various information. A display example of the display unit 50 when the call history application, scheduler application, or phone book application is started and executed will be described later.

Next, processing by the mobile phone of the first embodiment of the present invention will be described with reference to a processing flow for displaying a call history by the mobile phone of the first embodiment of the present invention shown in FIG.

[0043] In addition, the emotion identification device 20 accepts an input operation by an operation key (not shown) provided in the mobile phone of the present invention from a mobile phone user, and an emotion should be specified (1. The other party, 2. the mobile phone user himself, or both, are determined (step S401).

When the call device 10 starts a call (step S402, YES), the call device 10 outputs a reception voice signal and a transmission voice signal to the emotion identification device 20 (step S403). The emotion identifying device 20 is a factor of emotion information composed of affection, joy, anger, sadness, and neutral (normal) for the target speech signal of the speech signal input from the communication device 10. Each time, the degree of emotion is continuously estimated at predetermined time intervals (step S404). When the call by the call device 10 ends (YES in step S405), the emotion identification device 20 analyzes the numerical value for each factor estimated during the call, and the numerical force has one characteristic factor. The emotion expressed by the one factor is stored as the emotion of the other party or mobile phone user in this call (step S406. At this time, as shown in FIG. I remember with the feelings I made).

[0045] After that, the display control unit 40 starts the call history application 301 by accepting the input operation by the operation key provided in the mobile phone of the present invention by the mobile phone user power (step S407, YES). Program code for application 301 Accordingly, the emotion specified for each call stored in the emotion specifying device 20 and information related to the call are read out, and the information is displayed on the display unit 50 in a predetermined display format (step S408).

[0046] [Display format using call history]

FIG. 5 shows a display example of a call history by the mobile phone according to the first embodiment of the present invention. Fig. 5 (a) is a transmission history generated based on the information stored in the emotion identification device 20 shown in Fig. 3, and Fig. 5 (b) is based on the information stored in the emotion identification device 20 shown in Fig. 3. This is the incoming call history generated at. Fig. 5 (c) is an example of a mark indicating emotion displayed in the outgoing call history or incoming call history.

[0047] In order to display the outgoing call history shown in Fig. 5 (a), the display control unit 30 sets the item "outgoing / incoming" in the information list stored in the emotion identifying device 20 shown in Fig. 3 to "outgoing" ”(Corresponding to data a for one call), and the other party whose emotion set in step S401 should be specified, that is, the item“ signal source ”is“ received voice ”. The data of the items “call start time”, “call destination telephone number”, and “emotion” are extracted from the minute data, and the items of the call history in FIG. 5 (a), “date and time”, “name”, “emotion” "Is displayed at the corresponding location. Note that the name “Taro Matsushita” is displayed in the “name” field of the outgoing call history in FIG. 5 (a) instead of the telephone number “09000000000”! This is because the name was displayed instead of this telephone number because it was registered in the phone book application 303 as personal information of “Taro”. As a result, the mobile phone user who has confirmed the call history can easily identify the other party.

[0048] In FIG. 5 (a), the emotion of the other party is displayed. However, the mobile phone user's own emotion is displayed, and the other party's emotion and the cellular phone user's own emotion are displayed. It may be configured to display both at the same time! / ヽ. When displaying the emotion of the mobile phone user himself / herself, the emotion of the item “signal source” force S “sending speech” is extracted from the information list stored in the emotion identification device 20 shown in FIG. , Display. Also, it is not necessary to display both of these emotions with the characters that express those emotions (sorrow, affection, etc.) and the marks that represent those emotions. Further, it may be displayed in another display form (still image, moving image, etc.). [0049] In order to display the incoming call history shown in FIG. 5 (b), the display control unit 30 also sets the item "outgoing / incoming" in the information list stored in the emotion identification device 20 shown in FIG. "Incoming call" (data corresponding to one call b and c), and the other party who should specify the emotion set in step S401, that is, the item "Signal source" is "Received voice" From the data for one call, the data of the items "call start time", "call destination phone number", and "emotion" are extracted, and the items of the incoming call history in Fig. 5 (b) are "Received date" and "Name". , “Emotion” is displayed in the corresponding part. Note that the name “Jiro Matsushita” is displayed in the “name” field of the incoming call history in FIG. 5B instead of the telephone number “09000001111”! This is the same as the reason described above. is there.

[0050] The call history is displayed on the mobile phone according to the first embodiment of the present invention.

It is possible to determine the feelings of the other party or the mobile phone user based on the content of the past call, just by looking at it, so the past call can be made without any effort by the mobile phone user. Can help you remember the contents.

[0051] [Display format using schedule book]

Next, processing for displaying a schedule by the mobile phone according to the first embodiment of the present invention will be described. The processing flow for displaying the schedule is the same as that in FIG. 4 except that the scheduler application 302 is started and executed instead of starting and executing the call history application 301 in steps S407 and S408 described above. FIG. 6 shows a display example of the schedule book by the mobile phone according to the first embodiment of the present invention. Fig. 6 (a) is a calendar displayed from the schedule book, Fig. 6 (b) is a display example displaying emotions for each date in the calendar, and Fig. 6 (c) is a display on a specific date. It is a display example that displays the transition of emotion

[0052] When the display control unit 40 starts up the scheduler application 302 by accepting the input operation by the operation key provided in the mobile phone of the present invention also by the mobile phone user power (step S407, YES), first, the display control unit 40 in FIG. Display the calendar shown in a). Furthermore, when the display control unit 40 receives from the mobile phone user an input operation instructing to display the mobile phone user's own emotions on each date using the operation keys, the emotion identifying device 20 shown in FIG. From the list of stored information, extract the emotion of the mobile phone user who is the target of emotion identification, that is, the data for one call whose item “Signal Source” is “Transmission Voice”. Put out. In the list of information stored in the emotion identification device 20 shown in Fig. 3, the emotion of the mobile phone user identified by the call on September 9, 2005 is “joy”. Since the mobile phone user's own emotion identified in (1) is “normal”, the display control unit 40 displays a mark indicating a feeling of joy on September 9, 2005, in the corresponding date column. On September 10, 2005, a mark representing normal feeling is displayed. For other dates, these processes are performed in the same way, and the emotions for each date in the calendar are displayed as shown in Fig. 6 (b).

[0053] In the list of information stored in emotion identification device 20 shown in Fig. 3, the emotions of mobile phone users are identified by one call on September 9, 2005 and September 10, 2005, respectively. Since there was only one emotion that was given power, it would be good to display a mark representing that one emotion on each date. However, there may be multiple different emotions identified in each call on the same date in the information list. In such a case, the display control unit 40, among a plurality of calls on a single date, the emotion specified by the call with the newest call date and time, the emotion specified by the call with the longest call time, The emotion that has been identified most frequently is displayed on the display unit 50 as a total emotion on that date. In addition, although the structure which displays the total feeling in a certain day was demonstrated here, you may display the total feeling in predetermined periods, such as a week and a 1-month unit.

[0054] Alternatively, if there are multiple different emotions specified in each of multiple calls in the information list on one date, the level of the emotion is set for the emotion specified for each call. In order to do so, weighting may be performed, and based on the degree of the emotion, it may be possible to identify the total emotion. The display control unit 40 weights the emotion specified for each call so that the newer the call date or the longer the call time, the larger the numerical value indicating the degree of that emotion (for example, the call date and time). Force to display the total emotion The number of seconds of the difference minus the date and time is numerically expressed as the degree of that emotion, or the number of seconds of the call time is digitized as the degree of that emotion!), Numerical value The degree of emotion that has been converted into a certain level! ヽ (which may be a numerical value of the emotions specified for each call and the sum of those values for each emotion;) The maximum size is indicated by the numerical value! / ヽ If the weighted emotion does not exceed the predetermined threshold, “normal” is displayed as the total emotion. According to this configuration

In order to display the feelings that are particularly strong among the feelings identified for every call made on a certain day as the total feelings for that day, The accuracy of identification can be improved.

[0055] By calculating and setting the above threshold based on the numerical value indicating the degree of each emotion, emotions that are difficult to express in everyday life (for example, affection, joy, anger, sadness, two neutrals) If the emotion is identified from a phone call, the emotion of “affection” is an emotion that is difficult to express in everyday life). Can be easily displayed. For example, the threshold is calculated by the formula shown in Equation 1.

[Number 1]

¾ value: (([degree of joy] X [weight of joy]) + ([degree of anger] X [weight of anger]) +

([Sorrow Degree] X [Sorrow Weight]) + ([Degree of Love X X [Sweet Weight])))

C [degree of joy] + [degree of anger] + [degree of sadness] + [degree of love]) X [weight of K value]

[0056] Here, the numbers of "joy weight", "anger weight", "sadness weight", and "love weight" are used for emotions that are difficult to express in life (ie, "love"). On the contrary, it becomes smaller for emotions that are easily expressed in life, and the value of “threshold weight” is the initial value of the threshold. For example, by increasing the “weight of love”, the threshold value calculated when “affection” is included in the emotions identified in multiple calls is increased, and as a result, the numerical value “ The degree of “affection” easily exceeds this threshold, but the degree of emotions other than “affection” does not easily exceed this threshold, so it is easy to identify “affection” as a comprehensive emotion. By setting the threshold in this way, even if the owner is an individual whose joy and power is specified for each call, the affection is specified in several calls, so that the overall feeling can be expressed in love. And can reduce emotional bias.

[0057] The display control unit 40 also selects a specific date using the operation keys (in FIG. 6 (b), select September 2! /), And the emotion of the mobile phone user on that date is displayed. When an input operation instructing to display the transition is received from the mobile phone user, the item “call start time” in the information list stored in the emotion identification device 20 shown in FIG. 3 is that specific date. so In addition, the mobile phone user's own emotions for which emotions are to be identified, that is, data for one call whose item “signal source” is “sending speech” are extracted. The display control unit 40 refers to the items “call start time” and “emotion” in the extracted data for one call, and for example, as shown in FIG. Display your feelings in chronological order.

[0058] Note that the display control unit 40 displays a mark representing the emotion of the mobile phone user as shown in FIGS. 6 (b) and 6 (c). It may be displayed. At this time, if the display control unit 40 receives an input operation from the mobile phone user instructing to display an emotion of the other person on each date by the operation key (at this time, the other person also includes the time of the input operation). Specified by the phone number entered or specified by the phone number registered in the phone book if the other person to be identified is already registered in the phone book) From the list of information stored in device 20, extract the emotions of others, that is, the item “Signal Source” is “Received Voice” and the item “Destination Phone Number” matches the entered phone number. To do.

[0059] By displaying the schedule on the mobile phone according to the first embodiment of the present invention, it is possible to determine which other person or mobile phone user has used to make a call with a specific other person in the past. Since it is possible to determine whether or not the emotion has been felt, it is possible to help the mobile phone user remember the past call partner without having to go through the trouble.

[0060] [Display format using phone book]

Next, processing for displaying a telephone directory by the mobile phone according to the first embodiment of the present invention will be described. The processing flow for displaying the phone book is the same as that in FIG. 4 except that the phone book application 303 is started and executed instead of starting and executing the call history application 301 in steps S407 and S408 described above. FIG. 7 shows a display example of the telephone directory by the mobile phone according to the first embodiment of the present invention. Fig. 7 (a) is a display example of the name and phone number of each person registered in the phone book, and Fig. 7 (b) is a display example of emotions for each individual registered in the phone book. Fig. 7 (c) shows a display example of an individual's emotional call status.

[0061] The display control unit 40 performs an input operation using an operation key provided in the mobile phone of the present invention. When the phone book application 303 is started by accepting the talk user power (step S407, YES), for example, as shown in Fig. 7 (a), the name of the person whose name begins with a line and its individual Are displayed for each individual. Further, when the display control unit 40 accepts an input operation for instructing to display the personal emotion being displayed by the operation key, the mobile phone user power is received, among the information lists stored in the emotion identification device 20 shown in FIG. Of the person whose emotion is to be identified, that is, the item “Signal source” is “Received voice” and the item “Called phone number” matches the phone number registered for that individual. Extract data for one call. In the list of information stored in the emotion identification device 20 shown in FIG. 3, the telephone number “09000000000” matches the telephone number of the displayed name “Taro Matsushita” and the telephone number “09000001111” is displayed. In order to match the telephone number of the name “Jiro Matsushita”, the display control unit 40 puts marks representing the extracted emotions “sorrow” and “love” in the corresponding “Taro Matsushita” and “Jiro Matsushita” fields. Display. For other names such as “Hanako Matsushita”, these processes are performed in the same way, and for each individual registered in the phone book as shown in Fig. 7 (b), it is obtained from the past call of that individual. Display the sentiment.

[0062] In the information list stored in emotion identification device 20 shown in Fig. 3, since the emotion of the individual being displayed has only one emotion identified in one call, that emotion It would be good to display the mark that represents. However, there may be cases where a person's emotions have multiple different emotions in the information list identified by multiple calls. In such a case, the display control unit 40 has the latest call date and time, the emotion specified by the call, the longest call time, the emotion specified by the call, and the most specific among the emotions of a certain individual. It is also possible to cause the display unit 50 to display a number of emotions that have been repeated many times as the overall emotion of the individual. Further, the display control unit 40 displays the appearance frequency of emotions specified by a plurality of calls with an individual as shown in the display example of the emotional call status of an individual in FIG. 7 (c). May be. In Fig. 7 (c), a specific individual displayed in the phone book is selected by the operation key ("Taro Matsushita" is selected in 7 (b)) and out of 25 calls with the selected individual. Show how many times each emotion was identified (and the percentage of all calls)! /

[0063] Alternatively, in a certain individual emotion, a plurality of specified by each of a plurality of calls If there are different emotions in the information list, the emotion specified for each call with the individual is weighted to set the level of that emotion, and the overall emotion is identified from the level of emotion. It is also possible to do. The display control unit 40 weights the emotion specified for each call with the individual so that the newer the call date and time or the longer the call time, the larger the numerical value indicating the degree of the emotion (for example, The number of seconds, which is the difference between subtracting the current date and time from the date and time of the call and subtracting the current date and time, is used as the degree of emotion, or the number of seconds of the call time is used as the degree of emotion. ), The degree of emotion expressed in numbers! Of 数値 / ヽ (which may be a numerical value of emotions specified for each call and the sum of those values for each emotion), the degree is indicated by the largest value exceeding a certain threshold. If the weighted emotion does not exceed the predetermined threshold, “normal” is displayed as the total emotion of the individual. According to this configuration, an emotion with a particularly high degree of emotion identified among all calls made with a certain individual is displayed as the total emotion of that individual. It is possible to improve the accuracy of identifying overall feelings. By calculating and setting the above thresholds based on numerical values indicating the degree of each emotion, emotions that are difficult to express in everyday life (for example, affection, joy, anger, sadness, and neutrality are also configured. If the emotion is identified from a call that has a feeling of “love” that is difficult to express in everyday life !, it is an emotion that is difficult to express. (For example, the threshold value is calculated by the formula shown in Equation 1.)

[0064] By displaying the phone book on the mobile phone according to the first embodiment of the present invention, the phone book can be used to call a specific other person in the past by simply looking at the phone book! Since it is possible to judge whether it is hot or not, it is possible to help the mobile phone user remember the past call partner without taking time and effort.

[0065] [Other display formats]

In the mobile phone according to the first embodiment of the present invention described above, the example in which information representing emotion is additionally displayed in the display format when executing the PIM application has been described in detail. Even without using the display format, the emotion identification device 20 remembers Extract data that satisfies the specified conditions from the data in the list of information, display the extracted data and the calculation results of the data power, etc. It is also possible to inform mobile phone users about how their feelings are. FIG. 8 shows an example of sorting for each emotion by the mobile phone according to the first embodiment of the present invention.

[0066] The display control unit 40 uses the operation keys provided on the mobile phone of the present invention to perform an input operation to instruct the mobile phone user to display a list of call partners whose emotions were "sad" in past calls. Of the information list stored in the emotion identification device 20 shown in FIG. 3, the item “Signal source” is “Received voice” and the item “Emotion” is “Sorrow”. Extract something. The display control unit 40 further counts the number of times the callee telephone number has been extracted when the callee telephone number that has already been extracted is extracted again. When the display control unit 40 extracts all the corresponding telephone numbers in the information list, as shown in FIG. 8, the personal names corresponding to the telephone numbers in the descending order of the number of extractions (individual names are called If the phone number is already registered in the phone book application 303, the phone number is identified) and the number of times of extraction is displayed. In addition, in the display example of FIG. 8, in addition to the individual name and the number of extractions, the total number of calls with the extracted call destination telephone number and the ratio of the number of extractions to the total number of calls are also displayed. . By displaying such information, the mobile phone user can roughly know what other people have felt in calls made with a specific other person in the past.

[0067] By the way, in the mobile phone according to the first embodiment of the present invention thus far, from the received voice signal and the transmitted voice signal inputted at the time of voice call, the emotions of other telephone users who have made the voice and the relevant The case of estimating mobile phone user emotions has been described. The case of estimating the emotion of a person who describes a character string from a character string will be described below.

FIG. 9 shows another configuration diagram of the mobile phone according to the first embodiment of the present invention. The mobile phone according to the first embodiment of the present invention includes a PIM (Personal Information Manager) application group 30, a display unit 50, a communication device 60, an emotion identification device 70, and a display control unit 80. The description of the same reference numerals as in FIG. 1 is as described above, so the description is omitted. Abbreviated.

The call device 60 includes a mail transmission / reception unit 601 and a character information output unit 602. The mail transmission / reception unit 601 performs mobile wireless communication with the mobile radio base station, receives an e-mail addressed to the mail address assigned to the mobile phone of the present invention, and addresses an arbitrary mail address from the mobile phone of the present invention. To send e-mail. The character information output unit 602 is a character data described in the e-mail received via the mail transmitting / receiving unit 601 or the e-mail for transmission (the character string described in the title name or e-mail text in the e-mail). At least a part of the character data about the) is output to the emotion identification device 70. The character information output unit 602 outputs the received e-mail data to the display control unit 80 to display the e-mail, and displays the e-mail by operating the operation key (not shown) provided in the mobile phone of the present invention. E-mail data for transmission created on 50 is input from the display control unit 80.

The emotion identification device 70 includes an emotion estimation unit 701, an emotion accumulation unit 702, an emotion identification unit 703, and an emotion information storage unit 704. Emotion estimation unit 701 estimates the emotion of the mail creator describing the character string from the character data input from communication device 20. The emotion estimation unit 701, for example, expresses the degree of emotion expressed by the values of 0, 1, 2 for each factor of emotion information composed of love, joy, anger, sadness, neutral (normal) (0: emotion None, 1: Emotional weakness, 2: Emotional strongness, input character data (a character string that has at least one character strength, a mark that expresses an image, etc.) This mark is called a pictogram Is estimated for each sentence from the head of the sentence, and each estimated value is sequentially output to the emotion storage unit 702.

The emotion accumulating unit 702 accumulates the numerical values for each factor input from the emotion estimating unit 701 in association with the input order. Emotion accumulation unit 702 inputs a numerical value for each of a series of factors from emotion estimation unit 701 (the series here refers to the input start force of numerical input from emotion estimation unit 701 to emotion accumulation unit 702) Then, the numerical value for each factor in the series is stored as a set of data (hereinafter, a set of data is referred to as data for one mail).

[0072] Emotion identification unit 703 reads the data for one email from emotion accumulation unit 702, and The numerical value for each factor is analyzed, one characteristic factor is identified from the read numerical value, and the emotion represented by the one factor is output to the emotion information storage unit 704. When identifying one characteristic emotion, the emotion identification unit 703 uses the factor with the largest numerical value in the data for one email as the characteristic emotion, thereby identifying the content that is strongly impressed in the email. Emotion can be identified with emphasis. Another method is to identify emotions with emphasis on the content of the entire email, by using the factors that have the largest sum of the numbers from the beginning of the email to the end of the email in the data for one email as the characteristic emotion. In addition, it is possible to identify emotions with emphasis on the reverberation of the email content by making the characteristic emotion the factor with the most powerful numerical value at the end of the email sentence in the data for one email. .

[0073] Emotion information storage section 704 inputs an emotion represented by one factor from emotion identification section 703, and further, information on mail transmission / reception performed by mail transmission / reception section 601 of communication device 60 (the mail) The mail transmission / reception unit 601 inputs the mail transmission / reception error or the mail transmission time (mail reception time, destination, and sender's mail address). The emotion information storage unit 704 stores emotions input from the emotion identification unit 703 and various information input from the mail transmission / reception unit 601 in association with each other. FIG. 10 shows another example of the information list stored in the emotion identifying device in the mobile phone according to the first embodiment of the present invention. In FIG. 10, the emotion information storage unit 704 is an emotion in which the mail power is also specified for each piece of data for one email (that is, the sentiment of the mobile phone user of the present invention who is the mail creator in the case of outgoing mail). If it is a received mail, it is configured to memorize the emotion of the mail partner who is the mail creator).

[0074] The display control unit 80 activates and executes one of the applications in the PIM application group, extracts data necessary for processing of the executed application from the emotion information storage unit 700, and displays the data in the display unit 50. Display various information. A display example of the display unit 50 when the call history application, scheduler application, or phone book application is started and executed will be described later.

Next, another process by the mobile phone of the first embodiment of the present invention is described with reference to a processing flow for displaying a mail reception history by the mobile phone of the first embodiment of the present invention shown in FIG. explain. When communication device 60 starts mail transmission or mail reception (YES in step S1101), it outputs character data described in the received electronic mail or electronic mail for transmission to emotion identification device 70. (Step S1102). The emotion identifying device 70 reads the text data input from the communication device 60 for each sentence from the beginning of the input character data for each factor of emotion information composed of affection, joy, anger, sadness, and neutral (normal). Or, it is estimated for each phrase (step S 1103). When the emotion identification device 70 finishes inputting the character data of the communication device 60 (step S 1104, YES), it analyzes the numerical value for each factor estimated from the beginning to the end of the series of character data, One characteristic factor is identified, and the emotion represented by the one factor is stored as the mail creator's emotion (step S 1105. At this time, as shown in FIG. Memorize with specific emotions).

[0077] After that, the display control unit 80 starts the mail history application 304 by accepting the input operation by the operation key provided in the mobile phone of the present invention by the mobile phone user power (step S1106, YES). According to the program code of the history application 304, the emotion specified for each email stored in the emotion identification device 70 and information related to the email are read, and the information is displayed on the display unit 50 in a predetermined display format. (Step S 1107).

[0078] Note that the specific process of displaying the mail history, schedule book, and phone book by the display control unit 80 is as described in [Display format using call history], [Display format using schedule book], In [Display format using phone book], the name of the data object to be extracted is different. (The mail creator whose emotion should be specified is indicated by “Receiving voice” and “Sending” in the item “Signal source” in Fig. 3. Instead of discriminating by “spoken voice”, it is determined whether it is a mobile phone user power mail partner by “send” or “receive” in the item “send / receive” in FIG. The item “Send / Receive Time” in FIG. 10 is used instead of the item “Call Start Time”, and the item “Recipient Email Address” in FIG. 10 is used instead of the item “Call Target Phone Number” in FIG. Is common except for , The description thereof is omitted. Also, in [Display format using schedule book], if there are multiple different emotions identified in multiple emails on one date in the information list, or [Use phone book In the [Display Format], if there are multiple different emotions specified by multiple emails in the information list in the display list, the display control unit 40 can receive multiple emails received on a single date, Among the multiple emails received, the date and time when the email was received is the newest, the emotion identified by the email, the number of characters in the email body is the most, the emotion identified by the email, and the number of times identified the most Are displayed on the display unit 50 as the total feelings of the date or the individual.

[0079] Or, if there are multiple different emotions specified in each of multiple emails in the information list on one date, or multiple different emails specified by each of multiple emails for an individual emotion When emotions are listed in the information list, it is possible to weight the emotions specified for each e-mail to set the level of emotions and to specify the total emotions based on the level of emotions. The display control unit 40 weights the emotion specified for each email so that the newer the email reception date or the more the number of characters in the email body, the larger the numerical value indicating the degree of the emotion (for example, E-mail reception date / time power The number of seconds after subtracting the current date and time for displaying comprehensive emotions is numerically expressed as the degree of the emotion, or the amount of data in the mail text is expressed as the degree of emotion. Of emotions that exceed a certain threshold and whose degree is indicated by the largest numerical value is displayed as a comprehensive emotion, while weighted emotions are displayed at a predetermined threshold. If there is nothing that exceeds, “normal” is displayed as a general emotion. According to this configuration, the emotions that are identified for every email performed on a certain day and that have a particularly high degree of emotion are displayed as the total emotion for that day. The accuracy of specifying the overall emotion can be improved.

[0080] As described above, according to the mobile phone of the first embodiment of the present invention, it is possible to think about the contents of past calls and the contents of e-mail text without forcing the mobile phone user to spend time and effort. It can help to put out. In addition, in conjunction with various existing PIM applications, mobile phone users can be notified by notifying mobile phone users of emotions of mobile phone users and the emotions of other parties that have communicated with those users using mobile phones. It is possible to provide an environment in which a person can easily remember past communications.

[0081] (Second Embodiment) In the mobile phone according to the second embodiment of the present invention, when the mobile phone user recalls the contents of a past call as in the first embodiment, the emotion that symbolizes the content is specified in advance. In addition, if the call voice is recorded, the presence / absence of the recorded data is also displayed, and the mobile phone user of the recorded data can be effectively reminded of the past call contents. The data portion that can be reproduced. As a result, the mobile phone user can remember the contents of the call while minimizing the time and effort required to remember the contents of the past call. FIG. 12 shows a configuration diagram of the mobile phone according to the second embodiment of the present invention.

[0082] The mobile phone according to the second embodiment of the present invention includes a call device 10, an emotion identification device 20, a PIM (Personal Information Manager) application group 30, a display control unit 40, a display unit 50, a playback control unit 90, and a speaker. It is composed of 100. The configuration of the mobile phone according to the second embodiment of the present invention is the same as the configuration of the mobile phone according to the first embodiment of the present invention shown in FIG. 1 except that the voice storage unit 105, the playback control unit 90, and the speaker 100 of the communication device 10 are provided. It is an added one. With the addition of this configuration, functions to be processed are added to the communication unit 101 of the communication device 10, the voice signal output unit 102, the emotion abandonment unit 204 of the emotion identification device 20, and the display control unit 40. The description of the other parts that are the same as those in FIG. 1 is the same as that described in the first embodiment, and thus the description thereof is omitted.

[0083] The voice signal output unit 102 emotionally receives a voice signal received by another telephone force via the communication unit 101, a voice signal of the mobile phone user picked up by the call microphone 104, or both voice signals. Output to specific device 20. Also, the voice signal output unit 102 outputs the voice received by other telephone capabilities to the call speaker 103, and outputs the call voice of the mobile phone user collected by the call microphone 104 to the communication unit 101. In addition, the audio signal output unit 102 outputs to the audio storage unit 105 the audio signal received by the other telephones via the communication unit 101 and the audio signal of the mobile phone user collected by the call microphone 104. To do.

[0084] When the voice storage unit 105 of the call device 10 is notified by the communication unit 101 that a call with another telephone has been started and receives a voice signal from the voice signal output unit 102, the voice storage unit 105 starts recording the voice signal. To do. The audio storage unit 105 receives the audio signal from the audio signal output unit 102. When the recording is finished, the recording of the audio signal is ended, and the communication unit 101 is notified that the recording is completed. The communication unit 105, when notifying the voice storage unit 105 that a call with another telephone has started, is identification information for identifying the call (the time at which the call was started or ended is used to identify the call). Information, or assigning a predetermined number to the call every time a call is started, where the call start time is treated as identification information for identifying the call) The voice storage unit 105 records the identification information and the voice signal to be recorded in association with each other (the voice recording unit 105 identifies the voice signal based on the identification information).

[0085] Emotion information storage unit 204 receives an emotion expressed by one factor from emotion identification unit 203, and further, information related to a call made by communication unit 101 of call device 10 (the call is transmitted) Any incoming call, call start time 'call end time, and other phone identification information (for example, the phone number of the callee), and the presence of audio data that recorded the call by the call device 10. Input from the communication unit 101. The emotion information storage unit 204 stores the emotion inputted from the emotion identification unit 203, the various information inputted from the communication unit 101, and the presence / absence of recorded audio data in association with each other (the emotion identification unit 203 receives the received voice signal). When one characteristic factor is identified from each of the data for one call for each transmitted voice signal, the force that is the signal source of either the received voice signal or the transmitted voice signal is stored as one factor. ). FIG. 13 shows a list of information stored in the emotion identifying device in the mobile phone according to the second embodiment of the present invention. The information list stored in the emotion identifying device in the mobile phone according to the second embodiment of the present invention is obtained by adding the item “recorded data presence / absence” to the information list in FIG.

[0086] The reproduction control unit 90 is an audio signal specified by identification information (identification information is input by an input operation using an operation key provided on a mobile phone) among the audio signals stored in the audio storage unit 105. Is output and output to the speaker 100 to output sound.

Next, processing by the mobile phone of the second embodiment of the present invention will be described with reference to a processing flow for displaying a call history by the mobile phone of the second embodiment of the present invention shown in FIG. The process assigned the same reference numerals as the process flow for displaying the call history by the mobile phone according to the first embodiment of the present invention shown in FIG. 4 is as described in the first embodiment. Therefore, explanation is omitted.

[0088] When the call device 10 starts a call (step S402, YES), the call device 10 outputs the received voice signal and the transmitted voice signal to the emotion identifying device 20, and further starts recording the voice signal. (Step S1401). The emotion identifying device 20 is a factor of emotion information composed of affection, joy, anger, sadness, and two uitals (normal) for the target speech signal of the speech signal input from the call device 10. Each time, the degree of emotion is continuously estimated at a predetermined time interval (step S404). When the call by the call device 10 ends (YES in step S405), the emotion identification device 20 analyzes the numerical value for each factor estimated during the call, and sets the characteristic factor to 1 And identify the emotion expressed by one of the factors as the other party's or mobile phone user's emotion in this call (step S1402). The presence / absence of voice data is stored in association with the identified emotion). Further, when the input of the audio signal is completed, the call device 10 ends the recording of the audio signal, and records the recorded data and identification information for identifying the recorded data in association with each other (step S1403).

[0089] After that, the display control unit 40 starts the call history application 301 by accepting the input operation using the operation keys provided in the mobile phone of the present invention by the mobile phone user power (step S407, YES). According to the program code of the application 301, the emotion identified for each call stored in the emotion identification device 20, the information related to the call and the presence / absence of recorded voice data are read, and the information is displayed on the display unit 50. Display in a predetermined display format (step S 1404).

[0090] [Display format using call history]

FIG. 15 shows a display example of a call history by the mobile phone according to the second embodiment of the present invention. Fig. 15 (a) is a transmission history generated based on the information stored in the emotion identification device 20 shown in Fig. 13, and Fig. 15 (b) shows the information stored in the emotion identification device 20 shown in Fig. 13. This is the original received history.

[0091] In order to display the outgoing call history shown in Fig. 15 (a), the display control unit 30 sets the item "outgoing / incoming" in the information list stored in the emotion identifying device 20 shown in Fig. 13 to "outgoing". (In this case, the data a for one call is applicable) From the data for one call with the other party, that is, the item “Signal Source” is “Received Voice”, the data of the items “Call Start Time”, “Destination Phone Number”, “Emotion”, “Recorded Data” Is extracted and displayed at the corresponding locations of the items of “Originating date”, “Name”, “Emotion”, and “Recorded voice” in the outgoing history in Fig. 15 (a).

[0092] In order to display the incoming call history shown in FIG. 5 (b), the display control unit 30 also sets the item "outgoing / incoming" in the information list stored in the emotion identification device 20 shown in FIG. "Incoming call" (data corresponding to one call b and c), and the other party who should specify the emotion set in step S401, that is, the item "Signal source" is "Received voice" From the data for one call, the data of the items “call start time”, “call destination phone number”, “emotion”, “recorded data presence / absence” are extracted, and the incoming call history in Fig. 15 (b) is extracted. Displayed in the corresponding sections of the items “Date and time of incoming call”, “Name”, “Emotion”, and “Recorded voice”.

[0093] [Display format using schedule book]

Next, a process for displaying a schedule by the mobile phone according to the second embodiment of the present invention will be described. The processing flow for displaying the schedule is the same as that in FIG. 14 except that the scheduler application 302 is started and executed instead of starting and executing the call history application 301 in steps S407 and S1404 described above. FIG. 16 shows a display example of the schedule book by the mobile phone according to the second embodiment of the present invention. Fig. 16 (a) is a calendar displayed in the schedule book, Fig. 16 (b) is a display example that displays emotions for each date in the calendar, and Fig. 16 (c) is for a specific date. It is a display example that displays the transition of emotion. FIGS. 16 (a) and 16 (b) are the same as FIGS. 6 (a) and 6 (b) of the first embodiment, and the processing of the display control unit 40 for displaying these is also performed. Since it is the same as that of the first embodiment, description thereof is omitted.

[0094] The display control unit 40 selects a specific date using the operation keys (in FIG. 16 (b), selects September 2), and changes the emotion of the mobile phone user on that date. When an input operation for instructing display is accepted from a mobile phone user, the item “call start time” in the information list stored in the emotion identification device 20 shown in FIG. 13 is that specific date, and The mobile phone user's own emotions for which emotions are to be identified, that is, data for one call whose item “signal source” is “sending speech” are extracted. The display control unit 40 extracts Referring to the items “call start time”, “emotion”, and “recorded data presence / absence” in the data for one call, for example, as shown in Fig. 6 (c), the mobile phone user himself on September 2 “Sound recording” is displayed along with the feeling of, and is displayed in chronological order.

[0095] [Display format using phone book]

Next, processing for displaying a telephone directory by the mobile phone according to the second embodiment of the present invention will be described. The processing flow for displaying the phone book is the same as that in FIG. 14 except that the phone book application 303 is started and executed instead of starting and executing the call history application 301 in steps S407 and S1404 described above. FIG. 17 shows a display example of the phone book by the mobile phone according to the first embodiment of the present invention. Fig. 17 (a) is a display example of each person's name and phone number registered in the phone book, and Fig. 17 (b) is an example of emotion display for each individual registered in the phone book. .

[0096] The display control unit 40 starts up the phone book application 303 by accepting the input operation using the operation keys provided in the mobile phone of the present invention by the mobile phone user power (step S407, YES). For example, FIG. As shown in (a), the name of the person whose name begins with a line is displayed for each individual person's name and the individual's telephone number. Further, when the display control unit 40 receives an input operation for instructing to display the personal emotion being displayed by the operation key from the mobile phone user, the display control unit 40 of the information list stored in the emotion identification device 20 shown in FIG. Of the person whose emotion is to be identified, that is, the item “Signal source” is “Received voice” and the item “Called phone number” matches the phone number registered for that individual. Yes Extract data for one call. In the list of information stored in the emotion identification device 20 shown in FIG. 13, the telephone number “09000000000” matches the telephone number of the displayed name “Taro Matsushita” and the telephone number “09000001111” is displayed. In order to match the telephone number of “Jiro Matsushita”, the display control unit 40 displays the marks representing the extracted emotions “sorrow” and “love” in the corresponding “Taro Matsushita” and “Jiro Matsushita” fields. . Further, if the item “Presence / absence of recording data” is “Yes” in the extracted data for one call, the display control unit 40 displays “Recorded voice present” in the corresponding column as shown in FIG. indicate.

[0097] [Playback of recorded audio]

The mobile phone according to the second embodiment of the present invention has [display format using call history], [schedule This section describes the process of playing back the recorded audio after displaying the presence or absence of the recorded audio as described in [Display Format Using Single Book] and [Display Format Using Phone Book].

[0098] Select the display location where “Recorded audio is present” is selected using the operation key provided on the mobile phone, and play back when the mobile phone user receives an input operation instructing to play the recorded audio. The control unit 90 identifies the information displayed in the emotion information storage unit 204 as to which one call data the selected display location is displayed based on, and identifies the data for one call. The voice signal to be reproduced is read out of the voice signals stored in the voice storage unit 105 from the item “call start time” in (in the voice storage unit 105, the call start time as the identification information and the voice to be recorded The signal is stored in association with the signal). The reproduction control unit 105 outputs the read audio signal to the speaker 100, so that the mobile phone can output sound up to the end of the recorded voice.

[0099] According to the mobile phone of the second embodiment of the present invention, as in the first embodiment, when the mobile phone user remembers the content of a past call, it is sometimes specified in advance and symbolizes the content. In addition to displaying emotions, if the call voice is recorded, the presence or absence of the recorded data is also displayed, and the recorded data is played back so that the mobile phone user can confirm the contents. Can remember.

[0100] Note that the mobile phone according to the second embodiment of the present invention does not need to record every call one by one, so that a mobile phone user can record a voice signal during a call by performing a predetermined operation. It may be configured. When the emotion specified for each call, information about the call, and the presence or absence of recorded audio data are displayed in a predetermined display format, if there is no recorded audio data, for example, Fig. 15 (b), Fig. 17 As shown in (b), “No sound” is displayed at the corresponding location.

[0101] [Details on how to record and play call audio]

In the above-described recording and playback of call voice, simply recording the voice signal from the start of the call to the end of the call and playing the recorded voice signal from the beginning to the end. About an embodiment in which recording and playback of voice can be performed from a location that satisfies a predetermined condition, thereby reducing the amount of data of the voice signal to be recorded and recalling the contents of past calls in the call voice without taking time. explain. FIG. 18 shows the second of the present invention. The processing flow (Example 1) which records the audio | voice by the mobile telephone of embodiment is shown.

[0102] When the call device 10 starts a call (step S402, YES), the call device 10 outputs a received voice signal and a transmitted voice signal to the emotion identifying device 20. (Step S403). The emotion identifying device 20 is a factor of emotion information composed of affection, joy, anger, sadness, and neutral (normal) for the speech signal to which emotion should be identified among the audio signals input from the call device 10. Every time, the degree of emotion is continuously estimated at a predetermined time interval (step S404). In the process of step S404, if there is a factor that exceeds the threshold that is at least one of the estimated emotion information factors (S1801, YES), the call device 10 records the received voice signal and the transmitted voice signal. (S1802. In this process, the emotion estimation unit 201 for determining whether there is a factor with a numerical value exceeding a threshold value among the estimated emotion information factors is stored in the voice storage unit 105. A control signal that starts recording the received voice signal and the transmitted voice signal will be output). The emotion identification device 20 repeats these processes until the call by the call device 10 is completed (step S405, YES). For example, the emotion identification device 20 has the ability to start a call with respect to the received voice signal for each of the emotional information elements composed of affection, joy, anger, sadness, and neutral (normal) as shown in Fig. 2. If the recording is started when the factor of certain emotion information is 2 or more when continuously estimated at a predetermined time interval, the communication device 10 has a pleasure factor in the interval from 10 seconds to 15 seconds after the call starts. Since it was estimated to be 2, the recording of the call voice from this interval to the end of the call is started (first recorded audio), and the anger factor is 2 in the interval of 45 seconds after the start of the call for 50 seconds It is estimated that the recording of the call voice from this section to the end of the call is started (second recorded voice), and the sorrow factor is estimated to be 2 in the section of 55 seconds after the start of the call. Therefore, this segment strength also records the call voice until the end of the call. Start (third recorded voice).

[0103] After the call by the call device 10, the emotion identification device 20 analyzes the numerical value for each factor estimated during the one call, specifies one characteristic factor from the numerical value, The emotion expressed by the one factor is stored as the feeling of the other party or the user of the mobile phone in this call (step S 1402. At this time, as shown in FIG. 13, information about the call and recorded voice data are recorded. Is stored in association with the identified emotion). Also, When the input of the audio signal is completed, the call device 10 ends the recording of each call audio, and the recorded audio that has started recording after the factor specified in the process of step S 1402 is estimated to be 2 (for example, step If the factor specified in the process of S1402 is pleasure, the first recorded voice) is recorded in association with the identification information for identifying the recorded data (S1803. In this process, one characteristic The emotion specifying unit 203 that has identified the factor 203 will notify the voice storage unit 105 of the factor). Note that the communication device 10 may delete the second and third recorded voices after step S1803. In addition, the communication device 10 leaves the second and third recorded voices as second and third candidates representing the emotion of the call, and proceeds to the process of step S 1803 to obtain the second and third recorded voices. The identification information for identifying the recorded data may be recorded in association with (In this case, [Display format using call history], [Display format using schedule book], [Phone book When displaying the presence / absence of recorded audio as described in [Display format used], display the number of recorded audios in order of priority, or display only the number of recorded audios).

[0104] After this, [Display format using call history], [Display format using schedule book],

After displaying the presence / absence of recorded audio as described in [Display format using phone book], the process of playing back the recorded audio will also cause the voice location power that was the main cause of identifying the emotion of the recorded audio. Since playback can be started, it is possible to provide an environment in which it is easy to remember communication over past calls without having to spend 4 hours after recording audio playback. Also, it is not always necessary to record all audio signals from the start of the call to the end of the call! Therefore, the amount of sound data to be recorded can be reduced.

[0105] Further, another embodiment for recording a call voice will be described. FIG. 19 shows a processing flow (Example 2) for recording a call voice by the mobile phone according to the second embodiment of the present invention.

[0106] When the call device 10 starts a call (step S402, YES), the call device 10 outputs a received voice signal and a transmitted voice signal to the emotion identification device 20, and further starts recording the voice signal. (Step S403). The emotion identifying device 20 is a factor of emotion information composed of affection, joy, anger, sadness, and two uitals (normal) for the target speech signal of the speech signal input from the call device 10. Each time, the degree of emotion is continuously estimated at a predetermined time interval (step S404). In the process of step S404, If there is a factor with a numerical value exceeding the threshold that is at least one of the emotional information factors (step S 1901, YES), the emotion identification device 20 provides information on the factor and the time when the factor exceeded the threshold. (For example, the elapsed time since the start of the call. It is only necessary to know at which point in the call the factor exceeded the threshold) (Step S 1902. In this process, the emotion estimation unit 201 determines the factor of the emotion information. If it is estimated that there is a factor with a numerical value exceeding the threshold value, the emotion accumulation unit 202 stores the factor and the time when the factor exceeds the threshold value). The emotion identification device 20 repeats these processes until the call by the call device 10 is completed (step S405, YES). For example, the emotion starter 20 can continuously determine the call start power for the received voice signal at the end of the call for each emotion information factor consisting of affection, joy, anger, sadness, and neutral (normal) shown in FIG. If the factor of emotion information is 2 or more and the factor and the time when the factor exceeded the threshold are memorized, the call device 10 will be in the interval from 10 seconds to 15 seconds after the start of the call. Since the joy factor was estimated to be 2, the joy factor and the time 10 seconds after the start of the call were memorized (first tag information), and the interval from 45 to 50 seconds after the start of the call Therefore, the anger factor and the time 45 seconds after the start of the call are memorized (second tag information), and 50 to 55 seconds after the start of the call. The sadness factor was estimated to be 2 in the interval It stores a child, and the time after the call start 50 seconds, the (third tag information).

[0107] After the end of the call by the communication device 10, the emotion identification device 20 analyzes the numerical value for each factor estimated during the one call, specifies one characteristic factor from the numerical value, The emotion expressed by the one factor is stored as the other party's or mobile phone user's emotion in this call, and the information about the time when the specified factor exceeds the threshold (tag information) is stored (step information). S 1903. At this time, in addition to the correspondence between the information related to the call shown in FIG. 13, the presence / absence of the recorded voice data, and the identified emotion, tag information is also stored in association with each other). Further, when the input of the audio signal is completed, call device 10 ends the recording of the audio signal, and records the recorded data and identification information for identifying the recorded data in association with each other (step S 1403).

[0108] After this, [Display format using call history], [Display format using schedule book], As described in [Display format using the phone book], after displaying the presence / absence of the recorded voice, playback of the recorded voice can start the voice location power that can identify the tag information power. Therefore, it is possible to provide an environment in which it is easy to remember a communication by a past call without taking a long time after reproduction of recorded sound. Moreover, since it is not always necessary to record all voice signals until the start of the call, the amount of sound data to be recorded can be reduced. If there are multiple tag information for the specified factor, a screen prompting you to select one of the tag information after displaying the presence or absence of the recorded audio is displayed and selected. Playback should be started from the audio location that can be identified from the tag information. Furthermore, by starting playback of the recorded voice from a location that is a predetermined time before the voice location that can be identified from the tag information, it is possible to avoid missing the voice location that is the main cause of identifying the emotion of the recorded speech.

[0109] Although the invention has been described in detail and with reference to specific embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention. is there.

[0110] This application is based on a Japanese patent application filed on December 16, 2005 (Japanese Patent Application No. 2005-363837), the contents of which are incorporated herein by reference.

Industrial applicability

[0111] According to the information processing terminal of the present invention, it is possible to support recalling the contents of past calls and the contents of e-mail texts without forcing the user to spend time and effort. This is useful in the field of information processing terminals that can identify the emotion of the creator of the email and the emotion of the speaker during the call from the character string described in the email and the voice during the call.

Claims

The scope of the claims

[1] An input means for inputting emotion specifying information composed of at least character data or speech;

Based on the emotion identifying information input by the input means! /, An emotion identifying means for identifying an emotion;

An information processing terminal comprising: display means for displaying information related to the emotion specified by the emotion specifying means.

[2] An information processing terminal according to claim 1,

E-mail sending and receiving means for sending or receiving e-mail

The input means inputs character data including at least a character string or a mark power included in the email transmitted or received by the mail transmitting / receiving means, and the emotion specifying means is based on the character data input by the input means. Identify emotions,

The information processing terminal, wherein the display means displays information related to emotions specified by the emotion specifying means and corresponding to the electronic mail transmitted or received by the mail transmitting / receiving means.

[3] An information processing terminal according to claim 2,

The information processing terminal displays at least a transmission time or a reception time, a transmission destination or a transmission source, and information on the emotion for each electronic mail in the order in which the electronic mail is transmitted or received by the mail transmission / reception means. .

[4] An information processing terminal according to claim 2,

The information processing terminal, wherein the display unit displays, for each date, information related to the emotion corresponding to an e-mail transmitted or received on the date by the mail transmitting / receiving unit.

[5] An information processing terminal according to claim 2,

The display means displays, for each target person registered in the telephone directory function, information on the emotion corresponding to the e-mail transmitted to the target person or the e-mail received from the target person by the mail transmitting / receiving means. Information processing terminal to display.

[6] The information processing terminal according to any one of claims 2, 4 and 5, The information processing terminal, wherein the display means displays any one of the information related to the emotion specified for each of the plurality of electronic mails transmitted or received by the mail transmitting / receiving means.

[7] An information processing terminal according to claim 1,

It has a calling means for making calls,

The input means inputs a call voice by the call means,

The emotion identifying means identifies an emotion based on the call voice input by the input means,

The information processing terminal, wherein the display means displays information relating to the emotions specified by the emotion specifying means, to the called party, the caller, or both the called party and the caller who made a call by the calling means.

[8] An information processing terminal according to claim 7,

The information processing terminal, wherein the display means displays at least a call time or a call arrival time, a callee or a caller, and information related to the emotion for each call in the order in which the call is made or received by the call means.

[9] The information processing terminal according to claim 7,

The information processing terminal, wherein the display unit displays, for each date, information related to the emotion corresponding to a call that is transmitted or received on the date by the call unit.

[10] An information processing terminal according to claim 7,

The display means displays, for each target person registered in the telephone directory function, information related to the emotion about the target person who has called by the call means.

[11] The information processing terminal according to any one of claims 7, 9 and 10,

The information processing terminal, wherein the display unit displays any one of the information related to the emotion specified for each of a plurality of calls that are transmitted or received by the call unit.

[12] The information processing terminal according to any one of claims 7 to 11, wherein

Call storage means for storing call voice by the call means;

Call playback means for playing back the call voice stored in the call storage means, The information processing terminal, wherein the call reproduction means reproduces a call voice specifying information related to the emotion displayed by the display means, among the call voices stored in the call storage means.

[13] An information processing terminal according to claim 12,

The call storage means stores a portion in which the emotion specified by the emotion specifying means is reflected in the call voice by the call means,

The information processing terminal, wherein the call reproduction means reproduces at least the part of the call stored in the call storage means.

[14] The information processing terminal according to claim 13,

The information processing terminal, wherein the call reproduction means reproduces the location with a preset time before the start time of the location as a reproduction start time.

[15] The information processing terminal according to any one of claims 1 to 14,

The information processing terminal displays the information related to the emotion identified by the emotion identifying unit with at least a character representing the emotion or a mark representing the emotion.