WO2013175665A1

WO2013175665A1 - Communication apparatus

Info

Publication number: WO2013175665A1
Application number: PCT/JP2012/082453
Authority: WO
Inventors: 有香三好
Original assignee: Ｎｅｃカシオモバイルコミュニケーションズ株式会社
Priority date: 2012-05-24
Filing date: 2012-12-14
Publication date: 2013-11-28

Abstract

A communication apparatus (100-1) determines, on the basis of input voice, the emotion of the speaker of the voice, divides the voice into words, calculates the use frequency for each word as divided, calculates the importance level of the word on the basis of both the determined emotion and the calculated use frequency, and transmits information in accordance with the calculated importance level.

Description

Communication device

The present invention relates to a communication device, a server, an information providing method, and a program that provide information related to voice.

In recent years, e-mail exchanges are increasing as a method of communication. However, there are cases where it is difficult to communicate only with characters such as e-mail.

On the other hand, voice communication (call) can make it easier to convey that it is difficult to communicate with characters alone, such as the tone of the speaker.

For example, a technique for judging an emotion based on an image of a user and a voice of the user has been considered (see, for example, Patent Document 1). If this technology is used for a telephone call, the user can convey information that cannot be conveyed only by characters.

JP 2009-151766 A

When it is necessary to confirm the content of a conversation in voice communication later, it is common practice to take notes centering on content that seems to be important in the conversation. However, in recent years, in order to save the trouble of taking notes and avoid writing omissions in the notes, methods such as recording a whole conversation using a recorder or the like are often used. It is coming.

However, since the user records the entire conversation and extracts (recognizes) important matters from the conversation, it is a problem that it is burdensome for the user to check all the recorded contents. There is. In addition, even if the user confirms this, since it is judged by the user's own subjectivity whether it is an important matter, the person speaking does not realize that the content being spoken as an important matter is an important matter. There is a fear.

In addition, it is difficult to say that important matters in conversation can be recognized only by judgment based on the emotions of users as described in Patent Document 1.

An object of the present invention is to provide a communication device, a server, an information providing method, and a program that solve the above-described problems.

The communication device of the present invention
An emotion determination unit that determines an emotion of a speaker of the voice based on the input voice;
A usage frequency calculation unit that divides the voice into words and calculates a usage frequency for each word;
An importance calculator that calculates the importance of the word based on the emotion determined by the emotion determiner and the use frequency calculated by the use frequency calculator;
A notifying unit for notifying information according to the importance calculated by the importance calculating unit.

The server of the present invention
An emotion determination unit that determines the emotion of the speaker of the voice represented by the voice data based on the voice data transmitted from the communication device;
A usage frequency calculation unit that divides the voice data into words and calculates a usage frequency for each word;
An importance calculator that calculates the importance of the word based on the emotion determined by the emotion determiner and the use frequency calculated by the use frequency calculator;
And a communication unit that transmits information corresponding to the importance calculated by the importance calculation unit to the communication device.

The information providing method of the present invention includes:
A process of determining the emotion of the speaker of the voice based on the input voice;
Processing to divide the voice into words;
A process of calculating a use frequency for each of the divided words;
Processing to calculate the importance of the word based on the emotion and the usage frequency;
A process of notifying information according to the calculated importance is performed.

The program of the present invention is
A program for causing a computer to execute,
A procedure for determining an emotion of a speaker of the voice based on the inputted voice;
Dividing the speech into words;
A procedure for calculating a use frequency for each of the divided words;
Calculating the importance of the word based on the emotion and the usage frequency;
And a procedure for informing information according to the calculated importance.

As described above, in the present invention, important matters of conversation can be easily recognized.

It is a figure which shows one Embodiment of the communication system which has a communication apparatus of this invention. It is a figure which shows an example of an internal structure of the communication apparatus shown in FIG. FIG. 3 is a diagram illustrating an example of determination criteria stored in advance in an emotion determination unit illustrated in FIG. 2. It is a figure which shows the other example of an internal structure of the communication apparatus shown in FIG. 3 is a flowchart for explaining an information providing method in the communication apparatus shown in FIG. 1. It is a figure which shows an example of the information which the alerting | reporting part shown in FIG. 2 alert | reported. It is a figure which shows the other example of the information which the alerting | reporting part shown in FIG. 2 alert | reported. It is a figure which shows the other form of implementation of the communication system which has a communication apparatus of this invention. It is a figure which shows an example of an internal structure of the communication apparatus shown in FIG. It is a figure which shows an example of the internal structure of the server shown in FIG. It is a sequence diagram for demonstrating the information provision method in the form shown in FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an embodiment of a communication system having a communication device of the present invention.

In this embodiment, as shown in FIG. 1, the communication device 100-1 and the communication device 100-2 can communicate with each other via the communication network 200.

The communication devices 100-1 to 100-2 are communication terminals that are used by users and can communicate with each other.

FIG. 2 is a diagram illustrating an example of an internal configuration of the communication apparatus 100-1 illustrated in FIG.

As shown in FIG. 2, the communication device 100-1 shown in FIG. 1 includes an emotion determination unit 101, a usage frequency calculation unit 102, an importance calculation unit 103, and a notification unit 104.

The emotion determination unit 101 determines the emotion of the speaker of the voice based on the input voice. This determination method uses the voice recognition function to determine the emotion of the speaker who is emitting the voice based on the strength of the voice, the height, the interval between the sounds (sounding condition), and the like. . For example, if the voice is strong (the volume is high), the speaker's emotion is considered high. Further, when the voice is high, it is considered that the emotion of the speaker is high. In addition, when the interval between sounds is short, the emotion of the speaker is considered high. In this way, each determination element and the level of emotion are associated in advance as emotion association data, and the emotion determination unit 101 determines an emotion based on the association (determination criterion). The determination method is not particularly specified here.

FIG. 3 is a diagram illustrating an example of determination criteria stored in advance in the emotion determination unit 101 illustrated in FIG.

2, the emotion determination unit 101 shown in FIG. 2 is stored in association with emotional levels, voice strength, voice level, and sound interval. In FIG. 3, the emotion level is “high”, “medium”, “low”, the voice strength is “strong”, “medium”, “weak”, and the voice level is “high”, “medium”, “Low” indicates the sound interval as “short”, “medium”, and “long”. This is an expression for convenience of explanation, and may be an appropriate numerical value representing each. Also, the number of levels is not limited to three. Further, such a determination criterion may be set in advance by the user, or may be statistically calculated based on the actual data acquired in the past.

The voice input here may be a voice during a call, or a surrounding voice (for example, voice spoken during a meeting or lecturer voice during a lecture) inputted to a microphone or the like. Also good.

The usage frequency calculation unit 102 divides the input voice into words. Here, in order for the usage frequency calculation unit 102 to divide the speech into words, the usage frequency calculation unit 102 needs to have a speech recognition function. Further, the usage frequency calculation unit 102 calculates the usage frequency for each divided word. For example, when the voice is a voice during a call, the usage frequency calculation unit 102 may calculate the number of times each divided word is used from the start of the call to the end of the call. The usage rate may be calculated. Note that the usage frequency calculation unit 102 may divide the speech into a preset unit such as a phrase or a keyword instead of a word, and perform processing for each divided unit.

The importance calculating unit 103 calculates the importance of each word based on the emotion determined by the emotion determining unit 101 and the usage frequency calculated by the usage frequency calculating unit 102. At this time, the importance calculation unit 103 sets the importance to a higher value as the emotion determined by the emotion determination unit 101 is higher. Also, the importance level calculation unit 103 sets the importance level to a higher value as the usage frequency calculated by the usage frequency calculation unit 102 is higher. Then, the importance level calculation unit 103 sets a value obtained by multiplying the respective importance levels as the importance level of the word. Note that the importance calculation unit 103 may assign a weight or the like set in advance to each importance when multiplying each importance. In other words, the importance calculated by the importance calculation unit 103 indicates the degree of interest of the speech speaker and the degree of enthusiasm for transmitting the word.

Also, the importance calculation unit 103 may calculate a score (score) corresponding to the importance calculated for each word. For example, when the importance is a high value, the importance calculation unit 103 calculates a high score of “80 points”, or when the importance is a low value, calculates a low score of “20 points”. It may be what you do.

The notification unit 104 notifies information based on the importance calculated by the importance calculation unit 103. Here, the notification method may be display of information using a display or the like, transmission of information by sound using a speaker, or notification using vibration. The information notified by the notification unit 104 may be, for example, information indicating an outline of the call content or may be a summary characterizing the call content, but the importance level calculated by the importance level calculation unit 103 is high. Information that includes words. The information notified by the notification unit 104 may simply be a word having the highest importance calculated by the importance calculation unit 103 or a predetermined number of words from the highest importance. When the importance calculation unit 103 calculates the score as described above, the notification unit 104 notifies the score.

FIG. 4 is a diagram showing another example of the internal configuration of the communication apparatus 100-1 shown in FIG.

As shown in FIG. 4, the communication device 100-1 shown in FIG. 1 includes an emotion determination unit 101, a usage frequency calculation unit 102, an importance calculation unit 103, a notification unit 104, and a recording unit 105. It has been.

The recording unit 105 records the input voice. Here, the voice recorded by the recording unit 105 may be a voice during a call, or a surrounding voice input to a microphone or the like (for example, a voice spoken during a meeting or a lecturer voice during a lecture). It may be.

The emotion determination unit 101 determines the speaker's emotion based on the voice recorded by the recording unit 105. This determination method is the same as described above.

The usage frequency calculation unit 102 divides the voice recorded by the recording unit 105 into words. Further, the usage frequency calculation unit 102 calculates the usage frequency for each divided word. The method of calculating the usage frequency is the same as described above.

The internal configuration of the communication device 100-2 may be the same as the internal configuration of the communication device 100-1, or may have a general call function.

Hereinafter, an information providing method in the communication apparatus 100-1 shown in FIG. 1 will be described.

FIG. 5 is a flowchart for explaining an information providing method in communication apparatus 100-1 shown in FIG.

First, when voice is input to the communication device 100-1 in step 1, the emotion determination unit 101 determines the emotion of the speaker who has uttered the voice in step 2. As described above, this determination method may be based on a predetermined determination criterion or may be based on statistical data.

In step 3, the usage frequency calculation unit 102 divides the input voice into words. Subsequently, in step 4, the usage frequency calculation unit 102 calculates the usage frequency of each divided word. The calculation method of the usage frequency is as described above.

Then, in step 5, the importance calculation unit 103 calculates the importance of each word based on the emotion determined by the emotion determination unit 101 and the use frequency calculated by the use frequency calculation unit 102. The importance calculation method is as described above.

Subsequently, in step 6, the notification unit 104 notifies information based on the importance calculated by the importance calculation unit 103.

FIG. 6 is a diagram showing an example of information notified by the notification unit 104 shown in FIG.

2, as shown in FIG. 6, the notification unit 104 shown in FIG. 2 displays call records and information (words) with high importance calculated by the importance calculation unit 103 as “important matters”. In addition, information related to important matters (for example, information after the second importance level) is displayed as “topics”.

FIG. 7 is a diagram showing another example of information notified by the notification unit 104 shown in FIG.

In the notification unit 104 shown in FIG. 2, as shown in FIG. 7, the score (score) calculated for the call (conversation) and the high importance information (word) calculated by the importance calculation unit 103 are “Speak”. It is displayed as "a word that the person was pleased with."

As described above, the communication apparatus 100-1 determines the emotion of the voice uttered by the speaker, calculates the use frequency, and indicates the importance calculated from each, so that the user I can recognize what I wanted to convey.

As described above, emotion determination, usage frequency calculation, and importance calculation may be performed by a device other than the communication device 100-1.

FIG. 8 is a diagram showing another embodiment of the communication system having the communication device of the present invention.

In this embodiment, as shown in FIG. 8, the communication devices 110-1 to 110-2 and the server 300 can communicate with each other via the communication network 200.

The communication devices 110-1 to 110-2 are communication devices that can be used by users and can communicate with each other.

The server 300 performs emotion determination, usage frequency calculation, and importance calculation for the voices input to the communication devices 110-1 to 110-2.

FIG. 9 is a diagram illustrating an example of an internal configuration of the communication device 110-1 illustrated in FIG.

As shown in FIG. 9, the communication device 110-1 shown in FIG. 8 includes a communication unit 111, a notification unit 114, and a recording unit 115. The internal configuration of communication device 110-2 shown in FIG. 8 is also the same.

The recording unit 115 records the input voice. Here, the voice recorded by the recording unit 115 may be a voice during a call or a surrounding voice input to a microphone or the like (for example, a voice spoken during a meeting or a lecturer voice during a lecture). It may be.

The communication unit 111 makes a call with the communication device 110-2. The communication unit 111 detects the start and end of a call. In addition, the communication unit 111 transmits audio data indicating the input audio to the server 300. Here, the voice indicated by the voice data transmitted by the communication unit 111 may be a voice during a call, or a surrounding voice input to a microphone or the like (for example, a voice spoken during a meeting or a lecturer during a lecture) Or the like. At this time, if the recording unit 115 is recording voice, the communication unit 111 transmits voice data indicating the voice recorded by the recording unit 115 to the server 300.

The notification unit 114 notifies information transmitted from the server 300. Here, the notification method may be display of information using a display or the like, transmission of information by sound using a speaker, or notification using vibration.

FIG. 10 is a diagram showing an example of the internal configuration of the server 300 shown in FIG.

As shown in FIG. 10, the server 300 shown in FIG. 8 includes an emotion determination unit 301, a usage frequency calculation unit 302, an importance level calculation unit 303, and a communication unit 304.

The emotion determination unit 301 determines the emotion of the voice speaker indicated by the voice data based on the voice data transmitted from the communication device 110-1 or the communication device 110-2. This determination method is the same as that in the emotion determination unit 101 shown in FIG.

The usage frequency calculation unit 302 divides the voice data transmitted from the communication device 110-1 or the communication device 110-2 into words. In addition, the usage frequency calculation unit 302 calculates the usage frequency for each divided word. The calculation method of the usage frequency is the same as that in the usage frequency calculation unit 102 shown in FIG.

The importance level calculation unit 303 calculates the importance level of words based on the emotion determined by the emotion determination unit 301 and the usage frequency calculated by the usage frequency calculation unit 302. The importance calculation method is the same as that in the importance calculation unit 103 shown in FIG.

The communication unit 304 transmits predetermined information to the communication devices 110-1 to 110-2 based on the importance calculated by the importance calculation unit 303. The predetermined information may be, for example, information indicating the outline of the call content, or may be a summary characterizing the call content, but the high importance word calculated by the importance calculation unit 303 is It is included information. The information transmitted by the communication unit 304 is simply information indicating the word with the highest importance calculated by the importance calculation unit 303 or information indicating a predetermined number of words from the highest importance. Also good. When the importance calculation unit 303 calculates the score as described above, the communication unit 304 transmits the score.

Hereinafter, an information providing method in the form shown in FIG. 8 will be described.

FIG. 11 is a sequence diagram for explaining the information providing method in the form shown in FIG. Here, a case will be described as an example in which communication device 110-1 records a voice during a call and transmits voice data indicating the recorded voice to server 300.

First, when the communication unit 111 of the communication device 110-1 starts a call in step 11, the recording unit 115 starts recording the voice during the call in step 12.

Thereafter, when the communication unit 111 terminates the call at step 13, the communication unit 111 transmits voice data indicating the voice recorded by the recording unit 115 to the server 300 at step 14. At this time, the communication unit 111 may transmit the audio data after compression processing.

When the communication unit 304 of the server 300 receives the audio data transmitted from the communication device 110-1, the emotion determination unit 301, based on the received audio data, in step 15, the audio data indicated by the audio data is displayed. Determine the emotion of the speaker. If the audio data transmitted from the communication device 110-1 is compressed, the emotion determination unit 301 decompresses the audio data after the communication unit 111 decompresses the audio data. Emotion is judged based on

Also, in step 16, the usage frequency calculation unit 302 divides the voice data transmitted from the communication device 110-1 into words. Subsequently, in step 17, the usage frequency calculation unit 302 calculates the usage frequency of each divided word.

Then, in step 18, the importance calculation unit 303 calculates the importance of each word based on the emotion determined by the emotion determination unit 301 and the use frequency calculated by the use frequency calculation unit 302.

When the importance calculation unit 303 calculates the importance of each word, in step 19, the communication unit 304 transmits predetermined information based on the importance calculated by the importance calculation unit 303. This information is the information as described above.

When the communication unit 111 of the communication device 110-1 receives the information transmitted from the server 300, the notification unit 114 notifies the information in step 20. The content of the notification is the same as the content notified by the notification unit 104 shown in FIG.

As described above, since a device external to the communication device 110-1 performs emotion determination, usage frequency calculation, and importance calculation, even a communication device that does not have the function of performing such processing. The present invention can be used.

As described above, according to the present invention, a conversation performed using a communication terminal (communication device) having a voice communication function is automatically determined and recorded using voice recognition. For example, you can use voice recognition technology to count the number of words used in a conversation, and give a ranking to inform you of the topic of the conversation and what you wanted to convey most after the call. . The most spoken word is considered as the most wanted word. In addition, by using the emotion judgment function based on the level of the voice, strength, weakness, and sound-to-sound (clogging), it is possible to connect the speaker's words and emotions. , You can automatically check what you have angered or what the other party was most eager to convey (what is important) and automatically check it later.

In this way, by using voice recognition, the content of the conversation is automatically judged, items that the user was not aware of, important matters that the other party wanted to convey, feelings of the other party in the conversation, The user can easily check without a work load after a call.

In addition, by using the keyword ranking obtained, the speech of the speaker is examined, and emotion evaluation data is used to score the conversation according to what percentage of the conversation is occupied by positive emotions such as “joy”. By doing this, you can realize entertainment functions. This creates new enjoyment and value for the call and increases the opportunity for voice communication.

Further, the processing performed by each component provided in each of the communication device 100-1 and the server 300 described above may be performed by a logic circuit that is produced according to the purpose. In addition, a computer program (hereinafter referred to as a program) in which processing contents are described as a procedure is recorded on a recording medium readable by each of the communication device 100-1 and the server 300, and the program recorded on the recording medium is stored in the communication device. 100-1 and server 300 may be read and executed. The recording medium readable by each of the communication device 100-1 and the server 300 includes a transferable recording medium such as a floppy (registered trademark) disk, a magneto-optical disk, a DVD, and a CD, as well as the communication device 100-1 and the server. Reference numeral 300 denotes a memory such as a ROM or a RAM, an HDD, or the like built in each of 300. The program recorded on the recording medium is read by a CPU (not shown) provided in each of the communication device 100-1 and the server 300, and the same processing as described above is performed under the control of the CPU. Here, the CPU operates as a computer that executes a program read from a recording medium on which the program is recorded.

Note that the functions of the communication device 100-1 and the server 300 described above include an IC (Integrated Circuit) recorder, a mobile phone, a mobile terminal, a tablet or notebook PC (Personal Computer), a smartphone, a PDA (Personal Digital Assistants). ), And can be applied to devices such as game machines.

A part or all of the above embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Supplementary note 1) An emotion determination unit that determines an emotion of a speaker of the voice based on the input voice;
A usage frequency calculation unit that divides the voice into words and calculates a usage frequency for each word;
An importance calculator that calculates the importance of the word based on the emotion determined by the emotion determiner and the use frequency calculated by the use frequency calculator;
A communication apparatus comprising: a notification unit that notifies information according to the importance calculated by the importance calculation unit.
(Additional remark 2) It has a recording part which records the said input voice,
The emotion determination unit determines the emotion of the speaker based on the voice recorded by the recording unit,
The communication apparatus according to appendix 1, wherein the usage frequency calculation unit divides the voice recorded by the recording unit into words.
(Supplementary note 3) The communication apparatus according to Supplementary note 1 or 2, wherein the voice is a voice during a call.
(Additional remark 4) The said importance calculation part calculates the score according to the said importance about the said word,
The communication device according to any one of appendices 1 to 3, wherein the notification unit notifies the score.
(Additional remark 5) The said alerting | reporting part alert | reports the word with the highest said importance calculated by the said importance calculation part, The communication apparatus of any one of Additional remark 1 to 4 characterized by the above-mentioned.
(Additional remark 6) Based on the audio | voice data transmitted from the communication apparatus, the emotion determination part which determines the emotion of the speaker of the audio | voice which this audio | voice data shows,
A usage frequency calculation unit that divides the voice data into words and calculates a usage frequency for each word;
An importance calculator that calculates the importance of the word based on the emotion determined by the emotion determiner and the use frequency calculated by the use frequency calculator;
And a communication unit that transmits information corresponding to the importance calculated by the importance calculation unit to the communication device.
(Additional remark 6) Based on the input audio | voice, the process which determines the emotion of the speaker of the said audio | voice,
Processing to divide the voice into words;
A process of calculating a use frequency for each of the divided words;
Processing to calculate the importance of the word based on the emotion and the usage frequency;
The information provision method which performs the process which alert | reports the information according to the calculated said importance.
(Appendix 7)
A procedure for determining an emotion of a speaker of the voice based on the inputted voice;
Dividing the speech into words;
A procedure for calculating a use frequency for each of the divided words;
Calculating the importance of the word based on the emotion and the usage frequency;
The program for performing the procedure which alert | reports the information according to the calculated said importance.

As mentioned above, although this invention was demonstrated with reference to embodiment, this invention is not limited to the said embodiment. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2012-118699 filed on May 24, 2012, the entire disclosure of which is incorporated herein.

Claims

An emotion determination unit that determines an emotion of a speaker of the voice based on the input voice;
A usage frequency calculation unit that divides the voice into words and calculates a usage frequency for each word;
An importance calculator that calculates the importance of the word based on the emotion determined by the emotion determiner and the use frequency calculated by the use frequency calculator;
A communication apparatus comprising: a notification unit that notifies information according to the importance calculated by the importance calculation unit.
The communication device according to claim 1,
A recording unit for recording the input voice;
The emotion determination unit determines the emotion of the speaker based on the voice recorded by the recording unit,
The communication device according to claim 1, wherein the usage frequency calculation unit divides the voice recorded by the recording unit into words.
The communication device according to claim 1 or 2,
The communication apparatus according to claim 1, wherein the voice is a voice during a call.
The communication apparatus according to any one of claims 1 to 3,
The importance calculation unit calculates a score corresponding to the importance for the word,
The communication unit is configured to notify the score.
The communication device according to any one of claims 1 to 4,
The communication unit is configured to notify a word having the highest importance calculated by the importance calculation unit.
An emotion determination unit that determines the emotion of the speaker of the voice represented by the voice data based on the voice data transmitted from the communication device;
A usage frequency calculation unit that divides the voice data into words and calculates a usage frequency for each word;
An importance calculator that calculates the importance of the word based on the emotion determined by the emotion determiner and the use frequency calculated by the use frequency calculator;
And a communication unit that transmits information corresponding to the importance calculated by the importance calculation unit to the communication device.
A process of determining the emotion of the speaker of the voice based on the input voice;
Processing to divide the voice into words;
A process of calculating a use frequency for each of the divided words;
Processing to calculate the importance of the word based on the emotion and the usage frequency;
The information provision method which performs the process which alert | reports the information according to the calculated said importance.
On the computer,
A procedure for determining an emotion of a speaker of the voice based on the inputted voice;
Dividing the speech into words;
A procedure for calculating a use frequency for each of the divided words;
Calculating the importance of the word based on the emotion and the usage frequency;
The program for performing the procedure which alert | reports the information according to the calculated said importance.