JP6746963B2 - Conversation evaluation device, program, and conversation evaluation method - Google Patents

Description

The present invention relates to a technique for evaluating voice conversation.

Conventionally, a technique for evaluating various matters by analyzing a speech voice has been proposed. For example, Patent Document 1 discloses a technique of estimating the psychological or physiological state of a speaker from the interval of fundamental tones in the pitch sequence of the spoken voice.

Japanese Patent No. 4495907

However, the technique of Patent Document 1 merely estimates the state of a specific speaker, and objectively evaluates, for example, a voice conversation between a plurality of speakers (for example, an impression of a voice as a response to an utterance). I can't. In consideration of the above circumstances, the present invention aims to objectively evaluate a voice conversation.

In order to solve the above problems, the conversation evaluation apparatus according to the first aspect of the present invention generates a feature acquisition unit that acquires a feature amount of a voice forming a conversation and a related information that is different from the feature amount regarding the conversation. And a conversation evaluation unit that evaluates the conversation according to the feature amount and the related information. In the above aspect, it is possible to objectively evaluate the conversation according to the feature amount of the voice that constitutes the conversation. In addition, since the related information different from the feature amount is added to the evaluation of the conversation together with the feature amount, it is possible to appropriately evaluate the conversation as compared with the configuration in which only the feature amount is reflected in the evaluation.

In a preferred aspect of the present invention, the characteristic acquisition unit acquires the characteristic amount under a condition according to the related information. In the above aspect, since the characteristic amount is acquired under the condition according to the related information, there is an advantage that the characteristic amount can be appropriately acquired as compared with the configuration in which the related information is not used for acquiring the characteristic amount.

A conversation evaluation apparatus according to a second aspect of the present invention includes a feature acquisition unit that acquires a feature amount of a voice forming a conversation, an information generation unit that generates related information of a conversation different from the feature amount, and a feature amount. And a conversation evaluation unit that evaluates the conversation accordingly, and the feature acquisition unit acquires the feature amount under a condition according to the related information. In the above aspect, it is possible to objectively evaluate the conversation according to the feature amount of the voice that constitutes the conversation. Further, since the characteristic amount is acquired under the condition according to the related information, there is an advantage that the characteristic amount can be appropriately acquired as compared with the configuration in which the related information is not used for acquiring the characteristic amount.

In a preferred example of the conversation evaluation device according to each of the above-described aspects, the feature acquisition unit acquires the pitch of each of the first voice and the second voice that form a conversation as a feature amount, and the conversation evaluation unit uses the first voice. The conversation is evaluated according to the pitch difference between the second voice and the second voice. In the above aspect, since the conversation is evaluated according to the pitch difference between the first voice and the second voice, the quality of the impression of the response voice is judged from the viewpoint of the relationship between the pitch of the utterance voice and the pitch of the response voice. It can be evaluated objectively.

In each of the aforementioned aspects, the relevant information is, for example, at least the temporal situation of the conversation, the history of past conversations between the speakers of the conversation, the relationship between the speakers of the conversation, and the attributes of each speaker of the conversation. This is one piece of information.

It is a block diagram of the conversation evaluation apparatus of 1st Embodiment. It is a flow chart of conversation evaluation processing. It is a block diagram of the conversation evaluation apparatus of 2nd Embodiment. It is a block diagram of the conversation evaluation apparatus of 3rd Embodiment. It is a block diagram of the conversation evaluation apparatus of 4th Embodiment.

<First Embodiment>
FIG. 1 is a configuration diagram of a conversation evaluation device 100 according to the first embodiment of the present invention. The conversation evaluation device 100 according to the first embodiment is an analysis device that evaluates a conversation between the user U1 and the user U2, and is preferably used for training a good conversation, for example. A conversation is composed of the voice V1 (an example of the first voice) pronounced by the user U1 and the voice V2 (an example of the second voice) pronounced by the user U2.

In the first embodiment, for example, the user U1 pronounces a voice V1 of an utterance including an inquiry and a talk, and a voice V2 of a response including an answer to the inquiry from the user U1 and an answer to the talk from the user U1 to the user. Suppose U2 is pronounced. The voice V2 pronounced by the user U2 is, for example, a voice meaning an interjection. For example, "yes", "yes", etc., stagnation such as "e-to", "that-", etc. (stagnation in response), "yes", "no", etc. answers (affirmation/denial to the question), Words such as “oh” and “oh” that express the emotion of the speaker, or words and phrases such as “e?” and “what?” that mean a question (repeat) to the utterance can be exemplified as the interjection.

As illustrated in FIG. 1, the conversation evaluation device 100 according to the first embodiment is a computer system including a control device 12, a storage device 14, a display device 16, an input device 18, a sound collecting device 22, and a sound collecting device 24. Will be realized in. For example, the conversation evaluation device 100 can be realized by a portable information processing device such as a mobile phone or a smartphone or an information processing device such as a personal computer. It is also possible to realize the conversation evaluation device 100 by a plurality of devices configured separately from each other.

The sound pickup device 22 and the sound pickup device 24 are voice input devices that pick up ambient sounds. The sound collecting device 22 generates a sound signal X1 representing the sound V1 sounded by the user U1, and the sound collecting device 24 generates a sound signal X2 representing the sound V2 sounded by the user U2. The A/D converter for converting each of the audio signal X1 and the audio signal X2 from analog to digital is omitted for convenience of illustration.

The control device 12 is configured to include a processing circuit such as a CPU (Central Processing Unit), and centrally controls each element of the conversation evaluation device 100. Specifically, the control device 12 analyzes the voice signal X1 generated by the sound collecting device 22 and the voice signal X2 generated by the sound collecting device 24 to evaluate the conversation between the user U1 and the user U2. To do. The control device 12 of the first embodiment calculates an index (hereinafter, referred to as “evaluation value”) S of impression of the response of the user U2 to the utterance of the user U1.

The display device 16 (for example, a liquid crystal display panel) displays various images under the control of the control device 12. For example, the evaluation result (evaluation value S) of the conversation between the users U1 and U2 is displayed on the display device 16. The input device 18 receives an instruction from the user U (for example, the user U1 or the user U2) to the conversation evaluation device 100. For example, a plurality of operators operated by the user U (U1, U2) and a touch panel that detects contact with the display surface of the display device 16 are preferably used as the input device 18.

The storage device 14 stores programs executed by the control device 12 and various data used by the control device 12. For example, a known recording medium such as a semiconductor recording medium or a magnetic recording medium, or a combination of a plurality of recording media may be arbitrarily adopted as the storage device 14. The control device 12 of the first embodiment executes a program stored in the storage device 14 to execute a plurality of functions for evaluating the conversation between the user U1 and the user U2 (feature acquisition unit 32, information generation). The unit 34 and the conversation evaluation unit 36) are realized. A configuration in which the functions of the control device 12 are distributed to a plurality of devices, or a configuration in which a dedicated electronic circuit realizes some or all of the functions of the control device 12 may be employed.

The characteristic acquisition unit 32 acquires the characteristic amount of the voice V1 of the user U1 and the characteristic amount of the voice V2 of the user U2. The feature acquisition unit 32 of the first embodiment extracts the feature amount of the voice V1 of the user U1 by analyzing the voice signal X1, and extracts the feature amount of the voice V2 of the user U2 by analyzing the voice signal X2. Specifically, the prosodic feature amount is extracted for each of the voice V1 and the voice V2. Prosody is a linguistic and phonetic characteristic that can be perceived by a listener, and means a property that cannot be understood only from general notation of language.

The feature acquisition unit 32 of the first embodiment extracts the pitch P1 of the voice V1 of the user U1 and the pitch P2 of the voice V2 of the user U2 as feature amounts. For example, the feature acquisition unit 32 extracts the average pitch P1 in the utterance section of the voice signal X1 and the average pitch P2 in the utterance section of the voice signal X2. The utterance section is a section in which utterance continues (a section from a start point to an end point of a series of utterances). A known voice analysis technique may be arbitrarily adopted to extract the pitch P1 and the pitch P2.

When the user U2 responds with the voice V2 of the pitch P2 having a specific relationship to the pitch P1 of the voice V1 spoken by the user U1, the user U1 can comfortably hear the voice V2 of the user U2. There is a tendency to perceive that the response is safe and has a good impression. Specifically, when the user U2 pronounces a pitch P2 that is in a harmony relationship with the pitch P1 of the user U1, the response of the user U2 is perceived as a good impression. Further, the impression of the response by the user U2 is particularly greatly influenced by the end portion of the utterance section of the voice V1 by the user U1 near the start point of the utterance section of the voice V2. Therefore, the feature acquisition unit 32 of the first embodiment specifies the pitch P1 of a section of a predetermined length (for example, 180 msec) located at the end of the utterance section of the voice V1 of the user U1.

The information generation unit 34 in FIG. 1 generates information (hereinafter referred to as “related information”) R different from the feature amount extracted by the feature acquisition unit 32, regarding the conversation between the users U1 and U2. The related information R is information related to the conversation between the users U1 and U2. In the first embodiment, the related information R indicating the temporal situation of conversation is illustrated. Specifically, the information generation unit 34 generates related information R indicating the conversation date and time (for example, date and time) and the conversation duration (for example, elapsed time from the start of conversation) as the conversation situation. For example, the information generator 34 identifies the conversation date and time and the conversation duration by referring to the time measured by a timing circuit (not shown). That is, for example, the date and time of the start point of the latest utterance section in the voice V1 or voice V2 is specified as the conversation date and time, and the elapsed time from the start time of the earliest utterance section in the voice V1 or voice V2 to the current time is the conversation duration. Specified.

The conversation evaluation unit 36 conducts a conversation between the users U1 and U2 according to the feature amount (pitch P1, pitch P2) extracted by the feature acquisition unit 32 and the related information R generated by the information generation unit 34. evaluate. That is, the conversation evaluation unit 36 calculates the evaluation value S according to the pitch P1 and the pitch P2 and the related information R. As can be understood from the above description, in the first embodiment, not only the feature amount of the voice (V1, V2) forming the conversation but also the related information R other than the feature amount is taken into consideration in the conversation evaluation. The evaluation value S calculated by the conversation evaluation unit 36 is displayed on the display device 16.

As described above, when the user U2 responds with the voice V2 of the pitch P2 that is in a harmony with the pitch P1 of the voice V1 of the user U1, the user U1 tends to feel a good impression. is there. In consideration of the above tendency, the conversation evaluation unit 36 of the first embodiment calculates the evaluation value S according to the pitch difference ΔP (ΔP=|P1-P2|) between the pitch P1 and the pitch P2. .. Specifically, the conversation evaluation unit 36 calculates the evaluation value S such that the evaluation value S has a larger value as the pitch difference ΔP is closer to the consonance relation. The consonance relationship illustrated in the first embodiment is, for example, a pitch relationship in which the frequency ratio is close to an integer ratio (for example, perfect once, perfect eighth, perfect fifth, and perfect fourth).

As described above, the conversation evaluation unit 36 according to the first embodiment reflects the pitch P1 and the pitch P2 in the evaluation value S, and also includes the related information R regarding the conversation between the users U1 and U2. Then, the evaluation value S is calculated. A specific relationship between the related information R (conversation date and time, conversation duration) and the evaluation value S is illustrated below.

Since it is highly likely that conversations at night or on holidays are between intimate friends, compared to, for example, weekday daytime conversations (typically business conversations), user U1 U2 tends to have a good impression. In consideration of the above tendency, when the conversation date and time specified by the related information R corresponds to nighttime or holiday, the evaluation value S is larger than that when the conversation date and time corresponds to daytime on weekdays. As described above, the conversation evaluation unit 36 calculates the evaluation value S. For example, the conversation evaluation unit 36 adds a predetermined value to the evaluation value S when the conversation date/time corresponds to nighttime or holiday.

Further, when the conversation continues for a long time, there is a high possibility that the conversation between the user U1 and the user U2 will be lively, feeling the good impressions of each other. In consideration of the above tendency, the conversation evaluation unit 36 calculates the evaluation value S such that the evaluation value S becomes a larger value as the conversation continuation length designated by the related information R becomes longer. For example, the conversation evaluation unit 36 adds a predetermined value to the evaluation value S when the conversation duration exceeds a predetermined threshold. On the other hand, when the conversation lasts for an excessively long time, the mutual impression may be deteriorated due to the fatigue of the users U1 and U2. Considering the above tendency, it is possible to deduct the evaluation value S when the conversation duration designated by the related information R exceeds a predetermined threshold. It is also possible to use a plurality of different thresholds. For example, the first threshold value and the second threshold value are set (first threshold value<second threshold value), and when the conversation duration is a numerical value between the first threshold value and the second threshold value, the evaluation value S is added. It is assumed that the evaluation value S is deducted when the conversation duration is below the first threshold or above the second threshold. It is also possible to change the point addition value or the point deduction value for the evaluation value S step by step for each range with each threshold as the boundary.

FIG. 2 is a flowchart of a process in which the control device 12 of the first embodiment evaluates a conversation between the users U1 and U2 (hereinafter referred to as "conversation evaluation process"). For example, the conversation evaluation process is started in response to an instruction from the user U (U1, U2) to the input device 18 or the start of the utterance by the user U.

When the conversation evaluation process of FIG. 2 is started, the feature acquisition unit 32 sequentially analyzes the pitch P1 of the voice V1 of the user U1 and the pitch P2 of the voice V2 of the user U2 by analyzing the voice signal X1 and the voice signal X2. (SA1). Further, the information generating unit 34 generates the related information R (the conversation date and time and the conversation duration in the first embodiment) with reference to the time measured by the clock circuit, for example (SA2). The conversation evaluation unit 36 calculates an evaluation value S according to the feature amount (pitch P1, pitch P2) extracted by the feature acquisition unit 32 and the related information R generated by the information generation unit 34 (SA3). Note that the extraction of the characteristic amount by the characteristic acquisition unit 32 (SA1) and the generation of the related information R by the information generation unit 34 (SA2) can be reversed.

As illustrated above, in the first embodiment, it is possible to objectively evaluate the conversation between the users U1 and U2 according to the feature amounts of the voice V1 and the voice V2 that constitute the conversation. .. Further, the conversation is evaluated by considering the related information R of the conversation in addition to the feature amounts of the voice V1 and the voice V2 that make up the conversation. Therefore, the conversation is compared with the configuration in which only the feature amount is reflected in the evaluation result. It is possible to evaluate appropriately. Particularly in the first embodiment, since the conversation is evaluated according to the pitch difference ΔP between the pitch P1 of the voice V1 and the pitch P2 of the voice V2, the voice V2 of the user U2 with respect to the voice V1 of the user U1 From the viewpoint of pitch (that is, pitch difference), it is possible to objectively evaluate the quality of the impression of the response by the user U2.

<Second Embodiment>
A second embodiment of the present invention will be described. It should be noted that in each of the following exemplary embodiments, the elements having the same functions and functions as those in the first embodiment are assigned the reference numerals used in the description of the first embodiment, and the detailed description thereof will be appropriately omitted.

FIG. 3 is a configuration diagram of the conversation evaluation device 100 according to the second embodiment. As illustrated in FIG. 3, the storage device 14 of the conversation evaluation device 100 according to the second embodiment stores the history information H for each combination of the users U. The history information H is information relating to the history of past conversations (conversation history) between the users U. Specifically, the history information H of the second embodiment is the frequency of conversations conducted in the past between the users U (hereinafter referred to as “conversation frequency”) and the elapsed time from the first conversation between the users U. (Hereinafter referred to as “relationship period”). The conversation frequency means the number of conversations within a predetermined length of time (for example, one month). The conversation frequency of the related information R is updated for each conversation between the users U. The conversation frequency and the related period can be paraphrased as an index of the degree of intimacy between the users U.

The information generator 34 of the second embodiment generates the related information R by referring to the history information H stored in the storage device 14. For example, the user U1 and the user U2 instruct the conversation evaluation device 100 about their own identification information by appropriately operating the input device 18. The information generation unit 34 searches the storage device 14 for the history information H between the users U1 and U2 indicated by the identification information, and the related information R including the conversation frequency and the related period designated by the history information H. To generate. The operation of the feature acquisition unit 32 to extract the pitch P1 of the voice V1 and the pitch P2 of the voice V2 is the same as in the first embodiment.

When the conversation frequency is high, the user U1 and the user U2 have a good relationship, and there is a high possibility that they are talking while maintaining a good impression of each other. In consideration of the above tendency, the conversation evaluation unit 36 calculates the evaluation value S so that the evaluation value S becomes a larger value as the conversation frequency designated by the related information R becomes higher. For example, the conversation evaluation unit 36 adds a predetermined value to the evaluation value S when the conversation frequency exceeds a predetermined threshold. It is also possible to use a plurality of different thresholds. For example, when the conversation frequency is a numerical value between the first threshold value and the second threshold value (first threshold value<second threshold value), the evaluation value S is added, while when the conversation frequency is less than the first threshold value or the second threshold value. It is assumed that the evaluation value S is deducted when the threshold value is exceeded. It is also possible to change the point addition value or the point deduction value for the evaluation value S step by step for each range with each threshold as the boundary.

Also, when the relationship period from the first conversation is long, similarly, the user U1 and the user U2 have a good relationship, and there is a high possibility that they are talking while maintaining a good impression with each other. In consideration of the above tendency, the conversation evaluation unit 36 calculates the evaluation value S such that the evaluation value S becomes a larger numerical value as the relationship period designated by the related information R is longer. For example, the conversation evaluation unit 36 adds a predetermined value to the evaluation value S when the relationship period exceeds a predetermined threshold value. It is also possible to use a plurality of different thresholds. For example, the evaluation value S is added when the relationship period is a numerical value between the first threshold value and the second threshold value (first threshold value<second threshold value), while the relationship period is less than the first threshold value or the second threshold value. It is assumed that the evaluation value S is deducted when the threshold value is exceeded. It is also possible to change the point addition value or the point deduction value for the evaluation value S step by step for each range with each threshold as the boundary.

As described above, in the second embodiment as well, the conversation is evaluated in consideration of the related information R of the conversation in addition to the feature amounts of the voice V1 and the voice V2 forming the conversation. Therefore, as in the first embodiment, it is possible to appropriately evaluate the conversation as compared with the configuration in which only the feature amount is reflected in the evaluation. Particularly in the second embodiment, since the conversation history between the users U (for example, the conversation frequency and the related period) is used as the related information R, an appropriate evaluation based on the past tendency of conversation between the users U is realized. To be done.

In the above description, the identification information instructed by the operation on the input device 18 is used, but the method of identifying the user U1 and the user U2 is arbitrary. For example, the user U1 is identified by the speaker identification for the voice signal X1, the user U2 is identified by the speaker identification for the voice signal X2, and the history information H between the users U1 and U2 is searched. Is also possible. A known recognition technique can be arbitrarily adopted for the speaker identification of the user U1 and the user U2.

<Third Embodiment>
FIG. 4 is a configuration diagram of the conversation evaluation device 100 according to the third embodiment. As illustrated in FIG. 4, the storage device 14 of the conversation evaluation device 100 of the third embodiment stores the speaker information Q for each combination of the users U. The speaker information Q is information indicating the relationship between the users U. Specifically, the speaker information Q of the third embodiment specifies the mutual relationship (friend, family, acquaintance, colleague, etc.) between the users U and the intimacy degree between the users U. The mutual relationship and familiarity can be specifically instructed by the operation of the user U on the input device 18, but can be reflected in the speaker information Q from information registered in, for example, SNS (Social Networking Service). Is.

The information generator 34 of the third embodiment generates the related information R by referring to the speaker information Q stored in the storage device 14. For example, the information generation unit 34 searches the storage device 14 for the speaker information Q between the users U1 and U2 identified by the input of the identification information or the speaker identification as in the second embodiment, and The related information R including the mutual relationship and familiarity designated by the speaker information Q is generated. The operation of the feature acquisition unit 32 to extract the pitch P1 of the voice V1 and the pitch P2 of the voice V2 is the same as in the first embodiment.

When the mutual relationship is a friend, the user U1 and the user U2 have a good relationship, and there is a high possibility that they are talking while maintaining a good impression of each other. In consideration of the above tendency, when the mutual relationship specified by the related information R is a friend, the conversation is performed so that the evaluation value S becomes a larger numerical value than in the case where the mutual relationship is another relationship. The evaluation unit 36 calculates the evaluation value S. Specifically, the conversation evaluation unit 36 adds a predetermined value to the evaluation value S when the mutual relationship between the user U1 and the user U2 is a friend.

Similarly, when the degree of intimacy is high, the user U1 and the user U2 also have a good relationship, and there is a high possibility that they are talking while maintaining a good impression of each other. In consideration of the above tendency, the conversation evaluation unit 36 calculates the evaluation value S such that the evaluation value S becomes a larger value as the degree of intimacy specified by the related information R becomes higher. For example, the conversation evaluation unit 36 adds a predetermined value to the evaluation value S when the degree of intimacy exceeds a predetermined threshold. It is also possible to use a plurality of different thresholds. For example, when the degree of intimacy is a numerical value between the first threshold value and the second threshold value (first threshold value<second threshold value), the evaluation value S is added, while when the intimacy degree is less than the first threshold value or the second threshold value, It is assumed that the evaluation value S is deducted when the threshold value is exceeded. It is also possible to change the point addition value or the point deduction value for the evaluation value S step by step for each range with each threshold as the boundary.

As illustrated above, in the third embodiment as well, the conversation is evaluated in consideration of the related information R of the conversation in addition to the feature amounts of the voice V1 and the voice V2 that form the conversation. Therefore, as in the first embodiment, it is possible to appropriately evaluate the conversation as compared with the configuration in which only the feature amount is reflected in the evaluation. In the third embodiment, in particular, the speaker information Q (for example, mutual relationship and intimacy) between the users U is used as the related information R, so that an appropriate evaluation based on the actual relationship between the users U is realized. To be done.

<Fourth Embodiment>
FIG. 5: is a block diagram of the conversation evaluation apparatus 100 of 4th Embodiment. As illustrated in FIG. 5, the storage device 14 of the conversation evaluation device 100 according to the fourth embodiment stores the attribute information A for each user U. The attribute information A is information indicating the attribute (feature or property) of the user U. Information that depends on the voice pronounced by the user U is particularly suitable as the attribute information A. The attribute information A of the fourth embodiment specifies the sounding frequency of the user U. The sounding frequency is an average pitch (average pitch) of the sound produced by the user U.

The information generation unit 34 of the fourth embodiment searches the storage device 14 for the attribute information A of each of the user U1 and the user U2 identified by the input of the identification information and the speaker identification as in the second embodiment. , And generates related information R including the sounding frequency specified by each attribute information A. That is, the related information R of the fourth embodiment is information of each user U who conducts the conversation of the evaluation target, and like the related information R illustrated in the first to third embodiments, the characteristic acquisition unit. 32 is an example of information of a type different from the feature amount extracted by 32.

In the first to third embodiments, the configuration in which the related information R generated by the information generation unit 34 is reflected in the conversation evaluation by the conversation evaluation unit 36 is illustrated, but in the fourth embodiment, the feature acquisition unit 32 is used. The related information R is reflected in the extraction of the feature amount. That is, the characteristic acquisition unit 32 of the fourth embodiment extracts the characteristic amount under the condition according to the related information R generated by the information generation unit 34.

Specifically, the feature acquisition unit 32 extracts, from the audio signal X1 generated by the sound collection device 22, an acoustic component within a predetermined band including the sounding frequency of the user U1 designated by the related information R, and after extraction. The pitch P1 is specified from the sound component of. That is, the pitch P1 is specified only in the tone range in which the user U1 normally sounds. Similarly, the feature acquisition unit 32 specifies the pitch P2 from the acoustic component within the predetermined band including the sounding frequency of the user U2 designated by the related information R in the sound signal X2 generated by the sound collection device 24. It is also possible to use the attribute information A that specifies the pronunciation range of the user U as the related information R.

The conversation evaluation unit 36 evaluates the conversation between the users U1 and U2 according to the pitch P1 and the pitch P2 identified by the feature acquisition unit 32 using the related information R as described above. Specifically, the conversation evaluation unit 36 calculates the evaluation value S according to the pitch difference ΔP between the pitch P1 and the pitch P2. The related information R is not added to the evaluation by the conversation evaluation unit 36 in the fourth embodiment. However, similarly to the exemplification of the first to third embodiments, it is possible to add the related information R to the evaluation by the conversation evaluation unit 36 also in the fourth embodiment.

As illustrated above, also in the fourth embodiment, as in the first embodiment, the conversation between the users U1 and U2 is objectively determined according to the feature amounts of the voice V1 and the voice V2 that constitute the conversation. It is possible to evaluate. In addition, in the fourth embodiment, the feature quantity (pitch P1, pitch P2) is extracted under the condition according to the related information R regarding the conversation, so that a comparison is made with the configuration in which the related information R is not used for extracting the feature quantity. Therefore, there is an advantage that the feature amount can be appropriately extracted. For example, in the fourth embodiment, it is possible to highly accurately extract the feature amount by limiting the frequency band corresponding to the sounding frequency designated by the related information R.

In the above example, the pronunciation frequency of the user U is illustrated as the attribute information A, but the content of the attribute information A is not limited to the above example. For example, it is possible to use the attribute information A that specifies the gender of the user U. The feature acquisition unit 32 specifies the pitch P within the frequency band assumed for the sex designated by the related information R. For example, when the gender of the user U1 designated by the related information R is female, the feature acquisition unit 32 extracts the pitch P1 from the acoustic component of the high frequency range assumed for the female in the audio signal X1, When the gender of the user U1 is male, the pitch P1 is extracted from the acoustic component in the low range assumed by the male in the audio signal X1.

When the pitch difference between the voice V1 of the user U1 and the voice V2 of the user U2 exceeds one octave, one of the pitch P1 and the pitch P2 is brought closer to the other by an integral multiple of one octave. Therefore, it is preferable that the pitch difference between the two is corrected within one octave (hereinafter referred to as "pitch correction"). When the sexes of the user U1 and the user U2 are different (that is, the pitch difference is large), it is estimated that the pitch correction is highly necessary. Considering the above tendency, when the genders specified by the related information R are different between the user U1 and the user U2, the feature acquisition unit 32 performs pitch correction, and the user U1 and the user U2. A configuration in which the feature acquisition unit 32 omits pitch correction when the genders are common is also suitable.

<Fifth Embodiment>
Like the third embodiment, the information generation unit 34 of the fifth embodiment refers to the speaker information Q stored in the storage device 14 to determine the intimacy degree between the user U1 and the user U2. The related information R designated is generated. Similar to the fourth embodiment, the characteristic acquisition unit 32, under the condition according to the related information R generated by the information generation unit 34, each characteristic amount (pitch of the voice V1 of the user U1 and the voice V2 of the user U2). P1 and pitch P2) are extracted. Specifically, the characteristic acquisition unit 32 extracts the characteristic amount at a frequency according to the degree of intimacy specified by the related information R.

For example, when the degree of intimacy is high, since the user U1 and the user U2 have a good relationship, the evaluation value S is expected to be a relatively large numerical value. When the degree of intimacy is low, it is difficult to predict the magnitude of the evaluation value S. Therefore, it is assumed that there is a low need for frequent conversation evaluation when the degree of intimacy is high, and a frequent need for evaluating the conversation when the degree of intimacy is low. In consideration of the above tendency, the feature acquisition unit 32 of the fifth embodiment lowers the frequency of extracting the feature amount (pitch P1, pitch P2) as the degree of intimacy specified by the related information R is higher. ..

Specifically, the feature acquisition unit 32 extracts the feature amount at a lower frequency when the degree of intimacy exceeds a predetermined threshold than when the degree of intimacy falls below the threshold. For example, when the degree of intimacy is lower than the threshold value, it is set for each set of the speech sections of the voice V1 and the voice V2 that follow each other (that is, once for each set of the utterance by the user U1 and the response by the user U2. Pitch P1 and pitch P2 are extracted (in frequency). On the other hand, when the degree of intimacy exceeds the threshold value, the frequency is once for each of a plurality of sets of speech sections of the voice V1 and the voice V2 (that is, once for a plurality of times of the utterance by the user U1 and the response by the user U2. ) The pitch P1 and the pitch P2 are extracted. Since the evaluation of the conversation by the conversation evaluation unit 36 is performed every time the feature amount is extracted by the feature acquisition unit 32, the higher the degree of intimacy specified by the related information R, the more frequently the conversation evaluation unit 36 evaluates the frequency (and further the display). The frequency of updating the evaluation value S displayed on the device 16 decreases. It is also possible to use a plurality of different thresholds. For example, a configuration in which a frequency is set for each range having each of a plurality of threshold values as boundaries and the feature acquisition unit 32 extracts the feature amount at a frequency corresponding to a range to which the intimacy degree belongs among the plurality of ranges is assumed.

As illustrated above, also in the fifth embodiment, similar to the first embodiment, the conversation between the user U1 and the user U2 is objectively objective according to the feature amounts of the voice V1 and the voice V2 forming the conversation. It is possible to evaluate. Further, in the fifth embodiment, since the feature quantity (pitch P1, pitch P2) is extracted under the condition according to the related information R, the related information R is extracted in the extraction of the feature quantity as in the fourth embodiment. There is an advantage that the feature amount can be appropriately extracted as compared with the configuration that is not used. For example, in the fifth embodiment, the frequency of extraction of the feature quantity is controlled according to the related information R, so that the feature quantity obtained by the feature acquisition unit 32 can be compared with the configuration in which the related information R is not used to extract the feature quantity. It is possible to reduce the amount of calculation required for the extraction and the conversation evaluation by the conversation evaluation unit 36.

In the example of the fifth embodiment, the feature amount extraction condition (specifically, the frequency) is controlled according to the degree of intimacy between the users U, but the content of the related information R to be reflected in the feature amount extraction condition. Is not limited to the above examples. For example, it is possible to apply the arbitrary related information R exemplified in the first to third embodiments to the control of the feature quantity extraction condition. For example, a configuration in which the feature amount extraction condition is controlled according to the conversation frequency and the relation period illustrated in the second embodiment (for example, the higher the conversation frequency or the longer the relation period is, the more frequently the feature amount is extracted. It is also envisioned that the configuration).

Further, the configuration of the first to third embodiments in which the related information R is added to the evaluation of the conversation by the conversation evaluation unit 36, and the feature quantity extraction condition by the characteristic acquisition unit 32 are controlled according to the related information R. It is also possible to combine the configurations of the fourth embodiment and the fifth embodiment. The different types of related information R can be preferably applied to the evaluation of the conversation and the control of the feature amount extraction condition by the conversation evaluation unit 36, but the related information R can also be commonly applied. For example, in the configuration in which the related information R includes the conversation frequency, the extraction frequency of the feature amount by the feature acquisition unit 32 is controlled according to the conversation frequency, and the conversation evaluation unit 36 evaluates the conversation as illustrated in the second embodiment. It is also possible to use the conversation frequency.

<Modification>
Each aspect illustrated above can be variously modified. Specific modes of modification will be exemplified below. Two or more aspects arbitrarily selected from the following exemplifications can be appropriately merged within a range not inconsistent with each other.

(1) In each of the above-described modes, the conversation composed of the voice V1 pronounced by the user U1 and the voice V2 pronounced by the user U2 was evaluated, but the voice evaluated by the conversation evaluation device 100 is the user U. Is not limited to the vocal sound (that is, the real voice). Specifically, one of the voice V1 and the voice V2 can be a synthesized voice generated by a known voice synthesis technique. For example, the same configuration as each of the above-described embodiments may be adopted for the evaluation of the conversation composed of the voice V1 pronounced by the user U1 and the voice V2 generated by the voice synthesis. That is, by analyzing the utterance content by voice recognition of the voice U1 of the user U1, a voice V2 of an appropriate response to the utterance of the user U1 is generated. It is also possible to selectively select a plurality of prerecorded voices as the voice V2. Further, a configuration for evaluating a conversation composed of a voice V1 generated by voice synthesis and a voice V2 pronounced by the user U2, and a conversation composed of voice V1 and voice V2 generated by voice synthesis are evaluated. Configurations may also be employed.

As described above, in the configuration in which the voice V1 and the voice V2 are generated by voice synthesis, the sound collecting device 22 and the sound collecting device 24 are omitted. Further, in the configuration using the synthesized voice, the feature acquisition unit 32 may also acquire a voice synthesis parameter that specifies the acoustic characteristic of the voice (for example, pitch or volume) as the feature amount of the voice V1 or the voice V2. It is possible. With the above configuration, the process of extracting the feature amount of the voice V1 by analyzing the voice signal X1 and the process of extracting the feature amount of the voice V2 by analyzing the voice signal X2 can be omitted. As can be understood from the above description, the feature acquisition unit 32 is comprehensively expressed as an element that acquires the feature amount of the voice (V1, V2) forming the conversation, and the feature amount is obtained by the analysis process for feature extraction. In addition to elements extracted from the audio signal, elements that acquire the feature amount by any method other than extraction are also included. That is, “extraction” of the feature amount is an example of “acquisition” of the feature amount.

(2) In each of the above-described embodiments, the evaluation value S calculated by the conversation evaluation unit 36 is displayed on the display device 16, but the form of the evaluation result by the conversation evaluation unit 36 is not limited to the evaluation value S. For example, it is possible to display an evaluation comment according to the evaluation value S on the display device 16 (whether or not the evaluation value S is displayed). Moreover, the output method of the evaluation result is not limited to the display. For example, it is possible to output the evaluation value S and the evaluation comment by voice.

(3) The method of calculating the evaluation value S according to the characteristic amount (pitch P1, pitch P2) and the related information R is not limited to the above-described examples. For example, it is also possible to calculate the evaluation value S by a calculation (for example, a weighted sum) that applies a numerical value that evaluates conversation according to the feature amount and a numerical value that is calculated according to the related information R. Further, for example, even in a configuration in which the relationship between the feature amount and the evaluation value S (for example, the type and coefficient of the arithmetic expression that defines the relationship between the two) is changed according to the related information R, both the feature amount and the related information R are set. It is possible to calculate a corresponding evaluation value S.

(4) The feature amount extracted by the feature acquisition unit 32 is not limited to the pitch (P1, P2). For example, the feature acquisition unit 32 can extract the volume of each of the voice V1 and the voice V2 as the feature amount. The conversation evaluation unit 36 evaluates the conversation, for example, according to the volume difference between the voice V1 and the voice V2. For example, the conversation evaluation unit 36 calculates the evaluation value S such that the evaluation value S becomes larger as the volume difference between the voice V1 and the voice V2 becomes closer to a predetermined value.

It is also possible for the feature acquisition unit 32 to extract the interval between the utterance section of the voice V1 and the utterance section of the voice V2 (hereinafter referred to as the "utterance interval") as the feature amount. When the utterance interval during conversation is appropriate, there is a tendency that the voice of the other party in the conversation is perceived as a utterance that is comfortable and has a good impression. In consideration of the above tendency, it is preferable that the conversation evaluation unit 36 calculates the evaluation value S so that the evaluation value S becomes larger as the utterance interval is closer to the predetermined value.

(5) The method by which the information generation unit 34 generates the related information R is not limited to the above-described exemplary embodiments. Specifically, the information generating unit 34 can generate the related information R from the result of analyzing the voice signal X1 and the voice signal X2. For example, the information generation unit 34 estimates the genders of the user U1 and the user U2 by using the result of the feature acquisition unit 32 specifying the pitch P1 of the voice V1 and the pitch P2 of the voice V2. Similarly to the fourth embodiment, it is possible to generate the related information R that specifies the sexes of the user U1 and the user U2.

(6) It is also possible to realize the conversation evaluation device 100 by a server device (a single device or a server system configured by a plurality of devices) that communicates with a terminal device such as a mobile phone or a smartphone. For example, the conversation evaluation device 100 receives the voice signal X1 and the voice signal X2 from the terminal device, and evaluates the conversation between the user U1 and the user U2 by the same method as each of the above-described modes (for example, an evaluation value). S) is transmitted to the terminal device.

(7) As described above, the conversation evaluation device 100 exemplified in each of the above-described modes can be realized by the cooperation of the control device 12 and the program. For example, the program of the first aspect corresponding to the first to third embodiments is a feature acquisition that causes a computer (for example, one or more processing circuits) such as the control device 12 to acquire a feature amount of voice that constitutes a conversation. The unit 32 functions as an information generation unit 34 that generates related information R of a type different from the feature amount and a conversation evaluation unit 36 that evaluates the conversation according to the feature amount and the related information R.

In addition, the program of the second aspect corresponding to the fourth embodiment or the fifth embodiment causes a computer (for example, a single or a plurality of processing circuits) such as the control device 12 to acquire the feature amount of voice that constitutes a conversation. The acquisition unit 32 is a program that functions as an information generation unit 34 that generates related information R of a type different from the feature amount for conversation, and a conversation evaluation unit 36 that evaluates the conversation according to the feature amount. , The characteristic amount is acquired under the condition according to the related information R.

The program of each aspect illustrated above can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but any known recording medium such as a semiconductor recording medium or a magnetic recording medium is used. The recording medium of this type may be included. It is also possible to distribute the program to the computer in the form of distribution via a communication network.

(8) The preferred aspect of the present invention can be specified as the operation method (conversation evaluation method) of the conversation evaluation device 100 exemplified in each of the above-described embodiments. In the conversation evaluation method according to the first aspect, a computer (a single computer or a system composed of a plurality of computers) acquires a feature amount of a voice forming a conversation, and the related information R of a type different from the feature amount regarding the conversation. Is generated and the conversation is evaluated according to the feature amount and the related information R. In the conversation evaluation method according to the second aspect, the computer acquires the characteristic amount of the voice that constitutes the conversation, generates the related information R of a type different from the characteristic amount of the conversation, and according to the characteristic amount and the related information R. This is a method of evaluating conversation, and in the acquisition of the characteristic amount, the characteristic amount is acquired under the condition according to the related information R.

100... Conversation evaluation device, 12... Control device, 14... Storage device, 16... Display device, 18... Input device, 22... Sound collection device, 24... Sound collection device, 32... Feature acquisition unit, 34... Information generation unit, 36... Conversation evaluation department.

Claims (5)

A characteristic acquisition unit that acquires the pitch of a voice that constitutes a conversation between the first user and the second user ;
The frequency of conversations conducted in the past between the first user and the second user, and the number of conversations conducted in the past between the first user and the second user. An information generation unit that generates related information including at least one of a relation period that is an elapsed time from the first conversation ,
A conversation evaluation device, comprising: a conversation evaluation unit that evaluates the conversation according to the pitch and the related information. The conversation evaluation device according to claim 1, wherein the feature acquisition unit acquires the pitch under a condition according to the related information. The characteristic acquisition unit acquires the pitch of each of the first audio and second audio constituting the conversation,
The conversation evaluation unit, conversation evaluation apparatus according to claim 1 or claim 2 for evaluating the conversation in accordance with pitch difference between the second audio as the first audio. Computer,
A characteristic acquisition unit that acquires the pitch of a voice that constitutes a conversation between the first user and the second user ,
The frequency of conversations conducted in the past between the first user and the second user, and the number of conversations conducted in the past between the first user and the second user. An information generation unit that generates related information including at least one of a relation period that is an elapsed time from the first conversation , and
A program that functions as a conversation evaluation unit that evaluates the conversation according to the pitch and the related information. Acquiring the pitch of the voice that constitutes the conversation between the first user and the second user,
The frequency of conversations conducted in the past between the first user and the second user, and the number of conversations conducted in the past between the first user and the second user. Generate relevant information including at least one of the relationship period, which is the elapsed time from the first conversation,
Evaluate the conversation according to the pitch and the related information
A conversation evaluation method implemented by a computer.

Images