JP2009139390A - Information processing system, processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing system, processing method and program for smooth user communication. <P>SOLUTION: This information processing system for achieving smooth communication between users processes communications from a first speaker to a second speaker and has a second speaker feature extraction means which extracts a feature of the second speaker, and a communication processing means which processes input data from the first speaker based on the feature of the second speaker. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an information processing system, a processing method, and a program for smoothly performing communication between users.

  Various systems have been proposed so that the output of human-machine or human-communication system via machine can be smoothly handled even by different users.

  For example, in the field of speech recognition, a speech recognition system using features of a speaker's face image has been proposed (see Patent Document 1).

  As shown in FIG. 30, the speech recognition system described in Patent Document 1 includes a speech recognition input unit, a video signal input unit, an unspecified speaker speech recognition unit, a specific speaker speech recognition unit, and image processing. And a recognition result integration unit. The image processing unit includes a face area extraction unit, a face image database, and an image comparison unit. The specific speaker voice recognition unit includes a voice processing unit, a voice database, and a voice recognition processing unit.

  This system provides a speech recognition method that identifies a speaker using features of a speaker's face image and can realize a high recognition rate even with respect to inputs of a plurality of specific speakers.

  In addition, an information processing method has been proposed in which a synthesized speech is output from a sentence using the treatment relationship between the sender and receiver of the sentence (see Patent Document 2).

  As shown in FIG. 31, the information processing method described in Patent Document 2 includes a mailbox, a command input unit, an operation management unit, an e-mail management unit, a text reading unit, a voice transmission unit, and a text. It comprises an analysis unit, a sentence generation unit, a treatment determination unit, and a semantic expression summary unit.

  The determination of the treatment relationship in this system is performed using the profile information of the sender and receiver stored in advance.

Also, a voice response device has been invented that identifies a user type and selects voice recognition, dialog control, and voice synthesis (see Patent Document 3).
As shown in FIG. 32, the voice response device described in Patent Document 3 includes a user type identification unit, a voice recognition unit, a dialogue control unit, a database, and a voice synthesis unit.

In this system, a user type is identified for a single user, voice recognition, dialogue control and speech synthesis are performed using the identification information, and a response voice suitable for the user is output.
Japanese Patent Laid-Open No. 11-282492 Japanese Patent Laid-Open No. 10-149361 JP 2004-163541 A

  However, the first problem of the technique described in the patent document is that speech input, text conversion or machine translation, speech synthesis suitable for the characteristics of the second speaker with respect to the input from the first speaker The communication process such as cannot be performed.

  The reason is that, in Patent Documents 1 and 3, the means for acquiring the personal attribute of the second speaker is not provided.

  Further, in Patent Document 2, since the personal attribute of the user is acquired from the profile of the sender and receiver of the text, the second speaker's profile is not given in advance. This is because the characteristics of the second speaker cannot be acquired.

  Therefore, the present invention has been invented in view of the above problems, and an object thereof is to provide an information processing system, a processing method, and a program for smoothly performing communication between users.

  The present invention that achieves the above object is an information processing system for processing communication from a first speaker to a second speaker, wherein the second speaker extracts characteristics of the second speaker. An information processing system comprising: feature extraction means; and communication processing means for performing optimization processing of input data from the first speaker based on the characteristics of the second speaker.

  In addition, the present invention that achieves the above object is a processing method for processing communication from a first speaker to a second speaker, wherein the first speaker extracts features of the second speaker. And a communication process for performing a communication process for input data from the first speaker based on the characteristics of the second speaker.

  The present invention that achieves the above object is a program of an information processing system that executes processing of communication from a first speaker to a second speaker, and extracts features of the second speaker A program for causing an information processing system to execute a first speaker feature extraction process and a communication process for performing an optimization process on input data from the first speaker based on the characteristics of the second speaker. .

  According to the present invention, the speaker characteristics of the second speaker can be obtained from the first speaker without preparing the profile of the second speaker in advance for the utterance of the speaker having various attributes. Communication processing such as speech recognition, text conversion or machine translation, speech synthesis, etc. for the second speaker can be performed.

<First Embodiment>
A first embodiment will be described.

  The first embodiment is a system that performs communication processing such as translation and speech synthesis between a first speaker that emits communication by voice, text, and the like and a second speaker that receives the communication. The second speaker feature extracting means for extracting the feature amount of the second speaker is provided to extract the personal attribute of the second speaker in real time. Then, the similarity between the speaker attributes of the second speaker extracted by the feature extraction means of the second speaker and the attributes of each model of the model group stored in the dictionary / model database is calculated, Select the one with the highest degree, and perform communication processing such as speech recognition, text conversion or machine translation, speech synthesis, etc. suitable for the second speaker for the voice and text of the first speaker is there.

  The dictionary / model stored in the dictionary / model database in the communication processing system according to the first embodiment includes a dictionary / model for speech recognition, a dictionary / model for machine translation, a dictionary / model for text conversion, and a speech synthesis model. Various dictionaries / models such as dictionaries / models are constructed in advance based on the characteristics of the second speaker. For example, in the construction of a language model for speech recognition, the utterance types from the first speaker to the second speaker are the utterance to the elderly man type, the utterance to the elderly woman type, "Speaking to men" type, "Speaking to older women" type, "Speaking to youth men" type, "Speaking to youth women" type, "Speaking to young men" type, and "Young A speech recognition language model can be constructed by classifying into "speech to women" type, "speech to children men" type, and "speech to children women" type. Similarly, a dictionary / model for machine translation based on the characteristics of the second speaker, a dictionary / model for text conversion, a dictionary / model for speech synthesis, and the like are built in advance and stored in the dictionary / model database.

  In addition, when performing speech recognition of the first speaker as a communication process, by using the extracted features of the second speaker, appropriate speech recognition of the first speaker can be performed from the dictionary / model database. A voice recognition dictionary / model can be selected. For example, when the feature of the second speaker extracted by the second speaker feature extraction means is “child, woman”, the language model for speech recognition for the utterance from the first speaker is “child, woman”. ”Can be used to select the speech recognition language model of“ children and women ”used when the first speaker speaks to“ children and women ”from the speech recognition dictionary / model database. . Specifically, the similarity between the second speaker's speaker attribute information (children, women) and the attribute information of various speech recognition dictionaries / models stored in the dictionary / model database is calculated. Select the one with the maximum similarity. In this way, it is possible to perform speech recognition of the first speaker by selecting a speech recognition language model having an attribute of “children and women”. Similarly, even if the second speaker is a speaker having other attributes, the language model for speech recognition that matches the speaker's attributes can be selected by the dictionary / model selection means to perform speech recognition. It becomes possible.

  When performing machine translation of the language of the first speaker as the communication processing means, the translation dictionary stored in the dictionary / model database is selected using the characteristics of the second speaker. For example, in the case of English-Japanese translation, a Japanese generation dictionary is constructed in advance according to the attributes of the second speaker and stored in the dictionary / model database. Taking the English personal pronoun “your” as an example, the Japanese translation candidates corresponding to “your” have translations such as “your”, “your”, and “my”. Therefore, the target language block of the source language dictionary word “your” is the Japanese headword “You”, part-of-speech information, and information indicating the speaker attributes of the second speaker “young, male” The first target language dictionary block composed of information such as, the second word composed of information such as the headword "Kimi no" and part-of-speech information, and "youth, male" indicating the speaker attributes of the second speaker And the third block consisting of the headword “my”, part-of-speech information, and information such as “children, men” indicating the speaker attributes of the second speaker, the English personal pronoun “your” You can build a dictionary. For example, if you enter English “What is your name?” From the first speaker into the system and the second speaker is presumed to be “children, men”, first the English morphological analysis and syntax As a result of the analysis, information on the morphological element English interrogative pronoun “What”, Be verb “be”, personal pronoun “your”, and noun “name” can be obtained. On the other hand, on the Japanese side, “What” to “What”, “be” to “da”, “name” to “name”, “your” to “your” and “you” from the English-Japanese translation dictionary. The three candidates “No” and “My” are read. Next, using the information of the above-mentioned “children and men”, the similarity between the speaker features of the three Japanese candidates “your” is calculated, and the “my” block with “children and men” is calculated. Since the similarity of is maximized, the translation of “your” becomes “my”. Next, the morphemes for generating Japanese are “what”, “da”, “my”, and “name”. Next, machine translation can be performed using the case frame and translation rules of the Japanese independent auxiliary verb “DA”, and the output “What is my name?” Can be provided. Similarly, if the attribute of the second speaker is “Youth, Male”, the translation result is “What is your name?”, And if the attribute of the second speaker is “Young, Male”, the translation The result generates a translation result “What is your name?”.

  In addition, when the communication process is a speech synthesis process and the second speaker is an elderly person, the dictionary / model selection means creates a dictionary / model for speech synthesis for the elderly using the characteristics of the second speaker. By selecting and slightly increasing the duration of the synthesized speech waveform and adjusting the sound pressure level and pitch frequency threshold appropriately, the text, speech, etc. of the first speaker can be adjusted at a speed suitable for the elderly. Slowly speaking, slightly increasing the volume, and controlling the pitch of the voice.

  Next, a specific configuration in the first embodiment will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of the first embodiment for carrying out the present invention.

  Referring to FIG. 1, in the first embodiment of the present invention, an input means 101 for inputting data such as voice, image, or character information of a first speaker, and a communication processing means 100 for processing the input data. And second speaker feature extraction means 105 for extracting the characteristics of the second speaker, and output means 116 for outputting information such as characters, voices and images output from the communication processing means 100. .

  The communication processing unit 100 performs communication processing with one or more configurations from among a dictionary / model selection unit, a speech recognition unit, a text conversion unit, a machine translation unit, a speech synthesis unit, and the like.

  In the present embodiment, in order to perform various processes of the communication processing unit 100, various dictionaries / models are built in advance and stored in the dictionary / model database as described above.

  For example, in the construction of a language model for speech recognition, the utterance types from the first speaker are the utterance to the elderly man type, the utterance to the elderly woman type, and the utterance to the elderly man type. , "Speaking to a young woman" type, "Speaking to a young man" type, "Speaking to a young woman" type, "Speaking to a young man" type, and "Speaking to a young woman" type The language model for speech recognition is constructed by classifying into "speech for children men" type and "speech for children women" type. Similarly, a dictionary / model for machine translation based on speaker characteristics of the second speaker, a dictionary / model for text conversion, a dictionary / model for speech synthesis, and the like are built in advance and stored in the dictionary / model database.

  The second speaker feature extraction by the second speaker feature extracting means 105 extracts the speaker feature from other biological information such as voice data, face image, fingerprint, etc. of the second speaker. . In addition, when a profile of the second speaker is provided, an IC card capable of communicating the personal attributes of the sensor or the speaker, or a terminal device incorporated in an IC card capable of communicating the personal attributes of the speaker is used. Thus, the characteristics of the second speaker can be extracted. For example, attribute information such as age and gender can be extracted by using voice data of the second speaker, image data including a face image, or the like.

  Next, the operation of the first embodiment for carrying out the present invention will be described in detail with reference to FIG. 1 and FIG.

  Data is input to the system through the input means 101 (step 2201).

  Next, speaker features indicating the personal attributes of the second speaker are extracted by the second speaker feature extracting means 105 (step 2202).

  Next, using the feature of the second speaker extracted from the second speaker feature extraction unit 105, the dictionary / model selection unit performs a dictionary / model selection process for communication processing (step 2203). . For example, the input is a sentence. When machine translation is performed as a communication process, a dictionary / model selection unit selects a translation dictionary / model suitable for the speaker characteristics of the second speaker. In addition, when speech is input and speech recognition is performed as communication processing, the dictionary / model selection means selects a speech recognition dictionary / model suitable for the speaker characteristics of the second speaker. The dictionary / model selection means resembles the characteristics of the second speaker extracted from the second speaker feature extraction means 105 and the attribute information of various dictionaries / models stored in the dictionary / model database. The degree is calculated, and the one having the maximum similarity is selected.

  Next, communication processing is performed by the communication processing means 100 using the dictionary / model selected in the above step (step 2204). As a communication processing method, speech recognition, text conversion, machine translation, speech synthesis, and the like can be performed.

  Then, the processing result in the communication processing means 100 is output through the output device (step 2205).

  The effect of the first embodiment for carrying out the present invention will be described.

  In the first embodiment of the present invention, by providing the second speaker feature extraction means, the speaker feature of the second speaker is used without preparing the profile of the second speaker in advance. Communication processing is possible.

  Specifically, when speech recognition is performed as communication processing, a speech recognition method suitable for the characteristics of the second speaker is provided, and the accuracy of speech recognition can be improved.

  For example, when the speaker feature of the second speaker extracted by the second speaker feature extraction means is “child, woman”, the language model for speech recognition for the speech from the first speaker is “child” The speech recognition language model having the second speaker feature of “children and women” is selected from the dictionary / model database for speech recognition using the attribute information of “female”. Specifically, the similarity between the second speaker's speaker attribute information (children, women) and the attribute information of various voice recognition dictionaries / models stored in the dictionary / model database is calculated, Select the one with the maximum similarity. In this way, it is possible to select a speech recognition language model having the attribute of “children and women” and perform speech recognition of the utterance of the first speaker. Similarly, when the second speaker is a speaker with other attributes, it is possible to select a language model for speech recognition that matches the speaker's attributes using the dictionary / model selection means and perform speech recognition. Become.

  Also, when machine translation is implemented as communication processing, it is suitable for the characteristics of the second speaker by selecting the translation dictionary stored in the dictionary / model database using the characteristics of the second speaker. The machine translation technique is provided, and it has an effect that a variety of translation results can be realized.

  For example, in the case of English-Japanese translation, a Japanese generation dictionary is constructed in advance according to the attributes of the second speaker and stored in the dictionary / model database. Taking the English personal pronoun “your” as an example, the Japanese translation candidate corresponding to “your” can have translations such as “your”, “your”, and “my”. Therefore, the target language block of the source language dictionary word “your” is the Japanese headword “You”, part-of-speech information, and information indicating the speaker attributes of the second speaker “young, male” The first target language dictionary block composed of information such as, the second word composed of information such as the headword "Kimi no" and part-of-speech information, and "youth, male" indicating the speaker attributes of the second speaker And the third block consisting of the headword “my”, part-of-speech information, and information such as “children, men” indicating the speaker attributes of the second speaker, the English personal pronoun “your” Build a dictionary. If an English sentence `` What is your name? '' Is entered from the first speaker into the system and the second speaker is presumed to be `` children, men '', machine translation will begin with an English morpheme. Analysis and syntactic analysis are performed, and as a result, information about the morphological element English interrogative pronoun “What”, Be verb “be”, personal pronoun “your”, and noun “name” is obtained. On the other hand, on the Japanese side, “What” to “What”, “be” to “da”, “name” to “name”, “your” to “your” and “you” from the English-Japanese translation dictionary. The three candidates “No” and “My” are read. Next, using the information of the above-mentioned “children and men”, the similarity between the speaker features of the three Japanese candidates “your” is calculated, and the “my” block with “children and men” is calculated. Since the similarity of is maximized, the translation of “your” becomes “my”. Next, the morphemes for generating Japanese are “what”, “da”, “my”, and “name”. Machine translation is performed using the case frame and translation rules of the Japanese independent auxiliary verb "DA", and the output can be "What is my name?" Similarly, if the attribute of the second speaker is “Youth, Male”, the translation result is “What is your name?”, And if the attribute of the second speaker is “Young, Male”, the translation By providing a translation method that produces a translation result “What is your name?”, A variety of translation results can be realized.

  Further, when voice synthesis is performed as communication processing, there is an effect of improving the practicality of synthesized voice.

For example, if the second speaker is an elderly person, the dictionary / model selection means selects the dictionary / model for speech synthesis using the second speaker characteristics, and slightly increases the duration of the synthesized speech waveform, To provide a speech synthesis method that can control the voice pitch by adjusting the sound pressure level and the threshold of the pitch frequency appropriately, making the elderly slowly speak at a suitable speed, slightly increasing the volume. This has the effect of increasing the practicality of the synthesized speech.
<Second Embodiment>
In the second embodiment, in addition to the configuration of the first embodiment, first speaker feature extraction means for extracting features of the first speaker is further provided, and the personal attributes of both speakers are real-time. Similar to the attributes of each model of the dictionary / model database prepared by the dictionary / model selection means using the first speaker attribute and the second speaker's speaker attribute The degree of similarity is calculated, the one having the maximum degree of similarity is selected, and communication processing such as speech recognition, text conversion or machine translation, and speech synthesis is performed.

  In the communication processing system using both the speaker characteristics of the first speaker and the speaker characteristics of the second speaker in the second embodiment, the dictionary / model stored in the dictionary / model database Considering both the speaker characteristics of the first speaker and the speaker characteristics of the second speaker in advance, such as a dictionary for speech recognition, a dictionary for machine translation, a dictionary for text conversion, a dictionary for speech synthesis, etc. And build. For example, the construction of the speech recognition database is based on the combination of the speaker characteristics of the first speaker and the speaker characteristics of the second speaker in advance. Various speech recognition dictionaries / models are constructed and stored in the dictionary / model database with contents that can be reflected by features. For example, the utterance types for child men are `` utterance from old man to child man '' type, `` utterance from old woman to child man '' type, `` utterance from young man to child man '' type, It can be classified into the types of “utterance from young woman to child male” type, “utterance from young man to child male” type, and “utterance from young woman to child male” type. Similarly, the utterance types for children and women are the “speech from old man to child woman” type, the “speak from old woman to child woman” type, the “speak from young man to child woman” type, It can be classified into the types of “utterances from young women to child women”, “utterances from young men to child women”, and “utterances from young women to child women”. By using the language indicating the utterance type classified as described above, various voice recognition dictionaries / models for “utterance to children” can be constructed and stored in the dictionary / model database. Similarly, dictionaries / models for machine translation, text conversion, speech synthesis, etc. are also constructed in advance and stored in the dictionary / model database.

  In the second embodiment, when speech recognition is performed as a communication process, the features of the first speaker and the features of the second speaker are simultaneously used and stored in the dictionary / model database. Select a dictionary / model for speech recognition. For example, when the first speaker is a youth woman and the second speaker is a child man, the dictionary / model selection means will show the speaker characteristics of the first speaker “Youth, Woman” and the second Using “children, men” indicating the speaker characteristics of the speaker, the similarity with the dictionary / model stored in the dictionary / model database is calculated, and the one with the maximum similarity is selected. Specifically, in speech recognition processing for utterances from the first speaker, the selection of the acoustic model is performed by the speech recognition acoustic model stored in the dictionary / database using the speaker attributes of the first speaker. The similarity with the person's attribute is calculated, and the one with the maximum similarity is selected. The selection of the language model uses the speaker relationship obtained by the speaker relationship determination means using the speaker feature of the first speaker and the speaker feature of the second speaker and the features of both speakers at the same time. Then, the similarity with the attribute information of the speech recognition language model stored in the dictionary / database is calculated, and the one with the maximum similarity is selected. In this way, a speech recognition dictionary / model having the attributes of “young man and woman” as the first language speaker and “children and men” as the second speaker is selected for speech recognition. Can be done. Similarly, even if the speaker feature of the first speaker and the speaker feature of the second speaker are different features, the speech recognition dictionary / model is selected as described above, and both speakers are selected. It is possible to perform speech recognition suitable for the features of

  When machine translation is performed as communication processing means, morphological analysis, syntax analysis, target language generation processing for generating a target language, and the like are included. In the target language generation process, the dictionary / model selection means selects the translation dictionary stored in the dictionary / model database by using the speaker characteristics of the first speaker and the speaker characteristics of the second speaker at the same time.

  In order to explain, a Chinese-Japanese translation dictionary is preliminarily constructed. On the Chinese side of the translation dictionary, applicable first speaker features are given. On the Japanese side, translations are given according to the speaker characteristics of the second speaker.

  Taking the Chinese pronoun pronoun “late life” as an example, “late life” is generally a humble term and can be used by men and women younger than the listener. Therefore, attribute information “source language speaker characteristics (younger)” is given to the block of “late life” that is the source language. The Japanese translation candidates corresponding to "Late" are "Watakushi", "I", "I", etc., so the dictionary for machine translation that can be constructed is "Late" and "Junior" on the source language side In the source language block that is composed, on the generation side, the first target language generation block composed of “Watashi”, part of speech information, and target language speaker characteristics (older, female), “I”, part of speech Information, a second target language generation block composed of target language speaker characteristics (older, male), “I”, part-of-speech information, and target language speaker characteristics (younger, male) A dictionary can be constructed with the second target language generation block.

  Next, “sentence 1 in FIG. 33” will be described as an example of translation. For example, when the first speaker is an adolescent male and the second speaker is “old man, male”, the machine translation at this time first performs Chinese morphological analysis and syntactic analysis. Syntactic information composed of the element pronoun “late life”, aspect subscript “present”, verb “word 1 in FIG. 33”, and noun “word 2 in FIG. 33” is obtained. On the other hand, on the Japanese side, from the Chinese-Japanese translation dictionary, the pronouns “Bansei” to “Watakushi”, “I” and “I”, the aspect subscript “Ai” to “I”, the verb “word in FIG. 33” “Read” is read from “1”, and “Book” is read from the noun “Word 2 in FIG. 33”.

  Next, using the information on the speaker characteristics of the first speaker “Youth, male” and the speaker characteristics of the second speaker “Old man, male” By comparing the ages, the results of the source language speaker characteristics (younger) and the target language speaker characteristics (older) are obtained. Furthermore, the first speaker's speaker feature “Youth, male, source language speaker characteristics (younger)” and the second speaker's speaker feature “old man, male, target language speaker characteristics (year) The above is used to calculate the similarity with the attribute information of the translation dictionary, and the one with the maximum similarity is selected. As a result, the translation of “I” is selected from the three translation candidates of “late life”.

  Subsequently, machine translation is performed using the morpheme information, the syntax information having the case frame, and the translation rules, and a translation result “I am reading a book” is generated. Similarly, when the first speaker is a young man and the second speaker is an elderly woman, the output of the above translation example is to generate a translation result that reads "I am reading a book." Become. When the first speaker is a young man and the second speaker is a child, the translation results will be translated as “I am reading a book”.

  Further, when speech synthesis is performed as the communication processing means, the speaker relationship is determined using the speaker characteristics of the first speaker and the characteristics of the second speaker at the same time. In particular, if the speaker on the second language side is an elderly person, the dictionary / model selection means selects the dictionary / model for speech synthesis using the second speaker characteristics, and the duration of the synthesized speech waveform is slightly increased. Then, by appropriately adjusting the threshold values of the sound pressure level and the pitch frequency, it is possible to slowly increase the volume at a speed suitable for the elderly, to slightly increase the volume, and to control the pitch of the voice.

  Next, a specific configuration in the second embodiment will be described in detail with reference to the drawings.

  FIG. 2 is a block diagram showing a configuration of the second embodiment.

  Referring to FIG. 2, the second embodiment is similar to the configuration of the first embodiment shown in FIG. 1 in that the input means 101, the second speaker feature extraction means 105, and the output Means 116 are provided. In addition, a first speaker feature extracting unit 104 for extracting features of the first speaker and a communication processing unit 200 are provided.

  The communication processing means 200 performs communication processing in one or more configurations from among speaker relation determination means, dictionary / model selection means, speech recognition means, text conversion means, machine translation means, speech synthesis means, etc. carry out.

  Note that the speaker relationship determination means is based on the first and second speaker features extracted by the first speaker feature extraction means 104 and the second speaker feature extraction means 105. The relationship between the speaker and the second speaker is determined.

  In the present embodiment, various dictionaries / models for performing various processes of the communication processing means 200 are previously constructed and stored in the dictionary / model database. For example, the utterance types for child men are `` utterance from old man to child man '' type, `` utterance from old woman to child man '' type, `` utterance from young man to child man '' type, It can be classified into the types of “utterance from young woman to child male” type, “utterance from young man to child male” type, and “utterance from young woman to child male” type. Similarly, the utterance types for children and women are the “speech from old man to child woman” type, the “speak from old woman to child woman” type, the “speak from young man to child woman” type, It can be classified into the types of “utterances from young women to child women”, “utterances from young men to child women”, and “utterances from young women to child women”. Using the language indicating the utterance type classified as described above, various speech recognition dictionaries / models for “utterance to children” are constructed and stored in the dictionary / model database. Similarly, dictionaries / models for machine translation, text conversion, speech synthesis, etc. are also constructed in advance and stored in the dictionary / model database.

  Next, the operation of the second embodiment for carrying out the present invention will be described in detail with reference to FIG. 2 and FIG.

  Data is input to the system through the input means 101 (step 2301).

  Next, the first speaker feature extraction means 104 extracts speaker features indicating the personal attributes of the first speaker, and the second speaker feature extraction means 105 indicates the personal attributes of the second speaker. Speaker features are extracted (step 2302).

  Using the speaker features of the first speaker extracted from the first speaker feature extraction unit 104 and the speaker features of the second speaker extracted from the second speaker feature extraction unit 105, Speaker relationship is determined by speaker relationship determination means. For example, by comparing the ages of the two speakers, a determination result of “older” and “younger” is obtained (step 2303).

  Using the speaker relationship information obtained by the speaker relationship determination means, the dictionary / model selection means performs a dictionary / model selection process for communication processing (step 2304).

  For example, when the input is a sentence and machine translation is performed as a communication process, a dictionary / model selection unit selects a translation dictionary / model suitable for speaker characteristics of both speakers. When speech is input and speech recognition is performed as communication processing, a dictionary / model for speech recognition suitable for speaker characteristics of both speakers is selected by the dictionary / model selection means. In the dictionary / model selection means, the attribute information of both speakers extracted from the speaker feature extraction means 104 and the speaker feature extraction means 105, and the speaker relation information obtained by the speaker relation determination means, The dictionary / model selection process is performed by calculating the similarity with the attribute information of various dictionaries / models stored in the dictionary / model database and selecting the one having the maximum similarity.

  Next, communication processing is performed by the communication processing means 200 using the dictionary / model selected in the above step (step 2305). As a communication processing method, speech recognition, text conversion, machine translation, speech synthesis, and the like can be performed.

  Then, the processing result of the communication processing means 200 is output through the output device (step 2306).

  The effect of the second embodiment will be described.

  In the second embodiment, by providing the first speaker feature extracting means and the second speaker feature extracting means, the speaker characteristics of both speakers can be obtained without preparing the profiles of both speakers in advance. Communication processing using both can be performed.

  Specifically, when speech recognition is performed as communication processing, there is an effect that it is possible to provide a method for performing speech recognition suitable for the characteristics of both speakers.

  For example, when the first speaker is a youth woman and the second speaker is a child man, the dictionary / model selection means will show the speaker characteristics of the first speaker “Youth, Woman” and the second Using “children, men” indicating the speaker characteristics of the speaker, the similarity with the dictionary / model stored in the dictionary / model database is calculated, and the one with the maximum similarity is selected. Specifically, in speech recognition processing for utterances from the first speaker, the selection of the acoustic model is performed by the speech recognition acoustic model stored in the dictionary / database using the speaker attributes of the first speaker. The similarity with the person's attribute is calculated, and the one with the maximum similarity is selected. The selection of the language model is based on the speaker relation obtained by the speaker relation judging means using the speaker characteristics of the first speaker and the speaker characteristics of the second speaker and the characteristics of both speakers at the same time. Then, the similarity with the attribute information of the speech recognition language model stored in the dictionary / database is calculated, and the one with the maximum similarity is selected. In this way, a speech recognition dictionary / model having the attributes of “young man and woman” as the first language speaker and “children and men” as the second speaker is selected for speech recognition. Can be done. Similarly, even if the speaker feature of the first speaker and the speaker feature of the second speaker are different features, the speech recognition dictionary / model is selected as described above, and both speakers are selected. It is possible to perform speech recognition suitable for the features of

  Further, when machine translation is realized as communication processing, morphological analysis, syntax analysis, target language generation processing for generating a target language, and the like are included. In the target language generation process, the dictionary / model selection means selects the translation dictionary stored in the dictionary / model database by using the speaker characteristics of the first speaker and the speaker characteristics of the second speaker at the same time.

  For example, when performing Chinese-Japanese machine translation, in the example of Chinese “sentence 1 in FIG. 33”, the Chinese pronoun “late life” is generally humbly, and the younger than the listener is male or female. But you can use it. For example, attribute information of “source language speaker characteristics (younger)” is given to the block of “late life”, which is the source language of the Japanese-Chinese translation dictionary. The Japanese translation candidates corresponding to "Late" are "Watakushi", "I", "I", etc., so the dictionary for machine translation that can be constructed is "Late" and "Junior" on the source language side The source language block is composed of the target language generation block composed of “Watashi” and the target language speaker characteristics (older, female), and “I” and the target language speaker characteristics (older, male). ) And a block composed of information such as “I” and target language speaker characteristics (younger, male), etc., can be used to construct a dictionary.

  Next, “sentence 1 in FIG. 33” will be described as an example of translation. For example, when the first speaker is an adolescent male and the second speaker is “old man, male”, the machine translation at this time first performs Chinese morphological analysis and syntactic analysis. Syntactic information composed of the element pronoun “late life”, aspect subscript “present”, verb “word 1 in FIG. 33”, and noun “word 2 in FIG. 33” is obtained. On the other hand, on the Japanese side, from the Chinese-Japanese translation dictionary, the pronouns “Bansei” to “Watakushi”, “I” and “I”, the aspect subscript “Ai” to “I”, the verb “word in FIG. 33” “Read” is read from “1”, and “Book” is read from the noun “Word 2 in FIG. 33”.

  Next, using the information on the speaker characteristics of the first speaker “Youth, male” and the speaker characteristics of the second speaker “Old man, male” By comparing the ages, the results of the source language speaker characteristics (younger) and the target language speaker characteristics (older) are obtained. Furthermore, the first speaker's speaker feature “Youth, male, source language speaker characteristics (younger)” and the second speaker's speaker feature “old man, male, target language speaker characteristics (year) The above is used to calculate the similarity with the attribute information of the translation dictionary, and the one with the maximum similarity is selected. As a result, the translation of “I” is selected from the three translation candidates of “late life”.

  Next, machine translation is performed using the morpheme information, the syntax information having the case frame, and the translation rule, and a translation result “I am reading a book” is generated. Similarly, when the first speaker is a young man and the second speaker is an elderly woman, the output of the above translation example is to generate a translation result that reads "I am reading a book." Become. The first speaker is a young man, the second speaker is a child, and the translation results are translated by providing a translation method that produces a translation result that says "I am reading a book" Diversity of results can be realized.

  Further, when voice synthesis is performed as communication processing, there is an effect of improving the practicality of synthesized voice.

For example, the speaker relationship is determined using the speaker characteristics of the first speaker and the characteristics of the second speaker at the same time. In particular, if the speaker on the second language side is an elderly person, the dictionary / model selection means selects the dictionary / model for speech synthesis using the characteristics of both speakers, and slightly increases the duration of the synthesized speech waveform. Providing a voice synthesis method that can adjust the sound pressure level and the pitch frequency threshold appropriately to make the elderly slowly grow at a suitable speed, slightly increase the volume, and control the voice pitch As a result, the practicality of the synthesized speech can be improved.
<Third Embodiment>
A third embodiment will be described.

  FIG. 3 is a block diagram showing a configuration of the third embodiment for carrying out the present invention.

  Referring to FIG. 3, the third embodiment of the present invention is the same as the second embodiment shown in FIG. 2, except that the input means 101, the first speaker feature extraction means 104, In addition to the second speaker feature extraction means 105 and the output means 116, a speaker dialogue history database 310 for storing dialogue histories of both speakers and a communication processing means 300 are provided. In the conversation history database 310 of both speakers, the conversation history of the speakers is recorded in time series together with the speaker characteristics of both speakers.

  In the communication processing means 300, one or two or more of speaker relation determination means, dialog history management means, dictionary / model selection means, speech recognition means, text conversion means, machine translation means, speech synthesis means, etc. Implement communication processing in the configuration.

  Further, utterances from the speaker are always recorded in the dialogue history database 310 in time series according to the characteristics of the speaker. When the speaker is a new speaker, the feature amount of the speaker and the utterance are stored in the speaker interaction history database 310 as a new user. If the speaker is not a new speaker, the conversation history of the speaker is searched and recorded in the speaker conversation history database 310 in time series.

  Further, the dictionary / model database in the present embodiment can construct a system by the same one as in the second embodiment.

  Further, as shown in FIG. 29, the dialogue history management means 114, based on the characteristics and the speaker relations of both speakers output from the speaker relationship judgment means 113, from the speaker dialogue history database 310, Dialog history analysis means 2901 for extracting and analyzing the dialog history, and the speaker's extracted by the first speaker feature extraction means 104 and the second speaker feature extraction means 105 according to the analysis result of the dialog history In the determination result from the speaker feature determination unit 2902 and the speaker feature determination unit 2902 for determining whether the relationship between the feature and the speaker output from the speaker relationship determination unit 113 is correct, "There is an error" Is determined, feedback processing means 2903 for performing feedback processing on speaker characteristics is configured.

  The dialogue history analysis unit 2901 extracts the dialogue history of both speakers from the speaker dialogue history database 310 based on the characteristics and the speaker relationship of both speakers output from the speaker relationship judging unit 113, and the speaker A process including generating a vector including a feature, a dialog style, a keyword of a dialog content, or a model in which the vector is time-sequentially performed.

  The speaker feature determination unit 2902 uses the feature vector or model of the dialogue history generated by the speaker dialogue history analysis unit 2901, and the first speaker feature extraction unit 104 and the second speaker feature. It is determined whether or not the determination result for the speaker characteristics extracted by the extraction unit 105 and the speaker relationship output from the speaker relationship determination unit 113 is correct. For example, a feature quantity indicating the language expression of the Japanese “female word” and a feature quantity indicating the language expression of the “male language” are prepared and stored in the speaker dialogue history database 310 for dialogue. By comparing with the feature vector of the speaker obtained from the history analysis means 2901, the gender of the speaker can be determined.

  When the determination result from the speaker feature determination unit 2902 determines that “there is an error”, the feedback processing unit 2903 performs feedback processing on the speaker feature. For example, the first speaker and the second speaker were both assumed to be male, but the input sentence of the first speaker was "Thank you for bringing the key. Thank you. I often forget things recently. ”When the speaker feature determination means 2902 determines that the first speaker is“ female ”and performs feedback processing, Gender attribute values can be corrected.

  In this way, by using information obtained from the state and progress of the speaker's dialogue, it is possible to automatically determine whether or not the speaker's personality is “correct” by automatically determining the speaker's personal attribute error. A function of detecting and correcting can be realized.

  Next, the operation of the third mode for carrying out the present invention will be described in detail with reference to FIG. 3 and FIG.

  Data is input to the system through the input means 101 (step 2401).

  Next, the first speaker feature extraction means 104 extracts speaker features indicating the personal attributes of the first speaker, and the second speaker feature extraction means 105 indicates the personal attributes of the second speaker. Speaker features are extracted (step 2402).

  Using the speaker feature of the first speaker extracted from the first speaker feature extraction unit 104 and the speaker feature of the second speaker extracted from the second speaker feature extraction unit 105, the speaker The relationship determination means determines the speaker relationship. For example, by comparing the ages of both speakers, the determination result of “older” and “younger” is obtained (step 2403).

  Based on the characteristics and speaker relationships of both speakers output from the speaker relationship determining means 113, the conversation history of both speakers is extracted from the speaker conversation history database 310 and analyzed by the dialog history analyzing means 2901. Based on the analysis result, the speaker feature determination means 2902 determines whether or not the characteristics and speaker relationship of the two speakers used are correct. When it is determined that “there is an error”, feedback processing is performed by the feedback processing unit 2903 so as to perform speaker feature extraction again (step 2404).

Using the correct speaker attributes and speaker relationship information obtained in the previous step, the dictionary / model selection means performs dictionary / model selection processing for communication processing (step 2405). For example, when the input is a sentence and machine translation is performed as a communication process, a dictionary / model selection unit selects a translation dictionary / model suitable for speaker characteristics of both speakers. When speech is input and speech recognition is performed as communication processing, a dictionary / model for speech recognition suitable for speaker characteristics of both speakers is selected by the dictionary / model selection means.
In the dictionary / model selection means, the attribute information of both speakers extracted from the speaker feature extraction means 104 and the speaker feature extraction means 105, and the speaker relation information obtained by the speaker relation determination means, The dictionary / model selection process is performed by calculating the similarity with the attribute information of various dictionaries / models stored in the dictionary / model database and selecting the one having the maximum similarity.

  Subsequently, communication processing is performed by the communication processing means 100 using the dictionary / model selected in the above step (step 2406). As a communication processing method, speech recognition, text conversion, machine translation, speech synthesis, and the like can be performed.

  Then, the processing result of the communication processing means 300 is output through the output device (step 2407).

  The effect of the third embodiment will be described.

  As described above, in the third embodiment of the present invention, in addition to the effects of the second embodiment, the speaker dialogue history management means 114 and the speaker dialogue history database 310 are provided, thereby providing a speaker. The speaker characteristics extracted from the first speaker feature extraction means 104 and the second speaker feature extraction means 105 using the information obtained from the conversation status and progress of the speaker On the other hand, by performing correctness determination and feedback processing, an error can be automatically detected, and a function that can correctly use speaker characteristics of both speakers for communication processing can be realized. Have

  A first embodiment of the present invention will be described with reference to the drawings. Such an example corresponds to the first embodiment.

  This embodiment is shown in FIG. A text input unit 102, a second speaker feature extraction unit 105, a text conversion unit 106, a dictionary / model database 107, a dictionary / model selection unit 112, and a text output unit 115 are configured.

  The communication processing unit 100 in this embodiment includes a text conversion unit 106 and a dictionary / model selection unit 112.

  A keyboard is used as the text input means 102 and a display is used as the text output means 115.

  The second speaker feature extraction means 105 is means for extracting speaker features in real time. Examples of methods for extracting speaker characteristics include a method for estimating speaker attributes such as a speaker's age and gender from speaker's voice data, a method for estimating speaker attributes from an image including a speaker's face, and the like. Can be mentioned. In general, methods for extracting speaker features from an image including a speaker's face generally include face image glaze scale processing, image angle normalization processing, image size normalization processing, and image feature extraction from input image data. The speaker feature is extracted by performing processing and speaker attribute estimation processing.

  A technique using Gabor Wavelet is known as a speaker face image feature extraction technique. For example, non-patent document Pattern Recognition 30 (6), pp. 837-846, 1997, “Phantom Faces for Face Analysis”. An effective pixel position sampling method called Retina Sampling is also known as a method for setting feature points in a facial organ. For example, there is a non-patent document Audio and Video based Person Authentication-AVBPA99, pp. 125-129, 1999, “Face Authentication by retinotopic sampling of the Gabor decomposition and Support Vector Machines”.

  In addition, the conventional techniques for estimating the speaker's personal attributes from the speaker's facial features are known as a method using a facial texture, a method using a distance from the average face, and a method using multiple features. It has been.

  For example, in the method using the distance to the average face, an average face is created for each age and gender, and the feature points are taken independently for the average face, and the feature points are taken similarly for the input face, The distance between the feature points of the input face and the average face is calculated by using the square error and evaluating with an evaluation function representing the feature points of the facial organ, and the average face that is the minimum distance is determined as the age of the speaker. And the technology of gender is known. For example, the above-described technique can be used in the conventional technique disclosed in Japanese Patent Laid-Open Nos. 2003-99779 and 2006-344236 and the apparatus of this embodiment.

  In addition to the prior art described above, a hidden Markov model or a genetic algorithm can be used, but is not limited thereto.

  Further, speaker attributes such as the speaker's age and sex can be estimated from the speaker's voice data. The general method is to first obtain the average speech feature of each age from the labeled speech data of each age indicating speaker attributes such as gender and age as learning data, and obtain the speech signal by spectrum analysis etc. . Next, the feature amount of the voice of the speaker is acquired by performing spectrum analysis or the like on the voice signal from the voice data of the speaker. Next, by calculating the similarity between this feature quantity and the average speech feature quantity of each age obtained from the learning data, and selecting the one with the maximum similarity, the age and sex of the speaker Etc. can be estimated.

  For example, an acoustic feature is obtained by acoustic analysis of speech uttered by a speaker, and children, young people, young people, seniors, and elderly people prepared in advance for each gender using the obtained feature values. The method of calculating the degree of similarity with each of the standard patterns and acquiring the speaker's age and gender according to the most similar standard pattern can also acquire the speaker characteristics.

  Further, the speaker feature extraction means 105 is not limited to the method of extracting from the voice data and face image of the speaker, but can communicate the IC card or the personal attribute of the speaker that can communicate the personal attribute of the sensor or speaker. A method of using a terminal device built in an IC card, a method of acquiring the biological information of the speaker by other means, and a method of acquiring information such as the race, age, and sex of the speaker from the speaker profile are also used. May be.

The dictionary / model selection means 112 selects a dictionary / model suitable for the speaker from the database 107 storing the dictionary / model to which identification information indicating the characteristics of the speaker is added, and performs speech recognition, text conversion, or machine It is a means to select various dictionaries and models for translation and speech synthesis.
The algorithm for selecting a dictionary / model may use, but is not limited to, an inner product, a Jaccard coefficient, a cosine, a similarity based on a Dice coefficient, a similarity based on a chi-square, and a similarity based on the reciprocal of the Euclidean distance.

その正規化された重みは

とし、ある話者Ｂの話者特徴ベクトルは、

とし、その正規化された重みは、

とする。Ｘ・ＹをＸとＹと同じ属性がある場合に同じ属性同士の重みを掛け合わせた合計を返す演算であるとするとき、内積、Jaccard係数、余弦、Dice係数などの方法による話者Ｂと標準話者Ａとの類似度を計算する式は次に示す。

上の式に示すような計算方法で話者の特徴と辞書・モデルが持つ標準話者の特徴との類似度を計算し、類似度が最大となるものは辞書・モデル選択の対象とする。 For example, the speaker feature vector of standard speaker A is

Its normalized weight is

And a speaker feature vector of a certain speaker B is

And its normalized weight is

And If X and Y have the same attributes as X and Y, and the operation returns the sum of the weights of the same attributes multiplied by the speaker B by the method of inner product, Jaccard coefficient, cosine, Dice coefficient, etc. The formula for calculating the similarity with the standard speaker A is as follows.

The similarity between the speaker's characteristics and the standard speaker's characteristics of the dictionary / model is calculated by the calculation method shown in the above formula, and the one having the maximum similarity is selected as a dictionary / model selection target.

  The dictionary / model database 107 is a database composed of speech recognition, text conversion or machine translation, speech synthesis, thesaurus dictionary, and the like. Various dictionaries and models are generalized by attributes such as the age and gender of a large number of speakers, and feature amounts of standard speakers of each age group are added to the dictionary / model as identification information and constructed. For example, various dictionaries can be prepared by generalizing according to gender by children, young people, young people, middle-aged people, and elderly people.

  The text conversion means 106 performs a paraphrase expression process for the input text. Processing such as morphological analysis and syntax analysis is performed on the input from the text input means 102. In addition, using the speaker attributes acquired from the second speaker feature extraction unit 105, the dictionary / model selection unit 112 performs a paraphrase from the dictionary / database 107 to a language expression suitable for the second speaker. Can do.

  For example, when performing Japanese paraphrase, a paraphrase dictionary is preliminarily constructed according to the attributes of the second speaker and stored in the dictionary / model database. Here, the Japanese compound connection compounding phrase “I want” will be described as an example. The dictionary on the paraphrase generation side is constructed as follows. The source language block of the “wanted” dictionary that is the target of paraphrasing is the unique part “desired”, the part-of-speech information “telecom compounding word (type)”, utilization information, connection number information, etc. Consists of. On the generation side, it is constructed with two blocks: “Get” with higher politeness and “Please” with lower politeness. The first block is composed of a unique part “I want you”, part-of-speech information, usage information, connection number information, second speaker attributes (old man, man, woman) and the like. The second block is composed of a unique part “please”, part-of-speech information, utilization type information, connection number information, second speaker attributes (child, young, male, female) and the like.

  For example, the first speaker entered the system “I want to see this week's report” and the second speaker was estimated to be “old man, man”. In this case, the paraphrasing process first performs Japanese morphological analysis and syntax analysis. As a result, the morpheme elements are the noun “this week”, the case particle “no”, the noun “report”, the case particle “o”, Information such as the verb “show”, the connection particle “te”, the telecom compounding predicate “want”, the auxiliary verb “da”, and the final particle “ga” can be obtained. At this time, two candidates “I want you” and “Please” are read from the dictionary of the word “I want” that shows politeness.

  Next, using the information of the speaker attribute of the second speaker “old man, male”, the degree of similarity between the speaker characteristics of the two candidates “Please” and “Please” is calculated. Since the similarity of the block of “I want to get” with “old man, man” is the maximum, the object of paraphrase of “I want” is “I want to get”.

  Next, the morphemes for generating paraphrased sentences are "this week", "no", "report", "show", "show", "te", "get me want", auxiliary verb "da", final particle " "".

  Next, a target sentence for paraphrasing is generated using Japanese syntax rule information. As a result, I would like to see you this week's report. Similarly, when the speaker attribute of the second language speaker is “young, male” or “young, female”, the target sentence is “Please show this week's report”.

  A second embodiment will be described with reference to the drawings. This example corresponds to the first embodiment for carrying out the present invention.

  Referring to FIG. 5, the speech synthesizing means 110 according to the second embodiment of the present invention uses the text output from the text converting means 106 as synthesized speech instead of the text output means 115 in the first embodiment shown in FIG. And a voice output unit 111 for outputting synthesized voice. The other points are the same as in the first embodiment.

  The speech synthesis unit 110 uses the second speaker feature extracted by the second speaker feature extraction unit 105 for the output from the text conversion unit 106, and the dictionary / model selection unit 112 uses the dictionary / model selection unit 112. A speech synthesis dictionary suitable for the second speaker feature is selected from the database 107, and speech synthesis processing is performed.

  For example, if the speaker on the second language side is an elderly person, the dictionary / model selection means 112 selects the dictionary / model for speech synthesis from the dictionary / model database 107 using the second speaker feature. By making the duration of the synthesized speech waveform slightly longer, and adjusting the sound pressure level and pitch frequency thresholds appropriately, you can slowly squeeze at a speed suitable for the elderly, increase the volume slightly, By providing a speech synthesis method capable of controlling the length, it has the effect of enhancing the practicality of synthesized speech.

  The voice output unit 111 is a voice output unit that outputs a synthesized voice through a speaker using the synthesized voice waveform signal output from the voice synthesis unit 110 as a voice signal.

  In the present embodiment, for example, in response to the input of “I want to see this week's report” in the example sentence of the first embodiment, in the present embodiment, the above paraphrase processing is performed. As a result, it is possible to perform speech synthesis by selecting a dictionary / model for speech synthesis using the speaker characteristics of the second speaker. As a result, if the second speaker is `` old man, male '' or `` old man, female '', by slightly increasing the duration of the synthesized speech waveform and adjusting the sound pressure level and pitch frequency threshold appropriately, Slowly speaking at a speed suitable for elderly people, slightly increasing the volume, controlling the pitch of the voice, can provide "Please show me this week's report" with the output of synthesized speech . Also, if the second speaker is "Young, Male" or "Young, Female", you can provide "Please show this week's report" with the output of synthesized speech, with a general speech synthesis effect .

  A third embodiment will be described with reference to the drawings. This example corresponds to the first embodiment for carrying out the present invention.

  As shown in FIG. 6, the present embodiment includes a machine translation unit 109 instead of the text conversion unit 106 of the first embodiment, and the other points are the same as those of the first embodiment.

  The machine translation means 109 is means for translating the input text from the text input means 102 into a second language. Various dictionaries necessary for machine translation are obtained from the dictionary / model database 107 by the dictionary / model selection unit 112 using the features of the second speaker extracted by the second speaker feature extraction unit 105. A translation dictionary suitable for the speaker's characteristics is selected and translated.

  For example, with reference to FIGS. 25 and 26, the output function of machine translation results suitable for the characteristics of the second speaker will be described.

  The data described in FIG. 25 is a Japanese-English translation dictionary for Japanese “name”, and the Japanese language as the source language has a data structure with “name” as an example. The source language field includes word heading information, reading information, standard notation information, part of speech information, sentence style information, and the like. The target language field includes reading information, part-of-speech information, information indicating target language speaker characteristics, sentence style information, and the like. Specifically, on the Japanese side, a dictionary entry for only the Japanese word “name” is “word heading (“ name ”)”, “reading (“ name ””), and “part of speech (noun)” which is part of speech information. (Common noun)) ”corresponds to“ E_reading (“name”) ”and“ part of speech (NOUN (c)) ”indicating part of speech information on the English side. In addition, “standard notation (“ your name is ””) ”which is a standard notation indicating a Japanese question sentence using the Japanese word“ name ”,“ part of speech (noun (common noun)) ”which is part of speech information, In contrast to the dictionary entry that consists of the information "J_sentence style (question)" indicating the sentence style of the Japanese question sentence, the English side, which is the target language, is "E_reading (" name ")" And “part of speech (NOUN (c))” indicating the part of speech information, a sentence style, source language speaker characteristics, target language speaker characteristics, and the like. The attribute values of the speaker features on the source language side and the speaker features on the target language side are the younger, older, Use attribute information composed of gender. Also, a set of attribute values such as young women, young women, senior men, and senior women can be used.

  For example, when the first speaker inputs "What's your name?", When the second speaker is a young man, the age and gender attribute values are "target language speaker characteristics (younger, children, younger In order to match “young man” in the “male” ”, the utterance of“ What is your name? ”From the first speaker, On the other hand, it can be translated as “What is your name?”. Similarly, when the attribute value of the second speaker is “Children, Young Men”, the translation result is “What is your name?” And the attribute value of the second speaker is “Mother Men”. When the translation result is `` May I have your name? '' And the attribute value of the second speaker is `` Mature woman, old man '', the translation result is `` Could you please tell me your name '' ? "And the attribute value of the second speaker is" young woman, youth woman ", the translation result is output as" I would like to ask you whether you would grant me permission to have your name? " be able to.

  Example 4 will be described with reference to the drawings. This example corresponds to the first embodiment for carrying out the present invention.

  Referring to FIG. 7, the speech synthesizer 110 according to the fourth embodiment of the present invention includes a speech synthesizer 110 for synthesizing the translation result output from the machine translator 109 in the third embodiment shown in FIG. And voice output means 111 for outputting synthesized voice. The other points are the same as in the third embodiment.

The machine translation means 109 uses the second speaker feature extracted by the second speaker feature extraction means 105 in response to the input from the text input means 101, and the dictionary / model selection means 112 uses the dictionary / database 107. The target language dictionary suitable for the characteristics of the second speaker is selected and machine translation is performed.
In the speech synthesis unit 110, the second speaker feature extracted by the second speaker feature extraction unit 105 is used for the translation result from the machine translation unit 109, and the second feature is extracted from the dictionary / model database 107. Speech synthesis is performed by selecting a speech synthesis dictionary and model to be applied to speaker characteristics.

  The voice output unit 111 outputs a synthesized voice through a speaker using the synthesized voice waveform signal output from the voice synthesis unit 110 as a voice signal.

  For example, in the case of English-Japanese translation, if the first speaker inputs "How old are you?", According to this example, the translation result is obtained when the Japanese side, the second speaker side, is a young woman. Translates to "How old are you?" And outputs synthesized speech. On the Japanese side, when the elderly, the translation result is translated as “How many?”, The volume of the synthesized speech is increased slightly, the pitch of the voice is increased slightly, and the voice is spoken slowly. The Japanese side can provide a synthesized voice with a gentle voice, translated as “How old?” When the child is a child.

  Example 5 will be described with reference to the drawings. This example corresponds to the first embodiment for carrying out the present invention.

  Referring to FIG. 8, in this embodiment, instead of the text input means 102 of the second embodiment, a voice input means 103 for inputting the voice of the first speaker is provided, and a voice recognition means 108 for recognizing the input voice is provided. I have. The other points are the same as in the second embodiment.

  The voice input means 103 takes in the voice of the speaker using a directional microphone to take in the voice and performs voice input analysis processing. The microphone is not limited to a place or environment, and particularly refers to a microphone incorporated in a portable terminal. The voice input analysis process is to perform various kinds of voice processing on the voice signal by taking in the voice data of the speaker in various environments in a mobile terminal such as an automatic voice interpreter. For example, processing to convert an analog signal into a digital signal at a sampling frequency of 8 kHz or 16 kHz, processing to convert a signal for performing speech recognition into parallel data, and store it in a register, noise proof processing, etc.

The voice recognition means 108 performs voice recognition on the voice signal output from the voice input means 103 and outputs a recognition result. The speech recognition processing performed by the speech recognition processing means 108 performs continuous speech recognition processing such as LPC speech analysis, speech section detection, pattern matching, and determination. The speech recognition method need not be a specific one, and an existing method that is generally used, such as an HMM, a neural network, or an N-gram language model, may be employed.
The dictionary / model selection of the speech recognition processing in the speech recognition means 108 is performed by the dictionary / model selection means 112 using the feature of the second speaker extracted by the second speaker feature extraction means 105. From the model group stored in the database 107, a speech recognition language model suitable for the characteristics of the second speaker is selected.

  For example, when the speaker feature of the second speaker extracted by the second speaker feature extraction means is “child, woman”, the speech recognition language model is selected for the speech from the first speaker. Using the attribute information of “children and women”, a speech recognition language model having a second speaker characteristic of “children and women” can be selected from the dictionary / model database for speech recognition. Specifically, the similarity between the second speaker's speaker attribute information (children, women) and the attribute information of various voice recognition dictionaries / models stored in the dictionary / model database is calculated, Select the one with the maximum similarity. In this way, it is possible to perform speech recognition by selecting a speech recognition language model having an attribute of “children, women”. Similarly, when the second speaker is a speaker having other attributes, a speech recognition language model that matches the attributes of the speaker is selected by the dictionary / model selection means, and speech recognition is performed. Recognition accuracy can be improved.

  In the text conversion processing 106, the output result from the speech recognition means 108 is stored in the dictionary / model database 107 by the dictionary / model selection means 112 by the dictionary / model selection means 112 using the characteristics of the second speaker. Text conversion processing is performed by selecting a dictionary / model for text conversion processing from among the existing model groups.

  For example, in the paraphrase dictionary for “apple”, the generation block for paraphrasing for children includes “apple”, “noun” part-of-speech information for reading information, semantic classification information for “fruit”, and “speaker's story”. Consists of second speaker feature information that is "person feature (children)", and the generation block for paraphrasing for adults and foreigners is part of speech information that is "Apple" and "noun" of reading information and "fruit" If it consists of semantic classification information and second speaker feature information that is “speaker features of the second speaker (adult, foreigner)”, the paraphrasing process in the text conversion processing means 106 determines the features of the second speaker. By referring to the input of “Eat Apple”, the second speaker is “Eat an apple” when the child is a child, and “Eat Apple” when the second speaker is an adult. In other words.

  Further, the speech synthesis unit 110 uses the second speaker feature acquired from the second speaker feature extraction unit 105 to the conversion result from the text conversion unit 106, and the dictionary / model selection unit 112 A dictionary / model for speech synthesis is selected from the model group stored in the dictionary / model database 107.

  Example 6 will be described with reference to the drawings. This example corresponds to the first embodiment for carrying out the present invention.

  Referring to FIG. 9, this embodiment includes machine translation means 109 instead of the text conversion means 106 of the fifth embodiment. The other points are the same as in the fifth embodiment.

  The voice input means 103 in the present embodiment performs voice input analysis processing by taking in the voice of the first speaker (speaker in the first language). The process for analyzing the input voice may be the same as the voice input means of the fifth embodiment.

  The voice recognition means 108 recognizes the input voice of the speaker in the first language. The voice recognition method may be processed in the same manner as the voice recognition process of the fifth embodiment.

  The dictionary / model selection of the speech recognition processing in the speech recognition means 108 is performed by the dictionary / model selection means 112 using the feature of the speaker in the second language extracted by the second speaker feature extraction means 105. A dictionary / model for speech recognition is selected from the model group stored in the model database 107. As a result, speech recognition suitable for the characteristics of the speaker of the second language can be performed.

  The machine translation means 109 outputs the output result from the speech recognition means 108 from the model group stored in the dictionary / model database 107 by the dictionary / model selection means 112 using the characteristics of the speaker of the second language. Translation processing is performed by selecting a dictionary / model such as a word dictionary for translation and a rule dictionary.

  For example, in the case of English-Japanese translation, if the speech recognition result of “Please eat the apple!” Is to be translated, the word “apple” in the English-Japanese translation word dictionary is assumed to contain information on the source language side. It consists of the block shown and two generated blocks. Specifically, the Japanese generation side reads the reading information of “apple”, the part-of-speech information of “noun”, the semantic classification information of “fruit”, and the attribute information of the speaker of the second language “story of the second speaker” The first Japanese generation block for children consisting of "person characteristics (children)", the reading information of "Apple", the part-of-speech information of "noun", the semantic classification information of "fruit", and the speaker of the second language The second Japanese generation block for adults composed of “speaker characteristics of the second speaker (adult)” which is attribute information of the second speaker. Similarly, the dictionary of the verb “eat” is composed of a source language dictionary block of “eat” and two target language generation blocks “eating” and “eating”. Each block is provided with reading information, part-of-speech information, semantic classification information, second speaker attribute information, and the like. When translating “Please eat the apple!”, A series of processes such as morphological analysis, syntax analysis, and target language generation processing are performed. When the target language is generated, the translation selection is performed by calculating the similarity to the speaker attribute of each translation word candidate using the second speaker characteristics. Choose “apples” for children and “Eat apples!”, Select “apples” for adults, and use polite grammar rules to provide translations of “please eat apples!” Can do.

  The speech synthesizer 110 may perform the same processing as the speech synthesizer 110 of the fifth embodiment.

  The audio output unit 111 may perform the same processing as the audio output unit 111 of the fifth embodiment.

  As described above, the speech of the first speaker (speaker of the first language) is input to the voice input means 103, and the second speaker feature acquired by the second speaker feature extraction means 105 is used. Then, the dictionary / model selection means 112 selects a dictionary / model for speech recognition from the dictionary / model database 107 and performs speech recognition. For the recognition result from the speech recognition means 108, the dictionary / model selection means 112 selects the machine translation dictionary from the dictionary / model database 107 using the characteristics of the second speaker, and performs machine translation processing. . For the translation result from the machine translation means 109, the dictionary / model selection means 112 selects the speech synthesis dictionary from the dictionary / model database 107 using the characteristics of the second speaker, and performs speech synthesis processing. . The output from the voice synthesis processing means 110 is output through voice output means such as a speaker.

  A seventh embodiment will be described with reference to the drawings. This example corresponds to the second embodiment for carrying out the present invention.

  Referring to FIG. 10, the present embodiment includes a text input means 102, a communication processing means 200, a first speaker feature extraction means 104, a second speaker feature extraction means 105, a dictionary / model. It comprises a database 107 and text output means 115.

  The communication processing unit 200 includes a text conversion unit 106, a dictionary / model selection unit 112, and a speaker relationship determination unit 113.

  Further, in this embodiment, various dictionaries / models for performing various processes of the communication processing means 200 are constructed in advance and stored in the dictionary / model database 107. For example, the utterance types for child men are `` utterance from old man to child man '' type, `` utterance from old woman to child man '' type, `` utterance from young man to child man '' type, It can be classified into the types of “utterance from young woman to child male” type, “utterance from young man to child male” type, and “utterance from young woman to child male” type. Similarly, the utterance types for children and women are the “speech from old man to child woman” type, the “speak from old woman to child woman” type, the “speak from young man to child woman” type, It can be classified into the types of “utterances from young women to child women”, “utterances from young men to child women”, and “utterances from young women to child women”. By using the language indicating the utterance type classified as described above, various voice recognition dictionaries / models for “utterance to children” can be constructed and stored in the dictionary / model database. Similarly, dictionaries / models for machine translation, text conversion, speech synthesis, etc. are also constructed in advance and stored in the dictionary / model database.

  In the text input means 101, the text output means 115, the dictionary / model database 107, and the second speaker feature extraction means 105, the parts corresponding to those in the first embodiment are configured by the same processing. To do.

  The first speaker feature extraction unit 104 may perform the same processing with the same configuration as the second speaker feature extraction unit 105 of the first embodiment.

  In the speaker relation determining means 113, the first speaker feature extracted by the first speaker feature extracting means 104 and the second speaker feature extracted by the second speaker feature extracting means 105 are used. To determine the speaker relationship. For example, when the personal attribute of the first speaker is “Youth, Male” and the personal attribute of the second speaker is “Old Man, Female”, the comparison result is the attribute of the speaker on the first language side. Is obtained from “characteristics of the first speaker (younger, youth male)” and attributes of the second speaker are derived from “characteristics of the second speaker (older, older women)”.

  The dictionary / model selection unit 112 selects a dictionary / model for text conversion from the dictionary / model database 107 using the speaker relationship between the two speakers based on the output from the speaker relationship determination unit 113.

  As described above, in the present embodiment, the characteristics of both speakers are extracted at the same time, and the dictionary / model selection unit 112 performs the text conversion dictionary based on the result indicating the relationship between the speakers through the speaker relationship determination unit 113.・ Select a model and perform text conversion processing considering the characteristics of both speakers.

  Example 8 will be described with reference to the drawings. This example corresponds to the second embodiment for carrying out the present invention.

  Referring to FIG. 11, the speech synthesis means 110 according to the eighth embodiment of the present invention uses the text output from the text conversion means 106 instead of the text output means 115 in the seventh embodiment shown in FIG. A voice synthesizing unit 110 that performs synthesized voice is provided, and a voice output unit 111 that outputs synthesized voice is provided. The other points are the same as in the seventh embodiment.

  The speech synthesizer 110 uses the determination result indicating the relationship between the two speakers output from the speaker relationship determination unit 113 with respect to the output from the text conversion unit 106, and the dictionary / model selection unit 112 performs the dictionary / A speech synthesis dictionary / model indicating the speaker relationship between the two speakers is selected from the model database 107, and parameters for speech synthesis processing are controlled.

  The voice output unit 111 outputs a synthesized voice through a speaker using the synthesized voice waveform signal output from the voice synthesis unit 110 as a voice signal.

  A ninth embodiment will be described with reference to the drawings. This example corresponds to the second embodiment for carrying out the present invention.

  Referring to FIG. 12, this embodiment includes a machine translation means 109 instead of the text conversion means 106 of the seventh embodiment. The other points are the same as in the seventh embodiment.

  The machine translation means 109 is means for translating the input text from the text input means 102 into a second language. Selection of various dictionaries necessary for machine translation is performed by outputting the speaker features extracted by the first speaker feature extraction unit and the second speaker feature extraction unit to the speaker relationship determination unit 113. Then, the speaker relationship is determined. Using the output result from the speaker relation determination means 113, a machine translation dictionary is selected from the dictionary / model database 107 and machine translation processing is performed.

  For example, if the present embodiment is introduced into the “conference interpreter system”, when translating a memo from a conference participant to a PC, a translation result that matches the utterance of a single speaker with the attributes of each listener is provided. Will be able to.

  Here, the explanation will be made assuming a conference interpretation system capable of interpreting Chinese and Japanese. Suppose that there are three speakers: a young male speaker A (Ichiro), an elderly female speaker B, and a young male speaker C. In addition, the translation dictionary stored in the dictionary database 107 includes the contents of “late life”, “scream”, and “Ichiro”.

  First of all, taking Chinese “late life” as an example, “late life” is a personal pronoun and can generally be used by younger people than listeners by both men and women. Therefore, attribute information “source language speaker characteristics (younger)” is assigned to the source language block of “late life”. The Japanese translation candidates corresponding to "Late" are "Watakushi", "I", "I", etc., so the dictionary for machine translation that can be constructed is "Late" and the characteristics of the original language speaker on the source language side Source language block composed of (younger), and on the generation side, the first target language generated from "Watakushi", part-of-speech information, and target language speaker characteristics (older, female) Block, “I”, part-of-speech information, etc., second target language generation block composed of information such as target language speaker characteristics (older, male), “I”, part-of-speech information, etc., purpose A dictionary is constructed with a third target language generation block composed of information such as language speaker characteristics (younger, male).

  Next, those who can use the Chinese verb “scream” have no relationship between age and gender, but on the Japanese side, they have different politeness, independent auxiliary verb “Da” and five-level verb “Sue” can do. Therefore, the translation dictionary of “scream” gives the attribute of the source language speaker feature (Φ) on the source language side. The meaning indicates that it can be used for all users. On the other hand, on the generation side, the first generation block composed of the heading information of the independent auxiliary verb “DA”, the specific part information, the part of speech information, the target language speaker characteristics (younger, male), etc., It is composed of a second generation block composed of heading information of the five-step verb “sue”, unique part information, part-of-speech information, and target language speaker characteristics (older, female). The noun “Ichiro” is a general translation dictionary and does not have to be given information on age or sex.

Next, a self-introduction example sentence “Yosei Shoichiro.” From speaker A will be explained as a translation example.
When translating this sentence into Japanese, first the Chinese morphological analysis and syntactic analysis were performed. As a result, the syntactic information consisted of the morpheme element pronoun “Late”, the verb “scream”, and the noun “Ichiro”. Is obtained. On the other hand, on the Japanese side, from the Chinese-Japanese translation dictionary, from the pronouns “Yangsei” to “Watakushi”, “I” and “I”, from the verb “scream” to “da” and “sue”, from the noun “Ichiro” Load “Ichiro”.

  Next, when the utterance of the speaker A is translated into the speaker B, first, the features of the two speakers are extracted by the first speaker feature extracting unit 104 and the second speaker feature extracting unit 105. . Next, using the information of speaker A's characteristics “young man and man” and speaker B's characteristic “old man and woman”, the speaker relationship determination means compares the ages of both speakers, and the features of the original language speaker. The results of (younger) and target language speaker characteristics (older) are obtained. Furthermore, the first speaker's speaker feature “Youth, male, source language speaker characteristics (younger)” and the second speaker's speaker feature “older, female, target language speaker features (year) The above is used to calculate the similarity with the attribute information of the translation dictionary, and the one with the maximum similarity is selected. As a result, the translation of “Watakushi” is selected from the three translation candidates of “late life”. Similarly, “speak” is selected from the translation word candidates of “scream”.

  Therefore, the morphemes on the Japanese language generation side consist of “Watakushi”, “Sue”, and “Ichiro”. Next, the target language is generated using the case frame of the five-step verb “Sue” and the Japanese syntax generation rules, and the translation result “I say Ichiro Watakushi” can be presented to speaker B.

  Similarly, when the utterance of the speaker A is translated into the speaker C, the translation result “I am Ichiro” can be presented to the speaker C.

  A tenth embodiment will be described with reference to the drawings. This example corresponds to the second embodiment for carrying out the present invention.

  Referring to FIG. 13, the speech synthesis means 110 according to the ninth embodiment of the present invention is output from the machine translation means 109 instead of the text output means 115 in the eighth embodiment shown in FIG. The speech synthesis means 110 for synthesizing the translation result and the speech output means 111 for outputting the synthesized speech are provided. The other points are the same as in the eighth embodiment.

  In FIG. 13, the first speaker feature extraction means 104 extracts the first speaker feature. The second speaker feature extraction means 105 extracts the features of the second speaker. The relationship between the two speakers is determined by the speaker relationship determining means 113 using the extracted speaker characteristics of the first and second speakers. Using the determined speaker relation result, the dictionary / model selection means 112 selects a dictionary for machine translation from the dictionary / model database 107 and uses it for the machine translation means 109 to perform translation processing. Next, for the result of machine translation, using the determined speaker relation result, the dictionary / model selection means 112 selects a dictionary / model for speech synthesis from the dictionary / model database 107 and performs speech synthesis. . The synthesized speech from the speech synthesizer 110 is output through the speech output unit 111.

  Example 11 will be described with reference to the drawings. This example corresponds to the second embodiment for carrying out the present invention.

  Referring to FIG. 14, in this embodiment, instead of the text input means 102 of the eighth embodiment, voice input means 103 for inputting the voice of the first speaker is provided, and voice recognition means 108 for recognizing the input voice is provided. I have. The other points are the same as in the eighth embodiment.

  The voice input means 103 takes in the voice of the first speaker and performs voice input analysis processing. The process for analyzing the input voice may be the same as the voice input means of the fifth embodiment.

  The voice recognition means 108 recognizes the input voice of the speaker in the first language. The voice recognition method may be processed in the same manner as the voice recognition process of the fifth embodiment.

  The selection of the dictionary / model of the speech recognition processing in the speech recognition means 108 is extracted by the features of the first speaker extracted by the first speaker feature extraction means 104 and the second speaker feature extraction means 105. Using the characteristics of the second speaker, the speaker relationship determination unit 113 determines the speaker relationship, and using the determination result, the dictionary / model selection unit 112 stores it in the dictionary / model database 107. Select a dictionary / model for speech recognition from a group of models.

  In the text conversion means 106, the output result from the speech recognition means 108 is stored in the dictionary / model database 107 by the dictionary / model selection means 112 using the speaker relation acquired by the speaker relation determination means 113. A text conversion process is performed by selecting a paraphrase word dictionary, a conversion rule dictionary, or the like from a group of models.

  The voice synthesis unit 110 and the voice output unit 111 can perform the same processing as the voice synthesis unit 110 and the voice output unit 111 of the fifth embodiment.

  Example 12 will be described with reference to the drawings. This example corresponds to the second embodiment for carrying out the present invention.

  Referring to FIG. 12, this embodiment includes machine translation means 109 instead of the text conversion means 106 of the eleventh embodiment. The other points are the same as in the eleventh embodiment.

  The voice input means 103 and the voice recognition means 108 can perform processing with the same configuration as the voice input means 103 and the voice recognition means 108 of the eleventh embodiment.

  In the machine translation unit 109, the speaker relationship determination unit 113 determines the speaker relationship, and using the determination result, the dictionary / model selection unit 112 uses the determination result to store among the model groups stored in the dictionary / model database 107. A machine translation processing is performed by selecting a word dictionary for machine translation, a translation rule dictionary, a translation model, and the like.

  For example, in the case of Chinese-Japanese translation, when the speech recognition result of “sentence 2 in FIG. 33” is translated, the age of the speaker is displayed on the generation side of the word “word 3 in FIG. 33” in the dictionary dictionary for Chinese-Japanese translation. In addition, there is a dual-purpose language generation block for “younger” listeners, “~ please” and “older” for older speakers than “speaker's age”. Two generation blocks can be provided: “eat” (younger) and “eating” (older). In the machine translation process, “sentence 2 in FIG. 33” is subjected to morphological analysis and syntax analysis, and the rule selection for generating Japanese indicates the speaker relationship acquired by the speaker relationship determining means 113. Select using the results. Then, as a result of translating the input sentence of “sentence 2 in FIG. 33” into Japanese, “please eat apples” and “please eat apples” can be generated. Needless to say, when the second speaker is “younger” than the speaker, “Please eat apple” is generated, and when the second speaker is “older” than the speaker, "Please enjoy" is generated.

  Example 13 will be described with reference to the drawings. Such an example corresponds to the third embodiment for carrying out the present invention.

  Referring to FIG. 16, in this embodiment, a text input means 102, a first speaker feature extraction means 104 for extracting a first speaker feature, and a second story for extracting a second speaker feature. A feature extraction unit 105, a communication processing unit 300, a dictionary / model database 107, a speaker interaction history database 310, and a text output unit 115 are included. The communication processing unit 300 in this embodiment includes a text conversion unit 106, a dictionary / model selection unit 112, and a dialogue history management unit 114.

The text input means 102, text output means 115, first speaker feature extraction means 104, second speaker feature extraction means 105, and dictionary / model database 107 in this embodiment shown in FIG. Can be realized by the same construction method as the corresponding part of the twelfth embodiment.
The speaker's dialog history database 310 records the speaker's dialog history in chronological order along with the speaker characteristics of both speakers.

  Next, the communication processing means 300 in the present embodiment will be described with reference to the drawings. The speaker relationship determining means 113, the dictionary model selecting means 112, and the text converting means 106 in the communication processing means 300 in this embodiment are realized by the same construction method as the corresponding parts in the twelfth embodiment. can do. As shown in FIG. 29, the dialogue management unit 114 includes a dialogue history analysis unit 2901, a speaker feature determination unit 2902, and a feedback processing unit 2903.

  The dialogue history analysis means 2901 extracts the conversation history of both speakers from the speaker dialogue history database 310 based on the characteristics and the speaker relations of both speakers output from the speaker relationship judgment means 113, and the speaker A process including generating a vector including a feature, a dialog style, a keyword of a dialog content, or a model in which the vector is time-sequentially performed.

  In the speaker feature determination unit 2902, the first speaker feature extraction unit 104 and the second speaker feature are used using the feature vector or model of the dialogue history generated by the speaker dialogue history analysis unit 2901. It is determined whether or not the determination result regarding the speaker characteristics extracted by the extraction unit 105 and the speaker relationship output from the speaker relationship determination unit 113 is correct.

  For example, a feature quantity indicating the language expression of the Japanese “female word” and a feature quantity indicating the language expression of the “male language” are prepared and stored in the speaker dialogue history database 310 for dialogue. By comparing with the feature vector of the speaker obtained from the history analysis means 2901, the gender of the speaker can be determined.

  The feedback processing unit 2903 performs feedback processing on the speaker characteristics when it is determined that “there is an error” in the determination result from the speaker feature determination unit 2902.

  For example, the first speaker and the second speaker were both assumed to be male, but the input sentence of the first speaker was "Thank you for bringing the key. Thank you. I often forget things recently. ”When the speaker feature determination means 2902 determines that the first speaker is“ female ”and performs feedback processing, Gender attribute values can be corrected.

  In this way, by using information obtained from the state and progress of the speaker's dialogue, it is possible to automatically determine whether or not the speaker's personality is “correct” by automatically determining the speaker's personal attribute error. A function of detecting and correcting can be realized.

  Next, a fourteenth embodiment for carrying out the invention will be described with reference to the drawings. Such an example corresponds to the third embodiment for carrying out the present invention.

  This embodiment is shown in FIG. Referring to FIG. 17, the speech synthesizing unit 110 according to the fourteenth embodiment is a voice that synthesizes the text output from the text converting unit 106 instead of the text output unit 115 in the thirteenth embodiment shown in FIG. A synthesis unit 110 is provided, and a voice output unit 111 that outputs a synthesized voice is provided. The other points are the same as in the thirteenth embodiment.

  The voice synthesizing means 110 and the voice output means 111 are the same as the corresponding parts in the twelfth embodiment.

  In this way, for text input from the first speaker, the result of text conversion in consideration of the characteristics of the first speaker, the characteristics of the second speaker, and the conversation history of both speakers is synthesized speech. Can be output.

  Example 15 will be described with reference to the drawings. Such an example corresponds to the third embodiment for carrying out the present invention.

  This embodiment is shown in FIG. Referring to FIG. 18, the present embodiment includes a machine translation means 109 instead of the text conversion means 106 of the thirteenth embodiment. The other points are the same as in the thirteenth embodiment.

  The machine translation means 109 is means for translating the input text from the text input means 102 into a second language. Selection of various dictionaries necessary for machine translation is performed by outputting the speaker features extracted by the first speaker feature extraction unit and the second speaker feature extraction unit to the speaker relationship determination unit 113. Then, the speaker relationship is determined. In response to the output result from the speaker relationship determination means 113, the conversation history of the speaker is extracted from the speaker conversation history database, and the dialog management means 114 determines whether or not the speaker characteristics are correct. Using the correct speaker characteristics and speaker relationships output from the dialogue management means 114, the dictionary / model selection means 112 selects a machine translation dictionary from the dictionary / model database 107 and performs machine translation processing.

  As described above, for the text input from the first speaker, the result of machine translation processing considering the characteristics of the first speaker, the characteristics of the second speaker, and the conversation history of both speakers is provided. be able to.

  Example 16 will be described with reference to the drawings. Such an example corresponds to the third embodiment for carrying out the present invention.

  This embodiment is shown in FIG. Referring to FIG. 19, in this embodiment, the speech synthesis means 110 according to the fifteenth embodiment uses the translation result output from the machine translation means 109 instead of the text output means 115 in the fifteenth embodiment shown in FIG. A voice synthesizing unit 110 that performs synthesized voice processing is provided, and a voice output unit 111 that outputs synthesized voice is provided. The other points are the same as those in the fifteenth embodiment.

  The voice synthesizing means 110 and the voice output means 111 are the same as the corresponding parts in the fourteenth embodiment.

  In this way, for the text input from the first speaker, machine translation results that take into account the characteristics of the first speaker, the characteristics of the second speaker, and the conversation history of both speakers are output as synthesized speech can do.

  Example 17 will be described with reference to the drawings. Such an example corresponds to the third embodiment for carrying out the present invention.

  This embodiment is shown in FIG. Referring to FIG. 20, the voice synthesizing means 110 according to the present embodiment includes voice input means 103 for inputting the voice of the first speaker, instead of the text input means 102 in the fourteenth embodiment shown in FIG. The voice recognition means 108 for recognizing the input voice is provided. The other points are the same as those in Example 14.

  In this way, for speech input from the first speaker, the result of text conversion processing considering the characteristics of the first speaker, the characteristics of the second speaker, and the conversation history of both speakers is synthesized speech. Can be output.

  Example 18 will be described with reference to the drawings. Such an example corresponds to the third embodiment for carrying out the present invention.

  This embodiment is shown in FIG. Referring to FIG. 21, this embodiment includes machine translation means 109 instead of the text conversion means 106 of the seventeenth embodiment. The other points are the same as in the seventeenth embodiment.

  The machine translation means 109 is means for translating the input text from the text input means 102 into a second language. Selection of various dictionaries necessary for machine translation is performed by outputting the speaker features extracted by the first speaker feature extraction unit and the second speaker feature extraction unit to the speaker relationship determination unit 113. Then, the speaker relationship is determined. In response to the output result from the speaker relationship determination means 113, the conversation history of the speaker is extracted from the speaker conversation history database, and the dialog management means 114 determines whether or not the speaker characteristics are correct. Using the correct speaker characteristics and speaker relationships output from the dialogue management means 114, the dictionary / model selection means 112 selects a machine translation dictionary from the dictionary / model database 107 and performs machine translation processing.

In this way, for speech input from the first speaker, machine translation results that take into account the characteristics of the first speaker, the characteristics of the second speaker, and the conversation history of both speakers are output as synthesized speech can do.
Note that the application of each embodiment and example in the present invention is not limited to a single computer device or system, and a system can be configured by a plurality of terminals or computers.
For example, corresponding to the second embodiment, the input means 101 and the first speaker feature extraction means 104 are used as the first terminal, and the output means 111 and the second speaker feature extraction means 105 are used as the second terminal. It is easy to arrange the communication processing means 200 in the terminal on the server computer so that each terminal and the server computer execute processing while communicating with each other via a network.

  The communication system of the present invention is a text and voice output method including a function for realizing each module, a voice communication program for causing an electronic device or the like to execute each procedure, and an electronic device in which these programs are recorded. The present invention can be provided by a recording medium, a program product that can be incorporated in an internal memory of an electronic device that includes these programs, a computer such as a portable terminal or a server that includes the program, a voice interpretation device, and the like.

  According to the present invention, as described above, the communication between the speakers in the same language or in different language systems in the voice dialogue system, the video conference system, the videophone automatic interpretation system, or the voice interpretation system in the communication system of both speakers is explained. Assistance can be performed. In addition, the present invention uses a speaker's personal attributes for speech recognition, paraphrasing, machine translation, and speech synthesis, thereby providing users with high speech recognition accuracy, machine translation accuracy, abundant speech synthesis functions, and versatile language expression functions. Can be provided. In particular, the present invention can realize smooth communication between speakers by constructing a system as needed for speakers of the same language as well as speakers of different language systems. it can.

It is a block diagram showing a schematic structure of an information processing system in a first embodiment of the present invention. It is a block diagram which shows schematic structure of the information processing system in 2nd embodiment of this invention. It is a block diagram which shows schematic structure of the information processing system in 3rd embodiment of this invention. FIG. 3 is a block diagram showing a schematic configuration of Example 1 in the first embodiment of the present invention. FIG. 3 is a block diagram showing a schematic configuration of Example 2 in the first embodiment of the present invention. FIG. 6 is a block diagram showing a schematic configuration of Example 3 in the first embodiment of the present invention. FIG. 7 is a block diagram showing a schematic configuration of Example 4 in the first embodiment of the present invention. FIG. 10 is a block diagram showing a schematic configuration of Example 5 in the first embodiment of the present invention. FIG. 10 is a block diagram showing a schematic configuration of Example 6 in the first embodiment of the present invention. FIG. 6 is a block diagram showing a schematic configuration of Example 1 in the second embodiment of the present invention. It is a block diagram which shows schematic structure of Example 2 in 2nd embodiment of this invention. FIG. 10 is a block diagram showing a schematic configuration of Example 3 in the second embodiment of the present invention. It is a block diagram which shows schematic structure of Example 4 in 2nd embodiment of this invention. FIG. 10 is a block diagram showing a schematic configuration of Example 5 in the second embodiment of the present invention. It is a block diagram which shows schematic structure of Example 6 in 2nd embodiment of this invention. It is a block diagram which shows schematic structure of Example 1 in 3rd embodiment of this invention. It is a block diagram which shows schematic structure of Example 2 in 3rd embodiment of this invention. It is a block diagram which shows schematic structure of Example 3 in 3rd embodiment of this invention. It is a block diagram which shows schematic structure of Example 4 in 3rd embodiment of this invention. It is a block diagram which shows schematic structure of Example 5 in 3rd embodiment of this invention. It is a block diagram which shows schematic structure of Example 6 in 3rd embodiment of this invention. It is a flowchart which shows the algorithm of 1st embodiment in this invention. It is a flowchart which shows the algorithm of 2nd embodiment in this invention. It is a flowchart which shows the algorithm of 3rd embodiment in this invention. It is an example which shows the kind of Japanese-English machine translation dictionary structure in 2nd Embodiment. FIG. 26 is a diagram showing a translation result that can be generated from the dictionary example of FIG. It is a figure which shows the identification information of the model for speech recognition. It is an Example which shows the Japanese style information of the dictionary for machine translation stored in a dictionary and model database. It is a schematic block diagram of the conventional speech recognition apparatus. It is a schematic block diagram which reads out the conventional mail text and outputs a synthetic voice. It is a schematic block diagram of the conventional voice response apparatus. It is a schematic block diagram of the conventional voice response apparatus. It is a figure for demonstrating this invention.

Explanation of symbols

100 Communication processing means
101 Input means
102 Text input means
103 Voice input means
104 First speaker feature extraction means
105 Second speaker feature extraction means
106 Text conversion means
107 Dictionary / Model Database
108 Voice recognition means
109 Machine translation means
110 Speech synthesis means
111 Audio output means
112 Dictionary / model selection means
113 Speaker relation judging means
114 Dialog history management means
115 Text output means
116 Output means
200 Communication processing means
300 Communication processing means
310 speaker dialogue history database

Claims (37)

An information processing system for processing communication from a first speaker to a second speaker,
Second speaker feature extraction means for extracting features of the second speaker;
An information processing system comprising: communication processing means for processing input data from the first speaker based on the characteristics of the second speaker. First speaker feature extraction means for extracting features of the first speaker;
The information according to claim 1, wherein the communication processing unit processes input data from the first speaker based on the characteristics of the first speaker and the characteristics of the second speaker. Processing system. The communication processing means includes
Speaker relation determining means for determining speaker relations based on the characteristics of the first speaker and the characteristics of the second speaker;
The information processing system according to claim 2, wherein input data from the first speaker is processed with reference to the determined relationship of the speakers. A conversation history database that stores the conversation history of the speaker;
The communication processing means processes input data from the first speaker based on the characteristics of the first speaker or the characteristics of the second speaker and the conversation history of the conversation history database speaker. The information processing system according to claim 2 or claim 3, wherein The communication processing means includes
Dictionary and model database,
A dictionary / model selection means for selecting a dictionary or model from a dictionary / model database based on the characteristics of the second speaker;
5. Data conversion means for converting input data from the first speaker into data suitable for the second speaker using the selected dictionary or model. An information processing system according to any one of the above.   The data conversion means is a text conversion means for converting input data from a first speaker into text suitable for a second speaker using a selected dictionary or model. 5. The information processing system according to 5.   The data conversion means is voice synthesis means for performing voice synthesis suitable for a second speaker on input data from a first speaker using a selected dictionary or model. 5. The information processing system according to 5.   The data conversion means is a translation means for translating input data from the first speaker into a language expression suitable for the second speaker using the selected dictionary or model. 5. The information processing system according to 5.   The data converting means translates input data from the first speaker into a language expression suitable for the second speaker using the selected dictionary or model, and the translated language expression is converted into the second speaker. The information processing system according to claim 5, wherein the information processing system is a translation / speech synthesizer that performs speech synthesis suitable for the user.   The information processing system according to any one of claims 1 to 9, further comprising voice recognition means for voice recognition of the voice of the first speaker and outputting as input data.   11. The information processing according to claim 3, wherein the communication processing unit selects a dictionary or a model from the dictionary / model database based on the determination by the speaker relation determination unit. system.   The communication processing means is a dialog management that performs collation with various patterns obtained by analyzing the conversation history of the speaker from the conversation history database of the speaker based on the speaker feature extracted from the speaker feature extraction means. The information processing system according to claim 3, further comprising: means.   13. The speaker feature extraction unit according to claim 1, wherein the first speaker feature extraction unit or the second speaker feature extraction unit extracts speaker features from the speech data of the speaker. Information processing system.   The said 1st speaker feature extraction means or a 2nd speaker feature extraction means extracts the feature of a speaker from a speaker's face image, The Claim 1 characterized by the above-mentioned. Information processing system.   The information processing system according to any one of claims 1 to 12, wherein the first speaker feature extraction unit or the second speaker feature extraction unit extracts speaker features by a sensor. .   The first speaker feature extracting unit or the second speaker feature extracting unit includes an IC card capable of communicating the personal attribute of the speaker, and a terminal device built in the IC card capable of communicating the personal attribute of the speaker. The information processing system according to any one of claims 1 to 12, wherein speaker characteristics are extracted by using the IC card or the terminal.   The first speaker feature extraction unit or the second speaker feature extraction unit outputs personal attributes indicating the gender, age, race, body posture, or biological information of the speaker. The information processing system according to claim 12. A processing method for processing communication from a first speaker to a second speaker,
A first speaker feature extraction process for extracting features of the second speaker;
And a communication process for processing input data from the first speaker based on the characteristics of the second speaker. A second speaker feature extraction process for extracting features of the first speaker;
The processing method according to claim 18, wherein the communication processing processes input data from a first speaker based on characteristics of the first speaker and characteristics of the second speaker. . The communication process is:
A speaker relationship determination process for determining a speaker relationship based on the characteristics of the first speaker and the characteristics of the second speaker;
The processing method according to claim 19, wherein input data from the first speaker is processed with reference to the determined relationship of the speakers. The communication process is characterized in that input data from the first speaker is input based on the characteristics of the first speaker or the characteristics of the second speaker and the conversation history of the speaker. Item 20. The processing method according to Item 19 or Item 20.   The communication processing means selects a dictionary or model from a dictionary / model database based on the characteristics of the second speaker, and uses the selected dictionary or model to input data from the first speaker. The processing method according to any one of claims 18 to 21, further comprising a data conversion process for converting the data into data suitable for the second speaker.   23. The data conversion process according to claim 22, wherein the data conversion process is a process of converting input data from the first speaker into a text suitable for the second speaker using the selected dictionary or model. The processing method described.   23. The process according to claim 22, wherein the data conversion process is a process of performing speech synthesis suitable for the second speaker on the input data from the first speaker using the selected dictionary or model. The processing method described.   The data conversion process is a process of translating input data from the first speaker into a language expression suitable for the second speaker, using the selected dictionary or model. The processing method as described in.   In the data conversion process, using the selected dictionary or model, the input data from the first speaker is translated into a language expression suitable for the second speaker, and the translated language expression is translated into the second speaker. The processing method according to claim 22, wherein the processing is a speech synthesis suitable for the user.   27. The processing method according to claim 18, further comprising a voice recognition process for recognizing the voice of the first speaker and outputting the voice as input data.   28. The processing method according to claim 20, wherein in the communication process, a dictionary or a model is selected from a dictionary / model database based on the determination in the speaker relation determination process.   The communication process includes a dialog management process that performs verification with various patterns obtained by analyzing the conversation history of the speaker from the conversation history database of the speaker based on the extracted speaker characteristics. The processing method according to any one of claims 21 to 28.   30. The processing method according to claim 18, wherein a feature of the first speaker or a feature of the second speaker is extracted from the voice data of the speaker.   30. The processing method according to claim 18, wherein a feature of the first speaker or a feature of the second speaker is extracted from the face image of the speaker.   30. The processing method according to claim 18, wherein the feature of the first speaker or the feature of the second speaker is extracted by a sensor.   By using the IC card capable of communicating the personal attribute of the speaker, the terminal device built in the IC card capable of communicating the personal attribute of the speaker, the characteristics of the first speaker or 30. The processing method according to claim 18, wherein a feature of the second speaker is extracted.   30. The processing method according to claim 18, wherein a personal attribute indicating a speaker's sex, age, race, body posture, or biological information is output. A program of an information processing system that executes processing of communication from a first speaker to a second speaker,
A first speaker feature extraction process for extracting features of the second speaker;
A program for causing an information processing system to execute communication processing for processing input data from a first speaker based on characteristics of the second speaker. Causing the information processing system to execute a second speaker feature extraction process for extracting features of the first speaker;
36. The program according to claim 35, wherein the communication processing processes input data from the first speaker based on the characteristics of the first speaker and the characteristics of the second speaker. The communication process is:
A process of determining speaker relationships based on the characteristics of the first speaker and the characteristics of the second speaker;
37. The program according to claim 36, further comprising a process of processing input data from a first speaker with reference to the determined speaker relationship.

Images