JPH02183371A - Automatic interpreting device - Google Patents

Abstract

PURPOSE:To transmit the emotion of a speaker and to improve translation accuracy by extracting emotional information from the picture of the speaker and executing sound synthesization with translation corresponding to these emotional information. CONSTITUTION:The expression of the speaker is recognized with suitable time scale in a picture recognization device through a camera 11a and in a change extraction device 6, a necessary target such as the individual change of the motion of a hand, the motion of a face, the motion of eyebrows, the change of eyes and the change of a mouth, etc., is read and totally judged in an emotion extraction device 7. Then, the prescribed emotional information are selected. The sound of the speaker is recognized by a sound recognization part 1 and translated in a machine translation part 2. At such a time, the commonsense mode of expression stored to a knowledge base 4 or the mode of expression corresponding to the emotional information extracted from the emotion extraction device 7 are selected. Then, the sound is synthesized in a sound synthesization part 3, the strength and pitch of the sound are adjusted so as to be matched with the emotional information and the sound synthesization is executed. Thus, the translated sound matched with the emotion of the speaker is outputted.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an automatic interpreting device for interpreting a spoken language of a speaker into another spoken language.

[Prior Art] Recently, with the development of speech recognition technology, synthesis technology, and machine translation technology, automatic speech translation has become possible at a certain level, and its practical use is nearing.

This type of automatic interpretation device uses a machine translation section to perform machine translation based on the recognition results of a speech recognition section that recognizes the speaker's voice, and then synthesizes this translation result into speech using a speech synthesis section and outputs it. There is something.

[Problems to be Solved by the Invention] By the way, with the above-mentioned conventional automatic interpreting devices, it is not possible to output translated speech that incorporates the speaker's accurate feelings and emotions, so it is difficult to convey these emotions. However, the disadvantage was that the level of translation accuracy was low.

Therefore, an object of the present invention is to make it possible to output translated speech that incorporates the speaker's emotions.

[Means for Solving the Problems] Technical means of the present invention for solving these problems include a voice recognition means for recognizing the voice of a speaker, a facial expression recognition means for recognizing the facial expressions of the speaker, an emotion extraction means for extracting emotional information corresponding to a change in facial expression recognized by the facial expression recognition means; a machine translation means for machine translation based on the voice recognition result and the emotion information; and a translation result of the machine translation means and the emotion. An automatic interpreting device includes a speech synthesis means for synthesizing speech based on information.

[Embodiment] Hereinafter, an automatic interpretation device according to an embodiment of the present invention will be described based on the accompanying drawings.

As shown in FIG. 1, the automatic interpreting device according to the embodiment includes a speech recognition device 1 that recognizes the speaker's voice input manually from a microphone 10, etc., and facial expression recognition means that recognizes the speaker's facial expression. ing. The facial expression recognition means includes an image recognition device 5 that is connected to a video device 11 that exchanges images between the speaker and the receiver, and that receives and recognizes an image of the speaker from a camera lla. The image recognition device 5 is also connected to emotion extraction means for extracting emotional information corresponding to changes in the speaker's facial expressions that the device 5 fails to recognize. This emotion extraction means includes a change extraction device 6 that extracts changes in the speaker's facial expressions, such as hand movements, facial movements, eyebrow/eye/mouth movements, etc.;
A knowledge base 8 that stores predetermined emotional information corresponding to these changes in facial expressions, and an emotion extraction device 7 that extracts and outputs emotional information from the knowledge base 8 that corresponds to the changes in facial expressions. It is configured. This automatic interpretation device also includes a machine translation device 2 that performs a predetermined machine translation based on the voice recognition result of the speech recognition device 1 and the emotional information, and a machine translation device 2 that performs predetermined machine translation based on the translation result of the machine translation device 2 and the emotional information. It is equipped with a voice synthesis device 3 that synthesizes voice and outputs it to a speaker. The machine translation device 2 has a knowledge base 4 that has accumulated phrases corresponding to speech recognition results and emotional information.
The relevant data is selected from the list.

Therefore, according to the automatic interpreting device according to this embodiment, the speaker's voice is recognized by the voice recognition unit 1. Usually, it is sent to the machine translation unit 2 in units of words or phrases. Machine translation device 2 performs syntactic analysis and semantic analysis.

Translate by performing context analysis, etc. At this time, common-sense expressions accumulated in the knowledge base 4 and expressions corresponding to the emotional information extracted from the emotion extraction device 7 are selected. The selected expression is synthesized into speech by the speech synthesis section 3, and at this time too, the strength and pitch of the speech are adjusted to match the emotional information extracted from the emotion extraction device 7, and the speech synthesis is performed.

To explain in more detail, the process by which emotion information is extracted by the emotion extraction device 7 is as follows. The speaker's facial expression is recognized by the image recognition device 5 at an appropriate time scale through the camera lla. The recognized image is sent to the change extraction device 6
Main targets include hand movements, facial movements, nodding, negative sideways movements, etc.), and eyebrow movements.

Individual changes such as changes in eyes (tears) and changes in one mouth (laughter) are read. The read individual targets are comprehensively judged by the emotion extraction device 7 and predetermined emotion information is selected. This emotional information is stored in the knowledge base 8 as conditional knowledge for each target.

Therefore, the translation is performed using phrasing that fits the speaker's emotions, and the translated voice is output with intonation and pitch that match the speaker's emotions.

[Effects of the Invention] As explained above, according to the automatic interpreting device of the present invention, emotional information is extracted from the image of the speaker, and translation and speech synthesis corresponding to this emotional information are performed. The translation accuracy improves to the extent that this emotion can be conveyed.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of an automatic interpretation device according to an embodiment of the present invention. : Voice recognition device 2: Machine translation device = r1 Voice synthesis device 4: Knowledge base for machine translation 5: Image recognition device 6: Change extraction device 7: Emotion extraction device 8: Knowledge base for emotion extraction

Claims (1)

[Claims] a voice recognition means for recognizing the voice of the speaker; a facial expression recognition means for recognizing the facial expression of the speaker; an emotion extraction means for extracting emotional information corresponding to a change in the facial expression recognized by the facial expression recognition means; An automatic interpreting device comprising: a machine translation device that performs machine translation based on the result and emotional information; and a speech synthesis device that synthesizes speech based on the translation result of the machine translation device and the emotional information.