JP3926365B2 - Audio converter - Google Patents

Description

The present invention is, for example, relates to speech conversion equipment that outputs sound by converting the utterance of the speech input original language into the target language.

  Voice interpretation technology was developed as software premised on the use of high-performance workstations and personal computers. If the scope of conversation is limited to travel conversations, the performance has reached a practical level. . However, as a voice interpreting device, for ordinary users to use on a daily basis, hardware that is large enough to be easily carried on overseas trips, etc. and a user interface that can be easily operated are designed to be equivalent. Functional software needs to be ported to the hardware.

  Conventionally, the work of porting voice interpretation software to a B5 size notebook personal computer has been promoted.

  On the other hand, recent advances in hardware technology have made it possible to implement a translation function by voice input mainly for conversations used on overseas trips using portable information devices. Such other conventional translation functions are bidirectional, for example, having both a Japanese-to-English conversion function and an English-to-Japanese conversion function.

Such other prior art inventions include a foreign language translation device (see, for example, Patent Document 1) and a speech input translation device (see, for example, Patent Document 2). In these inventions, the shape of the device, the arrangement of the display unit, and the contents thereof are determined so that two people having different languages can interact using one device in a face-to-face format.
JP-A-8-77176 JP-A-8-278972

  However, notebook personal computers of about B5 size are not large enough for users to carry around and use in various places. Moreover, since it must be operated with a normal keyboard or mouse, it is not easy to use as a user interface. Furthermore, the amount of computational resources such as CPU performance and working memory capacity required for speech recognition is generally proportional to the size of the recognition target vocabulary.

  Since the computational resources are limited in small hardware, it is difficult to implement words necessary and sufficient as a speech interpretation device as a recognition target vocabulary, and there is a problem that the utility value as a speech interpretation device is lowered. The above is the subject concerning the above prior art (first subject).

  Next, a problem with the other prior art will be described (second problem).

  That is, in the above-described other prior art translation device, if the resolution of the display area is small with a small information device that fits in the pocket of clothes, all necessary information for two users is displayed. I can't. Therefore, there has been a problem that usability as a translation device is reduced. In addition, when a plurality of display units are mounted, there is a problem that power consumption increases and the operation time of the translation apparatus is shortened. In addition, since the translation device does not handle the utterances of two users exclusively, there is a problem that when the utterances overlap, the recognition rate of speech recognition is lowered and the performance as the translation device is lowered.

The present invention, in consideration of such a second problem of the conventional translation apparatus, and an object thereof is to provide a voice conversion equipment that can be improved usability of the display contents in comparison with the conventional .

  In addition, in consideration of the first problem of the above-described conventional speech interpretation apparatus, other inventions related to the present invention can be further downsized and can be easily operated. An object is to provide a voice conversion device, a voice conversion method, a program, and a medium.

The first aspect of the present invention is an input unit for inputting voice in the first or second language;
(1) When the voice of the first language is received from the input unit, the voice is recognized and, based on a predetermined control instruction, (1-a) the voice-recognized first language Or (1-b) converting the conversion target determined based on the speech recognition result into the second language, and outputting at least the notation data in the converted language. (2) When the second language voice is received from the input unit, the voice is recognized and, based on a predetermined control instruction, (2-a) the second voice recognized Output language notation data, or (2-b) convert the conversion target determined based on the speech recognition result to the first language, and output at least the notation data of the converted language A speech translator to
A support unit for supporting the determination of the conversion target of the speech translation unit;
A display unit for displaying the notation data of the converted language output from the speech translation unit based on the predetermined control instruction;
A control unit that performs the predetermined control instruction on at least the speech translation unit and the display unit ;
Language conversion direction detection for detecting information for determining the translation direction of which translation from the first language to the second language or from the second language to the first language should be performed by the above Part
The control unit specifies the translation direction for the speech translation unit based on the detection result, and controls the input unit,
The language conversion direction detection unit is composed of a gyro sensor installed inside the voice conversion device,
It is a speech conversion device that selects the translation direction at a position where a uttering user holds the speech conversion device.

According to the configuration of the present invention described above, for example, a voice conversion device that can be easily operated with a button or a touch panel held by a user with one hand is used. And two means of facing users (one user uses the first language and the other user uses the second language), the means for acquiring the operating authority manually, or A means to manually give the operation authority to the other party, or a means to automatically acquire it, explicitly indicate which one has the operation authority, and display and input means that the user can easily operate provide. Thereby, for example, it is possible to improve the usability of the display contents as compared with the conventional case without increasing the display power.
The first to tenth other inventions related to the present invention for solving the first problem in the prior art will be described below.

According to a first other aspect of the present invention, there is provided voice input means for inputting voice in the first language;
Voice recognition means for voice recognition of the input voice;
First extraction / display means for extracting and displaying one or more word strings of the first language corresponding to the speech recognition result;
A conversion target selecting means for selecting any word string to be converted from the displayed word string to a second language;
When all or part of the selected word string is specified, a second extraction / display unit that extracts and displays candidate terms corresponding to the specified content;
Candidate selection means for selecting any candidate from the displayed candidates;
Conversion for determining a conversion target to the second language based on the selected word string and the selected candidate, and converting the determined conversion target to the speech language of the second language And a voice conversion device.

According to a second other aspect of the invention, the first extracting / displaying unit displays the plurality of word strings to be selected and the selected word string in respective predetermined areas. A display unit with a display screen for
The second extraction / display unit is the speech conversion apparatus according to the first other aspect of the present invention, which is a unit that displays the term candidates on a partial area of the display screen in a window shape.

  According to a third aspect of the present invention, when the first extracting / displaying unit displays the selected word string on the display screen, the corresponding term is applied to a part of the word string. The speech conversion apparatus according to the second other aspect of the present invention, which is means for adding and displaying information indicating that the candidate can be displayed.

  According to a fourth other aspect of the present invention, the third other aspect includes a screen display specifying means for specifying a part of the word string on which the added information is displayed on the display screen. It is an audio conversion device of the invention.

  According to a fifth other aspect of the invention, the conversion means determines, as the conversion target, a result obtained by replacing the specified part of the word string with the selected candidate term. It is a voice conversion device of the 1st other invention.

According to another aspect of the present invention, a voice input step for inputting a voice in the first language;
A voice recognition step for voice recognition of the input voice;
A first extraction / display step of extracting and displaying one or more word strings of the first language corresponding to the speech recognition result;
A conversion target selection step for selecting any word string to be converted from the displayed word string to a second language;
A second extraction / display step for extracting and displaying term candidates corresponding to the specified contents when all or part of the selected word string is specified;
A candidate selection step for selecting any candidate from the displayed candidates;
Conversion for determining a conversion target to the second language based on the selected word string and the selected candidate, and converting the determined conversion target to the speech language of the second language And a voice conversion method.

  According to a seventh other aspect of the present invention, in the speech conversion apparatus according to any one of the first to fifth aspects, the speech recognition means, the first display / extraction means, and the conversion target selection means. And a program for causing a computer to function as all or part of the second display / extraction means, the candidate selection means, and the conversion means.

  The eighth other invention is the voice conversion method of the sixth other invention, wherein the voice recognition step, the first display / extraction step, the conversion target selection step, and the second A program for causing a computer to execute all or part of the display / extraction step, the candidate selection step, and the conversion step.

  The ninth other invention is a medium carrying the program of the seventh other invention, which can be processed by a computer.

  The tenth other aspect of the invention is a medium carrying the program of the eighth other aspect of the invention, which can be processed by a computer.

  With the above configuration, in another invention related to the technology related to the present invention, for example, it is possible to provide small hardware that a user can hold with one hand and easily operate with a button or a touch panel. For example, it is possible to classify and hold words included in an example sentence to be speech-interpreted, and to implement only a small number of words representing the class as a recognition target vocabulary in the speech recognition unit. When a sentence including a word representing a class is spoken, an example including the word can be searched and presented to the user. Normally, the user selects a desired example and outputs a translated speech. However, if necessary, the user can replace the word with another word in the class and output the translated speech. For example, if you want to enter “Do you have aspirin” in Japanese, replace it with the word “medicine” representing the class to which the word “aspirin” belongs, and say “Do you have any medicine” in Japanese. Then, the “medicine” part can be replaced with “aspirin”. Through such stepwise operations, the utility value as a speech interpreting apparatus is maintained without implementing a large-scale recognition target vocabulary.

  The present invention has an advantage that the usability of display contents can be improved as compared with the conventional case.

  In the following, the configuration and operation of a speech interpreting apparatus according to an embodiment of a speech conversion apparatus of another invention related to the present invention will be described with reference to the drawings, and another invention related to the present invention will be described. The operation of the voice conversion method will be described simultaneously.

  FIG. 1 is a block diagram showing a hardware configuration of the speech interpretation apparatus according to the present embodiment.

  The voice input / output device 102 receives the user's utterance in the source language and outputs the voice translated into the target language. The image output device 103 displays an example of the source language to be interpreted by the interpreting device. The image instruction device 105 and the button 106 are used for allowing the user to select an example displayed on the image output device 103. The arithmetic and control unit 101 converts the data related to the source language input from the voice input / output device 102, the image instruction device 105, and the button 106 into data related to the target language in a voice language, and the voice input / output device 102 and the image output device 103 Output to. The external large-capacity nonvolatile storage device 104 holds a program and data for instructing the processing procedure to the arithmetic control device 101. The external data input / output terminal 107 is used by the arithmetic and control unit 101 to exchange programs and data with external devices. The power supply device 108 supplies power necessary for driving the arithmetic control device 101.

  Here, the voice input means of another invention of the technique related to the present invention corresponds to the voice input device 102, the first extraction / display means of the other invention of the technique related to the present invention, and the second The extraction / display unit corresponds to a component including the image output device 103, the arithmetic control device 101, and the like. The screen display specifying means of another invention related to the present invention corresponds to the image instruction device 105 and the button 106. The first language of another invention of the technology related to the present invention corresponds to the source language in the present embodiment, and the second language of the other invention of the technology related to the present invention is the present embodiment. Corresponds to the target language in the form of

  A specific configuration example in which a PC / AT compatible motherboard is used for the arithmetic and control unit 101 is shown in FIG. The voice input / output device 203 is connected using the USB terminal of the motherboard 201. The image output device 204 is connected using the digital RGB interface terminal of the motherboard 201. The external large-capacity nonvolatile storage device 104 uses a 2.5-inch hard disk drive 202 and is connected to the motherboard 201 through an IDE interface. A flash memory disk may be used instead of the hard disk drive. Further, a Li-ion secondary battery 208 is used as the power supply device 108, and + 5V and + 12V voltages are supplied to the motherboard 201. Out of the input / output terminals of the motherboard 201, an analog display output terminal, a local area network terminal, and a keyboard connection terminal are pulled out to constitute an external data input / output terminal 207.

  A detailed configuration of the image output apparatus 204 is shown in FIG. A cold cathode ray tube backlight mounted on the back side is connected to an LCD unit 301 having a display area of 4 inches and a resolution of VGA using 18 bits from the digital RGB interface of the motherboard 302. Also, a video synchronization signal and a backlight control signal are connected.

  A detailed configuration of the image instruction device 205 and the button 206 is shown in FIG. A 3.8-inch pressure-sensitive touch panel 402 is connected to the touch panel controller 401, and the X coordinate and Y coordinate of the indicated position are converted into RS232C standard serial data and connected to the serial terminal COM1 of the motherboard 405. The buttons 403 and 404 are connected to the touch panel controller 401, respectively, and button ON / OFF information is added to the indicated position information. The serial data received by the device driver software of the touch panel controller 401 mounted on the motherboard 405 is decoded, so that the button 403 corresponds to the left button when the mouse is connected to the motherboard 405, and the button 404 corresponds to the right button. A mouse click event occurs.

  The detailed configuration of the voice input / output device 203 is shown in FIG. The USB audio interface 504 converts the input analog sound into digital data and transmits it to the mother board 505, and converts the digital data transmitted from the mother board 505 (corresponding to 201 in FIG. 2) into analog sound. A USB interface is used to send and receive digital data. A microphone 503 is used to collect analog audio. The output of the USB interface 504 is amplified by the audio amplifier 502 and output from the speaker 501. Instead of the USB audio interface 504, an audio interface mounted on the motherboard 505 may be used.

  FIG. 6 shows a perspective view of an example in which the configuration of FIG. 2 is mounted on a housing that the user can hold with one hand, and FIGS. 7A to 7C show three views thereof. An image instruction device 205, an image display device 204, and a button 205 are mounted on the main housing 601. Reference numerals 603 and 604 correspond to a button 403 and a button 404, respectively. A voice input / output device 203 is mounted on the sub-housing 602. When the interpretation apparatus is not used, the display surface of the image display apparatus 204 is covered and protected by the sub-housing 602.

  When using this interpreting apparatus, the sub-housing 802 is used after being moved to a predetermined position where the directivity direction of the voice input / output apparatus 203 (microphone 803) faces the user's face as shown in FIG. The three views are shown in FIGS. 9 (a) to 9 (c). That is, the speaker 804 mounted on the sub-housing 802 is lifted until it faces the user, and the microphone 803 is also lifted. In this state, the LCD 805 with a touch panel can be used.

  10A to 10C show how the main casing 601 is mounted. The 4-inch VGALCD unit 301 and the touch panel 402 are overlapped and mounted as a touch panel equipped LCD 1005. FIGS. 11A to 11C show how the sub-housing 602 is mounted.

  FIG. 12 shows a software configuration diagram as an embodiment of a program and data of another invention of the technology related to the present invention. In FIG. 12, 1201 instructs each component, and a control unit that controls the flow of data from each component, 1202 displays information from the control unit 1201 and sends input from the user to the control unit 1201. A GUI (Graphical User Interface) unit, 1203 is a voice input unit that records a user's voice according to an instruction from the control unit 1201, 1204 is a voice recognition unit that continuously recognizes a user's voice sent from the voice input unit, and 1205 is an original. An example database that holds correspondence between language and target language examples, 1206 is an example database 1205, a word class dictionary that holds words that have been classified, and 1207 is an example from the voice recognition result sent from the control unit 1201 An example selection unit 1208 for selecting an example with reference to the database 1205 follows an instruction from the control unit 1201. The word selection unit 1209 selects a classified word from the examples selected by the example selection unit 1207. A word 1209 is a word that can be changed to the classified word specified by the control unit 1201. An alternative word selection unit for selecting an alternative word by referring to the word class dictionary 1206, and a language for converting an example specified by the control unit 1201 into a target language by referring to the example database 1205 and the word class dictionary 1206 The conversion unit 1211 includes a speech synthesis unit that synthesizes and outputs an example sentence in a target language specified by the control unit. Here, the example database 1205 to the language conversion unit 1210 are collectively referred to as a translation unit 1220.

  Here, the speech recognition means of another invention related to the present invention corresponds to the speech recognition unit 1204, and the conversion target selection means of another invention related to the present invention corresponds to the example selection unit 1207 or the like. Correspond. Further, the screen display specifying means of another invention related to the present invention corresponds to the word selecting unit 1208 and the like, and the candidate selecting means of the other invention related to the present invention is an alternative word selecting unit 1209 and the like. Corresponding to The conversion means according to another invention of the technology related to the present invention corresponds to the components including the language conversion unit 1210 and the speech synthesis unit 1211.

  FIG. 14 shows a specific example of the example database 1205. The example corresponds to one sentence of the dialogue, and the correspondence between the source language and the target language is held together with information (components of the source language, dependency of the components) predetermined in each example. The source language words enclosed in <> indicate that they are classified words. Indicates that a classified word can be replaced with a word of the same class.

  FIG. 15 shows a specific example of the word class dictionary 1206. Here, a class is a word with a high degree of abstraction such as “fruit”, and a word belonging to a class is a word that expresses a specific entity of the class, such as “apple” or “mandarin orange”. is there. Note that it is possible to efficiently select an example by changing the abstraction level of the classification according to the performance of the speech recognition unit 1204. Further, the word class dictionary 1206 may be configured by hierarchizing classes.

  FIG. 16 shows details of the GUI unit 1202 displayed on the LCD 805 with a touch panel. 1601 is a translation direction designating unit that designates the direction of translation, 1603 is a speech recognition result display unit that displays a speech recognition result recognized by the speech recognition unit 1204, and 1604 displays a selected example sentence by an example selection unit 1207. 1605 is an example selection result display unit that displays an example specified by the user, 1606 is a translation result display unit that displays an example converted by the language conversion unit into a target language, and 1607 and 1608 are buttons 806, respectively. , Which corresponds to the button 807, and is input by the user. In addition, the user can perform pointing input on the LCD 805 with a touch panel.

  FIG. 13 is a software flowchart of another invention related to the present invention. 1301 is a step of selecting a direction of translation, 1302 is a step of inputting voice by the microphone 803 and performing voice recognition, 1303 is a step of searching an example from the example database 1205 based on the voice recognition result, and 1304 is a user from the searched example Selecting an example, 1305 determining whether the example selected in step 1304 is to be modified or translated, 1306 selecting a word to be corrected in the example selected in step 1304, 1307 A step of outputting a list of words that can be replaced with the word to be corrected selected in step 1306, 1308 a step of selecting a word desired by the user from the list of words output in step 1307, and 1309 a word changed in step 1308 Steps to replace the example Flop. 1310 is a step of converting the example sentence determined in step 1305 into the target language, and 1311 is a step of synthesizing the example converted into the target language in step 1309 and outputting from the speaker 804.

  Hereinafter, the operation of the software of another invention related to the present invention will be described with reference to the flowchart in FIG. 13 and the display content of the GUI unit 1202 displayed on the LCD 805 with a touch panel in FIGS. As an example, a case where the user wants to translate the sentence “Are there aspirin?” Will be described. Specifically, the user inputs “Is there a medicine” and then performs an operation of replacing the “medicine” portion with “aspirin”. In another invention related to the present invention, two types of input operations can be performed using a touch panel and buttons. Therefore, description will be given below in order of touch panel input and button input.

  FIG. 17 shows display contents of the GUI unit 1202 from step 1301 to step 1303 in the case of touch panel input. In step 1301, the user clicks the translation direction designation unit 1701 by touch panel input and designates translation in the Japanese-English direction. At this time, the GUI unit 1202 transmits the translation direction to the control unit 1201, and the control unit 1201 instructs the voice input unit 1203 to input a voice. The user uses the microphone 803 to say “Is there any medicine?” The voice input unit 1203 transmits the input voice to the voice recognition unit 1204. In step 1302, it is assumed that the speech recognition unit 1204 performs speech recognition corresponding to the specified translation direction, and transmits a recognition result “Is there a 7-day medicine” including misrecognition to the control unit 1201. The control unit 1201 transmits the voice recognition result to the GUI unit 1202 and the example selection unit 1207. The GUI unit 1202 displays the transmitted speech recognition result on the recognition result display unit 1702. On the other hand, in step 1303, the example selection unit 1207 searches for an example by the following method based on the voice recognition result, and transmits the searched example to the control unit 1201.

  The example selection unit 1207 extracts “7 days”, “medicine”, and “yes” as a set of important words defined in the example database 1205 from the speech recognition result “Is there 7 day medicine?”.

  Here, “7 days” belongs to the classified word <days>, and “drug” belongs to the classified word <drug>. “Yes” does not belong to any classified word.

  The example selection unit 1207 sequentially checks the dependency relationships of the components in FIG. 14, and sequentially selects from the examples in which the number of established relationships is large among the examples in which one or more dependency relationships are established. For example, for the example of example number 1, since there is no “over” in the above set of key words, the number of established dependency relationships is zero. As for the example of example number 2, since “something” does not exist in the above set of key words, (1 → 2) does not hold in the dependency of the constituent elements, but (2 → 3) is (See FIG. 14). Therefore, the number of established dependency relationships is 1.

If the example selection unit 1207 is designed to select an example having a dependency relationship of 1 or more from the example database 1205, the example of example number 1 is not selected in FIG. 14, and the example of example number 2 is selected. Is done. Since “something” does not exist in the set of key words, for the example of the selected example number 2,
・ Is there any medicine?
Is output.

In the following description, other examples in the example database 1205,
・ "Is it a medicine"
・ "It is a drug"
Will be described as being selected in the same manner as described above.

  The control unit 1201 transmits the example sentence transmitted from the example selection unit 1207 to the GUI unit 1202. The GUI unit 1202 displays the selected example sentence on the example candidate display unit 1703.

  The display content of the GUI unit 1202 in step 1304 is shown in FIG. In step 1304, the user selects an example “Is there a medicine” with the same sentence meaning by clicking 1801 on the touch panel input from the example candidates displayed on the example candidate display unit 1703. To do. At this time, the GUI unit 1202 transmits the selected example sentence to the control unit 1201.

  The display contents of the GUI unit 1202 in step 1305 are shown in FIG. In step 1305, the GUI unit 1202 displays the selected example sentence on the example result display unit 1901 and clears the example candidate display unit 1902. After that, it is selected whether to determine the example and translate it, or to modify the example and replace the classified word with a replaceable word. At this time, the user can determine an example by clicking the example result display unit 1901 on the touch panel. The determined example is transmitted to the control unit 1201. Further, the user can shift to a mode for replacing words in the example by double-clicking the example result display unit 1901 on the touch panel.

  The display content of the GUI unit 1202 when the example is determined in step 1305 is shown in FIG. In step 1310, the control unit 1201 transmits the example “Is there any medicine” determined by the user to the language conversion unit 1210. The language conversion unit 1210 converts the target language “Any medicine” using the example database 1205 and transmits the conversion result to the control unit 1201. The control unit 1201 transmits the conversion result to the GUI unit 1202 and the speech synthesis unit 1211. In step 1311, the GUI unit 1202 displays the conversion result on the interpretation result display unit 2001. On the other hand, the voice synthesizer 1211 synthesizes the conversion result as a voice and outputs it from the speaker 804.

  The display content of the GUI unit 1202 in step 1306 is shown in FIG. In step 1306, when the user selects the word selection mode in step 1305, the word to be changed is selected. At this time, the control unit 1201 instructs the word selection unit 1208 to select a word. The word selection unit 1208 extracts the word “medicine” classified from the example and transmits it to the control unit 1201. The control unit 1201 transmits a word to the GUI unit 1202, and the GUI unit 1202 displays an underlined “medicine” displayed in the example result display unit 2101 to indicate that the word can be changed by the user. The user clicks the word “medicine” to be corrected by touch panel input. The GUI unit 1202 transmits the selected word to the control unit 1201.

The display content of the GUI unit 1202 in step 1307 is shown in FIG. In step 1307, a list of alternative words for the word “medicine” designated by the user in step 1306 is displayed. The control unit 1201 transmits the word “drug” designated by the user to the alternative word selection unit 1209. The alternative word selection unit 1209 refers to the word class dictionary 1206 shown in FIG. 15 and is a word of the same class as the word “drug” specified by the user. “Aspirin”
・ Cold medicine
・ Troach
“Gastrointestinal” is extracted and transmitted to the control unit 1201. The control unit 1201 transmits a list of alternative words to the GUI unit 1202, and the GUI unit 1202 displays a list of alternative words in the list window 2201.

  The display content of the GUI unit 1202 in step 1308 is shown in FIG. Step 1308 selects a desired word from the list of alternative words shown in the list window 2201. At this time, the GUI unit 1202 obtains an alternative word “aspirin” by clicking on the alternative word 2301 desired by the user through the user's touch panel input, and transmits the acquired alternative word to the control unit 1201.

  The display content of the GUI unit 1202 in step 1309 is shown in FIG. Step 1309 changes the example to “Are there aspirin” with the designated alternative word “aspirin”. After that, the GUI unit 1202 changes the example displayed in the example result display unit 2401 to “Is there aspirin?” And displays it. Then, the process returns to step 1305.

  In FIG. 25, the user repeats steps 1305 to 1308, and the user selects the example determination in step 1305, converts “Is there aspirin” into the target language “Any aspirin”, and outputs a synthesized speech GUI unit 1202 Is the display content.

  Next, the case of button input will be described. In the following description, SW1 physically corresponds to the button 806 and SW2 physically corresponds to the button 807.

  The display contents of the GUI unit 1202 from step 1301 to step 1303 are shown in FIG. In step 1301, the translation in the Japanese-English direction is designated by clicking SW1, and the translation in the English-Japanese direction is designated by clicking SW2. In this case, click SW1 to specify Japanese-English translation. At this time, the GUI unit 1202 transmits the translation direction to the control unit 1201, and the control unit 1201 instructs the voice input unit 1203 to input a voice. The user uses the microphone 803 to say “Is there any medicine?” The voice input unit 1203 transmits the input voice to the voice recognition unit 1204. In step 1302, it is assumed that the speech recognition unit 1204 performs speech recognition corresponding to the specified translation direction, and transmits a recognition result “Is there a 7-day medicine” including misrecognition to the control unit 1201. The control unit 1201 transmits the voice recognition result to the GUI unit 1202 and the example selection unit 1207. The GUI unit 1202 displays the transmitted speech recognition result on the recognition result display unit 1702. On the other hand, in step 1303, the example selection unit 1207 transmits the example to the control unit 1201 based on the voice recognition result.

  The example selection unit 1207 extracts “7 days”, “medicine”, and “yes” as a set of important words defined in the example database 1205 from the speech recognition result “Is there 7 day medicine?”.

  Here, “7 days” belongs to the classified word <days>, and “drug” belongs to the classified word <drug>. “Yes” does not belong to any classified word.

  The example selection unit 1207 sequentially checks the dependency relationships of the components in FIG. 14, and sequentially selects from the examples in which the number of established relationships is large among the examples in which one or more dependency relationships are established. For example, for the example of example number 1, since there is no “over” in the above set of key words, the number of established dependency relationships is zero. As for the example of example number 2, since “something” does not exist in the above set of key words, (1 → 2) does not hold in the dependency of the constituent elements, but (2 → 3) is (See FIG. 14). Therefore, the number of established dependency relationships is 1.

If the example selection unit 1207 is designed to select an example having a dependency relationship of 1 or more from the example database 1205, the example of example number 1 is not selected in FIG. 14, and the example of example number 2 is selected. Is done. Since “something” does not exist in the set of key words, for the example of the selected example number 2,
・ Is there any medicine?
Is output.

In the following description, other examples in the example database 1205,
・ "Is it a medicine"
・ "It is a drug"
Will be described as being selected in the same manner as described above.

  The control unit 1201 transmits the example sentence transmitted from the example selection unit 1207 to the GUI unit 1202. The GUI unit 1202 displays the selected example sentence on the example candidate display unit 1703.

  The display content of the GUI unit 1202 in step 1304 is shown in FIG. In step 1304, the user selects an example “Is there a medicine” with the same sentence meaning by using a button input from the example candidates displayed on the example candidate display unit 1604. In the selection method, the designated line moves up by one line by clicking SW1, and the designated line moves down by one line by clicking SW2. To select an example, double-click SW1 to select it. At this time, the GUI unit 1202 transmits the selected example sentence to the control unit 1201.

  The display contents of the GUI unit 1202 in step 1305 are shown in FIG. In step 1305, the GUI unit 1202 displays the selected example sentence on the example result display unit 1901 and clears the example candidate display unit 1902. After that, it is selected whether to determine the example and translate it, or to modify the example and replace the classified word with a replaceable word. At this time, the user can determine an example by clicking SW2 with a button input. The determined example is transmitted to the control unit 1201. Further, by clicking SW1 with a button input, it is possible to shift to the word replacement mode in the example, and it is transmitted to the control unit 1201.

  The display content of the GUI unit 1202 when the example is determined in step 1305 is shown in FIG. In step 1310, the control unit 1201 transmits the example “Is there any medicine” determined by the user to the language conversion unit 1210. The language conversion unit 1210 converts the target language “Any medicine” using the example database 1205 and transmits the conversion result to the control unit 1201. The control unit 1201 transmits the conversion result to the GUI unit 1202 and the speech synthesis unit 1211. In step 1311, the GUI unit 1202 displays the conversion result on the interpretation result display unit 2001. On the other hand, the voice synthesizer 1211 synthesizes the conversion result as a voice and outputs it from the speaker 804.

  The display content of the GUI unit 1202 in step 1306 is shown in FIG. In step 1306, when the user selects the word selection mode in step 1305, the word to be changed is selected. At this time, the control unit 1201 instructs the word selection unit 1208 to select a word. The word selection unit 1208 extracts the word “medicine” classified from the example and transmits it to the control unit 1201. The control unit 1201 transmits a word to the GUI unit 1202, and the GUI unit 1202 displays an underlined “medicine” displayed in the example result display unit 2101 to indicate that the word can be changed by the user. The user selects a word “medicine” to be corrected by button input. That is, clicking SW1 moves one word to the left, and clicking SW2 moves one word to the right. It is also possible to select a correction word by double-clicking SW1. The GUI unit 1202 transmits the selected word to the control unit 1201.

The display content of the GUI unit 1202 in step 1307 is shown in FIG. In step 1307, a list of alternative words for the word “medicine” designated by the user in step 1306 is displayed. The control unit 1201 transmits the word “drug” designated by the user to the alternative word selection unit 1209. The alternative word selection unit 1209 refers to the word class dictionary 1206 shown in FIG. 15 and has the same class as the word “medicine” designated by the user. “Aspirin”
・ Cold medicine
・ Troach
·"Gastrointestinal drug"
Is extracted and transmitted to the control unit 1201. The control unit 1201 transmits a list of alternative words to the GUI unit 1202, and the GUI unit 1202 displays a list of alternative words in the list window 2201.

  The display content of the GUI unit 1202 in step 1308 is shown in FIG. Step 1308 selects a desired word from the list of alternative words shown in the list window 2201. At this time, the GUI unit 1202 acquires the alternative word “Aspirin” desired by the user by inputting the button on the user, and transmits it to the control unit 1201. As for the input method, clicking SW1 moves the cursor to the word one word above, and clicking SW2 moves the cursor to the word one word below. When selecting a word, it is possible to select it by double-clicking SW1.

  The display content of the GUI unit 1202 in step 1309 is shown in FIG. Step 1309 changes the example to “Are there aspirin” with the designated alternative word “aspirin”. After that, the GUI unit 1202 changes the example displayed in the example result display unit 2401 to “Is there aspirin?” And displays it. Then, the process returns to step 1305.

  FIG. 25 shows the GUI section when the user repeats steps 1305 to 1308, selects the example determination in step 1305, converts “Is there aspirin” into the target language “Any aspirin”, and outputs synthesized speech 1202 is a display content.

  In the above description, the user input to the GUI unit 1202 is limited to the touch panel input and the button input. However, it is also possible to select and determine words and examples by voice using voice recognition processing. . It is also possible to operate by combining touch panel, button, and voice input modalities. Moreover, although Japanese and English were taken up as an example, it can be similarly implemented for other languages such as Chinese, and other inventions related to the present invention do not depend on the language.

  In addition, in the above embodiment, the word string of another invention related to the technology related to the present invention has been described by taking an example of a sentence composed of a plurality of words. It may be composed of one word like

  In the above embodiment, the first extraction / display unit and the second extraction / display unit of the technology related to the present invention have been described as being realized by the same display device. For example, the configuration may be realized by independent display devices.

  As described above, the speech interpreting apparatus, which is another example of the technology related to the present invention, is a speech interpreting apparatus that selects an example based on input by speech and performs translation. Hardware includes an audio input / output device as an audio modality, an image output device as an image modality, and one or more buttons and an image instruction device as a contact modality. An arithmetic control device for converting data relating to a source language input from the image instruction device and the button into data relating to a target language in a spoken language and outputting the output data to the voice input / output device and the image output device; An external large-capacity nonvolatile storage device that holds a program and data for instructing the processing procedure to the arithmetic control device, and the arithmetic control device is external And the external data input and output terminals for exchanging vessel and the program and data, a voice interpreting device characterized by being constituted by a power supply for supplying power necessary for driving the arithmetic and control unit.

  Another example is the above-described speech interpreting apparatus using a PC / AT compatible motherboard as the arithmetic and control unit.

  Another example is the above-described speech interpreting apparatus using a hard disk drive of 2.5 inches or less as the external large-capacity nonvolatile storage device.

  Another example is the above-described speech interpreting apparatus using a flash memory disk as the external large-capacity nonvolatile storage device.

  Another example is the above-mentioned speech interpreting apparatus using a liquid crystal display device having a resolution of 240 dots or more in the vertical direction and 240 dots or more in the horizontal direction as the image output apparatus.

  Another example is the voice interpreting apparatus characterized in that two buttons are used as the buttons and functionally correspond to mouse buttons when a mouse is connected to the motherboard. .

  In another example, the voice interpreting apparatus is characterized in that a touch panel having a size equivalent to the display surface of the liquid crystal display device or a size including the previous display surface is used as the image instruction device. It is.

  In another example, the external data input / output terminal uses a keyboard connection terminal, an analog display output terminal, and a local area network terminal among the input / output terminals of the motherboard. It is.

  In another example, the audio input / output device includes a USB audio interface that inputs and outputs analog audio data and digital audio data through the USB terminal of the motherboard, and a microphone that collects user's utterances and supplies the audio to the USB audio interface. And an audio amplifier that amplifies the output of the USB audio interface, and a speaker connected to the audio amplifier.

  In another example, the audio input / output device includes an audio interface of the motherboard, a microphone that collects a user's utterance and applies it to the audio interface, an audio amplifier that amplifies the output of the audio interface, and the audio The voice interpreting apparatus is constituted by a speaker connected to an amplifier.

  In another example, the voice interpreting device is characterized in that the power supply device is constituted by a lithium ion secondary battery.

  As another example, the voice interpreting device can be held by one user in one hand, the button can be easily operated by the thumb of the one hand, and the image pointing device can be operated by the other hand. And the direction of the normal line of the display surface of the image display device and the directionality of the voice input / output device are designed to be easily directed to the user's face. The above-mentioned voice interpretation device.

  As another example, the voice interpreting device is constituted by a main housing in which buttons, an image instruction device and an image display device are mounted, and a sub-housing in which a voice input / output device is mounted. If the voice interpreter is not used, the display surface of the image display device is covered and protected by the sub-housing, and if the voice interpreting device is used, the sub-housing has a directivity direction of the voice input / output device. The speech interpretation apparatus is used after being moved to a predetermined position facing the user's face.

  Another example of the technology related to the present invention is an audio interpreting apparatus that selects an example based on an input by voice and performs translation. The software of the audio interpreting apparatus performs input / output with a user. A GUI unit, a source language input unit that inputs speech and performs speech recognition, a translation unit that translates from the source language input from the source language input unit to a target language, and an object translated by the translation unit A speech interpretation apparatus comprising: a speech synthesizer that synthesizes and outputs a language; and a control unit that controls the source language input unit, the GUI unit, the translation unit, and the speech synthesizer. .

  Another example is the above-described speech interpreting apparatus characterized in that, as the above-mentioned example, one sentence in a dialogue is a unit.

  Another example is the above-described speech interpreting apparatus characterized in that, as the above-described example, a sentence pattern frequently used in travel conversation is held.

  Another example is the above-described speech interpretation apparatus, wherein the words included in the above examples are classified together with related words that can replace the words.

  In another example, the source language input unit includes a voice input unit that performs voice input according to an instruction from the control unit, and performs continuous voice recognition on the voice input from the voice input unit to generate a word string. The speech interpreting apparatus is configured by a speech recognition unit for conversion.

  In another example, the translation unit includes an example database that holds correspondence between source language and target language examples, a word class dictionary that holds class information of words included in the example database, and a source language input unit. Based on the input from the example database, an example selection unit that selects a corresponding example from the example database, a word selection unit that selects a word to be corrected from the examples selected by the example selection unit, and the word selection unit An alternative word selection unit that selects from the word class dictionary a word that can be replaced with the selected word, and a language conversion unit that converts the target word into a target language based on the determined example. The above-mentioned voice interpretation device.

  In another example, the GUI unit includes a translation direction designating unit that designates a translation direction on the display unit, a speech recognition result display unit that displays a speech recognition result output from the source language input unit, and the example. An example candidate display unit for displaying an example selected from the example database by the selection unit, an example result display unit for displaying the example selected by the user, and an interpretation result for outputting an example of the target language output by the language conversion unit It is the above-mentioned speech interpretation apparatus characterized by comprising a display unit.

  In another example, the GUI unit selects a desired example by touch panel operation or button operation when the user selects an example from examples displayed on the example candidate display unit. The voice interpreting apparatus.

  In another example, the word selection unit adds a mark to the correctable word in the example result display unit of the GUI unit when one or more correctable words are presented to the user. The voice interpreting device.

  In another example, the mark of the correctable word is underlining the word, highlighting the word, bolding the word, or blinking the word. The above-mentioned speech interpreting apparatus.

  In another example, the word selection unit determines the GUI unit by touch panel operation, button operation, or voice operation by voice recognition when the user selects a correction word. It is a voice interpreting device.

  In another example, when the alternative word selection unit selects an alternative word, the alternative word selection unit obtains an alternative candidate list using a word class dictionary, and the GUI unit lists the alternative candidate list. The voice interpreting apparatus is characterized by being displayed in a line.

  In another example, when selecting an alternative candidate from the alternative candidate list, the alternative candidate is selected by a touch panel operation of the GUI unit, a button operation, or a voice operation by voice recognition. The voice interpreting apparatus.

  In another example, when the GUI unit can be changed to an example desired by the user, the example is determined by a touch panel operation or a button operation, and the language conversion unit translates the target language. The speech interpreting apparatus, wherein the speech synthesizer outputs the synthesized speech of the example.

  As is clear from the above description, the small hardware can be easily carried as a voice interpreting device when the user goes abroad. Further, since the user interface can be easily operated with one hand, it can be easily used in various scenes such as shopping and restaurants. Furthermore, since a voice is input using a word representative of a class and an example is confirmed, it can be replaced with a related word of the same class, so that even a small recognition target vocabulary does not reduce the utility value as a speech interpreter.

Next, a speech input translation device according to an embodiment of a speech conversion device of the present invention and a speech conversion device according to a technology related to the present invention for solving the second problem in the conventional technology described above. An example will be described with reference to the drawings.

  The configuration of the present embodiment is shown in FIG.

  As shown in the figure, a basic speech translation function is realized by a speech input unit 4101, a translation support unit 4108, a speech translation unit 4102, a display unit 4103, and a speech output unit 4107.

  Here, since the internal configuration of the apparatus according to the present embodiment has already been described in the above embodiment, a detailed description thereof is omitted here.

  The correspondence between the configuration of the present embodiment (see FIG. 26) and the configuration shown in FIG. 12, for example, is as follows. 26 corresponds to the speech input unit 1203 in FIG. 12, and the speech translation unit 4102 corresponds to the translation unit 1220, the speech recognition unit 1204, and the like. The translation support unit 4108 and the display unit 4103 correspond to the GUI unit 1202 and the like, and the speech output unit 4107 corresponds to the speech synthesis unit 1211 and the like.

  Next, the characteristic components of this embodiment will be described.

  In FIG. 26, the language conversion direction control unit 4105 determines which of the two users has the authority to operate the translation apparatus, controls the input form of the speech input unit 4101, and controls the speech translation unit 4102. The translation direction is designated and the display content of the display unit 4103 is designated. Here, one of the two users uses Japanese (corresponding to the first language of the present invention), and the other uses English (corresponding to the second language of the present invention). .

  The language conversion direction detection unit 4104 collects information necessary for the language conversion direction control unit 4105 to determine a user who has operating authority. The dialogue management unit 4106 sequentially holds the translation pairs displayed on the display unit 4103, and displays them on the display unit 4103 in one of the languages as a history of dialogues exchanged between the users. (See FIG. 29).

  In the following example, the operation of the translation apparatus for Japanese and English will be described with reference to FIG. 27 and the like, and an embodiment of the speech conversion method of the present invention will be described at the same time.

  Here, for example, as shown in FIG. 27, it is assumed that a Japanese user is facing the lower side and an English user is facing the upper side in the figure with the translation device in between.

  The speech input translation device in FIG. 27 is a speech input translation device in which the language conversion direction detection unit 4104 is a button. Assume that the language conversion direction control unit 4105 indicates the conversion direction from Japanese to English as an initial state. The input unit 4101 includes a microphone 4202 and a microphone 4206, but the input of the microphone 4206 is blocked in order to input Japanese. The user presses the voice input button 4201 and then speaks into the microphone 4202 of the input unit 4101 (for example, “Is there any medicine?”). The uttered voice is translated by the voice translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  When the speech translation unit 4102 receives Japanese speech from the speech input unit 4101, the speech translation unit 4102 recognizes the speech, extracts one or more Japanese word strings corresponding to the recognition result, and displays the display unit 4103. Is displayed as an example candidate.

  That is, the example candidates are displayed in the example candidate selection window 4203 of the display unit 4103 (for example, three examples of “Is it a drug”, “Is there a drug”, and “Is a drug”), and the translation support unit 4108 is displayed. When the user selects one of them (for example, selecting with the touch panel), the selected example is displayed in the example result window 4204 (for example, “Is there a medicine?”), And the English text translated from the example Is uttered from the audio output unit 4107 (for example, “Do you have medicine?”). A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, (“Do you have medicine?”, “Do you have medicine?”)). In order to prompt the other party to operate the translation apparatus, the button 4205 which is the language conversion direction detection unit 4104 is pressed in order to prompt the other party to operate the translation apparatus.

Based on the information from the language conversion direction detection unit 4104, the language conversion direction control unit 4105 instructs the speech translation unit 4102 and the display unit 4103 to convert from English to Japanese. The display content of the display unit 4103 is rotated by 180 ° as shown in FIG. 28 so as to be easy to use for face-to-face English users, and is displayed in English. In the input unit 4101, the input of the microphone 4307 is blocked in order to input English, and the microphone 4303 is enabled. The dialogue history window 4301 displays the English of the translation pair from the dialogue management unit 4106. Specifically, for example, a translation pair consisting of “Do you have medicine?” And “Do you have medicine?” And a translation pair consisting of “Yes” and “Yes, I do.” When held in the management unit 4106, the dialogue window 4401 shown in FIG. 29 displays (Sun): “Is there any medicine?”, (English): “Yes”. The user presses the voice input button 4302 and then speaks into the microphone 4303 of the input unit 4101 (for example, “Yes, certain”). The voice uttered is the voice translation unit 41.
Translated by 02.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  When the speech translation unit 4102 receives English speech from the speech input unit 4101, the speech translation unit 4102 recognizes the speech, extracts one or more English word strings corresponding to the recognition result, and displays them in the display unit 4103. Display as a candidate.

  That is, candidates are displayed in the example candidate selection window 4304 of the display unit 4103 (for example, three examples of “Yes, I do.”, “Surely.”, “Certainly.”), And the translation support unit 4108 is used. When the user selects one of them (for example, selecting with the touch panel), the selected example is displayed in the example result window 4305 (for example, “Yes, I do.”), And the translated Japanese example is displayed. The text is uttered from the voice output unit 4107 (for example, “Yes”). A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, a translation pair (“Yes”, “Yes, I do.”)). In order to prompt the other party to operate the translation apparatus, the button 4306 which is the language conversion direction detection unit 4104 is pressed.

  Based on information from the language conversion direction detection unit 4104, the language conversion direction control unit 4105 instructs the speech translation unit 4102 and the display unit 4103 to convert from Japanese to English. The display content of the display unit 4103 is rotated by 180 ° as shown in FIG. 29 so as to be easy to use for face-to-face Japanese users, and becomes Japanese display. Since the input unit 4101 inputs Japanese, the input of the microphone 4405 is blocked and the microphone 4403 is enabled. In the dialogue history window 4401, the Japanese of the translation pair is displayed from the dialogue management unit 4106. The user presses the voice input button 4402 and then speaks into the microphone 4403 of the input unit 4101 (for example, “Thank you”). The uttered voice is translated by the voice translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  That is, candidates are displayed in the example candidate selection window 4413 of the display unit 4103 (for example, one example of “Thank you”), and when the user selects one of them using the translation support unit 4108 ( The selected example is displayed on the example result window 4414 (for example, “Thank you”), and the translated Japanese text is uttered from the voice output unit 4107 (for example, “Thank you. "). The translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, (Thank you, "Thank you.") Translation pair). In order to further ask the partner's answer, when the partner is prompted to operate the translation device, a button 4404 which is the language conversion direction detection unit 4104 is pressed.

The speech input translation device of FIG. 30 is an example of a technology invention related to the present invention, and is a speech input translation device in which the language conversion direction detection unit 4104 is the tilt angle sensor 4502 of the microphone 4501. That is, the angle sensor 4502 is used to determine whether the microphone 4501 is inclined toward the Japanese user or the English user. In the state of FIG. 30, the language conversion direction control unit 4105 indicates the conversion direction from Japanese to English. The user presses the voice input button 4503 and then speaks into the microphone 4501 of the input unit 4101 (for example, “Is there any medicine?). The uttered voice is translated by the voice translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  That is, candidates are displayed in the example candidate selection window 4504 of the display unit 4103 (for example, three examples of “Is it a medicine”, “Is there a medicine”, and “Is a medicine”), and are used by using the translation support unit 4108 When the user selects one of them (for example, with the touch panel), the selected example is displayed in the example result window 4505 (for example, “Is there a medicine?”), And the English text translated from the example is output as a voice. It is uttered from the part 4107 (for example, “Do you have medicine?”). A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, (“Do you have medicine?”, “Do you have medicine?”)). When the partner is prompted to operate the translation device, the microphone 4501 is pointed toward the English user.

  Based on the information from the language conversion direction detection unit 4104, the language conversion direction control unit 4105 instructs the speech translation unit 4102 and the display unit 4103 to convert from English to Japanese. The display content of the display unit 4103 is rotated by 180 ° as shown in FIG. 31 so as to be easy to use for the face-to-face English user and is displayed in English. In the dialog history window 4601, the English of the translation pair is displayed from the dialog management unit 4106. The user presses the voice input button 4602 and then speaks into the microphone 4603 of the input unit 4101 (for example, “Yes, certainly”). The uttered voice is translated by the voice translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  That is, candidates are displayed in the example candidate selection window 4604 of the display unit 4103 (for example, three examples of “Yes, I do.”, “Surely.”, “Certainly.”), And the translation support unit 4108 is used. When the user selects one of them (for example, selecting with the touch panel), the selected example is displayed in the example result window 4605 (for example, “Yes, I do.”), And the translated Japanese example is displayed. The text is uttered from the voice output unit 4107 (for example, “Yes”). A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, a translation pair (“Yes”, “Yes, I do.”)). In order to find the answer of the other party, the microphone 4603 is pointed toward the Japanese user.

  Based on information from the language conversion direction detection unit 4104, the language conversion direction control unit 4105 instructs the speech translation unit 4102 and the display unit 4103 to convert from Japanese to English. The display content of the display unit 4103 is rotated by 180 ° as shown in FIG. 32 so as to be easy to use for face-to-face Japanese users, and is displayed in Japanese. In the dialogue history window 4701, the Japanese of the translation pair is displayed from the dialogue management unit 4106. The user presses the voice input button 4702 and then speaks into the microphone 4703 of the input unit 4101 (for example, “Thank you”). The uttered voice is translated by the voice translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  That is, a candidate is displayed in the example candidate selection window 4704 of the display unit 4103 (for example, one example of “Thank you”), and when the user selects one of them using the translation support unit 4108 ( The selected example is displayed in the example result window 4705 (for example, “Thank you”), and the translated Japanese text is uttered from the voice output unit 4107 (for example, “Thank you. "). The translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, (Thank you, "Thank you.") Translation pair). Further, in order to prompt the other party to operate the translation apparatus, the microphone 4703 is directed toward the English user in order to obtain the other party's answer.

The speech input translation device of FIG. 33 is an example of the speech conversion device of the present invention, and is a speech input translation device that is a gyro sensor 4801 in which the language conversion direction detection unit 4104 detects the inclination of the main body. It is assumed that the language conversion direction control unit 4105 indicates the conversion direction from Japanese to English in the state of the gyro sensor. The input unit 4101 includes a microphone 4802 and a microphone 4803, but the input of the microphone 4803 is blocked in order to input Japanese. The user presses the voice input button 4804 and then speaks into the microphone 4802 of the input unit 4101 (for example, “Is there any medicine?). The uttered voice is translated by the voice translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  That is, candidates are displayed in the example candidate selection window 4805 of the display unit 4103 (for example, three examples of “Is it a medicine”, “Is there a medicine”, and “Is a medicine”), and are used by using the translation support unit 4108 When the user selects one of them (for example, with the touch panel), the selected example is displayed in the example result window 4806 (for example, “Is there any medicine?”), And the English text translated from the example is output as a voice. It is uttered from the part 4107 (for example, “Do you have medicine?”). A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, (“Do you have medicine?”, “Do you have medicine?”)). In order to ask the other party's answer, when the other party is prompted to operate the translation apparatus, the other party is held upside down by holding the speech translation apparatus upside down.

Based on the information from the gyro sensor 4901, the language conversion direction control unit 4105 instructs the speech translation unit 4102 and the display unit 4103 to convert from English to Japanese. The display content of the display unit 4103 is rotated by 180 ° as shown in FIG. 34 so as to be easy to use for the face-to-face English user and is displayed in English. In the input unit 4101, the input of the microphone 4902 is blocked in order to input English, and the microphone 4903 is enabled. In the dialogue history window 4904, the English of the translation pair is displayed from the dialogue management unit 4106. The user presses the voice input button 4905 and then speaks into the microphone 4903 of the input unit 4101 (for example, “Yes, certainly”). The uttered voice is translated by the voice translation unit 4102.
Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  In other words, candidates are displayed in the example candidate selection window 4906 of the display unit 4103 (for example, three examples of “Yes, I do.”, “Surely.”, “Certainly.”), And the translation support unit 4108 is used. When the user selects one of them (for example, using the touch panel), the selected example is displayed in the example result window 4907 (for example, “Yes, I do.”), And the translated Japanese example is displayed. The text is uttered from the voice output unit 4107 (for example, “Yes”). A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, a translation pair (“Yes”, “Yes, I do.”)). In order to ask the other party's answer, when the other party is prompted to operate the translation apparatus, the other party is held upside down by holding the speech translation apparatus upside down.

  Based on information from the gyro sensor 5001, the language conversion direction control unit 4105 instructs the speech translation unit 4102 and the display unit 4103 to convert from Japanese to English. The display content of the display unit 4103 is rotated by 180 ° as shown in FIG. 35 so as to be easy to use for the face-to-face Japanese user and is displayed in Japanese. Since the input unit 4101 inputs Japanese, the input of the microphone 5002 is blocked and the microphone 5003 is enabled. In the dialogue history window 5004, the Japanese of the translation pair is displayed from the dialogue management unit 4106. The user presses the voice input button 5005 and then speaks into the microphone 5003 of the input unit 4101 (for example, “Thank you”). The uttered voice is translated by the voice translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  That is, candidates are displayed in the example candidate selection window 5006 of the display unit 4103 (for example, one example of “Thank you”), and when the user selects one of them using the translation support unit 4108 ( The selected example is displayed in the example result window 5007 (for example, “Thank you”), and the translated Japanese text is uttered from the voice output unit 4107 (for example, “Thank you. "). The translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, (Thank you, "Thank you.") Translation pair). In order to further ask the partner's answer, when the partner is prompted to operate the translation device, the partner is held in the hand so that the speech translation device is turned upside down.

The speech input translation device of FIG. 36 is an example of the invention of the technology related to the present invention, and the speech input translation device is the microphone array unit 5101 in which the input unit 4101 and the language conversion direction detection unit 4104 can detect the direction of the sound source. It is. The microphone array unit 5101 has a function of collecting sound with sharp directivity after specifying the direction of the sound source. Generally, the microphone array unit 5101 includes a plurality of microphone units arranged geometrically and each microphone unit. It is constituted by an arithmetic unit that converts an output into a single output by digital signal processing.

  When a Japanese user starts to speak (for example, “no,”), the microphone array unit 5101 detects the direction of the voice of the speaker, and is ready to speak. The background color of the display unit 4103 is a color that alerts the user (for example, red) while the utterance is not possible, and the permission color is given (for example, green) when the utterance is possible. Based on the information of the microphone array unit 5101, the language conversion direction control unit 4105 instructs the conversion direction from Japanese to English. When the user utters in a state where it can be generated (for example, “Is there any medicine?), The uttered speech is translated by the speech translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  That is, candidates are displayed in the example candidate selection window 5102 of the display unit 4103 (for example, three examples of “Is it a medicine”, “Is there a medicine”, and “Is a medicine”), and are used by using the translation support unit 4108 When the user selects one of them (for example, with the touch panel), the selected example is displayed in the example result window 5103 (for example, “Is there a medicine?”), And the English text translated from the example is output as a voice. It is uttered from the part 4107 (for example, “Do you have medicine?”). A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, (“Do you have medicine?”, “Do you have medicine?”)).

  When the other party starts speaking to answer (for example, “Hmm,”), the microphone array unit 5201 detects the direction of the voice of the speaker and is ready to speak. Based on information from the microphone array unit 5201, the language conversion direction control unit 4105 instructs the speech translation unit 4102 and the display unit 4103 to convert from English to Japanese. The display content of the display unit 4103 is rotated by 180 ° as shown in FIG. 37 so as to be easy to use for the face-to-face English user and is displayed in English. In the dialogue history window 5202, the English of the translation pair is displayed from the dialogue management unit 4106. When an English user speaks in a utterable state (for example, “Yes, certain”), the uttered speech is translated by the speech translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  That is, candidates are displayed in the example candidate selection window 5203 of the display unit 4103 (for example, three examples of “Yes, I do.”, “Surely.”, “Certainly.”), And the translation support unit 4108 is used. When the user selects one of them (for example, selecting with the touch panel), the selected example is displayed in the example result window 5204 (for example, “Yes, I do.”), And the translated Japanese example is displayed. The text is uttered from the voice output unit 4107 (for example, “Yes”). A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, a translation pair (“Yes”, “Yes, I do.”)).

  When the other party starts speaking to answer (for example, “Ah,”), the microphone array unit 5301 detects the direction of the voice of the speaker and becomes ready to speak. Based on information from the microphone array unit 5301, the language conversion direction control unit 4105 instructs the speech translation unit 4102 and the display unit 4103 to convert from Japanese to English. The display content of the display unit 4103 is rotated by 180 degrees as shown in FIG. In the dialogue history window 5302, the Japanese of the translation pair is displayed from the dialogue management unit 4106. When the user utters (eg, “thank you”) in a state where it can be generated, the uttered speech is translated by the speech translation unit 4102.

  Note that the translation operation in the translation unit 4102 has been described with reference to FIG. 14 and the like in the above embodiment, and therefore, detailed description thereof is omitted here, but the outline is as follows.

  That is, candidates are displayed in the example candidate selection window 5303 of the display unit 4103 (for example, one example of “Thank you”), and when the user selects one of them using the translation support unit 4108 ( The selected example is displayed in the example result window 5304 (for example, “Thank you”), and the Japanese text translated from the example is uttered from the voice output unit 4107 (for example, “Thank you. "). The translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, (Thank you, "Thank you.") Translation pair).

  The touch panel, buttons, and voice input modalities can be combined, or the buttons can be replaced with the touch panel. Moreover, although Japanese and English were taken as an example, it can implement similarly about other languages, such as Chinese, and this invention does not depend on a language.

  As is clear from the above description, according to the above configuration, since the operation screen of one language is displayed on the entire surface of the display unit, the usability of the translation apparatus is maintained even with a small display unit. In addition, it is easy to understand which operation authority is in accordance with the contents of the screen, so that the two utterances do not overlap. Therefore, the recognition rate of voice recognition does not decrease, and the performance as a translation device does not decrease.

  In the above-described embodiment, the second language has been described with respect to the case where the voice output is performed together with the display of the translation result. However, the present invention is not limited to this.

  In the above embodiment, a case has been described in which two users use the device in a form facing each other with the translation device interposed therebetween. However, the present invention is not limited to this. For example, two people use the device side by side. It is good also as composition to do.

Specifically, the configuration is as shown in FIGS. 39 and 40. As shown in these figures, a user in the first language (for example, Japanese) uses the voice input button 4201a, and a user in the second language (for example, English) uses the voice input button 4201b. To do. In this configuration, one microphone 5501 is provided in the upper center of the apparatus. In this case the same effect as the above configuration.

  In the above-described embodiment, for example, as shown in FIGS. 27 and 28, the case where the example result is to be translated has been described. However, the present invention is not limited to this. For example, in the word string displayed in the example result window 4204 Then, a list of alternative words for the word specified by the user may be displayed, a desired word may be selected from the alternative words, and a result reflecting the selection result may be a translation target. That is, the configuration in this case is obtained by applying the configuration described in FIGS. 21 to 24 to the configuration shown in FIG.

  Specifically, the configuration shown in FIGS. That is, a list of alternative words for the word specified by the user (FIG. 41) is displayed from the word string displayed in the example result window 4204a (FIG. 42), and the desired word is displayed from the alternative words. For example, an aspirin word 2301 is selected (FIG. 43), and an example result reflecting the selection result is displayed in an example result window 4204a (FIG. 44), which is to be translated. Subsequent translation operations and the like are the same as those shown in FIG. As a result, the range of translation objects is further expanded and usability is improved.

Invention of the technology related to the present invention, all or part of the means of the above-described voice conversion apparatus (or, elements, circuits, parts etc.) A program for executing the functions of the computer, and the computer It is a program that works in cooperation.

Further, the invention of techniques related to the present invention, all or part of the steps of the voice conversion method described above (or processes, operations, actions, etc.) A program for causing a computer to execute the operations of A program that operates in cooperation with a computer.

Further, the invention of techniques related to the present invention is a carrying medium a program for implementing the whole or part of the steps of the whole or a computer part of the operation of the voice conversion method described above, by a computer It is a readable medium and the read program is a medium for executing the operation in cooperation with the computer.

Further, the invention of techniques related to the present invention is a carrying medium a program for implementing the whole or a computer part of the functions of all or part of the means of the above-described voice conversion device, a computer It is a medium that can be read and the read program executes the function in cooperation with the computer.

Also, part of the means of the invention of techniques related to the present invention (or devices, elements, circuits, parts etc.), some of the steps of the invention techniques related to the present invention (or processes, operations, actions, etc. ) Means several means or steps among the plurality of means or steps, or means some function or part of operation within one means or step. is there.

Further, a part of the invention (or an element, a circuit, a part, etc.) of the technology related to the present invention means several devices among the plurality of devices, or one device. Means a part (or an element, a circuit, a part, etc.), or means a part of the function of one means.

Further, one use form of the invention program of the technology related to the present invention may be an aspect in which the program is recorded on a computer-readable recording medium and operates in cooperation with the computer.

Further, one use form of the invention program of the technology related to the present invention may be an aspect in which the program is transmitted through a transmission medium, read by a computer, and operated in cooperation with the computer.

  The recording medium includes a ROM and the like, and the transmission medium includes a transmission medium such as the Internet, light, radio waves, sound waves, and the like.

  The computer of the present invention described above is not limited to pure hardware such as a CPU, but may include firmware, an OS, and peripheral devices.

  As described above, the configuration of the present invention may be realized by software or hardware.

Speech conversion equipment according to the present invention has the effect that it is possible to improve the usability of the display contents in comparison with the prior art, for example, it converts the utterance of the speech input original language into the target language useful as a speech conversion equipment for audio output.

The block diagram which shows the hardware constitutions of the speech interpretation apparatus of one Embodiment of the technique related to this invention of one another Detailed block diagram of Figure 1 when using a PC / AT compatible motherboard Detailed block diagram of the image output device 204 Detailed block diagram of image pointing device 205 and button 206 Detailed block diagram of the voice input / output device 203 Overall view of the cabinet when not using the voice interpreter (A) is a front view showing the detailed structure of the speech interpreting apparatus shown in FIG. 6, (b) is a side view showing the detailed structure of the speech interpreting apparatus shown in FIG. 6, and (c) is a side view of the speech interpreting apparatus shown in FIG. Plan view showing detailed structure Overall view of the housing when using the voice interpreter (A) is a front view showing the detailed structure of the speech interpreting apparatus shown in FIG. 8. (b) is a side view showing the detailed structure of the speech interpreting apparatus shown in FIG. 8. (c) is the side view of the speech interpreting apparatus shown in FIG. Plan view showing detailed structure 2A is a front view for illustrating a method of mounting each component of FIG. 2 on the main housing 801. FIG. 2B is a side view for illustrating a method of mounting each component of FIG. (C) is a plan view showing a method of mounting each component of FIG. 2 on the main housing 801 2A is a front view for illustrating a method for mounting each component in FIG. 2 on the sub-housing 802. FIG. 4B is a side view for illustrating a method for mounting each component in FIG. (C) is a top view for showing the method of mounting each component of FIG. The block diagram which shows the structure of the software of the speech interpretation apparatus of one Embodiment of the technique relevant to this invention. Flow chart showing the flow of software processing The figure which shows an example of the content of the example database 1205 The figure which shows an example of the content of the word class dictionary 1206 The figure which shows the display content of the GUI part 1202 The figure which shows the display content of the GUI part 1202 from step 1301 to 1303 The figure which shows the display content of the GUI part 1202 in the process of step 1304. The figure which shows the display content of the GUI part 1202 in the process of step 1305. The figure which shows the display content of the GUI part 1202 in the process from step 1310 to 1311 The figure which shows the display content of the GUI part 1202 in the process of step 1306. The figure which shows the display content of the GUI part 1202 in the process of step 1307. The figure which shows the display content of the GUI part 1202 in the process of step 1308. The figure which shows the display content of the GUI part 1202 in the process of step 1309 The figure which shows the display content of the GUI part 1202 in the process from step 1310 to 1311 The block diagram which shows the structure of the speech input translation apparatus of one embodiment of this invention The figure which shows utilization of the Japanese of the speech translation apparatus whose language conversion direction detection part 4104 is a button The figure which shows utilization of English of the speech translation apparatus whose language conversion direction detection part 4104 is a button The figure which shows utilization of the Japanese of the speech translation apparatus whose language conversion direction detection part 4104 is a button The figure which shows utilization of the Japanese of the speech translation apparatus whose language conversion direction detection part 4104 is an angle sensor of a microphone axis | shaft. The figure which shows utilization of English of the speech translation apparatus whose language conversion direction detection part 4104 is an angle sensor of a microphone axis | shaft. The figure which shows utilization of the Japanese of the speech translation apparatus whose language conversion direction detection part 4104 is an angle sensor of a microphone axis | shaft. The figure which shows utilization of the Japanese of the speech translation apparatus whose language conversion direction detection part 4104 is a gyro sensor. The figure which shows utilization of English of the speech translation apparatus whose language conversion direction detection part 4104 is a gyro sensor. The figure which shows utilization of the Japanese of the speech translation apparatus whose language conversion direction detection part 4104 is a gyro sensor. The figure which shows utilization of the Japanese of the speech translation apparatus whose input part 4101 and language conversion direction detection part 4104 are microphone array units. The figure which shows utilization of English of the speech translation apparatus whose input part 4101 and the language conversion direction detection part 4104 are microphone array units. The figure which shows utilization of the Japanese of the speech translation apparatus whose input part 4101 and language conversion direction detection part 4104 are microphone array units. The figure for demonstrating utilization of the Japanese of the speech translation apparatus of another embodiment of this invention The figure for demonstrating utilization of English of the speech translation apparatus of another embodiment of this invention The figure for demonstrating the function of the alternative word in utilization of the Japanese of the speech translation apparatus of another embodiment of this invention. The figure for demonstrating the function of the alternative word in utilization of the Japanese of the speech translation apparatus of another embodiment of this invention. The figure for demonstrating the function of the alternative word in utilization of the Japanese of the speech translation apparatus of another embodiment of this invention. The figure for demonstrating the function of the alternative word in utilization of the Japanese of the speech translation apparatus of another embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Arithmetic control device 102 Voice input / output device 103 Image output device 104 External large capacity non-volatile storage device 105 Image instruction device 106 Button 107 External data input / output terminal 108 Power supply device 201 Motherboard 202 2.5 inch hard disk drive 203 Voice input / output device 204 Image output device 205 Image instruction device 206 Button 207 External data output terminal 208 Li-ion secondary battery 301 Backlit 4 inch VGALCD unit 302 Motherboard 401 Touch panel controller 402 3.8 inch pressure sensitive touch panel 403 button 404 button 405 Motherboard 501 Speaker 502 Audio Amplifier 503 Microphone 504 USB Audio Device 505 Motherboard 601 Main Case 602 Sub Case 603 Button 60 Button 701 front view 702 right side view 703 top view 801 main housing 802 sub-housing 803 microphone 804 speaker 805 LCD with a touch panel
901 Front view 902 Right side view 903 Top view 1001 Front view 1002 Right side view 1003 Top view 1004 Motherboard 1005 LCD with touch panel
1006 2.5 inch hard disk drive 1007 button 1008 button 1101 front view 1102 right side view 1103 top view 1104 microphone 1105 speaker 1106 USB audio device 1107 audio amplifier 1201 control unit 1202 GUI unit 1203 audio input unit 1204 audio recognition unit 1205 example database 1206 Word class dictionary 1207 Example selection unit 1208 Word selection unit 1209 Alternative word selection unit 1210 Language conversion unit 1211 Speech synthesis unit 1301 Step for determining the direction of translation 1302 Step for performing speech recognition 1303 Step for retrieving example from example database 1304 Example for example Step 1305 for selecting Step 1306 for determining whether to determine an example or to correct 1306 Step for determining a word to be corrected Step 1307 Obtaining an alternative word list 1308 Deciding an alternative word 1309 Modifying an example 1310 Performing language conversion 1311 Performing a speech synthesis unit 1601 Translation direction designation unit 1602 Translation direction designation unit 1603 Recognition result display unit 1604 Example candidate display unit 1605 Example result display unit 1606 Interpretation result display unit 1607 Button SW1
1608 Button SW2
1701 Translation direction designation part 1702 Recognition result display part 1703 Example candidate display part 1801 Selected example 1901 Example result display part 1902 Example candidate display part 2001 Interpretation result display part 2101 Example result display part 2201 List window 2301 Selected alternative word 2401 Example result display unit 4105 Language conversion direction control unit 4106 Dialog history management unit 4104 Language conversion direction detection unit

Claims (1)

An input unit for inputting voice in the first or second language;
(1) When the voice of the first language is received from the input unit, the voice is recognized and, based on a predetermined control instruction, (1-a) the voice-recognized first language Or (1-b) converting the conversion target determined based on the speech recognition result into the second language, and outputting at least the notation data in the converted language. (2) When the second language voice is received from the input unit, the voice is recognized and, based on a predetermined control instruction, (2-a) the second voice recognized Output language notation data, or (2-b) convert the conversion target determined based on the speech recognition result to the first language, and output at least the notation data of the converted language A speech translator to
A support unit for supporting the determination of the conversion target of the speech translation unit;
A display unit for displaying the notation data of the converted language output from the speech translation unit based on the predetermined control instruction;
A control unit that performs the predetermined control instruction on at least the speech translation unit and the display unit ;
Language conversion direction detection for detecting information for determining the translation direction of which translation from the first language to the second language or from the second language to the first language should be performed by the above Part
The control unit specifies the translation direction for the speech translation unit based on the detection result, and controls the input unit,
The language conversion direction detection unit is composed of a gyro sensor installed inside the voice conversion device,
A speech conversion device for selecting a translation direction at a position where a user who speaks holds the speech conversion device.

Images