JP6298806B2 - Speech translation system, control method therefor, and speech translation program - Google Patents

Description

  The present invention relates to a speech translation system, a speech translation method, and a speech translation program.

  In general, a speech translation system uses a speech recognition engine for recognizing input speech when translating speech in one language into speech in another language (for example, Patent Documents 1 and 2). In such a speech recognition engine, for example, after the spoken speech is collated with an acoustic model database and "sound" is converted into "reading", the "reading" is collated with the language model database and "character" The word sequence is further adjusted and displayed as a series of text as necessary. In the speech translation system, the input speech of a certain language recognized in this way is translated into another language by the translation engine, and the translation result is converted into output speech by the speech synthesis engine.

Japanese Patent Laying-Open No. 2015-40946 JP 2011-22813 A JP 2005-284880 A

  By the way, in the conventional speech translation system, if the input speech is erroneously recognized by the speech recognition engine, the erroneous speech recognition result is translated, so that the translation result is also incorrect. Measures to reduce the misrecognition rate include high accuracy of speech recognition processing, lengthening of recognition target words (processing a plurality of words as much as possible), re-input of speech by a speaker, or speech recognition result Confirmation.

  However, in order to increase the accuracy of the speech recognition processing, a heavy load processing for many vocabularies is required, and as a result, processing time tends to increase. In this case, high-speed processing is possible. If hardware is used, the cost of the apparatus increases. Also, if the recognition target word is lengthened, a group of a plurality of words is output as a speech recognition result, so that the subsequent handling in the translation engine becomes complicated or complicated, and the translation processing takes time and the translation accuracy is high. It may decrease. Further, when the speaker re-inputs the voice or confirms the voice recognition result, the process for generating and displaying the message is complicated, and it takes time to obtain the final translation result. As a result, the burden on the user (user, speaker) increases and convenience decreases.

  Therefore, the present invention has been made in view of such circumstances, and provides a speech translation system, a speech translation method, and a speech translation program that can easily improve the accuracy of speech recognition and thus speech translation. With the goal.

  In order to solve the above problems, the present inventor has intensively conducted research including collection of sampling data and analysis and analysis of test data. As a result, the normal speech recognition engine uses, for example, an acoustic model created based on the analysis results of standard adult speech data. For example, the speech recognition rate when an elderly person or a child speaks Has been found to be low, and the present invention has been completed.

  That is, a speech translation system according to an aspect of the present invention includes a speech input unit for inputting a speaker's speech, a speech recognition unit for recognizing the content of speech input to the speech input unit, and a speech recognition unit. A translation unit that translates the content recognized in the different language content, a speech synthesis unit that synthesizes the speech of the content translated by the translation unit, a speech output unit that outputs the speech synthesized by the speech synthesis unit, And a storage unit. Then, the storage unit stores the unrecognizable speech that could not be recognized by the speech recognition unit, together with the correct recognition content of the unrecognizable speech, as a speech database. Further, the speech recognition unit refers to the speech database, collates the speech of another speaker input to the speech input unit with unrecognizable speech stored in the speech database, and the collation result (for example, the degree of coincidence between the two or On the basis of the similarity, etc., a voice inquiry process for providing correct recognition content of the voice to the translation unit is executed.

  In addition, “correct recognition content of unrecognizable speech” can be said to be correct “reading” of unrecognizable speech. As the acquisition method, for example, the content of the vocabulary of unrecognizable speech stored in the storage unit is listened to by the speaker or actually heard and recognized by another person. "Is judged. In the latter case, for example, unrecognizable speech that cannot be recognized by any age or generation of people may be excluded from the storage target in the storage unit. Furthermore, “speaker” and “other speaker” are distinguished from each other for convenience in specifying the present invention. However, even when “speaker” and “other speaker” are the same, this term is also used. It is included in the technical scope of the invention.

  The timing at which the voice recognition unit executes the collation processing is not particularly limited. For example, when the voice of another speaker cannot be recognized (after voice recognition is once executed), or The voice inquiry process may be executed before the voice is recognized.

  The speech translation system according to an aspect of the present invention may further include an information acquisition unit that acquires information related to the attributes of the speaker and other speakers. Then, the storage unit may store the unrecognizable voice and the correct recognition content in association with the attribute of the speaker as the voice database. Further, the speech recognition unit collates the attributes of other speakers with the attributes stored in the speech database, and executes the above-described speech inquiry process based on the collation result (for example, the degree of coincidence or similarity). can do.

  Specifically, an example in which the “attribute” is the age of the speaker or the age range (also referred to as a generation) or gender is given. At this time, even for the same “reading” vocabulary, a plurality of data records corresponding to different inflections and tones (syllable tones, word tones, phrasal tones, sentence tones, etc.) depending on the generation and gender, It may be created as a part and stored in the storage unit. In addition, as a method for acquiring information on “attributes”, when a user (user, speaker) uses a service related to a speech translation system, or an application that is a speech translation program is installed in a computer such as an information terminal. For example, the user information may be filled in on the user information registration screen, or the attribute question questionnaire may be answered when using the speech translation system.

  Alternatively, the speech translation system according to an aspect of the present invention recognizes the speech input unit for inputting the speech of the speaker, the speech recognition unit that recognizes the content of the speech input to the speech input unit, and the speech recognition unit. A translation unit that translates the translated content into a different language content, a speech synthesis unit that synthesizes the speech of the content translated by the translation unit, a speech output unit that outputs the speech synthesized by the speech synthesis unit, and speech recognition The adaptive processing using the unrecognizable speech is performed on the storage unit that stores the unrecognizable speech that could not be recognized by the unit and the first acoustic model used for the recognition of the input speech in the speech recognition unit. An acoustic model generation unit that generates two acoustic models is provided. The speech translation system according to this aspect may be combined with the above-described speech database and speech query processing using the speech database.

  In this case, the speech translation system further includes an information acquisition unit that acquires information related to the attributes of the speaker, and the acoustic model generation unit includes the attributes of the speaker, for example, the age of the speaker or the age range (generation) or gender. A second acoustic model may be generated every time, and the speech recognition unit may be configured to recognize the content of the input speech using the second acoustic model corresponding to the attributes of other speakers. .

  In addition, a method for controlling a speech translation system including a speech input unit, a speech recognition unit, a translation unit, a speech output unit, and a storage unit according to an aspect of the present invention includes the following steps. That is, the method includes a step of inputting the voice of the speaker by the voice input unit, a step of recognizing the content of the voice input to the voice input unit by the voice translation unit, and a voice recognition unit by the translation unit. The step of translating the recognized contents into the contents of different languages, the step of synthesizing the speech of the content translated by the translation unit by the speech synthesis unit, and outputting the speech synthesized by the speech synthesis unit by the speech output unit And the storage unit recognizes the voice ("unrecognizable voice") that could not be recognized by the voice recognition unit (or could not be recognized; And storing as a voice database. Then, in the step of recognizing the content of the voice, referring to the voice database, the voice of another speaker input to the voice input unit is collated with the unrecognizable voice stored in the voice database, and based on the collation result Then, the voice inquiry process for providing the correct recognition content of the voice to the translation unit is executed.

  In addition, the speech translation program according to one aspect of the present invention includes a computer (not limited to a single type or a single type, but may be a plurality or a plurality of types; the same shall apply hereinafter), a voice input unit for inputting a speaker's voice, a voice input A speech recognition unit for recognizing the content of speech input to the unit, a translation unit for translating the content recognized by the speech recognition unit into content in a different language, a speech synthesis unit for synthesizing speech of the content translated by the translation unit, and The voice synthesizing unit functions as a voice output unit that outputs the synthesized voice and a storage unit. Then, the program causes the storage unit to store unrecognizable speech that could not be recognized by the speech recognition unit, together with correct recognition contents of unrecognizable speech, as a speech database. In addition, the program refers to the speech recognition unit with reference to the speech database, collates speech of other speakers input to the speech input unit with unrecognizable speech stored in the speech database, and based on the collation result The voice inquiry process for providing the correct recognition content of the voice to the translation unit is executed.

  According to the present invention, correct recognition content of speech that cannot be recognized by normal processing can be easily obtained by collating speech uttered by the user with unrecognizable speech stored in advance in the speech database. Therefore, the voice can be recognized without increasing the device cost due to the high accuracy and high speed processing of the speech recognition itself, complicating various processes and reducing the translation accuracy, and increasing the burden on the user and the convenience. It is possible to improve the accuracy of recognition and thus the accuracy of speech translation easily and efficiently.

1 is a system block diagram schematically showing a preferred embodiment of a network configuration and the like related to a speech translation system according to the present invention. It is a system block diagram which shows roughly an example of a structure of the user apparatus (information terminal) in the speech translation system by this invention. It is a system block diagram which shows roughly an example of a structure of the server in the speech translation system by this invention. It is a flowchart which shows an example of the process including the speech database construction in the speech translation system by this invention. It is a flowchart which shows an example of the process including the speech inquiry in the speech translation system by this invention. It is a flowchart which shows another example of the process including the speech inquiry in the speech translation system by this invention. It is a flowchart which shows another example of the process including the speech database construction in the speech translation system by this invention. It is a flowchart which shows another example of the process including the speech inquiry in the speech translation system by this invention. It is a flowchart which shows another example of the process including the speech inquiry in the speech translation system by this invention. It is a flowchart which shows an example of the process including the acoustic model production | generation (improvement) in the speech translation system by this invention.

  Hereinafter, embodiments of the present invention will be described in detail. The following embodiments are examples for explaining the present invention, and are not intended to limit the present invention only to the embodiments. The present invention can be variously modified without departing from the gist thereof. Furthermore, those skilled in the art can employ embodiments in which the elements described below are replaced with equivalent ones, and such embodiments are also included in the scope of the present invention. Furthermore, positional relationships such as up, down, left, and right shown as needed are based on the display shown unless otherwise specified. Furthermore, various dimensional ratios in the drawings are not limited to the illustrated ratios.

(System configuration)
FIG. 1 is a system block diagram schematically showing a preferred embodiment such as a network configuration related to a speech translation system according to the present invention. The speech translation system 100 includes a server 20 that is electronically connected via a network N to an information terminal 10 (user device) used by a user (speaker or other speaker).

  The information terminal 10 employs a user interface such as a touch panel and a display with high visibility, for example. The information terminal 10 here is a portable tablet terminal device including a mobile phone represented by a smartphone having a communication function with the network N. The information terminal 10 further includes a processor 11, a storage resource 12, a voice input / output device 13, a communication interface 14, an input device 15, a display device 16, and a camera 17. In addition, the information terminal 10 operates as a part of the speech translation system according to the embodiment of the present invention by operating the installed speech translation application software (at least a part of the speech translation program according to the embodiment of the present invention). It functions as a whole.

  The processor 11 includes an arithmetic logic unit and various registers (program counter, data register, instruction register, general-purpose register, etc.). Further, the processor 11 interprets and executes speech translation application software, which is the program P10 stored in the storage resource 12, and performs various processes. The speech translation application software as the program P10 can be distributed from the server 20 through the network N, for example, and may be installed and updated manually or automatically.

  The network N includes, for example, a wired network (a short-range communication network (LAN), a wide-area communication network (WAN), a value-added communication network (VAN), etc.) and a wireless network (mobile communication network, satellite communication network, Bluetooth ( Bluetooth (registered trademark), WiFi (Wireless Fidelity), HSDPA (High Speed Downlink Packet Access), etc.).

  The storage resource 12 is a logical device provided by a storage area of a physical device (for example, a computer-readable recording medium such as a semiconductor memory), and an operating system program, a driver program, various data, etc. used for processing of the information terminal 10 Is stored. Examples of the driver program include an input / output device driver program for controlling the audio input / output device 13, an input device driver program for controlling the input device 15, an output device driver program for controlling the display device 16, and the like. Can be mentioned. Furthermore, the voice input / output device 13 is, for example, a general microphone and a sound player capable of reproducing sound data.

  The communication interface 14 provides a connection interface with the server 20, for example, and is configured from a wireless communication interface and / or a wired communication interface. The input device 15 provides an interface for accepting an input operation by a tap operation such as an icon, a button, or a virtual keyboard displayed on the display device 16, and is externally attached to the information terminal 10 in addition to the touch panel. Various input devices can be exemplified.

  The display device 16 provides various information as an image display interface to a user or a conversation partner as necessary, and examples thereof include an organic EL display, a liquid crystal display, and a CRT display. The camera 17 is for capturing still images and moving images of various subjects.

  The server 20 is constituted by, for example, a host computer having a high arithmetic processing capability, and expresses a server function by operating a predetermined server program in the host computer, for example, a speech recognition server, a translation server, And a single or a plurality of host computers functioning as a speech synthesis server (in the drawing, it is indicated by a single, but is not limited thereto). Each server 20 includes a processor 21, a communication interface 22, and a storage resource 23 (storage unit).

  The processor 21 is composed of an arithmetic and logic unit for processing arithmetic operations, logical operations, bit operations and the like and various registers (program counter, data register, instruction register, general-purpose register, etc.), and is stored in the storage resource 23. P20 is interpreted and executed, and a predetermined calculation processing result is output. The communication interface 22 is a hardware module for connecting to the information terminal 10 via the network N. For example, the communication interface 22 is a modulation / demodulation device such as an ISDN modem, an ADSL modem, a cable modem, an optical modem, or a soft modem.

  The storage resource 23 is a logical device provided by, for example, a storage area of a physical device (a computer-readable recording medium such as a disk drive or a semiconductor memory). Each of the storage resources 23 includes one or more programs P20, various modules L20, various types. A database D20 and various models M20 are stored.

  The program P10 is the above-described server program that is the main program of the server 20. In addition, the various modules L20 perform a series of information processing related to requests and information transmitted from the information terminal 10, so that they are appropriately called and executed during the operation of the program P10 (moduleized subprograms). ). Examples of the module L20 include a speech recognition module, a translation module, and a speech synthesis module.

  The various databases D20 include various corpora required for speech translation processing (for example, in the case of Japanese and English speech translation, a Japanese speech corpus, an English speech corpus, a Japanese character (vocabulary) corpus, an English character) (Vocabulary) corpus, Japanese dictionary, English dictionary, Japanese-English bilingual dictionary, Japanese-English bilingual corpus, etc.), a speech database described later, a management database for managing information related to users, and the like. In addition, examples of the various models M20 include an acoustic model and a language model used for speech recognition described later.

  An example of operations and operations of speech translation processing in the speech translation system 100 configured as described above will be further described below.

[First Embodiment]
(Speech database construction process 1 in speech translation)
FIG. 4 is a flowchart showing an example of processing including speech database construction in the speech translation system 100. Such speech database construction constitutes a part of speech translation processing by the speech translation system 100.

  A user (speaker) first activates the application by tapping an icon (not shown) of speech translation application software displayed on the display device 16 of the information terminal 10. As a result, a screen for selecting a language for speech translation is appropriately displayed on the display device 16, and the language of the user (here “Japanese”) and the language of the conversation partner (here “English”), for example. ) Can be selected. Thereafter, when a voice input screen for accepting the utterance content by the user is displayed on the display device 16, voice input from the voice input / output device 13 is possible.

  In this state, when the user (speaker) inputs a voice in, for example, Japanese (step ST1), the processor 11 generates a voice signal based on the voice input, and the voice signal is transmitted through the communication interface 14 and the network N. Send to server 20. As described above, the information terminal 10 itself, or the processor 11 and the voice input / output device 13 function as a “voice input unit”.

  The processor 21 of the server 20 receives the voice signal through the communication interface 22 and performs voice recognition processing (step SJ1). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech recognition module, Japanese speech corpus, acoustic model, language model, etc.) from the storage resource 23, and obtains “sound” of the input speech. Convert to "reading" (character). As described above, the processor 21 functions as a “voice recognition unit”, and the server 20 functions as a “voice recognition server” as a whole.

  If the speech recognition is “possible” (“Yes” in step SJ1), the processor 21 proceeds to multilingual translation processing for translating the recognized “reading” (characters) of the recognized speech into another language. Transition (step SJ2). At this time, the processor 21 calls the necessary module L20 and database D20 (translation module, Japanese character corpus, Japanese dictionary, English dictionary, Japanese-English bilingual dictionary, Japanese-English bilingual corpus, etc.) from the storage resource 23 and recognizes them. The resulting input speech “reading” (character string) is properly sorted and converted into Japanese phrases, clauses, sentences, etc., the English corresponding to the conversion result is extracted, and these are sorted according to the English grammar. To natural English phrases, clauses, sentences, etc. As described above, the processor 21 also functions as a “translation unit”, and the server 20 functions as a “translation server” as a whole.

  On the other hand, if the speech recognition is “impossible” (“No” in step SJ1), the processor 21 proceeds to a speech database construction process (step SJ5). Here, the voice that could not be recognized is stored and accumulated as “unrecognizable voice” in the voice database area that is one of the databases D20 secured in the storage resource 23 (step SJ51). Then, at the appropriate timing, the correct recognition content of the “unrecognizable voice” is acquired and stored in the same voice database (step SJ52). Specifically, examples of the acquisition method in this case include the methods (1) to (3) listed below. In any case, the processor 21 of the server 20 stores the “correct recognition content” in the voice database of the storage resource 23 in association with “unrecognizable voice”.

  (1) The user who spoke is notified on the spot that the voice is unrecognizable by displaying it on the information terminal 10, and the correct "reading" (character) of the voice is immediately input from the information terminal 10. get. The processor 11 of the information terminal 10 transmits the correct reading (that is, the correct recognition content) to the server 20 each time.

  (2) After “unrecognizable speech” is accumulated in the storage resource 23 to some extent, people who have different attributes (for example, age and gender) listen to their speech and are correctly recognized. (That is, correct recognition content) is transmitted or input to the server 20 each time or in a batch.

  (3) In the menu of the speech translation application executed on the information terminal 10, a questionnaire-type page or column for requesting cooperation for recognition of “unrecognizable speech” is prepared, and the speech translation application is executed (unspecified The user listens to one or a plurality of “unrecognizable voices” and transmits the correct reading (that is, correct recognition content) to the server 20 each time.

  Next, when the translation of the speech content is completed, the processor 21 proceeds to speech synthesis processing (step SJ3). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech synthesis module, English speech corpus, acoustic model, language model, etc.) from the storage resource 23, and the English phrase that is the translation result, Convert clauses, sentences, etc. to natural speech. As described above, the processor 21 also functions as a “speech synthesis unit”, and the server 20 functions as a “speech synthesis server” as a whole.

  Next, the processor 21 generates a voice signal for voice output based on the synthesized voice, and transmits the voice signal to the information terminal 10 through the communication interface 22 and the network N. The processor 11 of the information terminal 10 receives the audio signal through the communication interface 14 and performs an audio output process (step ST2).

(Speech inquiry processing in speech translation 1-1)
FIG. 5 is a flowchart illustrating an example of processing including speech inquiry in the speech translation system 100. Such speech inquiry constitutes a part of speech translation processing by the speech translation system 100. The process here is the same as that shown in FIG. 4 except that “user (other speaker)” performs voice input instead of “user (speaker)” and executes the process of step SJ6 instead of step SJ5. This is substantially the same as the processing shown. Therefore, the description of processes other than those related to this difference will be omitted below. The speech translation process shown in FIG. 5 is an example of a procedure for executing the speech inquiry process when the speech of the user (other speaker) cannot be recognized (after speech recognition is once performed).

  If the speech recognition is “possible” as in the process shown in FIG. 4 (“Yes” in step SJ1), the processor 21 proceeds to multilingual translation processing (step SJ2), while the speech recognition is performed. Is “impossible” (“No” in step SJ1), the processor 21 proceeds to the voice inquiry process (step SJ6).

  Here, the processor 21 calls and references the voice database from the storage resource 23, and collates the voice that could not be recognized with the “unrecognizable voice” stored in the voice database (step SJ61). At this time, for example, if it is determined that the degree of coincidence or similarity or the like in the voice matching between the two is greater than or equal to a predetermined value, the processor 21 determines “correct recognition” of the “unrecognizable voice” stored in the voice database. The “content” is output to the multilingual translation processing (step SJ2) side as the “correct recognition content” of the speech that could not be recognized (step SJ62).

(Speech inquiry process 1-2 in speech translation)
FIG. 6 is a flowchart showing another example of processing including speech inquiry in the speech translation system 100. Such a speech inquiry also constitutes a part of speech translation processing by the speech translation system 100. In this processing, “user (other speaker)” performs voice input (step ST1) instead of “user (speaker)”, and the processing of step SJ7 is executed instead of step SJ5. Except for this, it is substantially the same as the processing shown in FIG. Therefore, the description of processes other than those related to this difference will be omitted below. Moreover, the speech translation process shown in FIG. 6 is an example of a procedure for executing the speech inquiry process before recognizing the speech of the user (another speaker).

  That is, the user (another speaker) inputs a voice in Japanese (step ST1), and the processor 21 of the server 20 that receives the voice signal moves to a voice inquiry process (step SJ7). Here, the processor 21 calls and refers to the voice database from the storage resource 23, and collates the input voice with the “unrecognizable voice” stored in the voice database (step SJ71; corresponding to step SJ61).

  Then, the processor 21 determines whether or not the input voice corresponds to “unrecognizable voice” stored in the voice database (step SJ72; substantially included in the process of step SJ71). For example, when it is determined that the degree of coincidence or similarity or the like in the voice matching between the two is greater than or equal to a predetermined value, the processor 21 determines that the “applicable” (“Yes” in step SJ72) corresponds to the corresponding “ The “correct recognition content” of “unrecognizable speech” is output to the multilingual translation processing (step SJ2) side as the “correct recognition content” of the input speech (step SJ73).

  On the other hand, if it is determined that the degree of coincidence or similarity in the voice matching between the two is less than a predetermined value, the processor 21 determines that the voice is “not applicable” (“No” in step SJ72), and performs voice recognition processing ( The process proceeds to step SJ1). That is, in this case, the voice by the user (another speaker) is not “unrecognizable voice”, and is recognized by the normal voice recognition process, or the possibility thereof is extremely high.

  As described above, in the example of the speech translation process shown in FIG. 6, since the speech inquiry process is executed before the speech of the user (other speaker) is recognized, the speech recognition process (step SJ1) shown in FIG. ) Is not required.

[Second Embodiment]
(Speech database construction process 2 in speech translation)
FIG. 7 is a flowchart showing another example of processing including construction of a speech database in the speech translation system 100. The construction of the speech database also constitutes a part of speech translation processing by the speech translation system 100. In this processing, prior to voice input (step ST1) by the user (speaker), user information (step ST0) is input by the user (speaker), and “unrecognizable voice” is stored. Prior to the accumulation (step SJ51), the process is substantially the same as the process shown in FIG. 4 except that the “user information” is stored and accumulated. Therefore, the description of processes other than those related to this difference will be omitted below.

  Here, when the user (speaker) starts the speech translation application, for example, before the screen for selecting the target language for speech translation is displayed on the display device 16 of the information terminal 10 or after selecting the target language. An information registration screen for inputting information related to the user is displayed on the display device 16 of the information terminal 10. Although it does not restrict | limit especially as information regarding a user, Attribute information, such as a user's age, sex, a birthplace, a residence, is contained.

  In this state, when the user (speaker) inputs user information (step ST0), the processor 11 generates an information signal based on the information input, and the information signal is transmitted to the server 20 through the communication interface 14 and the network N. Send to. As described above, the information terminal 10 itself or the processor 11 also functions as an “information acquisition unit”.

  When the processor 21 of the server 20 receives the information signal through the communication interface 22, the processor 21 proceeds to a voice database construction process (step SJ5). Here, the input user information (including attributes) is stored in a voice database area, which is one of the databases D20 secured in the storage resource 23 (step SJ50). Next, when voice recognition by the user (speaker) (step ST1) and subsequent voice recognition processing (step SJ1) are impossible, the processor 21 determines the “unrecognizable voice” as the voice recognition process. The information is stored and accumulated in the voice database in association with the user attribute information (step SJ51). Further, the correct recognition content of the “unrecognizable voice” is acquired at an appropriate timing and stored in the same voice database (step SJ52).

(Speech inquiry process 2-1 in speech translation)
FIG. 8 is a flowchart illustrating another example of processing including speech inquiry in the speech translation system 100. Such a speech inquiry also constitutes a part of speech translation processing by the speech translation system 100. The processing here is the same as that shown in FIG. 7 except that “user (other speaker)” performs voice input instead of “user (speaker)” and executes the processing of step SJ6 instead of step SJ5. This is substantially the same as the processing shown. Therefore, the description of processes other than those related to this difference will be omitted below. In addition, the speech translation process shown in FIG. 8 is an example of a procedure executed when the speech of the user (another speaker) cannot be recognized (after speech recognition is once executed), as in the example shown in FIG. It is.

  When the user (other speaker) information is input from the information terminal 10 as in the process illustrated in FIG. 7, the processor 21 proceeds to the voice inquiry process (step SJ6). Here, the processor 21 calls and references the voice database from the storage resource 23, and narrows down by the user attributes (age, gender, etc.).

  Specifically, for example, when the user is a man in his 70s, the processor 21 extracts (selects) “unrecognizable speech” associated with “male in his 70s” stored in the speech database, Alternatively, if necessary, a new secondary voice database is created from the extracted voice data. Similarly, for example, when the user is a woman (girl) younger than 10 years, the processor 21 extracts (screens) “unrecognizable speech” associated with “female under 10 years” stored in the speech database. Or, if necessary, a new secondary audio database is created from the extracted audio data.

  Then, similarly to the process shown in FIG. 7, when the speech recognition is “possible” following the speech input (step ST1) (“Yes” in step SJ1), the processor 21 proceeds to the multilingual translation process. Transition (step SJ2). On the other hand, when the speech recognition is “impossible” (“No” in step SJ1), the processor 21 proceeds to the voice inquiry process (step SJ6) again. Here, the processor 21 calls and refers to the voice database from the storage resource 23 and refers to the “unrecognizable voice” extracted based on the user's attributes for the voice that could not be recognized (or a new voice created from them). In the voice database (step SJ61).

(Speech inquiry process 2-2 in speech translation)
FIG. 9 is a flowchart showing another example of processing including speech inquiry in the speech translation system 100. Such a speech inquiry also constitutes a part of speech translation processing by the speech translation system 100. In this processing, instead of “user (speaker)”, “user (other speaker)” performs information input (step ST0) and voice input (step ST1), and step is replaced with step SJ5. Except for executing the processing of SJ7, it is substantially the same as the processing shown in FIG. Therefore, the description of processes other than those related to this difference will be omitted below. Moreover, the speech translation process shown in FIG. 9 is an example of a procedure for executing the speech inquiry process before recognizing the speech of the user (another speaker).

  That is, a user (another speaker) inputs user information (step ST0), and the processor 21 of the server 20 that has received the information signal moves to a voice inquiry process (step SJ7). Here, the processor 21 calls and references the voice database from the storage resource 23, and narrows down by the user attributes (age, sex, etc.) (step SJ70; same as step SJ60). Next, the user (another speaker) inputs the voice in Japanese (step ST1), and the processor 21 that receives the voice signal calls the voice database from the storage resource 23 and refers to the input voice. Are collated with “unrecognizable speech” (or a new speech database “unrecognizable speech” created from them) extracted based on the attribute of step SJ71; corresponding to step SJ61).

  Then, the processor 21 determines whether or not the input voice corresponds to “unrecognizable voice” associated with the attribute (step SJ72; substantially included in the process of step SJ71). For example, when it is determined that the degree of coincidence or similarity or the like in the voice matching between the two is greater than or equal to a predetermined value, the processor 21 determines that the “applicable” (“Yes” in step SJ72) corresponds to the corresponding “ The “correct recognition content” of “unrecognizable speech” is output to the multilingual translation processing (step SJ2) side as the “correct recognition content” of the input speech (step SJ73).

  On the other hand, if it is determined that the degree of coincidence or similarity in the voice matching between the two is less than a predetermined value, the processor 21 determines that the voice is “not applicable” (“No” in step SJ72), and performs voice recognition processing ( The process proceeds to step SJ1). That is, in this case, the voice by the user (another speaker) is not “unrecognizable voice”, and is recognized by the normal voice recognition process, or the possibility thereof is extremely high.

  As described above, in the example of the speech translation process shown in FIG. 9, the speech inquiry process is executed before recognizing the speech of the user (another speaker) as in the example shown in FIG. The determination process in the voice recognition process (step SJ1) as shown in FIG.

[Third Embodiment]
(Acoustic model generation processing in speech translation)
FIG. 10 is a flowchart showing an example of processing including acoustic model generation (improvement) in the speech translation system 100. Such acoustic model generation also constitutes part of speech translation processing by the speech translation system 100.

  Here, first, the process up to the construction of the speech database shown in FIG. 4 (first embodiment) (steps ST1, ST2, SJ5) or the process up to the construction of the speech database shown in FIG. 7 (second embodiment) (steps). ST0, ST1, ST2, SJ5) are executed.

  Next, the processor 21 of the server 20 moves the process to step SJ8, and calls the conventional acoustic model (first acoustic model) used for speech recognition as the model M20 from the storage resource 23 (step SJ81). Then, a known adaptive process using the “unrecognizable speech” stored in the speech database (for example, the adaptive process cited in Patent Document 1) is performed on the conventional acoustic model (step SJ82). Thus, a new acoustic model (second acoustic model) is generated (step SJ83) and stored in the storage resource 23.

  In step SJ83, when a speech database constructed by the processing shown in FIG. 4 (first embodiment) is used, one new acoustic model can be generated from one conventional acoustic model. Moreover, when using the audio | voice database constructed | assembled by the process shown in FIG. 7 (2nd Embodiment), several new acoustic models for every user attribute (age, generation, sex, etc.) from one conventional acoustic model, for example. Can be generated. At this time, the new acoustic model is stored in the storage resource 23 in association with the user attribute. Hereinafter, a case where a new acoustic model is generated for each attribute as in the latter case will be described as an example.

  Next, when the user (other speaker) information is input from the information terminal 10 as in FIG. 8 (second embodiment), the processor 21 proceeds to acoustic model selection processing (step SJ9). Here, the processor 21 designates a new acoustic model according to the user's attributes (age, sex, etc.) from among a plurality of new acoustic models stored in the storage resource 23, and uses it for speech recognition. Make preparations.

  And about the process (step SJ1, SJ2, SJ3) after step SJ9 in 3rd Embodiment, it replaces with the conventional acoustic model, and uses the acoustic model newly created and selected according to the attribute. Since it is substantially the same as a series of processes in FIG. 6 (first embodiment) and FIG. 9 (second embodiment), description thereof is omitted here.

  According to the speech translation system 100 configured as described above, the control method thereof, and the speech translation program, speech (unrecognizable speech) that was not initially recognized among speech uttered by the user (speaker) is collected in advance. In addition, by appropriately acquiring the correct recognition contents, a voice database including both data is constructed and stored in the storage resource 23.

  Then, the speech uttered by the user (other speaker) cannot be recognized by normal processing by comparing it with the unrecognizable speech stored in the speech database after speech recognition or prior to speech recognition. It is possible to easily obtain the correct content of speech recognition. Therefore, the voice recognition itself can be performed without increasing the device cost due to high accuracy and high-speed processing, complicating various processes and decreasing translation accuracy, and increasing the burden on the user and reducing convenience. It is possible to improve the accuracy of recognition and thus the accuracy of speech translation easily and efficiently.

  Further, as described above, when the user is an elderly person or a child, the recognition rate of the spoken speech tends to be low. In the speech translation system 100, the unrecognizable speech and the correct recognition contents thereof are displayed. It can be stored in the storage resource 23 as a voice database in association with attributes (age, generation, sex, etc.). Therefore, by narrowing down the corresponding unrecognizable speech according to the user attribute, the speech recognition rate of the speech uttered by the user, and thus the speech translation rate, can be further increased.

  Furthermore, according to the speech translation system 100, the unrecognizable speech stored in the storage resource 23 is used to perform an adaptive process on the conventional acoustic model to generate a new acoustic model. Speech recognition can be performed using a new acoustic model. Even if it does in this way, the increase in the apparatus cost resulting from the high precision and high-speed processing of voice recognition itself, the complexity of various processes and the fall of translation accuracy, and the increase in a user's burden and the convenience will be caused. Therefore, the accuracy of speech recognition and thus the accuracy of speech translation can be improved easily and efficiently.

  In this case, a new acoustic model is generated for each attribute of the user (speaker), for example, age or age range (generation) or gender, and the new acoustic model corresponding to the attribute of the user is used. Recognition can also be performed. In this way, since speech recognition using an acoustic model corresponding to the user's attribute is possible, the speech recognition rate of the speech uttered by the user, and hence the speech translation rate, can be further increased.

  Note that, as described above, each of the above embodiments is an example for explaining the present invention, and is not intended to limit the present invention to the embodiment. The present invention can be variously modified without departing from the gist thereof. For example, those skilled in the art can replace the resources (hardware resources or software resources) described in the embodiments with equivalents, and such replacements are also included in the scope of the present invention.

  Moreover, although each said embodiment described the example which performs each process of speech recognition, translation, and a speech synthesis by the server 20, you may comprise so that these processes may be performed in the information terminal 10. FIG. In this case, the module L20 used for these processes may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. Furthermore, the database D20 of the voice database and / or the model M20 such as an acoustic model may be stored in the storage resource 12 of the information terminal 10, or may be stored in the storage resource 23 of the server 20. As described above, the speech translation system may not include the network N and the server 20.

  Furthermore, the process shown in FIG. 5 or 6 (first embodiment) or the process shown in FIG. 8 or 9 (second embodiment) may be combined with the process shown in FIG. 10 (third embodiment). That is, the voice inquiry process in the first or second embodiment may be executed before or after the generation and use of the new acoustic model described as the third embodiment. Furthermore, the original speech whose content could not be properly translated in multilingual translation was stored as a kind of “unrecognizable speech” or “untranslatable speech” in the speech database. You may accumulate. Furthermore, in the third embodiment, a new acoustic model that does not depend on user attributes may be generated without generating a new acoustic model for each user attribute.

  Of course, a gateway server for converting a communication protocol between the information terminal 10 and the network N may be interposed. The information terminal 10 is not limited to a portable device, and may be a desktop personal computer, a notebook personal computer, a tablet personal computer, a laptop personal computer, or the like.

  According to the present invention, it is possible to easily obtain correct recognition content of speech that cannot be recognized by normal processing. For example, a program in the field of providing services related to conversations between people who cannot understand each other's language, It can be widely used for activities such as designing, manufacturing, providing and selling systems and methods.

10 Information terminal (speech translation system)
11 processor 12 storage resource 13 voice input / output device 14 communication interface 15 input device 16 display device 17 camera 20 server (speech translation system)
21 processor 22 communication interface 23 storage resource 100 speech translation system D20 database L20 module M20 model N network P10, P20 program

Claims (8)

A voice input unit for inputting the voice of the speaker;
A voice recognition unit for recognizing the content of the voice input to the voice input unit;
A translation unit that translates the content recognized by the voice recognition unit into content in a different language;
A speech synthesizer for synthesizing speech of the content translated by the translation unit;
A voice output unit for outputting the voice synthesized by the voice synthesis unit;
A storage unit for storing unrecognizable speech that could not be recognized by the speech recognition unit;
With
The voice recognition unit includes the following (1) or (2) ;
( 1 ) Let the user different from the speaker hear the unrecognizable voice stored in the storage unit, and receive the correct recognition content of the unrecognizable voice from the user.
( 2 ) Requesting a user different from the speaker to input the correct recognition content of the unrecognizable speech stored in the storage unit, and receiving the correct recognition content of the unrecognizable speech from the user.
Run the processing,
The storage unit stores the unrecognizable speech and the correct recognition content of the unrecognizable speech as a speech database,
The speech recognition unit refers to the speech database, collates the speech of another speaker input to the speech input unit with the unrecognizable speech stored in the speech database, and based on the collation result, Executing a voice query process for providing the correct recognition content of the voice to the translation unit;
Speech translation system. The voice recognition unit executes the voice inquiry process when the voice of the other speaker cannot be recognized or before the voice of the other speaker is recognized.
The speech translation system according to claim 1. An information acquisition unit for acquiring information related to attributes of the speaker and the other speaker;
The storage unit stores the unrecognizable speech and the correct recognition content as the speech database in association with the attributes of the speaker,
The speech recognition unit collates the attribute of the other speaker with the attribute stored in the speech database, and executes the speech inquiry process based on the collation result.
The speech translation system according to claim 1 or 2. The attribute is the speaker's age or age range or gender,
The speech translation system according to claim 3. A second acoustic model is obtained by performing adaptive processing using the unrecognizable speech and the correct recognition content of the unrecognizable speech on the first acoustic model used for recognizing the input speech in the speech recognition unit. An acoustic model generation unit for generating
The speech translation system according to any one of claims 1 to 4. An information acquisition unit for acquiring information related to the attribute of the speaker;
The acoustic model generation unit generates the second acoustic model for each attribute of the speaker,
The speech recognition unit recognizes the content of the input speech using the second acoustic model corresponding to the attribute of the other speaker.
The speech translation system according to claim 5. A method for controlling a speech translation system including a speech input unit, a speech recognition unit, a translation unit, a speech synthesis unit, a speech output unit, and a storage unit,
Inputting a voice of a speaker by the voice input unit;
Recognizing the content of speech input to the speech input unit by the speech translation unit;
Translating the content recognized by the speech recognition unit into content of a different language by the translation unit;
Synthesizing speech of contents translated by the translation unit by the speech synthesis unit;
Outputting the voice synthesized by the voice synthesis unit by the voice output unit;
Storing unrecognizable speech that could not be recognized by the speech recognition unit by the storage unit;
Have
In the step of recognizing the contents of the voice, the following (1) or (2) ;
( 1 ) Let the user different from the speaker hear the unrecognizable voice stored in the storage unit, and receive the correct recognition content of the unrecognizable voice from the user.
( 2 ) Requesting a user different from the speaker to input the correct recognition content of the unrecognizable speech stored in the storage unit, and receiving the correct recognition content of the unrecognizable speech from the user.
Run the processing,
In the storing step, the unrecognizable speech and the correct recognition content of the unrecognizable speech are stored as a speech database,
In the step of recognizing the content of the speech, the speech database is further referred to, the speech of another speaker input to the speech input unit is compared with the unrecognizable speech stored in the speech database, Based on the collation result, a voice inquiry process for providing correct translation content of the voice to the translation unit is performed.
A method for controlling a speech translation system. Computer
Voice input unit for inputting the voice of the speaker,
A voice recognition unit for recognizing the content of the voice input to the voice input unit;
A translation unit that translates the content recognized by the voice recognition unit into content in a different language;
A speech synthesizer for synthesizing speech of the content translated by the translation unit;
A voice output unit for outputting the voice synthesized by the voice synthesis unit;
A storage unit for storing unrecognizable speech that could not be recognized by the speech recognition unit;
Function as
In the voice recognition unit, the following (1) or (2) ;
( 1 ) Let the user different from the speaker hear the unrecognizable voice stored in the storage unit, and receive the correct recognition content of the unrecognizable voice from the user.
( 2 ) Requesting a user different from the speaker to input the correct recognition content of the unrecognizable speech stored in the storage unit, and receiving the correct recognition content of the unrecognizable speech from the user.
To execute the processing,
In the storage unit, the unrecognizable speech and the correct recognition content of the unrecognizable speech are stored as a speech database,
The speech recognition unit refers to the speech database, collates the speech of another speaker input to the speech input unit with the unrecognizable speech stored in the speech database, and based on the collation result, Causing voice translation processing to provide the translation unit with the correct recognition content of the voice,
Speech translation program.

Images