WO2018186416A1

WO2018186416A1 - Translation processing method, translation processing program, and recording medium

Info

Publication number: WO2018186416A1
Application number: PCT/JP2018/014317
Authority: WO
Inventors: 旋造田代
Original assignee: 旋造田代
Priority date: 2017-04-03
Filing date: 2018-04-03
Publication date: 2018-10-11
Also published as: JP2018180590A; JP6243071B1

Abstract

[Problem] To provide a translation processing method that enables translation variations to be made multifunctional and that can improve versatility. [Solution] As steps to be executed by one communication terminal that includes a storage means onto which is pre-installed a translation application that executes mutual translation between a base language and a second language, the following steps are provided: a communication content reception step for receiving at least one input, via an input means, as transmission reference data, such input being audio data, text data, or image data in the base language; a to-second-language translation step in which the translation application is initiated, the transmission reference data received in the communication content reception step is translated into at least one of audio data, text data, and image data in the second language to thereby generate transmission data; a transmission step for transmitting, by a transmission means, the transmission data which was translated in the to-second-language translation step; a reception step for receiving, using a reception means, at least one the audio data, text data, and image data in the second language as reception reference data; a to-base-language translation step in which the translation application is initiated, the reception reference data received in the reception step is translated into at least one of audio data, text data, and image data in the base language to thereby generate reception data; and a communication content output step for outputting, via an output means, the reception data which was translated in the to-base-language translation step.

Description

Translation processing method, translation processing program, and recording medium

The present invention relates to a translation processing method, a translation processing program, and a recording medium for mutual translation between a basic language and another language when communicating with each other via a telecommunication line.

Conventionally, if the two languages are different when communicating with each other via a telecommunication line, such as a call (conversation) via a telephone line, a chat via an internet line or a round-trip letter by E-Mail, Various translation functions for translating for understanding are well known.

In this case, in the case of face-to-face conversations, if one of them has a translator, it is possible to establish a conversation and communicate even if they do not have language skills for the other party's language. Is possible.

On the other hand, unlike E-Mail etc., where there is a time lag between receiving and replying, and translation is possible during that time, such as calls and chats (including timelines of various SNSs), etc. It is necessary to realize two kinds of communication, that is, translation from a basic language to another language on the transmission side and translation from another language to the basic language at the time of reception.

Therefore, even if they do not have a translator (translation function), they can listen to the voice and convert it into text data in order to realize a different language when sending and a basic language when receiving. A technique is known in which text data after translation is generated, and the text data is further read aloud to enable near real-time communication (see, for example, Patent Document 1).

JP 2005-513619 A

However, the translator disclosed in Patent Document 1 is merely a category of simultaneous interpretation that realizes listening and reading at the time of a voice call, and has a problem of low versatility.

An object of the present invention is to provide a translation processing method, a translation processing program, and a recording medium that can make a variation of translation multifunctional and improve versatility.

In order to achieve the above object, in the invention described in claim 1, as a step of causing a single communication terminal having a storage unit, in which a translation application for performing mutual translation between a basic language and another language is installed, to be executed, A communication content receiving step for receiving at least one of speech data and text data in the basic language as transmission reference data via the input means, and starting the translation application and sending reference data received in the communication content receiving step. Other language translation step for translating into speech data and text data in other languages to generate transmission data, transmission step for transmitting the transmission data translated in the other language translation step, and voice data in other languages Receive text data as reception reference data by receiving means A basic verbalization translation step of translating the reception reference data received in the reception step by activating the translation application into speech data and text data in a basic language, and a basic languageization translation step And a communication content output step for outputting the translated received data via the output means.

In the invention according to claim 2, in the transmission step, the transmission data is transmitted by a transmission means through a telecommunication line, and in the reception step, the reception reference data is received by a reception means through a telecommunication line. , That is.

In the invention according to claim 3, the telecommunications line is an internet line.

In the invention according to claim 4, the telecommunications line is a telephone line.

In the invention according to claim 5, the transmission step is executed without going through a telecommunication line, and the receiving step is executed without going through a telecommunication line.

In the invention according to claim 6, before executing the other language translation step, the other language to be translated based on address data including a registered name of a communication partner registered in advance in the storage means And a different language specifying step for specifying.

In the invention according to claim 7, when the transmission reference data received by the input means is voice data in the basic language, the listening application installed in the storage means in advance is started and the other language is started. A basic text data generating step for generating basic text data in the basic language for generating the outgoing data composed of text data according to the above.

In the invention according to claim 8, when the transmission reference data received by the input means is text data in the basic language, a reading application installed in advance in the storage means is started to activate the other language. A basic voice data generation step of generating basic voice data in the basic language for generating the outgoing data composed of voice data according to.

In the invention according to claim 9, when the transmission reference data received by the input means is text data in the basic language, the speech data in the other language translated in the other language translation step A transmission step of transmitting text data in the basic language received by the input means in addition to the transmission data by the transmission means.

In the invention according to claim 10, from the time when the input means accepts the voice data in the basic language to the time when the sending means in the other language translated in the other language is sent by the sending means. The period includes an interrupt canceling step for canceling reception by the receiving means.

In the invention according to claim 11, when the transmission reference data received by the input means is a sign language operation in the basic language, the sign language analysis application installed in the storage means in advance is activated to perform the sign language operation. A basic analysis data generating step for generating basic analysis data in the basic language for generating the transmission data composed of at least one of voice data, text data, and image data in the other language, , Are provided.

In the invention described in claim 12, the translation processing method described in any one of claims 1 to 6 is executed by a calculation means of a communication terminal.

The invention according to claim 13 is a recording medium on which a translation processing program for causing a computing means of a communication terminal to execute the translation processing method according to any one of claims 1 to 6 is recorded.

According to the present invention, the input data in the basic language received via the input means can be generated with the output data after being converted into another language, and the output data can be output simultaneously. With respect to the data, output data after being converted into a basic language can be generated and output through the output means at the same time.

This makes it possible for a single communication terminal to achieve mutual translation between the basic language and another language, even for visually impaired people or hearing impaired people. Moreover, even if the eyes and ears are not inconvenient, it is possible to have text data mutually, and it is possible to suppress misrecognition of the contents of a call instead of a call memo or during an important call such as a contract. .

Therefore, even if it has a translation function only on one side, both voice and text data can be used effectively, and translation variations can be made multifunctional to improve versatility. be able to.

In the transmission step, the transmission data is transmitted by the transmission means through the telecommunication line, and in the reception step, the reception reference data is received by the reception means through the telecommunication line. Transmission data (in other languages) can be transmitted, and reception reference data (in other languages) before translation can be received.

Here, if the telecommunication line is an internet line, it can be used for mutual translation using SNS or the like, and if the telecommunication line is a telephone line, a TV including a normal voice call or a video signal such as a video signal. It can be used for mutual translation using telephone.

Also, since the sending step is executed without going through the telecommunications line and the receiving step is run without going through the telecommunications line, it can be used for mutual translation such as face-to-face interpretation without using the telecommunications line.

In this way, for voice input or text (including images / videos) input in a basic language, both voice data and text data after being converted to another language are generated, and at the same time, both the data are transmitted to the other party. In addition, when receiving voice data or text data in another language, both voice data and text data after being converted into a basic language can be generated, and at the same time, both data can be output as voice and text.

In addition, before executing the other language translation step, by providing another language identification step for identifying another language to be translated based on the address data including the registered name of the communication partner registered in advance in the storage means, Communication relations without having to select / designate other languages using phone numbers (country codes, etc.) or e-mail addresses (domain names, etc.) included in the address data for specifying the destination or receiver in advance. The translation function can be validated by specifying the country or language before establishing.

In addition, when the transmission reference data received by the input means is voice data in a basic language, a basic language for generating transmission data composed of text data in another language by starting a listening application installed in advance in the storage means By providing the basic text data generation step for generating the basic text data according to the above, it is possible to effectively use the text data of the basic language as the translation source.

In addition, when the transmission reference data received by the input means is text data in a basic language, a basic language for generating a transmission data composed of voice data in another language by starting a reading application installed in the storage means in advance. By providing the basic speech data generation step for generating the basic speech data according to the above, it is possible to effectively use the text data of the basic language as the translation source.

In addition, when the transmission reference data received by the input means is text data in the basic language, the text data in the basic language received by the input means in addition to the transmission data by the speech data of other languages translated in the other language translation step By providing a sending step for sending the message by the sending means, the text data of the basic language as the translation source can be effectively used as a substitute for mistranslation or memo.

In addition, an interrupt canceling step for canceling reception by the receiving means is provided during a period from when the input means accepts the voice data in the basic language to when the sending means sends the outgoing data in another language translated in the other language translation step. Thus, the voice input of the other party can be rejected until the intentional utterance is completed, and the turbidity (overlap) of the conversation can be suppressed.

In addition, when the call reference data received by the input means is a sign language action in a basic language, the sign language action is analyzed by starting a sign language analysis application installed in the storage means in advance, and then voice data / text data in another language A language is used as a sign language operation by providing a translation processing method including a basic analysis data generation step for generating basic analysis data in a basic language for generating transmission data composed of at least one of image data In some cases, it can be used for mutual translation.

Moreover, according to the translation processing program of the present invention, the above-described translation processing method can be executed by the computing means of the communication terminal.

According to the recording medium of the present invention, a translation processing program for causing the calculation means of the communication terminal to execute the translation processing method described above can be easily used.

The translation processing method concerning one embodiment of the present invention, a translation processing program, and a system application example concerning a recording medium are shown, (A) is an explanatory view of an example which installs a translation application in a communication terminal, (B) is It is a block block diagram of the principal part of a communication terminal. It is explanatory drawing which shows the example of simultaneous interpretation of the text data by a chat system. It is explanatory drawing of the usage example of the speech data and text data after other languages based on the speech input by a basic language. (A) is an explanatory diagram of an example of using voice data and text data after other languages based on text input in a basic language, and (B) is voice data and text data after being converted into a basic language based on voice input in another language. It is explanatory drawing of the usage example. It is a flowchart of an example of the translation routine which the control part which concerns on one embodiment of this invention performs. It is explanatory drawing of an example of the translation mode selection screen of the communication terminal which concerns on one embodiment of this invention. It is explanatory drawing which shows the example of the sign language interpretation of the text data by a chat system.

Next, an embodiment according to the present invention will be described with reference to the drawings. In the following description, for example, the directions of the top, bottom, left, and right sides of the communication terminal 1 shown in FIG. 1A are the same as the top, bottom, left, right, and back sides of the page with reference to the state shown in FIG. It will be explained as a thing. The communication terminal 1 according to the present embodiment causes a single communication terminal 1 having storage means in which a translation application for performing mutual translation between a basic language and another language is installed in advance.

As shown in FIG. 1A, the communication terminal 1 realizes a voice input unit 2 and a voice output unit 3 for realizing a telephone call by a telephone function through various communication methods of the telecommunication line NT, and a display function. Display unit 4, a storage unit 5 that stores a program related to overall functions as communication terminal 1 including a translation processing program for realizing a translation processing method to be described later, and a communication terminal according to a program stored in storage unit 5 And a control unit (CPU) 6 that processes one of various functions.

Note that the communication terminal 1 shown in FIG. 1A is a smartphone, but has a computer function such as a tablet terminal or a personal computer. In addition to the voice data call function, the text data chat function E -Any terminal that can use the Mail function, SNS (Social Networking Service) function, etc. is not particularly limited.

It should be noted that the various communication methods can include mainly a wireless telephone communication line method, an Internet connection line method, a short-distance wireless communication method such as infrared ray or Bluetooth (registered trademark), and the like.

The voice input unit 2 allows voice input from a built-in microphone (microphone) provided below the communication terminal 1. The voice input unit 2 is connected to a tag, a connector (not shown) or the like provided on the peripheral surface of the communication terminal 1 or a short-distance wireless communication connection such as Bluetooth (registered trademark). Voice input from a microphone can be included.

The audio output unit 3 allows audio output from a built-in speaker provided above the communication terminal 1. The audio output unit 3 is connected to a tag, a connector (not shown) or the like provided on the peripheral surface of the communication terminal 1 or a short-distance wireless communication connection such as Bluetooth (registered trademark). Audio output from the speaker can be included.

The display unit 4 can permit various operations using a touch panel system in addition to image display using a color liquid crystal panel or the like. Various operations by the user can include operations by a switch 7 provided on the peripheral surface or the surface of the communication terminal 1 in addition to the so-called touch operation using the display unit 4.

Therefore, in the following description, in the case of the description specialized for the display function of the display unit 4, it will be described as the display unit 4, and all operations other than the touch panel operation and the switch operation that are not particularly limited are shown in FIG. As described above, the operation by the operation unit 8 that outputs the operation signal (command signal) to the control unit 6 will be generically described.

The display unit 4 activates, for example, a telephone icon AP1 for executing a telephone function, an E-Mail icon AP2 for executing an E-Mail function, a chat icon AP3 for executing a chat function, and a translation application. Various function icons such as a translation icon AP4 are displayed.

The storage unit 5 is a so-called ROM (Read only memory) or RAM (Random access memory), and realizes a computer function with the control unit 6.

In addition, the storage unit 5 stores internal data, removable external storage using various media, online storage including cloud C by Internet connection, etc. as a storage location for data according to the usage form such as primary storage, etc. Can be included.

In addition, the storage unit 5 may be referred to as an execution program (hereinafter referred to as “translation driver”) of a translation application (hereinafter simply referred to as “application”) downloaded through a telecommunication line (Internet connection line) NT, for example. Is stored in advance.

Further, the storage unit 5 stores address information (address book) for displaying the name of the other party on the display unit 4 in a table system when a call is made or received by the telephone function.

In this address information, in addition to the name for identifying the other party, the telephone number of the other party in the case of a telephone function, and the E-Mail address (including the domain name) of the other party in the case of an E-Mail function by Internet connection ), An SNS function that is a community-based membership system service can include member registration information (appropriate member public data under personal information protection) with the service provider.

These pieces of personal information are appropriate and indispensable information when using each function, and can identify the country and language.

As shown in FIG. 1B, the control unit 6 of the communication terminal 1 receives, from the reception interface (I / F) 9, reception data when a radio wave is received by a built-in antenna (not shown), for example. . Similarly, the control unit 6 transmits transmission data when transmitting a radio wave by the built-in antenna to the transmission interface (I / F) 10.

The control unit 6 controls the speech recognition unit 11, the speech generation unit 12, the character recognition unit 13, and the character generation unit 14 for translation processing using a translation driver, and transmits / receives voice data or text data. In addition, the control unit 6 displays an image (for example, JPG method or the like) or video (moving image or moving image with sound: MP4 or the like) photographed by the photographing camera 15 provided on the front side (and / or the back side) of the communication terminal 1. , An image control function is provided for storing in the storage unit 5 or for use in a TV phone or sign language translation described later.

The voice recognition unit 11 receives voice data based on the voice input by the user to the voice input unit 2 or voice data received via the reception interface 9. The voice recognition unit 11 has a reading function for analyzing the voice data.

The voice generation unit 12 uses voice data output from the voice output unit 3 or voice data transmitted from the transmission interface 10 based on voice data output from the voice recognition unit 11 or text data output from the character recognition unit 13. To generate. The voice generation unit 12 has a reading function for outputting the generated voice data from the voice output unit 3.

The text recognition unit 13 receives text data based on a character input touched by the user on the display unit 4 or text data received via the reception interface 9. The character recognition unit 13 recognizes the character data code of the text data.

The character generation unit 14 generates text data to be displayed on the display unit 4 or text data to be transmitted from the transmission interface 10 based on voice data output from the speech recognition unit 11 or text data output from the character recognition unit 13. Generate a character data code.

The photographing camera 15 can photograph the front side of the communication terminal 1 in the self-shooting mode, for example. In addition, since the camera function itself is well-known, the detailed description is abbreviate | omitted here.

The control unit 6 can cause the sign language analysis unit 16 to analyze the video imaged by the imaging camera 15.

The sign language analysis unit 16 has a function of analyzing operations related to sign language photographed by the photographing camera 15 and generating, for example, character data and voice data.

Also, the sign language analysis unit 16 can analyze the motion related to the sign language photographed by the photographing camera 15 and generate substitute image data such as a simple motion image including a hand image and a face image. A detailed example of the generation of the character / sound / image data will be described later.

In such a basic configuration, the control unit 6 sets the language (for example, Japanese) set by the operating system stored in the storage unit 5 as a basic language that is normally used by the user, and stores the address stored in the storage unit 5 by the translation driver. The language (for example, English) used by the communication partner included in the personal information of the information is set as another language to be translated.

Note that the other language is not limited to one language and can be a language according to the type of translation driver. Therefore, the translation driver can use one or more stored in the storage unit 5.

As shown in FIG. 2, when the user selects the chat icon AP3 displayed on the display unit 4 of the communication terminal 1, the control unit 6 activates the chat function and causes the display unit 4 to display a chat screen.

At this time, if the other party selects the other party from the other party selection screen attached to the chat function, it is possible to automatically identify whether the other party uses the basic language or another party. Thereby, when the other party is a user who uses another language, the control unit 7 executes a function of translating the corresponding used language into another language.

In the following description, the communication terminal 21 used by the other party is also a smartphone including an audio input unit 22 and an audio output unit 23, and a display unit 24 for realizing a display function. At this time, it is assumed that no translation application is installed in the communication terminal 21.

Also, in the following explanation, for convenience of explanation, input / output on words such as voice and characters is indicated by “XX”, input / output on screen display is indicated by [XX], and data processing The above input / output is indicated by <XX>.

In addition, since the text data is a character code, it is also referred to as a simple “code” or “character code” in the following description. The voice data or text data is also simply referred to as “voice” or “text” (or “character” converted for display).

Here, the user refers to the character input pad displayed at the bottom of the screen of the display unit 4 and performs a touch operation. For example, “Tone Doo? ”Is entered in the upper right of the screen of the display unit 4 with a frame [a tone? ] Is displayed. The character code data processing at this time uses known processing.

On the other hand, the control unit 6 recognizes the character code input by the character recognition unit 13, and generates a character code (for example, <How have you ＞ been>) of another language from the recognized character code using a translation driver. The control unit 6 transmits the generated character code <How have you been> to the communication terminal 21 of the other party via the transmission interface 10.

The partner communication terminal 21 (the control unit thereof) displays the character [How ［have you been] on the upper left side of the screen of the display unit 24 in a chat mode based on the received character code <How have you been>.

In response to this, when the partner inputs the reply character “so and so”, the character [so and so] is displayed at the upper right of the screen of the display unit 24 of the communication terminal 21 owned by the partner, The character code <so and so> is sent.

The control unit 6 of the communication terminal 1 that has received the character code <so and so> causes the character recognition unit 13 to recognize the character code <so and so>, and then translates the character code <maaama>. > Is generated. Further, the control unit 7 displays a character [maama] on the upper left of the screen of the display unit 4.

As described above, regarding the mutual communication between the text data and the text data between the communication terminal 1 and the communication terminal 21, the language used by the partner is another language, and the translation application is installed in the partner communication terminal 21. Even if not, communication can be easily established.

In addition to the above, the control unit 6 that enables such a basic translation function includes a speech translated into another language by simply inputting either speech or characters in the basic language using its own communication terminal 1. Output and character notation can be realized in the communication terminal 21 of the other party.

In addition, the control unit 6 causes the communication terminal 1 to realize the voice output and the character notation translated into the basic language only by inputting either one of speech or characters in another language using the communication terminal 21 by the other party. be able to.

Hereinafter, an example of voice input in the communication terminal 1 (see FIG. 3) and character input in the communication terminal 1 (see FIG. 4) will be described. Here, the communication terminal 1 uses a chat application with voice or a telephone application with character display. Note that a translation with a character output function may be additionally programmed in a mode to a normal telephone application (phone icon AP1).

Similarly, a normal chat application (chat icon AP3) may be programmed to add translation with voice function in a mode. Further, it is assumed that when the user starts up these applications and designates a partner, the control unit 6 determines to translate based on information stored in the address data.

As shown in FIG. 3, when the user inputs “tone” to the voice input unit 2, the control unit 6 causes the voice recognition unit 11 to read the voice and generate a voice <tone>. To do.

Next, the control unit 6 causes the character generation unit 14 to generate a character <tone> on the basis of the read voice <tone> and causes the display unit 4 to display the character [tone]. The translated character <How have you been> is generated and the character [How have you been] is displayed on the display unit 4. Further, the control unit 6 causes the voice generation unit 12 to generate a voice <How have you been> based on the generated character <How have you been>.

Thereby, the control unit 6 superimposes the generated three data of the character <tone>, the character <How ＜ have you been>, and the voice <How have you been> from the transmission interface 10 to the communication terminal 21. Send to.

Based on the received data, the communication terminal 21 displays the pre-translation characters [Tone Doo] and the post-translation characters [How have you been] on the display unit 24 and the voice “How have you been”. Output from the output unit 23.

In this way, the communication terminal 21 can acquire three data of the character <tone>, the character <How have you been>, and the voice <How have you been> without installing the translation application. Can do.

On the other hand, when the user of the communication terminal 21 inputs “so and so” to the voice input unit 22 in response to the voice “How have you been”, the voice <so and so> is transmitted to the communication terminal 1. To do.

The control unit 6 of the communication terminal 1 causes the voice recognition unit 11 to recognize the voice <so 受信 and so> when the reception interface 9 receives the voice <so and so>.

Thereby, the control unit 6 causes the character generation unit 14 to generate the character <so ＜ and so> and causes the display unit 4 to display the character [so］ and so], and also generates and displays the translated character <maamaa>. The character “maamaa” is displayed in part 4. Further, the control unit 6 causes the voice generation unit 12 to generate a voice <maamaa> based on the generated character <maaama>, and causes the voice output unit 3 to output the transliterated voice “maaamaa”.

As described above, the communication terminal 1 does not have the translation application installed on the communication terminal 21, so that the pre-translation character <so and so>, the post-translation character <ma-ama>, and the post-translation voice <ma-ama> Can be generated on the display unit 4 and the audio output unit 3 to be recognized by the user. Note that these three pieces of data may be generated by the communication terminal 21 as long as the translation application is installed in the communication terminal 21.

As shown in FIG. 4, when the user inputs a character “tone” using the character input pad displayed on the display unit 4, the control unit 6 causes the character recognition unit 13 to recognize the character code.

Next, based on the character <tone>, the control unit 6 causes the character generator 14 to generate the character <tone> and display the character [tone] on the display unit 4 and translate it. The character <How have you been> is generated and the character [How have you been] is displayed on the display unit 4.

Also, the control unit 6 causes the voice generation unit 12 to generate a voice <How have you been> based on the generated character <How have you been>.

On the other hand, when the user of the communication terminal 21 inputs “so and so” to the voice input unit 22 (or inputs the characters “so に and so”) to the voice “How have you been”, the voice <so and so> (or characters <so and so>) are transmitted to the communication terminal 1.

When the control unit 6 of the communication terminal 1 receives the voice <so and so> (or the character <so and so>) by the reception interface 9, the voice recognition unit 11 causes the voice <so and so> (or the character recognition unit 13). To recognize the characters <so and so>).

Thereby, the control unit 6 causes the character generation unit 14 to generate the character <so ＜ and so> and causes the display unit 4 to display the character [so］ and so], and also generates and displays the translated character <maamaa>. The character “maamaa” is displayed in part 4.

In addition, the control unit 6 causes the voice generation unit 12 to generate a voice <maamaa> based on the generated character <maamaa>, and causes the voice output unit 3 to output the translated voice “maaama”.

Next, a specific routine executed by the control unit 7 will be described based on the flowchart of FIG.

As described above, the communication terminal 1 is preinstalled with a translation application that executes mutual translation between the basic language and another language in the storage unit 5 and stores an address book.

(Step S1)
In step S 1, the control unit 6 receives the designation of the communication application by the operation of the operation unit 8 by the user, refers to the address book corresponding to the communication application from the storage unit 5, and causes the display unit 4 to display the address book. The process proceeds to step S2.

(Step S2)
In step S2, the control unit 6 determines whether or not a call destination (communication partner) designation operation has been performed by a user operation. When it is determined that the call destination has been designated (Yes), the control unit 6 proceeds to step S3.

On the other hand, if the control unit 6 does not determine that the call destination has been specified (No), the control unit 6 continues this routine until the call destination is specified (or until another application is operated). Monitor. Then, the control unit 6 establishes communication (here, a telephone function) with the designated call destination.

(Step S3)
In step S3, the control unit 6 specifies the call destination based on the call destination designated by the user, analyzes the address information corresponding to the call destination, and proceeds to step S4.

(Step S4: Step for identifying other languages)
In step S4, the control unit 6 specifies the country or language including whether the language used by the communication partner is a basic language or another language based on the analyzed address information, and proceeds to step S5.

In the following description, it is assumed that the language used by the communication partner in step S4 is specified as another language.

(Step S5: Communication content reception step)
In step S5, the control unit 6 accepts at least one input of voice data or text data in the basic language as transmission reference data via the voice input unit 2 or the display unit 4 as input means, and the process proceeds to step S6. Transition.

(Step S6: Translation into other languages)
In step S6, the control unit 6 activates the translation application, translates the transmission reference data received in step S5 into voice data and text data in other languages, generates transmission data, and proceeds to step S7.

If the voice input from the voice input unit 2 is input in step S5, the control unit 6 recognizes the voice data in the basic language, which is the transmission reference data received by the voice input unit 2, and stores it in the storage unit 5. Start the listening application installed in advance.

And in this step S6, the control part 6 will perform the basic text data production | generation step which produces | generates the basic text data by the basic language for producing | generating the transmission data which consists of text data by another language.

On the other hand, if the character input is from the display unit 4 in step S5, the control unit 6 recognizes the text data in the basic language, which is the transmission reference data received from the display unit 4, and is installed in the storage unit 5 in advance. Start the reading application.

(Step S7: Transmission step)
In step S7, the control unit 6 transmits the transmission data translated in step S6 from the transmission interface 10 as a transmission means through the telecommunication line, and proceeds to step S8.

In addition, in step S7, when the transmission reference data received by the display unit 4 is text data in the basic language, the control unit 6 adds the text data in the basic language in addition to the transmission data in the speech data in the other languages that have been translated. May be transmitted in a superimposed manner.

Note that since the communication state (call state) with the other party is established in step S2, the user completes the transmission in steps S3 to S7 (until the word is exhausted or the other party receives data). ) A cancel step may be executed (accepted) to depress the switch 7 and prevent interruption from the other party. As a result, voice turbidity (translation turbidity) can be suppressed.

(Step S8)
Next, the control unit 6 determines whether or not there is reception from the other party. In addition, the control part 6 can also receive the communication content from a user as interruption by the routine of step S5 continuously. If it is determined that there is a reception from the other party (Yes), the control unit 6 proceeds to step S9.

On the other hand, if the control unit 6 does not determine that there is a reception from the other party (No), the control unit 6 continues to monitor this routine or the interruption in step S5.

(Step S9: Reception step)
In step S9, the control unit 6 receives voice data or text data in another language from the communication partner as reception reference data by the reception interface 9 as a reception unit through the telecommunication line, and proceeds to step S10.

(Step S10)
In step S10, the control unit 6 specifies whether the received data is text data or voice data, and proceeds to step S11.

(Step S11)
In step S11, the control unit 6 determines an output method, that is, whether to perform character analysis or voice analysis, based on the received data, and proceeds to step S12.

(Step S12: Basic verbalization translation step)
In step S12, the control unit 6 activates the translation application, translates the reception reference data received in step S9 into speech data and text data in a basic language, generates reception data, and proceeds to step S13.

(Step S13: Communication content output step)
In step S13, the control unit 6 outputs the received data translated in step S12 through the audio output unit 3 and the display unit 4 as output means and outputs the characters and ends this routine. The control unit 6 thereafter repeats the above routine until the communication (call) is completed.

As described above, the translation processing method, the translation processing program, and the recording medium according to the present embodiment are translated into the communication terminal 21 of the communication partner with respect to either the voice or the character input to the communication terminal 1. Voice and text can be output.

Also, in response to transmission of either voice or characters from the communication terminal of the communication partner, the translated voice and characters can be output to the communication terminal 1.

Here, the characters before translation, which is the basic language of the communication terminal 1 and the communication terminal 21, can be transmitted and received with each other.

In addition to those already described above, the methods according to the above embodiments may be used in appropriate combination.

In addition, the present invention is implemented with various modifications within a range not departing from the gist thereof.

For example, sign language is a visual language that performs finger movement and non-finger movement at the same time, and is recognized as a language along with speech language. Therefore, mutual translation with other languages can be realized by converting the visual language based on this sign language into the basic language and converting it into character data, voice data, or alternative image data.

In this case, other languages may include translations in the case of converting the native language into characters, sounds, and images, as well as re-converting into other languages based on the characters, sounds, and images of the native language. it can.

In addition, since the sign language behavior varies from country to country, translation similar to the language may be required. Accordingly, the storage unit 5 stores data that can be translated with respect to the sign language action as well as the language.

Sign language, although depending on the level of proficiency, is relatively slower than conversation but is often faster than character input. Therefore, convenience can be improved for some users by analyzing the sign language action and generating character data and voice data.

The above-mentioned “substitute image data such as a simple motion image including hand and face images” can be used as it is, but the responsiveness can be secured, but the amount of communication data increases. End up. Accordingly, the sign language motion video may be used as it is. However, in order to reduce the amount of communication data, the substitute image expressed in the screen of the display unit 4 as easily as possible is replaced with the substitute image data displayed on the display unit 4. Is desirable.

In reality, a user who requires sign language has vision, so at least if the sign language action can be converted into character data at the time of language input, the above-described text by characters at the time of output If displayed, it will be established as a translation function for mutual understanding without problems.

Furthermore, if it can be converted into voice data and transmitted to the other party, the burden can be reduced when the other party is a normal hearing person.

In addition, when you want to learn sign language and improve communication, or you want to use it like a pictograph (stamp) (for example, a still image or JIF movie that shakes the palm of a known "bye-bye") The above-described substitute image data is convenient.

Therefore, for example, when the translation icon AP4 shown in FIG. 1 is selected, a translation mode selection menu is displayed on the display unit 4 as shown in FIG.

The translation mode selection menu displayed on the display unit 4 includes “1) offline face-to-face translation”, “2) SNS translation”, “3) general call translation”, “ There are 6 types: “4) Simultaneous translation of conference”, “5) Video phone translation” and “6) Sign language translation”.

One communication terminal 1 shown in the present embodiment includes a voice input unit 2 for voice input, an operation unit 8 for character input, and a photographing camera 15 for video input as input means. Further, one communication terminal 1 has an audio output unit 3 for outputting audio and a display unit 4 for outputting characters / images as output means. Further, one communication terminal 1 is capable of translating a basic language and another language.

That is, one communication terminal 11 generates output data after being converted into another language for input data in a basic language received via the input means, and simultaneously outputs the output data (to the output means). In addition, it is possible to generate output data after making it into a basic language with respect to input data in another language, and at the same time, output it through the output means.

Therefore, as a matter of course, it is not a communication method using two

communication terminals

1 and 21, but for example, in the case of face-to-face translation when a user who uses a basic language and a user who uses another language are in the same place and have a conversation Furthermore, it can be used with one communication terminal 1.

As described above, “1) Offline face-to-face translation” is a translation mode that can be used when two or more foreign language speakers have a conversation using one communication terminal 1.

Further, one communication terminal 1 can use SNS on the cloud through an Internet line as a telecommunication line. Therefore, since the language used can be specified using the address book on the cloud as an SNS, for example, on the timeline, etc., the display language can be changed using the mutual translation function between the basic language and other languages. Mutual translation is possible.

At this time, the basic language and the other language may translate the translated text by tapping a screen display portion of a text (including a sign language image or the like), for example, Depending on the system, the entire text display may be translated at the labeling timing or the like.

That is, one communication terminal 11 generates output data after being converted into another language for input data in the basic language received via the input means, and at the same time outputs the output data (on the cloud). In addition, it is possible to generate output data after being converted into a basic language with respect to input data in another language, and at the same time, output it via the output means (display unit 4).

In this way, “2) SNS translation” is a translation that can be used, for example, when a translation that specifies a mutual language is performed by text or voice using a cloud-type SNS member list that can be managed on the Internet. Mode. .

Further, one communication terminal 1 can make a call through a telephone line or an internet line as a telecommunication line.

That is, one communication terminal 11 generates output data after being converted into another language for input data in the basic language received via the input means, and at the same time outputs the output data (to a telephone line or the like). In addition, it is possible to generate output data after being converted into a basic language with respect to input data in another language, and at the same time, output it via the output means (the audio output unit 3 or the display unit 4).

As described above, “3) General call translation” is a translation mode that can be used for conversations between foreigners and other parties on a normal telephone line.

Also, one communication terminal 1 can communicate through a short-range wireless communication line, a telephone line, and an Internet line as an electric communication line.

That is, one communication terminal 11 generates output data after being converted into another language for input data in a basic language received via the input means, and at the same time outputs the output data (short-range wireless communication line / telephone). (Line, internet line, etc.) can be output, and output data after being converted into a basic language is generated for input data in other languages, and at the same time, output via output means (voice output unit 3 or display unit 4) can do.

Thus, for example, the basic language and the other language can be translated into each other not only by the same conference room but also by two or more persons (two or more communication terminals) in a remote place. At this time, the number of other languages is not limited to one language, and a plurality of languages can be translated.

In this way, “4) Simultaneous Conference Translation” is a translation mode that can be used when two or more users have conversations between foreign languages. .

Similarly, if “5) TV phone translation” is selected, the simultaneous translation mode can be activated in the TV phone using the photographing camera 15. At this time, for example, as shown in FIG. 7, it is desirable that the other party's communication terminal 21 is also equipped with a photographing camera 25.

If “6) sign language translation” is selected, the sign language translation mode for analyzing the sign language motion photographed by the photographing camera 15 and converting it into character data, voice data, and substitute image data can be activated.

Specifically, when the user of the communication terminal 1 selects the translation mode with chat (or telephone) function in which “6) sign language translation” of the translation icon AP4 is selected, the control unit 6 captures an image photographed by the photographing camera 15. Execute sign language mode including the analyzed translation. When the communication environment with the user of the communication terminal 21 is established through an electric communication line such as a telephone line, the control unit 6 uses, for example, a sign language action unit (corresponding to a word vocabulary unit) from an image captured by the photographing camera 15. The sign language analysis unit 16 is made to analyze the content of the operation.

For example, when the user of the communication terminal 1 performs an action pointing at himself (I), an action pointing at the other party (you), an action of rotating both hands up and down (loved), The sign language analysis unit 16 analyzes each sign language operation.

At this time, an image of each sign language operation unit is stored in the storage unit 5. The video of each sign language operation unit may be temporarily stored in a frame memory or the like different from the storage unit 5, for example.

Based on the sign language action analyzed by the sign language analysis unit 16, the control unit 6 causes the character generation unit 14 to generate Japanese characters that are masculine expressions. A character [I love you] is displayed on the display unit 4 and a translated English character is generated and the character [I love you] is displayed on the display unit 4.

That is, the control unit 6 activates a sign language analysis application installed in the storage unit 5 in advance when the sign language operation as the transmission reference data received by the photographing camera 15 as the input means is used as a basic language. Analyzing the sign language movement with the basic analysis data in the basic language for generating outgoing data consisting of at least one of speech data, text data, and image data in other languages A basic analysis data generation step (corresponding to step S6) for generating “I love you>) is executed.

In addition, the translation by this sign language action includes the case where the sign language action is analyzed to generate , and the case where it is secondarily translated into another language.

More realistically, the sign language analysis application of the present embodiment includes an analysis conversion function for analyzing sign language movement and converting it into Japanese, a translation function for translating the converted Japanese into English, and an English-based version based on the translated English. It can also be said that three language conversion functions (translation functions), that is, a conversion function for converting to a sign language action of the above, are executed.

Also, the control unit 6 causes the voice generation unit 12 to generate a voice based on the generated character .

Furthermore, the control unit 6 causes the display unit 4 to display an image of the analyzed sign language action. At this time, if the translated word is English, the sign language analysis unit 16 uses the image data stored in the storage unit 5 in advance to sign language operations such as an English-language sign language operation or a similar icon / stamp (JPG, video GIF, etc.). ) To enable transmission.

Thereby, the control unit 6 superimposes the generated four data of the character , the character , the voice , and the sign language image <(I love you)>. Then, the data is transmitted from the transmission interface 10 to the communication terminal 21.

Based on the received data, the communication terminal 21 displays the pre-translation character [I love you], the post-translation character [I love you], and the post-translation sign language image [(I love you)] on the display unit 24. The voice “I love you” is output from the voice output unit 23 while being displayed.

In this manner, even if the translation application is not installed, the communication terminal 21 has the character , the character , the voice , and the sign language image [(I love you) ] Can be acquired. Note that these four pieces of data may be generated by the communication terminal 21 if a translation application is installed in the communication terminal 21.

On the other hand, when the user of the communication terminal 21 inputs “I am happy” to the voice input unit 22 in response to the voice “I love you”, the user transmits the voice to the communication terminal 1. .

The control unit 6 of the communication terminal 1 causes the voice recognition unit 11 to recognize the voice when the reception interface 9 receives the voice .

Thus, the control unit 6 causes the character generation unit 14 to generate the character , displays the character [I am happy] on the display unit 4, and translates the character <happy! > Is generated on the display unit 4 and the character [happy! ], And a sign language image [(happy)] of the corresponding sign language action is displayed. The control unit 6 can also generate the speech <joyful> by the speech generation unit 12 based on the generated character <joyful> and output the transliterated speech “happy” from the speech output unit 3.

Thus, even if the communication terminal 1 does not have a translation application installed on the communication terminal 21, the character before translation and the character after translation <happy! >, Translated sign language image <(happy)>, translated speech <joyful>, and can be expanded on the display unit 4 and the voice output unit 3 to be recognized by the user. Note that these four pieces of data may be generated by the communication terminal 21 if a translation application is installed in the communication terminal 21.

It should be noted that the present invention is not limited to the configuration and application shown in the above embodiment, and various modifications are possible. For example, the translation data stored in the storage unit (including on the cloud) 5 is updated by learning speech recognition, video recognition, natural language processing, etc. by a learning function (AI function) such as deep-learning. It is also possible to store data sequentially.

As described above, the translation processing method, the translation processing program, and the recording medium according to the present invention are functions that are executed by a single communication terminal, but the translation variations can be made multifunctional to improve versatility. And is useful for a translation processing method, a translation processing program, and a general recording medium for mutual translation between a basic language and another language when communicating with each other via a telecommunication line. .

DESCRIPTION OF SYMBOLS 1 Communication terminal 2 Voice input part 3 Voice output part 4 Display part 5 Memory | storage part 6 Control part 7 Switch 8 Operation part 9 Reception interface 10 Transmission interface 11 Voice recognition part 12 Voice generation part 13 Character recognition part 14 Character generation part 15 Shooting camera 16 Sign language analyzer (image generator)
21 Communication Terminal 22 Audio Input Unit 23 Audio Output Unit 24 Display Unit 25 Shooting Camera AP1 Telephone Icon AP2 E-Mail Icon AP3 Chat Icon C Cloud NT Electric Communication Line

Claims

As a step of causing a single communication terminal having a storage means preinstalled with a translation application that performs mutual translation between a basic language and another language,
A communication content receiving step for receiving at least one of voice data, text data, and image data in the basic language as transmission reference data via an input means;
Another language that activates the translation application and translates the transmission reference data received in the communication content reception step into at least one of the speech data, text data, and image data of the other language to generate transmission data A translation step,
A sending step of sending the sending data translated in the other language translation step by sending means;
A reception step of receiving at least one of the voice data, text data, and image data in the other language as reception reference data by a reception unit;
A basic languageized translation step of generating reception data by translating the reception reference data received in the reception step by starting the translation application into at least one of voice data, text data, and image data in the basic language When,
A communication content output step for outputting the received data translated in the basic language translation step through an output means;
A translation processing method comprising:
In the transmission step, the transmission data is transmitted through a telecommunication line by a transmission means,
In the receiving step, the reception reference data is received by a receiving means through a telecommunication line.
The translation processing method according to claim 1, wherein:
The telecommunication line is an internet line;
The translation processing method according to claim 2, wherein:
The telecommunication line is a telephone line;
The translation processing method according to claim 2, wherein:
The calling step is performed without going through a telecommunication line,
The receiving step is performed without going through a telecommunication line;
The translation processing method according to claim 1, wherein:
Before executing the other language translation step, another language specifying step for specifying the other language to be translated based on address data including a registered name of a communication partner registered in advance in the storage unit;
The translation processing method according to any one of claims 1 to 5, further comprising:
When the transmission reference data received by the input unit is voice data in the basic language, the listening application installed in advance in the storage unit is activated to generate the transmission data including text data in the other language Generating basic text data in the basic language for generating basic text data;
The translation processing method according to claim 1, further comprising:
When the transmission reference data received by the input unit is text data in the basic language, a reading application installed in advance in the storage unit is activated to generate the transmission data including voice data in the other language Generating basic voice data in the basic language for generating basic voice data;
The translation processing method according to claim 1, further comprising:
When the transmission reference data received by the input means is text data in the basic language, it is received by the input means in addition to the transmission data by the voice data of the other language translated in the other language translation step. A sending step of sending the text data in the basic language by the sending means;
The translation processing method according to claim 8, further comprising:
An interrupt for canceling reception by the receiving means during a period from when the input means accepts the speech data in the basic language to when the sending means in the other language is transmitted by the sending means. Cancellation step,
The translation processing method according to claim 1, further comprising:
When the transmission reference data received by the input means is a sign language action in the basic language, a sign language analysis application installed in the storage means is activated to analyze the sign language action, and then the voice in the other language A basic analysis data generation step of generating basic analysis data in the basic language for generating the transmission data consisting of at least one of data, text data, and image data;
The translation processing method according to claim 1, further comprising:
A translation processing program for causing a computing means of a communication terminal to execute the translation processing method according to any one of claims 1 to 11.
A recording medium recording a translation processing program for causing a computing means of a communication terminal to execute the translation processing method according to any one of claims 1 to 11.