KR20210029636A - Real-time interpretation service system that hybridizes translation through artificial intelligence and interpretation by interpreter - Google Patents

Abstract

The present invention relates to a hybrid real-time interpretation service system for hybridizing translation through artificial intelligence and translation by an interpreter expert, and more particularly, to a hybrid real-time interpretation service system for hybridizing translation through artificial intelligence and interpretation by an interpreter expert, in which one or both of real-time translation through artificial intelligence and real-time translation by an interpreter expert are selected to provide an interpretation service suitable for a conversational environment and provide meeting minute information in a text format, so that a separate conversation recording is unnecessary. To this end, the present invention includes: a plurality of user terminals for requesting translation in a user′s requested language and performing voice or video calls with one or more other users as conversation partners; an APP server that is connected to enable data communication with terminals of the users and relay the terminals so as to enable the voice or video calls, receives voice or image information of the user terminals to generate text information through voice recognition, uses artificial intelligence in a preset language to generate translated text information, and transmits the translated text information to the user terminals upon request; and at least one expert terminal that receives, upon request of the user terminal, the voice information or video information, which is included in the voice or video call between users by the APP server, of the other user′s terminal of the requested user terminal from the APP server to provide the information to the interpreter expert, and provides translated audio information or translated image information to the user terminal in the set language in the requested user terminal.

Description

Real-time interpretation service system that hybridizes translation through artificial intelligence and interpretation by interpreter}

The present invention relates to a real-time interpretation service system in which translation through artificial intelligence and translation by an interpreter expert are hybrid. More specifically, real-time translation through artificial intelligence and real-time translation by an interpreter can be selected or both can be selected to suit the conversation environment. It relates to a real-time interpretation service system in which translation through artificial intelligence and interpretation of an interpreter expert are hybridized through artificial intelligence that does not require an operation for recording a separate conversation because it can receive an interpretation service and can receive information about meeting minutes in text format.

In general, people who do not speak foreign languages often experience inconvenience and difficulties because they cannot communicate well when traveling abroad or talking or talking with foreigners. However, without the help of other people who know the foreign language well, they have to go to dictionaries or conversation books on their own to communicate with foreigners, but this made real-time response difficult and it was difficult to communicate properly.

Recently, with the development of smart phone technology, automatic translators have been developed, and Korean and Japanese languages with similar language systems have high reliability and are practically used. For English, because the language system is different from Korean, the reliability of translation is lower than that of Japanese. Translators and their translation programs are widely known to the extent that they can be practically used for or simple complex text.

On the other hand, it is a field where a lot of research efforts are devoted to speech recognition for machine recognition of human speech, and the automatic recognition rate of speech is increased due to the development of sampling technology and self-learning function using neural networks, and it is practically used in some fields Has become.

In addition, as a technology for outputting a character as a voice, a voice output device for converting a database (DB) of pronunciation combinations of each character, pronunciation of other words, and pronunciation of sentences and outputting it as a voice is already widely used.

In addition, an interpreter service is provided to users by converting other languages into their own languages using a smart phone that modern people always carry.

However, since the existing translation or interpretation system provided in various forms is provided through simple machine translation, the reliability and accuracy of interpretation quality decreases, and it is difficult to perform smooth translation or interpretation during multi-party conversations, and the terminal and server Since it can be used only in an online environment that is directly connected, there is a restriction on the use of the service according to the environmental conditions, and the conversation record is not stored, causing inconvenience in that a separate recording process must be performed for recording.

Korean Patent Registration No. 10-1753649

The present invention was devised to solve the above problems, and according to the conversation situation and the requested interpretation quality, as the translation through the artificial intelligence (AI) translation unit and the professional translation of the interpreter expert are provided alternatively or in full at the request of the user. Its purpose is to provide a real-time interpretation service system in which translation through artificial intelligence can be easily selected and the reliability of interpretation quality is improved, and interpretation by an interpreter expert is hybridized.

The present invention has the following features to achieve the above object.

The present invention includes a plurality of user terminals that request translation in a user's requested language and perform an audio or video call with one or more other users who are conversation partners; It is connected to enable data communication with the plurality of user terminals and relays them to make an audio or video call, receives voice or video information of the user terminals to generate text information through voice recognition, and the corresponding text information An APP server that generates translated text information in a preset language using artificial intelligence and transmits it to a plurality of user terminals according to a request; And receiving voice information or video information of the other user terminal of the requested user terminal by the APP server in an audio or video call between users at the request of the user terminal and providing it to an interpreter expert, and the requested user terminal And at least one expert terminal that provides translated audio information or translated image information to a corresponding user terminal in a set language of.

Here, a voice input means for receiving voice information of a user or an interpreter and transmitting it to the user terminal or an expert terminal, and a voice output means for receiving voice information or translated voice information from the user terminal and providing it to the interpreter expert or user. .

In addition, the user terminal includes a communication module unit that enables data communication with the APP server, a voice receiver that receives the user's voice information, a voice transmission unit that provides translated voice information, the user's translation request information, and translation language setting information. , Interpreter selection information, conversation topic selection information, or an interface unit for receiving user input information including artificial intelligence translation request information, and voice information and user input information received from the voice receiving unit according to user input information from the interface unit. Transmits to the APP server so that the expert terminal of the selected interpreter expert is included in the voice or video call between users, performs a request for transmission of translated text information using artificial intelligence according to the request, and translates the expert terminal received from the APP server. And a user terminal control unit that provides voice information or translated text information using artificial intelligence to a user through a voice transmission unit or display means.

In addition, the APP server is a communication unit that enables data communication with the plurality of user terminals or the at least one expert terminal, and when the translation request information of the user terminal is received through the communication unit, the domain selected by providing conversation subject domain list information After providing the list of interpreter experts according to the list information to the user terminal, the user terminal and the expert by transmitting the request information for possible translation to the expert terminal of the corresponding interpreter according to the information selected by the user from the provided interpreter expert list information. A matching management unit that manages the connection between the terminals to perform translation, and when the matching of the matching management unit is performed, the voice information or image information to be translated provided from the other user terminal of the user terminal that has transmitted the translation request information is transferred to the expert terminal. A real-time translation relay unit that repeats a series of processes of transmitting and receiving the translated audio or video information from the expert terminal and transmitting it in real time to the user terminal that has transmitted the translation request information, and between the user terminals and the expert terminal. An STT conversion unit that converts voice information or video information into text information through voice recognition, and when there is a request for transmission of translated text information using the artificial intelligence of the user terminal, the STT conversion unit transmits the text information of the user terminal of the conversation partner. It includes an artificial intelligence (AI) translation unit that generates translation text information in which the text information is translated into a preset language through artificial intelligence and transmits the transmission request to a user terminal.

In addition, the expert terminal includes a communication module unit that enables data communication with the APP server, an expert voice receiving unit that receives the translated voice information of the interpreter expert from the voice input unit, and the user’s voice information through the voice output unit. An expert voice transmission unit provided to the user, an expert interface unit provided to an interpreting expert when request information for possible translation is received from the APP server, and inputting selection information on whether or not the interpreting expert is possible, and user's voice information from the APP server Or an expert terminal control unit that provides image information to an interpreter through an expert voice transmission unit or a display means, and transmits the translated audio information of the interpreter expert received through the expert voice reception unit to the APP server.

In addition, the APP server receives the text information of the voice information and translated voice information of the plurality of user terminals and expert terminals from the STT converter and lists them in chronological order according to the identification information of each user terminal and expert terminal. Included.

In addition, text information and translation text information of user terminals and expert terminals are received and stored from the STT conversion unit and the artificial intelligence translation unit of the APP server, and a translation database stored for each conversation subject domain information is included.

In addition, when there is a translation request using an interpreter expert or artificial intelligence from the user terminal, a payment server in which payment is made by calculating billing information according to a setting is included.

In addition, when any one of the plurality of user terminals is selected as the main speaker terminal, the remaining user terminals are specified as listener terminals, and an expert terminal performing the translation of audio or video information of each listener terminal The audio or video information is translated into the language set in the main speaker terminal.

According to the present invention, as the translation through the artificial intelligence (AI) translation unit and the professional translation of the interpreter expert are provided alternatively or both according to the user's request, the user can easily select according to the conversation situation and the requested interpretation quality. There is an effect that the reliability can be improved.

In addition, as the artificial intelligence translation unit compares the artificial intelligence translation result with the translation result of an interpreter corresponding to the translation result, the artificial intelligence translation quality can be improved as artificial intelligence learning is performed.

In addition, through the meeting minutes generation unit of the APP server, there is an effect that the user can receive the meeting minutes information without having to separately record the contents of the conversation.

In addition, it is possible to provide an interpretation service using a mobile router in a multi-party conversation or in a local offline environment rather than an online environment.

1 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system according to an embodiment of the present invention.
2 is a block diagram showing the internal configuration of a real-time interpretation service system according to an embodiment of the present invention.
3 is a use state diagram schematically showing the operation of the real-time interpretation service system according to an embodiment of the present invention.
4 and 5 are diagrams illustrating an interface screen provided when a user terminal accesses an APP server according to an embodiment of the present invention.
6 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system according to another embodiment of the present invention.
7 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system according to another embodiment of the present invention.
8 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system according to a deformable embodiment of the present invention.
9 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system in the form of a conference mode according to a deformable embodiment of the present invention.
10 is a diagram illustrating an interface screen of a user terminal in which the conference mode according to FIG. 9 is displayed.
11 is a diagram illustrating a chat screen for each user terminal in the conference mode according to FIG. 9.

In order to explain the present invention and the operational advantages of the present invention and the object achieved by the implementation of the present invention, the following describes a preferred embodiment of the present invention and looks at with reference thereto.

First, terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention, and expressions in the singular may include a plurality of expressions unless clearly differently in context. In addition, in the present application, terms such as "comprise" or "have" are intended to designate the existence of features, numbers, steps, actions, components, parts, or a combination thereof described in the specification, but one or more other It should be understood that the presence or addition of features, numbers, steps, actions, components, parts, or combinations thereof, does not preclude the possibility of preliminary exclusion.

In describing the present invention, when it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

1 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system according to an embodiment of the present invention.

Referring to the drawings, a real-time interpretation service system 1000 according to an embodiment of the present invention requests translation in a user's requested language, and a plurality of user terminals ( 100), and the plurality of user terminals 100 and connected to enable data communication, relaying therebetween to make an audio or video call, and receiving audio or video information of the corresponding user terminals 100 to perform voice recognition. The APP server 300 and the user terminal 100 for generating text information through and transmitting the text information to a plurality of user terminals 100 according to a request by generating translated text information using artificial intelligence in a preset language. ) At the request of the APP server 300 to receive the voice information or video information of the other user terminal 100 of the user terminal 100 requested by the APP server 300 in an audio or video call between users from the APP server 300 It is provided to an interpreter expert, and consists of at least one expert terminal 200 that provides translation voice information or translated image information to the corresponding user terminal 100 in a language set for the requested user terminal 100.

Here, the user terminal 100 is a terminal owned by two or more multi-party chatters having different languages, and interpreting through the real-time interpretation service system 1000 according to the present invention when an artificial intelligence translation or translation by an interpreter is required. Service will be provided.

The user terminal 100 may be applied to any terminal capable of data communication by connecting to an APP server 300 to be described later, such as a desktop, a notebook, or a tablet, in addition to a portable terminal including a smart phone.

In addition, the APP server 300 relays the translation request information when the translation request information is received in the multilateral conversation in which the conversation is performed in different languages to perform an audio or video call, and according to the request of the user terminal 100, each user terminal 100 ) To convert the voice information or video information received from) into text information in STT form through voice recognition, and perform the translation using artificial intelligence, and transfer the generated translated text information to each user terminal (100). Can be transmitted.

In addition, when any one or all of the parties to the multilateral conversation requests a translation from an interpreter, the APP server 300 transmits the request information for possible translation to the expert terminal 200 possessed by the interpreter and checks whether it is possible. If possible, the expert terminal 200 is included in the multi-party voice or video call.

At this time, when the translation request information of the interpreter expert is received from the specific user terminal 100, the APP server 300 provides the user's language information and the conversation subject domain information to the corresponding user terminal 100 to provide the user's input information. Accordingly, the interpreter expert list information previously stored in the corresponding language and the corresponding domain is extracted and provided to the user terminal 100 to request selection information.

In addition to personal information of the corresponding interpreter expert, the interpreter expert list information may provide review information of an existing user on the quality of interpretation, cost information, and the like.

Accordingly, when some or all of the multilateral conversation parties transmit the translation request information and input a series of user input information to match the interpreter expert and connect an audio or video call, each user terminal 100 selects the expert terminal 200. ) Receives the voice information or video information of the user terminal 100 of the corresponding user terminal 100 from the APP server 300, and transmits the translated audio information or translated image information to the APP server 300. Through this, it is transmitted to the corresponding user terminal 100.

Of course, if there are two parties to the conversation, and they select one interpreter, the expert terminal 200 of the corresponding interpreter continues to receive audio or video information from the two user terminals 100 and translates it. Information and translated video information will be provided.

In addition, the APP server 300 includes an STT converter 340 to translate text information through voice recognition from voice information or video information of each user terminal 100 and voice information or video information of an interpreter through voice recognition. Each text information may be generated and transmitted to the user terminal 100.

Accordingly, the user of the user terminal 100 receives and listens to the translated voice information or the translated image information, and at the same time receives the text information through the display means of the user terminal 100 so that the user can concentrate on the conversation content more.

In addition, as described above, when an artificial intelligence translation is requested by the APP server 300, translation text information according to the artificial intelligence translation is also provided and compared with the translated text information of an interpreter.

In the case of the hybrid real-time interpretation service system according to the present invention, a translation service through an interpreter expert and a translation service through artificial intelligence can be provided alternatively or at the same time. It can be provided so as to be possible, and in consideration of the expertise of a conversation topic, if a certain level of high-quality translation service is possible with only translation through artificial intelligence, it has a characteristic that can reduce the cost burden to the user.

For example, if an interpreter's call connection is offline due to the network situation, the conversation can be continued by replacing it with a translation service through artificial intelligence, and both are provided with a translation service by an interpreter and a translation service through artificial intelligence at the same time. Compared to the comparison, if the user determines that there is no significant difference in quality, the translation service through an interpreter can be stopped halfway and only the translation service through artificial intelligence can be selectively provided.

2 is a block diagram showing the internal configuration of a real-time interpretation service system according to an embodiment of the present invention.

Referring to the drawings, looking at the internal configuration of the real-time interpretation service system 1000 according to an embodiment of the present invention, first, in the case of the user terminal 100, a communication module unit 110 that enables data communication with the APP server 300. ), the voice receiving unit 120 receiving the user's voice information, and the voice transmitting unit 130 providing translated voice information, the user's translation request information, translation language setting information, interpretation expert selection information, conversation topic selection information Or the interface unit 140 for receiving user input information including artificial intelligence translation request information, and the voice information and user input information received from the voice receiving unit 120 according to user input information from the interface unit 140. The expert terminal 200 of the interpreter expert selected by sending it to the APP server 300 is included in the voice or video call between users, and performs a request for transmission of translated text information using artificial intelligence according to the request, and the APP server 300 ). It consists of a user terminal control unit 150 that provides the user with translation voice information of the expert terminal 200 or translation text information using artificial intelligence received from the voice transmission unit 130 or a display means.

Here, a separate voice input means 400 may be provided so that the voice receiver 120 can receive the user's voice information, and if necessary, a microphone, which is a voice input means generally provided in the user terminal 100, etc. Through the voice receiver 120 may receive the user's voice information.

In addition, a separate voice output means 410 may be provided in order to provide the translated voice information to the user by the voice transmission unit 130, and if necessary, a speaker, which is a voice output means generally provided in the user terminal 100 The user may be provided with the translated voice information of the voice transmission unit 130 through or the like.

In addition, the user terminal control unit 150 includes user input information input through the interface unit 140, for example, user's translation request information, translation language setting information, interpretation expert selection information, conversation topic selection information, or artificial When intelligent translation request information is received, it is transmitted to the APP server 300, and information provided for various request information provided from the APP server 300 is provided through a display means of the user terminal 100.

Meanwhile, the APP server 300 of the real-time interpretation service system 1000 includes a communication unit 310 for enabling data communication with the plurality of user terminals 100 or the at least one expert terminal 200, and the communication unit 310 When the translation request information of the user terminal 100 is received through the conversation subject domain list information, the list of interpreter experts according to the selected domain list information is provided to the corresponding user terminal 100, and then from the provided interpreter expert list information. Matching management unit 320 that manages the connection between the user terminal 100 and the expert terminal 200 by transmitting the translation request information to the expert terminal 200 of the corresponding interpreter expert according to the selected information selected by the user. And, when the matching of the matching management unit 320 is performed, the expert terminal 200 transmits voice information or image information to perform translation provided from the counterpart user terminal 100 of the user terminal 100 that has transmitted the translation request information. ), and repeats a series of processes of receiving the voice information or video information translated from the expert terminal 200 and transmitting it to the user terminal 100 that transmitted the translation request information in real time ( 330) and the STT conversion unit 340 for converting voice information or image information between the user terminals 100 and the expert terminal 200 into text information through voice recognition, and the artificial intelligence of the user terminal 100 When there is a request for transmission of the translated text information used, the text information of the user terminal 100 of the conversation counterpart is received from the STT conversion unit 340, and the text information is translated into a preset language through artificial intelligence. The artificial intelligence (AI) translation unit 350 and the APP server 300 that generate and transmit the transmission request to the user terminal 100 are provided with the plurality of user terminals 100 and experts from the STT conversion unit 340. By receiving the text information of the voice information and the translated voice information of the terminal 200, each user terminal 100 and the expert terminal 200 ) Is composed of a minutes generating unit 360 listing in chronological order according to the identification information.

Here, the STT conversion unit 340 and the artificial intelligence translation unit 350 are provided as a separate artificial intelligence-based STT server 700 as shown in FIG. 7 to be described later, so that the APP server 300 is a matching management unit 320. And by relaying the voice or video call between each user terminal 100 and the expert terminal 200 through the real-time translation relay unit 330, and at the same time transmitting the audio or video information to the artificial intelligence-based STT server 700 STT-type text information may be generated through recognition, and translated text information may be generated through artificial intelligence upon request and provided to the user terminal 100.

Of course, if the artificial intelligence-based STT server 700 is configured separately, the meeting minutes generation unit 360 is included in the server to generate the minutes list information, and the translation database 500 to be described later is also based on the artificial intelligence. It is interlocked with the STT server 700 to receive the corresponding audio or video information, text information, and translation text information from thereto and store it.

Meanwhile, the expert terminal 200 includes a communication module unit 210 that enables data communication with the APP server 300, and an expert voice receiving unit 220 that receives the translated voice information of an interpreter expert from the voice input unit 400. ), and an expert voice transmission unit 230 that provides the user's voice information to an interpreter expert through the voice output means 410, and when the request information for possible translation is received from the APP server 300, it is provided to an interpreter expert. The expert interface unit 240 for receiving selection information on whether or not the interpreter is available, and the voice or video information of the user from the APP server 300 are sent to the expert voice transmission unit 230 or the display means 260 to the interpreter expert. It is provided through, and consists of an expert terminal controller 250 that transmits the translated voice information of the interpreter expert received through the expert voice receiving unit 220 to the APP server 300.

3 is a use state diagram schematically showing the operation of the real-time interpretation service system according to an embodiment of the present invention.

The operation process of the real-time interpretation service system according to the present invention will be described with reference to the drawings. FIG. 3 is an example in which a conversation party whose setting language is Korean and a conversation party whose setting language is English perform a voice call with each other.

Accordingly, the participants of the conversation with the set language of Korean selected an interpreter who translates English into Korean, and those with the set language of English selected an interpreter who translates Korean into English.

Of course, one interpreter can be designated and configured to provide translation services to both parties to the conversation.

As a result, the conversation participant whose setting language is Korean will say, "Hello! It's good to see you again." In the case of saying “Hello, we bumped up here!”, the voice information is transmitted to the expert terminal 200 of the interpreter expert selected by the person speaking with the set language of English, and the translated voice information of the interpreter expert is “Hello, we bumped up here!” The voice is transmitted to the conversation party whose set language is English through the APP server 300.

Of course, since translation through artificial intelligence was selected, translated text information through artificial intelligence is provided as "Hi, we catch up!" as text information.

4 and 5 are diagrams illustrating an interface screen provided when a user terminal accesses an APP server according to an embodiment of the present invention.

Referring to the drawing, when the user terminal 100 accesses the APP server 300, a menu screen is first provided for selecting the user's setting language and the conversation subject domain (see Fig. 4(a)).

Accordingly, when a set language is selected and a domain is selected (see Fig. 4(b) and (c)), interpreter expert list information according to the set language and domain is provided from the APP server 300 to the user terminal 100 If the user selects one of these, the corresponding interpreter is included in the conversation (see (b) of FIG. 5).

Looking at the chat window (refer to (b) of FIG. 5), first, ① is a menu for the entire conversation, and ② is a menu for selecting the end of the conversation. Of course, when the conversation end menu is selected, a selection menu may be provided whether to receive the meeting minutes list information or a recording file of the entire audio or video call.

On the other hand, ③ indicates that the voice information of the user of the user terminal 100 is displayed as text information, and ④ indicates that the voice information of the conversation partner is displayed as text information.

⑤ is an interpreter's voice information displayed as text information, and is a whisper function menu that can only be heard by an interpreter expert or communicate requests to ⑥ and ⑦ interpreter experts.

⑧ is a menu for speaking while pressing a button to deliver your own voice information during a voice call, ⑨ is a menu for conversation time, and ⑩ is a menu for selecting a video call and an audio call.

Meanwhile, as shown in (c) of FIG. 5, a menu for selecting whether to provide a translation service through artificial intelligence is provided.

6 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system according to another embodiment of the present invention.

Referring to the drawings, the real-time interpretation service system 1000 according to the present embodiment includes user terminals 100 and expert terminals 200 from the STT conversion unit 340 and the artificial intelligence translation unit 350 of the APP server 300. ) Text information and translation text information are received and stored, and a translation database 500 stored for each conversation subject domain information is included.

The translation database 500 may provide data stored during artificial intelligence learning at the request of the artificial intelligence translation unit 350 so that the learning is performed.

In addition, in the present embodiment, when there is a request for translation using an interpreter expert or artificial intelligence from the user terminal 100, a payment server 600 that calculates billing information according to a setting and makes payment is included.

7 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system according to another embodiment of the present invention.

Referring to the drawings, in the real-time interpretation service system 100 according to the present embodiment, the artificial intelligence-based STT server 700 is separately configured in the above-described embodiment, and the existing multi-party conversation is possible from the artificial intelligence-based STT server 700. A database 500 is provided to receive and store the performed audio or video information, text information, and translated text information.

Here, the artificial intelligence-based STT server 700 is a voice or video call between each user terminal 100 and the expert terminal 200 through the APP server 300 matching management unit 320 and real-time translation relay unit 330 When relaying, the corresponding voice or video information is transmitted in real time and text information in the form of STT is generated by voice recognition.

In addition, when there is a request from the user terminal 100, translation text information may be generated through artificial intelligence and provided to the user terminal 100.

Of course, in the case of separate configuration of the artificial intelligence-based STT server 700 as described above, the meeting minutes generation unit 360 in the APP server 300 in the above-described embodiment is not included, and is included in the artificial intelligence-based STT server 700. The minutes list information is generated, and the minutes list information is transmitted according to the request of the APP server 300.

8 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system according to a deformable embodiment of the present invention.

Referring to the drawings, the real-time interpretation service system 1000 according to the present embodiment receives a plurality of user-specific voice information, converts it into text information, and transmits voice information or text information to the outside; Receive text information or voice information transmitted from outside and provide it to an interpreter expert, and receive translation text information or translated voice information that has been translated from the language of the text or voice information into a preset language from the interpreter. When voice information is input, it is connected to at least one expert terminal 200 that converts it into translated text information, and data communication with the mobile router 800 and the expert terminal 200, and is connected to the mobile router 800. Voice information or text information for each user is received and transmitted to the expert terminal 200 to request the transmission of the translated voice information in which the language of the voice information for each user is translated into another preset language, and the voice information or text for each user The language of the information is translated into a preset language through the artificial intelligence (AI) translation unit 350 to generate AI translated voice information or AI translated text information, and the translated voice information received from the expert terminal 200 or It is composed of an APP server 300 that transmits the translation text information to the mobile router 800 requesting the translation.

Here, even in an offline situation in which data communication between the user terminal 100 and the APP server 300 is not possible, the mobile router 800 is connected to enable data communication with the APP server 300. To this end, the mobile router 800 is an APP The server 300 enables general Ethernet communication, Wi-fi communication, as well as satellite communication, so that a real-time interpretation service can be provided even when the general user terminal 100 is offline.

In addition, in this embodiment, a device having a separate identification number in addition to the user terminal 100 and having a voice input means 400 and a voice output means 410, that is, a device such as an ear set or a headset, can also be connected. It can be applied to conferences as well as conference calls.

9 is a conceptual diagram showing a schematic configuration of a real-time interpretation service system in the form of a conference mode according to a deformable embodiment of the present invention, and FIG. 10 is a view showing an interface screen of a user terminal displaying the conference mode according to FIG. 9; 11 is a diagram illustrating a conversation screen for each user terminal in the conference mode according to FIG. 9.

Referring to the drawings, the real-time interpretation service system 1000 according to the present embodiment is applied to the conference mode type, and any one user terminal 100 among a plurality of user terminals 100 is selected as the main speaker terminal 100a. In this case, the remaining user terminals 100 are specified as listener terminals 100b.

Accordingly, the main speaker terminal 100a may be designated as a remote seminar or lecture or conference chair, and it is preferable that the user terminal with the largest amount of conversation is designated.

In addition, the listener terminal 100b is a user terminal 100 other than the main speaker terminal 100a. Transmits the received and translated translation voice information or translated video information to the APP server 300 so that it is transmitted to the corresponding listener terminal (100b), and the user voice information of the listener terminal (100b) received from the listener terminal (100b) or The image information is translated back into the language set of the main speaker terminal 100a, so that it is transmitted to the main speaker terminal 100a through the APP server 300.

Accordingly, as shown in FIG. 11, the main speaker terminal 100a using Korean as the setting language is specified, the listener terminal 100b using Chinese as the setting language, and the listener terminal 100b using English as the setting language, respectively. When specified, the main speaker terminal 100a may receive translation audio information or translated image information for audio information or image information of each listener terminal 100b in Korean, respectively.

That is, the translation service between the listener terminals 100b is not performed through a separate expert terminal 200, so that the audio information or the video information of each of the listener terminals 100b is not supported by a translation service.

This is because the form of the conference mode is mainly focused on the user who plays the role of the main speaker, so the conversation between listeners (audience) does not require translation.

As described above, the present invention has been described with reference to an embodiment shown in the drawings, but this is only exemplary, and those of ordinary skill in the art can recognize that various modifications and other equivalent embodiments are possible therefrom. I will understand.

Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100: user terminal 200: expert terminal
300: APP server 400: voice input means
410: voice output means 500: database
600: payment server 700: artificial intelligence-based STT server
800: mobile router 1000: interpretation service system

Claims (9)

A plurality of user terminals 100 requesting translation in a user's requested language and performing an audio or video call with one or more other users who are conversation partners;
It is connected to enable data communication with the plurality of user terminals 100 and relays them to make an audio or video call, and receives audio or video information of the corresponding user terminals 100 to receive text information through voice recognition. An APP server 300 that generates and transmits the corresponding text information to a plurality of user terminals 100 according to a request by generating translated text information using artificial intelligence in a preset language; And
In response to the request of the user terminal 100, the APP server 300 transmits the voice or video information of the other user terminal 100 of the user terminal 100 requested by the APP server 300 in an audio or video call between users. 300), provided to an interpreter expert, and at least one expert terminal 200 for providing translation voice information or translated image information to the corresponding user terminal 100 in the set language of the requested user terminal 100; A real-time interpretation service system in which translation through artificial intelligence and interpretation of an interpreter expert are hybridized.
The method of claim 1,
A voice input means 400 for receiving voice information of a user or an interpreter and transmitting it to the user terminal 100 or the expert terminal 200, and
The translation through artificial intelligence and the interpretation of the interpreter expert are hybridized, characterized in that it further comprises a voice output means 410 for receiving voice information or translated voice information from the user terminal 100 and providing it to an interpreter or a user. Real-time interpretation service system.
The method of claim 1,
The user terminal 100 is
A communication module unit 110 that enables data communication with the APP server 300,
A voice receiving unit 120 receiving the user's voice information,
A voice transmission unit 130 that provides translated voice information,
An interface unit 140 for receiving user input information including user's translation request information, translation language setting information, interpretation expert selection information, conversation topic selection information, or artificial intelligence translation request information, and
According to the user input information from the interface unit 140, the voice information received from the voice receiving unit 120 and the user input information are transmitted to the APP server 300, and the expert terminal 200 of the selected interpreter Alternatively, it is included in a video call, and performs a request for transmission of translated text information using artificial intelligence according to the request, and translated voice information of the expert terminal 200 received from the APP server 300 or translated text information using artificial intelligence. A real-time interpretation service system in which translation through artificial intelligence and interpretation of an interpreter expert are hybridized, comprising: a voice transmission unit 130 or a user terminal control unit 150 that provides a user through a display means.
The method of claim 3,
The APP server 300
A communication unit 310 that enables data communication with the plurality of user terminals 100 or the at least one expert terminal 200,
When the translation request information of the user terminal 100 is received through the communication unit 310, the subject domain list information is provided, and interpreter expert list information according to the selected domain list information is provided to the user terminal 100 and then provided. Transmitting request information for possible translation to the expert terminal 200 of the corresponding interpreter according to the information selected by the user from the list of interpreter experts, and managing the connection between the user terminal 100 and the expert terminal 200 A matching management unit 320,
When the matching of the matching management unit 320 is performed, the voice information or image information to be translated provided from the counterpart user terminal 100 of the user terminal 100 that has transmitted the translation request information is transferred to the expert terminal 200. A real-time translation relay unit 330 that repeats a series of processes of transmitting and receiving the translated audio or video information from the expert terminal 200 and transmitting it in real time to the user terminal 100 that has transmitted the translation request information. And
An STT conversion unit 340 for converting voice information or image information between the user terminals 100 and the expert terminals 200 into text information through voice recognition, and
When there is a request for transmission of the translated text information using the artificial intelligence of the user terminal 100, the text information of the conversation partner user terminal 100 is received from the STT conversion unit 340 and the corresponding text information is recorded through artificial intelligence. Translation and interpretation through artificial intelligence, characterized in that it comprises an artificial intelligence (AI) translation unit 350 that generates translation text information that is translated into a set language and transmits the transmission request to the user terminal 100. Real-time interpretation service system with hybrid interpretation.
The method of claim 4,
The expert terminal 200 is
A communication module unit 210 that enables data communication with the APP server 300,
An expert voice receiving unit 220 receiving the translated voice information of an interpreter expert from the voice input means 400,
An expert voice transmission unit 230 for providing the user's voice information to an interpreter expert through the voice output means 410;
When the request information for possible translation is received from the APP server 300, the expert interface unit 240 is provided to an interpreter and receives selection information on whether or not the interpreter is available, and
The voice information or video information of the user from the APP server 300 is provided to an interpreting expert through the expert voice transmitting unit 230 or the display means 260, and the translated voice of the interpreting expert received through the expert voice receiving unit 220 A real-time interpretation service system in which translation through artificial intelligence and interpretation of an interpreter expert are hybridized, comprising: an expert terminal controller 250 that transmits information to the APP server (300).
The method of claim 4,
The APP server 300
Identification information of each user terminal 100 and expert terminal 200 by receiving text information of the voice information and translated voice information of the plurality of user terminals 100 and expert terminals 200 from the STT converter 340 A real-time interpretation service system in which translation through artificial intelligence and interpretation of an interpreter expert are hybridized, characterized in that it further includes a minutes generating unit 360 listing in chronological order according to the following.
The method of claim 4,
Text information and translation text information of the user terminals 100 and expert terminals 200 are received and stored from the STT conversion unit 340 and the artificial intelligence translation unit 350 of the APP server 300,
A real-time interpretation service system in which translation through artificial intelligence and interpretation of an interpreter expert are hybridized, characterized in that it further includes a translation database 500 stored for each conversation topic domain information.
The method of claim 4,
Translation and interpretation through artificial intelligence, characterized in that when there is a translation request from the user terminal 100 using an interpreter expert or artificial intelligence, a payment server 600 that calculates billing information according to the setting and makes payment is further included. A real-time interpretation service system in which expert interpretation is hybridized.
The method of claim 2,
When any one user terminal 100 of the plurality of user terminals 100 is selected as the main speaker terminal 100a, the remaining user terminals 100 are specified as the listener terminal 100b. Artificial intelligence, characterized in that the expert terminal 200 performing translation of audio or video information translates the audio or video information of the corresponding listener terminal 100b into a set language of the main speaker terminal 100a. A real-time interpretation service system that combines translation through translation and interpretation of experts.

Images