KR20170030075A - Method for providing interpreting sound information, customized interpretation of server and system implementing the same - Google Patents

Abstract

A method of providing interpretation voice information, and a customized interpretation server and system for operating the same are disclosed. A method of providing interpreted speech information according to various embodiments of the present invention is a method of providing interpreted speech information for a source language in a customized interpretation server, the method comprising: displaying UI items of content related to a source word through an application installed in a web page or a listener terminal step; Collecting, in real time, first interpretation speech information generated in a predetermined language for the source language; And providing the collected first interpretation voice information to the listener terminal in real time in response to receiving a user selection input for the UI item.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for providing interpretation voice information and a customized interpretation server and system for operating the same,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for providing interpreted speech information, a customized interpretation server and system for operating the interpreted speech information, and a technique for providing interpreted speech information to a speaker in various environments.

Conventional real-time interpretation services or simultaneous interpretation services have been provided primarily using the FM simultaneous interpretation system. In general, the FM simultaneous interpretation system includes an audio control unit, a modulator, an antenna combiner, an interpreter unit, a receiver, and the like.

An object of the present invention is to provide interpretation voice information collected in real time to a speaker who wishes to listen to it through an application in real time.

Another object of the present invention is to provide an environment in which a specific person can upload interpreted speech information of a different language to the server when the provision of the contents provided with real-time interpretation speech information is terminated.

Another problem to be solved by the present invention is to establish various business environments related to commerce based on content including interpreted voice information.

Another object of the present invention is to provide an interpretation method that does not require renting a separate receiver and / or earphone to a listener to provide a real-time interpretation service.

The problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems which are not mentioned can be clearly understood by those skilled in the art from the following description.

According to another aspect of the present invention, there is provided a method of providing interpreted speech information, the method comprising the steps of: Through an application installed in the mobile terminal; Collecting, in real time, first interpretation speech information generated in a predetermined language for the source language; And providing the collected first interpretation voice information to the listener terminal in real time in response to receiving a user selection input for the UI item.

In some embodiments, the method for providing interpreted speech information comprises: constructing a first language item corresponding to the first interpreted speech information; And displaying the first language item on a screen displayed through the web page or the application at a position adjacent to a position at which the UI item of the content is displayed.

In some embodiments, when the real-time collection of the first interpretation speech information is terminated, the second interpretation speech information corresponding to a different language from the first interpretation speech information is sent to the user Collecting from a terminal or an arbitrary user terminal; And providing at least one of the first interpretation voice information or the second interpretation voice information to the listener terminal in response to receiving a user selection input for a UI item of the content.

In some embodiments, the method for providing interpreted voice information may further comprise paying a predetermined reward for the user account that provided the second interpreted voice information.

In some embodiments, the method for providing interpreted speech information may further include displaying a first language item corresponding to the first interpreted speech information and a second language item corresponding to the second interpreted speech information to be distinguished from each other have.

A method for providing interpreted speech information according to various embodiments of the present invention is a method for providing interpreted speech information for a source language in a customized interpretation server, the method comprising: receiving commerce content for a specific product or a specific service from a content producer terminal Wherein the commerce content is content representing the specific commodity or the specific service in a source language; Displaying a UI item of the commerce content through a web page or an application; Outputting first interpretation speech information generated in a predetermined language for the source language in response to receiving a user selection input for a UI item of the commerce content; Collecting second interpretation speech information corresponding to a language different from the first interpretation speech information from the seller terminal; And displaying commerce contents reflecting the second interpretation speech information.

In some embodiments, the interpretation voice information providing method further includes the step of displaying the information of the specific commodity or the information of the content creator in response to a request from the commodity supplier or the content producer to supply the specific commodity or the specific service .

In some embodiments, the method for providing interpreted speech information may further include transmitting the commerce content reflecting the second interpreted speech information to an external media server.

A customized interpretation server according to various embodiments of the present invention is a customized interpretation server for providing interpretation speech information for a source language, comprising: a communication unit; And displaying UI items of contents related to the original language through an application installed on a web page or a listener terminal, collecting first interpretation speech information generated in a predetermined language for the original language in real time, And a control unit for providing the collected first interpretation voice information to the listener terminal through the communication unit in real time in response to receiving the selection input.

In some embodiments, the control unit configures a first language item corresponding to the first interpretation speech information, displays a configured first language item at a position adjacent to a position at which the UI item of the content is displayed, 1 collects the second interpretation speech information corresponding to a language different from the first interpretation speech information from the preset user or an arbitrary user when the real-time collection of the interpretation speech information is finished, In response to receiving the user selection input for the first interpretation speech information or the second interpretation speech information, the at least one of the first interpretation speech information or the second interpretation speech information to the listener terminal through the communication section.

Other specific details of the invention are included in the detailed description and drawings.

By providing the interpretive audio information providing method of the present invention and the server performing the interpretation audio information to the listener through the application in real time, the listener can easily listen to the desired interpretation voice among the various contents provided by the application in real time .

Further, by providing an interpretation audio information providing method and an environment in which a specific person in the server can upload the interpretation audio information of a different language to the server, it is possible to provide a more active voice translation content Platform can be implemented, and a certain reward can be paid to a specific person, which can encourage enthusiasm for participation in the voice translation content market.

Also, it is possible to construct various business environments by generating / distributing contents related to commerce based on the interpretation voice information providing method of the present invention and the contents including the interpretation voice information in the server performing the method.

In addition, according to the interpretive audio information providing method and server of the present invention, a listener can receive an interpreter signal using his / her terminal (listener terminal), so that the listener can receive a separate receiver and / A real-time interpretation service can be provided even if the user does not rent the service.

In addition, according to the interpretive audio information providing method and server of the present invention, it is possible to reduce the rental cost caused by rental of a separate receiver and / or an earphone, It is possible to prevent the loss caused by the breakage or the like.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a conceptual diagram of a customized interpretation system in accordance with various embodiments of the present invention.
FIG. 2 is a flow chart illustrating the operation of providing first interpreted speech information or second interpreted speech information for content in a customized interpretation server according to various embodiments of the present invention.
3 is an exemplary diagram showing the function of the personalized interpretation system in the intestinal interpretation situation.
4 is an exemplary diagram showing the function of a customized interpretation system in a remote interpretation situation.
5 is an exemplary diagram showing the function of a customized interpretation system in a small-scale interpretation situation using a server.
6 is an exemplary diagram showing the function of a customized interpretation system in a small-scale interpretation situation in which a server is not used.
7 is an exemplary diagram illustrating an operation for restricting the provision of content to certain conditions in accordance with various embodiments of the present invention.
Figure 8 is a flow diagram illustrating the operation of providing commerce content in a customized interpretation server in accordance with various embodiments of the present invention.
9 is a schematic diagram of a customized interpretation system in accordance with another embodiment of the present invention.
FIG. 10 is a view schematically showing a detailed configuration of the listener terminal of FIG. 1. FIG.
11 is a view schematically showing a real-time interpretation method according to an embodiment of the present invention.
12 is a view schematically showing a display screen of a listener terminal for performing a real-time interpretation method according to an embodiment of the present invention.
13 is a view schematically showing a real-time interpretation method according to an embodiment of the present invention.
FIG. 14 is a view schematically showing a real-time interpretation method according to another embodiment of the present invention.
15 is a view schematically showing a display screen of a listener terminal displaying an application for performing a real-time interpretation method according to an embodiment of the present invention.
16 is a view schematically showing a real-time interpretation method according to another embodiment of the present invention.
17 is a view schematically showing another display screen of a listener terminal displaying an application for performing a real-time interpretation method according to an embodiment of the present invention.
18 is a view schematically showing another display screen of a listener terminal displaying an application for performing a real-time interpretation method according to another embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. It should be understood, however, that the invention is not limited to the disclosed embodiments, but may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, Is provided to fully convey the scope of the present invention to a technician, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. The terms " comprises "and / or" comprising "used in the specification do not exclude the presence or addition of one or more other elements in addition to the stated element. Like reference numerals refer to like elements throughout the specification and "and / or" include each and every combination of one or more of the elements mentioned. Although "first "," second "and the like are used to describe various components, it is needless to say that these components are not limited by these terms. These terms are used only to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical scope of the present invention.

Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense that is commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

The terms spatially relative, "below", "beneath", "lower", "above", "upper" And can be used to easily describe a correlation between an element and other elements. Spatially relative terms should be understood in terms of the directions shown in the drawings, including the different directions of components at the time of use or operation. For example, when inverting an element shown in the figures, an element described as "below" or "beneath" of another element may be placed "above" another element . Thus, the exemplary term "below" can include both downward and upward directions. The components can also be oriented in different directions, so that spatially relative terms can be interpreted according to orientation.

The 'original language' mentioned in this document can be interpreted as a language uttered by a speaker, and is not limited to a specific language.

In addition, 'interpretation voice information' mentioned in this document can be interpreted as a voice corresponding to a language different from the original language.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram of a customized interpretation system 1000 in accordance with various embodiments of the present invention.

As shown, the customized interpretation system 1000 generates content containing at least one of an image or a voice signal (e.g., a translation voice signal) and provides it to a user (e.g., a listener) in a live mode, (READY). The provision of such contents can be performed through an application installed in a user terminal or a web service using the Internet.

2 is a flowchart showing an operation of providing first interpretation voice information or second interpretation voice information for contents in the customized interpretation server 100 according to various embodiments of the present invention.

For convenience of explanation, the description process of FIG. 2 will be described with reference to FIG. 3 to FIG. FIG. 3 is an exemplary view showing the function of the customized interpretation system 1000 in the introductory interpretation situation, FIG. 4 is an exemplary view showing the function of the customized interpretation system 1000 in the remote interpretation situation, 6 is a diagram illustrating functions of the personalized interpretation system 1000 in a small scale interpretation situation in which the personalized interpretation server 100 is not used. Fig.

First, the customized interpretation server 100 may display a UI item of content related to a source language through an application or a web page (S210). The content related to the source language may include, for example, images or voices that occur in the form of a specific language at a meeting or a seminar. The UI item may be an icon or UI menu having an execution function for viewing or listening to such content through an application or a web page.

In various embodiments, the customized interpretation server 100 may open a space (e.g., a content room) where video and audio corresponding to specific content can be allocated in the application. For example, each UI window in FIG. 15 to be described later may be a thumbnail corresponding to the content room. Such a space is interlocked with a UI item (e.g., a thumbnail) of the corresponding content, and a voice signal (e.g., a translation voice signal) collected from a transmission client installed on a site or the like may be transmitted or allocated to the opened space.

Next, the personalized interpretation server 100 can collect the first interpretation speech information in real time (S220). Here, the first interpretation speech information may be a speech signal generated by translating the original language into a predetermined language.

For example, in the intramural interpretation situation shown in FIG. 3, a speaker (e.g., a speaker) can speak in a native language (e.g., English), and simultaneously or successively, the interpreter can speak in a predetermined language You can translate the original language. In this case, the first interpretation voice information, which is the translated voice, may be collected through the console installed in the field. This first interpretation speech information can be collected at the transmission client 10 and the transmission client 10 can deliver the first interpretation speech information to the customized interpretation server 100. [

In an embodiment of the present invention, the transmission client 10 may be an electronic device installed at the site or at any location for collecting voice (e.g., interpretation voice information) or video from an external device (e.g., console). In this case, the transmission client 10 can transmit the collected image or voice to a space (for example, a specific content room) previously opened on the application or web page by the customized interpretation server 100.

In the remote interpretation situation as shown in Fig. 4, the transmission client 10 can collect the speaker's voice (e.g., original word) or the scene image on behalf of the console. However, the present invention is not limited thereto, and the transmission client 10 may collect voice or image signals via the console as shown in FIG. Then, the transmission client 10 can transmit the collected voice or image to the original language server 110. The source language server 110 can provide the collected original words to the interpreter, and the interpretation server 120 can collect the interpretation voice information from the interpreter. In this case, the interpretation speech information corresponds to the first interpretation speech information described above. That is, the interpreter is a person who translates the original language of the speaker into a specific language designated in advance. 4, the configuration of the customized interpretation server 100 is functionally classified into the native language server 110 and the interpretation server 120. However, the customized interpretation server 100 is not limited thereto, And may perform the functions of the interpretation server 120.

Also, the transmission client 10 may be implemented through an application installed in the interpreter terminal 300 or the interpreter terminal 300 as shown in FIG. In this case, the interpreter terminal 300 can collect the first interpretation speech information, which the interpreter speaks, and transmit it to the customized interpretation server 100.

On the other hand, the first interpretation voice information can be provided to the listener only by the application installed in the interpreter terminal 300 without going through the customized interpretation server 100. [ For example, in a small-scale interpretation situation in which the customized interpretation server 100 is not used as shown in FIG. 6, the interpreter terminal 300 may be connected to the console, the transmission client 10, And can perform functions of the interpretation server 100.

Referring again to FIG. 2, the customized interpretation server 100 may provide the first interpretation speech information to the listeners in real time according to a user request (S230). For example, a listener who wishes to view content can select (e.g., click or touch) a UI item of content exposed in an application or a web page through the listener terminal 400, and the customized interpretation server 100 can select The first interpretation speech information collected in the form of embodiments can be provided in real time to the listener terminal 400 of FIGS. 3 through 5 through a space previously provided in the application or web page. In this case, the customized interpretation server 100 may provide video or original speech information in addition to the first interpretation speech information to the listener terminal 400 in real time. 6, the application of the interpreter terminal 300 may provide the first interpretation voice information to the listener terminal 400 instead of the customized interpretation server 100. [

According to some embodiments, the customized interpretation server 100 may display a first language item corresponding to the first interpreted speech information in an area adjacent to a location where a UI item of a specific content is displayed. The first language item may be an icon including a text or an image (e.g., a flag image) indicating the language of the first interpretation speech information, but is not limited to a specific form. Thus, listeners can intuitively confirm in which language the corresponding content is being broadcast on an application or a web page.

Next, the customized interpretation server 100 can confirm whether the real-time content is terminated (S240). For example, if a lecture or a seminar at the site ends, the interpreter can terminate the interpretation, and the customized interpretation server 100 confirms that the first interpretation speech information is no longer received, .

If the real-time content has not been terminated, steps S210 to S240 may be continuously performed. If the real-time content is terminated, the customized interpretation server 100 may provide a re-viewing service such as VOD.

Specifically, when the real-time content is terminated, the personalized interpretation server 100 processes real-time content by adjusting the content of the content (e.g., voice or image) and the first interpretation audio information or performing additional subtitle insertion processes And a UI item (e.g., VOD text processing) of the processed content can be configured and displayed.

According to various embodiments, the customized interpretation server 100 can perform the second interpretation voice information upload service in the process of performing the refresh service or in a separate process (S250). This second interpretation voice information uploading service is a service in which a voice of a language not prepared in advance for a specific content in the personalized interpretation system 1000 is received from an arbitrary user and is shared with the listener. Here, the second interpretation speech information may be speech information corresponding to a language other than the first interpretation speech information.

For example, it is assumed that a VOD content in which a Korean translation voice is inserted into a customized interpretation server 100 in response to a speech spoken in English is assumed. At this time, a listener who can speak either English or Korean can generate a translated voice in Japanese after watching the corresponding content, and upload the voice to the customized interpretation server 100. Here, Japanese corresponds to the second interpretation speech information. By doing so, the customized interpretation server 100 can provide the listener with the contents output in Japanese (second interpretation speech information) in addition to the Korean (first interpretation speech information) prepared in advance. That is, it is possible to expand the contents by utilizing the talents of a large number of listeners.

Then, the customized interpretation server 100 can start the contents reflecting the second interpretation speech information (S260), and can provide the first interpretation speech information or the second interpretation speech information to the listeners through the posted content (S270).

In this case, the customized interpretation server 100 may pay a predetermined reward for the user account providing the second interpretation speech information. Thus, it is possible to create an environment in which the second interpretation speech information can be more actively collected. Such predetermined reward may be, but is not limited to, cyber money or points.

When the second interpretation speech information is provided, the customized interpretation server 100 generates a first language item representing the first interpretation speech information and a second language item representing the second interpretation speech information at a position adjacent to the UI item of the view content Items can be distinguished from each other and displayed. By doing so, the listener can intuitively confirm the source or reliability of the provided interpretation voice information.

7 is an exemplary diagram showing an operation for restricting the provision of contents to specific conditions.

The customized interpretation system 1000 or the customized interpretation server 100 can provide the above-mentioned real-time content or re-viewing content only by inputting a predetermined password on the application, such as the 701 screen. In the case of an interpreter situation requiring stronger security, the customized interpretation server 100 may transmit only real-time contents or re-view contents (not shown) to the culler's terminal 400 located in the communication radius 707 of the beacon 705 Can be provided.

According to some embodiments, this security condition may be performed step by step. That is, the customized interpretation server 100 can request that information (for example, password) set in advance on the application or web page under the first condition be input, and if the listener terminal 400 receives the specific beacon 705 in the second condition, Or within the communication radius 707 of < / RTI > The customized interpretation server 100 may provide some or all of the real-time content or the re-viewed content depending on whether the first condition and the second condition are satisfied.

8 is a flow diagram illustrating an operation for providing commerce content in a customized interpretation server 100 in accordance with various embodiments of the present invention. The embodiment of FIG. 8 may represent an embodiment in which the commerce platform is implemented through the personalized interpretation system 1000, and redundant contents of the process of FIG. 2 described above may be omitted or simplified.

First, the customized interpretation server 100 can collect commerce content (S810). The commerce content may be a content including a video or a voice described in a source language for a specific commodity or a specific service, and may be, for example, advertisement contents.

According to various embodiments, the customized interpretation server 100 may collect such commerce content through a matching system (S810). Specifically, the customized interpretation server 100 can match a goods supplier that supplies a specific product or a specific service, and a content producer that produces such goods and services as content in the form of image or voice.

To this end, the customized interpretation server 100 can collect information (for example, product / service name, number, feature, size, effect, etc.) of a product or service to which a specific product is to be supplied And expose the collected product information to an application or a web page. In addition, the customized interpretation server 100 can collect information (e.g., content production history, specialized field, etc.) of the content creator who produces the content in the membership subscription or information writing process, Application or web page. Thus, the commerce content can be produced by communicating between the commodity supplier and the content producer, and the produced commerce content can be provided to the customized interpretation server 100.

Next, the customized interpretation server 100 forms the UI items of the collected commerce content and displays it on the application or web page (S820), and transmits the first interpretation voice information for the commerce content to the listener terminal 400 (S830).

The first interpretation speech information in FIG. 8 may be collected in real time like the first interpretation speech information in FIG. 2, but may be implemented in a form already included in the commerce content according to various embodiments. That is, the commerce content may be generated in real time, but may be in a state in which the content producer has completed the production, so that the first interpretation voice information may be provided in a processed form included in the commerce content.

Then, as shown in FIG. 2, a specific listener can upload the second interpretation voice information for the commerce content to the customized interpretation server 100 (S840).

In the embodiment of FIG. 8, the listener who uploads the second interpretation voice information may be a salesperson of the goods or services included in the contents. For example, when a water bottle of a specific type to be advertised in English is interpreted in Korean and uploaded as commerce content to the customized interpretation server 100, a listener (Japanese-speaking) who listens to it in Japan will sell the water bottle in Japan You might want to. Then, the listener can upload the translated speech information translated into Japanese into the customized interpretation server 100 by using his or her talent to encourage customers in Japan to purchase the water bottle.

Then, the customized interpretation server 100 can publish the commerce content reflecting the second interpretation speech information (e.g., Japanese) through its own website or application (S850) or transmit it to an external media server such as YouTube (S860 ). In this case, the manager of the customized interpretation server 100 may source the commodity article (e.g., goods or service) of commerce content from the commodity supplier and supply the second interpretation voice information to the uploaded salesperson.

According to the embodiment of FIG. 8 as described above, various business environments can be constructed by generating / distributing contents related to commerce based on contents including interpreted speech information.

9 through 18, a personalized interpretation system 1000 according to various embodiments of the present invention will be described. The configuration and functions of the customized settlement server 100, the speaker terminal 200, the interpreter terminal 300 and the listener terminal 400 disclosed in Figs. 9 to 18 are applied to the above-described Figs. 1 to 8 .

Figure 9 is a schematic illustration of a customized interpretation system 1000 in accordance with another embodiment of the present invention.

Referring to FIG. 9, the customized interpretation system 1000 includes a customized interpretation server 100, a speaker terminal 200, an interpreter terminal 300, and a listener terminal 400.

The personalized interpretation server 100, the speaker terminal 200, the interpreter terminal 300, and the listener terminal 400 can transmit and receive various data and / or information to / from each other via the network. The network includes a wired or wireless network. Wireless networks include wireless networks of various communication types such as mobile communication, local area communication, radiocommunication, optical communication, satellite communication, and the like. The customized interpretation server 100 may also include a storage unit (e.g., memory), a communication unit (e.g., a wireless communication module), and a control unit (e.g., a processor). The customized interpretation server 100 may be configured to include a plurality of servers physically or functionally distinguished (the native language server 110 and the interpretation server 120).

The original language server 110 may receive at least one speech signal of a speech from the speaker terminal 200 and transmit the same to the interpreter terminal 300 and the listener terminal 400.

The interpreter server 120 may receive an interpreter signal corresponding to the original speech signal of the at least one speech from the interpreter terminal 300 and may transmit the interpreter signal to the listener terminal 400.

Here, the original language represents the original language (before the interpretation) used by the speaker, and the interpreter language represents the target language to be interpreted by the interpreter from the original language.

On the other hand, the number of servers is not limited to the specific embodiment, and the customized interpretation server 100 may be configured to include a plurality of servers physically or functionally separated or may be constituted by one server. As will be apparent to those skilled in the art.

A speaker terminal (200) represents a terminal used by a speaker. The speaker terminal 200 may be provided in an electronic system including a microphone module, a wireless communication unit, or a computer system.

In the present embodiment, the speaker terminal 200 receives a source speech signal for at least one speech (in the form of speech) from a user (i.e., a speaker) and transmits the source speech signal to the source language server 110 .

The interpreter terminal 300 represents a terminal used by an interpreter. The interpreter terminal 300 may be provided in an electronic system or a computer system including a microphone module and / or a speaker module and a wireless communication unit.

In this embodiment, the interpreter terminal 300 can receive the source speech signal for at least one speech from the source language server 110. The interpreter terminal 300 outputs the original speech signal, receives an interpreter signal (in the form of voice) corresponding to the original speech signal from a user (that is, an interpreter), and transmits the interpreter speech signal to the interpretation server 120 ).

A hearer terminal (400) represents a terminal used by a listener. The listener terminal 400 may be provided as a computer system that a user (i.e., a listener) can use while moving.

For example, the listener terminal 400 may be a computer system such as a smart phone, a tablet, a PDA (Personal Digital Assistant), a laptop, and the like, but the present invention is not limited thereto.

That is, the listener terminal 400 may be provided in any computer system that is connectable to the network and has mobility. In the present invention, the listener terminal 400 may receive at least one source speech signal from the source language server 110 and receive an interpretation signal corresponding to the source speech signal from the interpretation server 120, Reception of a source language signal from the translator server 110 and reception of an interpreter signal from the interpretation server 120 may be simultaneously or separately received.

FIG. 10 is a view schematically showing the detailed configuration of the listener terminal 400 of FIG.

9, the listener terminal 400 includes a wireless communication unit 410, an A / V input unit 420, a user input unit 430, a sensing unit 440, an output unit 450, a storage unit 460, An interface unit 470, a control unit 480, and a power supply unit 490.

The wireless communication unit 410 may wirelessly communicate with an external device. The wireless communication unit 410 can wirelessly communicate with an external device using a wireless communication method such as mobile communication, WiBro, Bluetooth, WiFi, Zigbee, ultrasonic, infrared, and RF have.

However, the radio communication method of the listener terminal 400 is not limited to the specific embodiment. The wireless communication unit 410 may transmit data and / or information received from the external device to the control unit 480 and may transmit the data and / or information transmitted from the control unit 480 to the external device. For this, the wireless communication unit 410 may include a mobile communication module 411 and a short-range communication module 412.

Also, the wireless communication unit 410 may acquire location information of the listener terminal 400 including the location information module 413. [ The location information of the listener terminal 400 may be provided from, for example, a GPS positioning system, a WiFi positioning system, a cellular positioning system or a beacon positioning system, but the present invention is not limited thereto, The location information may be provided from the positioning systems of the mobile terminal. The wireless communication unit 410 may transmit the position information received from the positioning system to the control unit 480.

The A / V input unit 420 is for inputting video or audio signals, and may include a camera module 421 and a microphone module 422.

The user input unit 430 receives various information from the user. The user input unit 430 may include input means such as a keypad, a button, a switch, a touch pad, and a jog wheel. When the touch pad has a mutual layer structure with a display module 451 described later, a touch screen can be configured.

The sensing unit 440 senses the state of the listener terminal 400 or the state of the user. The sensing unit 440 may include sensing means such as a touch sensor, a proximity sensor, a pressure sensor, a vibration sensor, a geomagnetic sensor, a gyro sensor, a velocity sensor, an acceleration sensor, and a biometric sensor. The sensing unit 440 may be used for user input.

The output unit 450 notifies the user of various types of information. The output unit 450 can output information in the form of text, image, or voice. To this end, the output unit 450 may include a display module 451 and a speaker module 452.

The display module 451 may be a PDP (Plasma Display Panel), an LCD (Liquid Crystal Display), a TFT (Thin Film Transistor) LCD, an OLED (Organic Light Emitting Diode), a flexible display, May be provided in any form well known in the art.

The output unit 450 may further comprise any type of output means well known in the art.

The storage unit 460 stores various data and commands. The storage unit 460 may store system software and various applications for operation of the listener terminal 400.

The storage unit 460 may be a random access memory (RAM), a read only memory (ROM), an erasable-programmable ROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a hard disk, a removable disk, Lt; RTI ID = 0.0 > computer-readable < / RTI >

The interface unit 470 serves as a channel with an external device connected to the listener terminal 400. The interface unit 470 receives data and / or information from an external device or receives power and transmits the received data and / or information to the internal components of the listener terminal 400, Or supply internal power.

The interface unit 470 includes, for example, a wired / wireless headset port, a charging port, a wired / wireless data port, a memory card port, a universal serial bus An audio input / output port, a video input / output (I / O) port, and the like.

The control unit 480 controls other components to control the overall operation of the listener terminal 400. The control unit 480 can execute the system software stored in the storage unit 460 and various applications.

The power supply unit 490 includes a wireless communication unit 410, an A / V input unit 420, a user input unit 430, a sensing unit 440, an output unit 450, a storage unit 460, an interface unit 470, And supplies power necessary for the operation of the control unit 480. The power supply 490 may include an internal battery.

9, one speaker terminal 200 and one interpreter terminal 300 are shown in FIG. 9. However, the present invention is not limited thereto, and the speaker terminal 200 and the interpreter terminal 300 The number may be variously modified according to the embodiment.

11 is a view schematically showing a real-time interpretation method according to an embodiment of the present invention.

Referring to FIG. 11, in step S510, the speaker terminal 200 transmits a source signal received from a specific person to the source language server 110. FIG. (S510)

Subsequently, in step S520-1, the source language server 110 can transmit the source language signal received from the speaker terminal 200 to the interpreter terminal 300, and then, in step S520-2, And transmits the original speech signal received from the terminal 200 to the listener terminal 400. (S520-1, S520-2)

In this case, steps S520-1 and S520-2 may be performed simultaneously, or may be separately performed. In addition, the steps S520-1 and S520-2 may be performed in the order of the divided steps, or may be performed in the reverse order according to the setting.

On the other hand, in step S530, the interpreter terminal 300 may receive an interpreter signal corresponding to the original speech signal from the user (that is, an interpreter), and then transmit the interpreter signal to the interpretation server 120. [ At this time, the interpretation server may be included in the customized interpretation server 100 shown in FIG. 11, or may be a separate server separated from the original language server as described above. (S530)

Then, in step S540, the interpretation server 120 transmits the interpretation speech signal received from the interpreter terminal 300 to the listener terminal 400 in step S530. (S540)

Thereafter, in step S550, the listener terminal 400 can receive a user's selection regarding at least one of the source speech signal and the interpretation speech signal. (S550)

Subsequently, in step S560, the listener terminal 400 outputs the language signal selected by the user. (S560)

When the user of the listener terminal 400 selects the original speech signal, the listener terminal 400 outputs the original speech signal. If the user of the listener terminal 400 selects the interpreter signal, the listener terminal 400 Can output an interpreter signal. Alternatively, when the user of the listener terminal 400 selects both the original speech signal and the interpretation speech signal, the listener terminal 400 may distinguish between the original speech signal and the interpretation speech signal.

FIG. 12 is a view schematically showing a display screen of a listener terminal 400 performing a real-time interpretation method according to an embodiment of the present invention.

Referring to FIG. 12, the listener terminal 400 can display the screen 10 related to the original speech signal and the screen 20 relating to the interpreter speech signal. The user can control the output of the source language signal and / or the translator language signal by inputting a control operation including a touch or the like to each screen.

Each screen may display a progress bar 11 indicating the output status and one or more various buttons 12 for controlling the user.

For example, when one of the various buttons 12 is set as a button for performing a recording function, when the user presses a corresponding control operation including a touch, a click, etc., the listener terminal 400 outputs the original speech signal And / or an interpreter signal to be stored in the listener terminal 400, and it is possible to reproduce the original sound signal and / or the translator signal by using the same button or the same button as the upper button.

13 is a view schematically showing a real-time interpretation method according to an embodiment of the present invention.

The real-time interpretation method of FIG. 13 differs from the real-time interpretation method described with reference to FIG. 11 in that transmission of the speech list and selection of at least one speech among the speech list are performed. Hereinafter, for the sake of convenience of explanation, a detailed description of the substantially same steps as those of the real-time interpretation method described with reference to FIG. 11 will be omitted or simplified.

Referring again to FIG. 13, in step S610, the original language server 110 transmits a list of speeches including at least one speech to the listener terminal 400. FIG. (S610). At this time, the source language server 110 may be implemented as a customized interpretation server 100 together with the interpretation server as shown in FIG. 13, whereas the source language server 110 is separated from the interpretation server, Lt; / RTI > server.

In step S620, the listener terminal 400 receives a user selection regarding at least one speech from the speech list received from the source language server 110, and transmits the speech information selected by the user to the source language server 110 do. (S620)

Thereafter, in steps S640-1 and S640-2, the source language server 110 transmits a source language signal related to at least one speech selected by the user to the interpreter terminal 300 and the listener terminal 400, respectively. (S640-1, S640-2)

In step S660, the interpretation server 120 transmits an interpreter signal corresponding to the original speech signal related to the speech selected by the user to the listener terminal 400. [ (S660)

FIG. 14 is a view schematically showing a real-time interpretation method according to another embodiment of the present invention.

The real-time interpretation method of FIG. 14 differs from the real-time interpretation method described with reference to FIG. 11 in that transmission of the speech list and retrieval of at least one speech corresponding to the keyword are performed. For the sake of convenience of explanation, detailed description of steps substantially the same as those of the real-time interpretation method described with reference to Fig. 11 will be omitted or simplified.

Referring to FIG. 14, in step S710, the original language server 110 transmits a list of speeches including at least one speech to the listener terminal 400. FIG. (S710)

Then, in step S720-1, the listener terminal 400 receives the keyword from the user and transmits the keyword information input by the user to the source language server 110. [ (S720-1)

Then, in step S720-2, the original language server 110 searches for at least one speech corresponding to the keyword in the speech list, and transmits the search result to the listener terminal 400. [ (S720-2)

Then, in step S720-3, the listener terminal 400 receives the user selection regarding at least one of the search results received from the source language server 110, and transmits the speech information selected by the user to the source language server 110 send. (S720-3)

Then, in steps S740-1 and S740-2, the source language server 110 transmits a source language signal related to at least one speech selected by the user to the interpreter terminal 300 and the listener terminal 400, respectively. (S740-1)

Then, in step S760, the interpretation server 120 transmits an interpreter signal corresponding to the original speech signal relating to the speech selected by the user to the listener terminal 400. [ (S760)

FIG. 15 is a view schematically showing a display screen of a listener terminal 400 displaying an application for performing a real-time interpretation method according to an embodiment of the present invention.

Referring to FIG. 15, an application for real-time interpretation method of the present invention is shown through screens 701 to 703.

The interpretation method described above with reference to FIGS. 1 to 14 may be performed on the application shown in FIG. 15, and the application in which the real-time interpretation method can be performed may be stored in advance in the listener terminal, A screen relating to the application can be displayed.

At this time, the screen for the stored application can be displayed by including the area for performing the interpretation method described above with reference to FIGS. 1 to 14 on the upper part of the screen as shown in 701, An area for performing broadcast contents (Telecast), an entertainment (Entertainment), a lecture, and the like can be further displayed.

That is, when the user (i.e., listener) selects any one of the area for performing the real-time interpretation method, the broadcast content screen area, the entertainment area, and the lecture area on the screen related to the application displayed on the listener terminal, The application displayed on the terminal can display the selected area on the screen of the listener terminal.

The broadcast content screen means a TV program content including a terrestrial wave, a cable, a partial piece and the like, and an entertainment area means a video and / or audio content excluding a TV program content such as a music video, music, it means.

More specifically, if the user selects the broadcast content area, at least one broadcast content image determined in advance, as shown in 702, may be separately displayed. In the case of selecting any one of the broadcast content images, The selected broadcast content video can be played back. In this case, the interpreting method described above with reference to FIGS. 1 to 14 may be applied as it is, and the broadcast content image including the translator selected by the user may be reproduced.

In addition, if the user selects the entertainment image, at least one predetermined entertainment image, as shown in 703, may be separately displayed. If any one of the entertainment images is selected, the selected entertainment image is reproduced . In this case, the real-time interpretation method described above with reference to FIGS. 1 to 14 may be applied, so that an entertainment image including a translator selected by the user may be reproduced.

16 is a view schematically showing a real-time interpretation method according to another embodiment of the present invention.

16, one or more listener terminals 801 to 812 are shown. In the one or more listener terminals 801 to 812, an application in which the real-time interpretation method described with reference to FIG. .

On the other hand, real-time interpretation can be demanded in large-scale events, but may be required in small events or discussions. In this case, if a customized interpretation server including an interpretation server or a native language server is used, the efficiency may be reduced because the load increases as compared with the real time interpretation performed in the terminal itself without using the server.

Accordingly, according to an embodiment of the present invention, in the case of a small-scale event (for example, an event in which 20 or fewer persons are participating, but the number may vary according to the setting), the real- It is possible to perform real-time interpretation using only the listener's terminal without the intervention of the server.

More specifically, at a ceremony in which 12 listeners participate, such as 801 to 812, a listener (for example, 801) who can interpret a specific original language spoken by the speaker can input a voice to be interpreted using the application , Other listeners can receive the interpreted voice through their application.

17 is a view schematically showing another display screen of a listener terminal displaying an application for performing a real-time interpretation method according to an embodiment of the present invention.

A specific listener can perform a chat room opening through an application. For example, it is possible for a listener of a listener who participated in the event to perform the opening of the room, or to open a room for the listener outside the event that is not participating in the event.

In this case, the chat room refers to a virtual space in which other listeners including a listener capable of interpreting the specific original language can transmit and receive data including text, pictures, moving pictures, voice messages, etc. in real time in real time .

On the other hand, a listener who can interpret a specific original language can set a security code for entering a chat room in the process of opening the chat room.

That is, when a security code for entering a chat room is set, it means that only the listener who has input the set security code (a number of a specific digit and / or a certain number of digits) is allowed to enter the chat room. When other culprits want to enter the chat room through the same setting, a listener who is able to interpret the specific original language can input the security code only by entering the security code set when the chat room is opened.

Each of the other listeners including the listener who is able to interpret the specific original language participating in the chat room opened by the listener who is able to interpret a specific original language can listen to at least one of the at least one interpretation language By entering a choice, you can determine the interpreter language you want.

For example, when the interpretation is performed in the chat room in Korean, Chinese, Japanese, and French, the application in which the real-time interpretation method can be performed is as shown in the screen 902 of FIG. And other listeners including the listener who is able to interpret the specific original language can select a desired interpretation language by selecting at least one of the indications indicating the four interpretation languages.

18 is a view schematically showing another display screen of the listener terminal 400 displaying an application for performing a real-time interpretation method according to another embodiment of the present invention.

Referring to FIG. 18, a listener terminal 1500 is shown. In the listener terminal 1500, an application in which the real-time interpretation method described with reference to FIG. 15 is performed may be stored in advance.

At this time, according to an embodiment of the present invention, an application in which the real-time interpretation method can be performed may include a listening / interpreting mode for a visually impaired person.

More specifically, a user (i.e., listener) can execute a listening / interpretation mode for the visually impaired by inputting a predetermined control operation including touch, click, etc. in an application in which the real-time interpretation method can be performed .

When the listening / interpreting mode for the visually impaired is executed, a progressive bar 1010 and a vertical progress bar 1011 can be displayed as shown in FIG. 18, Are displayed on the left side of the screen and the bottom of the screen, respectively. However, this is only an example, and may be displayed in various shapes such as a horizontal shape and a vertical shape anywhere in the screen.

On the other hand, in the case of the vertical progress bar 1011, when the user moves up and down a control operation including touch, click, drag, and the like to the vertical progress bar 1011 The user can adjust the volume of the voice output on the listener terminal (for example, the interpretation language, the voice of the broadcast content video, etc.).

In the case of the horizontal progress bar 1010, when the user inputs a control operation including touch, click, drag, and the like to the horizontal progress bar 1010 left and right, (For example, an interpretation language, a sound of a broadcast content video, etc.) output on the listener terminal can be regulated.

For example, if the user of the listener terminal 1500 is listening to the real-time interpretation language in the listening / interpreting mode for the visually impaired after executing the listening / interpreting mode for the visually impaired, The output volume of the real-time interpretation language can be adjusted to be larger or smaller by adjusting the response bar 1011 up or down. Also, the user can adjust the playback speed of the real-time interpretation language by adjusting the horizontal progress bar 1010 to the left or right, and the playback time can be adjusted together with the progress bar 1010.

On the other hand, although not clearly shown, in the real-time interpretation method according to the embodiments of the present invention, at least some of the steps may be performed in a different order from the order shown, or may be modified to be performed simultaneously with the other steps.

Also, although not explicitly shown, the real-time interpretation method according to the embodiments of the present invention may be configured such that the interpreter server 120 receives a plurality of interpreter signals corresponding to a source language signal from one or a plurality of interpreter terminals 300 And the interpretation server 120 may be modified to transmit at least one interpretation signal corresponding to the language setting by the listener among the plurality of interpreter signals to the listener terminal 400. [

The steps of a method or algorithm described in connection with the embodiments of the present invention may be embodied directly in hardware, in software modules executed in hardware, or in a combination of both. The software module may be a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD- May reside in any form of computer readable recording medium known in the art to which the invention pertains.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

Claims (10)

A method of providing interpreted speech information for a source language in a customized interpretation server,
Displaying a UI item of contents related to a source language through an application installed in a web page or a listener terminal;
Collecting, in real time, first interpretation speech information generated in a predetermined language for the source language; And
And providing the collected first interpretation speech information to the listener terminal in real time in response to receiving a user selection input for the UI item. The method according to claim 1,
Configuring a first language item corresponding to the first interpretation speech information; And
Further comprising the step of displaying the first language item on a screen displayed through the web page or the application at a position adjacent to a location where the UI item of the content is displayed. The method according to claim 1,
Collecting second interpretation speech information corresponding to a language different from the first interpretation speech information with respect to the source language from a preset user terminal or an arbitrary user terminal when real-time collection of the first interpretation speech information is finished; And
Further comprising providing at least one of the first interpretation speech information or the second interpretation speech information to the listener terminal in response to receiving a user selection input for a UI item of the content . The method of claim 3,
Further comprising paying a predetermined reward for a user account providing the second interpretation speech information. 5. The method of claim 4,
Further comprising displaying a first language item corresponding to the first interpreted speech information and a second language item corresponding to the second interpreted speech information to be distinguished from each other. A method of providing interpreted speech information for a source language in a customized interpretation server,
The method comprising: receiving commerce content for a specific commodity or a specific service from a content producer terminal, the commodity content being content representing the specific commodity or the specific service in a source language;
Displaying a UI item of the commerce content through a web page or an application;
Outputting first interpretation speech information generated in a predetermined language for the source language in response to receiving a user selection input for a UI item of the commerce content;
Collecting second interpretation speech information corresponding to a language different from the first interpretation speech information from the seller terminal; And
And displaying commerce contents reflecting the second interpretation speech information. The method according to claim 6,
Further comprising the step of displaying information of the specific product or information of the content creator in response to a request from a goods supplier or a contents producer supplying the specific goods or a specific service. The method according to claim 6,
And transmitting the commercial contents reflecting the second interpretation speech information to an external media server. A personalized interpretation server for providing interpretation voice information about a source language,
A communication unit; And
A method for displaying a UI item of a content related to a source language through an application installed on a web page or a listener terminal, collecting first interpretation speech information generated in a predetermined language for the source language in real time, A controller for providing the collected first interpretation voice information to the listener terminal through the communication unit in real time in response to receiving the input;
And a personalized interpretation server. 10. The method of claim 9,
Wherein,
A first language item corresponding to the first interpretation voice information is displayed and a first language item is displayed at a position adjacent to a location where the UI item of the content is displayed, Collecting second interpretation speech information corresponding to a language different from the first interpretation speech information from a predetermined user or an arbitrary user for the original language and receiving a user selection input for a UI item of the content And provides at least one of the first interpretation voice information or the second interpretation voice information to the listener terminal through the communication unit in response to the request.

Images