KR20190025261A - Conversion service system and its method in real-time automatic programming language based on natural language speech recognition - Google Patents

Abstract

The present invention relates to a service system and a method for automatically performing real-time conversion into a programming language based on natural language speech recognition. The service system includes: a first user terminal and a second user terminal for performing conversion-related application services using a programming language; and a service provision server for converting data into the programming language, wherein in a state in which the first and second user terminals are synchronized with each other and set with mutually different first and second languages by using an application service related to the conversion into the programming language, the service provision server receives voice data of the first language from the first user terminal to convert the voice data into character data of the first language through a voice recognition service, translates the converted character data of the first language into character data of the second language through a character translation service, displays the translated character data on a screen of the first user terminal, and converts the translated character data of the second language into voice data of the second language to transmit the voice data of the second language to the second user terminal in real-time. According to the present invention, the programming language can be automatically converted in real-time.

Description

[0001] The present invention relates to a conversion service system and a method thereof for real-time automatic programming language based on natural language speech recognition,

The present invention relates to a speech recognition based real-time automatic programming language for automatically recognizing input speech and displaying it as a character, converting the character into a programming language and displaying it as a character and outputting it as a speech in a programming language, ≪ / RTI >

In general, people who do not have a good programming language often have a good idea but do not know the program. However, without help from others who know the program, it is necessary to search for a dictionary or booklet, but it is difficult to real time response and it is difficult to program the intention properly.

Recently, the development of smart phone technology has developed an automatic interpreter between countries, and it is a field where many research efforts have been made on speech recognition for machine recognition of human voice. The development of sampling technology and self learning function using neural network The automatic recognition rate of voice is increasing and it is put into practical use in some fields.

In addition, a technique of outputting characters by voice is widely used as a voice output device that converts a pronunciation combination of each character, a pronunciation of other words, a pronunciation of a sentence, etc. into a database (DB) and outputs it as a voice. However, a real-time conversion device is required to receive a real-time help in a program language with a computer and a terminal, but is not known at present.

Korean Patent Publication No. 10-2015-0085959

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a method and apparatus for inputting a voice of a natural language and converting it into a character by voice recognition, ), And converts the converted character into a speech signal and displays it on a corresponding user terminal. In addition, the speech recognition based real-time automatic language conversion And to provide a conversion service system and a method thereof in a programming language.

In order to achieve the above-mentioned object, a first aspect of the present invention provides a computer program product comprising: a user terminal for performing a conversion-related application service in a programming language; And a voice recognition module for receiving voice data of the first language from the first user terminal in a state where the first language and the second language are synchronized with each other using a translation related application service in a programming language of the user terminal, And translating the converted character data of the first language into character data of the second language through the character translation service and displaying the converted character data on the screen of the first user terminal And a conversion service providing server for converting the translated character data of the second language into speech data of the second language and providing the service to the second user terminal in real time, Real-time automatic programming language.

Here, the conversion service providing server in the programming language may convert the text data into character data of the first language through the voice recognition service and provide a service to be displayed on the screen of the first user terminal, It is preferable that the converted character data of the first language is modified using the translation related application service and transmitted to the conversion service providing server in the programming language.

Preferably, the conversion service providing server in the programming language provides a service to be displayed on the screen of the second user terminal by converting it into character data of the second language through the voice recognition service, The converted character data of the second language can be modified using the translation application service in the programming language and transmitted to the conversion service providing server in the programming language.

Preferably, the conversion service providing server in the programming language further comprises a first and a second language voice-and-character storing module for converting the character data of the first and second languages respectively corresponding to the voice data into a database (DB) ; First and second language conversion storage modules for converting dictionary and conversion information of each language for conversion of the first and second languages into a database and storing the database; Converting the voice data of the first and second languages into character data of the first and second languages, respectively, and converting the character data of the first and second languages into the first and second language data, respectively, First and second language voice-to-character conversion modules for converting voice data into voice data of a language; First and second language conversion modules for translating character data of the first and second languages converted from the first and second language voice-to-character conversion modules into corresponding counterpart languages; And a controller for receiving the voice data of the first language from the first user terminal and converting the voice data of the first language into character data of the first language by using the first language voice-to-character conversion module and the first language voice- And a translation module for receiving the character data of the first language converted from the first language voice-to-character conversion module and translating the character data into character data of the second language using the first language translation module and the first language translation storage module .

According to a second aspect of the present invention, there is provided a computer program product comprising at least one user terminal for performing conversion related application services in a programming language; And a voice recognition module for receiving voice data of the first language from the user terminal in a state where the first and second languages are set in different first and second languages using a conversion related application service in the programming language of the user terminal, Language character data, translating the converted character data of the first language into character data of the second language through a character translation service, and displaying the translated character data on the screen of the user terminal, A conversion service providing server for converting a character data of a second language into voice data of the second language and providing a service to the user terminal in real time; .

Here, the conversion service providing server in the programming language may convert the text data into character data of the first language through the speech recognition service and provide a service to be displayed on the screen of the user terminal, It is preferable that the converted character data of the first language is modified using the application service and transmitted to the conversion service providing server in the programming language.

A third aspect of the present invention is a method for providing a conversion service to a real-time automatic programming language based on speech recognition using a system including a conversion service providing server in a programming language connected to a user terminal through a communication network, Synchronizing with each other using a conversion-related application service in a programming language of a user terminal, and setting the first and second languages in different languages; (b) converting voice data of the first language transmitted from the first user terminal through the voice recognition service of the conversion service providing server in the programming language into character data of the first language; (c) translating the character data of the first language converted in the step (b) into character data of the second language through the character translation service of the conversion service providing server in the programming language, Providing a service to be displayed on a screen of the display unit; And (d) converting the character data of the second language translated in the step (c) into speech data of the second language through the conversion service providing server in the programming language, and transmitting the character data to the second user terminal in real time And providing a service so as to provide a conversion service method to a real-time automatic programming language based on speech recognition.

Here, after the step (b), after converting the text data into character data of the first language through the speech recognition service of the conversion service providing server in the programming language, the service is provided so that it is displayed on the screen of the first user terminal And modifying the character data of the first language converted in the step (b) by using the conversion related application service in the programming language of the first user terminal, and transmitting the modified character data to the conversion service providing server in the programming language It is preferable to further include.

Preferably, the method further comprises the steps of: converting the text data into text data of the second language through the speech recognition service of the conversion service providing server in the programming language, and providing the text data to be displayed on the screen of the second user terminal; And converting the converted character data of the second language to the conversion service providing server in the programming language by using the conversion related application service in the programming language of the user terminal.

Preferably, after the step (a), the first and second language voice-and-character storage modules of the conversion service providing server in the programming language respectively store the first And converting the character data of the second language into a database (DB) and storing the converted character data in a database (DB) A first language voice-to-character conversion module of the conversion service providing server in the programming language, and a second language voice-to-character conversion module in the first language voice-character storage module, Converting the speech data of the first language transmitted from the first language into character data of the first language; And translating the character data of the first language converted from the first language voice-to-character conversion module into character data of the second language through the first language translation storage module; Providing a service to display character data of a second language translated from the first language translation module on the screen of the first user terminal via the second language translation module; Converting the character data of the second language translated from the first language translation module into voice data of the second language through the conversion module and the second language voice-and-character storage module, And providing the service to be transmitted.

According to a fourth aspect of the present invention, there is provided a method for providing a conversion service to a real-time automatic programming language based on speech recognition using a system including a conversion service providing server in a programming language connected to at least one user terminal through a communication network, a ') setting the first and second languages to different languages using a conversion-related application service in the programming language of the user terminal; (b ') converting the voice data of the first language transmitted from the user terminal through the voice recognition service of the conversion service providing server in the programming language into the character data of the first language; (c ') translating the character data of the first language converted in the step (b') into character data of the second language through the character translation service of the conversion service providing server in the programming language, Providing a service to be displayed on a screen of the display unit; And converting the character data of the second language translated in the step (c ') into speech data of the second language via the conversion service providing server in the programming language, and transmitting the converted speech data to the user terminal in real time And providing a service so as to provide a conversion service method to a real-time automatic programming language based on speech recognition.

Hereinafter, after the step (b '), a service is provided so as to be displayed on the screen of the user terminal after converting into the character data of the first language through the speech recognition service of the conversion service providing server in the programming language And modifying the converted character data of the first language by using the conversion related application service in the programming language of the user terminal, and transmitting the modified character data to the conversion service providing server in the programming language.

A fifth aspect of the present invention provides a computer-readable recording medium storing a program capable of executing a conversion service method in a real-time automatic programming language based on speech recognition as described above.

The real-time automatic programming language based speech recognition based speech recognition method according to the present invention can be implemented by a computer readable code on a computer readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

For example, the computer-readable recording medium includes a ROM, a RAM, a CD-ROM, a magnetic tape, a hard disk, a floppy disk, a removable storage device, a nonvolatile memory, , And optical data storage devices.

According to the conversion service system and method of real-time automatic programming language based on natural language speech recognition of the present invention as described above, natural language is input into a character by speech recognition, and the converted character is input to the input character And displays the converted characters on a corresponding user terminal, converts the converted characters into a voice signal, and outputs the voice signal to the other party. Thus, a real-time automatic programming language There is an advantage to be able to convert.

FIG. 1 is a block diagram illustrating a speech recognition service based real-time automatic programming language according to an exemplary embodiment of the present invention. Referring to FIG.
FIG. 2 is a block diagram illustrating a conversion service providing server according to a preferred embodiment of the present invention. Referring to FIG.
FIG. 3 is a flowchart illustrating a method of converting a speech recognition-based real-time automatic programming language according to an exemplary embodiment of the present invention.

The above and other objects, features, and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings, which are not intended to limit the scope of the present invention. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as " including " an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention. Also, the terms " part, " " module, " and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the following embodiments of the present invention may be modified into various other forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to enable those skilled in the art to more fully understand the present invention.

Each block of the accompanying block diagrams and combinations of steps of the flowcharts may be performed by computer program instructions (execution engines), which may be stored in a general-purpose computer, special purpose computer, or other processor of a programmable data processing apparatus The instructions that are executed through the processor of the computer or other programmable data processing equipment will generate means for performing the functions described in each block or flowchart of the block diagram. These computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement the functionality in a particular manner so that the computer usable or computer readable memory It is also possible for the instructions stored in the block diagram to produce an article of manufacture containing instruction means for performing the functions described in each block or flowchart of the flowchart.

Computer program instructions may also be loaded onto a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible that the instructions that perform the data processing equipment are capable of providing the steps for executing the functions described in each block of the block diagram and at each step of the flowchart.

Also, each block or step may represent a portion of a module, segment, or code that includes one or more executable instructions for executing the specified logical functions, and in some alternative embodiments, It should be noted that functions may occur out of order. For example, two successive blocks or steps may actually be performed substantially concurrently, and it is also possible that the blocks or steps are performed in the reverse order of the function as needed.

FIG. 1 is a block diagram of a whole system for explaining a conversion service system using a real-time automatic programming language based on speech recognition according to an embodiment of the present invention. FIG. FIG. 2 is a block diagram illustrating a server in detail.

Referring to FIGS. 1 and 2, a real-time automatic programming language conversion service system based on speech recognition according to an embodiment of the present invention includes at least one user terminal 100-1 to 100-N and a programming language A conversion service providing server 200, and the like.

Each of the user terminals 100-1 to 100-N is connected to the conversion service providing server 200 in a programming language through the communication network 10. The communication network 10 may be a large capacity, (Wi-Fi), WiGig, Wireless Broadband Internet, Wibro, and Wi-Fi to provide high-speed multimedia services. Interoperability for Microwave Access, Wimax).

The Internet includes a plurality of services such as HTTP (Hyper Text Transfer Protocol), Telnet, File Transfer Protocol (FTP), Domain Name System (DNS), Simple Mail Transfer Protocol (SMTP) Refers to a worldwide open computer network structure that provides Simple Network Management Protocol (SNMP), Network File Service (NFS), Network Information Service (NIS), and the like. Each user terminal 100-1 to 100- To the conversion service providing server 200, as shown in FIG. Meanwhile, the Internet may be a wired or wireless Internet, or may be a core network integrated with a wired public network, a wireless mobile communication network, or a portable Internet.

If the communication network 10 is a mobile communication network, it may be a synchronous mobile communication network or an asynchronous mobile communication network. As an example of the asynchronous mobile communication network, a WCDMA (Wideband Code Division Multiple Access) communication network is exemplified. In this case, although not shown in the drawing, the mobile communication network may include, for example, a radio network controller (RNC). Meanwhile, although the WCDMA network is described as an example, it may be a next generation communication network such as a 3G LTE network, a 4G network, and a 5G network, or an IP network based on other IPs. The communication network 10 transmits signals and / or data of the conversion service providing server 200 to the user terminals 100-1 to 100-N in a programming language.

Each of the user terminals 100-1 to 100-N includes at least one of a smart phone, a smart pad, and a smart note communicating through a wireless Internet or a portable Internet. (Digital Multimedia Broadcasting) phone, a tablet PC, an iPad, a desktop personal computer (PC), a personal computer, a mobile phone, , A notebook PC, or the like, and a wired / wireless home appliance / communication device having a user interface for connecting to the conversion service providing server 200 in a programming language such as a notebook PC.

In particular, when each of the user terminals 100-1 to 100-N is implemented as a normal smart phone, the smart phone may be various applications that the user desires, unlike a general mobile phone (a feature phone) This phone is based on an open operating system that can be freely used and deleted by downloading the program. It can be used for all mobile phones equipped with mobile office functions as well as functions such as voice / It is desirable to understand that the present invention is a communication device that does not have a voice communication function but includes all Internet phones or Tablet PCs capable of accessing the Internet.

Such an open-type operating system may include, for example, Symbian of NOKIA, BlackBerry of RIMS, iPhone of Apple, Microsoft's Windows Mobile, Google's Google Android, and Samsung's ocean.

As described above, since the smartphone uses an open operating system, a user can arbitrarily install and manage various application programs, unlike a mobile phone having a closed operating system.

That is, the smartphone basically includes a control unit, a memory unit, a screen output unit, a key input unit, a sound output unit, a sound input unit, a camera unit, a wireless network communication module, a near field wireless communication module, and a battery for power supply.

The controller is a generic term for controlling the operation of the smartphone, and includes at least one processor and an execution memory, and is connected to each functional unit provided in the smart phone through a bus.

The controller controls the operation of the smartphone by loading at least one program code included in the smartphone into the execution memory through the processor and calculating the result by transferring the result to at least one functional unit through the bus .

The memory unit is a general term of a non-volatile memory included in a smart phone, and stores and maintains at least one program code executed through the control unit and at least one data set in which the program code is used. The memory unit basically stores a system program code and a system data set corresponding to an operating system of a smartphone, a communication program code and a communication data set for processing a wireless communication connection of the smartphone, at least one application program code and an application data set , And the program code and data set for implementing the present invention are also stored in the memory unit.

The screen output unit is composed of a screen output device (e.g., an LCD, an LED device) and an output module for driving the screen output device. The screen output unit is connected to the control unit through a bus, And outputs it to the device.

The key input unit is composed of a key input device having at least one key button (or a touch screen device interlocked with the screen output unit) and an input module for driving the key input unit. The control unit is connected to the control unit via a bus, Or inputs data necessary for the operation of the control unit.

The sound output unit includes a speaker for outputting a sound signal and a sound module for driving the speaker. The sound output unit is connected to the control unit through a bus, and outputs a result of operation corresponding to the sound output from the various operation results of the control unit through the speaker . The sound module decodes sound data to be output through the speaker and converts the sound data into a sound signal.

The sound input unit includes a microphone for receiving a sound signal and a sound module for driving the microphone, and transmits the sound data input through the microphone to the control unit. The sound module encodes and encodes a sound signal input through the microphone.

The camera unit includes an optical unit, a CCD (Charge Coupled Device) and a camera module for driving the CCD unit, and obtains bitmap data input to the CCD through the optical unit. The bitmap data may include both still image data and moving image data.

The wireless network communication module is a collective term for communicating wireless communication and includes at least one antenna, an RF module, a baseband module, and a signal processing module for transmitting and receiving a radio frequency signal of a specific frequency band. And transmits the calculation result corresponding to the wireless communication among the various calculation results of the control unit through the wireless communication or receives the data through the wireless communication and transmits the data to the control unit, , Communication, and handoff procedures.

Also, the wireless network communication module includes a mobile communication structure for performing at least one of connection, location registration, call processing, call connection, data communication, and handoff to a mobile communication network according to the CDMA / WCDMA standard. Meanwhile, according to the intention of those skilled in the art, the wireless network communication module may further include a portable Internet communication structure for performing at least one of connection to the portable Internet, location registration, data communication, and handoff according to the IEEE 802.16 standard, It is evident that the present invention is not limited by the wireless communication configuration provided by the communication module.

The short-range wireless communication module is composed of a short-range wireless communication module that connects a communication session using a radio frequency signal as a communication medium within a predetermined distance. Preferably, the short-range wireless communication module includes RFID communication, Bluetooth communication, Wi- And wireless communication. The short-range wireless communication module may be integrated with the wireless network communication module.

In particular, when each of the user terminals 100-1 to 100-N applied to the embodiment of the present invention is implemented as a smart phone, for example, an application store (App Store) or a conversion service providing server 200 in a programming language Converting to a programming language Through the download of related application programs, it is possible to perform conversion application services in a programming language.

As described above, the conversion service can be used in the real-time programming language by accessing the conversion service providing server 200 in the programming language through the conversion-related application service in the programming language installed in the smartphone.

In addition, a voice signal of each user is received through a microphone using a conversion-related application service in a programming language installed in each of the user terminals 100-1 and 100-2, and the voice signal is converted into digital voice data, To the conversion service providing server 200 in a programming language.

In addition, by using conversion-related application services in a programming language installed in each of the user terminals 100-1 and 100-2, voice data of each user transmitted from the conversion service providing server 200 in the programming language via the communication network 10 And can display the character data on the screen.

The user terminals 100-1 to 100-N may include a multilingual keyboard module (not shown) capable of inputting characters by connecting a country-specific keyboard, A selection button for selecting and modifying character data or translated character data, a correction button, a translation request button for requesting translation of a voice-input character, and a voice output request button for requesting voice output of translated character data, .

When the correct speech recognition or correct translation is not obtained in the character display by the speech input or the translation character display, that is, the error part which can not be automatically processed by the speech recognition service and translation service of the conversion service providing server 200 in the programming language The user can arbitrarily display the current recognition character and display it as a block shade to support the service so that the user can select and re-enter / modify the selected character. In addition, the user terminals 100-1 to 100- So as to support a character input by a combination key or a character input by a menu display method.

The conversion service providing server 200 in a programming language is connected to each of the user terminals 100-1 to 100-N via the communication network 10 and receives speech data of the first language from the first user terminal 100-1 And converts the converted character data of the first language into character data of the second language through the character translation service to convert the converted character data into character data of the first language through the voice recognition service, 1) on the screen of the display unit.

In addition, the conversion service providing server 200 in the programming language may convert the text data of the first language into the character data of the first language through the speech recognition service and provide a service to be displayed on the screen of the first user terminal 100-1 have. At this time, it is preferable that the converted character data of the first language is modified using the translation related application service in the programming language of the first user terminal 100-1 and transmitted to the conversion service providing server 200 in the programming language .

In addition, the conversion service providing server 200 in the programming language can convert the text data into character data of the second language through the speech recognition service and provide a service to be displayed on the screen of the second user terminal 100-2 have. At this time, it is preferable that the converted character data of the second language is modified using the translation related application service in the programming language of the second user terminal 100-2 and transmitted to the conversion service providing server 200 in the programming language .

In this programming language, the conversion service providing server 200 may include a communication module 210, first and second language voice-and-character storage modules 220a and 220b, first and second language translations 220a and 220b, Storage modules 230a and 230b, first and second language voice-to-character conversion modules 240a and 240b, first and second language translation modules 250a and 250b, a control module 260, and the like.

The communication module 210 transmits and receives data and / or signals to / from the user terminals 100-1 through 100-N connected through the communication network 10.

The first and second language voice-and-character storing modules 220a and 220b have a function of storing character data of the first and second languages corresponding to the voice data of the first and second languages into a database (DB) .

The first and second language translation storage modules 230a and 230b function to store dictionary and translation information of each language for translation of the first and second languages into a database (DB).

The first and second language voice-to-character conversion modules 240a and 240b convert voice data of the first and second languages into character data of the first and second languages, respectively, through a voice recognition service (for example, a voice recognition program) And converting the character data of the first and second languages into voice data of the first and second languages, respectively.

The first and second language translation modules 250a and 250b are configured to translate the first and second language voice-to-character conversion modules 240a and 240b from the first and second language voice-to-character conversion modules 240a and 240b through a character translation service And converts character data of a language into a corresponding program language.

The control module 260 is a module for controlling the overall operation of the conversion service providing server 200 in a programming language. In particular, the control module 260 receives the voice data of the first language from the first user terminal 100-1, Character conversion module 240a and the first language voice-and-character storage module 220a so as to be converted into character data of the first language.

The control module 260 receives the converted first language character data from the first language voice-to-character conversion module 240a and transmits the first language translation module 250a and the first language translation storage module 230a To be translated into character data of the second language.

In addition, the control module 260 provides a service to display the translated character data of the second language from the first language translation module 250a on the screen of the first user terminal 100-1, Language voice-to-character conversion module 240b and the second language voice-and-character storage module 220b into voice data of a second language and transmit the voice data to the second user terminal 100-2 in real time .

In addition, the control module 260 provides a service for displaying the translated character data of the first language from the second language translation module 250b on the screen of the second user terminal 100-2, Language voice-to-character conversion module 240a and the first language voice-and-character storage module 220a to convert the voice data into voice data of a first language and transmit the voice data to the first user terminal 100-1 in real- .

The reason why the first and second language voice-to-character conversion modules 240a and 240b are respectively provided is that the signal processing amount for speech recognition is complicated, and a voice having a different language is recognized and converted into a corresponding character And a voice-character database (DB) is constructed for each language, respectively, for recognition processing of voice.

As described above, in the embodiment of the present invention, when the first language is implemented in Korean and the second language in assembly language, if the first user speaks in Korean, Korean is recognized by the speech recognition service and the Hangul is displayed on the screen of the first user terminal 100-1 owned by the first user and the displayed Hangul is automatically converted into an assembly, Is displayed on the screen of the first user terminal 100-1 and the translated assembler is converted into a voice signal and output to the second user terminal 100-2 owned by the second user.

When the second user speaks with the assembly language, the assembly language is recognized by the speech recognition service of the conversion service providing server 200 in the programming language, and the assembly language of the second user terminal 100-2 owned by the second user The displayed assembly is automatically translated into Korean and the Hangul is displayed on the screen of the second user terminal 100-2. In addition, the translated Hangul is converted into a voice signal, And the voice is output to the first user terminal 100-1.

As described above, the present invention is not limited to achieving complete speech recognition and conversion to a programming language. It is also possible to use a variety of sample pronunciation databases (DB) in advance in order to utilize current speech recognition technology, automatic translation technology, And a basic translation database (DB) are constructed and used, and information such as voice, words, and translation is also combined with a registration and learning function, so that a practical application can be practically realized more realistically.

On the other hand, the conversion service providing server 200 in the programming language recognizes the input voice transmitted from each of the user terminals 100-1 to 100-N and converts the converted character information to the control module 260, And each input is distinguished from the conversion-related application in the programming language of the terminals 100-1 to 100-N and displayed.

At this time, if the user views the voice recognition character displayed on the screen of each of the user terminals 100-1 to 100-N, the block is selected with respect to a character not recognized correctly or with a selection key for an erroneously recognized character, Only the voice is inputted repeatedly.

When the voice input is completed, the conversion service providing server 200 in the programming language automatically translates the converted contents into the corresponding language. That is, the first and second language voice-to-character conversion modules 240a and 240b and the first and second language translation modules 250a and 250b of the conversion service providing server 200 in the programming language store the character- And translates and controls each language into a corresponding language through the first and second language translation modules 250a and 250b.

The first and second language translation modules 250a and 250b translate the languages corresponding to each other into the corresponding language using the dictionary stored in the first and second language translation storage modules 230a and 230b and the translation database DB , And the translated contents are transmitted to the control module 260 again and output to the conversion related applications in the programming languages of the user terminals 100-1 to 100-N.

The translated contents are converted into voice information again by the first and second language voice-to-character conversion modules 240a and 240b, respectively, and the voice information is transmitted to the control module 260, (210). Accordingly, since the communication module 210 outputs the voice of the other language, it performs the conversion function in the automatic programming language.

Meanwhile, the conversion service can be performed through a single user terminal (for example, 100-1) in the real-time automatic programming language based on the speech recognition of the present invention.

That is, the conversion service providing server 200 in the programming language, in a state where the first and second languages are set to different languages using the conversion related application service in the programming language of the user terminal 100-1, And converting the converted character data of the first language into character data of the second language through a character translation service, and converting the converted character data of the first language into character data of the second language And provide a service to be displayed on the screen of the user terminal 100-1.

In addition, the conversion service providing server 200 in the programming language may convert the translated character data of the second language into voice data of the second language and provide a service to be transmitted to the user terminal 100-1 in real time.

In addition, the conversion service providing server 200 in the programming language may convert the text data of the first language into the character data of the first language through the voice recognition service, and provide a service to be displayed on the screen of the user terminal 100-1. At this time, it is preferable that the converted character data of the first language is modified using the translation related application service in the programming language of the user terminal 100-1, and transmitted to the conversion service providing server 200 in the programming language.

Hereinafter, a conversion service method using a real-time automatic programming language based on speech recognition according to an embodiment of the present invention will be described in detail.

FIG. 3 is a general flowchart for explaining a conversion service method in a real-time automatic programming language based on speech recognition according to an embodiment of the present invention. Referring to FIG. 3, Based automatic real-time programming language.

Referring to FIGS. 1 to 3, a speech recognition based real-time automatic programming language conversion service method according to an embodiment of the present invention includes: first, After synchronizing with each other using a conversion-related application service in a programming language, the first and second languages are set to different languages (SlOO).

Thereafter, the voice data of the first language transmitted from the first user terminal 100-1 through the voice recognition service of the conversion service providing server 200 in the programming language is converted into the character data of the first language (S200) .

After translating the character data of the first language converted in step S200 into character data of the second language through the character translation service of the conversion service providing server 200 in the programming language, And provides a service to be displayed on a screen of the display unit 100-1 (S300).

Next, the character data of the second language translated in step S300 is converted into voice data of the second language through the conversion service providing server 200 in the programming language, and then converted into voice data of the second language by the second user terminal 100-2 The service is provided so as to be transmitted in real time (S400).

In addition, after the above-described step S200, the character data is converted into character data of the first language through the speech recognition service of the conversion service providing server 200 in the programming language, and then displayed on the screen of the first user terminal 100-1 And converting the character data of the first language converted in step S200 to a programming language by using a conversion related application service in the programming language of the first user terminal 100-1, To the providing server (200).

Further, after step S100, the voice data of the second language transmitted from the second user terminal 100-2 through the speech recognition service of the conversion service providing server 200 in the programming language is converted into the character data of the second language And converting the converted character data of the second language into character data of the first language through the character translation service of the conversion service providing server 200 in the programming language, 2), converting the translated character data of the first language into speech data of the first language through the conversion service providing server 200 in the programming language, And providing a service for real-time transmission to the first user terminal 100-1.

In addition, a service is provided so as to be converted into character data of a second language through a speech recognition service of the conversion service providing server 200 in a programming language, and then displayed on the screen of the second user terminal 100-2 And modifying the converted character data of the second language by using the conversion related application service in the programming language of the second user terminal 100-2 and transmitting the modified character data to the conversion service providing server 200 in the programming language As shown in FIG.

After step S100, the first and second language voice-and-character storage modules 220a and 220b of the conversion service providing server 200 in the programming language respectively correspond to the voice data of the first and second languages (DB) of the character data of the first and second languages and storing the character data of the first and second languages in a database (DB) through the first and second language translation storage modules 230a and 230b of the conversion service providing server 200 in the programming language (DB) and storing dictionary and translation information of each language for translation of the second language, and storing the dictionary and translation information of the first language voice-to-character conversion module 240a and the second language voice-to- Converting the voice data of the first language transmitted from the first user terminal 100-1 through the first language voice-and-character storage module 220a into the character data of the first language; The first language translation module 25 of the server 200 Translating the character data of the first language converted from the first language voice-to-character conversion module 240a into character data of the second language through the first language translation storage module 230a, Providing a service for displaying the translated character data of the second language from the first language translation module 250a on the screen of the first user terminal 100-1 through the conversion service providing server 200; Translated from the first language translation module 250a through the second language voice-to-character conversion module 240b and the second language voice-and-character storage module 220b of the conversion service providing server 200 in the programming language, Converting the character data of the language into voice data of the second language, and providing the service to the second user terminal 100-2 so as to transmit the voice data to the second user terminal 100-2 in real time.

The second language voice-to-character conversion module 240b and the second language voice-and-character storage module 220b of the conversion service providing server 200 in the programming language can also transmit Converting the voice data of the second language into character data of the second language and transmitting the voice data of the second language through the second language translation module 250b and the second language translation storage module 230b of the conversion service providing server 200 in the programming language Translating the character data of the second language converted from the second language voice-to-character conversion module 240b into character data of the first language; Providing a service such that the translated character data of the first language is displayed on the screen of the second user terminal 100-2 from the first user terminal 250b, Language speech-to-character conversion module 240a and the first Converts the translated character data of the first language from the second language translation module 250b into the voice data of the first language through the language voice-and-character storage module 220a and transmits it to the first user terminal 100-1, And transmitting the real-time service to the mobile terminal.

Meanwhile, a conversion service method can be performed through a user terminal (for example, 100-1) in the real-time automatic programming language based on the speech recognition of the present invention.

In other words, it is possible to set different first and second languages using a conversion-related application service in the programming language of the user terminal 100-1, Converting the voice data of the first language transmitted from the user terminal 100-1 into character data of the first language; and converting the voice data of the first language into character data of the first language through the character translation service of the conversion service providing server 200 in the programming language A step of translating the character data of the language into the character data of the second language and then providing a service so that the character data is displayed on the screen of the user terminal 100-1; Converting the translated character data of the second language into voice data of the second language and providing the translated service to the user terminal 100-1 in real time The.

At this time, after the step of converting the voice data of the first language transmitted from the user terminal 100-1 to the character data of the first language through the voice recognition service of the conversion service providing server 200 in the programming language, Converting service data into character data of a first language through a voice recognition service of the conversion service providing server 200 and providing a service to be displayed on a screen of the user terminal 100-1; And converting the converted first language text data to a conversion language service providing server 200 in a programming language by using conversion related application services in a programming language of the first language 100-1.

Meanwhile, the speech recognition based real-time automatic programming language conversion service method according to an embodiment of the present invention can also be implemented as a computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

For example, the computer-readable recording medium includes a ROM, a RAM, a CD-ROM, a magnetic tape, a hard disk, a floppy disk, a removable storage device, a nonvolatile memory, , And optical data storage devices.

In addition, the computer readable recording medium may be distributed and executed in a computer system connected to a computer communication network, and may be stored and executed as a code readable in a distributed manner.

Although the preferred embodiments of the conversion service system and the method thereof have been described with reference to the speech recognition based real-time automatic programming language according to the present invention, the present invention is not limited thereto, And various modifications may be made within the scope of the drawings, which also belong to the present invention.

100-1 to 100-N: a user terminal,
200: Provide conversion service in programming language Server,
210: communication module,
220a and 220b: first and second language voice-character storage modules,
230a and 230b: first and second language translation storage module,
240a and 240b: first and second language voice-to-character conversion module,
250a and 250b: first and second language conversion module,
260: Control module

Claims (17)

A user terminal for performing conversion application services in a programming language; And converting the speech data of the first language from the first user terminal in a state where the first language and the second language are synchronized with each other using conversion related application services in a programming language of the first and second user terminals, And converting the converted character data of the first language into character data of the second language through a character translation service to convert the character data of the first language into character data of the first language through a speech recognition service, Providing a service to be displayed on the screen and converting the converted character data of the second language into voice data of the second language and providing the service to the second user terminal in real time, Speech recognition based real - time automatic programming language conversion service system. The method according to claim 1,
Wherein the conversion service providing server in the programming language converts the text data into character data of the first language through the speech recognition service and provides a service to be displayed on the screen of the first user terminal,
Wherein the converting unit converts the character data of the first language by using the translation related application service in the programming language of the first user terminal and transmits the modified character data to the conversion service providing server in the programming language. Language conversion service system. The method according to claim 1,
Wherein the conversion service providing server in the programming language receives the voice data of the second language from the second user terminal and converts the voice data into character data of the second language through the voice recognition service, Providing a service for translating character data of a second language into character data of the first language and displaying the translated character data on the screen of the second user terminal and converting the translated character data of the first language into voice data of the first language And provides the service to the first user terminal in real-time so that the first user terminal can receive the service in real time. The method of claim 3,
The conversion service providing server in the programming language converts the text data into character data of the second language through the speech recognition service and provides a service to be displayed on the screen of the second user terminal,
And converting the converted character data of the second language to the conversion service providing server in the programming language by using the translation related application service in the programming language of the second user terminal. Language conversion service system. The method according to claim 1,
The conversion service providing server in the programming language,
First and second language voice-and-character storing modules for storing character data of the first and second languages respectively corresponding to the voice data of the first and second languages into a database (DB);
First and second language translation storage modules for storing dictionary and translation information of each language for translation of the first and second languages in a database;
Converting the voice data of the first and second languages into character data of the first and second languages, respectively, and converting the character data of the first and second languages into the first and second language data, respectively, First and second language voice-to-character conversion modules for converting voice data into voice data of a language;
First and second language translation modules for translating character data of the first and second languages converted from the first and second language voice-to-character conversion modules into corresponding counterpart languages; And
Receiving the voice data of the first language from the first user terminal and controlling the voice data to be converted into character data of the first language by using the first language voice-to-character conversion module and the first language voice-and-character storage module A translation module for receiving the character data of the first language converted from the first language voice-to-character conversion module and translating the character data into character data of the second language using the first language translation module and the first language translation storage module And providing a service for displaying character data of a second language translated from the first language translation module on the screen of the first user terminal, and for providing the second language voice-to-character conversion module and the second language voice- A control module for converting the voice data into voice data of the second language using the storage module and providing the voice data to the second user terminal in real- Module based on the speech recognition system. 6. The method of claim 5,
Wherein the control module receives the voice data of the second language from the second user terminal and receives character data of the second language by using the second language voice-to-character conversion module and the second language voice- And a second language translation module for receiving the character data of the second language converted from the second language voice-to-character conversion module and using the second language translation module and the second language translation storage module, Wherein the first language voice-to-character conversion module and the second language voice conversion module provide a service for displaying the translated character data of the first language from the second language translation module on the screen of the second user terminal, Language voice-and-character storage module to convert the voice data into voice data of the first language and to transmit the voice data to the first user terminal in real- Speech recognition based on real-time auto-programming language conversion service system, comprising a step of providing a. At least one user terminal for performing conversion application services in a programming language; And a voice recognition module for receiving voice data of the first language from the user terminal in a state where the first and second languages are set in different first and second languages using a conversion related application service in the programming language of the user terminal, Language character data, translating the converted character data of the first language into character data of the second language through a character translation service, and displaying the translated character data on the screen of the user terminal, A conversion service providing server for converting a character data of a second language into voice data of the second language and providing a service to the user terminal in real time; . 8. The method of claim 7,
Wherein the conversion service providing server in the programming language converts the text data into character data of the first language through the voice recognition service and provides a service to be displayed on the screen of the user terminal,
A real-time automatic programming language based on speech recognition, characterized in that character data of the converted first language is modified using a translation-related application service in the programming language of the user terminal and is transmitted to the conversion service providing server in the programming language Conversion service system. There is provided a method of providing a conversion service to a real-time automatic programming language based on speech recognition using a system including a conversion service providing server in a programming language connected to a user terminal through a communication network,
(a) synchronizing the first and second user terminals with each other using a conversion-related application service in a programming language, and then setting the first and second languages to different first and second languages;
(b) converting voice data of the first language transmitted from the first user terminal through the voice recognition service of the conversion service providing server in the programming language into character data of the first language;
(c) translating the character data of the first language converted in the step (b) into character data of the second language through the character translation service of the conversion service providing server in the lograming language, Providing a service to be displayed on a screen of the terminal; And
(d) converting the character data of the second language translated in the step (c) into voice data of the second language through the conversion service providing server in the programming language, and transmitting the converted voice data to the second user terminal in real time And providing a service based on the speech recognition service. 10. The method of claim 9,
Converting the text data of the first language into text data of the first language through the speech recognition service of the conversion service providing server in the programming language after the step (b), and providing the service to be displayed on the screen of the first user terminal Wow,
Further comprising the step of modifying the character data of the first language converted in the step (b) by using the conversion related application service in the programming language of the first user terminal and transmitting the modified character data to the conversion service providing server in the programming language A method for transforming service into a real - time automatic programming language based on speech recognition. 10. The method of claim 9,
Converting the voice data of the second language transmitted from the second user terminal through the voice recognition service of the conversion service providing server in the programming language into the character data of the second language after the step (a)
Translating the converted character data of the second language into character data of the first language through a character translation service of the conversion service providing server in the programming language and then displaying the translated character data on the screen of the second user terminal , ≪ / RTI &
Converting the translated character data of the first language into speech data of the first language through the conversion service providing server in the programming language and then providing the service to the first user terminal in real time Wherein the speech recognition service is a real-time automatic programming language based on speech recognition. 12. The method of claim 11,
Providing a service to be displayed on the screen of the second user terminal after converting it into character data of the second language through the speech recognition service of the conversion service providing server in the programming language,
Further comprising the step of modifying the converted character data of the second language by using the conversion related application service in the programming language of the second user terminal and transmitting the modified character data to the conversion service providing server in the programming language Based real - time automatic programming language. 10. The method of claim 9,
Wherein the first and second language voice-and-character storing modules of the conversion service providing server in the programming language respectively correspond to the first and second language voice data, respectively, after the step (a) Converting the character data of the language into a database (DB)
Storing and translating dictionary and translation information of each language for translation of the first and second languages through the first and second language translation storage modules of the translation service providing server in the programming language;
A first language voice-to-character conversion module of the conversion service providing server in the programming language and voice data of a first language transmitted from the first user terminal through the first language voice-character storage module, Data;
Converting the character data of the first language converted from the first language voice-to-character conversion module into character data of the second language through the first language translation module of the conversion service providing server in the programming language and the first language translation storage module; , ≪ / RTI >
Providing a service for displaying character data of a second language translated from the first language translation module on a screen of the first user terminal through a conversion service providing server in the programming language;
A second language voice-to-character conversion module of the conversion service providing server in the programming language and character data of a second language translated from the first language translation module through the second language voice-character storage module, Converting the voice data into voice data, and providing the service to the second user terminal in real time; and converting the voice data into voice data. 14. The method of claim 13,
A second language voice-to-character conversion module of the conversion service providing server in the programming language and voice data of the second language transmitted from the second user terminal through the second language voice-character storage module, Converting the data into character data;
Converting the character data of the second language converted from the second language voice-to-character conversion module through the second language translation module of the conversion service providing server and the second language translation storage module into the character data of the first language , ≪ / RTI >
Providing a service for displaying character data of a first language translated from the second language translation module on a screen of the second user terminal through a conversion service providing server in the programming language;
A first language voice-to-character conversion module of the conversion service providing server in the programming language and character data of a first language translated from the second language translation module through the first language voice-character storage module, Converting the voice data into voice data, and providing the service to the first user terminal in real-time so as to transmit the voice data to the first user terminal. A method for providing a conversion service to a real-time automatic programming language based on speech recognition using a system including a conversion service providing server in a programming language connected to at least one user terminal through a communication network,
(a ') setting different first and second languages using a conversion-related application service in a programming language of the user terminal;
(b ') converting the voice data of the first language transmitted from the user terminal through the voice recognition service of the conversion service providing server in the programming language into the character data of the first language;
(c ') translating the character data of the first language converted in the step (b') into character data of the second language through the character translation service of the conversion service providing server in the programming language, Providing a service to be displayed on a screen of the display unit; And converting the character data of the second language translated in the step (c ') into speech data of the second language via the conversion service providing server in the programming language, and transmitting the converted speech data to the user terminal in real time And providing a service to the speech recognition service based on the speech recognition service. 16. The method of claim 15,
Providing a service to be displayed on the screen of the user terminal after converting the text data into character data of the first language through the speech recognition service of the conversion service providing server in the programming language after the step (b ');
Further comprising the step of modifying the converted character data of the first language by using the conversion related application service in the programming language of the user terminal and transmitting the modified character data to the conversion service providing server in the programming language A method of transformation service in a real - time automatic programming language. A computer-readable recording medium having recorded thereon a computer program for executing the method according to any one of claims 9 to 16.

Images