KR20200049404A - System and Method for Providing Simultaneous Interpretation Service for Disabled Person - Google Patents

Abstract

The present invention provides a simultaneous interpretation service providing system for a disabled person which outputs a speaker′s content to be transferred in a voice, a sign language image, and text to provide the disabled person with the speaker′s content to be transferred and provides the text at the same time to prevent a misunderstanding caused by an error in the sign language image, and a method thereof. To achieve this, according to the present invention, the simultaneous interpretation service providing system for a disabled person comprises: an interpreter terminal and a listener terminal operated by operation of an interpreter and a listener, respectively, and executing an application installed between a simple manner of selecting a simultaneous interpretation service displayed on a display window (screen) to be connected to a simultaneous interpretation server; and the simultaneous interpretation server collecting and converting the speaker′s voice or sign language image for simultaneous interpretation into digital data, transmitting the digital data to the interpreter terminal, collecting an interpretation voice or sign language image from the interpreter terminal, converting the interpretation voice or sign language image into second text data thereof, and transmitting a simultaneous interpretation voice, a simultaneous interpretation sign language image, and the second text data to the listener terminal.

Description

System and Method for Providing Simultaneous Interpretation Service for Disabled Persons

The present invention relates to a system and method for providing simultaneous interpretation services for persons with disabilities, and in particular, for persons with disabilities who provide simultaneous interpretation services when an emergency situation requires interpretation by linking at least one disabled terminal and at least one interpreter terminal. The system and method for providing an interpreter service.

Recently, as international exchanges have been actively conducted, the number of cases of talking or speaking with foreigners in other languages has increased, and accordingly, an interpreter is required for smooth communication with foreigners.

As an interpreter for communication with foreigners, as a two-way interpreter that can communicate with foreigners in remote locations using different languages, Korean Patent Publication No. 2002-54192 (invention name: for foreigners In the automatic telephone interpretation system and method), when a foreign user queries in their native language, the contents of the query are automatically interpreted and delivered to the local guide, and when the local guide answers the query in the native language, the response is automatically interpreted and delivered to the foreign user. Disclosed is an automatic telephone interpretation system.

However, in the case of the hearing impaired, it is difficult to use such an interpreter because the interpreter's voice is not heard by an interpreter who conducts simultaneous interpretation.

In particular, as a system currently provided as a means of interpretation for the disabled, a subtitle providing system is applied. The subtitle providing system is used in a television, a large conference seat, or a Telecommunications Relay Service (TRS) provided to people with disabilities in developed countries such as the United States.

However, the subtitle system provided on a television or a large conference room is not a system used by people with disabilities to communicate with their counterparts, and subtitle calls are also used as an auxiliary means to make phone calls with distant parties. Sign language or handwriting is most often used to communicate with the other person in an everyday life.

However, it is difficult to use it as a means of general communication because the other person without knowledge of sign language cannot understand, and writing is not only a discomfort for the other person to use the same handwriting to communicate, but also an environment where the disabled live with the other person. The situation is becoming an obstacle to moving forward.

Registered Patent Publication No. 10-1454745 (Registration Date 2014.10.20.) Registered Patent Publication No. 10-0911717 (Registration date 2009.08.04.)

Accordingly, the present invention was devised to solve the above problems, and provides simultaneous interpretation services for the disabled who provide simultaneous interpretation services when an emergency situation requires interpretation by linking at least one disabled terminal and at least one interpreter terminal. The purpose is to provide a system and method.

In addition, according to the present invention, the contents to be delivered by the speaker (disabled person) can be viewed through voice, sign language, and text, so that the contents of the speaker can be provided to another person with a disability as well as by providing text. The purpose of the present invention is to provide a system and method for providing a simultaneous interpretation service to prevent misunderstandings due to errors in the system.

In addition, the present invention provides a simultaneous interpretation service providing system that can remove the temporal and spatial constraints of the interpreter by enabling the simultaneous interpretation by listening to the speaker's voice through application installation using a smartphone owned by the interpreter and The purpose is to provide the method.

Other objects of the present invention are not limited to those mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

A feature of the system for providing a simultaneous interpretation service for a disabled person according to the present invention for achieving the above object is installed by a simple method of selecting a simultaneous interpretation application displayed on a display window (screen) driven by an interpreter and a listener. An interpreter terminal and a listener terminal connected to a simultaneous interpretation server by executing an application, and a speaker voice or speaker sign language image for simultaneous interpretation is collected and converted into digital data, transmitted to the interpreter terminal, and the interpreted voice from the interpreter terminal Or a simultaneous interpretation server that collects an interpreted sign language image and converts it into second text data according to the interpreted voice or interpreted sign language image, and delivers the interpreted voice, interpreted sign language image, and second text data to a listener terminal.

Preferably, the simultaneous interpretation server converts the collected speaker voice or speaker sign language image into first text data according to the speaker voice or speaker sign language image, and delivers the speaker voice, speaker sign language image, and first text data to the listener terminal. It is characterized by.

Preferably, the simultaneous interpretation server interprets the speaker voice collecting unit for collecting the speaker voice, the audio processor for digitally encoding and encoding the speaker voice collected by the speaker voice collecting unit, and the speaker voice compressed by the audio processor A transmission unit transmitting at least one of a terminal and a listener terminal, transmitting an interpreter voice transmitted from the interpreter terminal to a listener terminal, and transmitting a speaker sign language image or an interpreter sign language image collected by the video collection unit to a listener terminal, and the interpreter An interpreter voice collecting unit that collects an interpreter voice corresponding to the speaker voice transmitted from the terminal, and converts the collected interpreter voice into second text data according to the interpreter voice, and converts the speaker voice collected by the speaker voice collector Converts to the first text data according to the voice and collects images A text conversion unit for converting the interpreted sign language image or the speaker sign language image to second text data by using a sentence according to the sign language image, and a video collection unit for collecting the speaker sign language image and the interpreted sign language image transmitted from the interpreter terminal And, an image segmentation unit for dividing the sign language image collected by the image collection unit into time-based images for each hand movement and non-manual signals (face expressions and gestures) of the deformed speaker (speaker or interpreter), Characters including vowels and consonants, numbers, and special characters that are matched by arranging the images divided by the image division unit in chronological order and comparing them with characters previously stored in the storage unit according to hand movements and non-resin signals. And a character generation unit that generates a sentence by combining the detected characters in chronological order.

Preferably, the transmitting unit is characterized in that at least one of the speaker voice, the interpreter voice, the speaker sign language image and the interpreter sign language image is transmitted to the listener terminal through the text conversion unit.

Preferably, the translator voice collection unit selects the translator voice or speaker sign language image collected for each first unique code included in the transmitted speaker voice or speaker sign language image, and when there are three or more interpreter terminals, the first unique code and It is characterized by selecting an interpreter voice or interpreter sign language image collected for each second unique code included in the interpreter terminal.

Preferably, the simultaneous interpretation server includes a matching processing unit that compares the matching degree of at least three or more interpretation voices or interpretation sign language images collected by the interpretation and voice collection unit, and one interpretation voice having the highest matching degree in the matching processing unit, or It is characterized in that it further comprises an interpreter speech selector to select the interpreter sign language image.

Preferably, the matching processing unit is characterized by comparing the interpretation degree with the number of times that different syllables are generated by comparing the interpreted voice or call sign image with each syllable.

Features of a method for providing a simultaneous interpretation service for a disabled person according to the present invention for achieving the above object are (A) collecting speaker voices or speaker sign language images collected in a multilingual language through a speaker voice collection unit, and (B ) Digitally encoding, encoding and compressing the speaker voice or speaker sign language image collected through the audio / video processing unit, and transmitting the compressed speaker voice or speaker sign language image to at least one of the interpreter terminal and the listener terminal through the transmission unit, ( C) converting the collected speaker voice or speaker sign language image into first text data according to the speaker voice or speaker sign language image through a text conversion unit, and the first text data together with the speaker voice or speaker sign language image to the listener terminal (D) to the interpreter terminal through the interpreter voice collection unit Collecting an interpreted audio or interpreted sign language image transmitted from the (E) transmitting the collected interpreted audio or interpreted sign language image to a listener terminal through a transmission unit, and (F) collecting the interpreted audio or interpreted sign language Converting the image into second text data according to an interpreted voice or interpreted sign language image through a text conversion unit, and transmitting the second text data together with the interpreted voice or interpreted sign language image to a listener terminal; and (G) a transmission unit. And transmitting the second text data according to the interpreted voice and the interpreted voice or the second text data according to the interpreted sign language image and the interpreted sign language image to a listener terminal.

Preferably, in step (D), the interpreted voice or interpreted sign language image is selected as the interpreted voice or interpreted sign language image collected for each first unique code included in the speaker voice or the speaker sign language image transmitted to the interpreter terminal, When there are three or more interpreter terminals, it is characterized in that the interpreter voice or interpreter sign language image collected for each second unique code included in the interpreter terminal together with the first unique code is selected.

Preferably, in step (E), when there are three or more interpreter terminals to which the interpreter voice or the interpreter sign language image is transmitted, the matching degree of at least three interpreter voice or interpreter sign language images collected through a matching processor is compared with each other. And a step of selecting one interpretation voice or interpretation sign language image having the highest matching degree through the interpretation voice selection unit and transmitting the selected interpretation voice or interpretation sign language image to the listener terminal through the transmission unit. Is done.

Preferably, the matching degree is characterized by comparing the matching degree with the number of times that different syllables are generated by comparing the interpreted voice or the interpreted sign language image for each syllable.

Preferably, the conversion of the first text data and the second text data includes hand movement and non-manual signals (face expression) of the hand of a speaker (speaker or interpreter) that transforms the collected sign language image through an image segmentation unit. And gestures) for each time-based image, and the segmented images are arranged in chronological order through the sentence generation unit, and matched through comparison with characters previously stored in the storage unit according to hand movements and non-resin signals. And detecting characters including vowels and consonants, numbers, special characters, and the like, and combining the detected characters in chronological order to generate a sentence.

The system and method for providing a simultaneous interpretation service for the disabled according to the present invention as described above has the following effects.

First, instead of the simultaneous interpretation receiver that was provided in the existing simultaneous interpretation environment, you can listen to the simultaneous interpretation voice through application installation using your own smartphone, and you can freely select the listening language.

Second, listeners with disabilities can easily see what the speaker wants to convey by using voice, sign language, and text, so that they can easily check the content of the speaker, and provide text together to prevent misunderstandings due to errors in sign language images. It has the effect.

Third, there is no temporal or spatial limitation of the interpreter by providing the simultaneous interpretation by listening to the speaker's voice through the application installation using the smartphone owned by the interpreter, so that there is no sign language for the disabled. It has the effect of easily obtaining an interpreter.

Fourth, by providing a low-cost medium / small-scale system and method, instead of the existing high-cost simultaneous interpretation and operation system, it is possible to reduce the cost of operating the simultaneous interpretation system.

1 is a block diagram showing the configuration of a system for providing simultaneous interpretation service for a disabled person according to an embodiment of the present invention
Figure 2 is a first embodiment showing the configuration of the simultaneous interpretation server in Figure 1 in detail
Figure 3 is a second embodiment showing the configuration of the simultaneous interpretation server in Figure 1
4 is a flow chart for explaining a method for providing simultaneous interpretation service for a disabled person according to an embodiment of the present invention

Other objects, features and advantages of the present invention will become apparent through the detailed description of the embodiments with reference to the accompanying drawings.

A preferred embodiment of a system and method for providing location information for a disabled person according to the present invention will be described with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete and the scope of the invention to those skilled in the art. It is provided to inform you. Therefore, the configuration shown in the embodiments and drawings described in this specification is only one of the most preferred embodiments of the present invention and does not represent all of the technical spirit of the present invention, and various equivalents that can replace them at the time of this application It should be understood that there may be water and variations.

1 is a block diagram showing the configuration of a system for providing simultaneous interpretation service for a disabled person according to an embodiment of the present invention.

As shown in Figure 1, the system for providing simultaneous interpretation services for the disabled of the present invention includes an interpreter terminal 100, a simultaneous interpretation server 200, and a listener terminal 300.

The interpreter terminal 100 and the listener terminal 300 are installed with a simultaneous interpretation application for receiving simultaneous interpretation information and simultaneous interpretation service from the simultaneous interpretation server 200 through a communication network. At this time, the simultaneous interpretation application may be provided by the simultaneous interpretation server 200, or may be provided and installed in another smartphone application market, for example, the Apple App Store or Google Android Market.

And the interpreter terminal 100 and the listener terminal 300 is driven by the interpretation of the interpreter and listener, an application installed in a simple way to select (touch or button) the simultaneous interpretation application displayed on the display window (screen) To access the simultaneous interpretation server 200.

Also, the interpreter terminal 100 and the listener terminal 300 may be basically a smart phone, a tablet PC, a mobile phone, a personal digital assistant (PDA), and other mobile computing devices capable of voice recognition and transmission, but are not limited thereto. Does not. Further, the interpreter terminal 100 and the listener terminal 300 may be wearable devices having a voice recognition and transmission function, a communication function, and a data processing function.

At this time, at least one of the speaker and the listener may be disabled, and when the listener answers the interpreter to the speaker, the speaker and the listener will be changed. Accordingly, it should be noted that only the listener terminal 300 is described in the drawing, and the speaker terminal is not separately described.

As an embodiment, if the speaker is deaf, the speaker may be provided in sign language. In this case, the interpreter receives the sign language video, interprets the content (sign language), and delivers it to the listener terminal 300 through the simultaneous interpretation server 200. Accordingly, the listener may be provided with text and an interpreted voice corresponding to sign language content to the listener terminal 300. At this time, the simultaneous interpretation server 200 may provide text according to the speaker sign language video to the listener terminal 300.

As another example, when the listener is a hearing impaired person, the interpreter receives the speaker's voice and interprets it to deliver the sign language video to the listener terminal 300 through the simultaneous interpretation server 200. Accordingly, the listener may be provided with an interpreted sign language video of a speaker's voice in a multilingual language. At this time, the simultaneous interpretation server 200 may provide text according to the interpreted sign language image to the listener terminal 300.

To this end, the simultaneous interpretation server 200 collects speaker voice or sign language images for simultaneous interpretation, converts them into digital data, delivers them to the interpreter terminal 100, and interprets the voice or sign language images interpreted from the interpreter terminal 100. Is collected and converted into second text data according to an interpreted voice or sign language image, and transmits the interpreted voice, sign language image and second text data to a listener terminal 300. Accordingly, the listener selects and confirms at least one of an interpreted voice, a sign language image, and second text data delivered to the listener terminal 300.

At this time, the simultaneous interpretation server 200 may convert the collected speaker voice or sign language image into first text data and transmit the first text data to the listener terminal 300.

In addition, the simultaneous interpretation server 200 compares the interpreted voices collected from the multiple interpreter terminals 100 with each other when the number of interpreted voices collected from the interpreter terminal 100 is three or more, and has the highest matched interpretation voice. Select an interpreter voice. In addition, the simultaneous interpretation server 200 may convert the selected interpretation voice into second text data, and transmit the interpreted voice and second text data to the listener terminal 300.

Meanwhile, the simultaneous interpretation server 200 may provide a simultaneous interpretation service by linking the at least one interpreter terminal 100 and the at least one listener terminal 300. Specifically, the simultaneous interpretation server 200 may provide a cloud instance for simultaneous interpretation to at least one interpreter terminal 100 and at least one listener terminal 300, and the interpreter terminal 100 And it is possible to enable simultaneous interpretation between the listener terminal 300.

In other words, the interpreter and listener may be provided with a simultaneous interpretation service through the interpreter terminal 100 and the listener terminal 300 linked to the cloud instance. Therefore, the voice of the speaker may be transmitted to at least one interpreter terminal 100 through the cloud instance, and then the voice language interpreted by the interpreter terminal 100 may be transmitted to the at least one listener terminal 300 through the cloud instance. Can be.

As described above, the present invention is driven by the operation of the interpreter and listener, and is connected to the simultaneous interpretation server 200 by executing the installed application in a simple way to select the simultaneous interpretation application displayed on the display window (screen).

Example 1

FIG. 2 is a first embodiment showing the configuration of the simultaneous interpretation server in FIG. 1.

2, the simultaneous interpretation server 200 includes a speaker voice collection unit 201, an audio / video processing unit 202, a transmission unit 203, an interpretation voice collection unit 204, and a matching processing unit 205. ), An interpreting and voice selecting unit 206 and a text converting unit 207.

The speaker voice collection unit 201 collects speaker voices conducted in multiple languages.

The audio / video processing unit 202 digitalizes, encodes, and compresses the speaker speech collected by the speaker speech collection unit 201. Compression can be encoded by various compression methods such as MP3.

The transmission unit 203 transmits the speaker voice compressed by the audio / video processing unit 202 to the interpreter terminal 100. In addition, the transmitting unit 203 transmits the interpreted voice transmitted from the interpreter terminal 100 to the listener terminal 100.

Meanwhile, the transmission unit 203 may transmit a text file in which at least one of the speaker voice and the interpretation voice is converted to text through the text conversion unit 207 to the listener terminal 100.

The interpreter voice collection unit 204 collects interpreter voices corresponding to the speaker voices transmitted from the interpreter terminal 100. At this time, the interpreter voice collection unit 204 may select the interpreter voice collected for each first unique code included in the transmitted speaker voice. In addition, when the interpreter voice collection unit 204 has three or more interpreter terminals 100, the interpreter voice collected for each second unique code included in the interpreter terminal 100 together with the first unique code may be selected.

The matching processing unit 205 compares the matching degree of at least three interpreted voices collected by the interpreted voice collection unit 204 with each other. That is, the interpreted voices are compared for each syllable, and the matching degree is compared with the number of times that different syllables are generated. At this time, the reference syllable is not set in advance, and in the process of comparing interpreted voices, syllables commonly used with each other may be used as the reference syllable.

The interpreter voice selector 206 selects one interpreter voice having the highest matching degree from the matching processor 205.

The text converting unit 207 converts the interpreted speech selected by the interpreted speech selector 206 into second text data according to the interpreted speech. At this time, when an interpreter voice is transmitted from one interpreter terminal 100, the interpreter voice transmitted from one interpreter terminal 100 is converted into second text data without configuring the interpreter voice selector 206. In addition, the text conversion unit 207 converts the speaker speech collected by the speaker speech collection unit 201 into first text data according to the speaker speech.

Example 2

FIG. 3 is a second embodiment showing the configuration of the simultaneous interpretation server in FIG. 1.

3, the simultaneous interpretation server 200 includes a speaker voice collection unit 201, an audio / video processing unit 202, a transmission unit 203, an interpretation voice collection unit 204, and a matching processing unit 205. ), An interpreting and voice selection unit 206, a text conversion unit 207, an image collection unit 208, an image division unit 209, a sentence generation unit 210, and a storage unit 211.

The speaker voice collection unit 201 collects speaker voices conducted in multiple languages.

The audio / video processing unit 202 digitally encodes, encodes, and compresses the sign language images collected by the speaker voice and image collection unit 208 collected by the speaker voice collection unit 201. Compression can be encoded by various compression methods such as MP3.

The transmitting unit 203 transmits the speaker's voice or the speaker's sign language compressed by the audio / video processing unit 202 to the interpreter terminal 100. In addition, the transmitting unit 203 transmits an interpreter voice or a speaker sign language image transmitted from the interpreter terminal 100 to the listener terminal 100. At this time, the speaker sign language image refers to an image of the sign language performed by the speaker.

Meanwhile, the transmission unit 203 may transmit a text file in which at least one of the speaker voice and the interpretation voice is converted to text through the text conversion unit 207 to the listener terminal 100. Also, the transmitting unit 203 may transmit the speaker sign language image or the interpreted sign language image collected by the image collection unit 208 to the listener terminal 100.

The interpreter voice collection unit 204 collects interpreter voices corresponding to the speaker voices transmitted from the interpreter terminal 100. At this time, the interpreter voice collection unit 204 may select the interpreter voice collected for each first unique code included in the transmitted speaker voice. In addition, when the interpreter voice collection unit 204 has three or more interpreter terminals 100, the interpreter voice collected for each second unique code included in the interpreter terminal 100 together with the first unique code may be selected.

The matching processing unit 205 compares the matching degree of at least three interpreted voices collected by the interpreted voice collection unit 204 with each other. That is, the interpreted voices are compared for each syllable, and the matching degree is compared with the number of times that different syllables are generated. At this time, the reference syllable is not set in advance, and in the process of comparing interpreted voices, syllables commonly used with each other may be used as the reference syllable.

The interpreter voice selector 206 selects one interpreter voice having the highest matching degree from the matching processor 205.

The text converting unit 207 converts the interpreted speech selected by the interpreted speech selector 206 into second text data according to the interpreted speech. At this time, when an interpreter voice is transmitted from one interpreter terminal 100, the interpreter voice transmitted from one interpreter terminal 100 is converted into second text data without configuring the interpreter voice selector 206. In addition, the text conversion unit 207 converts the interpreted sign language image collected by the image collection unit 208 into second text data by using a sentence according to the sign language image.

Meanwhile, the text conversion unit 207 converts the speaker speech collected by the speaker speech collection unit 201 into first text data according to the speaker speech. In addition, the text conversion unit 207 converts the speaker sign language image collected by the image collection unit 208 into first text data according to the sign language image.

The image collection unit 208 collects a speaker sign language image. Also, the image collection unit 208 collects an interpreted sign language image transmitted from the interpreter terminal 100. In this case, the speaker sign language image refers to an image of sign language performed by a speaker, and the interpreter sign language image refers to an image of sign language performed by an interpreter.

The image segmentation unit 209 is a time unit image for each hand movement and non-manual signals (face expressions and gestures) of the speaker (speaker or interpreter) that transforms the sign language image collected by the image collection unit 208. Divide by.

The sentence generation unit 210 lists the images divided by the image division unit 209 in chronological order, and matches them through comparison with characters previously stored in the storage unit 211 according to hand movements and non-resin signals. Detects characters including vowels and consonants, numbers, and special characters. In addition, the language generation unit 210 generates sentences by combining the detected characters in chronological order.

The operation of the system for providing simultaneous interpretation service according to the present invention configured as described above will be described in detail with reference to the accompanying drawings. The same reference numerals as in FIGS. 1 to 3 refer to the same members that perform the same functions.

4 is a flowchart for explaining a method for providing simultaneous interpretation service according to an embodiment of the present invention.

Referring to FIG. 4, first, a speaker voice or a speaker sign language image collected in a multilingual language is collected through the speaker voice collection unit 201 (S10).

Then, the speaker voice or speaker sign language image collected through the audio processing unit 202 is digitally encoded, encoded, and compressed, and the speaker voice or speaker sign language image compressed through the transmission unit 203 is interpreted by the interpreter terminal 100 and the listener terminal ( 300) is transmitted (S20). At this time, the compression may be encoded by various compression methods such as MP3. In addition, the interpreter terminal 100 may be at least one.

Meanwhile, the collected speaker voice or speaker sign language image may be converted into first text data according to the speaker voice through the text conversion unit 207, and the first text data may be transmitted to the listener terminal 300 together with the speaker voice. (S30). In this case, as one embodiment, the first text data may be transmitted to the interpreter terminal 100.

When the speaker voice or speaker sign language video is input to the interpreter terminal 100 of the interpreter, the interpreter transmits the interpreter voice or interpreter sign language video interpreted by listening to the speaker's voice or viewing the speaker sign language video to the simultaneous interpretation server 200 do. At this time, the interpreter selects a voice language for interpretation, and transmits an interpreted voice or an interpreted sign language image of the selected voice language to the simultaneous interpretation server 200. At this time, the simultaneous interpretation server 200 transmits the first text data according to the speaker voice or the speaker sign language image converted by the text conversion unit 207 to the interpreter terminal 100 together with the speaker voice or the speaker sign language image, so that the interpreter It is possible to make the interpretation more accurate by allowing the first text data to check the error caused by interpreting or viewing only the speaker's voice or the speaker's sign language.

Then, the simultaneous interpretation server 200 collects an interpreted voice or an interpreted sign language image transmitted from the interpreter terminal 100 through the interpreted voice collection unit 204 (S40). In this case, the interpreted voice or interpreted sign language image may be selected for each interpreted voice or interpreted sign language image collected for each first unique code included in the speaker voice or interpreter sign language image transmitted to the interpreter terminal 100. In addition, when the interpreter voice collection unit 204 has three or more interpreter terminals 100, the interpreter voices collected for each second unique code included in the interpreter terminal 100 together with the first unique code are selected.

Subsequently, the interpreted voice or the interpreted sign language image collected through the transmitting unit 203 is transmitted to the listener terminal 300 (S80). At this time, the interpreted voice transmitted to the listener terminal 300 may be compressed by digitalizing and encoding it through the audio processing unit 202. At this time, the compression may be encoded by various compression methods such as MP3.

On the other hand, the collected interpreted voice or interpreted sign language image is converted into second text data according to the interpreted voice or interpreted sign language image through the text conversion unit 207, and the interpreter voice or interpreted sign language image is transmitted to the listener terminal 300. The second text data can be transmitted together.

At this time, when there are three or more interpreter terminals 100 to which the interpreter voice is transmitted (S50), the matching degree of at least three interpreter voices collected through the matching processor 205 is compared with each other (S60). That is, the interpreted voices are compared for each syllable, and the matching degree is compared with the number of times that different syllables are generated. At this time, the reference syllable is not set in advance, and in the process of comparing interpreted voices, syllables commonly used with each other may be used as the reference syllable.

Then, the interpreter voice selector 206 selects one interpreter voice having the highest matching (S70), and transmits the interpreter voice or the interpreter sign language image selected through the transmitter 203 to the listener terminal 300. Transmit (S80).

Meanwhile, the selected (collected) speaker voice or interpreter sign language image is converted into second text data according to the interpreter voice or interpreter sign language image through the text conversion unit 207, and the interpreter voice or interpreter is interpreted by the listener terminal 300. The second text data may be transmitted together with the sign language image (S90).

At this time, the interpreted sign language image is a time-based image for each of the hand movements and non-manual signals (face expressions and gestures) of the hand of the speaker (speaker or interpreter) to transform the sign language image collected through the image division unit 209. Divide.

Then, the images divided through the sentence generation unit 210 are arranged in chronological order, and vowels, consonants, and numbers that are matched through comparison with characters previously stored in the storage unit 211 according to hand movements and non-resin signals. , Characters including special characters are detected. Then, the detected characters are combined in chronological order to generate sentences and converted into second text data.

Then, the listener listens to the interpreted voice provided by the interpreter interpreter server 200 through his listener terminal 300, or views the interpreter sign language image. At this time, the listener terminal 300 is a second voice data according to the voice of the speaker, the speaker's voice, the second text data according to the speaker's voice, the speaker's sign language, and the interpreter's voice or the interpreter's sign language by the listener's manipulation. At least one of the first text data according to the interpretation voice and the first text data according to the interpreted sign language image may be simultaneously confirmed.

Therefore, by allowing the listener to listen to the speaker's content to be transmitted through voice, sign language, and text, it is easy to catch the point emphasized by the speaker, and it is possible to prevent misinterpretation due to the misinterpretation of the interpreter.

Although the technical spirit of the present invention described above has been specifically described in a preferred embodiment, it should be noted that the above-described embodiment is for the purpose of description and not for limitation. In addition, those skilled in the art of the present invention will understand that various embodiments are possible within the scope of the technical spirit of the present invention. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100: interpreter terminal 200: simultaneous interpretation server
201: speaker voice collection unit 202: audio / video processing unit
203: Transmission unit 204: Interpretation and speech collection unit
205: Matching processing unit 206: Interpretation and voice selection unit
207: text conversion unit 208: image collection unit
209: image division unit 210: language generation unit
211: storage unit 300: listener terminal

Claims (8)

An interpreter terminal and a listener terminal driven by an interpreter and a listener and running a simultaneous interpreter application displayed on a display window (screen) to access a simultaneous interpreter server;
Collects speaker voice or speaker sign language images for simultaneous interpretation, converts them into digital data, delivers them to the interpreter terminal, collects interpreted voice or interpreter sign language images from the interpreter terminal, and interprets voice or interpret sign language images according to the second A system for providing simultaneous interpretation services for persons with disabilities, including a simultaneous interpretation server that converts text data into a listener terminal and delivers the interpreted voice, interpreted sign language images, and second text data. According to claim 1,
The simultaneous interpretation server converts the collected speaker voice or speaker sign language image into first text data according to the speaker voice or speaker sign language image, and transmits the speaker voice, speaker sign language image, and first text data to the listener terminal. A system for providing simultaneous interpretation services for disabled people. According to claim 1,
The simultaneous interpretation server
A speaker voice collection unit for collecting a speaker voice,
An audio processing unit that digitalizes, encodes, and compresses the speaker voices collected by the speaker voice collection unit,
The audio processing unit transmits the compressed speaker voice to at least one of an interpreter terminal and a listener terminal, transmits an interpreted voice transmitted from the interpreter terminal to a listener terminal, and listens to a speaker sign language image or interpreter sign language image collected by the video collection unit A transmission unit to transmit to the terminal,
An interpreter voice collection unit for collecting an interpreter voice corresponding to the speaker voice transmitted from the interpreter terminal;
Translating the collected interpretation voice into second text data according to the interpretation voice, converting the speaker voice collected by the speaker voice collection unit into first text data according to the speaker voice, and interpreting sign language image collected by the video collection unit Or a text conversion unit for converting the speaker sign language image into second text data by using a sentence according to the sign language image,
An image collection unit for collecting a speaker sign language image and an interpreted sign language image transmitted from the interpreter terminal;
An image segmentation unit that divides the sign language image collected by the image collection unit into time-based images for each hand movement and non-manual signals (face expressions and gestures) of the deformed speaker (speaker or interpreter);
Characters including vowels and consonants, numbers, and special characters that are matched by arranging the images divided by the image division unit in chronological order and comparing them with characters previously stored in the storage unit according to hand movements and non-resin signals. A system for providing simultaneous interpretation services for persons with disabilities, including a character generator that detects and generates sentences by combining the detected characters in chronological order. The method of claim 3.
The transmitting unit transmits a text file in which at least one of the speaker voice, the interpreter voice, the speaker sign language image, and the interpreter sign language image is converted to text through the text conversion unit to a listener terminal. . The method of claim 3,
The interpreter voice collection unit selects the interpreter voice or speaker sign language image collected for each first unique code included in the transmitted speaker voice or speaker sign language image,
When the number of interpreter terminals is three or more, a system for providing simultaneous interpretation service for persons with disabilities, characterized in that the interpreter voice or interpreter sign language video collected for each second unique code included in the interpreter terminal together with the first unique code is selected. The method of claim 3,
The simultaneous interpretation server
Matching processing unit for comparing the matching degree of at least three or more interpreted speech or interpreted sign language images collected from the interpreted speech collection unit,
A system for providing simultaneous interpretation services for persons with disabilities, further comprising an interpreter voice selector for selecting one interpreter voice or interpreter sign language image having the highest matching degree in the matching processor. The method of claim 6,
The matching processing unit compares the interpretation voice or call sign language with each syllable, and compares the matching degree with the number of times that different syllables are generated. (A) collecting a speaker's voice or a speaker's sign language video in a multilingual language through a speaker's voice collection unit,
(B) digitally encoding and compressing the speaker voice or speaker sign language image collected through the audio / video processing unit, and transmitting the compressed speaker voice or speaker sign language image to at least one of an interpreter terminal and a listener terminal through the transmission unit; ,
(C) converting the collected speaker voice or speaker sign language image into first text data according to a speaker voice or speaker sign language image through a text conversion unit, and the first text together with the speaker voice or speaker sign language image to the listener terminal Passing data,
(D) collecting an interpreted voice or an interpreted sign language image transmitted from the interpreter terminal through an interpreted voice collecting unit,
(E) transmitting the collected translator voice or interpreter sign language image to a listener terminal through a transmission unit,
(F) converting the collected interpreted voice or interpreted sign language image into second text data according to the interpreted voice or interpreted sign language image through a text conversion unit, and the second text data together with the interpreted voice or interpreted sign language image to a listener terminal The step of passing,
(G) Simultaneous interpretation service for persons with disabilities, including passing the collected second and second text data according to the interpreted voice and the interpreted voice to the listener terminal through the translator How to provide.

Images