KR20150086902A - Finger-language translation providing system for deaf person - Google Patents

Abstract

According to an embodiment of the present invention, an automatic sign language translation providing system for the hearing impaired users can provide the real-time lecture video for the hearing impaired users in a remote place. The automatic sign language translation providing system includes: a sign language service providing server which is connected through a wired or wireless communication network to provide the lecture video data including the lecture video and sound information; a hearing impaired user terminal which access the sign language service providing server to receive and output the wanted lecture video data; and a sign language supporter terminal which converts the lecture video and the sound information wanted by the hearing impaired user terminal to sign language and subtitle data to transmit the converted data with the lecture video to the hearing impaired user terminal.

Description

{FINGER-LANGUAGE TRANSLATION PROVIDING SYSTEM FOR DEAF PERSON}

One embodiment of the present invention relates to a system for providing automatic translation of a sign language for a hearing impaired person who automatically provides image information converted into sign language for speech.

In general, a deaf or hearing impaired person refers to a person who is unable to perceive the sound as an ear due to abnormal or improper hearing function. Therefore, the hearing impaired people see the sign of the set of predefined hands and communicate to the other party or receive the doctor. In addition, the education of the hearing impaired is also educated using sign language.

Currently, there are about 350,000 hearing-impaired people in Korea. There are many hearing-impaired people in each country in the world. However, it is difficult for ordinary people who do not know sign language, which is a communication means of hearing impaired people, to communicate with people with hearing impairments . Especially, many disabled people who are employed by corporations or public institutions are having difficulty communicating with their colleagues or supervisors, and as ordinary colleagues or supervisors who do not know sign language, it is difficult to communicate with the deaf .

Furthermore, as the development of computer, internet and smartphone technologies has enabled various real-time education, the e-learning system for the hearing-impaired is not well established yet.

As described above, it is a realistic problem that there is a lack of online contents produced for the hearing-impaired person, and even if there is an online sign language education program for the hearing-impaired person, there is a problem in education between the instructor and the student in real time.

Even if the teacher and the hearing-impaired for hearing-impaired people are educated online in real time, real-time education between the teacher and the hearing-impaired is difficult because of the real difficulty that the speaker's voice can not be translated in real time by sign language .

Also, even among family members or friends who have a normal hearing function with their hearing-impaired children as their children, they often do not know sign language. Therefore, even if the family members or friends of the hearing-impaired person do not know the sign language, a means is required to effectively display the voice as sign language and transmit it to the hearing-impaired person.

Korean Patent Registration No. 10-1151865 'Portable Speech Device for the Deaf' Registration Utility Model No. 20-0318046 " System for the Deaf &

One embodiment of the present invention is a system for automatic deaf-mute translation for hearing-impaired persons who automatically provides image information converted into sign language to a deaf person and enables communication between the deaf and the deaf to provide.

The automatic salvation translation system for the hearing impaired according to an embodiment of the present invention is a system for providing handwritten automatic translation for the hearing impaired, in which the sign language image information for the word information is stored in advance and the sign language image information matched with the word information provided from the outside is extracted, A sign language service providing server for providing a sign language service; And an automatic sign language interpretation application, recognizing externally input speech information by executing the sign language automatic application, converting the speech information into word information, and providing the speech information to the sign language service providing server, And receiving and outputting the mature image information for the word information from the terminal.

The sign language video information may include sign language video data and caption data for the sign language video data.

And a sign language support terminal for generating sign language image data and caption data for the word information and providing the sign language image data and the caption data to the sign language service providing server.

Wherein the sign language service providing server comprises: a communication module for transmitting and receiving data between the handover terminal and the sign language support terminal; A signal processing module for signal processing the sign language video data and the caption data uploaded by the sign language support terminal; A streaming providing module for streaming the signal-processed mime-image data and the caption data to the disabled terminal; A word information DB including word information recorded in a sign language standard dictionary; A storage module for storing data transmitted and received by the communication module, the storage module storing registration information of the disabled person and a sign language support company, sign language video data for each word information uploaded by the sign language support terminal, and caption data; A word information extraction module for extracting matched word information by comparing and analyzing word information provided from the disabled terminal and word information stored in the word information DB; A mature image information extraction module for extracting matched mature image data and caption data by comparing and analyzing the extracted word information and mature image data and caption data for each word information stored in the storage module; A disabled person management module for registering and managing disabled person information requesting the sign language video information, and providing a convenience function including a Paint and Web sharing function to the disabled person terminal; And a control module for providing a sign language automatic translation application to the disabled terminal and the sign language support terminal and controlling the operation of each of the components

The handy terminal includes a first communication unit for transmitting and receiving data with the hydration service providing server; A voice recognition unit for recognizing voice information inputted from outside; A speech separator for separating the recognized speech information into spaces and characterizing the speech information; A first photographing unit for photographing the image of the disabled person; An image information transmission unit for transmitting a question image of the disabled person to the photographed image and the output image; A first information input unit for inputting registration information of the disabled person and inputting question data desired by the disabled person; A user authentication unit for authenticating a user based on registration information of the disabled person; A first display unit for displaying data transmitted and received by the first communication unit; And a first control unit which is equipped with an automatic sign language interpretation application and controls the operation of each of the structures by executing the automatic sign language interpretation application.

Wherein the word information extraction module compares and searches a part matched with the word information stored in the word information DB for each unit objectized by the speech separation unit, So that matching word information can be extracted.

The voice recognition unit may remove voice in a specific frequency band from voice information input from the outside and recognize the voice information that has been removed.

Wherein the sign language support terminal comprises: a second communication unit for transmitting and receiving data with the sign language service providing server; A second photographing unit for photographing a sign language video performed by a sign language support company for each word information; A second information input unit connected to the sign language service providing server for inputting registration information and authentication information of the sign language support company and inputting education information on the sign language video; A subtitle support unit for converting subtitle description of the sign language video into subtitle data; A digital editing unit for editing the sign language video and the caption data to generate the sign language video information; A sign language video uploading unit for uploading the sign language video information to the sign language service providing server; A second display unit for displaying data transmitted and received by the second communication unit; And a second control unit which is equipped with an automatic sign language translation application and controls the operation of each of the structures by executing the automatic sign language interpretation application.

The sign language service providing server receives the first encryption information from the disabled terminal, restores the encryption key included in the first encryption information, encrypts the restored encryption key and the sign language video information requested for the disabled terminal, And may further include a security module provided to the terminal.

Wherein the security module includes: a sharing determination unit for determining whether to share the sign language video information requested from the disabled terminal; An encrypted information analyzer for analyzing the first encryption information received from the disabled terminal; An encryption command generation unit for generating an encryption command according to an analysis result of the first encryption information; And an encryption data generating unit for encrypting the encryption key included in the first encryption information and the data requested to be shared from the disabled terminal and generating data, and the encrypted data may include reference data for generating a modified key .

Wherein the handover terminal includes a data sharing request unit for requesting the sign language service providing server to share desired sign language video information; An encrypted data analysis unit for analyzing the encrypted data transmitted from the sign language service providing server; A decryption command generation unit for generating a decryption command according to an analysis result of the encrypted data transmitted from the sign language service providing server; A serial number comparing unit for comparing the encryption key included in the encrypted data transmitted from the sign language service providing server and the encryption key of the disabled terminal; And a data decoding unit for decoding the encrypted data when the encryption key included in the encrypted data transmitted from the hydration service providing server matches the encryption key of the disabled terminal.

According to one embodiment of the present invention, there is provided a system for automatically translating sign language for the hearing impaired, which is provided to reduce the difficulty of a person with a disability who is difficult to communicate with a non-disabled person due to hearing impairment, It is possible to communicate automatically with non-handicapped people who are not aware of sign language.

In addition, according to an embodiment of the present invention, a hearing-impaired person can access a web regardless of environmental conditions and receive services for smooth communication with a non-disabled person through remote assistance.

1 is a diagram schematically showing a system for providing automatic translation of a sign language for the hearing impaired according to an embodiment of the present invention.
2 is a block diagram schematically showing the configuration of the hydration service providing server of FIG.
3 is a block diagram schematically showing the configuration of the security module of FIG.
4 is a view for explaining an example of the operations of the word information extraction module and the sign language image information extraction module of FIG.
5 is a block diagram schematically showing a configuration of a disabled terminal of FIG.
FIG. 6 is a block diagram schematically showing the configuration of the sign language support terminal of FIG. 1;

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, in which those skilled in the art can readily implement the present invention.

2 is a block diagram schematically showing the configuration of a sign language service providing server of FIG. 1, and FIG. 3 is a block diagram of a sign language service providing server of FIG. 4 is a view for explaining an example of the operation of the word information extraction module and the sign language video information extraction module of FIG. 2, FIG. 5 is a block diagram of the security module of FIG. 1, FIG. 6 is a block diagram schematically showing the configuration of the sign language support terminal of FIG. 1; FIG.

Referring to FIG. 1, a system for providing automatic language translation for the hearing impaired according to an embodiment of the present invention includes a sign language service providing server 10, a disabled terminal 20, and a sign language support terminal 30. The automatic sign language translation provision system for the hearing impaired person connects the disabled handset terminal 20 and the sign language support terminal 30 to the sign language service providing server 10 through a wired / wireless communication network so that the handicapped person can know the sign language It is a system that enables communication with non-disabled person (1).

The sign language service providing server 10 is connected to the disabled terminal 20 and the sign language support terminal 30 via a wired / wireless communication network and is provided with a sign language video including sign language video data and caption data matched with word information of voice information It is a web server device that provides and manages information. The caption data is data for describing the sign language video data as caption characters. In addition, the sign language service providing server 10 extracts the sign language image information matching the word information provided from the outside, in which the sign language image information for the word information uploaded by the sign language support terminal 30 is stored in advance, And provides the extracted sign language video information to the disabled terminal 20.

The sign language service providing server 10 includes a communication module 110, a signal processing module 120, a streaming providing module 130, a word information DB 140, a storage module 145, a security module 150, An extraction module 160, a sign language video information extraction module 170, a disabled person management module 180, and a control module 190.

The communication module 110 is a device for transmitting and receiving data to and from the handy terminal 20 and the handset support terminal 30. The communication module 110 includes a handover terminal 20 and a sign language support terminal 30 ), A predetermined communication channel is established, and sign language video information, request information of the disabled terminal 20, request information of the sign language support terminal 30, and the like are transmitted to the sign language support server 10 using the communication protocol defined in the communication program provided in the sign language service providing server 10. [ Support video information, sign language video data, caption data, and the like. In the present invention, wired technologies such as ethernet, RS-232C, RS-485, TCP / IP, RS-232 and IEEE 1394 are used for data transmission and reception with the handy terminal 20 and the sign language support terminal 30, IEEE 802.11 WLAN (Wireless Local Area Network), IEEE 802.15 WPAN (Wireless Personal Area Network), UWB (Ultra WideBand), WiFi, ZigBee, Bluetooth, 3G, 4G, LTE and the like Wireless technology can also be applied.

The signal processing module 120 is a device for signal processing the sign language video data and caption data uploaded by the sign language support terminal 30. For example, the signal processing module 120 may remove noise included in the mapped image data and the caption data.

The streaming providing module 130 is a device for streaming the sign language video data and the caption data processed by the signal processing module 120 to the disabled terminal 20. That is, the streaming providing module 130 encodes the input video data and the caption data for the word information, and provides the encoded data to the disabled terminal 20 in a streaming manner.

The word information DB 140 includes word information recorded in the sign language standard dictionary. In addition, the word information DB 140 stores word information including word information recorded in a sign language standard dictionary, and also stores special word information for dialects, dialects, foreign words, and foreign words not recorded in the sign language standard dictionary .

The storage module 145 stores data transmitted and received by the communication module 110 and stores registration information of a disabled person and a sign language support company and sign language video data and caption data per word information uploaded by the sign language support terminal 30 . The storage module 145 may comprise a RAM, a ROM, an EEPROM (Electrically Erasable and Programmable Read Only Memory), an SDRAM (Synchronous Dynamic Random Access Memory), or an HDD, no.

The security module 150 receives the first encryption information from the disabled terminal 20, restores the encryption key included in the first encryption information, and transmits the restored encryption key and the requested authentication And provides the encrypted video information to the disabled terminal 20.

The security module 150 may include a sharing determination unit 151, an encryption information analysis unit 152, an encryption command generation unit 153, and an encryption data generation unit 154.

The sharing determination unit 151 determines whether sharing of sign language video information requested from the disabled terminal 20 is permitted. If the sign language service providing server 10 does not want to share the requested data from the disabled terminal 20, the requested data is encrypted by encrypting the general data.

The encryption information analysis unit 152 analyzes the first encryption information received from the disabled terminal 20. That is, the encryption information analysis unit 152 inquires the chip serial number information included in the first encryption information, and generates the corresponding location information.

The encryption command generation unit 153 generates an encryption command according to the analysis result of the first encryption information. That is, the encryption command generation unit 153 generates an encryption command when the chip serial number included in the first encryption information is inquired and the corresponding location information is generated.

The encryption data generation unit 154 encrypts the encryption key included in the first encryption information and the data requested to be shared from the disabled terminal 20 to generate data. The encrypted data may include reference data for generating a variant key. The encrypted data may include an ID, a chip serial number, location information, and sharing request data of the disabled terminal 20.

In addition, the encrypted data generation unit 154 may have a session key usage effect by padding different reference data to the encrypted data at different positions. When the encryption is normally performed by the encryption data generation unit 154, the header is formed in the encrypted data, and the encrypted data with the header added thereto is transmitted to the disabled terminal 20 through the communication module 110 .

The word information extraction module 160 is a device for extracting matched word information by comparing and analyzing word information provided from the disabled terminal 20 and word information stored in the word information DB 140. [ That is, the word information extraction module 160 compares the word information stored in the word information DB 140 with a matching part of the unitized object by the voice separation unit 230 of the disabled terminal 20, The parsed speech information may be extracted from the word information DB 140 by parsing the parsed speech information so that it matches the word information that is objectively searched by the speech separating unit 230 of the speech recognition unit 20.

The saliency image information extraction module 170 compares and analyzes the word information extracted by the word information extraction module 160 with the saliency image data and the caption data per word information stored in the storage module 145, Subtitle data is extracted. The thus-extracted sign language video data and caption data are provided to the disabled terminal 20 requesting the sign language automatic translation service.

The disabled person management module 180 is a device for registering and managing disabled person information requesting sign language video information, and can collectively manage authentication and authorization of a disabled person using the service. Also, the disabled person management module 180 can automatically log into the disabled person and the non-disabled person according to the ID of the disabled person. In addition, the disabled person management module 180 may provide a convenience function including the Paint and Web sharing function to the disabled terminal 20. That is, in addition to the function of registering and managing information of a disabled person, the disabled person management module 180 may add a Paint or Web sharing function to use external data required for sign language video information. In addition, the handicap management module 180 may provide functions such as information retrieval, community, mail, messenger, SMS, and blog (contents production) to enhance utilization and facilitate convenience for the disabled.

The control module 190 provides an automatic sign language translation application to the handy terminal 20 and the sign language support terminal 30 and provides the respective components The operation of the module 130, the word information DB 140, the storage module 145, the security module 150, the word information extraction module 160, the sign language video information extraction module 170, . The automatic sign language interpretation application is a program that is provided by the company that provides the automatic sign language interpretation service and is uploaded to a web market such as an app store or an Android market. The automatic sign language interpretation application is provided to the disabled terminal 20 or the sign language support terminal 30, After downloading and installing it, you will be provided with the automatic sign language translation service.

The handy terminal 20 accesses the sign language service providing server 10 to request automatic sign language translation service and transmits sign language video information matching the desired word information among the sign language video information uploaded by the sign language support terminal 30 Receiving and outputting the data. That is, the handy terminal 20 mounts the automatic sign language translation application, recognizes the voice information inputted from the outside by executing the automatic sign language application, converts the voice information into word information, and transmits the word information to the sign language service providing server 10 And receives and outputs the sign language video information for the word information from the sign language service providing server 10. The disabled terminal 20 is equipped with a wired / wireless interface and has an Internet browser capable of displaying the contents of the web so as to enable wired / wireless Internet communication. The terminal 20 is connected to an external terminal device and is capable of transmitting and receiving images, (PDA), a hand-held personal computer (HPC), a web pad, a notebook, a wireless application protocol (WAP) phone, a palm PC, an e-book A terminal, and a hand held terminal (HHT).

The disabled terminal 20 includes a first communication unit 210, a voice recognition unit 220, a voice separation unit 230, a first photographing unit 240, an image information transmission unit 245, a first information input unit 250 A data sharing request unit 251, an encrypted data analysis unit 252, a decryption instruction generation unit 253, a serial number comparison unit 254, a data decryption unit 260, a user authentication unit 270, A display unit 280, and a first control unit 290.

The first communication unit 210 is a device for transmitting and receiving data with the sign language service providing server 10 and connects the sign language service providing server 10 with a predetermined communication channel based on a protocol stack defined in the communication network, And transmits request information, sign language video data, caption data, and the like of the disabled terminal 20 using the communication protocol defined in the communication program provided in the providing server 10. In the present invention, wired technologies such as Ethernet, RS-232C, RS-485, TCP / IP, RS-232, and IEEE 1394 and IEEE 802.11 WLAN, UWB , A Wi-Fi method, a ZigBee method, a Bluetooth method, a 3G method, a 4G method, an LTE method, and the equivalent method. The voice recognition unit 220 is a device for recognizing voice information input from the outside through input means such as a microphone in a situation of daily conversation or the like. The voice recognition unit 220 recognizes the voice information as a specific word or sentence. The voice recognition unit 220 can recognize voice information using various known methods. For example, the speech recognition unit 220 can recognize the speech by the HMM method. The HMM (Hidden Markov Model) method is a method of modeling a basic unit (phoneme) for speech recognition, and includes a phoneme recognized by the speech recognition unit 220 and a database separately provided in the speech recognition unit 220 Speech can be recognized by combining words and sentences with phonemes. The HMM method is a dual probability processing method that estimates an unobservable process through another observable process. The HMM method recognizes speech as a minimum unit (phonemic) of speech recognition and thus has a higher recognition rate than other methods. In addition, the speech recognition unit 220 can remove the voice of a specific frequency band from the externally input voice information, and recognize the removed voice information.

The voice separator 230 separates the voice information recognized by the voice recognition unit 220 into a plurality of spaces and characterizes them. For example, when the phrase recognized by the voice recognition unit 220 is "father enters the room ", the voice separation unit 230 separates the words into" father, "" Can be objectified.

The first photographing unit 240 photographs an image of a disabled person requesting a sign language automatic translation service, and may be at least one web cam camera. The first photographing unit 240 may be capable of communicating with the manager of the sign language service providing server 10 or the sign language support server by watching the video on a one-to-one basis. According to the present invention, the disabled person can make a situation explanation or inquiry inquiry through the first photographing unit 240, so that the disabled person can actively participate in the conversation.

The image information transmitting unit 245 transmits a question image of the disabled person to the image captured by the first image capturing unit 240 and the output image to the sign language service providing server 10.

The first information input unit 250 is a device for inputting registration information of a disabled person and inputting question data desired by a disabled person. According to the present invention, the user may input question data desired by the disabled person through the first information input unit 250, and receive the sign language video data and the caption data converted by the sign language support terminal 30 according to the question data.

The data sharing request unit 251 requests the sign language service providing server 10 to share desired sign language video information. The data sharing request unit 251 may transmit desired sign language video data and caption data sharing request information among the sign language video information for receiving the sign language automatic translation service to the sign language service providing server 10.

The encrypted data analysis unit 252 analyzes the encrypted data transmitted from the sign language service providing server 10. That is, the encryption data analysis unit 252 analyzes the data header by receiving the encryption key using the chip serial number and the encrypted data using the data requested by the disabled person by the sign language service providing server 10.

The decryption command generation unit 253 generates a decryption command according to the analysis result of the encrypted data transmitted from the sign language service providing server 10. That is, the decryption command generation unit 253 generates a decryption command when the encrypted data analysis unit 252 determines that the data header is normal.

The serial number comparing unit 254 compares the encryption key included in the encrypted data transmitted from the sign language service providing server 10 with the encryption key of the disabled terminal 20. That is, the serial number comparison unit 254 compares the chip serial number in the encryption key included in the data encrypted by the sign language service providing server 10 with the chip serial number stored in the disabled terminal 20 Or not. The encryption key may be a chip serial number of a smart card provided in the terminal 20.

The data decryption unit 260 decrypts the encrypted data when the encryption key included in the encrypted data transmitted from the hydration service providing server 10 and the encryption key of the disabled terminal 20 coincide with each other.

The user authentication unit 270 is a device for authenticating a user based on registration information of a disabled person, and is a device for logging in or automatically logging in the sign language service providing server 10. [ To this end, the user authentication unit 270 may log in through the input of disabled person information (for example, an ID, a password, etc.).

The first display unit 280 is a device for displaying data transmitted and received by the first communication unit 210. The first display unit 280 may be a liquid crystal display (LCD), a liquid crystal display (LED), an organic light emitting diode (OLED) OLED) method can be used. In addition, the first display unit 280 is capable of outputting the sign language video data and the caption data. In particular, the first display unit 280 is designed to adjust the font size, the window size, and the like according to the preference of the handicapped. In addition, the first display unit 280 may be configured to be simple and color-sensitive to provide a visual effect as a whole, and the basic menu may be designed to be divided into icons.

The first control unit 290 is provided with an automatic sign language translation application and executes the automatic sign language translation application so that the respective components (i.e., the first communication unit 210, the speech recognition unit 220, the speech separation unit 230 A first information input unit 250, a data sharing request unit 251, an encrypted data analysis unit 252, a decryption command generation unit 253, The serial number comparison unit 254, the data decoding unit 260, the user authentication unit 270, and the first display unit 280).

The sign language support terminal 30 is a device for generating sign language video data and caption data for the word information requested by the disabled terminal 20 and providing it to the sign language service providing server 10. The sign language support terminal 30 may be a mobile communication terminal capable of executing a sign language automatic translation application, a mobile phone, a personal digital assistant (PDA), a smart phone, an International Mobile Telecommunication 2000 ) Terminal, a code division multiple access (CDMA) 2000 terminal, a universal mobile telecommunication service (UMTS), a notebook computer, a personal computer, and a portable multimedia player (PMP) It will be apparent that the present invention can also be applied to applications thereof.

The sign language support terminal 30 includes a second communication unit 310, a second image capturing unit 320, a second information input unit 330, a caption support unit 340, a digital editing unit 350, a sign language video upload unit 360 A second display unit 370, and a second control unit 380.

The second communication unit 310 is a device for transmitting and receiving data to and from the sign language service providing server 10. The second communication unit 310 connects a predetermined communication channel with the sign language service providing server 10 based on a protocol stack defined in the communication network, And transmits and receives audio data and the like using a communication protocol defined in a communication program provided in the providing server 10. [ In the present invention, wired technologies such as Ethernet, RS-232C, RS-485, TCP / IP, RS-232, and IEEE 1394 and IEEE 802.11 WLAN, UWB , A Wi-Fi method, a ZigBee method, a Bluetooth method, a 3G method, a 4G method, an LTE method, and the equivalent method.

The second photographing unit 320 is an apparatus for photographing a sign language image performed by the sign language support company by word information, and may be at least one web cam camera.

The second information input unit 330 is connected to the sign language service providing server 10 and inputs registration information and authentication information of the sign language support company. The second information input unit 330 receives education information on the sign language image (i.e., sign language video information, Etc.). The second information input unit 330 receives support and registration information from a sign language support company to assist the disabled person in the conversation activity.

The caption support unit 340 is a device for converting a caption data of a caption image into caption data, and converts the caption data into caption data for a sign language support person.

The digital editing unit 350 is a device for editing the sign language video and caption data to generate video information to be provided to the sign language service providing server 10. The digital editing unit 350 can construct a digital editing system and edit video and audio information by the second photographing unit 320. [

The sign language video upload unit 360 uploads the sign language video information to the sign language service providing server 10.

The second display unit 370 is an apparatus for displaying data transmitted and received by the second communication unit 310, and may be an LCD, an LED, an OLED, or an AMOLED system.

The second control unit 380 is provided with an automatic sign language translation application and executes the sign language automatic translation application so that the respective components (i.e., the second communication unit 310, the second photographing unit 320, The caption support unit 330, the caption support unit 340, the digital editing unit 350, the sign language video upload unit 360, and the second display unit 370).

As described above, the present invention is not limited to the above-described embodiments, but can be applied to a system for providing automatic translation for deaf-mute persons according to the present invention. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

10: Sign language service providing server 20:
30: sign language support terminal 110: communication module
120: signal processing module 130: streaming providing module
140: Word information DB 145: Storage module
150: Security module 151: Sharing decision unit
152: Encryption information analysis unit 153: Encryption command generation unit
154: Encrypted data generation unit 160: Word information extraction module
170: Sign language video information extraction module 180:
190: control module 210: first communication section
220: voice recognition unit 230: voice separation unit
240: first photographing unit 245: image information transmitting unit
250: First information input unit 251: Data sharing request unit
252: Encrypted data analysis unit 253: Decryption instruction generation unit
254: Serial number comparing unit 260: Data decoding unit
270: user authentication unit 280: first display unit
290: first control unit 310: second communication unit
320: second photographing unit 330: second information input unit
340: Subtitle support unit 350: Digital editor
360: sign language video upload unit 370: second display unit
380: second control section

Claims (11)

A sign language service providing server for storing sign language image information for word information in advance and for extracting sign language image information matched with word information provided from the outside and providing the extracted sign language image information; And
A sign language automatic translation application is installed, the speech information inputted from the outside is recognized by executing the sign language automatic application, the speech information is converted into word information, and the speech information is provided to the sign language service providing server, And a disabled terminal for receiving and outputting the sign language video information for the word information.
The method according to claim 1,
Wherein the sign language video information includes sign language video data and caption data for the sign language video data.
The method according to claim 1,
Further comprising a sign language support terminal for generating sign language video data and caption data for the word information and providing the sign language video data and the caption data to the sign language service providing server.
The method of claim 3,
The sign language service providing server
A communication module for transmitting and receiving data between the handover terminal and the sign language support terminal;
A signal processing module for signal processing the sign language video data and the caption data uploaded by the sign language support terminal;
A streaming providing module for streaming the signal-processed mime-image data and the caption data to the disabled terminal;
A word information DB including word information recorded in a sign language standard dictionary;
A storage module for storing data transmitted and received by the communication module, the storage module storing registration information of the disabled person and a sign language support company, sign language video data for each word information uploaded by the sign language support terminal, and caption data;
A word information extraction module for extracting matched word information by comparing and analyzing word information provided from the disabled terminal and word information stored in the word information DB;
A mature image information extraction module for extracting matched mature image data and caption data by comparing and analyzing the extracted word information and mature image data and caption data for each word information stored in the storage module;
A disabled person management module for registering and managing disabled person information requesting the sign language video information, and providing a convenience function including a Paint and Web sharing function to the disabled person terminal; And
And a control module for providing a sign language automatic translation application to the disabled terminal and the sign language support terminal and controlling the operation of each configuration.
The method of claim 4,
The handover terminal
A first communication unit for transmitting and receiving data with the hydration service providing server;
A voice recognition unit for recognizing voice information inputted from outside;
A speech separator for separating the recognized speech information into spaces and characterizing the speech information;
A first photographing unit for photographing the image of the disabled person;
An image information transmission unit for transmitting a question image of the disabled person to the photographed image and the output image;
A first information input unit for inputting registration information of the disabled person and inputting question data desired by the disabled person;
A user authentication unit for authenticating a user based on registration information of the disabled person;
A first display unit for displaying data transmitted and received by the first communication unit; And
And a first control unit for controlling the operation of each of the components by executing the automatic sign language interpretation application, the automated sign language interpretation system including a sign language automatic translation application.
The method of claim 5,
Wherein the word information extraction module compares and searches a part matched with the word information stored in the word information DB for each unit objectized by the speech separation unit, And extracting matched word information from the extracted word information.
The method of claim 5,
Wherein the speech recognition unit removes speech in a specific frequency band from the externally inputted speech information and recognizes the removed speech information.
The method of claim 3,
The sign language support terminal
A second communication unit for transmitting and receiving data with the hydration service providing server;
A second photographing unit for photographing a sign language video performed by a sign language support company for each word information;
A second information input unit connected to the sign language service providing server for inputting registration information and authentication information of the sign language support company and inputting education information on the sign language video;
A subtitle support unit for converting subtitle description of the sign language video into subtitle data;
A digital editing unit for editing the sign language video and the caption data to generate the sign language video information;
A sign language video uploading unit for uploading the sign language video information to the sign language service providing server;
A second display unit for displaying data transmitted and received by the second communication unit; And
And a second control unit for controlling the operation of each of the structures by executing the automatic sign language interpretation application, in which the automatic sign language interpretation application is installed.
The method of claim 5,
The sign language service providing server
A security module for receiving the first encryption information from the disabled terminal, restoring the encryption key included in the first encryption information, encrypting the restored encryption key and the sign language video information requested for the disabled terminal, Further comprising: an automatic translation translation system for a hearing impaired person.
The method of claim 9,
The security module
A sharing determination unit determining whether to share the sign language video information requested from the disabled terminal;
An encrypted information analyzer for analyzing the first encryption information received from the disabled terminal;
An encryption command generation unit for generating an encryption command according to an analysis result of the first encryption information; And
And an encrypted data generation unit for encrypting the encryption key included in the first encryption information and the data requested to be shared from the disabled terminal to generate data,
Wherein the encrypted data includes reference data for generating a modified key.
The method of claim 10,
The handover terminal
A data sharing request unit for requesting the sign language service providing server to share desired sign language video information;
An encrypted data analysis unit for analyzing the encrypted data transmitted from the sign language service providing server;
A decryption command generation unit for generating a decryption command according to an analysis result of the encrypted data transmitted from the sign language service providing server;
A serial number comparing unit for comparing the encryption key included in the encrypted data transmitted from the sign language service providing server and the encryption key of the disabled terminal; And
And a data decryption unit for decrypting the encrypted data when the encryption key included in the encrypted data transmitted from the sign language service providing server matches the encryption key of the disabled terminal, .

Images