KR102374599B1 - hand language animation producing method for hearing impairment people - Google Patents

Abstract

The present invention relates to a method for generating sign language video animation for the hearing impaired to provide clear and detailed expressions. According to the present invention, the method comprises: a word array generating step (S100) of generating, by a word array generation unit (100), a word array by converting an input sentence into a word array included in a sign language dictionary group through morphological analysis or preprocessing, or receiving a sentence converted into a sign language word order; an arrangement word sign language animation generation step (S200) of assigning, by an arrangement word sign language animation generation unit (200), a matching sign language animation image for each word belonging to the arrangement; and a sign language sentence animation generation step (S300) of generating, by a sign language sentence animation generation unit (300), a sign language animation image for the sentence by sequentially connecting (blending) the sign language animations for the arranged words.

Description

{ hand language animation producing method for hearing impairment people }

The present invention relates to a method and system for generating sign language animation for the hearing impaired.

Prior Art Laid-Open Patent Publication No. 10-2021-0026006 A sign language translation system and method for converting video audio into avatars and animations, in a sign language translation program,

(101) voice extraction unit configured by using number 201 to extract audio from the video, (102) text conversion unit configured by extracting features from the extracted voice so that the audio of the extracted video can be converted into text, and the converted text A (103) text refining unit composed of morphemes by refining the converted text to output accurate information, and a model in which the refined text is translated into sign language using deep learning technology to translate the refined text into sign language. (104) Sign language translation unit, (105) Sign language animation conversion unit, which consists of transforming sign language into animation by learning sign language data to a model so that the translated sign language can be generated into avatars and animations, and generated avatars and animations (106) output unit, which consists of placing the converted animation so as to output (202) voice unit composed of accurate data from the analyzed voice to be able to be analyzed; The (204)SCORE extractor, which consists of calculating the score of features extracted from voice data so that the SCORE can be calculated for (205) SCORE selection unit, (206) post-processing unit composed of combining elements with maximum scores by calculating a score so that the selected features can be combined, combining the elements to output the combined result Disclosed is a sign language translation system and method for converting the voice of an image into an avatar and animation, characterized in that it consists of a (207) output unit configured to output on the screen.

An object of the present invention is to provide a method and system for generating a word arrangement by pressing a sentence (text) pre-processed into Korean character input or sign language grammar, and using this to generate a sign language image animation for the hearing impaired.

The present invention requires the expression of more than 20,000 words (headwords) for precise conversion of sign language animation of Korean words, and by linking them with 5,000 levels of representative headwords, using the sign language animation image (pre-generated) of the connected representative headwords, words (headwords) An object of the present invention is to provide a method and system for generating sign language animations for the deaf who can express clearly, purify, and in detail the implementation process by going through the process of generating a sign language animation image of

In the method for generating sign language video animation for the hearing impaired of the present invention, the word arrangement generating unit 100 converts an input Korean sentence into a word arrangement included in a sign language dictionary group through morphological analysis or pre-processing, or in sign language word order. A word arrangement generating step (S100) of receiving the converted sentence and generating a word arrangement;

A sequenced words sign language animation generating step ( S200 ) of generating, by the sequenced words sign language animation generating unit 200 , a matching sign language animation image for each word belonging to the sequence; Sign language sentence animation generating unit 300 generates a sign language animation image for the sentence by sequentially connecting (blending) sign language animations for the arranged words, a sign language sentence animation generating step (S300); characterized.

Here, in the sequence words sign animation generating step (S200), the representative lemma sign animation image loading unit 210 stores a pre-stored sign language animation image matching the representative lemma having a smaller number than all the words belonging to the sign language dictionary group library Loading (created by), representative headword sign language animation image loading step (S210), and the word connection unit 230a, using the loaded information of the library (second storage unit), a related representative for each arrangement word a linking step of associating with the headword;

The sequence word sign language animation image matching unit 250 matches the sign language animation image for the representative headword (word) when the classification of the word corresponds to the representative headword, and is connected when the classification of the given word corresponds to the isomorphism Matching the sign language action of the representative lemma to the sign language action of the corresponding word, and matching the sign language action of the corresponding word by sequentially combining the sign language action of at least two related representative headwords connected when the classification of a given word corresponds to a compound word, It is characterized in that it consists of a sequence word sign language animation image matching step (S250).

According to the present invention, there is provided a method and system for generating a word arrangement by pressing a sentence (text) preprocessed with Korean character input or sign language grammar, and using this to generate a sign language image animation for the hearing impaired.

In addition, in the case of the present invention, more than 20,000 words (headwords) expressions are required for precise sign language animation conversion of Korean words, and sign language animation images (pre-generated) of the connected representative headwords are used by linking them with 5,000 levels of representative headwords. This is to provide a method and system for generating sign language animations for the deaf and hard of hearing, in which the implementation process is clear, purified, and detailed expression is possible by going through the process of generating a sign language animation image of a word (headword).

1 (a, b) is a complete and detailed configuration diagram of the sign image animation generation system for the hearing impaired of the present invention.
2 is an overall configuration diagram of a sign image animation generation method and system for the hearing impaired according to the present invention.
3 is an overall flowchart of a sign image animation generation method for the hearing impaired of the present invention.
4 is a detailed flowchart of the sign language animation generating step 200 of arranged words in the method of generating a sign language animation for the hearing impaired according to the present invention.
5 is a conceptual explanatory diagram of a word classification and connection step during the sign animation generation step of arranged words according to the present invention.
6 is a detailed explanatory diagram of the sequence words sign animation generation step of the present invention.
7 is a conceptual explanatory diagram of a sign image animation generation method and system for deaf people of the present invention, and word arrangement.

Hereinafter, a method and system for generating sign language animation for the hearing impaired according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. 1 (a, b) is an overall and detailed configuration diagram of a sign language animation generating system for the hearing impaired of the present invention, FIG. 2 is an overall configuration diagram of a sign language animation generating method and system for the hearing impaired of the present invention, FIG. 3 is The overall flowchart of the sign image animation generation method for the hearing impaired of the present invention, FIG. 4 is a detailed flowchart of the sign language animation generation step 200 of arranged words among the sign image animation generation method for the hearing impaired of the present invention, and FIG. 5 is the present invention It is a conceptual explanatory diagram of the word classification and connection step in the sign language animation generation step of arranged words, and FIG. 6 is a detailed explanatory diagram of the sign language animation generation step of the sequence words of the present invention.

As shown in FIGS. 1 to 4 , the sign image animation generation method for the hearing impaired of the present invention includes a word arrangement generation step (S100), a sign language animation generation step of arranged words (S200), and a sign language sentence animation generation step ( S300) is included.

First, in the word arrangement generating step (S100), the word arrangement generating unit 100 converts the inputted Korean sentence into a word arrangement included in the sign language dictionary group through morpheme analysis or preprocessing, or inputs the converted sentence in sign language word order takes and creates an array of words.

As shown in FIG. 7 , in the sentence generated in the word arrangement generation step S100 , it is preferable that the Korean sentence is a pre-processed sign language sentence suitable for the expression grammar of the sign language. Korean sentences can be generated through a preprocessing process in which tosses are removed and word order is transformed to match the expression grammar of sign language. Alternatively, a word arrangement (text) may be generated by receiving a sentence converted from sign language to word order from the beginning.

The generation of Korean sentences or sign language sentences may be text input using a keyboard input interface means, or a voice file generated by voice input may be converted into text by voice recognition.

In FIG. 7, as an example, a Korean sentence "my child suddenly lost consciousness" is posted, and the corresponding sign language sentence "child suddenly loses consciousness" shows the word arrangement.

Next, in the sequenced words sign language animation generating step ( S200 ), the sequenced words sign language animation generating unit 200 provides a matching sign language animation image to each word belonging to the sequence. It is a step of matching sign language images for each word in the array. This will be described in detail again below.

Finally, in the sign language sentence animation generation step ( S300 ), the sign language sentence animation generating unit 300 sequentially connects (blends) sign language animations for the arranged words to generate a sign language animation image for the sentence.

As shown in FIG. 2 , the sign sentence animation generation step ( S300 ) finally generates an animation image for the sign language sentence through animation blending, rendering, and video capturing steps that sequentially connect sign language animations for arranged words. do.

As shown in FIGS. 2 to 4 and 5 , in the method for generating a sign language image animation for the hearing impaired according to an embodiment of the present invention, the sign language animation generating step (S200) of arranged words is a representative headword sign language animation It consists of an image loading step ( S210 ), a word classification and connection step ( S230 ) and an arrangement word sign language animation image matching step ( S250 ) linking with the relevant representative headwords.

First, in the step of loading the representative headword sign language animation image (S210), the representative headword sign language animation image loading unit 210 stores a pre-stored sign language animation image that matches the representative headword having a smaller number than all the words belonging to the sign language dictionary group. It is loaded (created by) from .

As shown in FIG. 2 , in an embodiment of the present invention, sign language for 5,300 representative lemmas for example in the first library (first storage unit, representative lemma and sign language image storage unit: 201) by the sign language management system An animation image is created in advance and stored. The representative headword sign animation image loading unit 210 loads the sign language animation image matching 5,300 representative headwords, respectively.

Next, in the word classification and connection step (S230), the word classification and connection unit 230, for each of the arranged words, a representative lemma, a homomorphic word having the same sign language operation as the representative lemma, and a combination of a plurality of representative lemmas It is classified into a classification as one of the composed words, and is linked with a related representative headword.

Or, in the word connection step (S230a), the word connection unit uses a compound word classification relation consisting of a combination of a representative lemma, a homomorphic word having the same sign language operation as the representative lemma, and a plurality of representative lemmas for each of the arranged words. Connect with the headword.

Here, the number of arranged words (or headwords and sign words in FIG. 5) is, for example, 23,279, which is 3 to 6 times more than the number of representative headwords. Arranged words are classified as compound words consisting of representative lemma, sign language (sign language) action isomorphic with specific representative lemma (meaning sign language action is isomorphic), and compound words consisting of a combination of two or more representative lemma or isomorphism. Together, they are stored in the second library (second storage unit, word classification and connection relation storage unit: 203).

As shown in FIG. 5 , in an embodiment of the present invention, the code PLGI00005 for “Garodo” is given, and it is classified and designated as a homologous word such as “Garasadae”, a representative entry having a specific code, so that the classification and connection relationship are stored. "Intercept" is given code PLET 00043 and is classified and linked as being synonymous with "collection, looting, deprivation, extortion, and taking away", and the relationship is stored.

When describing the compound word with reference to FIG. 5 , "horizontal and vertical" is a compound word of two words, PLET 00040 code is given, "horizontal" with PLCC 00008 code, and "vertical" with PLC 00543 code. The connection relation is stored in the second library (second storage unit, word classification and connection relation storage unit: 203).

In addition, "powdered soap" having code PLHL 00002 is a compound word of "powder" having code PLCC 00009 and "soap" having code PLHL00075, and this classification and connection relationship is a second library (second storage unit, word classification and connection). It is stored in the relation storage unit: 203).

Next, the arranged word sign language animation image matching unit 250 matches the sign language animation image for the representative lemma (word) when the classification of the word corresponds to the representative lemma, and connects when the classification of the given word corresponds to the isomorphism The sign language action of the relevant representative headword is matched with the sign language action of the corresponding word. The sequenced word sign language animation image matching unit 250 matches the sign language action of the corresponding word by sequentially combining the sign language actions of at least two related representative lemma connected when the classification of a given word corresponds to a compound word.

As shown in Fig. 6, according to the information loaded in the word classification and connection step (S230), it is classified and connected as a compound word of "thorn" and "tree", and "thorn" by the arranged word sign language animation image matching unit 250 An animation image of a word is generated by sequentially matching and synthesizing the sign language animation image for " and the sign language animation image for "tree".

Hereinafter, a method and system for generating sign language animation for the hearing impaired of the present invention will be described in detail with reference to the accompanying drawings. As shown in FIG. 1( a , b ), the sign language animation generation system for the hearing impaired includes a word arrangement generation unit 100 , an arrangement words sign language animation generation unit 200 , and a sign language sentence animation generation unit 300 . ) is composed of

The word arrangement generating unit 100 converts an input Korean sentence into a word arrangement included in a sign language dictionary group through morpheme analysis or pre-processing, or receives a sentence converted into a sign language word order and generates a word arrangement.

Arranged words The sign language animation generating unit 200 provides a matching sign language animation image to each word belonging to the arrangement.

The sign language sentence animation generating unit 300 generates a sign language animation image for the sentence by sequentially connecting (blending) sign language animations for the arranged words.

As shown in FIGS. 1 to 5 , the arranged words sign language animation generating unit 200 of the sign language animation generating system for the hearing impaired includes the sign language animation image loading unit 210 and the word classification and connection unit 230 and It is configured to include an arrangement word sign language animation image matching unit 250 .

The sign language animation image loading unit 210 loads (creates) a sign language animation image corresponding to a representative headword having a smaller number than all words belonging to the sign language dictionary group from the library.

The word classification and connection unit 230 categorizes each of the arranged words into one of a representative lemma, an isomorphic word having the same sign language operation as the representative lemma, and a compound word consisting of a combination of a plurality of representative lemmas, and connects with a related representative lemma ( match).

The sequenced word sign language animation image matching unit 250 matches the sign language animation image with respect to the representative headword (word) when the classification of the word corresponds to the representative headword. In addition, when the classification of a given word corresponds to a homomorphic word, the sign language action of the related representative headword is matched with the sign language action of the corresponding word. In addition, when the classification of a given word corresponds to a compound word, sign language actions of at least two related related representative headwords are sequentially combined to match the sign language action of the corresponding word.

As shown in FIG. 7 , the sign image animation generating method and system for the hearing impaired of the present invention adjusts the sign language image speed for adjusting the image play speed of the sign language sentence animation generated by the sign language sentence animation generating unit 300 . It is preferable to further include a part 400 .

Although the present invention has been described with reference to the above-mentioned preferred embodiments, the scope of the present invention is not limited to these embodiments, and the scope of the present invention is defined by the following claims and is equivalent to the present invention various modifications and variations pertaining to

The reference numerals described in the claims below are merely to aid the understanding of the invention and do not affect the interpretation of the scope of rights.

100: word array generator
200: Array words sign language animation generator
300: Sign language sentence animation generation unit
400: sign language video speed control unit
210: representative headword sign language animation video loading part
230: word classification and connection part
250: arrangement word sign language animation image matching unit

Claims (7)

delete The word arrangement generating unit 100 converts the input Korean sentence into a word arrangement included in the sign language dictionary group through morphological analysis or pre-processing, or receives the converted sentence in sign language word order and generates a word arrangement, a word arrangement generating step (S100);
A sequenced words sign language animation generating step (S200) of generating, by the sequenced words sign language animation generating unit 200, a matching sign language animation image for each word belonging to the sequence;
Sign language sentence animation generating unit 300 generates a sign language animation image for a sentence by sequentially connecting (blending) sign language animations for arranged words, a sign language sentence animation generating step (S300);

The sequence words sign animation generating step (S200),
The representative headword sign animation image loading unit 210 loads (creates) a pre-stored sign language animation image matching the representative headword having a smaller number than all the words belonging to the sign language dictionary group from the library, loading the sign language animation image a step (S210) and
a linking step, in which the word linking unit 230a uses the loaded information of the library (second storage unit) to link each array word with a related representative lemma;
The sequence word sign language animation image matching unit 250 matches the sign language animation image for the representative headword when the classification of the word corresponds to the representative headword,
If the classification of a given word corresponds to a homomorphic word, the sign language action of the related representative headword is matched with the sign language action of the corresponding word,
When the classification of a given word corresponds to a compound word, an arrangement word sign language animation image matching step (S250) of sequentially combining the sign language actions of at least two related representative headwords connected to match the sign language action of the corresponding word;
is composed of,

The library includes a representative headword and a sign language image storage unit 201 in which a representative headword and a sign language image corresponding to each of the representative headwords are stored;
It consists of a word classification and connection relation storage unit 203 in which classification and representative lemma connection relation for words classified into three types of representative lemma and isomorphism and compound word are stored,
The representative headword sign animation image loading unit 210 loads the representative headword and sign language image from the representative headword and sign language image storage unit 201,
The word classification and connection unit 230 loads word classification and connection information from the word classification and connection relationship storage unit 203, scans the corresponding word, and generates a representative headword connection relationship. How to create video animation.
delete The word arrangement generating unit 100 converts the input Korean sentence into a word arrangement included in the sign language dictionary group through morphological analysis or pre-processing, or receives the converted sentence in sign language word order and generates a word arrangement, a word arrangement generating step (S100);
A sequenced words sign language animation generating step (S200) of generating, by the sequenced words sign language animation generating unit 200, a matching sign language animation image for each word belonging to the sequence;
Sign language sentence animation generating unit 300 generates a sign language animation image for a sentence by sequentially connecting (blending) sign language animations for arranged words, a sign language sentence animation generating step (S300);

The sequence words sign animation generating step (S200),
The representative headword sign animation image loading unit 210 loads (creates) a pre-stored sign language animation image matching the representative headword having a smaller number than all the words belonging to the sign language dictionary group from the library, loading the sign language animation image a step (S210) and
The word classification and connection unit 230 uses the loaded information of the library (second storage unit), for each of the arranged words, a representative lemma, an isomorphic word having the same sign language operation as the representative lemma, and a plurality of representative lemmas A word classification and connection step (S230) of classifying one of the compound words consisting of a combination and linking it with a related representative headword;
The sequence word sign language animation image matching unit 250 matches the sign language animation image for the representative headword when the classification of the word corresponds to the representative headword,
If the classification of a given word corresponds to a homomorphic word, the sign language action of the related representative headword is matched with the sign language action of the corresponding word,
When the classification of a given word corresponds to a compound word, an arrangement word sign language animation image matching step (S250) of sequentially combining the sign language actions of at least two related representative headwords connected to match the sign language action of the corresponding word;
is composed of,

The library includes a representative headword and a sign language image storage unit 201 in which a representative headword and a sign language image corresponding to each of the representative headwords are stored;
It consists of a word classification and connection relation storage unit 203 in which classification and representative lemma connection relation for words classified into three types of representative lemma and isomorphism and compound word are stored,
The representative headword sign animation image loading unit 210 loads the representative headword and sign language image from the representative headword and sign language image storage unit 201,
Sign language for the hearing impaired, characterized in that the word classification and connection unit 230 loads the headword classification and connection relationship information from the word classification and connection relationship storage unit 203 and scans the corresponding word to generate a representative headword connection relationship. How to create video animation.
delete delete delete

Images