KR102543936B1 - Apparatus and Method for Recognizing Sign Language - Google Patents

Abstract

The present invention relates to a language recognition device using sign language, and relates to a user interface controller (UI) that supports a user's input, etc., and a camera control that can capture the shape and movement direction of a signer's finger. Control) unit and a coordinate model (Coordinate Hands ML) extractor capable of recognizing the coordinates of the captured receiver's fingers, and the coordinate information recognized and stored through the first layer unit and the coordinate model (Coordinate Hands ML) extractor. and a shape model extraction unit (Shape Hands ML) that extracts the shape model of the finger and sign language recognition (Sign It relates to a sign language recognition device and method comprising a second layer unit including a language recognition unit, which extracts the shape through the location and coordinates of the joints of the fingers and continuously learns the shape to accumulate hand shape data; At the same time, the movement direction of the hand is also recognized, mapped to the hand shape data, and used without additional gloves or restrictive conditions, thereby dramatically increasing the number of recognizable sign languages and learning There is an effect of providing a more accurate and convenient sign language recognition device and method by continuously updating the data with the latest data.

Description

Sign language recognition device and method {Apparatus and Method for Recognizing Sign Language}

The present invention relates to an apparatus and method for recognizing a language using sign language, and relates to a user interface controller (UI) that supports a user's input, etc., and controls a camera capable of capturing the shape and movement direction of a sign language speaker's finger. (Camera Control) unit and a coordinate model (Coordinate Hands ML) extraction unit capable of recognizing the coordinates of the captured receiver's fingers, which are recognized and stored through the first layer unit and the Coordinate Hands ML extraction unit. A shape model extractor (Shape Hands ML) that receives coordinate information and extracts a shape model of the finger and sign language recognition that obtains a hand shape ID (ID) learned through continuous learning of the extracted shape model A sign language recognition device and method including a second layer unit including a sign language recognition unit.

Sign language is a method of conveying intentions expressed through gestures or hand gestures for people who have difficulty expressing their language verbally, such as the hearing impaired and speech impaired. It is done by integrating As such, sign language is a very difficult language for the general public to learn because it requires considerable effort to learn how to use sign language because language is expressed with fingers or gestures. situation you are experiencing. Therefore, only certain people, such as family members of the disabled or some social workers, learn sign language and play the role of sign language interpreters to communicate with hearing and speech impaired people. However, currently, there are not enough sign language interpreters, so people with disabilities face many difficulties. In order to solve this problem, sign language recognition recognizes and translates the sign language motions of the hearing and speech impaired people and provides the translated sign language motions to the general public so that the hearing and speech impaired people and the general public can easily communicate through sign language. The development of the device is urgently needed. As one of these developments, Korean Patent Publication No. 10-2003-0049265 suggests a hydration glove. The sign language glove is equipped with various sensors, and when a sign language sign language is worn by a sign language glove, a method of recognizing the sign language using the sensor mounted on the sign language glove is proposed. Since the above method is a method of wearing gloves, a method of recognizing sign language by recognizing a bending motion of a finger using a sensor attached to each finger and recognizing a spatial movement of a finger using a sensor attached to a hand, or However, recognizing the movement of a finger in space using only a sensor attached to the back of the hand may have a disadvantage in that practical measurement is difficult due to errors in the self-measurement sensor itself. As another solution, Korean Patent Publication No. 10-2015-0044084 also suggests a sign language glove, and in order to increase the accuracy of sensing by adding recognition information by additionally recognizing finger bending information and direction information as well as finger opening information However, it is judged that the problem of error has not been fundamentally solved because it is only a matter of adding only one recognition information along with the inconvenience of wearing sign language gloves. In order to solve the inconvenience of wearing gloves, Korean Registered Patent No. 10-1958201 recognizes only the movement of one hand without wearing sign language gloves, recognizes one sign language action, and combines the recognized sign language actions based on a specific critical time. has been registered by providing a device for interpreting the user's intention, but this invention provides a limited algorithm for recognizing one hand, and the present invention has a different purpose and configuration from the proposed method, There is no description about the technical characteristics of the device according to the present invention through the form and direction of movement and the construction of the database through learning, so there is a clear difference between the present invention and the purpose, configuration and effect.

The present invention was devised to solve the above problems, a coordinate model extractor that recognizes the shape of a finger joint through joint coordinates, and a shape model that derives the shape of a finger by receiving the recognized joint coordinates ( A shape model extractor, a hand shape ID (ID) extractor that acquires a hand shape ID (ID) by learning the recognized and extracted model, and a hand shape ID (ID) that recognizes the movement of the hand shape ID (ID) ID) It is an object of the present invention to provide a sign language recognition device including a movement line recognition unit and a sign language recognition method using the sign language recognition device. Another object of the present invention is to provide a sign language dictionary by improving accuracy through learning using such mapped data and continuously updating the learned data.

In order to achieve the above object, a sign language recognition device according to a preferred embodiment of the present invention includes a coordinate model extractor that recognizes the shape of a finger joint through joint coordinates, and a finger shape that receives the recognized joint coordinates. A shape model extractor that derives, a hand shape ID extractor that learns the recognized and extracted model to obtain a hand shape ID, and recognizes the movement of the hand shape ID. It is characterized in that it is configured to include a hand-shaped ID (ID) movement line recognition unit. The coordinate model extraction unit may be included in one layer as a part of the overall system configuration diagram, and the shape model extraction unit may also be included in another layer as a part of the overall system. The hand shape ID extraction unit may be configured by mapping and learning the data of the coordinate model extraction unit and the shape model extraction unit, and may be configured in one of the layers or another layer. The hand shape ID (ID) movement line recognition unit may also be configured in one of the layers or another layer, and the coordinate model extraction unit, the shape model extraction unit, and the hand shape ID (ID) extraction unit And the data extracted through the hand-shaped ID (ID) movement line recognition unit is translated and the contents are shown or displayed to the user in voice, text, or additional notification methods. In addition, the contents of the data extracted in this way are updated through learning, and the updated contents are continuously stored in a database to provide a sign language dictionary in which the latest contents that are learned each time are stored and utilized.

As described above, the sign language recognition device according to the present invention extracts the shape through the positions and coordinates of the joints of the fingers, continuously learns the shape, accumulates hand shape data, and recognizes the movement direction of the hand and recognizes the movement direction of the hand. By mapping and using shape data and enabling the use of one hand or both fingers without additional gloves or restrictive conditions, the number of recognizable sign language is dramatically increased, and by continuously updating data with the latest data through learning, more An accurate and simple sign language recognition device and method may be provided.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention together with the contents of the above-described invention, so the present invention is limited to those described in the drawings. It should not be construed as limiting.
1 is a diagram showing the system architecture of a sign language recognition apparatus according to the present invention.
2 is a diagram illustrating steps of obtaining a learned hand shape ID (ID) according to the present invention.
3 is a diagram showing an algorithm for acquiring the hand shape ID performed by a sign language recognition unit.
FIG. 4 is a diagram showing an additional algorithm performed in addition to the algorithm for acquiring the hand shape ID (ID) shown in FIG. 3 .
5 is a diagram illustrating the concept of a hand movement recognition algorithm.
FIG. 6 is a diagram showing the concept of the hand movement recognition algorithm disclosed in FIG. 5 in detail.
FIG. 7 is a diagram showing the configuration of a shape model extraction unit (Shape Hands ML) shown in FIG. 2 .
8 is a diagram illustrating a learning and recognition machine language data set (Maching Language Data Set) according to the present invention.
9 is a diagram illustrating an embodiment in which a sign language recognition device according to the present invention is implemented in a mobile terminal such as a portable smart device.
10 is a diagram showing a detailed flowchart of a method of the sign language recognition apparatus according to the present invention.
11 is a diagram showing a series of processes for preventing loss of image data according to the present invention.

The above objects, features and advantages will be described later in detail with reference to the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs will be able to easily implement the technical spirit of the present invention. In describing the present invention, if it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

The terms used in the present invention have been selected from general terms that are currently widely used as much as possible while considering the functions in the present invention, but these may vary depending on the intention of a person skilled in the art or precedent, the emergence of new technologies, and the like. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, not simply the name of the term.

When it is said that a certain part "includes" a certain component throughout the specification, it means that it may further include other components without excluding other components unless otherwise stated.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments of the present invention exemplified below may be modified in many different forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to more completely explain the present invention to those skilled in the art.

1 is a diagram showing the system architecture of a sign language recognition device according to the present invention. The system architecture of the present device may be composed of a plurality of layers according to convenience. As shown in the drawing, the first layer is a view layer, and a user interface (UI) controller that supports user input, etc., and a camera that can capture the shape and movement direction of a callee's finger. Control) unit and a coordinate model (Coordinate Hands ML) extraction unit capable of recognizing the coordinates of the captured receiver's fingers. The view layer performs a function of capturing joint coordinates of the sign language of the sign language with a camera and recognizing the coordinates of the joint when the sign language is performed by the sign language. It is possible to receive an input through the camera image capture operation and perform a series of related operations accordingly. Coordinate information recognized through the Coordinate Hands ML extraction unit may be primarily stored as a machine language coordinate model (ML Model (Coordinate Model)). A core layer (Native (Core) Layer), which is a second layer connected to the view layer unit, is shown in the drawing, receives coordinate information recognized and stored through the coordinate model (Coordinate Hands ML) extraction unit, and extracts the shape model of the finger. The shape model extraction unit (Shape Hands ML) and the hand shape ID (ID) learned through continuous learning of the extracted shape model in conjunction with the shape model extraction unit (Shape Hands ML) It may be configured to include a sign language recognition unit to acquire. This series of steps is schematically illustrated in FIG. 2 . 2, as shown in FIG. 1, each joint coordinate of a finger or hand is recognized through a coordinate model (Coordinate Hands ML) extraction unit, and the recognized joint coordinates are converted into a shape model (Shape Model) extraction unit (Shape Hands ML). It shows the step of acquiring the learned hand shape ID through the process of transmitting and continuously learning the recognized shape model through the sign language recognition unit. Through this process, each hand shape ID (ID) calculates a final sign language word through real-time combination. For example, a first hand shape ID (ID) (ID_A) is obtained and a second hand shape ID (ID) is obtained. After acquisition, a step of acquiring one sign language word through combination thereof is shown. 3 is a diagram illustrating an algorithm for acquiring the hand shape ID performed by a sign language recognition unit. For example, image information of at least 8 frames or more is required for recognizing one hand shape, and this can be designated as one turn (hand shape recognition unit). The turn information is used as a hand shape recognition unit and may also be used as unit (life) information in which the hand shape is maintained and maintained. As an example, as shown in the drawing, a situation in which the first ID information (ID_1) lasts for 5 frames and the second ID information (ID_2) lasts for 3 frames within 8 frame information is referred to as the Sequence Hand ID List (Sequence Hand ID List). List), and as shown in the frequency map of the drawing, the above case can be recognized as a sign language word alone (self) as already defined in the frequency map. . Even if a slight error occurs in this series of image frame information through continuous learning, the recognition rate can be improved by minimizing the error through the accumulated learning information. FIG. 4 is an additional algorithm performed in addition to the hand shape ID acquisition algorithm shown in FIG. It improves recognition accuracy and provides a function to generate more sign language words. As shown in the drawing, positional information (relative coordinates) for each hand shape can be stored through a position map, which is disclosed in detail as a code of Rect_1..Rect_n in a rectangular shape. In addition, a LongTerm Map is provided, and through this, a function of determining whether or not to maintain hand shape information remaining in the current Sequence Hand ID List can be performed, and for example, one turn ( Every time a turn passes, a life that determines whether or not to be maintained may be increased or decreased. 5 is a diagram illustrating the concept of a hand movement recognition algorithm. The recognition of the hand movement is an algorithm designed to process words having the same hand shape ID but different meanings according to the moving direction, and may be included in the sign language recognition unit shown in FIG. 1. Although not shown in the drawings, it may be performed by configuring an independent additional recognition unit according to the designer's convenience. The hand movement recognition algorithm as described above processes the movement of each recognized hand shape ID, and this movement process may change the dynamic frame according to the performance of the device on which the algorithm is performed. . As shown in the drawing, for example, the sign language representing 'oppa (brother)' can be implemented by moving the hand shape meaning 'mountain' upward, and in order to mean 'mountain', which is the original meaning, it can be implemented in the form of 6 It can be displayed by maintaining the frame. Also, in order to mean 'younger brother', it can be implemented by moving the hand shape meaning 'mountain' downward. FIG. 6 is a diagram showing the concept of the hand movement recognition algorithm disclosed in FIG. 5 in detail. By using the rectangular coordinate information (Rectangle(Rect_n)Postion Map) of the recognized hand shape ID (ID), the amount of change (Threshold) of the X coordinate between each movement line is converted into the right value and the left value. (Weight) can be measured, and the number of times (Weight) can be measured by converting the change (Threshold) of the Y coordinate between the moving lines into an Up value and a Down value. The number of required directions for each sign language word is determined using the four-direction counts (weights) of the Right value, Left value, Up value, and Down value collected through this process. Based on this, the direction of movement can be determined. In this measurement, the convenience of calculation may be secured by ignoring values below a certain amount of change (Threshold). For example, the movement line of the sign language word 'brother' can be assigned a value of 2 of the number of times (weight) of the up value, and the movement line of the sign language word 'younger brother' can be given the number of times (weight) of the down value. A value of 2 can be given.

A hand shape ID dictionary may be constructed for hand shape IDs extracted by recognizing and learning through the processes of FIGS. 1 to 6 . This dictionary is recognized as a heterogeneous database concept and continuously accumulates and accumulates learned hand shape IDs, so that the recognition rate of sign language words can be dramatically improved over time due to the accumulated database, and all aspects of daily life Words can also accumulate over time, dramatically improving the number of expressive words in sign language. In the hand shape ID dictionary, each hand shape may be mapped to a unique ID, and each sign language word may be mapped to a combination of hand shape IDs according to motions. . The form of the dictionary may include a plurality of hand shape IDs (IDs), sign language word numbers, sign language words, sign language explanations, etc. However, changes and additions are possible according to the purpose of the designer. The hand shape ID (ID) dictionary may also include a sign language word recognition list. For example, assuming that there are 209 currently recognizable sign language words, a sign language word can be formed by combining 228 hand shape IDs recognized, learned, and extracted in the present invention. For example, I/You/Meet A sign language word recognition list can be implemented by combining /welcome. In addition, this recognition list can be classified by category to further enhance its efficiency, and this recognition category can be divided into, for example, state/nature/season/time/age/family/emotion/personal/person. And the classification by category can be changed and added according to the purpose of the designer. FIG. 7 is a view showing a configuration diagram of a shape model extraction unit (Shape Hands ML) shown in FIG. 2 . The shape model extraction unit (Shape Hands ML) may use its own machine language (ML) engine according to the present invention, and may be composed of, for example, 18 layers, and this configuration is the machine It can be configured as shown in the figure as a layer configuration for maximizing machine language (ML) recognition and response performance. The configuration of these layers represents an embodiment of the present invention, and may be changed, added, or reduced according to the designer's purpose. In addition, an additional configuration for maximizing the recognition and response performance of the machine language (ML: Machine Language) can be added to a hyper parameter configuration, and this configuration is referred to as a filter (kernel) number (preferably 32 or 64 filters can be used), filter size (preferably 9 x 1 (Width x Height)), 1D convolution filter can be used has), Stride size (preferably 1 x 1 (Width x Height), Padding, Activation Function (preferably RELU can be used)) According to the purpose of the designer, it may be configured with additional functions.The shape model extraction unit (Shape Hands ML) may also define its data set (Date Set), The configuration may include an absolute coordinate configuration expressing the hand shape and a screen position coordinate configuration expressing the hand position. The absolute coordinate configuration is X for one coordinate. , Y, Z, which can be composed of a total of 21 points, and can have a value of (-1.0 to 1.0) In addition, the screen position coordinate configuration is one coordinate It can be composed of X, Y 2-axis values for , which can consist of a total of 21 points, and can have a value of (0.0 to 1.0) The absolute coordinates and screen position coordinates It is possible to configure a learning and recognition machine language data set (Maching Language Data Set), and in the present invention, both hands as well as one hand can be recognized and learned. For example, assuming that both hands are recognized, each left hand ) coordinate value + each right hand (Right Hand) coordinate value combination can be represented by a total of 210 configurations, which can be expressed as a formula as shown in FIG. 8. Here, L is the left hand, R is the right hand, A is the absolute value, and S is the relative value. For example, L_AX_1 can be expressed as the first absolute value of the X-axis of the left hand, and it can be seen that the absolute value of the value represents 0.097552724.

The sign language recognition device according to the present invention as described above can be implemented and installed in any device regardless of whether it is mobile or fixed, and due to recent industrial development and rapid development of information and communication technology, high-performance portable smart devices such as tablet PCs and smartphones As is spread, it can preferably be mounted on a portable smart device, and FIG. 9 shows an embodiment in which the system according to the present invention can be implemented in a mobile terminal such as the portable smart device. As shown in the figure, the sign language extracted and recognized by the plurality of layers according to the present invention can be expressed in the form of text, voice, and video by the sign language translator of the mobile device, and as an operating system of the mobile device, for example, Android. It shows that it is driven by the OS (Android Operation System). 10 is a diagram showing a detailed flowchart of a method of the sign language recognition apparatus according to the present invention. The sign language recognition apparatus according to the present invention first receives an external image from a camera in real time. The external image is a screen that captures a sign language operation performed by a sign language speaker and is captured and received by a camera in real time, and the received image is used by a Coordinate Hands ML extraction unit (hereinafter referred to as C. obtained in the form of each hand shape coordinate data (Coordinate). The obtained hand shape coordinate data is transmitted to a shape model extraction unit (hereinafter referred to as S.Model), and the received coordinate data is a hand that matches the hand shape learned through machine learning It is obtained as an ID (Hand ID). Data including the acquired hand ID and position information is stored in an internal storage, and sign language analysis is started when minimum data for determining sign language words are stored in the internal storage, The final sign language word is determined by figuring out the movement line through the frequency of hand ID appearance and location information in the internal storage. Through the above process, the life count of the data existing in the internal storage decreases, and the data whose life cycle is over are deleted from the internal storage. Such deletion can prevent an erroneous operation in determining a sign language word by deleting unused and neglected data from the internal storage. Through the above process, a final sign language word is obtained, and the sign language word can be displayed in various ways such as output on a screen or voice. These processes may be performed in the first layer and the second layer. Each hand shape coordinate data (Coordinate) received from the camera in real time and acquired through C.Model during the above series of processes can be immediately transmitted to S.Model, but can also be stored in the storage using an internal file. Since the image data received from the camera may be lost while waiting for the final response time of S.Model, a process of saving the hand shape coordinate data extracted from the image in the internal storage in real time can be performed to prevent such loss of image data. there is. At the same time, data input to the internal storage may be extracted using the output terminal of the internal storage, and data output from the internal storage may be transmitted to S.Model. While waiting for the response of S.Model, the hand shape coordinate data can be continuously saved in real time through the input terminal of the internal storage. It is configured to prevent image data from being lost through this series of processes. FIG. 11 is a diagram illustrating steps of inputting and outputting hand shape coordinate data to and from an internal storage to prevent loss of a series of image data.

As described above, although the preferred embodiments according to the present invention have been described above, the technical idea of the present invention is not limited thereto, and each component of the present invention is within the technical scope of the present invention to achieve the same purpose and effect. may be changed or amended in In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and in the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Various modified implementations are possible by those skilled in the art, and these modified implementations should not be individually understood from the technical spirit or perspective of the present invention.

Claims (10)

As a sign language recognition device,
a user interface controller (UI) unit supporting a user's input to the view layer;
a camera controller unit capable of capturing the shape and movement direction of a receiver's finger;
A first layer unit configured to include a Coordinate Hands ML (Coordinate Hands ML) extraction unit capable of recognizing the captured finger coordinates of the talker;
a shape model extractor (Shape Hands ML) extracting a shape model of the finger by receiving coordinate information recognized and stored through the coordinate model (Coordinate Hands ML) extractor;
A second configuration including a sign language recognition unit that obtains a hand shape ID learned through continuous learning of the extracted shape model in conjunction with the shape model extraction unit (Shape Hands ML). Including the layer part,
The shape model extraction unit (Shape Hands ML) may store position information (relative coordinates) for each hand shape through a position map in conjunction with the sign language recognition unit. In addition, by using the LongTerm Map, a function can be performed to determine whether or not to maintain the hand shape information remaining in the current Sequence Hand ID List. A sign language recognition device characterized in that a life that determines whether or not to be maintained can be increased or decreased each time.
According to claim 1,
The shape model extraction unit (Shape Hands ML) may be configured to include a hand shape recognition algorithm in conjunction with the sign language recognition unit, and the hand shape recognition algorithm recognizes one hand shape. It includes image information of at least 8 frames or more, and it can be designated as one turn (hand shape recognition unit), and the turn information is used as a hand shape recognition unit, and the hand shape is It can also be used as life information that is maintained and continued, and a situation in which the first ID information (ID_1) lasts for 5 frames and the second ID information (ID_2) lasts for 3 frame information within the 8 frame information is sequenced. An apparatus for recognizing sign language, which can be expressed as a hand ID list, and can recognize sign language words by using and mapping a previously prepared frequency map.
delete According to claim 1,
The Shape Model extraction unit (Shape Hands ML) works in conjunction with the Sign Language Recognition unit to use rectangular coordinate information (Rectangle (Rect_n) Position Map) of the recognized hand shape ID. The number of times (weight) can be measured by converting the change (Threshold) of the X coordinate between each movement line into the right (Right) value and the left (Left) value. It is possible to measure the number of times (weight) by converting it into a value and a down value, and the measured right value, left value, up value, and down value A sign language recognition device that can determine a movement direction based on the number of required directions for each sign language word by using the number of times (weights) in four directions.
According to claim 1,
The sign language recognition device is characterized in that it is configured to further include a hand shape ID (ID) dictionary constructed for the hand shape ID (ID) extracted by recognizing and learning, and each hand shape in the hand shape ID (ID) dictionary may be mapped to a unique ID, and each sign language word may be mapped to a combination of hand shape IDs according to motions, and the form of the dictionary may be mapped to a plurality of hand shape IDs (IDs). ), sign language word numbers, sign language words and sign language descriptions, etc.
According to claim 5,
The hand shape ID (ID) dictionary may also include a sign language word recognition list, and the sign language word recognition list may constitute a sign language word by combining the recognized, learned and extracted hand shape IDs. The sign language recognition device, characterized in that the recognition list can be classified and stored by category.
According to claim 1,
The shape model extraction unit (Shape Hands ML) is configured using its own machine language (ML) engine, and can be configured including 18 layers and a plurality of hyper parameters , and the 18 layers include 7 convolutional layers, 6 batch normalize layers, 2 max pooling layers, and 2 dropout layers and one Softmax layer, and the configuration of the plurality of hyper parameters includes the number of filters (sometimes referred to as kernels), filter size, and stride A sign language recognition device characterized by comprising a stride size, padding, and an activation function.
According to claim 1,
The shape model extraction unit (Shape Hands ML) includes a data set, and the data set consists of absolute coordinates representing the hand shape and screen position coordinates representing the position of the hand shape. It may be configured to include, The absolute (Absolutely) coordinate configuration may consist of three axis values of X, Y, Z for one coordinate, which may consist of a total of 21 points, (-1.0 ~ 1.0), and the relative (Screen Position) coordinate configuration can consist of two axes of X and Y for one coordinate, which can consist of a total of 21 points, (0.0 ~ 1.0) It may have a value of , and a learning and recognition machine language data set may be configured using the absolute coordinates and the screen position coordinates, and each left hand coordinate value + A sign language recognition device characterized in that it can be represented in a total of 210 configurations as a combination of each right hand coordinate value.
As a sign language recognition method,
Receiving an external image in real time through a camera;
converting the received image into each hand shape coordinate data (Coordinate) form through a coordinate model (Coordinate Hands ML) extractor (hereinafter referred to as C.Model);
Transmitting the converted hand shape coordinate data to a shape model extraction unit (hereinafter referred to as S.Model);
Converting the received coordinate data through the shape model extractor (hereinafter referred to as S.Model) into a hand ID matching the learned hand shape using machine learning, extracting the location information;
storing the converted hand ID and position information in an internal storage;
starting sign language analysis when minimum data for determining sign language words are stored in the internal storage;
Determining a final sign language word by determining a movement line through the frequency of occurrence of Hand ID and location information in the internal storage,
The shape model extraction unit (Shape Hands ML) links the extracted shape model with the sign language recognition unit that acquires the hand shape ID (ID) learned through continuous learning to provide a position map (Position Hands ML). Location information (relative coordinates) for each hand shape can be stored through the Map, and the hand shape information remaining in the current Sequence Hand ID List is maintained using the LongTerm Map. A sign language recognition method capable of performing a function of determining whether or not a sign language is present and increasing or decreasing a life that determines whether or not to be maintained whenever one turn passes.
According to claim 9,
deleting from the internal storage data whose life cycle is over due to a decrease in life count among data existing in the internal storage;
Each hand shape coordinate data (Coordinate) obtained through the C.Model is stored in an internal storage instead of immediately transmitted to the S.Model, and data input to the internal storage is extracted through an output terminal of the internal storage, Transmitting the extracted data to S.Model;
and continuously storing hand shape coordinate data in real time in the internal storage input terminal while waiting for a response from the S.Model.

Images