KR20200083338A - Hand gesture recognition apparatus and method using an artificial neural network learning - Google Patents

Abstract

Disclosed are a hand motion recognition method using artificial neural network learning and a device therefor. The hand motion recognition method using the artificial neural network learning comprises the following steps of: starting a learning mode according to the occurrence of an external signal or a specific noise; performing motion-based labeling on a detection signal acquired through a sensor part according to the starting of the learning mode; performing motion learning based on motion-based labeling; performing motion section recognition according to the result of motion learning; terminating the motion section recognition according to the occurrence of the external signal or the specific noise; and performing artificial neural network learning based on hand motion section recognition data according to the motion section recognition. Therefore, motions of hands of a robot can accurately and precisely be controlled according to the intention of a user.

Description

Hand gesture recognition method and apparatus using artificial neural network learning{HAND GESTURE RECOGNITION APPARATUS AND METHOD USING AN ARTIFICIAL NEURAL NETWORK LEARNING}

The present invention relates to a hand gesture recognition technology, and more particularly, to a hand gesture recognition method and apparatus using artificial neural network learning.

In a device for recognizing or assisting an operation by measuring a signal using a human body implantable electrode or a wearable electrode, the pattern of the signal is different for each individual, and there are different difficulties in correcting conditions in assisting the operation.

In order to correct such individual differences, a technology that learns a signal pattern using an artificial neural network and recognizes motions using the artificial neural network rather than a judgment based on a conventional program, and assists motions flexibly responds to individual diversity. It enables more accurate recognition or assistance.

However, in order to learn artificial neural networks for motion recognition or assistance, it is necessary to classify signals that occur during a specific period and convert or preprocess them appropriate for artificial neural network learning.

In particular, the robot prosthesis or the electric prosthesis is a hand gesture by operating the robot prosthesis according to the user's intention, unlike a prosthesis that simply mimics the shape of the hand, so it should be able to supplement the function of the human hand to some extent. do. However, the related research results are still insufficient, and a method for easily controlling the number of robots according to a user's intention is required.

The present invention is to meet the needs of the existing technology, the object of the present invention is to generate a signal by connecting a separate external device or generate a noise pattern using a specific motion to effectively handle the hand movement of a robot or electrician. An object of the present invention is to provide a method and apparatus for recognizing hand gestures using artificial neural network learning, which can be recognized and used as an input signal for artificial neural network learning.

Another object of the present invention is to recognize and effectively label the motion of an electric prosthesis, a robot prosthesis, or a similar device (a device worn on an arm) to effectively use the recognition signal for artificial neural network learning and thereby control the motion of the robot prosthesis. It is to provide a method and apparatus for hand gesture recognition that can be accurately and precisely controlled according to a user's intention.

Another object of the present invention is to provide a computer-readable recording medium in which a program for implementing a hand gesture recognition method using artificial neural network learning is described.

A hand gesture recognition method using artificial neural network learning according to an aspect of the present invention for solving the above technical problem includes: starting a learning mode according to an external signal generation or specific noise generation; Performing motion-based labeling on the detection signal acquired through the sensor unit according to the start of the learning mode; Performing motion learning based on the motion-based labeling; Performing an operation section recognition according to a result of the operation learning; Ending the recognition of the operation section according to the occurrence of an external signal or a specific noise; And performing artificial neural network learning based on the hand motion section recognition data according to the motion section recognition.

In one embodiment, the hand gesture recognition method, after the step of performing the artificial neural network learning, sensing acquired through the sensor unit based on the learning result of the artificial neural network learning based on the hand gesture section recognition data according to the input mode selection signal The method may further include executing an inference mode for hand gesture recognition of the signal.

In one embodiment, the hand gesture recognition method may further include generating a control signal for controlling the hand gesture of the electric number or robot number according to the reasoning result of the reasoning mode.

In one embodiment, the sensor unit includes an inertial sensor and an electromyography sensor, the inertial sensor is installed on the back of the hand or the number of robots, and the electromyography sensor is the number of motors or the number of robots. It is installed on a specific part of the arm, including the user's forearm and elbow, which is connected to.

A hand gesture recognition apparatus using artificial neural network learning according to another aspect of the present invention for solving the above technical problem, comprising: a trigger signal detection unit detecting an external signal occurrence or a specific noise occurrence for starting and ending a hand gesture section recognition; A labeling unit that performs motion-based labeling on the section detection signal input through the sensor unit; A learning unit performing motion learning based on the section detection signal that has been subjected to the motion-based labeling; A motion recognition unit performing motion recognition based on a section detection signal that has undergone the motion-based labeling and a learning result of the learning unit in a state of being switched from a learning mode to a normal mode according to a mode selection; And a signal generator for generating a control signal for controlling the number of electric motors or robots according to the result of the hand motion recognition of the motion recognition unit.

In one embodiment, the sensor unit includes an inertial sensor and an electromyography sensor, the inertial sensor is installed on the back of the hand or the number of robots, and the electromyography sensor is the number of motors or the number of robots. It can be installed on a specific part of the arm, including the user's forearm and elbow connected to it.

A hand gesture recognition apparatus according to another aspect of the present invention for solving the above technical problem includes a computer readable recording medium recording a program implementing a hand gesture recognition method using artificial neural network learning in any one of the above-described embodiments do.

In the case of using the hand gesture recognition method and apparatus using the above-described artificial neural network learning, a signal is generated by connecting a separate external device or a noise pattern is generated using a specific motion to effectively effect the hand gesture of the robot or electrician. It can be recognized and used as an input signal for learning the artificial neural network.

That is, the mode switching between the reset mode, the learning mode, and the normal mode may be performed through signal generation using an external device or short circuit or opening of the circuit. For example, it is possible to switch to a learning mode by causing a specific noise pattern to be learned. In addition, a part of the body can be made into a loop to generate a specific noise pattern. For example, it is possible to generate noise by grabbing a specific part of the right arm with the left hand and making a part of the body into a loop. Then, after switching to the learning mode, labeling may be performed with different motions, for example, labeling may be performed for each classification using scissors, rocks, and beams. After learning, the mode may be terminated again through a signal generated by an external device or a specific noise pattern.

In addition, according to the present invention, by recognizing and effectively labeling the motion of the electric number, the number of robots, or a similar device (a device worn on the arm), the recognition signal is effectively used for learning the artificial neural network and through this, the number of robots is operated It is possible to provide a method and apparatus for recognizing hand gestures that can accurately and precisely control according to a user's intention.

1 is a schematic block diagram for explaining the configuration of a wide range of a hand gesture recognition apparatus using artificial neural network learning according to an embodiment of the present invention.
FIG. 2 is a schematic block diagram for explaining a configuration of consultation that can be employed in the hand gesture recognition device of FIG. 1.
3 is a flowchart illustrating a mode selection process of the hand gesture recognition device of FIG. 2.
4 is a flowchart for explaining an artificial neural network learning process of the hand gesture recognition device of FIG. 2.
5 is a flowchart for explaining another embodiment of the artificial neural network learning process of the hand gesture recognition device of FIG. 2.
FIG. 6 is a block diagram showing a detailed configuration of a control unit employable in the hand gesture recognition device of FIG. 1.
7 is a schematic block diagram illustrating a configuration of a hand gesture recognition apparatus using artificial neural network learning according to another embodiment of the present invention.
8 is a view for explaining the configuration of the sensor unit of the hand gesture recognition device of FIG. 7.
FIG. 9 is a flowchart for explaining an artificial neural network learning process of the hand gesture recognition device of FIG. 7.
10 is a structural diagram of a convolutional neural network employable in the method and apparatus of the present embodiment.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are exemplified only for the purpose of explaining the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention It can be implemented in various forms and is not limited to the embodiments described herein.

Embodiments according to the concept of the present invention can be applied to various changes and can have various forms, so the embodiments will be illustrated in the drawings and described in detail herein. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosure forms, and includes all modifications, equivalents, or substitutes included in the spirit and scope of the present invention.

The terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprises” or “haves” are intended to indicate the presence of features, numbers, steps, operations, elements, parts, or combinations thereof described herein, one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, actions, components, parts or combinations thereof are not excluded in advance.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

1 is a schematic block diagram for explaining the configuration of a wide range of a hand gesture recognition apparatus using artificial neural network learning according to an embodiment of the present invention.

Referring to FIG. 1, the hand gesture recognition device 100 according to the present exemplary embodiment includes a sensor unit 10 and a control unit 20. The sensor unit 10 may include an inertial sensor unit including an inertial sensor. The inertial sensor is mounted on the electric prosthesis and can measure the acceleration according to the motion of the electric prosthesis. The number of motors may include the number of robots having a plurality of finger joints.

In addition, when the hand gesture recognition device 100 is mounted on the number of electric or robot hands, the hand gesture recognition device 100 may further include a driving unit 50 or an actuator that provides a force for the operation of the electric hand. have. In this case, the control unit 20 may include a prosthetic control unit 30 and control the operation of the driving unit 50 by generating a control signal from the prosthetic control unit 30. In addition, the hand gesture recognition apparatus 100 may further include a button unit 60 for separating and acquiring the detection signal of the sensor unit 10 as a section detection signal when learning or recognizing the hand gesture through artificial neural network learning. . The button unit 60 may be coupled to the hardware of the control unit 20 or may be installed in a motorized number.

As such, in this embodiment, the artificial neural network learning platform 200 may be included in the hand gesture recognition device 100 in a broad sense. In this case, the hand gesture recognition device may have a form in which the sensor part 10, the control part 20, the driving part 50, and the artificial neural network learning platform 200 are integrally mounted on the number of electric or robot hands. The artificial neural network learning platform 200 is coupled to the control unit 20 and includes a means for performing an artificial neural network learning function or a component for performing a function corresponding to the means. The artificial neural network learning platform 200 may include a database system.

The artificial neural network mounted on the aforementioned artificial neural network learning platform 200 is a statistical learning algorithm implemented in a form similar to a biological neural network for machine learning. That is, a node that mimics a neuron of a biological neural network constructs a network and, through learning, points to a model having a problem-solving ability by changing a weight indicating a binding strength of a synapse. Specifically, the artificial neural network implements a function, and there is a network consisting of nodes, neurons and synapses, and the neuron output is activated based on the sum of multiplication values of the weights of synapses connected to each neuron. (activation). The multiplication of these inputs and stored weights is implemented by a multiplier. The task of changing the weight according to the backpropagation-based learning rule to minimize the loss function of the final output through the artificial neural network is learning, and the process of obtaining the output for the input with the learned artificial neural network model It is reasoning. The shape of the output depends on the application and purpose of the system. For example, in the inference that an image is an input, an index of a label of what the image represents may be output. By adding special nodes to the artificial neural network, it is possible to improve the performance of signal classification or hand gesture recognition. The artificial neural network learning platform 200 may be implemented based on a central processing unit (CPU) or a graphics processing unit (GPU) to calculate parameters of a model and generate output.

The above-described control unit 20 may correspond to a microprocessor, arithmetic processing unit, and the like, and may include a central processing unit (CPU) or a core, and executes a program or a software module stored in a memory to perform this embodiment. It may be implemented to execute a hand gesture recognition method using artificial neural network learning.

FIG. 2 is a schematic block diagram for explaining a configuration of consultation that can be employed in the hand gesture recognition device of FIG. 1.

Referring to FIG. 2, the hand gesture recognition apparatus 40 according to the present embodiment labels inertial information by hand gestures from a sensor of a wearable sensor unit as a unique number and utilizes it for artificial neural network learning.

The hand gesture recognition device 40 according to the present embodiment includes an inertial sensor unit 12 and a control unit, and may be mounted on an electric hand. The control unit includes a state control unit 21, a signal recording unit 22 and a signal processing unit 23. The control unit is connected to the electric unit (see 50 in FIG. 1) through the built-in communication unit and is connected to the artificial neural network learning platform (see 200 in FIG. 1) to transmit and receive signals or data.

The inertial sensor unit 12 measures the movement of the hand or electric prosthesis in a three-dimensional space through one inertial sensor. To this end, the inertial sensor may be equipped with an accelerometer for measuring acceleration and a magnetometer for measuring angular velocity.

The state control unit 21 acquires the detection signal of the inertial sensor as a section detection signal according to the signal input from the button unit 60. That is, the state control unit 21 starts to measure or store the detection signal of the inertial sensor according to the first signal of the button unit (see 60 of FIG. 1), and measures the detection signal of the inertial sensor according to the second signal of the button unit. Or you can stop the save operation. To this end, the state control unit 21 may receive different signals from the button unit.

The button portion (see 60 in FIG. 1) may be configured of, for example, N (arbitrary natural number) labeling buttons and one hand gesture button. In that case, the N labeling buttons perform a function of inputting the number N to be labeled, and the other one hand gesture button may perform a function of inputting a signal for determining a start-complete (ON-OFF) time point of the hand gesture. have.

The signal recording unit 22 may store the signal measured by the sensor unit in the form of a buffer in a memory or a recording device.

The signal processing unit 23 may generate data for learning an artificial neural network by labeling a signal stored in a buffer form with a unique number. For example, data for learning an artificial neural network may be used to learn to match a preset labeling number for a specific hand gesture with a series of section detection signals or a group of section detection signals corresponding thereto.

3 is a flowchart illustrating a mode selection process of the hand gesture recognition device of FIG. 2.

Referring to FIG. 3, when the hand gesture recognition apparatus according to the present embodiment starts operation by power input or the like, the communication speed setting check or the communication speed control is performed so that the communication speed for hand gesture recognition is 40 Hz or more (S31).

Next, the signal measured by the sensor unit can be read through the sensor reading and stored in a buffer (S32).

Next, when the operation mode is selected as a specific mode or when the operation mode is selected as a specific mode (S33), the process proceeds to the corresponding mode. The operation mode may include a reset mode (S34), a normal mode (S35), and a learning mode (S36). The reset mode S34 may include deleting all data stored in the memory or buffer by initializing the operation mode. The normal mode S35 may include an operation of inferring a hand gesture corresponding to the sensing signal based on the section sensing signal of the sensor unit and the learning result in the learning mode. The normal mode S35 may be referred to as a normal mode or a hand gesture recognition mode. In addition, the learning mode (S36) deduces a series of section detection signals or section detection signals of a predetermined group or dataset through an artificial neural network based on a section detection signal of the sensor unit and a specific labeling signal of a specific motion of the button unit 60. And learn.

Next, the detection signal of the sensor unit, the recognition result, the learning result, the mode status, etc. may be transmitted to the control unit through Bluetooth or serial communication (S37).

Next, the index and the mode state can be managed by increasing the index (S38 ). It is possible to acquire a certain number of section detection signals in a specific mode by increasing the number of indexes by a predetermined number (for example, one by one), and to change to another preset mode when a preset number of times has elapsed.

4 is a flowchart for explaining an artificial neural network learning process of the hand gesture recognition device of FIG. 2.

Referring to FIG. 4, in the learning mode of the hand gesture recognition device of the present embodiment (S36), the control unit or the sensor unit interlocking with the control unit manages the state value to maintain the communication speed (S361).

Next, it is determined whether the hand gesture recognition device starts to operate at the time when the ON signal is input by pressing the hand gesture button (S352). On the other hand, when the button portion is omitted, the specific hand gesture may include a preset gesture, such as placing the thumb on the back of the hand, contacting the thumb with the index finger, or extending the entire finger completely.

As a result of determining whether to start the operation, if yes, label the unique number N for the unique hand action with the N (any natural number) hand action button (S363). Such labeling may be performed on the section detection signal measured later.

Next, when the labeling signal is set in the operation start state, hand gesture recognition may be started (S364). The signal measured from the inertial sensor is stored in a buffer (S365).

Next, a hand gesture section recognition operation is completed at the time of an OFF signal input by pressing a hand gesture button (S366). If the operation completion is not input for a predetermined time, the corresponding hand movement section recognition operation may be completed when a predetermined time elapses through the timer or the like.

Next, artificial neural network learning is performed based on a section detection signal according to hand gesture section recognition (S367). Artificial neural network learning may be used to generate a control signal for reliably controlling the motion of the electrician's finger through the labeled section detection signal.

Next, sensor information and status information are transmitted to the control unit. The transmission type may be Bluetooth or serial communication.

Meanwhile, the sensing signal measured through the sensor unit under the managed state value may be stored in the buffer and stored in the storage unit as raw data even when the labeling operation or the artificial neural network learning process is not performed (S362, S362, S365, S366, S368). In this case, the raw data may correspond to one form of sensor measurement results.

5 is a flowchart for explaining another embodiment of the artificial neural network learning process of the hand gesture recognition device of FIG. 2.

Referring to FIG. 5, in the hand gesture recognition method using artificial neural network learning according to the present embodiment, an external signal generation S362a or a specific noise generation S362b is detected. Then, the learning mode is started according to the occurrence of an external signal or a specific noise (S362c).

Next, motion-based labeling is performed according to the start of the learning mode (S363a).

Next, motion learning is performed based on motion-based labeling (S364a).

Next, another external signal generation (S366a) or another specific noise generation (S366b) is detected. Then, the learning mode is terminated according to another external signal generation or specific noise generation (S366c).

In addition, the hand gesture recognition method may execute a reasoning mode after ending the learning mode (S367). Here, the reasoning mode may be performed based on the dataset of the labeling section detection signal segmented according to the occurrence of an external signal or specific noise in the artificial neural network learning platform 200.

FIG. 6 is a block diagram showing a detailed configuration of a control unit employable in the hand gesture recognition device of FIG. 1.

Referring to FIG. 6, the control unit 20 of the hand gesture recognition device according to the present embodiment includes a mode switching unit 24, a trigger signal detection unit 25, a labeling unit 26, a learning unit 27, and a motion recognition unit 28 and a signal generating unit 29.

The mode switching unit 24 sets or resets a reset mode, a normal mode and a learning mode, or manages or controls an operation that is automatically switched from a specific operation mode to another operation mode according to a preset condition.

The trigger signal detection unit 25 manages to start or end a hand gesture recognition operation or a labeling operation according to an external signal, specific noise, or the like. The trigger signal detection unit 25 may operate according to the signal input of the button unit. Further, the trigger signal detection unit 25 may function to start or end a preset operation according to a specific movement or motion in a state where the button unit is omitted. The trigger signal detection unit 25 is a micro electro mechanical system (MEMS) inertial sensor, and can detect a motion formed by acceleration, vibration, shock, tilt, rotation, or a combination thereof as a trigger signal.

The labeling unit 26 may set preset specific labeling according to an input signal of a specific button or a preset hand or arm movement or motion among the button units. Labeling may be hand movement or finger shape labeling. For example, the labeling may include a series of labeling numbers for the shape of the fingers representing the scissors, rocks and beams. In addition, the labeling may additionally include a series of labeling numbers for the form of holding the object with the thumb and index finger, and the form of holding the object with the thumb and the remaining four fingers.

The learning unit 27 may include software and hardware for performing artificial neural network learning. The learning unit 27 infers the hand gesture recognition result based on the data set of the section detection signal labeled with a series of specific numbers or symbols through the labeling unit 26.

The motion recognition unit 28 recognizes a hand gesture for a specific section detection signal labeled based on the learning result of the learning unit 27.

The signal generation unit 29 generates a control signal corresponding to the hand gesture recognized by the motion recognition unit 28. The control signal is transmitted to the driving unit of the electrician and functions to move in the form of a finger corresponding to the recognized hand gesture. The signal generation unit 29 may correspond to at least a part of the functional unit or component of the prosthetic control unit 30.

7 is a schematic block diagram illustrating a configuration of a hand gesture recognition apparatus using artificial neural network learning according to another embodiment of the present invention. 8 is a view for explaining the configuration of the sensor unit of the hand gesture recognition device of FIG. 7.

Referring to FIG. 7, the hand motion recognition device 40 according to this embodiment uniquely combines inertial information by hand motion obtained from the inertial sensor of the wearable sensor part and specific muscle signals of muscles around the forearm obtained from the EMG sensor. Label it with a number and use it for learning the artificial neural network.

The hand gesture recognition device 40 according to the present embodiment includes an inertial sensor unit 12, an electromyography sensor unit 14, and a control unit, and may be mounted on an electric hand. As shown in Figure 8, the inertial sensor 12a of the inertial sensor unit 12 may be installed on the back of the hand of the electric prosthesis 100a, and the EMG sensor 14a of the EMG sensor unit 14 is around the forearm. It can be installed on the surface of the electric prosthesis (100a) coupled to the cutting body to contact the muscles of.

The control unit includes a state control unit 21, a signal recording unit 22 and a signal processing unit 23. The control unit is connected to the electric unit (see 50 in FIG. 1) through the built-in communication unit and is connected to the artificial neural network learning platform (see 200 in FIG. 1) to transmit and receive signals or data through a wired or wireless network.

The hand gesture recognition device 40 of this embodiment is substantially the same as the hand gesture recognition device of the embodiment described above with reference to FIG. 2 except that the EMG sensor unit 14 is added and the related components are implemented to use it. Therefore, redundant components and detailed descriptions thereof will be omitted.

The EMG sensor unit 14 is installed to measure the EMG signal, which is a minute biosignal generated from the remaining muscles of the cut of the body where the electric prosthesis is mounted. The EMG sensor unit 14 may include an EMG sensor mounted on an electric prosthesis, or may include an EMG sensor worn on a cutting part of the body in the form of a separate sensor assembly.

According to the present embodiment, the sensing signal of the inertial sensor and the EMG signal of the EMG sensor are acquired as the section sensing signal according to the signal input from the button unit through the state control unit 21, and the acquired section sensing signal is stored in the memory or storage device. In the buffer form, the section detection signal stored in the buffer form can be labeled with a unique number to generate a dataset for artificial neural network learning. The generated data set includes a labeled section detection signal including an inertial signal and an EMG signal, and is input to an artificial neural network and used for learning for hand gesture recognition.

FIG. 9 is a flowchart for explaining an artificial neural network learning process of the hand gesture recognition device of FIG. 7.

Referring to FIG. 9, in the hand gesture recognition apparatus according to the present embodiment, in a state in which the learning mode is started (S90), the controller measures an inertial signal according to a predetermined inertial force or inertial force through an inertial sensor (S91). If the measured inertia signal has an acceleration or angular velocity above a threshold value, the control unit recognizes the trigger signal corresponding to the manipulation of the hand gesture button or the specific hand gesture motion. If the measured inertia signal does not have an acceleration or angular velocity above a threshold, the control unit may maintain a previous mode (eg, a normal mode) state or a standby mode in a learning mode.

In response to the recognition of the trigger signal, the controller starts hand gesture recognition (S93), accordingly measures the EMG for hand gesture recognition (S94), and measures the inertial force for arm motion recognition (S95).

During the measurement operation of the sensor signal for hand gesture recognition, the control unit measures the inertial signal according to the predetermined inertia force or inertia force through the inertial sensor, and determines whether the measured inertia force has an acceleration or angular velocity above a threshold (S96).

As a result of the determination, if the measured inertial force has an acceleration or angular velocity above a threshold, the current hand gesture recognition task is terminated (S97), and a hand gesture number is assigned to the section detection signal measured for a certain time (S98). The labeled section detection signal may be input to artificial neural network learning and used to infer hand motion.

On the other hand, as a result of the determination in step S96, if the measured inertial force does not have an acceleration or angular velocity above a threshold value, the EMG measurement and the inertia measurement process may be continuously performed.

In addition, the determination step S96 may be replaced with a preset time by a timer or the like, or may be replaced by a number of repetitions of a unit interval time preset by a counter or the like.

10 is a structural diagram of a convolutional neural network employable in the method and apparatus of the present embodiment.

The hand gesture recognition apparatus for learning an artificial neural network according to the present embodiment may use a deep learning architecture as an artificial neural network. The deep learning architecture can be implemented to learn motion detection signals according to motion-based labeling by using a convolutional neural network (CNN), deconvolution, skip connection, etc. on the input signal. have.

As an example, the deep learning architecture may have a form including a convolutional network, a deconvolutional network, and a shortcut. As shown in FIG. 10, the deep learning architecture stacks a 3x3 color convolution layer and an activation layer (ReLU) and extracts a 2x2 size filter to extract local characteristics of the input data (X). (stride) is applied to 1 to perform the operation of the convolution block that is connected to the next lower depth level 4 times, and then a 2x2 size deconvolution layer and activation layer (ReLU) are applied. After connecting to the next higher depth level, the operation of the inverse convolution block, which stacks a 3x3 size color convolution layer and an activation layer, is repeated 4 times, where each of the convolution networks including the operation of the convolution block of each level is performed. The convolution operation may be performed by attaching the convolution result of the corresponding level of the inverse convolution network of the same level to the detection signal of the convolution block of the level (copy and contatenate) and performing convolution operations in each block.

The convolution block in the convolution network and the deconvolution network may be implemented by a combination of conv-ReLU-conv layers. Further, the output of the deep learning architecture may be obtained through a classifier connected to a convolutional network or a deconvolutional network, but is not limited thereto. The classifier may be used to extract a local feature from the section detection signal using a fully connectivity network (FCN) technique.

Further, the deep learning architecture may be implemented to additionally use an insulation module or a multi filter pathway in a convolution block depending on the implementation. Different filters in the inception module or multi-filter path may include 1x1 filters.

For reference, in a deep learning architecture, when an input signal, that is, an input 32 has a width 32, a height 32, and a predetermined number N of labeling channels, the size of the input signal X may be [32x32xN]. In a deep learning architecture convolutional neural network (CNN), a convolutional (CONV) layer is connected to a portion of an input signal and can be designed to calculate a dot product of the connected region and its weight. .

Here, the ReLU (rectified linear unit) layer is an activation function applied to each element, such as max(0,x). The ReLU layer does not change the size of the volume. The POOLING layer may output a reduced volume by performing downsampling or subsampling on a dimension represented by (horizontal, vertical).

Then, the fully-connected (FC) layer may calculate class scores and output a volume having a size of [1x1x10], for example. In this case, 10 numbers correspond to class scores for 10 categories. The pre-connection layer is connected to all elements of the previous volume. There, some layers may have parameters, while some layers may not. CONV/FC layers may include weight and bias as an activation function, not just input volume. Meanwhile, the ReLU/POOLING layers are fixed functions, and the parameters of the CONV/FC layer can be learned with a gradient descent so that the class score for each input signal is the same as the label of the corresponding input signal.

The artificial neural network employable in the above-described hand gesture recognition device is a separate pre-processor, a recognizer, a learner, or an independent functional unit or a combination thereof, which has a form connected to the classification system, or is partially mounted and connected to each constituent part. It may be provided. The artificial neural network may be designed to include machine learning or deep learning. Deep learning may include a convolutional neural network. Such an artificial neural network or an artificial intelligence system including the same may be referred to as a hand gesture recognition device including a learning means.

In addition, although all the components of the above-described hand gesture recognition device may be embodied as one independent hardware, some or all of the components are selectively combined to partially or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having a program module for performing the. The codes and code segments constituting the computer program may be easily inferred by those skilled in the art of the present invention. Such a computer program is stored in a computer readable storage medium (Computer Readable Media) and read and executed by a computer, thereby implementing an embodiment of the present invention.

Although the present invention has been mainly described with reference to the embodiments with reference to the drawings, it is only an example, and it is obvious that various modifications and equivalent other embodiments are possible from those skilled in the art. Therefore, the true technical protection scope of the present invention should be defined by the technical spirit of the appended claims.

Claims (6)

Starting a learning mode according to an external signal generation or specific noise generation;
Performing motion-based labeling on the detection signal acquired through the sensor unit according to the start of the learning mode;
Performing motion learning based on the motion-based labeling;
Performing an operation section recognition according to a result of the operation learning;
Ending the recognition of the operation section according to the occurrence of an external signal or a specific noise; And
And performing artificial neural network learning based on the hand motion section recognition data according to the motion section recognition. The method according to claim 1,
After the step of performing the artificial neural network learning, an inference mode for recognizing the hand gesture for the detection signal obtained through the sensor unit is executed based on the learning result of the artificial neural network learning based on the hand gesture section recognition data according to the input mode selection signal. The gesture recognition method further comprising the step of. The method according to claim 2,
And generating a control signal for controlling the hand motion of the electric hand or the robot hand according to the result of the reasoning of the reasoning mode. The method according to claim 3,
The sensor unit includes an inertial sensor and an EMG sensor, wherein the inertial sensor is installed on the back of the hand of the motorized or robotic robot, and the EMG sensor is connected to the motorized or robotic user. A method of hand gesture recognition that is installed on a specific part of the arm, including the forearm and elbow. A hand gesture recognition device using artificial neural network learning,
A trigger signal detection unit that detects external signal generation or specific noise generation to start and end the hand gesture section recognition;
A labeling unit that performs motion-based labeling on the section detection signal input through the sensor unit;
A learning unit performing motion learning based on the section detection signal that has been subjected to the motion-based labeling;
A motion recognition unit performing motion recognition based on a section detection signal that has undergone the motion-based labeling and a learning result of the learning unit in a state of being switched from a learning mode to a normal mode according to a mode selection; And
A hand motion recognition device including a signal generating unit for generating a control signal for controlling the number of electric motors or robots according to the hand motion recognition result of the motion recognition unit. The method according to claim 5,
The sensor unit includes an inertial sensor and an EMG sensor, wherein the inertial sensor is installed on the back of the hand of the motorized or robotic robot, and the EMG sensor is connected to the motorized or robotic user. A hand gesture recognition device installed on a specific part of the arm, including the forearm and elbow.

Images