KR20220066535A - Method, Server and System for Recognizing Motion in Video - Google Patents

Abstract

According to the present invention, a method for recognizing a motion in a video is performed by a server and comprises: a step of acquiring and storing posture information for a plurality of postures performed by a first character, wherein the posture information includes the direction and size of the first character performing the corresponding postures and joint position values of joints; a step of identifying the direction and size of a second character included in an input video; a step of inputting the input video into a joint prediction model to acquire joint position values of joints of the second character; a step of identifying specific posture information corresponding to the identified joint position values of the second character in the stored posture information; a step of correcting the identified direction and size of the second character with the direction and size of the first character in the identified specific posture information; a step of inputting the joint position values of the first character and the joint position values of the second character in the specific posture information into a discrimination model while the direction and size of the second character is corrected to calculate first error values of the joint position values of the second character with respect to the joint position values of the first character; and a step of determining first accuracy of the discrimination model based on the calculated first error values. The first character and the second character can be the same character.

Description

Motion recognition method in video, server and system {Method, Server and System for Recognizing Motion in Video}

The present invention relates to a method for recognizing a motion of a person in an image, and more particularly, to a method, server and system for recognizing a motion of a person in an image for recognizing the motion of a person in an image in a state where the person does not attach a marker.

Recently, motion capture technology of 3D images, which could only be seen in Hollywood movies, is widely used by home appliances, games, medical care, advertisements, animations, virtual YouTubers, and the like. Most of these motion capture devices attach markers or sensors to the human body, and then use a camera to recognize and analyze the marker images or data from the sensors to measure the position and direction of each joint in the human body. According to the operation method of the marker or sensor used here, it can be divided into four types: an acoustic method, a mechanical method, a magnetic method, and an optical method.

Among them, in the case of the movie character Hulk, which is well known to the public, markers are attached to the main segments of the performer using an optical motion capture device, and signals from 3 to 16 video cameras are simultaneously read into the computer to locate the markers. It was created by analyzing the movement of each part of the body.

Since this conventional optical method focuses on realistically expressing the motion of a character, the amount of data is huge, and a high-end computer and many experts are required to post-process this vast amount of data. In addition, when using a high-speed frame or high-definition camera, the processing load of the computer is increased, and since video signals from a plurality of video cameras must be synchronously transmitted at the same time, an expensive synchronous camera is used, and a huge cost is required. A problem exists.

Therefore, there is a need for a technology that can reduce costs and easily apply motion capture so that companies of various sizes can use it.

Republic of Korea Patent Publication No. 10-2005-7014813

An object of the present invention for solving the above-described problems is to obtain joint position values of each joint of a person based on an image obtained by photographing the motion of the person in real time.

Another object of the present invention is to provide a method for determining the reliability of a specific model that has generated the new learning data.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In the method for recognizing motion in an image performed by a server according to the present invention for solving the above-mentioned problem, the step of acquiring and storing each posture information for a plurality of postures performed by a first person - the respective posture information includes the direction, size, and joint position values of each of the joints of the first person performing the corresponding posture-, recognizing the direction and size of the second person included in the input image, using the input image as a joint prediction model input to, obtaining joint position values of each of the joints of the second person, identifying specific posture information corresponding to the identified joint position values of the second person among the stored respective posture information, the grasping correcting the direction and size of the second person to the direction and size of the first person in the identified specific posture information; Inputting the joint position value and the joint position value of the second person to the discrimination model, calculating a first error value of the joint position value of the second person with respect to the joint position value of the first person, and the calculated first error and determining the first accuracy of the discrimination model based on a value, wherein the first person and the second person may be the same person.

The method may further include machine learning the joint prediction model by inputting the first error value and the second input image to the joint prediction model.

In addition, the method further includes: inputting the joint position value of the third person stored in advance and the joint position value of the fourth person calculated by learning based on the learned joint prediction model into the discrimination model to machine learning the discrimination model , and the third person and the fourth person may be the same person.

Here, the joint prediction model generates reference coordinates of a fixed point based on the position of the second person for each frame in the input image, and connects the frames to the second person based on the reference coordinates joint position values of the joints of the second person may be obtained according to the movement of .

In addition, the first error value is a structural error value when the joint distance is different from a preset reference distance according to each joint position value of the second person or a speed error when the joint movement speed is different from the preset reference speed It can contain values.

In addition, based on the first accuracy, it is determined whether the first accuracy is less than a preset threshold, until the first accuracy is less than the threshold, repeating machine learning the joint prediction model or the discrimination model It may include further steps.

In addition, in the server for performing motion recognition in an image according to the present invention for solving the above problems, a memory for storing information, each posture information for a plurality of postures performed by the first person is obtained and stored - Each of the posture information includes the direction, size, and joint position value of each of the joints of the first person performing the corresponding posture, grasps the direction and size of the second person included in the input image, and uses the input image Input to the joint prediction model, obtain joint position values of each of the joints of the second person, identify specific posture information corresponding to the identified joint position values of the second person among the stored posture information, and the The direction and size of the identified second person are corrected to the direction and size of the first person in the identified specific posture information, and in a state in which the direction and size of the second person are corrected, By inputting the joint position value and the joint position value of the second person into the discrimination model, a first error value of the joint position value of the second person with respect to the joint position value of the first person is calculated, and the calculated first error value and a controller for determining the first accuracy of the discrimination model based on , wherein the first person and the second person may be the same person.

In addition, the control unit, by inputting the first error value and the second input image to the joint prediction model, machine learning the joint prediction model, and the joint position value of the third person stored in advance and the learned joint prediction model The joint position value of the fourth person calculated based on learning is input to the discrimination model to machine-lear the discrimination model, and the third person and the fourth person may be the same person.

Here, the joint prediction model generates reference coordinates of a fixed point based on the position of the second person for each frame in the input image, and connects the frames to the second person based on the reference coordinates joint position values of the joints of the second person may be obtained according to the movement of .

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

According to the present invention as described above, the effect of predicting the movement of the joint of a person without attaching a marker by obtaining the joint position value of each of the joints of the person based on the image captured in real time of the movement of the person is obtained. have.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a diagram schematically showing a system for performing motion recognition in an image according to the present invention.
2 is a diagram schematically illustrating an operation of a server according to the present invention.
3 is a flowchart illustrating a process in which a server recognizes a motion in an image according to the present invention.
4 is an exemplary diagram illustrating a process of obtaining a joint position value in a joint prediction model according to the present invention.
5 is an exemplary diagram of an error value generated for a distance between joints predicted through a joint prediction model according to the present invention.
6 is an exemplary diagram of an error value generated for the movement speed of the joint predicted through the joint prediction model according to the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only those of ordinary skill in the art to which the present invention pertains, to complete the disclosure of the present invention. It is provided to fully understand the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, "comprises." and/or "comprising" does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Before the description, the meaning of the terms used in this specification will be briefly described. However, it should be noted that the descriptions of terms are for the purpose of helping the understanding of the present specification, and are not used in the meaning of limiting the technical spirit of the present invention unless explicitly described as limiting the present invention.

1 is a diagram schematically showing a system 1 for performing motion recognition in an image according to the present invention.

The system 1 may have the effect of predicting the movement of a person's joints without the person attaching a marker by acquiring joint position values of each of the person's joints based on an image captured in real time .

Referring to FIG. 1 , a system 1 may include a photographing device 10 for capturing an image and a server 20 for recognizing motion within the image. Here, the photographing apparatus 10 may be separately provided to transmit the photographed image to the server 20 , or may be provided in connection with the server 20 .

Hereinafter, the system 1 may include fewer or more components than the components shown in FIG. 1 .

The photographing device 10 may mean a device capable of photographing an image. The photographing apparatus 10 may photograph an image according to a user's manipulation or a preset input. The photographing device 10 may be an electronic device equipped with a camera, and may be provided in connection with the server 20 or may be provided to enable transmission and reception of information through a communication unit (not shown) in a separately provided state. Also, the photographing device 10 may include a separate recording module (not shown) to record music played during video recording.

Here, the photographing device 10 may photograph a person 2, a moving object (not shown), a background, etc. located within a preset photographable radius, and record music played while photographing in real time.

The server 20 may include a communication unit 210 , a memory 220 , and a control unit 230 , but is not limited thereto and may include fewer or more components. Coordinates to be described below are three-dimensional coordinates.

The communication unit 210 is one that enables wireless communication between the server 20 and a wireless communication system, between the server 20 and the photographing device 10, or between the server 20 and an external server (not shown). It may include more than one module. In addition, the communication unit 210 may include one or more modules for connecting the server 20 to one or more networks.

Here, the communication unit 210 may receive information about the image and music included in the image from the photographing device 10 , and a control signal for the operation of the photographing device 10 generated by the server 20 . can be transmitted.

The memory 220 may store information supporting various functions of the server 20 . The memory 220 may store a plurality of application programs (or applications) driven in the server 20 , data for operation of the server 20 , and commands. At least some of these application programs may be downloaded from an external server (not shown) through wireless communication. In addition, at least some of these application programs may exist for a basic function of the server 20 . Meanwhile, the application program may be stored in the memory 220 , installed on the server 20 , and driven to perform an operation (or function) of the server 20 by the controller 230 .

In addition to the operation related to the application program, the controller 230 may generally control the overall operation of the server 20 . The controller 230 may provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 220 .

The controller 230 may control at least some of the components discussed with reference to FIG. 1 in order to drive an application program stored in the memory 220 . Furthermore, in order to drive the application program, the controller 230 may operate at least two or more of the components included in the server 20 in combination with each other.

Hereinafter, when the control unit 230 recognizes the movement of the person 2 through the marker with respect to the person 2 who has attached the marker, this is called a 'first person', and the control unit 230 When the movement of the person 2 is recognized without recognizing the marker for the person 2 who has attached the marker and moved, this will be referred to as a 'second person'. Accordingly, it may mean that the first person and the second person are the same person.

Specifically, the controller 230 may obtain and store the respective posture information for a plurality of postures performed by the first person in the input image. Here, the respective posture information may include the direction, size, and joint position value of each of the joints of the first person performing the corresponding posture.

The controller 230 may determine the direction and size of the second person included in the input image, input the input image to the joint prediction model, and obtain joint position values of the joints of the second person. The joint prediction model generates reference coordinates of a fixed point based on the position of the second person for each frame in the input image, and connects the frames to the movement of the second person around the reference coordinates. Accordingly, a joint position value of each of the joints of the second person may be obtained.

Here, unlike recognizing motion through a sensor, in most current 3D pose prediction methods, relative positions of other joint coordinates are predicted with the pelvic coordinates as the origin. Therefore, it is impossible to obtain motions such as moving up and down or jumping in the running motion. Accordingly, the control unit 230 acquires joint position values of each of the joints of the second person on the assumption that the second person is standing on the floor at a point where the second person does not move, for example, so that the foot joint of the second person touches the ground in the coordinate system. It can be predicted in the form of standing on its feet.

The joint prediction model may include a convolutional neural network (CNN) and a recurrent neural network. Hereinafter, the convolutional neural network will be called 'CNN', and the recurrent neural network will be called 'RNN'.

CNN spatially integrates the convolution layer, which creates a feature map by applying a plurality of filters to each area of the image, and the feature map to extract features that are invariant to changes in position or rotation. It may be formed in a structure that alternately repeats the pooling layer several times. Through this, various level features can be extracted from low-level features such as points, lines, and planes to complex and meaningful high-level features.

The convolution layer can obtain a feature map by taking a nonlinear activation function on the dot product of the filter and the local receptive field for each patch of the input image. Compared to other network structures, CNNs may have the feature of using filters with sparse connectivity and shared weights. Such a connection structure can reduce the number of parameters to be learned, make learning through the backpropagation algorithm efficient, and consequently improve prediction performance.

The integration layer (Pooling Layer or Sub-sampling Layer) may generate a new feature map by using local information of the feature map obtained from the previous convolutional layer. In general, the newly created feature map by the integration layer is reduced to a smaller size than the original feature map. Representative integration methods include Max Pooling, which selects the maximum value of the corresponding region in the feature map, and the corresponding feature map in the feature map. There may be an average pooling method for obtaining an average value of a region. In general, the feature map of the integrated layer can be less affected by the location of arbitrary structures or patterns present in the input image than the feature map of the previous layer. That is, the integration layer can extract features that are more robust to regional changes such as noise or distortion in the input image or previous feature map, and these features can play an important role in classification performance. Another role of the integration layer is to reflect the features of a wider area as you go up to the upper learning layer in the deep structure. It is possible to generate features that reflect the characteristics of the abstract image as a whole.

In this way, the features finally extracted through iteration of the convolutional layer and the integration layer are fully connected to a classification model such as a multi-layer perception (MLP) or a support vector machine (SVM). -connected layer) and can be used for classification model training and prediction.

RNN is a deep learning technique that is effective for learning the sequence through a structure in which a specific part is repeated. Because it needs to be recognized by referring to the preceding word, letter, or frame). The RNN can be mainly used to recognize sequential information such as speech and letters.

However, the joint prediction model according to the present invention is not limited to CNN and RNN, and may be formed of neural networks having various structures.

As an example, a transformer may be used instead of CNN. Transformer, a neural network structure widely used in the field of natural language processing, receives input as a sequence of 'words'. Thus, when it is put in a Transformer, it is possible to obtain a result that surpasses CNN. Accordingly, it can be replaced with a Transformer instead of a CNN.

The control unit 230 determines specific posture information corresponding to the joint position value of the identified second person among the posture information obtained for the plurality of postures performed by the first person in the input image, and the identified second person The direction and size of the second person may be corrected with the direction and size of the first person in the identified specific posture information.

The controller 230 may determine the specific posture information identical to the first posture of the second person based on the first posture information of the first person and correct the direction and size of the second person to the direction and size of the first person. have. Therefore, before determining the joint position value of the first person and the joint position value of the second person, it is possible to have the effect of enabling accurate determination through correction of the direction and size.

Here, the control unit 230 inputs the joint position value of the first person and the joint position value of the second person in the specific posture information to the discrimination model in a state in which the direction and size of the second person are corrected, and the first person It is possible to calculate a first error value of the joint position value of the second person with respect to the joint position value of . Thereafter, the controller 203 may determine the first accuracy of the discrimination model based on the calculated first error value. Here, the first error value is a structural error value when the joint distance is different from a preset reference distance according to each joint position value of the second person or a speed error value when the joint movement speed is different from the preset reference speed may include

Specifically, when the structure loss value is understood as a sequence of joint coordinates, the distance between the knee joint and the foot joint is the length of the shin, and should not change with time. Therefore, the inconsistency of the distance between the connected joints over time can be judged as a structural error value.

In addition, the velocity error value (Velocity Loss), the movement of the joint according to time, that is, the joint velocity should not be too fast. Therefore, the speed of the joint must be large enough to follow the movement of the joint on the actual data set, and be slow enough not to cause noise and move. Therefore, the speed of the joint can be determined as a speed error value.

The control unit 230 determines whether the first accuracy is less than a preset threshold value based on the first accuracy, and repeats the joint prediction model or the determination model until the first accuracy is less than the threshold value can machine learning. Accordingly, the server 20 may have the effect of acquiring a joint position value by recognizing the movement of a person in real time based on the image captured in real time by the photographing device 10 .

Hereinafter, when the control unit 230 recognizes the movement of the person 3 through the marker with respect to the person 3 who has attached the marker, this is called a 'third person', and the control unit 230 When the movement of the person 3 is recognized without recognizing the marker for the person 3 who has attached the marker and moved, this will be referred to as a 'fourth person'. Accordingly, it may mean that the third person and the fourth person are the same person. Additionally, the person 2 and the person 3 may be different persons or the same person, and the person 2 and the person 3 may move with different motions or move with the same motion. However, even if the person 2 and the person 3 move with the same motion, it is impossible to move with the same motion.

The controller 230 may machine-lear the joint prediction model by inputting the first error value and the second input image to the joint prediction model. The control unit 230 may perform machine learning of the discriminant model by inputting the joint position value of the third person stored in advance and the joint position value of the fourth person calculated by learning based on the learned joint prediction model to the discrimination model. have.

2 is a diagram schematically showing the operation of the server 20 according to the present invention.

Referring to FIG. 2 , the server 20 performs a posture for a plurality of postures performed by the first person based on the input image received from the photographing device 10 , the direction, size, and joint position of each of the joints. Each posture information including a value may be obtained.

The server 20 may determine the direction and size of the second person included in the input image, and input the input image to the joint prediction model to obtain joint position values of the joints of the second person. Here, the server 20 may identify specific posture information corresponding to the identified joint position value of the second person among the stored respective posture information. The server 20 corrects the identified direction and size of the second person to the direction and size of the first person in the identified specific posture information, and in a state in which the direction and size of the second person are corrected, the specific posture By inputting the joint position value of the first person and the joint position value of the second person in the information into the discrimination model, a first error value of the joint position value of the second person with respect to the joint position value of the first person can be calculated .

The server 20 may determine the first accuracy of the discrimination model based on the calculated first error value.

The server 20 performs machine learning of the joint prediction model by inputting the first error value and the second input image to the joint prediction model, and based on the previously stored joint position value of a third person and the learned joint prediction model By inputting the joint position value of the fourth person calculated by learning to the discrimination model, the discrimination model may be machine-learned.

Therefore, the joint prediction model and the discriminant model continue to have lower error values through machine learning. Accordingly, when a new image is input, joint position values can be obtained in real time according to the movement of the person in the new image. .

3 is a flowchart illustrating a process in which the server 20 according to the present invention recognizes a motion in an image.

4 is an exemplary diagram illustrating a process of obtaining a joint position value in a joint prediction model according to the present invention.

5 is an exemplary diagram of an error value generated for a distance between joints predicted through a joint prediction model according to the present invention.

6 is an exemplary diagram of an error value generated for the movement speed of the joint predicted through the joint prediction model according to the present invention.

First, referring to FIG. 3 , the process of recognizing a motion in an image of the present invention may be performed by the server 20 and may be performed through a specific program. Also, hereinafter, all operations of the server 20 may be performed by the controller 230 in the same manner.

The server 20 may obtain and store the posture information of the first person (S301).

The server 20 may obtain and store the respective posture information for a plurality of postures performed by the first person in the input image. Here, the respective posture information may include the direction, size, and joint position value of each of the joints of the first person performing the corresponding posture.

The server 20 may determine the direction and size of the second person (S302), and may obtain the joint position value of the second person based on the joint prediction model (S303).

The server 20 may determine the direction and size of the second person included in the input image, and input the input image to the joint prediction model to obtain joint position values of the joints of the second person.

Here, the joint prediction model generates reference coordinates of a fixed point based on the position of the second person for each frame in the input image, and connects the frames to move the second person around the reference coordinates Accordingly, a joint position value of each of the joints of the second person may be obtained.

4, when the server 20 obtains the joint position value for each frame in the input image through the joint prediction model, the reference coordinate of the fixed point based on the position of the second person 401 ( 402) as a reference, and thereby, by connecting frames, it is possible to obtain joint position values of each of the joints of the second person according to the movement of the second person around the reference coordinates. Here, the joint prediction model may include a convolutional neural network (CNN) and a recurrent neural network.

As an example, the server 20 may hold the reference coordinate 402 as a point on the floor that is a fixed point that does not move. The left side of Fig. 4 is the first frame in which the second person 401 jumps by 30 cm based on the reference coordinates 402 of the floor, and the right side of Fig. 4 is the second frame after the first frame, after 0.3 seconds have elapsed. As a result, the second person 401 jumps upward to the right based on the reference coordinates 402 of the floor. Here, in the server 20, the state of the second person 401 of the second frame is 0.3 seconds elapsed, but since it is in a position where a person cannot move from the position of the first frame, through the process of deleting this error, It is possible to obtain a joint position value of each of the joints of the second person 401 .

The server 20 may determine specific posture information corresponding to the joint position value of the second person (S304), and correct the direction and size of the first person based on the specific posture information of the second person (S305).

The server 20 recognizes specific posture information corresponding to the joint position value of the identified second person among the respective posture information obtained for a plurality of postures performed by the first person in the input image, and the identified second person The direction and size of the second person may be corrected with the direction and size of the first person in the identified specific posture information.

The server 20 may determine the specific posture information identical to the first posture of the second person based on the first posture information of the first person and correct the direction and size of the second person to the direction and size of the first person have. Therefore, before determining the joint position value of the first person and the joint position value of the second person, it is possible to have the effect of enabling accurate determination through correction of the direction and size.

The server 20 may calculate a first error value of the joint position value of the second person based on the discrimination model (S306).

In the state in which the direction and size of the second person are corrected, the server 20 inputs the joint position value of the first person and the joint position value of the second person in the specific posture information to the discrimination model, and the joint position of the first person A first error value of the joint position value of the second person with respect to the position value may be calculated.

Here, the first error value is a structural error value when the joint distance is different from a preset reference distance according to each joint position value of the second person or a speed error value when the joint movement speed is different from the preset reference speed may include

5, if the determined arm length (the length from the wrist to the elbow) of the second person 501 is 27cm, the arm length according to the joint position value of the second person 501 in FIG. 5 is 50cm, The first error value may be a structure error value.

Referring to FIG. 6 , the left side shows the state in which the arm of the second person 601 is raised above the head, and the right side after the time of 0.01 seconds has elapsed is the state where the arm of the second person 601 goes down, the joint motion speed Since cannot move over a certain speed, in this case, the first error value may be a speed error value.

The server 20 may determine the first accuracy of the discrimination model based on the first error value (S307).

The server 20 determines whether the first accuracy is less than a preset threshold value based on the first accuracy, and repeats the joint prediction model or the determination model until the first accuracy is less than the threshold value can machine learning. Accordingly, the server 20 may have the effect of acquiring a joint position value by recognizing the movement of a person in real time based on the image captured in real time by the photographing device 10 .

The server 20 may perform machine learning of the joint prediction model and the discrimination model (S308).

The server 20 may machine-lear the joint prediction model by inputting the first error value and the second input image to the joint prediction model. The server 20 may machine-lear the discrimination model by inputting the joint position value of the third person stored in advance and the joint position value of the fourth person calculated by learning based on the learned joint prediction model to the discrimination model. have.

3 illustrates that steps S301 to S308 are sequentially executed, this is merely illustrative of the technical idea of this embodiment, and those of ordinary skill in the art to which this embodiment belongs Since it will be possible to change the order described in FIG. 3 and execute it or execute at least one of steps S301 to S308 in parallel within a range that does not deviate from the essential characteristics, various modifications and variations may be applied, so that FIG. 3 is in a time-series order It is not limited.

The method according to the present invention described above may be implemented as a program (or application) to be executed in combination with a server, which is hardware, and stored in a medium.

The above-described program is C, C++, JAVA, machine language, etc. that a processor (CPU) of the computer can read through the device interface of the computer in order for the computer to read the program and execute the methods implemented as a program It may include code (Code) coded in the computer language of Such code may include functional code related to a function defining functions necessary for executing the methods, etc., and includes an execution procedure related control code necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer to be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains know that the present invention may be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

10: shooting device
20: server

Claims (10)

A method for recognizing motion in an image performed by a server, the method comprising:
Acquiring and storing each posture information for a plurality of postures performed by the first person - Each of the posture information includes the direction, size, and joint position value of each of the joints of the first person performing the posture -;
recognizing the direction and size of a second person included in the input image;
inputting the input image to a joint prediction model to obtain joint position values of joints of a second person;
identifying specific posture information corresponding to the identified joint position value of the second person among the stored posture information;
correcting the identified direction and size of the second person to the direction and size of the first person in the identified specific posture information;
In a state in which the direction and size of the second person are corrected, the joint position value of the first person and the joint position value of the second person in the specific posture information are input to the discrimination model, 2 Calculating a first error value of the joint position value of the person; and
determining the first accuracy of the discrimination model based on the calculated first error value;
wherein the first person and the second person are the same person.
According to claim 1,
Machine learning the joint prediction model by inputting the first error value and the second input image to the joint prediction model; further comprising, a motion recognition method.
3. The method of claim 2,
The method further comprising: inputting the joint position value of the third person stored in advance and the joint position value of the fourth person calculated by learning based on the learned joint prediction model into the discrimination model to machine learning the discrimination model; and ,
The third person and the fourth person are the same person.
According to claim 1,
The joint prediction model is,
For each frame in the input image, a reference coordinate of a fixed point is generated based on the position of the second person for each frame, and the second person is connected according to the movement of the second person based on the reference coordinate by connecting the frames. A method of recognizing a motion to obtain a joint position value of each of the joints.
According to claim 1,
The first error value is
According to each joint position value of the second person, a structure error value when the joint distance is different from a preset reference distance, or a speed error value when the joint movement speed is different from the preset reference speed, according to the joint position value of the second person.
According to claim 1,
determining whether the first accuracy is less than a preset threshold value based on the first accuracy, and repeatedly machine learning the joint prediction model or the discrimination model until the first accuracy is less than the threshold value; Further comprising, a motion recognition method.
In the server for performing motion recognition in the image,
memory for storing information;
Obtaining and storing respective posture information for a plurality of postures performed by the first person, wherein each posture information includes the direction, size, and joint position value of each of the joints of the first person performing the posture;
Identify the direction and size of the second person included in the input image,
By inputting the input image to the joint prediction model, the joint position value of each joint of the second person is obtained,
Identifying specific posture information corresponding to the joint position value of the second person identified among the stored posture information,
correcting the direction and size of the identified second person to the direction and size of the first person in the identified specific posture information;
In a state in which the direction and size of the second person are corrected, the joint position value of the first person and the joint position value of the second person in the specific posture information are input to the discrimination model, 2 Calculate the first error value of the joint position value of the person,
A control unit for determining the first accuracy of the discrimination model based on the calculated first error value;
The first person and the second person are the same person, the server.
According to claim 1,
The control unit is
Machine learning the joint prediction model by inputting the first error value and the second input image to the joint prediction model,
Machine learning the discriminant model by inputting the joint position value of the third person stored in advance and the joint position value of the fourth person calculated by learning based on the learned joint prediction model to the discrimination model,
The third person and the fourth person are the same person.
According to claim 1,
The joint prediction model is,
For each frame in the input image, a reference coordinate of a fixed point is generated based on the position of the second person for each frame, and the second person is connected according to the movement of the second person based on the reference coordinate by connecting the frames. Obtaining the joint position value of each of the joints, the server.
A program stored in a computer-readable recording medium in combination with a computer to execute the method of any one of claims 1 to 6.

Images