KR20210090498A - The method for analyzing 3D body type based on 2D image, the system and the program thereof - Google Patents

Abstract

The present invention relates to a method for analyzing a three-dimensional (3D) body shape based on a two-dimensional (2D) image, a system thereof, and a program thereof. The body shape analysis system according to the present invention includes: an image data receiving unit for receiving 2D image data of a body photographed; a model generating unit for generating body shape data based on a neural network by using the 2D image data, and generating a virtual body model based on the generated body shape data; and a body shape analyzing unit for analyzing a body shape with respect to the generated virtual body model. Therefore, the method for analyzing a 3D body shape, the system thereof, and the program thereof according to an embodiment of the present invention can precisely analyze body shape information of a subject with only a 2D image easily taken by a general camera module.

Description

The method for analyzing 3D body type based on 2D image, the system and the program thereof}

The present invention relates to a body shape analysis method and a system and computer program for executing the body shape analysis method.

“Body analysis” indicates body imbalance information including height, weight, waist circumference, spinal curvature, and left-right balance by observing the subject from multiple angles, such as front, side, and rear.

Existing body shape analysis devices have a limitation in that a special camera such as a depth camera or an infrared camera must be used in a designated place. In addition, there is a limit to relying on the experience of the measurer because when taking a picture, a grid pattern is placed on the wall or a guide line is used to divide the shot image under the subjective judgment of the user. Therefore, the existing body shape analysis apparatus has a problem in that the repeatability is not high.

Meanwhile, a three-dimensional motion capture device has been developed and can be used as a body shape analysis device because various biometric indicators such as joint angle and speed of the subject can be acquired through this device. However, this equipment is quite expensive, takes a lot of time for measurement, and it is necessary to secure a large space to install the equipment, and there are disadvantages that professional personnel who can use this equipment are required.

Recently, artificial intelligence technology for estimating the joints of the human body has been developed, but a 3D scanner that rotates 360 degrees is additionally required for accurate body shape analysis. The three-dimensional scanner has disadvantages in that it is not only very expensive, but also has spatial limitations and a long computation time.

(KR) Registered Patent No. 10-1938361 (KR) Patent Publication No. 10-2019-0140765 (KR) Patent Publication No. 10-2016-0070498

The present invention has been devised to solve the above problems, and in order to analyze the body shape more easily than the existing complex and expensive equipment, a method, system and It aims to provide the computer program.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

A method for analyzing a 3D body shape based on a 2D image according to an embodiment of the present invention includes: receiving, by a computer processor, 2D image data of a body photographed; generating, by the processor, body shape data based on a neural network using 2D image data, and generating a three-dimensional virtual body model based on the generated body shape data; and analyzing, by the processor, a body type with respect to the generated virtual body model.

The generating of the virtual body model of the 3D body shape analysis method according to an embodiment of the present invention includes generating control data of an imaging module and using the generated control data of the imaging module and the body shape data to obtain the virtual body. It may be characterized in that the model is created.

The generating of the virtual body model of the 3D body shape analysis method according to an embodiment of the present invention comprises extracting skeletal information from the 2D image data and generating the virtual body model based on the skeletal information. can do.

The neural network according to an embodiment of the present invention may be characterized as a generative adversarial network (GAN).

According to an embodiment of the present invention, a system for analyzing a 3D body shape based on a 2D image includes an image data receiver configured to receive 2D image data of a body photographed; a model generator for generating body shape data based on a neural network using the 2D image data, and generating a virtual body model based on the generated body shape data; and a body shape analysis unit that analyzes a body shape with respect to the generated virtual body model.

The model generating unit of the three-dimensional body shape analysis system according to an embodiment of the present invention generates control data of an imaging module and generates the virtual body model by using the generated control data of the imaging module and the body shape data can be done with

The model generating unit of the 3D body shape analysis system according to an embodiment of the present invention may extract skeletal information from 2D image data and generate the virtual body model based on the skeletal information.

A 3D body shape analysis program based on a 2D image according to an embodiment of the present invention

receiving, by the processor of the computer, 2D image data of the body; generating, by the processor, body shape data based on a neural network using 2D image data, and generating a virtual body model based on the generated body shape data; and analyzing, by the processor, a body type with respect to the generated virtual body model.

The three-dimensional body shape analysis method, the system, and the computer program according to an embodiment of the present invention can precisely analyze body shape information of a subject with only a 2D picture easily taken with a general camera module.

Therefore, there is an advantage that an expensive body shape analysis device can be replaced through the body shape analysis method according to the present invention.

In addition, since the body shape analysis method according to the present invention uses 2D image data, there is an advantage in that it can be calculated within a shorter time than that of a conventional expensive body shape analysis device.

Therefore, since the body shape analysis method according to the present invention does not require a large-scale body shape analysis device, there is no spatial limitation, so that ordinary people can easily perform their body shape analysis using their mobile terminal or smart phone. .

Effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a conceptual diagram of a body shape analysis system according to an embodiment of the present invention.
2 is a flowchart of a body shape analysis method according to an embodiment of the present invention.
3 is a block diagram of a model data generator of a body shape analysis system according to an embodiment of the present invention.
4 shows a learning process of a model data generator according to an embodiment of the present invention.
5 is a flowchart of a body shape analysis method according to another embodiment of the present invention.
6 is a flowchart of a body shape analysis method according to another embodiment of the present invention.
7 shows a result of performing body shape analysis according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments will be described in detail with reference to the drawings. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. and/or includes a combination of a plurality of related description items or any of a plurality of related description items.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it should be understood that other components may exist in between. something to do. On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in between.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

It should also be understood that the terms "first" and "second" are used herein for distinguishing purposes only, and are not meant to indicate or anticipate sequences or priorities in any way.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Throughout the specification and claims, when a part includes a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

Prior to the description of the present specification, some terms used in the present specification will be clarified.

As used herein, a neural network is a term encompassing all kinds of machine learning models designed to imitate a neural structure. For example, the neural network may include all kinds of neural network-based models such as an artificial neural network (ANN), a convolutional neural network (CNN), and the like.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram of a body shape analysis system according to an embodiment of the present invention. The body shape analysis system 1000 according to an embodiment of the present invention may include an image data receiver 100 , a model generator 200 , and a body shape analyzer 300 .

The image data receiving unit 100 according to an embodiment of the present invention may receive one or more pieces of 2D image data showing the whole body of the body to be photographed, and the model generating unit 200 may apply the 2D image data to an artificial intelligence algorithm. By applying it, contour information and skeletal information can be extracted, and body shape data can be generated to generate a virtual body model. The body shape analysis unit 300 may analyze body shape information such as height, predicted age, estimated weight, BMI, and left/right balance based on the frontal basis of the photographed target body using the generated virtual body model.

Meanwhile, the body shape analysis system 1000 according to an embodiment of the present invention may further include a display unit (not shown) that provides personalized exercise or food information based on the analyzed body shape data. Meanwhile, the display unit may display the generated virtual body model in the form of a 2D or 3D avatar in which the body shape is reflected.

In the personalized exercise provision method, an exercise type such as aerobic exercise or anaerobic exercise may be prescribed according to the analyzed body type, and food information suitable for the analyzed body type may be recommended.

Referring to FIG. 1 , the model generating unit 200 according to an embodiment of the present invention will be described in more detail.

The model generation unit 200 according to an embodiment of the present invention includes a model data generation unit 210 , a model-based learning data unit 220 , an actual image-based learning data unit 230 , a determining unit 240 , and a determining unit. The learning unit 250 and the model data generating unit-learning unit 260 may be included, and a neural network that is a generative adversarial network (GAN) may be applied.

A generative adversarial neural network (GAN) is a machine learning algorithm for generating images similar to the original data distribution, and is being used as a technology that can quickly and easily create fakes that look like real ones. A generative adversarial neural network (GAN) transforms and learns the variables of a neural network through mutual competition between two neural network models, a generator and a discriminator, and derives results.

The generator learns real data for the purpose of generating data close to reality and generates data based on it, and the discriminator learns to determine whether the data generated by the generator is real or false.

A generative adversarial neural network (GAN) composed of one generator and one discriminator may be applied to the model generator 200 according to an embodiment of the present invention, and the model data generator 210 according to an embodiment of the present invention may be applied. ) may correspond to the generator of the GAN algorithm, and the discriminator 240 according to an embodiment of the present invention may correspond to the discriminator of the GAN algorithm.

Accordingly, the model data generator 210 according to an embodiment of the present invention may receive 2D image data and generate a virtual body model based on the simulator. Data constituting the virtual body model may be learned to be similar to the body shape of the actual image data.

More specifically, the model data generator 210 calculates and generates body shape data for constructing a virtual body model from the 2D image data. More details on this will be described later with reference to FIGS. 3 and 4 .

The actual image-based learning data unit 230 according to an embodiment of the present invention generates body shape data by recognizing a human body shape from actual image data and removing pixel data of a background other than the human shape.

In addition, the actual image-based learning data unit 230 processes the image data by using the generated data about the body shape and the image data stored in the image data DB 10 together, and can be read by the determining unit 240 . It can be converted into data format.

The model-based learning data unit 220 according to an embodiment of the present invention may convert the model data newly generated by the model data generation unit 210 into a data form that can be read by the determination unit 240 .

The determining unit 240 according to an embodiment of the present invention receives learning data based on the actual image data from the real image-based learning data unit 230 and receives the training data based on the newly created model from the model-based learning data unit 220 . ), the training data based on the actual image data and the training data based on the generated model are received in an arbitrary order by the determining unit 240 .

The determining unit 240 according to an embodiment of the present invention classifies the virtual generated data and the actual image-based data, and confirms the classification result. That is, the determination unit 240 according to an embodiment of the present invention aims to classify the real image data or the virtual model data well, and when the learning is repeated, the real image data and the virtual model data are better classified. can be classified.

The discriminator-learning unit 250 according to an embodiment of the present invention may receive the result classified by the discriminator 240 and adjust the learning network of the discriminator 240 using the result value.

The model data generation unit-learning unit 260 according to an embodiment of the present invention receives the result classified by the determination unit 240 and uses the result value to build the learning network of the model data generation unit 210 . Can be adjusted.

Meanwhile, the determining unit-learning unit 250 and the model data generating unit-learning unit 260 according to an embodiment of the present invention may adjust the learning network using a machine learning algorithm.

2 is a flowchart of a body shape analysis method according to an embodiment of the present invention.

The body shape analysis method according to an embodiment of the present invention may be performed using the body shape analysis system 1000 according to FIG. 1 , and the processor receives the photographed 2D image data ( S110 ), using the 2D image data. to generate body shape data based on the neural network (S120), and generating a virtual body model based on the body shape data (S130). It may include analyzing the body shape with respect to the generated virtual body model ( S140 ) and providing the analyzed body shape data and personalized exercise or food information ( S150 ).

The present invention has technical characteristics in the step of generating body shape data based on the neural network using 2D image data (S120) and the step of generating a virtual body model based on the body shape data (S130), as shown in FIGS. 3 and 4 It will be described in more detail with reference to .

3 is a block diagram of the model data generation unit 210 of the body shape analysis system according to an embodiment of the present invention, and FIG. 4 shows a learning process of the model data generation unit according to an embodiment of the present invention.

Referring to FIG. 3 , the model data generation unit 210 according to an embodiment of the present invention includes a skeleton information extraction unit 1210 , a body model information generation unit 2210 , a body simulator 3210 , and an image data conversion unit ( 4210) may be included.

The skeleton information extracting unit 1210 according to an embodiment of the present invention may receive image data from the image data DB 10 to extract skeleton information and calculate data of the skeleton information. The skeleton information includes the position of the skeleton axis and the skeleton It may include an angle value between the two. Meanwhile, the image data DB 10 may be built into the body shape analysis system 1000 or connected to an external server.

The body model information generating unit 2210 according to an embodiment of the present invention may generate data for determining the body type of the body model and control data of the imaging module, and generates a variable corresponding to the type of data and a value of each variable. can do.

Data for determining the body type of the body model may be arm thickness, leg thickness, arm length, leg length, torso area, etc., and may be referred to as body shape data throughout the specification.

The body simulator 3210 according to an embodiment of the present invention may receive the skeletal information and the body shape data, and then generate a 3D body model while changing the values of the body shape data while fixing the data of the skeletal information. Meanwhile, since the value of the body shape data may vary depending on the position and direction of the imaging module, the body simulator 3210 according to an embodiment of the present invention can generate a three-dimensional body model while also changing the control data of the imaging module. have.

The photographing module according to an embodiment of the present invention may include a camera and the like, and the control data may include data indicating the position and direction of the camera.

That is, the body simulator 3210 according to an embodiment of the present invention may generate a virtual body model in consideration of the position and direction of the camera.

The image data conversion unit 4210 according to an embodiment of the present invention converts data constituting the 3D body model generated by the body simulator 3210 into 2D-based image data and uses the model-based learning data unit 220 . Sends 2D-based image data.

The model-based learning data unit 220 according to an embodiment of the present invention converts the generated image data for the virtual body model into a data form that can be read by the determining unit 240, and the image of the virtual body model The data and the image data received from the image data DB 10 may be used together to be regenerated into data readable by the determining unit 240 .

A configuration of a neural network applied to the model data generator 210 will be described with reference to FIG. 4 .

The skeleton information extracting unit 1210 and the body model information generating unit 2210 according to an embodiment of the present invention are configured as a framework of a neural network, and the skeleton information extracting unit 1210 is a skeleton detection network (Skeleton Detect Network). It may be composed of a learning framework learned as a FFNN (Convolutional Neural Network) network frame that performs the role of a convolutional layer, and FFNN (Feed-FFNN) that performs the role of a fully connected layer, and the body model information generator 2210 serves as a convolutional layer. Forward Neural Network) may be configured as a network frame.

Meanwhile, the learning framework applied to the skeleton information extractor 1210 may use a network variable learned using other types of image data in addition to the 2D image data provided from the image data DB 10 .

In addition, the framework applied to the body model information generating unit 2210 may be learning virtual model data in which 3D body shape information generated according to an embodiment of the present invention is reflected.

A process in which virtual model data generated according to an embodiment of the present invention is learned will be described with reference to FIG. 4 .

The 2D image data 60 is input to the skeleton information extraction unit 1210 and the body model information generation unit 2210 of the model data generation unit 210 .

The skeleton information extractor 1210 that has received the 2D image data 60 extracts the skeleton information vector 1211 and transmits it to the body simulator 3210 . The skeleton information vector 1211 according to an embodiment of the present invention may include a position of a skeleton axis, an angle value between skeletons, and the like.

The CNN network 2212 of the body model information generator 2210 that has received the 2D image data 60 extracts information necessary for configuring the body model existing in the image as a variable and outputs it as a vector. The vector output from the CNN network 2212 is input to two FFNN networks 2214 and 2216, respectively. The first FFNN network 2214 extracts a vector (2218, body shape data) for body shape information, and the second FFNN network 2216 extracts a vector (2220, imaging module control data) for information of the imaging module of the simulator. to extract

The body shape information according to an embodiment of the present invention may be arm thickness, leg thickness, arm length, leg length, torso area, etc., and the information of the photographing module may be data indicating the position and direction of the camera.

The body simulator 3210 according to an embodiment of the present invention includes the skeletal information vector 1211 received from the skeletal information extracting unit 1210 and the body shape information vector 2218 received from the body model information generating unit 2210 and photographing. A virtual body model 62 is generated using the module information vector 2220 .

More specifically, the body simulator 3210 may generate and learn the virtual body model 62 while changing the body shape data value and the control data value of the imaging module while the skeletal information data is fixed. The body simulator 3210 according to an embodiment of the present invention may also generate and learn a new virtual body model 62 by changing the movement of the virtual body model by changing the skeletal information data.

The photographing module 3212 of the body simulator 3210 according to an embodiment of the present invention captures an image of the virtual body model 62, and the image of the virtual body model 62 uses an image data converter 4210 is converted into 2D virtual image data 64 and stored in the image data converter 4210 .

5 is a flowchart of a body shape analysis method according to another embodiment of the present invention. 6 is a flowchart of a body shape analysis method according to another embodiment of the present invention.

5 and 6 , the body shape analysis method according to another embodiment of the present invention generates a virtual body model using skeletal information, body shape data, and control data of an imaging module of a body simulator as described with reference to FIG. 4 . can create

The body shape analysis method according to FIGS. 5 and 6 includes the body shape analysis method according to FIG. 2, the steps of receiving 2D image data (S210, S310), and the steps of analyzing the body shape with respect to the generated virtual body model (S240, S340) and analysis Steps (S250, S350) of providing the body shape data and personalized exercise/food information may be the same.

The body shape analysis method according to FIG. 5 may generate body shape data and control data of the imaging module based on the neural network (S220), and may generate a virtual body model based on the body shape data and control data of the imaging module (S230).

The body shape analysis method according to FIG. 6 extracts skeletal information, generates body shape data and control data of the imaging module based on a neural network (S320), and virtual body based on skeletal information, body shape data, and control data of the imaging module A model may be generated (S340).

7 shows a result of performing body shape analysis according to an embodiment of the present invention.

That is, FIG. 7 shows a virtual model of a photographed body using a three-dimensional body shape analysis program according to an embodiment of the present invention, so that the body shape analysis program according to the present invention predicts the age, height, and weight of the photographed subject. , left-right balance, waist circumference or thigh circumference, etc., as well as providing information on whether a straight neck or a turtle neck, and whether the pelvis or legs are left or right, as well as information on exercise types and recommended foods tailored to an individual's body type. can

The system described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, systems and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and carry out program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The above description is merely illustrative of the technical idea of the present invention, and a person of ordinary skill in the art to which the present invention pertains may make various modifications and variations without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

10: image data DB
100: image data receiving unit
200: model generation unit
210: model data generation unit
1210: skeletal information extraction unit
2210: body model information generating unit
3210; body simulator
4210: image data conversion unit
220: model-based learning data unit
230: actual image-based learning data unit
240: determination unit
250: discrimination unit-learning unit
260: model data generation unit-learning unit
300: body shape analysis unit
1000: body type analysis system according to the present invention

Claims (9)

receiving, by the processor of the computer, 2D image data of the body;
generating, by the processor, body shape data based on a neural network using 2D image data, and generating a three-dimensional virtual body model based on the generated body shape data; and
and analyzing, by the processor, a body shape with respect to the generated virtual body model. According to claim 1,
The step of generating the virtual body model comprises:
Generating control data of an imaging module and generating the virtual body model using the generated control data of the imaging module and the body shape data
A 3D body shape analysis method based on 2D images. According to claim 1,
The step of generating the virtual body model comprises:
extracting skeletal information from the 2D image data and generating the virtual body model based on the skeletal information
3D body shape analysis method based on 2D image. According to claim 1,
The 3D body shape analysis method based on a 2D image, characterized in that the neural network is a generative adversarial network (GAN). an image data receiving unit for receiving 2D image data of a body photographed;
a model generator that generates body shape data based on a neural network using the 2D image data and generates a three-dimensional virtual body model based on the generated body shape data; and
2D image-based three-dimensional body shape analysis system comprising a; a body shape analyzer that analyzes a body shape with respect to the generated virtual body model. 6. The method of claim 5,
The model generator generates control data of the imaging module and generates the virtual body model using the generated control data of the imaging module and the body shape data. 6. The method of claim 5,
The model generator extracts skeletal information from the 2D image data and generates the virtual body model based on the skeletal information. 6. The method of claim 5,
The 3D body shape analysis system based on 2D image, characterized in that the neural network is a generative adversarial network (GAN). In the computer program stored in a computer-readable recording medium combined with the processor of the computer,
A 3D body shape analysis program based on a 2D image, characterized in that the method of any one of claims 1 to 4 is executed.

Images