KR20230080191A - Apparatus and method for estimating body size based on full body picture and height using deep learning - Google Patents

Abstract

An apparatus for estimating a body size according to the present invention may include: a data receiving part that receives a frontal photo of the full body of a user and the height value of the user, and a processor that extracts a human body area from the frontal photo, calculates body proportions based on the human body area, and calculates body size based on the body proportions and height value.

Description

Apparatus and method for predicting body size based on body picture and height using deep learning

The present invention relates to an apparatus and method for predicting a body size that can be used when selecting a clothing size based on a full body picture and a height value using deep learning technology.

When a user purchases clothing online or non-face-to-face, it is impossible to accurately measure and try on, and since different sizing standards are applied to each seller, there is a high probability of returns due to dissatisfaction with the size after purchase. The domestic online apparel market is worth 14 trillion won, and the average return rate is about 30%.

If an online clothing shopper knows his or her body size accurately, there will be no problem in selecting the size, but most of the cases are not, and the body size can change over time, and it is inconvenient to measure it directly using a tape measure. In addition, there are many cases where you do not know which part to measure. Accordingly, there is a need for a technology that enables a prospective purchaser of online clothing shopping to easily, quickly, and accurately measure his/her own body size. If we provide apparel shoppers with a way to actually measure their body size prior to purchasing a product, and recommend a customized size by reflecting the size standard of the purchased item, the cost of loss caused by returns due to clothing size can be greatly reduced. can

In the past, statistical techniques such as the use of multiple buyers' reviews have been used to predict such body size. Conventional methods have limitations in predicting body size.

Accordingly, various studies are being conducted on techniques for accurately and quickly predicting a user's body size using deep learning technology. However, even the services introduced with these latest technologies do not have a small amount of information that the user must directly input, so users may feel uncomfortable using them, and they require 3D scanning or multi-angle photos, which cannot be performed with simple equipment. Discomfort exists.

Accordingly, there is a demand for a technique for accurately predicting a user's actual body size with minimal information while conveniently using simple equipment.

The foregoing background art is technical information that the inventor possessed for derivation of the embodiments of the present invention or acquired during the derivation process, and is not necessarily a known technology disclosed to the general public prior to filing the application of the embodiments of the present invention. does not exist.

An object to be solved by an embodiment according to the present invention is to provide a method and method for predicting a body size by inputting only a frontal picture of a user and a height value using deep learning technology in order to solve the above-described problem.

However, the problem to be solved by the present invention is not limited to the above-mentioned problem, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, an apparatus for predicting body size according to the present invention includes a data receiver for receiving a frontal photograph of a user's entire body and a height value of the user; and a processor that extracts a body region from the frontal photograph, calculates a body proportion based on the body region, and calculates a body size based on the body proportion and height value. More specifically, the processor may include: a body region extraction unit extracting the body region from the frontal photograph; a measurement point extraction unit generating a plurality of measurement points on the contour of the body region; a body proportion predictor configured to calculate the body proportion based on the body region and the measurement point; and a body size predictor configured to calculate the body size based on the body ratio and the height value.

More specific descriptions and other details of embodiments are included in the detailed description and drawings.

According to an embodiment of the present invention, since the precise body size measurement is performed based on the frontal picture, the purchaser can measure the body size without separate actual measurement.

According to an embodiment of the present invention, it is possible to solve the size problem for the selected clothing by automatically learning different size reference data for each company.

According to an embodiment of the present invention, the effect of providing customized clothing recommendations without the help of experts through continuous learning of the body size measurement result of the body size prediction system using the frontal photo, the company size reference data and the buyer's clothing purchase history there is

1 is a diagram for explaining a body size prediction method according to an embodiment of the present invention.
2 is a diagram for explaining a body region extraction unit according to an embodiment of the present invention.
3 is a diagram for explaining a measurement point extraction unit according to an embodiment of the present invention.
4 is a diagram for explaining a body proportion predictor according to an embodiment of the present invention.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In order to solve the above-mentioned problems of the non-face-to-face purchase method in the conventional online clothing market (conventionally, a high return rate and a size problem that is a cause of return), according to an embodiment of the present invention, an apparatus and method for predicting body size utilize deep learning. Therefore, it is possible to predict the body size based on the user's whole body frontal photographed image. According to one embodiment of the present invention, the device of the present invention may be implemented in the form of a system including a plurality of devices.

In an embodiment of the present invention, the apparatus for predicting body size may perform prior learning on information on size criteria for each clothing seller and use the information for predicting body size.

The purpose of this study is to provide the results of measuring the size and ratio of each part of the body through an artificial intelligence-based body size precision measurement model in a body size prediction system based on a full-body image.

The present invention is to solve the above-mentioned problems of the prior art, a method for learning data such as measured body size and ratio, company size criteria, buyer's characteristic classification, and clothing purchase history, and recommending customized clothing based on artificial intelligence. and to provide a system.

The apparatus for measuring body size according to the present invention may be implemented in the form of a terminal or, in some cases, may be implemented in the form of a system including various hardware. For example, the body size measuring device may be implemented in the form of a single device such as a smart phone, smart watch, smart glass, tablet, various wearable devices, laptop, desktop, etc., or may be implemented as one or more servers connected through a network.

1 is a diagram showing the structure of a body size measuring device according to an embodiment of the present invention.

A body size measuring device according to an embodiment of the present invention may include a data receiving unit and a processor.

The data receiving unit may receive a frontal photograph of the user's whole body and a height value of the user. To this end, the data receiving unit may include a device for receiving a frontal picture of the user and a height value. For example, when the body size measuring device is implemented as a smart phone, the data receiving unit may include a camera for taking a picture and a touch display for obtaining various texts and user input. In another embodiment, when the apparatus for measuring body size is implemented as one or more servers, the data receiving unit may include a communication unit for receiving photo data and an input value transmitted from an external terminal. The data receiver may be wired or wirelessly connected to transmit the received frontal photo image data or the height value of the user to the processor.

The processor may predict the user's body size based on the user's frontal picture and the height value received through the data receiving unit. Accordingly, the body size predicting apparatus according to the present invention can predict the user's body size more easily than the prior art. In particular, it is possible to predict the body size with only a frontal picture of the user's whole body and a height value, and there is no need to take pictures from various angles or input other body information such as weight, as in the prior art, in terms of user convenience. Significantly improved effects can be obtained. Specifically, the processor may extract a body region from the frontal picture received through the data receiving unit, calculate a body ratio based on the body region, and calculate a body size based on the body ratio and height value.

In one embodiment of the present invention, processors (processors) may be devices or components that perform various calculations, controls, and judgments. For example, processors include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and microprocessors. (Micro Processor), IC (Integrated Circuit), CPU (Central Processing Unit), MCU (Micro Controller Unit), ECU (electronic control unit) controllers, micro-controllers, microprocessors, It may be implemented using at least one of electrical units for performing other functions.

Processors may include a memory (not shown). The memory is electrically connected to processors. In terms of hardware, the memory may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, and the like. The memory may store various instructions and data for overall operations of processors, such as programs for processing or controlling processors.

In one embodiment of the present invention, the processor may include a body region extraction unit, a measurement point extraction unit, a body ratio estimation unit, and a body size estimation unit. Each component of the processor will be described in detail with reference to the drawings below.

2 is a diagram for explaining a body region extraction unit according to an embodiment of the present invention.

In one embodiment of the present invention, the body region extractor may extract the body region from the frontal picture received by the data receiver. The body region extraction unit may include an artificial neural network trained to extract a body region (a region in which the user's body is displayed within the entire image) within the entire image of the frontal photograph in which the user's body is displayed using Mask-RCNN. Mask-RCNN is a deep learning model that generates a segmentation mask for each object instance in an image in pixel units. Mask-RCNN may have a structure in which a mask branch for generating a segmentation mask in units of pixels is added to Faster-RCNN, which is an object detection model. Faster-RCNN can recognize the area (position) where an object exists in a specific image, and at this time, information about the space in the image is not needed. The Mask Branch added to Mask-RCNN can extract three-dimensional spatial information from the two-dimensional location information in the image, and thus extract the human body region, which is the part where the user's whole body appears in the frontal picture. there is. Mask-RCNN may include a backbone neural network, a Region Proposal Network (RPN), a RoI Align layer, a layer that predicts bounding boxes and classes, and a layer that generates a mask in units of pixels. The backbone neural network can extract image features within frontal photos. For example, the backbone neural network can be ResNet. A region of interest (RoI) can be extracted in the form of a bounding box from the image features extracted by the RPN backbone neural network. RoI Align layer Position alignment of the region of interest can be performed. Layers can then predict bounding boxes and classes, and create masks in units of pixels.

In one embodiment of the present invention, the measurement point extraction unit may generate a plurality of measurement points corresponding to the body regions extracted by the body region extraction unit.

The measuring point may be a location coordinate value or a point image indicating a location of a specific point in the picture. According to one embodiment of the present invention, the plurality of measurement points include two measurement points located at both ends of the shoulder (referred to as shoulder measurement points), two measurement points located at both armpits (referred to as chest measurement points), and two measurement points located at both ends of the waist. It may include at least one of two measurement points (referred to as waist measurement points), two measurement points located at both ends of the pelvis (referred to as pelvis measurement points), and two measurement points located at both ends of the neck and shoulder joint (referred to as neck measurement points). can In another embodiment, the measurement points may include two measurement points (thigh measurement points) located at both ends of each thigh.

The measurement point extraction unit may include a Convolutional Point Regression artificial neural network model learned to generate a plurality of measurement points in the frontal picture. The Convolutional Point Regression artificial neural network model uses a plurality of frontal photos of the whole body, information on a plurality of measurement points preset in correspondence with each of the whole bodies on the frontal photos, and skeletal information generated in response to each of the frontal photos as learning data, and the frontal photo of the whole body is input. It is a deep learning model trained to create measurement points corresponding to the input full-body frontal photos.

The skeletal information is generated by determining the joint points of the human body appearing on the full-body frontal photograph and applying the Pose-Estimation technique. Skeleton information is image information consisting of a plurality of points displayed on the joints of each human body corresponding to human body regions on a frontal photograph and a plurality of lines connecting the dots to correspond to the shape of the human body. Pose-Estimation is a technique for recognizing the posture of an object (person) in an image. In this process, a skeleton (skeleton information) matching the object may be generated for accurate recognition. Through Pose-Estimation, it is possible to three-dimensionally recognize postures where body parts are overlapped or difficult to grasp on a flat surface.

The convolutional point regression artificial neural network model may include a global feature learning unit and a local feature learning unit. The global feature learning unit may learn relative positions (global features) between a plurality of measurement points. The local feature learning unit may learn the relative position (local feature) of a specific body part determined based on the skeleton information and a measurement point corresponding to the corresponding body part. The Convolutional Point Regression artificial neural network model is designed to generate measurement points that reflect global and local features. For example, if the global feature learning unit learns the measurement points for each body part including the shoulder measurement point and the wrist measurement point, it can learn the distance ratio between the entire measurement points rather than the distance (clear value) between two adjacent measurement points. . The global feature may represent the location and ratio of measurement points with respect to the entire body. A local feature may indicate a relationship between a specific body part and a measurement point. For example, when the relative positional relationship between the position of the shoulder joint of the skeleton and the measurement point is learned from the shoulder image in the frontal photograph, the body characteristics of the corresponding part can be identified. Developed styles can be identified. The determined body proportion may be converted into a clear numerical value through input of the body size. As the clothing size (100, 105, etc.) may have different measurement methods depending on the clothing manufacturer, it may be used when recommending clothing if the body size is calculated as a ratio.

In one embodiment of the present invention, the body proportion predictor may predict the body proportion based on the body region and the measurement point. For example, the body ratio predictor may include a shoulder ratio (ratio of shoulder width to height), a chest ratio (ratio of chest circumference to height), At least one of neck ratio (ratio of neck circumference to height) and waist ratio (ratio of waist circumference to height) may be calculated. In this process, the pixel length of the height of the object in the frontal picture and the length between the measuring points for measuring the body size are obtained, and the ratio between the length between the measuring points and the length of the height can be calculated by comparing these two values.

The body proportion predictor may calculate the body proportion using a dense regression net artificial neural network model learned based on the segmentation image of the body region extracted by the body region extractor and the measurement point information generated by the measurement point extractor.

The Dense Regression Net artificial neural network model uses the segmentation image of the human body region, measurement point information, and actual body size information of the user as learning data. may have been learned.

In one embodiment of the present invention, the body size predictor may calculate the user's body size based on the body proportion determined by the body proportion predictor and the height value received through the data receiver. The body size predictor may calculate the shoulder width based on the shoulder ratio and height values. The body size predictor may calculate the chest circumference based on the chest ratio and the height value. The body size predictor may calculate the neck circumference based on the neck ratio and the height value. The body size predictor may calculate the waist circumference based on the waist ratio and the height value. The body size prediction unit may also predict hip circumference, thigh circumference, leg length, upper arm circumference, and the like.

Hereinafter, a body size prediction method according to another aspect of the present invention will be described.

The body size prediction method according to the present invention may include a data receiving step, a body region extracting step, a measuring point extracting step, and a body ratio predicting step.

The data receiving step is a step of receiving a frontal picture of the user's whole body and a height value of the user.

The human body region extraction step is a step of extracting a human body region from a frontal photograph.

The step of extracting measurement points is a step of generating a plurality of measurement points corresponding to the human body region.

The body proportion estimation step is a step of calculating a body proportion based on a body region and a measurement point. The body proportion prediction step is based on the body area and measurement points, shoulder ratio (ratio of shoulder width to height), chest ratio (ratio of chest circumference to height), neck ratio (ratio of neck circumference to height), waist ratio ( This is a step of calculating at least one of the ratio of waist circumference to height).

The body size estimating step is a step of calculating the body size based on the body ratio and height values. The body size estimating step may include calculating a shoulder width based on the shoulder ratio and the height value, calculating a chest circumference based on the chest ratio and the height value, and calculating a neck circumference based on the neck ratio and the height value. and calculating a waist circumference based on the waist ratio and the height value.

The step of extracting measurement points is performed by a Convolutional Point Regression artificial neural network model trained to generate a plurality of measurement points in the frontal picture. The Convolutional Point Regression artificial neural network model uses a plurality of full-body frontal photos, a plurality of preset measurement point information corresponding to each of the whole bodies on the photos, and skeletal information generated corresponding to each of the photos as learning data, and the whole body frontal photo is input , it is learned to create a measurement point corresponding to the input full-body frontal photograph. The skeletal information is generated by determining the joint points of the human body appearing on the full-body frontal photograph and applying the Pose-Estimation technique.

The Convolutional Point Regression artificial neural network model includes a global feature learning unit that learns relative positions (global features) between the plurality of measurement points; and a local feature learning unit configured to learn relative positions (local features) of a specific body part determined based on the skeleton information and a measurement point corresponding to the corresponding body part. The Convolutional Point Regression artificial neural network model is designed to generate measurement points that reflect global and local features.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. , and also includes those implemented in the form of a carrier wave (eg, transmission over the Internet). Also, the computer may include a processor or a control unit. Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention. Although the present invention has been described with respect to the preferred embodiments mentioned above, various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the scope of the appended claims will include such modifications and variations as long as they fall within the gist of the present invention.

Claims (16)

a data receiving unit for receiving a frontal photograph of a user's entire body and a height value of the user; and
and a processor configured to extract a body region from the frontal photograph, calculate a body proportion based on the body region, and calculate a body size based on the body proportion and the height value.
According to claim 1,
the processor,
a body region extraction unit extracting the body region from the frontal photograph;
a measurement point extraction unit generating a plurality of measurement points in the frontal photograph based on the body region;
a body proportion predictor configured to predict the body proportion based on the body region and the measurement point; and
a body size estimation unit calculating the body size based on the body proportion and the height value;
A body size predictor comprising a.
According to claim 2,
The measurement point is
Two measurement points located at both ends of the shoulder (called shoulder measurement points),
Two measurement points located on both armpits (called chest measurement points),
Two measurement points located at both ends of the waist (called waist measurement points),
two measurement points located at both ends of the pelvis (referred to as pelvic measurement points), and
At least one of two measuring points (referred to as neck measuring points) located at both ends of the neck and shoulder joint,
The body proportion prediction unit,
Based on the body region and the measurement points, shoulder ratio (ratio of shoulder width to height), chest ratio (ratio of chest circumference to height), neck ratio (ratio of neck circumference to height), waist ratio (ratio of waist circumference to height) Calculate at least one of the ratio),
The body size predictor,
Calculate the shoulder width based on the shoulder ratio and the height value,
Calculate the chest circumference based on the chest ratio and the height value,
Calculate the neck circumference based on the neck ratio and the height value,
A body size predicting device for calculating a waist circumference based on the waist ratio and the height value.
According to claim 2,
The human body region extraction unit,
A body size prediction device including an artificial neural network trained to extract a human body region from the entire image of the frontal photograph using Mask-RCNN.
According to claim 4,
The Mask-RCNN,
A body size prediction device having a structure in which a mask branch that generates a segmentation mask in pixel units is added to Faster-RCNN, an object detection model.
According to claim 5,
The Mask-RCNN,
a backbone neural network for extracting image features from the frontal photograph;
a Region Proposal Network (RPN) for extracting a Region of Interest (RoI) from the image features in the form of a bounding box;
a RoI Align layer that aligns the position of the ROI;
a layer that predicts the bounding box and class; and
A body size prediction device composed of layers that generate a mask in pixel units.
According to claim 6,
The backbone neural network is ResNet body size prediction device.
According to claim 2,
The measuring point extraction unit,
A body size predicting device including a Convolutional Point Regression artificial neural network model learned to generate a plurality of measurement points in the frontal photograph.
According to claim 8,
The Convolutional Point Regression artificial neural network model,
Using a plurality of full-body frontal photographs, a plurality of measurement point information preset corresponding to each of the whole body on the photograph, and skeletal information generated corresponding to each of the photographs as learning data,
When a full-body frontal photograph is input, it is learned to create a measurement point corresponding to the input full-body frontal photograph,
The skeletal information is generated by determining the joint points of the human body appearing on the frontal photograph of the body and applying Pose-Estimation technology.
According to claim 9,
The Convolutional Point Regression artificial neural network model,
a global feature learning unit learning relative positions (global features) between the plurality of measurement points; and
A local feature learning unit for learning relative positions (local features) of a specific body part determined based on the skeletal information and a measurement point corresponding to the corresponding body part;
A body size predicting device designed to generate a measurement point in which the global feature and the local feature are reflected.
According to claim 2,
The body proportion prediction unit,
A body size measurement device for calculating the body ratio using a dense regression net artificial neural network model learned based on the segmentation image of the body region and the measurement point information.
According to claim 11,
The Dense Regression Net artificial neural network model,
Using the segmentation image of the human body region, the measurement point information, and the actual body size information of each user as learning data,
The body size measuring device is configured to output a ratio of each part of the body to a height when the segmentation image of the human body region and the measurement point information are input.
a data reception step of receiving a frontal picture of a user's entire body and a height value of the user;
a human body region extraction step of extracting the human body region from the frontal photograph;
a measurement point extraction step of generating a plurality of measurement points corresponding to the human body region;
a body proportion estimation step of calculating the body proportion based on the body region and the measurement point; and
a body size predicting step of calculating the body size based on the body ratio and the height value;
Body size prediction method comprising a.
According to claim 13,
The measurement point is
Two measurement points located at both ends of the shoulder (called shoulder measurement points),
Two measurement points located on both armpits (called chest measurement points),
Two measurement points located at both ends of the waist (called waist measurement points),
two measurement points located at both ends of the pelvis (referred to as pelvic measurement points), and
At least one of two measuring points (referred to as neck measuring points) located at both ends of the neck and shoulder joint,
The body proportion prediction step,
Based on the body region and the measurement points, shoulder ratio (ratio of shoulder width to height), chest ratio (ratio of chest circumference to height), neck ratio (ratio of neck circumference to height), waist ratio (ratio of waist circumference to height) A step of calculating at least one of the ratio of),
The body size prediction step,
Calculate the shoulder width based on the shoulder ratio and the height value,
Calculate the chest circumference based on the chest ratio and the height value,
Calculate the neck circumference based on the neck ratio and the height value,
A body size prediction method comprising calculating a waist circumference based on the waist ratio and the height value.
According to claim 13,
In the step of extracting the measurement point,
The Convolutional Point Regression artificial neural network model trained to generate a plurality of measurement points in the frontal photograph is performed,
The Convolutional Point Regression artificial neural network model,
Using a plurality of full-body frontal photographs, a plurality of measurement point information preset corresponding to each of the whole body on the photograph, and skeletal information generated corresponding to each of the photographs as learning data,
When a full-body frontal photograph is input, it is learned to create a measurement point corresponding to the input full-body frontal photograph,
The skeletal information is generated by determining the joint points of the human body appearing on the frontal photograph of the body and applying Pose-Estimation technology.
According to claim 13,
The Convolutional Point Regression artificial neural network model,
a global feature learning unit learning relative positions (global features) between the plurality of measurement points; and
A local feature learning unit for learning relative positions (local features) of a specific body part determined based on the skeletal information and a measurement point corresponding to the corresponding body part;
A body size prediction method designed to generate a measurement point in which the global feature and the local feature are reflected.

Images