JP7365435B2 - Computer-implemented body measurement method - Google Patents

Description

The present invention relates to a computer-implemented body measurement method.

With the recent development of computer technology, attempts have been made to automatically measure the body dimensions of a person from a video or image of the person. Such automatic measurement techniques from videos or images are attracting attention, particularly in the clothing field, as a method that can measure a customer's body dimensions over a network.

Currently well-known automatic measurement methods typically convert a two-dimensional video or image of the subject into a three-dimensional representation of the subject's body using a convolutional neural network (CNN). This is done by creating a model and measuring dimensions from this three-dimensional model. As an example of such automatic measurement technology, BodyNet disclosed in Non-Patent Document 1 is widely known. In BodyNet, a three-dimensional model is generated by estimating the skeleton of a subject photographed in a two-dimensional video or image and then fleshing out (skinning) the estimated skeleton.

On the other hand, as another example of automatic measurement technology, HS-Nets disclosed in Non-Patent Document 2 is also known. In HS-Nets, a three-dimensional model is directly generated from a plurality of two-dimensional images taken of a subject in a predetermined posture and orientation, rather than performing skeletal estimation or fleshing out as in BodyNet. In this method, the shooting state of a two-dimensional image serving as a base is specified, but a three-dimensional model can be generated with superior accuracy compared to BodyNet of Patent Document 1.

Gul Varol, et al., “BodyNet: Volumetric Inference of 3D Human Body Shapes”, ECCV 2018, arXiv:1804.04875 [cs.CV], April 13, 2018 Endri Dibra, et al., “HS-Nets : Estimating Human Body Shape from Silhouettes with Convolutional Neural Networks”, IEEE Conf. Proc., 2016, 3DV, p.108-117.

However, the three-dimensional models obtained by the methods of Non-Patent Documents 1 and 2 are insufficiently accurate to be used for the purpose of obtaining accurate measurement data for detailed body features, and the three-dimensional models obtained by these methods are There is a problem in that it is difficult to obtain highly accurate measurement data from the original model.

Furthermore, although HS-Nets in Non-Patent Document 2 has the advantage of being able to generate a three-dimensional model with relatively higher accuracy than the BodyNet in Non-Patent Document 1, if ideal captured images can be used, HS-Nets It is difficult for an individual to take an ideal photographic image. Therefore, it is difficult to obtain a three-dimensional model with high precision using this method, and there is a problem that measurement data cannot be obtained with high precision.

Therefore, there is a need for an automatic measuring method that can measure with higher accuracy based on a photographed two-dimensional image.

SUMMARY OF THE INVENTION In view of the above problems, it is an object of the present invention to provide a computer-implemented body measurement method that measures a subject's body characteristics with high accuracy.

A computer-implemented method for measuring the body of a subject according to the first aspect of the invention includes:
an image obtaining step of obtaining at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region;
Each of the photographed images is converted into a body region represented by a first color corresponding to the body region of the photographed image and a background region represented by a second color corresponding to the background region of the photographed image. a binarization step of generating reference two-tone images corresponding to the photographed images, respectively;
a definition step of defining each of the reference two-tone images as a model generation image;
a model output step of outputting a three-dimensional model of the subject,
a substep of generating a three-dimensional model of the subject from the model generation image using body shape template data and a convolutional neural network;
a substep of generating comparative two-tone images corresponding to each of the model generation images based on the three-dimensional model of the subject;
a substep of comparing each of the body regions of the reference two-tone image and each of the body regions of the comparison two-tone image to calculate a difference value between the two;
If the difference value is larger than a predetermined threshold, and if the number of executions of this model output step is less than a predetermined number of times, the comparison two-tone image is corrected, and the model generation image is changed to the corrected comparison image. A model output step, comprising: a substep of changing to a two-tone image and executing the model output step again;
If the last calculated difference value is less than or equal to a predetermined threshold, one or more dimensions related to the body characteristics of the subject are measured based on the last generated three-dimensional model of the subject. The method includes the steps of:

According to such a configuration, when outputting a three-dimensional model of a subject, a three-dimensional model with a sufficiently small error between a comparison two-tone image generated from the three-dimensional model and a reference two-tone image based on a photographed image is generated. The generation of the three-dimensional model is repeated while modifying the two-tone image that is the basis for generating the three-dimensional model. Thereby, the body characteristics of the subject can be measured with high precision based on the three-dimensional model in which the body of the subject is modeled with high precision.

A computer-implemented method for measuring the body of a subject according to a second aspect of the invention includes:
an image obtaining step of obtaining at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region;
Each of the photographed images is converted into a body region represented by a first color corresponding to the body region of the photographed image and a background region represented by a second color corresponding to the background region of the photographed image. a binarization step of generating the same number of reference two-tone images as the captured images;
a model output step of generating a three-dimensional model of the subject from the reference two-tone image using body shape template data and a convolutional neural network;
including;
Before the step of generating the reference two-tone image, estimating the skeleton of the subject from the photographed image and correcting the shape of the body region included in the photographed image using the estimated skeleton; Or
Before the step of generating the three-dimensional model of the subject, the skeleton of the subject is estimated from the photographed image or the reference two-tone image, and the estimated skeleton is used to create a three-dimensional model of the subject included in the reference two-tone image. The method further includes the step of correcting the shape of the body region.

According to such a configuration, when generating a reference two-tone image that is a base for generating a three-dimensional model of a subject, based on the captured image or the skeleton of the subject estimated based on the two-tone image, A reference two-tone image suitable for generating a three-dimensional model is generated by correcting distortion of the subject's body region that may occur when capturing a photographed image. Thereby, a three-dimensional model of the subject can be modeled with high precision, and based on such a three-dimensional model, the physical characteristics of the subject can be measured with high precision.

A computer-implemented method for measuring the body of a subject according to a third aspect of the present invention includes:
an image obtaining step of obtaining at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region;
Each of the photographed images is converted into a body region represented by a first color corresponding to the body region of the photographed image and a background region represented by a second color corresponding to the background region of the photographed image. a binarization step of generating the same number of reference two-tone images as the captured images;
a model output step of generating a three-dimensional model of the subject from the reference two-tone image using body shape template data and a convolutional neural network;
including;
In this method, the body shape template data is composed of body shape templates generated from a group of people having one or more attributes of the subject.

According to such a configuration, a three-dimensional model of the target person can be modeled with high accuracy by using body template data composed of body shape templates generated from a group of people close to the target person. Based on a three-dimensional model, the physical characteristics of the subject can be measured with high accuracy.

According to the computer-implemented method for measuring a subject's body according to the present invention, the subject's physical characteristics can be measured with high accuracy.

FIG. 1 shows one embodiment of the computer-implemented body measurement method of the present invention. FIG. 2 shows details of the three-dimensional model generation step included in the body measurement method of FIG. 1. FIG. 3 illustrates one embodiment of a system including a computer in which the body measurement method of FIG. 1 is performed. FIG. 4 shows details of the three-dimensional model generation module included in the system of FIG. 3. FIG. 5 shows an example of a photographing screen displayed on an application used when acquiring a photographed image.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a computer-implemented method for measuring the body of a subject according to the present invention will be described with reference to the drawings. However, the embodiments described below are merely illustrative. The present invention is not limited to the embodiments described below.

The computer-implemented method for measuring the body of a subject of the present invention includes:
an image obtaining step of obtaining at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region;
Each of the photographed images is converted into a body region represented by a first color corresponding to the body region of the photographed image and a background region represented by a second color corresponding to the background region of the photographed image. a binarization step of generating reference two-tone images corresponding to the photographed images, respectively;
a definition step of defining each of the reference two-tone images as a model generation image;
a model output step of outputting a three-dimensional model of the subject;
measuring one or more dimensions associated with a physical characteristic of the subject based on the three-dimensional model of the subject.

The first feature of the method of the present invention is that when outputting a three-dimensional model of the subject, the error between the comparison two-tone image generated from the three-dimensional model and the reference two-tone image based on the photographed image is sufficiently small. The purpose is to repeat the generation of the three-dimensional model while modifying the two-tone image, which is the basis of the three-dimensional model generation, as necessary, until the three-dimensional model is obtained.
Specifically, the first feature of the method of the invention is:
a model output step of outputting a three-dimensional model of the subject;
a substep of generating a three-dimensional model of the subject from the model generation image using body shape template data and a convolutional neural network;
a substep of generating comparative two-tone images corresponding to each of the model generation images based on the three-dimensional model of the subject;
a substep of comparing each of the body regions of the reference two-tone image and each of the body regions of the comparison two-tone image to calculate a difference value between the two;
If the difference value is larger than a predetermined threshold, and if the number of executions of this model output step is less than a predetermined number of times, the comparison two-tone image is corrected, and the model generation image is changed to the corrected comparison image. and a substep of changing to a two-tone image and re-executing this model output step,
The three-dimensional model of the subject used in the measurement step is the three-dimensional model finally output in the model output step.

Thereby, the body characteristics of the subject can be measured with high precision based on the three-dimensional model in which the body of the subject is modeled with high precision.

A second feature of the method of the present invention is to correct distortions of the subject's body region that may occur when capturing captured images.
Specifically, the second feature of the method of the invention is:
Before the binarization step, a photographed image correction step of estimating the skeleton of the subject from the photographed image and correcting the shape of the body region included in the photographed image using the estimated skeleton, or ,
Before the defining step, the skeleton of the subject is estimated from the photographed image or the reference two-tone image, and the shape of the body region included in the reference two-tone image is corrected using the estimated skeleton. It includes a two-tone image correction step.

Thereby, a three-dimensional model of the subject can be modeled with high precision, and based on such a three-dimensional model, the physical characteristics of the subject can be measured with high precision.

A third feature of the method of the present invention is that the body shape template data is composed of body shape templates generated from a group of people having one or more attributes of the subject.

Thereby, a three-dimensional model of the subject can be modeled with high precision, and based on such a three-dimensional model, the physical characteristics of the subject can be measured with high precision.

The method of the present invention may have each of these first to third features alone, or may have any combination of these features. When the method of the present invention includes a combination of a plurality of these first to third characteristics, it becomes possible to measure the physical characteristics of the subject with higher accuracy.

FIG. 1 shows Method 1, which is an embodiment of the computer-implemented body measurement method of the present invention. The method of this embodiment has the first to third features of the present invention described above.
Method 1 can be executed, for example, by computer system 3, which will be described below.

In this method 1, first, in step 11, at least one captured image including a body region corresponding to the subject's body and a background region corresponding to a region other than the body region is acquired (image acquisition step).

Specifically, in this step 11, photographed images corresponding to one or more body orientations and postures required for model generation to be performed in the later model output step 14 are acquired. In this embodiment, an image taken from the front of a subject in a neutral posture generally called "A pose" (frontal image), and an image taken from the side of the subject (side image) are used. Two captured images are acquired, including:

This step 11 can be obtained by photographing the subject with a camera included in or connected to a computer used for photographing.

Here, if the computer or camera used for photographing has a function of correcting the distortion of the photographed image, it is desirable to use that function to obtain the photographed image with the distortion corrected.
For example, if a computer or camera used for photographing is equipped with a tilt sensor, distortion processing may be performed using the tilt data. For example, distortion in the vertical dimensions of a body region caused by the tilt of a camera when a subject is photographed can be corrected based on the tilt data.

Furthermore, if the computer or camera used for photographing is equipped with a depth sensor, it is preferable to acquire depth data of the subject at the time of photographing. Based on the depth information obtained based on the depth data, it is possible to check whether there are any large errors in the physical information such as the height of the subject that has been acquired in advance. Further, the depth information can also be used to accurately determine the boundary between the body region and the background region when distinguishing them in the subsequent binarization step 12.

Next, in step 12, each of the photographed images is converted so that the body region is represented in a first color corresponding to the body region in the photographed image, and the second color is represented in a second color corresponding to the background region in the photographed image. Reference two-tone images corresponding to the photographed images, including the background region, are generated (binarization step).

By the binarization conversion of the photographed image performed in step 12, details included in the body region and background region are omitted in the generated two-tone image, and the body region and background region are clearly distinguished. become distinct. In the two-tone image generated in this way, the body region corresponding to the subject's body is clearly separated from the background region. It becomes easier to recognize the body area corresponding to the person's body.
Note that the two-tone image obtained in this step 11 will be used as a reference image (reference two-tone image) for confirming the accuracy of the three-dimensional model generated in the later model output step 15. .

In this step 12, each of the one or more photographed images acquired in the image acquisition step 11 described above is subjected to binarization conversion to generate a corresponding two-tone image. In this embodiment, the front photographed image and the side photographed image acquired in the image acquisition step 11 are respectively binarized to generate a reference front two-tone image and a reference side two-tone image.

The binarization conversion of the captured image in step 12 may be performed by any conventionally known method. As such a method, a binarization method using deep learning techniques such as global contrast normalization (GCN), semantic segmentation, and instance segmentation (eg, DeepMask) can be adopted. These methods may be used alone or in combination.

The first and second colors representing the body region and the background region, respectively, in the reference two-tone image may be different from each other, and they may be colors with significantly different colors or contrasts that make it easy to distinguish the boundaries of these regions. It is preferable that there be. Typically, the first color and the second color are selected from white and black, and the first color may be white and the second color may be black, or vice versa.

Next, in step 13, the subject's skeleton is estimated from the captured image or the reference two-tone image, and the estimated skeleton is used to correct the shape of the body region included in the reference two-tone image (two-tone image correction step). This step 13 corresponds to the second feature of the present invention.

Specifically, in this step 13, the skeleton of the subject in the body region included in the photographed image or the reference two-tone image is estimated, and based on the estimated skeleton, the model to be performed in the later model output step 15 is Each reference two-tone image is corrected so that it becomes an image suitable for generation.
Estimation of the subject's skeleton is usually performed based on a reference two-tone image that has been subjected to binarization conversion, but if the subject's skeleton can be estimated with sufficient accuracy, it can be binarized. It may be performed based on a previously captured image.

The method for estimating the subject's skeleton is not particularly limited, and any conventionally known skeleton estimation method can be employed, such as a two-dimensional skeleton estimation method such as OpenPose or PoseNet, or a three-dimensional skeleton estimation method.

The shape of the body region included in the reference two-tone image can be corrected by, for example, correcting the captured image of the body region or the posture expressed in the body region of the reference two-tone image in model generation performed in the model output step 15 later. This is done by making corrections to achieve an appropriate posture. For example, due to distortion of the subject's body, the subject may not be able to take a posture suitable for model generation when being photographed, and in the subject's posture represented in the body area, one shoulder may overlap the other shoulder. The neck may be lower than normal, or the neck may be bent. In such a case, it is conceivable to correct the body region of the reference two-tone image based on the estimated skeleton so that the skeleton represents a posture suitable for model generation.

Note that in this embodiment, the skeleton estimation and subsequent distortion correction are performed on the reference two-tone image in the two-tone image correction step 13; however, in place of or in place of this two-tone image correction step 13, In addition, skeleton estimation and subsequent distortion correction may be performed on the captured image before it is binarized.
Specifically, in method 1 of the present embodiment, instead of or in addition to the two-tone image correction step 13, the skeleton of the subject is estimated from the photographed image, and the estimated skeleton is used to correct the photographed image. It may also include step 13A of correcting the shape of the included body region (photographed image correction step). This photographed image correction step 13A is carried out before the binarization step 12, unlike the two-tone image correction step 13 which is carried out after the binarization step 12.

Next, in step 14, each of the reference two-tone images is defined as a model generation image (definition step).

In this step 14, each of the reference two-tone images corrected in the two-tone image correction step 13 is used as an image to be used when first generating a three-dimensional model of the subject in the next model output step 15. It is defined as the image for initial model generation. In this embodiment, two images, a front image for model generation and a side image for model generation, are generated.

Next, in step 15, a three-dimensional model of the subject is output (model output step).

As described above, in this embodiment, in this step 15, until a three-dimensional model is obtained in which the error between the comparison two-tone image generated from the three-dimensional model and the reference two-tone image based on the photographed image is sufficiently small, The first feature of the present invention is that the three-dimensional model is repeatedly generated while modifying the two-tone image that is the basis for three-dimensional model generation.
This step 15 includes substeps 21 to 24 shown in FIG. 2, and by repeating these as necessary, a highly accurate three-dimensional model can be generated. These substeps 21 to 24 will be explained in detail below.

In the model output step 15 of this embodiment, first, in the sub-step 21, a three-dimensional model of the subject is generated from the model generation image, which is a two-dimensional image, using body template data and a convolutional neural network ( model generation substep).

In this sub-step 21, for example, HS-Nets described in Patent Document 2 can be used as a three-dimensional model generation method using body template data and a convolutional neural network. In three-dimensional modeling using HS-Nets, based on the body shape template data, the front image for model generation and the side image for model generation, which include body regions corresponding to the subject photographed from the front and side, respectively, are extracted. A three-dimensional model corresponding to the orientation and posture of the subject included in the image can be directly generated.

The body shape template data is learned data generated from a predetermined group of people. Specifically, the body shape template data may be a template mesh created using machine learning from three-dimensional scan data of a plurality of bodies acquired from a predetermined group of people.
At this time, before the 3D scan data is subjected to machine learning, the raw 3D scan data is preprocessed, and multiple feature points (landmarks) are manually specified on the preprocessed 3D scan data. It is desirable that Preprocessing that may be performed here includes, for example, removing noise components in the three-dimensional scan data, manually correcting missing parts, and smoothing the surface of the three-dimensional scan data.

The third feature of this embodiment of the present invention is that the body shape template data is composed of body shape templates generated from a group of people having one or more attributes of the subject. are doing. Here, one or more attributes of the target person include the target person's race, nationality, age, gender, body composition value, body shape such as BMI, etc., for example, the target person's race or nationality. .
Typical HS-Nets models a wide range of human body shapes, so they use body templates learned from data on a wide range of people with little bias, but models can be generated for a wide range of human figures. As a trade-off, there was the problem that sufficient accuracy could not be obtained. In this embodiment, by using a body shape template generated from a group of people whose body shape tendency is similar to the target person, such as the same race or nationality as the target person, high precision that can be used for such purposes is achieved. It becomes possible to generate a model.

The three-dimensional model generated in this sub-step 21 is composed of, for example, about 10,000 polygons. However, the number of polygons constituting the three-dimensional model is not particularly limited, and can be determined as appropriate depending on the required computational resources and the required smoothness of the curve in the three-dimensional model.

Next, in sub-step 22, two-tone comparison images corresponding to each of the model generation images are generated based on the three-dimensional model of the subject (comparison image generation sub-step).

Specifically, in this sub-step 22, comparative two-tone images are generated for each of the postures and orientations of the subject's body represented in the body region of the model generation image. These comparison two-tone images are images to be compared with the reference two-tone image in the subsequent image comparison sub-step 23, and are images of a subject with a posture and orientation corresponding to the body region included in the reference two-tone image. The image includes a body region corresponding to the three-dimensional model, and a background region corresponding to a region other than the body region. The body region and background region included in the comparison two-tone image are respectively represented by the same first color and second color as the body region and background region included in the reference two-tone image.

In this embodiment, since the three-dimensional model is generated using HS-Nets, in the model generation sub-step 21 described above, a front view having a posture and orientation corresponding to the body region included in the front image for model generation is generated. A three-dimensional model and a side three-dimensional model having a posture and orientation corresponding to the body region included in the model generation side image are respectively generated. Therefore, in this embodiment, by binarizing these front three-dimensional models and side three-dimensional models, a comparison front two-tone image with a posture and orientation corresponding to the body region included in the reference two-tone image is created. and a comparison side two-tone image can be generated.

Next, in sub-step 23, each of the body regions of the reference two-tone image and each of the body regions of the comparison two-tone image are compared to calculate a difference value between the two (comparison sub-step).

The comparison of images and the calculation of the difference value in this sub-step 23 can be performed, for example, by superimposing the reference two-tone image and the comparison two-tone image, respectively. At this time, in order to more accurately measure the difference between the two images, the area of the area where the body area in the reference two-tone image and the body area in the comparison two-tone image do not overlap is minimized. It is desirable to calculate the difference value after adjusting and overlapping the two images.
For example, when adjusting the positions of two images, it is desirable to adjust the vertical direction (height direction) of the body regions included in these images so as to fit them, and then superimpose them. In addition, the two images may be adjusted to reduce or eliminate the influence of the way the limbs are spread in the body region, and the lengths of the torso and legs in the body region may be estimated by step 13 (or step 13A) described above. The superimposition may be performed after correction based on the obtained skeleton.
Also,

The difference value calculated by comparing two images can be determined, for example, by calculating the difference in distance between the body regions of the two images and integrating the difference.

In this sub-step 23, the two images may be compared as a whole to calculate one difference value, or each specific region within the two images may be compared to calculate a plurality of difference values. Good too. For example, the difference value may be calculated for each of a plurality of predetermined body parts in a plurality of body regions. For example, by comparing two images, difference values may be calculated for each of the body parts such as the chest, torso, right arm, left arm, right leg, left leg, etc.

Next, in sub-step 24, if the difference value is larger than a predetermined threshold, and if the number of executions of this model output step 15 is less than the predetermined number, the comparison two-tone image is modified and the model generation image is The image is changed to a corrected two-tone image for comparison (image correction substep).

Specifically, in this sub-step 24, if the difference value calculated in the above-mentioned comparison sub-step 23 is less than or equal to a predetermined threshold, this model output step 15 is ended and the process proceeds to the next measurement step 15. do. On the other hand, if the difference value calculated in the comparison sub-step 23 described above is larger than the predetermined threshold, in other words, the difference between the reference two-tone image and the comparison two-tone image is large, and the model generation sub-step 21 generates If the generated three-dimensional model does not represent the subject's body accurately enough, a two-tone image for comparison may be used to re-execute this model output step 15 using another image for model generation. The image for model generation is changed to the corrected two-tone image for comparison.

The predetermined threshold value may be arbitrarily determined depending on the desired accuracy of the three-dimensional model to be generated.
In addition, when a plurality of difference values are calculated, if at least one difference value among the plurality of difference values is larger than a predetermined threshold value, it may be determined that the present model output step 15 is executed again. If the average of the difference values is larger than a predetermined threshold value, a determination may be made to execute the main model output step 15 again. Furthermore, different threshold values may be determined for each of a plurality of regions in the image for which difference values have been calculated, and the difference values and threshold values may be compared for each of these regions.

However, depending on the reference two-tone image used, there may be cases in which the difference value does not become less than the predetermined threshold even if this model output step 15 is repeated many times, and it is no longer possible to expect this. In order to stop repeating this step 15 when such a situation occurs, in this sub-step 24, even if the difference value calculated in the above-mentioned comparison sub-step 23 is larger than a predetermined threshold value, When the number of executions of this model output step 15 reaches a predetermined number of times, the repetition of this model output step 15 is completed, and the process moves to the next measurement step 15.
Here, the predetermined number of executions of this step 15, which is set to determine the end of the repetition of this step 15, can be arbitrarily determined depending on the computational resources and time required for generating the three-dimensional model. It is determined. Furthermore, if it is desired to avoid stopping the repetition as much as possible, the predetermined number of executions may be set to a sufficiently large value (for example, the maximum value that can be set).

The comparative two-tone image may be modified by any method. For example, the comparison two-tone image can be corrected by removing noise components estimated based on the difference values from the comparison two-tone image.
Alternatively, the comparison two-tone image can be modified by correcting the dimensions of the comparison two-tone image. Here, the dimension correction may be performed on the entire comparison two-tone image, or may be performed on one or more specific body parts in the body region within the comparison two-tone image. For example, in the comparison sub-step 23 described above, when the body region corresponding to the right arm of the subject is compared and the calculated difference value is relatively large, the region corresponding to the right arm of the body region in the two-tone image for comparison The dimensions of the image may be corrected to be larger or smaller so as to approach the reference image.
Note that the methods listed here may be performed alone or in combination of a plurality of methods.

After the comparison two-tone image is corrected in this sub-step 24, the corrected comparison two-tone image is set as a new model generation image, and the main model output step 15 is executed again. The re-executed model output step 15 starts again with sub-step 21, generates a new three-dimensional model from this newly set model generation image, and evaluates this in sub-steps 22-24.

In this way, in this model output step 15, the three-dimensional model is used to generate the three-dimensional model until it is confirmed that the generated three-dimensional model accurately represents the body of the photographed subject with the desired accuracy. Change the model generation term image and repeat generation of the 3D model. As a result, method 1 of the present embodiment can generate a highly accurate three-dimensional model.

Then, in step 16, if the last calculated difference value is less than or equal to a predetermined threshold, one or more of the body characteristics related to the subject's body characteristics are calculated based on the last generated three-dimensional model of the subject. Measure dimensions (measuring step).

The dimensions related to the physical characteristics of the subject measured in this step 16 may be any dimensions such as length, height, circumference, etc. related to any body part of the subject. For example, it may be measured as a costume size that fits the subject's body. The relevant dimensions include, for example, the subject's shoulder width, chest circumference, waist circumference, neck circumference, arm circumference, waist circumference, wrist circumference, thigh circumference, waist, hip, back length, sleeve length, sleeve length, rise length, inseam length, etc. could be.

Note that when measuring the dimensions, reference may be made to the physical information of the subject that has been acquired in advance. For example, if the value of the height of the subject is given in advance, the dimension can be calculated based on the ratio of the height of the subject to the three-dimensional model.

As described above, in the method 1 of the present embodiment, the model output step 15 of outputting a three-dimensional model of the subject includes a comparison two-tone image generated from the three-dimensional model and a reference two-tone image based on the photographed image. By repeating substeps 21 to 24 as necessary until a three-dimensional model with sufficiently small errors is obtained (first feature), a three-dimensional model in which the subject's body is modeled with high accuracy is output. Based on such a three-dimensional model, the physical characteristics of the subject can be measured with high accuracy.

Furthermore, in method 1 of the present embodiment, after the binarization step 12, a two-tone image correction step 13 is executed to correct distortion of the subject's body region that may occur when acquiring the captured image ( 2), a three-dimensional model of the subject can be modeled with high precision in the model output step 15, and based on such a three-dimensional model, the physical characteristics of the subject can be measured with high precision. can.

Furthermore, in method 1 of the present embodiment, the body shape template data is composed of body shape templates generated from a group of people having one or more attributes of the subject (third feature). In the output step 15, a three-dimensional model of the subject can be modeled with high precision, and based on such a three-dimensional model, the physical characteristics of the subject can be measured with high precision.

Thereby, a three-dimensional model of the subject can be modeled with high precision, and based on such a three-dimensional model, the physical characteristics of the subject can be measured with high precision.

FIG. 3 shows one embodiment of a system 3 that includes a computer capable of performing the body measurement method shown in FIG.
Note that the body measurement method shown in FIG. 1 does not necessarily need to be executed by the system 3, but can be executed by any computer system.

The system 3 includes a user terminal 31 and a server 32 as a computer that executes the body measurement method shown in FIG.

The user terminal 31 may be any terminal operated by the subject or another person at a location remote from the server 32 and capable of transmitting and receiving data to and from the server 32 via a network. For example, the user terminal 1 may be a desktop computer, a mobile computer, a tablet computer, a mobile phone, a smart phone, a wearable device, a handheld device, any embedded device, etc.

The server 32 may be any server capable of computing data transmitted from the user terminal 31 via the network. For example, the server 32 may be an on-premises server configured with one or more computer resources, or may be a cloud server implemented on a cloud network.

By transmitting and receiving data between the user terminal 31 and the server 32 via the network, the user terminal 31 and the server 32 can arbitrarily share and execute each step executed in the body measurement method of the present invention, which will be explained in detail below. can do.
For example, while the user terminal 31 executes steps for acquiring data that can be obtained near the subject, such as photographing the subject, and steps that perform processing with a relatively low computational load, the user terminal 31 The step of performing processing with a large calculation load may be executed in the server 32 which has more abundant calculation resources than the server 32 .

Specifically, in the system 3 of this embodiment, the user terminal 31 includes an image acquisition module 311, a binarization module 312, and a communication module 310. The user terminal 31 further includes a camera 315 and a depth sensor 316.
The image acquisition module 311 is configured to execute the image acquisition step 11 described above, and specifically, is configured to acquire a captured image by a camera 315 provided in the user terminal. The image acquisition module 311 is also configured to perform distortion correction processing on the captured image using depth data acquired by the depth sensor 316.
The binarization module 312 is configured to perform the binarization step 12 described above.
The communication module 310 is configured to send and receive data to and from the communication module 320 of the server 32, and, as an example, performs subsequent processing on the reference two-tone image generated by the binarization module 312 on the server 32. to the communication module 320 of the server 32.

The server 32 also includes a communication module 320, a two-tone image correction module 323, a definition module 324, a model output module 325, and a measurement module 326.
The communication module 320 is configured to transmit and receive data to and from the communication module 310 of the user terminal 31, and receives, for example, a reference two-tone image transmitted from the communication module 310 of the user terminal 31.
The two-tone image correction module 323 is configured to perform the two-tone image correction step 13 described above.
Definition module 324 is configured to perform definition step 14 described above.
Model output module 325 is configured to perform model output step 15 described above.
Measurement module 326 is configured to perform measurement step 16 described above.

Here, the model output module 325 includes a model generation sub-module 421, a comparison image generation sub-module 422, a comparison sub-module 423, and an image correction sub-module 424, as shown in FIG.
These sub-modules 421-424 are configured to perform the above-described sub-steps 21-24, respectively.

As described above, in the system 3 of the present embodiment, the user terminal 31 performs, among the steps included in the method 1 of the present invention, the image acquisition step 11 in which a photographed image can be acquired from the subject and the calculation load. While the server 32 is configured to execute the binarization step 12, which is relatively small, the server 32 is configured to execute the two-tone image correction step 13 and the model output step 15, which require a large calculation load.
Therefore, the system 3 of this embodiment can efficiently distribute the calculation load between the user terminal 31 and the server 32.

Note that in the system 3 of the present embodiment, the server 32 is equipped with a two-tone image correction module 323 configured to execute the two-tone image correction step 13; , the user terminal 31 may include a module 313 (not shown) configured to perform step 13A.

FIG. 5 shows an example of a photographing screen 50 displayed on the application 5 used when photographing a photographed image acquired in the image acquisition step of the method of the present invention (for example, in step 11 of the above embodiment). show. Application 5 may run on user terminal 31 of system 3, for example. The following description will be made on the assumption that the user terminal 31 is a smartphone and the application 5 is a smartphone application.

As shown in FIGS. 5A and 5B, a guide frame 51 corresponding to the posture and orientation of the subject required for imaging is displayed on the imaging screen 50 of the application 5. Here, the pose shown in FIGS. 5(a) and 5(b) is the A pose, the orientation shown in FIG. 5(a) is frontal, and the orientation shown in FIG. 5(b) is lateral. It is possible.

When the photographer uses the application 5 to photograph a subject, the image reflected in the lens of the camera 315 provided on the user terminal 31 on which the application operates is displayed on the photographing screen 50, and the guide frame 51 It continues to be displayed on the shooting screen 50 so as to overlap with the video.
When photographing the body of the subject, the photographer uses the camera 315 (and if necessary, the camera 315 is installed) so that the subject's body is approximately within the guide frame 51. The user terminal 31) adjusts the orientation of the user terminal 31), the distance to the subject, the magnification on the photographing screen 50 of the image reflected in the lens and displayed on the photographing screen 50, etc. After such adjustment allows the subject's body to sufficiently fit within the guide frame 51 on the photographing screen 50, the photographer photographs the subject's body.

In this way, the guide frame 51 displayed on the photographing screen 50 of the application 5 makes it possible to easily photograph the subject in the posture and orientation required in the later model output step (for example, step 15). , it is possible to easily obtain captured images with little distortion that are suitable for model generation.

When a photographer photographs a subject, if the subject's body is not sufficiently within the guide frame 51 on the photographing screen 50, the application 5 does not permit photography, and is configured to activate the touch area in the range where the characters "Shoot" are displayed on the screen of the application 5 to permit shooting only when the image is sufficiently within the guide frame 51. It's okay.
When the application 5 is configured as described above, the photographer can more easily photograph the subject in the posture and orientation required for photographing.

The application 5 is mainly used to capture at least one captured image obtained in the image acquisition step, but is also used to obtain other information useful in carrying out the body measurement method of the present invention. Good too.
For example, the application 5 may further include an interface for the subject to directly input useful information such as the subject's height, weight, age, etc. that can be referenced in measurements based on the three-dimensional model.

Note that the body measurement method executed by the computer according to this embodiment is not limited to the configuration of the above embodiment. Further, the body measurement method executed by the computer according to the present invention is not limited to the above-described effects. The body measurement method executed by the computer according to the present invention can be modified in various ways without departing from the gist of the present invention.

For example, method 1 shown in FIG. 1 includes substeps 21 to 24 of repeating the generation of a three-dimensional model according to the above-mentioned first feature of the present invention, and based on skeletal estimation according to the above-mentioned second feature of the present invention. 13 (or step 13A) of correcting the image, the method of the invention does not necessarily have both of these features.

For example, if the body measurement method of the present invention includes substeps 21 to 24 of repeatedly generating a three-dimensional model, step 13 (or step 13A) of correcting the image based on skeletal estimation is not necessarily performed. It's okay. Similarly, if step 13 (or step 13A) of correcting the image based on skeleton estimation is executed, the substeps 21 to 24 of repeating the generation of the 3D model are not performed, and the 3D model is generated only once. It may be the person who executes it. The body measurement method of the present invention can measure the subject's body characteristics with high accuracy even when only one of the steps related to these characteristics is included.

In addition, although a more detailed explanation will not be repeated here, even if the matters are not directly described above, conventionally known technical matters regarding body measurement methods will be included in the present invention. can also be adopted as appropriate.

Claims (10)

A computer-implemented method for measuring the body of a subject, the method comprising:
an image obtaining step of obtaining at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region;
Each of the photographed images is converted into a body region represented by a first color corresponding to the body region of the photographed image and a background region represented by a second color corresponding to the background region of the photographed image. a binarization step of generating reference two-tone images corresponding to the photographed images, respectively;
a definition step of defining each of the reference two-tone images as a model generation image;
a model output step of outputting a three-dimensional model of the subject,
a substep of generating a three-dimensional model of the subject from the model generation image using body shape template data and a convolutional neural network;
a substep of generating comparative two-tone images corresponding to each of the model generation images based on the three-dimensional model of the subject;
a substep of comparing each of the body regions of the reference two-tone image and each of the body regions of the comparison two-tone image to calculate a difference value between the two;
If the difference value is larger than a predetermined threshold, and if the number of executions of this model output step is less than a predetermined number of times, the comparison two-tone image is corrected, and the model generation image is changed to the corrected comparison image. A model output step, comprising: a substep of changing to a two-tone image and executing the model output step again;
If the last calculated difference value is less than or equal to a predetermined threshold, one or more dimensions related to the body characteristics of the subject are measured based on the last generated three-dimensional model of the subject. and a measuring step. The method according to claim 1, wherein the at least one captured image includes an image captured from the front of the subject and an image captured from the side of the subject. The method according to claim 1 or 2, wherein the at least one captured image is an image captured by a user terminal of the subject. The method according to claim 3, wherein the at least one captured image is an image that has been subjected to distortion correction based on depth data acquired simultaneously with the capturing of the captured image by a depth sensor provided in the user terminal. . The at least one photographed image is an image photographed on an application running on the user terminal, and the photographing screen of the application includes a guide frame corresponding to the posture and orientation of the subject required for photographing. The method according to claim 3 or 4, wherein: is displayed. Before the binarization step, a photographed image correction step of estimating the skeleton of the subject from the photographed image and correcting the shape of the body region included in the photographed image using the estimated skeleton, or ,
Before the defining step, the skeleton of the subject is estimated from the photographed image or the reference two-tone image, and the shape of the body region included in the reference two-tone image is corrected using the estimated skeleton. A method according to any one of claims 1 to 5, further comprising a two-tone image correction step. The method according to any one of claims 1 to 6, wherein the body shape template data is constituted by a body shape template generated from a group of people having one or more attributes of the subject. 8. The method of claim 7, wherein the one or more attributes of the subject include one or more of the subject's nationality or race. The difference value is obtained by superimposing the reference two-tone image and the comparison two-tone image, respectively, and at that time, the body area in the reference two-tone image and the comparison two-tone image are superimposed on each other. The method according to any one of claims 1 to 8, wherein the method is calculated by adjusting so that the area of a region that does not overlap with the body region is minimized. According to any one of claims 1 to 9, the modification of the comparison two-tone image includes performing at least one of removing a noise component estimated based on the difference value and correcting a dimension. the method of.

Images