WO2022154685A1

WO2022154685A1 - Smart system for estimating parameters and recommending clothing on the basis of 3d-scan images of consumers

Info

Publication number: WO2022154685A1
Application number: PCT/RU2021/000010
Authority: WO
Inventors: Максим Александрович ФЕДЮКОВ; Андрей Владимирович ПОСКОНИН; Олег Владимирович КОРНЕЕВ; Иван Александрович СИДОРОВ; Сергей Михайлович КЛИМЕНТЬЕВ; Наталья Дмитриевна ХАНБЕКОВА; Андрей Васильевич КОВАЛЬ
Original assignee: Общество С Ограниченной Ответственностью "Тексел"
Priority date: 2021-01-13
Filing date: 2021-01-13
Publication date: 2022-07-21

Abstract

A system for providing a recommendation when choosing clothing, comprising: a user device, which comprises a photographic camera, a communication means and a graphical user interface and is capable of generating images of the user, transmitting said images to a server, receiving recommendations from the server and delivering said recommendations to the user; and a server, which comprises a database of clothing measurements and is capable of receiving images of the user from the user device, generating user measurements based on said images, comparing the user measurements with the clothing measurements, sorting the result of the comparison to generate a fit rating, generating a recommendation and sending the recommendation to the user device. The recommendation may be a recommendation of a size of clothing of a particular brand or a recommendation of a particular item of clothing.

Description

INTELLIGENT SYSTEM FOR EVALUATION OF PARAMETERS AND RECOMMENDATIONS OF CLOTHES BY 3D-CKAHOTPAMMAM OF CONSUMERS

Technical field

The invention relates to the field of electronic distance trading. In particular, it relates to means for remote selection of garments for users of this system based on the users' RFP scans.

State of the art

The SARS-CoV-2 (COVID-19) virus epidemic and the unprecedented measures taken by governments to contain the spread of coronavirus infection have become one of the biggest changes in the retail industry in history. Thus, according to Sberbank analysts, during the first week of quarantine in Russia, the demand for clothing fell by 94%. According to McKinsey estimates, the fall in retailer revenue in 2020 could reach 27-30%. There has been a dramatic redistribution between apparel distribution channels (in 2019, 70% of sales came from offline channels, 30% from online channels) due to full store closures in many countries.

These events led to the deterioration of the financial situation and the bankruptcy of many retailers, mainly focused on offline sales. It is predicted that the introduced safety requirements (disinfection after each fitting, which leads to a rapid loss of presentation and an increase in costs, restrictions on the operation of trading floors, etc.) imposed on offline retailers by supervisory authorities and buyers will become an additional incentive for the speedy introduction of contactless fitting technologies. Despite the temporary nature of restrictive and prohibitive measures, the long-term trend towards an increase in the share of online sales channels due to a reduction in offline channels will accelerate. Almost since the beginning of the crisis, there has been a significant increase in customer interest in smartphone-based mobile virtual fitting solutions.

At the same time, to date, there is no known commercial implementation of a remote recommendation system when choosing clothes, which generates recommendations for the size of clothes of a particular brand or recommendations for a particular item of clothing for its purchase or rental.

Disclosure of invention

The system for recommendations when choosing clothes contains: - a user device containing a camera, a communication device and a graphical user interface and configured to generate user images, transmit them to the server, receive recommendations from the server and bring the recommendations to the user; and

- a server containing a database of clothing dimensional features, configured to receive user images from the user's device, generate user dimensional features based on them, compare the user's dimensional features with clothing dimensional features, sort the matching result to generate a fit rating, generate a recommendation and send a recommendation user device.

The clothing recommendation method is performed by the system containing the user's device and the server, and includes the following steps:

- formation by the device of the user of images of the user;

- transmission by the device of the user of images of the user to the server;

- formation by the server on the basis of user images of dimensional features of the user;

- comparison by the server of the dimensional characteristics of the user with the dimensional characteristics of clothing;

- sorting by the server of the comparison result to form the landing rating;

- formation by the server of recommendations based on the result of sorting;

- sending a recommendation to the user's device by the server; and

- bringing the user's device recommendations to the user.

The clothing recommendation may be a specific brand size recommendation or a specific garment recommendation.

Brief description of the drawings

In FIG. 1 is a flow chart of the invention in clothing size recommendation mode.

In FIG. 2 is a flow chart of the invention in garment recommendation mode.

In FIG. 3 shows an example of the presentation of landing information in the system according to the invention.

In FIG. 4 shows an example of visualization for the user of the measurement results in the system according to the invention. In FIG. 5 is a graphical illustration of the grid search method in two dimensions.

In FIG. 6 shows a diagram of the stratification of the test part of the dataset into K parts.

In FIG. 7 is a diagram of combining recommendations from various embodiments of the recommendation algorithm in the system according to the invention.

In FIG. 8 shows an example of an algorithm for the method of typing men by body type.

In FIG. Figure 9 shows an example of an algorithm for typing women by body type.

Implementation of the invention

An intelligent system for recommending clothes based on 3D scans of consumers in general provides the following functions:

- recommendation of clothing based on comparison of 3D scans of people and clothing specifications/parameters;

- typing of 3D-scanograms of people by figure and color profile;

- recommendation of clothing based on typing of 3V-scans of people;

- combining a detailed scan of a human head with a parametric model;

- texture filtering of 3D scans of people using machine learning methods;

- providing visualization of 3D scans of people in the recommended clothing using digital templates.

The present invention is directed to the implementation of the first of the above functions - the development of clothing recommendations based on a comparison of 3D scans of people and specifications or parameters of clothing items.

Input actions (signals, information data, mechanical actions, etc.) necessary for the performance of the specified function by the intelligent system: user action with the help of manipulators through the human-machine interface, input of initial and derived data of 3D-scanograms of individual consumers.

Output reactions (signals, information data, actions, etc.) provided by the product as a result of the performance of the specified function by the intelligent system: providing the consumer with a clothing recommendation. The present invention implements a method and a system for recommending clothing based on a comparison of the RF scans of people and the dimensional characteristics of a person determined from them with clothing parameters obtained directly for each item of clothing, or extracted from clothing specifications (the so-called spec sheet), which are part of technological packages (the so-called tech pack).

The method and system of the present invention has the ability to operate in two modes: clothing size recommendation mode and clothing item recommendation mode. Recommendations are formed by comparing the user's measurements and dimensional characteristics of garments or dimensional grids.

In FIG. 1 shows a block diagram of the algorithm in the clothing size recommendation mode. When working in the clothing size recommendation mode, the algorithm accepts full-length frontal and side photos of the user, taken using a special holder (delivery, tripod), and size grids of brands, their types and subtypes. Photos can be taken with any camera, including the camera of a mobile phone, tablet, laptop or other mobile device, a camera of a desktop computer, smart TV, smart home communication center, etc. Uploading user photos to the system can be done through a mobile device, computer, smart TV, etc. and using an internet connection or alternative image delivery methods (such as MMS messages).

Size grids of brands, their types and subtypes are retrieved from the database of size grids. Dimensional mesh data is parsed, as a result of which a set of dimensional features of dimensional meshes is formed.

After receiving a request for a recommendation (for example, through a mobile application or through a browser application on the user's device), the system receives data from the user that characterizes the user's figure, or extracts this data from the user's profile. Such data is generally height, weight, a portrait photograph of the user, at least one full-length frontal photograph, and at least one full-length side photograph. Based on these data, the system determines the dimensional characteristics of the user. Algorithms for determining the dimensional characteristics of a user from photographs are known and are not described here for brevity. An example is the international application PCT/IB2020/053263 of the same applicant, which describes an algorithm for parameterizing the user's body based on a set of his photographs, and all of which is incorporated into this application by reference. Additionally, the user can enter the results of traditional body measurements (shoulder width, arm length, leg length, chest circumference, waist circumference, etc.). which can be directly or indirectly taken into account when determining the dimensional features of the user.

Then the dimensional features of the dimensional grids are compared with the dimensional features of the user. Based on this mapping, a fit rating is calculated for each size grid and the results of the calculation are stored in an array of fit ratings.

Then, taking into account the fit ratings, the sizes are sorted, the results of which are stored in a sorted array of fit ratings.

After that, on the basis of previously obtained data, the most suitable size for a given user is determined and communicated to the user through the user interface in text and/or graphic and/or sound form.

In FIG. 2 is a block diagram of the algorithm in the mode of recommending garments. When operating in the garment recommendation mode, the method algorithm takes as input data on specific target garments, including their sizes. The operation of this algorithm is essentially similar to that of the clothing size recommendation mode, with the difference that the fit rating is determined for the target garments and not just for the sizes in the size grid.

Algorithms for matching dimensional features of a user with dimensional features of size grids and/or garments are known and are not described here for brevity. As an example, we can cite the international application PCT/IB2020/053263 of the same applicant, which describes an algorithm for comparing the dimensional characteristics of a user with dimensional characteristics of dimensional grids and garments and an algorithm for ranking the results of such a comparison, and all of which is incorporated into this application by reference.

In FIG. 3 shows an example of landing information presentation, and FIG. 4 shows an example of visualization for the user of the measurement results in the system according to the invention. These measurement results are shown in Fig. 4 in black, in practice, in the graphical user interface, they can be presented in color, while the color can carry information about the degree of compliance of the corresponding dimensional characteristics of the garment with the dimensional characteristics of the user, for example, green - fits (the dimensional attribute is just “just right”) , red - tight (size sign "small"), blue - loose (size sign "large").

Since the algorithm uses many hyperparameters, the system according to the invention has the ability to fine-tune them to based on the labeled data set. For this, the grid search method is used. This method performs an exhaustive search over a manually specified subset of the hyperparameter space of the algorithm. When using the grid search method, it is necessary to do cross-validation on the training dataset. To do this, the data set is divided proportionally into K parts. After that, the algorithm is run on K-1 parts of the training dataset, using the remaining one part as a validation part.

In FIG. 5 is a graphical illustration of the grid search method in 2D space, and FIG. 6 is a diagram of the stratification of the test part of the dataset into K parts.

Since the system labels data clusters with the highest accuracy, it is possible to label on pre-computed hyperparameters, and to improve accuracy, it is also possible to mix the results of recommendations of two or more hyperparameters. In FIG. 7 is a diagram of combining recommendations from various embodiments of the recommendation algorithm in the system according to the invention.

In many cases of practical use of the system according to the invention, the quality of the input data received from the user is insufficient for the reliable operation of the recommendation algorithm (the data may be incomplete, erroneous, sometimes deliberately distorted). To reduce the negative impact of data deficiencies on the result of the recommendation, the system has been supplemented with the ability to issue recommendations taking into account the typing of user figures.

Typing algorithms are necessary to correctly recommend clothes to the user by category, style, material, color, etc. For example, the algorithm should not recommend a fitted shirt to a man with a protruding belly: such a shirt will be tight in the stomach and will be excessively loose in the neck and shoulders. Thus, the algorithm should recommend clothes that compensate for or hide body imperfections, such as straight and loose-fitting clothes. For men with an athletic figure, on the contrary, the algorithm should recommend fitted clothing, as it will emphasize the athletic silhouette.

Within the main types, subtypes are distinguished, each of which bears the characteristics of two main types. The morphological characteristics of the physique of men according to Bunak are presented in Table. one .

Table 1. Characteristics of the physique of men

The morphological characteristics of the physique of men according to Bunak are presented in Table. 2.

Table 2. Characteristics of body types of women

As an example, in FIG. 8 and FIG. 9 shows one of the ways of typing a person - according to the type of figure: for men (Fig. 8) - 3 types of figures, for women (Fig. 9) - 5 figure types. For the main parameters, the main measurements are taken, such as chest circumference, waist circumference and hip circumference.

The types of figures determined in this way are used to develop recommendations for clothing, presented in Table. 3.

Table 3. Algorithmic recommendations for the type of posture

Table 4. Algorithmic recommendations for the spatial position of the hands

Table 5. Algorithmic recommendations for shoulder height

Table 6. Algorithmic recommendations for the shape of the lower extremities

Algorithms of recommendations based on typing the human body according to basic dimensional features are based on the following principles. In mass production, clothes are made not for an individual, but for a small number of types of figures, which anthropometric clothes provide for the entire initial set of options for consumer figures. Hence, the main goal of the dimensional typology is to identify a limited number of types of figures by a combination of dimensional features that provide maximum population satisfaction with anthropometric clothing. According to the calculations of M. V. Ignatiev, in order to achieve consumer satisfaction with clothing according to two leading features, it is necessary to produce 25 clothing numbers. To achieve the same satisfaction with the three leading features, 125 numbers are needed, and taking into account the five leading features requires more than 1000 clothing numbers.

The degree of connection between two features is measured by the correlation coefficient r _xy , which is determined by the following formula:

For a number of types of clothing, those of them are chosen as the leading dimensional features, the anthropometricity of clothing with which is the most priority. So, for a men's shirt, the neck girth is the most priority, and for bras - the chest girth.

Table 7. Leading dimensional features determined by recommendation algorithms and their boundary values

An important step in recommendation algorithms is to establish the interval between adjacent numbers of typical figures. The values of the intervals for the size features of the body are set by the standard and are 3.5-4.5% of the value of the size feature. In the Russian clothing industry, the interdimensional interval for the circumference of the chest, waist and hips is taken to be ± 2 cm, for the body length ± 3 cm. The inter-weight interval for the circumference of the hips and waist of the adult population is ± 2 cm, and for children, the waist circumference is ± 3 cm.

When solving the problem of determining the optimal number of types of figures, a three-sigma limit was taken, all figures were identified as typical, the frequency of which is at least 0.1% (at least one person per thousand). Thus, 360 types of figures were established for the male population and 509 types for the female population. The Russian standard for the production of clothing established 172 typical figures for men and 137 typical figures for women. For each type, the frequency of occurrence has been established for the whole country and for individual regions.

OST 17-326-81 establishes 137 types of figures for the production of women's clothing, which are divided into 4 full notes, 9 size and 3 age subgroups.

The completeness is set as the difference between O and Om, equal to:

Group I - 4 cm - low fullness;

Group II - 8 cm - medium fullness;

Group III - 12 cm - large fullness;

Group IV - 16 cm - very large fullness. The entire range of sizes from 88 to 136 (13 numbers) is divided into three subgroups: small sizes 88-104, large sizes 108-120 and extra large sizes 124-136.

The first complete group includes only a group of small sizes, the second - all three subgroups, the third - a subgroup of small and large sizes and the fourth - a subgroup of small sizes, thus, the entire range of sizes in female typology contains 7 subgroups.

For women, the standard provides for 6 heights from 146 to 176 cm, and the 6th height (176 cm) is provided only for the VII size group of the 1st weight subgroup.

Each weight group corresponds to a certain combination of age groups.

The typology of the male population is presented by OST 17-325-86 “Sewing, knitted, fur products. Figures of men are typical. Dimensional signs for designing clothes.

The standard establishes 172 typical figures for the production of clothing, grouped into 5 full-length groups and two sheet rows. The first row includes sizes from 84 to 104, the second - from 108 to 128.

To establish typical figures, the interval for height is assumed to be ± 3 cm, for chest girth + 2 cm, for waist girth ± 2 cm. The inter-full interval for waist girth is ± 3 cm.

Clothing is marked with three digits: the first indicates height, the second - the full chest circumference W and the third digit - the full circumference of the hips for women and the waist for men and children, for example: 158-96-100 indicates W height, 48 size, 1st fullness (for women).

All figures of women are divided into three size series according to the Ogsh attribute: small sizes 76-104, medium 104-128 and large 128-146 cm.

The interdimensional interval for figures with Ogsh from 76 to 104 cm is 4 cm, and from 104 to 128 - 6 cm.

Only 3 subgroups were distinguished by height: small - 156 cm, medium - 164 cm and large - 172 cm. The interval for growth is 8 cm.

The completeness is defined as the difference between Gb and Gg, i.e. P \u003d About - Ogsh. There are three fullness: small Pm = 0, medium Ps = 6 cm and large Pb = 12 cm.

Table 8. Numbering of women's clothing for a figure of average build

Table 9 - Numbering of men's clothing for figures of normal build

Clothing is made for figures with a height of 162 to 190 cm.

Very short R - 162 cm,

Short S - 168 cm,

Medium M - 174 cm,

Long L - 180 cm.

Extra long XL - 186 cm.

Example: A sports athletic figure of a dolichomorphic type of body proportions. The group includes figures from 44 to 52 sizes, from 168 to 190 cm in height (8-X). Basic figure: 50, 8 / 174.

An example of the implementation of the figure analysis algorithm for women

1. Body type (waist-to-hip ratio: straight, semi-fitted, fitted)

2. Height: (up to 165 - low, 166-172 - medium, 173 and above - high)

3. Body length: AF=FK proportional physique. AF is longer - elongated torso, FK is longer - legs are longer than torso.

4. Bust size: if the difference between bust and under bust is less than 8 cm, then the chest is S, if more than 18 cm, then L.

5. Shoulder width and hip width

Proportional: hips wider up to 2.5 cm

More than 2.5 cm: hip correction

Less than 2.5 cm: shoulder correction

6. Arm length: proportional (elbow at waist), long - accessories/layering recommended, short - horizontal accents not recommended

7. Face height and neck height a. Proportional: 2/3 face height b. Shortened: less than 2/3 c. Extended: more than 2/3

An example of the implementation of the figure analysis algorithm for men

1. Body type: a. Triangle: shoulders wider than hips b. Rectangle: the volume of the shoulders, waist and hips within s c. Inverted triangle: hips wider than shoulders d. Rounded: girth of the abdomen above a given threshold

2. Height: a. up to 172: low b. 173-176: middle c. 177 and above: high

3. Body length: a. Proportional torso: AF=FK b. Long torso: AF>FK c. Long legs: AF<FK

When implementing an embodiment of the invention, the system described above had the following performance:

- time to provide a clothing recommendation based on a comparison of 3D scans of people and clothing specifications / parameters (from the moment a 3D scan is received to the moment a recommendation is issued): up to 10 minutes;

- parameter of matching accuracy of 3 D-scans of people and clothing specifications/parameters - average matching error: up to 3 cm;

- tAP (Mean Average Precision, average normal accuracy) of the machine learning algorithm of the recommender system: 0.8;

- RMSE (Root-Mean-Square Error, standard deviation) of the recommender system machine learning algorithm: up to 5 cm;

- MAE (Mean Absolute Error, mean absolute deviation) of the recommender system machine learning algorithm: up to 3 cm;

- depth of the neural network of the machine learning algorithm of the recommender system: 18 layers.

Claims

CLAIM

one . A clothing recommendation system, comprising:

- a user device containing a camera, a communication device and a graphical user interface and configured to generate user images, transmit them to the server, receive recommendations from the server and bring the recommendations to the user; and

2. The system of claim. 1, in which the recommendation is a recommendation of the size of clothing of a particular brand.

3. The system of claim 1, wherein the recommendation is a recommendation for a specific garment.

4. The system according to claim. 1, in which the recommendation sent to the user's device is formed from several recommendations.

5. The system according to claim. 1, in which the dimensional features of the user are formed by the server, taking into account the results of typing the user's figure.

6. The method of recommending when choosing clothes, performed by a system containing a user's device and a server, including the following actions:

- formation by the device of the user of images of the user;

- transmission by the device of the user of images of the user to the server;

- sorting by the server of the comparison result to form the landing rating;

- formation by the server of recommendations based on the result of sorting;

- sending a recommendation to the user's device by the server; and

- bringing the user's device recommendations to the user.

7. The method of claim 6, wherein the recommendation is a recommendation for a specific brand clothing size.

8. The method of claim 6, wherein the recommendation is a recommendation for a specific garment.

9. The method according to claim 1, in which the recommendation sent to the user's device is formed from several recommendations.

10. The method according to claim 1, in which the dimensional features of the user are formed by the server, taking into account the results of typing the user's figure.