WO2019134560A1

WO2019134560A1 - Method for constructing matching model, clothing recommendation method and device, medium, and terminal

Info

Publication number: WO2019134560A1
Application number: PCT/CN2018/123510
Authority: WO
Inventors: 陈岩; 刘耀勇
Original assignee: Oppo广东移动通信有限公司
Priority date: 2018-01-08
Filing date: 2018-12-25
Publication date: 2019-07-11
Also published as: CN110021061B; CN110021061A

Abstract

The present document discloses a method for constructing a matching model, comprising: acquiring a specified quantity of model images comprising field depth information; constructing a three-dimensional model for each model according to the model image, and marking body type data, outfits, and accessories corresponding to the three-dimensional models to obtain a first image sample; using a set machine learning algorithm to train, according to an image sample, a preset deep neural network to obtain a matching model, wherein the image sample comprises the first image sample. The document further discloses a clothing recommendation method, a matching model construction device, a clothing recommendation device, a storage medium, and a terminal.

Description

Collocation model construction method, clothing recommendation method, device, medium and terminal

The present application claims priority to Chinese Patent Application No. 20181001539, filed on Jan. 08.

Technical field

The embodiments of the present application relate to mobile terminal technologies, for example, to a collocation model construction method, a clothing recommendation method, an apparatus, a medium, and a terminal.

Background technique

With the rapid development of the economy, the variety and style of apparel products have become extremely rich and complex, and at the same time, people's requirements for clothing and accessories are also increasing. Faced with a variety of styles and styles, people often want professional and scientific advice.

In the related art, the user's clothing matching knowledge is generally obtained by reading a fashion magazine; some apparel providers try to display the clothing matching scheme provided by the designer of the clothing brand through a display screen set in the shopping mall, thereby providing the user with the clothing matching solution. The brand's clothing recommendations, but this way of providing advice with the support is limited in intelligence, and can not achieve the user's expected results.

Summary of the invention

The embodiment of the present application provides a collocation model construction method, a clothing recommendation method, a device, a medium, and a terminal, which can provide an optimized clothing recommendation solution, and improve the intelligence and accuracy of the clothing recommendation function.

In an embodiment, the embodiment of the present application provides a collocation model construction method, including:

Acquiring a set number of model images including depth of field information, wherein the models in the set number of model images are wearing preset sets and accessories;

Constructing a three-dimensional model of each model according to the model image, and marking the body shape data, the suit and the accessories corresponding to the three-dimensional model to obtain a first image sample;

According to the image sample, the preset depth neural network is trained by using a set machine learning algorithm to obtain a collocation model, wherein the image sample includes the first image sample.

In an embodiment, the embodiment of the present application further provides a clothing recommendation method, including:

Obtaining at least one frame of the user image and the user's clothing style information;

Determining a corresponding human body model according to the user image;

Entering the human body model and the clothing style information into a pre-configured collocation model, and obtaining a clothing matching suggestion output by the collocation model, wherein the collocation model is a deep learning model trained according to a preset image sample, and the The image sample is obtained by marking the body type data, the set and the accessories of the set model;

Show the clothing matching suggestions.

In an embodiment, the embodiment of the present application further provides a collocation model construction device, and the device includes:

An image acquisition module, configured to acquire a set number of model images including depth information, wherein the models in the set number of model images are wearing preset sets and accessories;

a sample determination module, configured to construct a three-dimensional model of each model according to the model image, and mark the body shape data, the suit and the accessories corresponding to the three-dimensional model to obtain a first image sample;

The model training module is configured to train the preset depth neural network according to the image sample by using a set machine learning algorithm to obtain a collocation model, wherein the image sample includes the first image sample.

In an embodiment, the embodiment of the present application further provides a clothing recommendation device, the device comprising:

The information acquisition module is configured to acquire at least one frame of the user image and the user's clothing style information;

a human body model determining module, configured to determine a corresponding human body model according to the user image;

a matching determination module, configured to input the human body model and the clothing style information into a pre-configured matching model, and obtain a clothing matching suggestion outputted by the matching model, wherein the matching model is trained according to a preset image sample. a deep learning model, and the image sample is obtained by marking the body type data, the set and the accessories of the set model;

With the suggestion display module, set to display the clothing matching suggestions.

In an embodiment, the embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the foregoing collocation model construction method is implemented, or the computer program is The above-described clothing recommendation method is implemented when the processor executes.

In an embodiment, an embodiment of the present application further provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable by the processor, where the processor implements the foregoing matching when executing the computer program. The model construction method, or the processor recommendation method is implemented when the processor executes the computer program.

DRAWINGS

1 is a flowchart of a method for constructing a collocation model according to an embodiment of the present application;

2 is a flowchart of a clothing recommendation method according to an embodiment of the present application;

3 is a flow chart of another clothing recommendation method provided by an embodiment of the present application;

4 is a schematic structural diagram of a collocation model construction apparatus according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a clothing recommendation device according to an embodiment of the present application; FIG.

FIG. 6 is a schematic structural diagram of a terminal according to an embodiment of the present disclosure;

FIG. 7 is a schematic structural diagram of another terminal according to an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of a smart phone according to an embodiment of the present application.

Detailed ways

The present application will be described below in conjunction with the accompanying drawings and embodiments. The embodiments described herein are merely illustrative of the present application and are not intended to be limiting. For the convenience of description, only some but not all of the structures related to the present application are shown in the drawings.

In an embodiment, some example embodiments are described as a process or method depicted as a flowchart. Although the flowchart depicts multiple steps as a sequential process, many of the steps can be implemented in parallel, concurrently, or concurrently. Additionally, the order of one or more steps can be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the figures. The processing may correspond to methods, functions, procedures, subroutines, subroutines, and the like.

Embodiment 1

FIG. 1 is a flowchart of a method for constructing a collocation model according to an embodiment of the present application. The method may be performed by a collocation model construction device, where the device may be implemented by software and/or hardware. As shown in Figure 1, the method includes:

Step 110: Acquire a set number of model images including depth information.

In an embodiment, the model in the set number of model images is wearing a preset suit and accessories;

In an embodiment, the set number of model images is a plurality of model images different from at least one of a body shape, a gender, and an age of the model. In an embodiment, the embodiment of the present application does not quantify the set number, as long as the acquired model image including the deep information is sufficient to construct a three-dimensional model about the model. Among them, the model image is a model with different body types, different genders and different ages. The professional costume designer designs the costumes for these models, and adopts a three-dimensional (3-dimension, 3D) depth camera (which can be a mobile terminal). Shoot a model wearing a preset suit and accessories to get an image with depth information. Among them, the suit is a good costume matched by professional costume designers. Accessories include headwear, earrings, necklaces, bracelets, hats, scarves and bags. In an embodiment, the 3D depth camera can achieve the effect of 3D imaging using a structured light scheme. The structured light scheme, that is, the structured light projects a specific optical information onto the surface of the object, and is collected by the camera. The information such as the position and depth of the object is calculated according to the change of the optical signal caused by the object, thereby restoring the entire three-dimensional space.

In an embodiment, there are many ways to obtain a model image including the depth information, which is not specifically limited in the embodiment of the present application. One way may be to shoot a model wearing a matching costume (including taking a photo or video) according to a preset direction by controlling the 3D depth camera to obtain a set number of model images including depth information. The preset direction may be the front, rear, left, and right directions of the model. In an embodiment, the shooting direction of the 3D depth camera is not limited to the four directions listed above, and may be taken around the model one week. For example, controlling the 3D depth camera captures at least one frame model image from the front, rear, left, and right directions of the model. In another example, the control 3D depth camera is surrounded by a model that is set to match the costume for video shooting, and a model video is obtained. The model video may be subjected to frame processing using a set framing strategy to obtain a set number of model images including depth information. The framing strategy may be: extracting one frame of image every set time interval to obtain a set number of model images including depth information. The set time interval may be system default. In an embodiment, the shorter the time interval, the more model images are extracted, and the more accurate the three-dimensional model constructed by the model image.

Step 120: Construct a three-dimensional model of each model according to the model image, and mark the body shape data, the suit and the accessories corresponding to the three-dimensional model to obtain a first image sample.

In one embodiment, the model image includes pixel information and depth of field information, and a three-dimensional model of the model may be constructed based on the model image using a setting algorithm. The algorithm used in the construction of the three-dimensional model is not limited in the embodiment of the present application. Among them, the three-dimensional model is a three-dimensional model of a human body wearing a model with a professional designer and a good costume set and accessories. The stereo model of the human body has a specific proportional relationship with the real human body.

In one embodiment, the body shape data includes, but is not limited to, neck circumference, chest circumference, waist circumference, shoulder width, arm, hip circumference, and leg data. The circumference of the neck setting area in the three-dimensional image may be pre-defined to mark the neck circumference data. Similarly, the arm in the three-dimensional image is pre-defined to mark the arm data (including arm length data, arm circumference data, and arm circumference data, etc.). By analogy, the waist, shoulders, and legs are marked.

In one embodiment, the suits and accessories worn by the model can also be marked. For example, the clothes are marked along the outline of the clothes worn by the model. It can also be marked along the outline of the accessories worn by the model. In addition, the clothing style corresponding to the set and accessories is manually entered, and it is recorded as the first dress style. The marked three-dimensional model is represented in the form of an image matrix, and the image matrix and the first clothing style are stored as the first image sample. Among them, the style of clothing includes sports style, elegant style, mix and match style, rural style and rock style. In an embodiment, in order to enrich the number of image samples, the attribute parameters of the suit and the accessory may be deformed or adjusted, for example, the length, color or style of the jacket may be adjusted, or the accessory may be twisted. Exemplarily, an adjustment instruction of the set and the accessory is obtained, and the attribute parameters of the set and the accessory corresponding to the three-dimensional model are modified according to the adjustment instruction. Among them, the attribute adjustment suggestion given by the professional designer can be obtained, and the adjustment instruction is generated according to the attribute adjustment suggestion. Since the adjustment target corresponding to the adjustment instruction is a suit and an accessory, and the body shape data of the human body model has not changed, the three-dimensional model whose attribute parameter is adjusted may be marked according to the marking rule corresponding to the first image sample. At the same time, the adjusted suits and accessories are marked, including but not limited to, along the outline of the garment or accessory to determine the image data corresponding to the suit and accessories. Get the style of the adjusted suit and accessories as a second style. The second image sample is obtained according to the image matrix corresponding to the marked three-dimensional model (including marking the body type data and marking the set and accessories) and the second clothing style. In an embodiment, the step of deforming or adjusting the attribute parameters of the set and accessories is not necessary, and performing the adjustment step may enrich the number of samples used for the model training.

Step 130: Train the preset depth neural network according to the image sample by using a set machine learning algorithm to obtain a collocation model, wherein the image sample includes the first image sample.

In one embodiment, the image sample is sample data that marks the three-dimensional model of the model and marks the clothing style corresponding to the suit and accessories worn by the model, including but not limited to the first image sample.

In an embodiment, the set machine learning algorithms include a forward propagation algorithm and a backward propagation algorithm.

In an embodiment, the deep neural network may be a convolutional neural network, that is, the number of hidden layers and the number of nodes in each of the input layer, the hidden layer, and the output layer may be preset, and the first parameter of the convolutional neural network is initialized. Wherein, the first parameter includes the offset value of each layer and the weight of the edge, and the convolutional neural network is initially obtained. In the embodiment of the present application, the preset depth neural network is used to perform the two stages of forward propagation and backward propagation by using the image sample; when the error calculated by the backward propagation training reaches the expected error, the training ends and the matching is obtained. model. Exemplarily, the first image sample (including a positive sample and a negative sample) can be used to train the convolutional neural network in two stages of forward propagation and backward propagation; the error calculated in the backward propagation training reaches the expected error. In the case of the value, the training ends and the matching model is obtained. In an embodiment, the image sample may further include a second image sample, that is, the first image sample and the second image sample may be used to perform training on the convolutional neural network in two stages of forward propagation and backward propagation; The training ends when the error calculated by the backward propagation training reaches the desired error value.

In an embodiment, network parameters such as the number of layers of the deep neural network, the number of neurons, the convolution kernel, and/or the weight are not limited in the embodiment of the present application. The embodiment of the present application does not limit the execution body of the collocation model construction operation, and may be a server or a mobile terminal.

The technical solution of the embodiment obtains a set number of model images including depth information; constructs a three-dimensional model of each model according to the model image, and marks the body shape data, the suit and the accessories corresponding to the three-dimensional model to obtain the first Image samples; according to the image samples, the preset deep neural network is trained by using a set machine learning algorithm to obtain a matching model, which can make the matching model have the function of recommending the clothing matching scheme based on the body data and the clothing style. The above technical solution can solve the problem that the clothing matching solution provided by the related technology is limited in intelligence, can provide the clothing recommendation suggestion that achieves the expected effect of the user, and improves the intelligence and accuracy of the clothing recommendation function.

Embodiment 2

FIG. 2 is a flowchart of a clothing recommendation method according to an embodiment of the present application. The method can be performed by a clothing recommendation device, wherein the device can be implemented by software and/or hardware, and can generally be integrated into a mobile terminal, such as a mobile terminal having a 3D depth camera. As shown in Figure 2, the method includes:

Step 210: Acquire at least one frame of the user image and the user's clothing style information.

In an embodiment, the user image may be an image including depth of field information captured by a 3D depth camera, and may also be a historical image in a picture library. Clothing style information is the type of clothing style, including but not limited to sports style, elegance, mix and match, pastoral style and rock style.

In an embodiment, the operation of acquiring the image of the user by the mobile terminal may be performed by a system of the mobile terminal or by any application software having a shooting function in the mobile terminal, and the operation of acquiring the image of the user may be performed by the system under the operation instruction of the user. Or application software to execute. For example, when the clothing recommendation function is activated, the clothing style inquiry information is output, including a display selection dialog box, prompting the user to input the clothing style. Among them, the way the user enters the clothing style can be manually entered or selected by the clothing style options listed in the selection dialog box. The clothing style information input by the user is obtained, and the user is prompted to input at least one frame of the user image. For example, a camera activation event may be triggered to detect a user-entered clothing style information to prompt the user to capture at least one frame of the user image through the camera. The 3D depth camera is controlled to take a user image according to a shooting instruction input by the user. For another example, in the case that the clothing style information input by the user is detected, the prompt information may be displayed to ask the user to select to perform the following operations: selecting at least one frame of the user history image from the picture library, or controlling the camera to capture at least one frame of the user image. According to the operation instruction input by the user, it is determined which operation is performed (including controlling the 3D depth camera to capture the user image, or acquiring the user image from the picture library).

Step 220: Determine a corresponding human body model according to the user image.

In one embodiment, the human body model is a three-dimensional model constructed by acquiring model body information in advance through a 3D depth camera. There are many ways to construct a three-dimensional model, and the embodiments of the present application are not specifically limited. For example, when the clothing recommendation function is initialized, the user is photographed by the 3D depth camera from the preset direction to obtain a first depth image. The user is photographed by the preset direction to capture at least one frame of the user image in at least four directions of the user's front, back, left, and right directions, that is, at least four frames of the first depth image are obtained. A user's human body model can be constructed using a specific three-dimensional model building algorithm based on the first depth image. The selection of the three-dimensional model construction algorithm is not specifically limited in the embodiment of the present application. For example, the three-dimensional model construction algorithm may be a correlation detection algorithm and a three-dimensional texture mapping algorithm. For another example, when the clothing recommendation function is initialized, the 3D depth camera is controlled to surround the user for at least one week, and video recording is performed to obtain a user video. The user video may be framed by a specific framing strategy to obtain a user image of a plurality of frames taken at a set angle on a 360 degree circumference, which may be recorded as a second depth image. Similarly, the user's human body model can be constructed using a specific three-dimensional model construction algorithm according to the second depth image.

In an embodiment, the iris information of the user may also be extracted, and the iris information is stored in the human body model set in association with the human body model.

In one embodiment, the human body model building operation is a setting step of the clothing recommendation scheme, which is executed when the clothing recommendation function is initialized, and stores the constructed human body model on the mobile terminal. In an embodiment, the apparel recommendation function also provides a human body model update function by which the user can add, modify, or delete the stored human body model.

In the embodiment of the present application, when detecting that the clothing recommendation function is enabled, the user image is acquired, the preset feature point in the user image is extracted, and the facial feature information of the user image is determined according to the preset feature point. The preset feature point may be a system default, and may identify a feature point of the user identity, for example, a pixel corresponding to the iris or a corresponding pixel such as an eye, a nose, and a mouth. That is to say, the pixel corresponding to the iris in the user image can be extracted, and the iris information of the user image is determined according to the pixel. In an embodiment, pixels corresponding to eyes, noses, and mouths in the user image may also be separately extracted, thereby determining eye contours, nose contours, and mouth contours in the user image to generate users according to eye contours, nose contours, and mouth contours. portrait.

In an embodiment, the user may store more than one user's mannequin within the same mobile terminal. The human body model set corresponding to the user image can be determined by querying the human body model set by the facial feature information. For example, the human body model corresponding to the user image may be selected from the pre-built set of human body models according to the iris information. For another example, a user portrait can be constructed based on the contour of the eye, the contour of the nose, and the contour of the mouth, and the human body model corresponding to the user image can be filtered according to the user image.

In an embodiment, the embodiment of the present application may further generate a human body model according to at least one frame of the user image without constructing a human body model of the user in advance.

Step 230: Input the human body model and the clothing style information into a pre-configured matching model, and obtain a clothing matching suggestion output by the matching model.

In one embodiment, the collocation model is a deep learning model trained according to preset image samples, wherein the model data of the set model is marked to obtain image samples. The model can be a model of different body types, different genders and different ages as described in the embodiments of the present application, and the model is wearing a suit and accessories matched by professional costume designers. The collocation model can be a convolutional neural network model. In the embodiment of the present application, network parameters such as the number of layers of the collocation model, the number of neurons, the convolution kernel, and/or the weight are not limited.

In one embodiment, the apparel matching suggestions include clothing type suggestions, shoe matching suggestions, and suggestions for matching accessories.

In the embodiment of the present application, the matrix data corresponding to the human body model corresponding to the user image and the clothing style selected by the user are input into the matching model, and the body shape data corresponding to the human body model is extracted by the matching model, and the clothing style is determined, and the body shape data is determined. And clothing style matching clothing matching suggestions and the probability value corresponding to each clothing matching suggestion, output clothing matching suggestions and probability values. In an embodiment, since the collocation model is trained by the image sample, and the image sample includes the image matrix corresponding to the marked model three-dimensional model and the clothing style corresponding to the set and the accessory, according to the collocation model, based on the human body model and The clothing style chosen by the user can provide clothing matching suggestions that match the user's body shape data and clothing style. For example, if the user takes a self-photograph of a frame and enters a clothing style as a sports style, the matching model in the embodiment of the present application may be used to provide clothing matching suggestions. In an embodiment, the user image and the clothing style are acquired, and the corresponding human body model is determined according to the user image. The matrix data corresponding to the human body model and the clothing style are input and matched with the model, and the clothing matching suggestions output by the matching model are obtained.

In an embodiment, the clothing matching suggestions may be arranged in descending order according to the probability value, and the preset number of clothing matching suggestions and corresponding probability values are outputted.

Step 240: Display the clothing matching suggestion.

Illustratively, the display manner of the clothing matching suggestion may be a text description manner, and the text description corresponding to the clothing matching suggestion may be directly displayed in the form of a dialog box, and the display manner of the clothing recommendation suggestion is not limited to the above enumerated manner. For example, it is also possible to display the clothing suits and accessories corresponding to the clothing matching suggestions in the form of two-dimensional or three-dimensional images. For example, it is also possible to display an effect picture of the preset model wearing the clothing corresponding to the clothing and the accessory corresponding to the recommendation.

The technical solution of the embodiment obtains at least one frame of the user image and the clothing style information input by the user; determines a corresponding human body model according to the user image; and inputs the human body model and the clothing style information into a pre-configured matching model to obtain the matching The clothing output suggestion of the model output; display the clothing matching suggestion, according to the matching model, the body shape data corresponding to the user's human body model and the clothing matching suggestion corresponding to the clothing style selected by the user may be determined. The above technical solution can solve the problem that the clothing matching solution provided by the related technology is limited in intelligence, can provide the clothing recommendation suggestion that achieves the expected effect of the user, and improves the intelligence and accuracy of the clothing recommendation function.

Embodiment 3

FIG. 3 is a flowchart of another clothing recommendation method provided by an embodiment of the present application. As shown in FIG. 3, the method includes:

Step 310: When detecting that the apparel recommendation function is activated, acquiring at least one frame of the user image and the user's clothing style information.

For example, the clothing recommendation function switch may be added in the camera application, and when the opening instruction of the user input is detected, the clothing recommendation function switch is turned on. When it is detected that the clothing recommendation function switch is turned on, the user is prompted to take at least one frame of the user image. In an embodiment, the target frame may be displayed in the preview interface to provide the user with the face falling into the target frame during shooting to ensure that the user's face is captured. After the user image is captured, a query dialog box is displayed to prompt the user to enter the clothing style inquiry information. The inquiry dialog is detected to obtain the clothing style information input by the user.

In an embodiment, the mobile terminal may further provide an application capable of implementing a clothing recommendation function, and when detecting that the application is started, displaying a query dialog box to prompt the user to input clothing style information. When the clothing style information input by the user is detected, the clothing style information is stored in the preset storage space. The mobile terminal controls the camera to capture at least one frame of the user image.

Step 320: Extract a preset feature point in the user image, and determine facial feature information of the user image according to the preset feature point.

Step 330: Filter, according to the facial feature information, a human body model corresponding to the user image from a pre-built set of human body models.

Step 340: Input the human body model and the clothing style information into a pre-configured matching model, and obtain a clothing matching suggestion output by the matching model.

Step 350: Find an apparel model matching the style and size in the apparel matching suggestion from a preset apparel database, and display the apparel model.

In an embodiment, the preset apparel database may be a database storing image data of clothing and accessories, and the image data is stored in association with the description data in the database. The description data is a character that describes the characteristics of the clothing and accessories, including but not limited to the size, color or style of the clothes or accessories. The picture data in the preset clothing database may be picture data of clothing and accessories obtained from the network platform picture library through the web crawler. In an embodiment, the clothing database may also be a photo of a user's own clothing and accessories, a database of clothing and accessories in the user's own wardrobe, and the like. For example, a 3D depth camera can be used to take photographs of clothes and accessories from a preset direction, and a costume model can be constructed based on the photographs of the photographed clothes and accessories, and the costume model is stored in the clothing database.

According to the clothing matching suggestion, the corresponding clothing model is searched from the preset clothing database. In an embodiment, the clothing matching suggestion is matched with the description data, and the clothing model matching the style and the size in the clothing matching suggestion is determined, and the clothing model is displayed. In an embodiment, the probability corresponding to the clothing matching suggestion may be displayed. The value determines the display order of the clothing model, that is, the clothing model corresponding to the clothing matching suggestion with higher probability value is preferentially displayed. In one embodiment, a user model wearing the recommended apparel is rendered according to the apparel model and the human body model, and the user model is displayed to present the effect of the user wearing the matched clothes and accessories.

In an embodiment, the preset clothing database is not limited to a database pre-configured in the mobile terminal, and may also be a database of the online shopping platform. For example, the online shopping platform can provide three-dimensional clothing model data, and the mobile terminal acquires three-dimensional model data by calling an application programming interface (API) provided by the network platform. In an embodiment, the link address corresponding to the clothing model may be displayed while the clothing model is displayed, that is, if the clothing model is a set of sportswear, the display order of the link address corresponding to the sportswear may be determined according to the sales volume. .

Step 360: Acquire an adjustment operation input by the user for the user model.

In an embodiment, the apparel recommendation recommendation output by the recommendation model is in line with the professional designer's aesthetic suggestion, but does not necessarily achieve the user's expectation of the costume effect. In view of the above considerations, the embodiment of the present application can also provide a clothing fine-tuning function. That is, the user model of the costume corresponding to the apparel recommendation is displayed on the mobile terminal, and the adjustment operation input by the user for the user model is detected. Wherein, the adjustment operation may include an update indication for the length, color or accessory of the garment. For example, the user clicks on the pixel corresponding to the trousers and adds a worn effect to the trousers. Another example is that the user clicks on the headwear, modifies the number of headwear, and the like.

In the embodiment of the present application, when displaying the user model of the clothing model corresponding to the clothing recommendation suggestion, the user operation for the user model is acquired. When a user operation is detected, it is determined whether the operation object operated by the user is for clothing or accessories. If so, the property interface of the apparel or accessory is displayed for the user to modify the attribute data. Obtain the modified new attribute data and generate an adjustment operation based on the new attribute data.

Step 370: Modify the apparel parameter of the user model according to the adjustment operation, and display the modified new user model.

Among them, the clothing parameters include attribute data such as color, length, and style.

The apparel parameters of the apparel or accessories are updated according to the attribute data corresponding to the adjustment operation, and the modified new user model is displayed to display the adjusted clothing and accessories effects.

In an embodiment, the user may also be reminded to mark the clothing style corresponding to the modified clothing and accessories, as an adjustment record of the clothing matching suggestions outputted by the matching model, and save the adjustment record of the user's clothing matching suggestions output by the matching model. . When the adjustment record exceeds the set threshold, the clothing style and the matrix data corresponding to the user model wearing the modified costume are input into the matching model to update the matching model. This design allows the matching model to refer to the user's preferences for apparel recommendation, which can better meet the individual needs of users.

In an embodiment, the network platform may be queried according to the new user model, and the product link address corresponding to the clothing or accessories in the new user model is determined, so as to shorten the time spent by the user on the online shopping, and the online shopping of the user is improved. Experience.

In the technical solution of the embodiment, the clothing model is determined through the clothing matching suggestion, and the user model wearing the recommended clothing is rendered according to the clothing model and the human body model, and the user model is displayed to present the user corresponding to the clothing matching suggestion correspondingly. The effect of the recommended apparel; the adjustment operation for the user model that detects the user input can also be provided to meet the user's personalized clothing matching needs.

Embodiment 4

FIG. 4 is a schematic structural diagram of a collocation model construction apparatus according to an embodiment of the present application. The apparatus may be implemented by software and/or hardware and configured to perform the collocation model construction method provided by the embodiment of the present application. As shown in Figure 4, the device comprises:

The image obtaining module 410 is configured to acquire a set number of model images including depth information, wherein the models in the set number of model images are wearing preset sets and accessories;

The sample determination module 420 is configured to construct a three-dimensional model of each model according to the model image, and mark the body shape data, the suit and the accessories corresponding to the three-dimensional model to obtain a first image sample;

The model training module 430 is configured to train the preset depth neural network according to the image sample by using a set machine learning algorithm to obtain a collocation model, wherein the image sample includes the first image sample.

The technical solution of the embodiment provides a collocation model construction device, and has the function of recommending a clothing collocation scheme based on the body type data and the clothing style. The above technical solution can solve the problem that the clothing matching solution provided by the related technology is limited in intelligence, can provide the clothing recommendation suggestion that achieves the expected effect of the user, and improves the intelligence and accuracy of the clothing recommendation function.

In an embodiment, the set number of model images is a plurality of model images different from at least one of a body shape, a gender, and an age of the model.

In an embodiment, the image acquisition module 410 is configured to:

The 3D depth camera is controlled to shoot a model wearing the matching costume according to a preset direction, and a set number of model images including depth information is obtained.

In an embodiment, the image acquisition module 410 is configured to:

Controlling the 3D depth camera to perform video shooting around the model wearing the matching costume, and obtaining a model video;

The model video is subjected to frame processing by using a set framing strategy to obtain a set number of model images including depth information.

In an embodiment, the sample determination module 420 is configured to:

Marking the neck circumference, chest circumference, waist circumference, shoulder width, arms, hips and legs of the three-dimensional model;

Marking the set and accessories and marking the first dress style corresponding to the set and accessories;

The first image sample is obtained according to the image matrix corresponding to the marked three-dimensional model and the first clothing style.

In an embodiment, the apparatus further comprises:

An additional sample determining module is configured to obtain an adjustment instruction of the package and the accessory after marking the body shape data, the suit, and the accessory corresponding to the three-dimensional model to obtain the first image sample, and modify the corresponding three-dimensional model according to the adjustment instruction The set parameters of the suit and accessories; mark the modified set and accessories, and mark the second dress style corresponding to the modified set and accessories; the image matrix corresponding to the 3D model after modifying the attribute parameters and the second A second image sample is obtained in the style of clothing.

In an embodiment, the model training module 430 is configured to:

Using the image samples to train the preset deep neural network in two stages of forward propagation and backward propagation;

When the error calculated by the backward propagation training reaches the expected error value, the training ends and a matching model is obtained.

Embodiment 5

FIG. 5 is a schematic structural diagram of a clothing recommendation device according to an embodiment of the present application. The apparatus may be implemented by software and/or hardware and may be integrated into a mobile terminal having a 3D depth camera configured to perform a clothing recommendation operation. As shown in Figure 5, the device includes:

The information acquiring module 510 is configured to acquire at least one frame of the user image and the user's clothing style information;

The human body model determining module 520 is configured to determine a corresponding human body model according to the user image;

The collocation suggestion module 530 is configured to input the human body model and the clothing style information into a pre-configured collocation model, and obtain a clothing collocation suggestion outputted by the collocation model, wherein the collocation model is based on a preset image sample. a deep learning model of the training, and the image sample is obtained by marking the body type data, the set and the accessories of the set model;

The suggestion display module 540 is arranged to display the clothing matching suggestion.

The technical solution of the embodiment provides a clothing recommendation device, and the body shape data corresponding to the human body model of the user and the clothing matching proposal corresponding to the clothing style selected by the user may be determined according to the collocation model. The above technical solution can solve the problem that the clothing matching solution provided by the related technology is limited in intelligence, can provide the clothing recommendation suggestion that achieves the expected effect of the user, and improves the intelligence and accuracy of the clothing recommendation function.

In an embodiment, the information acquisition module 510 is configured to:

When the clothing recommendation function is detected to be activated, the clothing style inquiry information is output;

Obtaining clothing style information input by the user, and prompting the user to input at least one frame of the user image;

The 3D depth camera is controlled to capture a user image according to the operation instruction input by the user, or the user image is acquired from the picture library.

In an embodiment, the mannequin determination module 520 is configured to:

Extracting a preset feature point in the user image, and determining facial feature information of the user image according to the preset feature point;

The human body model corresponding to the user image is filtered from the pre-built set of human body models according to the facial feature information.

In an embodiment, the collocation suggestion module 540 is configured to:

Finding an apparel model matching the style and size in the clothing matching suggestion in the preset clothing database, and displaying the clothing model.

In an embodiment, the collocation suggestion module 540 is configured to:

A user model wearing the recommended clothing is rendered according to the clothing model and the human body model, and the user model is displayed.

In an embodiment, the apparatus further comprises:

The clothing parameter adjustment module is configured to: after displaying the user model, acquire an adjustment operation input by the user for the user model; modify the clothing parameter of the user model according to the adjustment operation, and display the modified new user model.

In an embodiment, the clothing matching suggestion includes at least one of the following: a clothing type matching suggestion, a shoe matching suggestion, and a jewelry matching suggestion.

Embodiment 6

The embodiment of the present application further provides a storage medium including computer executable instructions for executing a collocation model construction method when executed by a computer processor, the method comprising:

In an embodiment, the embodiment of the present application further provides another storage medium including computer executable instructions for executing a clothing recommendation method when executed by a computer processor, the method comprising:

Determining a corresponding human body model according to the user image;

Show the clothing matching suggestions.

Storage medium - any one or more types of memory devices or storage devices. The term "storage medium" is intended to include: a mounting medium such as a Compact Disc Read-Only Memory (CD-ROM), a floppy disk or a tape device; a computer system memory or a random access memory such as a dynamic random access memory; (Dynamic Random Access Memory, DRAM), (Double Data Rate Random Access Memory, DDR RAM), Static Random Access Memory (SRAM), Extended Data Output Random Access Memory (Extended Data Output Random Access Memory) , EDO RAM), Rambus Random Access Memory (Rambus RAM), etc.; non-volatile memory such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory components Wait. The storage medium may also include other types of memory or a combination thereof. Additionally, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system, the second computer system being coupled to the first computer system via a network, such as the Internet. The second computer system can provide program instructions to the first computer for execution. The term "storage medium" can include two or more storage media that can reside in different locations (eg, in different computer systems connected through a network). A storage medium may store program instructions (eg, embodied as a computer program) executable by one or more processors.

Certainly, a storage medium containing computer executable instructions provided by the embodiments of the present application, the computer executable instructions are not limited to the operation of the collocation model construction as described above, and may also perform the collocation model provided by any embodiment of the present application. Related operations in the build method.

Of course, the storage medium containing the computer executable instructions provided by the embodiment of the present application is not limited to the operation of the clothing recommendation as described above, and may also perform the clothing recommendation method provided by any embodiment of the present application. Related operations in .

Example 7

The embodiment of the present application provides a terminal, where the terminal has an operating system, and the collocation model construction device provided by the embodiment of the present application can be integrated into the terminal. The terminal can be a smart phone or a tablet (PAD). FIG. 6 is a schematic structural diagram of a terminal according to an embodiment of the present application. As shown in FIG. 6, the terminal includes a memory 610 and a processor 620. The memory 610 is configured to store a computer program, a model image, a three-dimensional model of the model, an image sample, and a matching model. The processor 620 reads and executes the computer program stored in the memory 610. The processor 620 implements the collocation model construction method described in any of the embodiments of the present disclosure when executing the computer program.

Example eight

In an embodiment, the embodiment of the present application provides another terminal, where the terminal has an operating system, and the clothing recommendation device provided by the embodiment of the present application may be integrated into the terminal. The terminal can be a smart phone or a tablet (PAD). FIG. 7 is a schematic structural diagram of another terminal according to an embodiment of the present application. As shown in FIG. 7, the terminal includes a camera 710, a memory 720, and a processor 730. The camera 710 is a 3D depth camera, and a user image including depth of field information can be captured using a structured light scheme. The memory 720 is configured to store a computer program, a user image, clothing style information, a human body model, a collocation model, and the like. The processor 730 reads and executes a computer program stored in the memory 720. The processor 730 implements the apparel recommendation method of any of the embodiments of the present disclosure when the computer program is executed.

Example nine

The camera, memory and processor listed in the above examples are all components of the terminal, and the terminal may also include other components. Take a smart phone as an example to illustrate the possible structure of the above terminal.

FIG. 8 is a schematic structural diagram of a smart phone according to an embodiment of the present application. As shown in FIG. 8 , the smart phone may include: a memory 801, a central processing unit (CPU) 802 (also referred to as a processor, hereinafter referred to as a CPU), a peripheral interface 803, and a radio frequency (RF) circuit. 805, audio circuit 806, speaker 811, display 812, camera 813, power management chip 808, input/output (I/O) subsystem 809, other input/control devices 810, and external port 804, these components pass One or more communication buses or signal lines 807 are in communication.

The illustrated smartphone 800 is merely one example of a mobile terminal, and the smartphone 800 can have more or fewer components than those shown in the figures, two or more components can be combined, or can have different Component configuration. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The memory 801 can be accessed by the CPU 802, the peripheral interface 803, etc., and the memory 801 can include a high speed random access memory, and can also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices. Or other volatile solid-state storage devices. A computer program, a matching model, and the like are stored in the memory 811.

Peripheral interface 803, which can connect the input and output peripherals of the device to CPU 802 and memory 801.

I/O subsystem 809, which can connect input and output peripherals on the device, such as display 812 and other input/control devices 810, to peripheral interface 803. The I/O subsystem 809 can include a display controller 8091 and one or more input controllers 8092 for controlling other input/control devices 810. Wherein, one or more input controllers 8092 receive electrical signals from other input/control devices 810 or transmit electrical signals to other input/control devices 810, and other input/control devices 810 may include physical buttons (press buttons, rocker buttons, etc.) ), dial, slide switch, joystick, click wheel. It is worth noting that the input controller 8092 can be connected to any of the following: a keyboard, an infrared port, a Universal Serial Bus (USB) interface, and a pointing device such as a mouse.

A display 812, which is an input interface and an output interface between the user terminal and the user, displays the visual output to the user, and the visual output can include graphics, text, icons, video, and the like.

The camera 813 acquires an optical image of the user using a structured light scheme, and converts the optical image into an electrical signal, and is stored in the memory 801 through the peripheral interface 803.

Display controller 8091 in I/O subsystem 809 receives an electrical signal from display 812 or an electrical signal to display 812. Display 812 detects contact on the display, display controller 8091 converts the detected contact into interaction with a user interface object displayed on display 812, i.e., enables human-computer interaction, and the user interface object displayed on display 812 can be operational The icon of the game, the icon of the network to the corresponding network, and the like. In an embodiment, display 512 is a screen. In an embodiment, the device may also include a light mouse, which is a touch sensitive surface that does not display a visual output, or an extension of a touch sensitive surface formed by the screen.

The RF circuit 805 is configured to establish communication between the mobile phone and the wireless network (ie, the network side) to implement data reception and transmission between the mobile phone and the wireless network. For example, sending and receiving short messages, emails, and the like. In one embodiment, RF circuit 805 receives and transmits an RF signal, also referred to as an electromagnetic signal, and RF circuit 805 converts the electrical signal into an electromagnetic signal or converts the electromagnetic signal into an electrical signal, and through the electromagnetic signal and communication network And other devices to communicate. RF circuitry 805 may include known circuitry for performing these functions including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec CODER-DECoder (CODEC) chipset, Subscriber Identity Module (SIM), etc.

The audio circuit 806 is arranged to receive audio data from the peripheral interface 803, convert the audio data into an electrical signal, and transmit the electrical signal to the speaker 811.

The speaker 811 is arranged to restore the voice signal received by the mobile phone from the wireless network through the RF circuit 805 to sound and play the sound to the user.

The power management chip 808 is configured to provide power and power management for the hardware connected to the CPU 802, the I/O subsystem 809, and the peripheral interface 803.

The terminal provided by the embodiment of the present application constructs a three-dimensional model of each model by using a model image including depth information, and marks the body shape data, the suit, and the accessories corresponding to the three-dimensional model to obtain a first image sample; The fixed machine learning algorithm trains the preset deep neural network to obtain the matching model, which can make the matching model have the function of recommending the clothing matching scheme based on the body data and the clothing style. The above technical solution can solve the problem that the clothing matching solution provided by the related technology is limited in intelligence, can provide the clothing recommendation suggestion that achieves the expected effect of the user, and improves the intelligence and accuracy of the clothing recommendation function.

In an embodiment, the embodiment of the present application further provides another terminal, and according to the collocation model, the body shape data corresponding to the human body model of the user and the clothing collocation proposal corresponding to the clothing style selected by the user may be determined. The above technical solution can solve the problem that the clothing matching solution provided by the related technology is limited in intelligence, can provide the clothing recommendation suggestion that achieves the expected effect of the user, and improves the intelligence and accuracy of the clothing recommendation function.

The collocation model construction device, the clothing recommendation device, the storage medium and the terminal provided in the above embodiments can execute the collocation model construction method and the clothing recommendation method provided by the embodiments of the present application, and have the corresponding functional modules and effects for executing the method. For the technical details that are not described in detail in the above embodiments, reference may be made to the collocation model construction method and the clothing recommendation method provided by any embodiment of the present application.

Claims

A collocation model construction method, including:

Acquiring a set number of model images including depth of field information, wherein the models in the set number of model images are wearing preset sets and accessories;

Constructing a three-dimensional model of each model according to the model image, and marking the body shape data, the suit and the accessories corresponding to the three-dimensional model to obtain a first image sample;

According to the image sample, the preset depth neural network is trained by using a set machine learning algorithm to obtain a collocation model, wherein the image sample includes the first image sample.
The method according to claim 1, wherein the set number of model images is a plurality of model images different from at least one of a body shape, a gender, and an age of the model.
The method according to claim 1 or 2, wherein acquiring a set number of model images including depth information includes:

The three-dimensional 3D depth camera is controlled to shoot a model wearing the matching costume according to a preset direction, and a set number of model images including depth information is obtained.
The method according to claim 3, wherein the 3D depth camera is controlled to shoot a model wearing the matching costume according to a preset direction to obtain a set number of model images including depth information, including:

Controlling the 3D depth camera to perform video shooting around the model wearing the matching costume, and obtaining a model video;

The model video is subjected to frame processing by using a set framing strategy to obtain a set number of model images including depth information.
The method according to claim 1 or 2, wherein marking the body shape data, the suit and the accessory corresponding to the three-dimensional model to obtain the first image sample comprises:

Marking the neck circumference, chest circumference, waist circumference, shoulder width, arms, hips and legs of the three-dimensional model;

Marking the set and accessories and marking the first dress style corresponding to the set and accessories;

The first image sample is obtained according to the image matrix corresponding to the marked three-dimensional model and the first clothing style.
The method according to claim 1 or 2, after marking the body shape data, the suit and the accessory corresponding to the three-dimensional model to obtain the first image sample, the method further includes:

Obtaining an adjustment instruction of the set and the accessory, and modifying an attribute parameter of the set and the accessory corresponding to the three-dimensional model according to the adjustment instruction;

Marking the modified set and accessories and marking the second dress style corresponding to the modified set and accessories;

And obtaining a second image sample according to the image matrix corresponding to the three-dimensional model after modifying the attribute parameter and the second dressing style.
The method according to any one of claims 1 to 6, wherein the preset depth neural network is trained according to the image sample by using a set machine learning algorithm to obtain a matching model, including:

Using the image samples to train the preset deep neural network in two stages of forward propagation and backward propagation;

When the error calculated by the backward propagation training reaches the expected error value, the training ends and a matching model is obtained.
A method of recommending clothing, including:

Obtaining at least one frame of the user image and the user's clothing style information;

Determining a corresponding human body model according to the user image;

Entering the human body model and the clothing style information into a pre-configured collocation model, and obtaining a clothing matching suggestion output by the collocation model, wherein the collocation model is a deep learning model trained according to a preset image sample, and the The image sample is obtained by marking the body type data, the set and the accessories of the set model;

Show the clothing matching suggestions.
The method according to claim 8, wherein acquiring at least one frame of the user image and the user's clothing style information comprises:

When the clothing recommendation function is detected to be activated, the clothing style inquiry information is output;

Obtaining clothing style information input by the user, and prompting the user to input at least one frame of the user image;

The 3D depth camera is controlled to capture a user image according to the operation instruction input by the user, or the user image is acquired from the picture library.
The method of claim 8, wherein determining a corresponding human body model based on the user image comprises:

Extracting a preset feature point in the user image, and determining facial feature information of the user image according to the preset feature point;

The human body model corresponding to the user image is filtered from the pre-built set of human body models according to the facial feature information.
The method according to any one of claims 8 to 10, wherein the clothing matching suggestion is displayed, comprising:

Finding an apparel model that matches the style and size in the apparel matching suggestion in a preset apparel database, and displaying the apparel model.
The method of claim 11 wherein displaying the apparel model comprises:

A user model wearing the recommended clothing is rendered according to the clothing model and the human body model, and the user model is displayed.
The method of claim 12, after displaying the user model, further comprising:

Obtaining an adjustment operation input by the user for the user model;

Modifying the apparel parameters of the user model according to the adjustment operation, and displaying the modified new user model.
The method according to any one of claims 8 to 13, wherein the clothing matching suggestion comprises at least one of the following: a clothing category matching suggestion, a shoe matching suggestion, and a jewelry matching suggestion.
A matching model building device, comprising:

An image acquisition module, configured to acquire a set number of model images including depth information, wherein the models in the set number of model images are wearing preset sets and accessories;

a sample determination module, configured to construct a three-dimensional model of each model according to the model image, and mark the body shape data, the suit and the accessories corresponding to the three-dimensional model to obtain a first image sample;

The model training module is configured to train the preset depth neural network according to the image sample by using a set machine learning algorithm to obtain a collocation model, wherein the image sample includes the first image sample.
The apparatus according to claim 15, wherein the set number of model images are a plurality of model images different from at least one of a body shape, a gender, and an age of the model.
A clothing recommendation device comprising:

The information acquisition module is configured to acquire at least one frame of the user image and the user's clothing style information;

a human body model determining module, configured to determine a corresponding human body model according to the user image;

a matching determination module, configured to input the human body model and the clothing style information into a pre-configured matching model, and obtain a clothing matching suggestion outputted by the matching model, wherein the matching model is trained according to a preset image sample. a deep learning model, and the image sample is obtained by marking the body type data, the set and the accessories of the set model;

With the suggestion display module, set to display the clothing matching suggestions.
The apparatus of claim 17, wherein the information acquisition module is configured to:

When the clothing recommendation function is detected to be activated, the clothing style inquiry information is output;

Obtaining clothing style information input by the user, and prompting the user to input at least one frame of the user image;

The three-dimensional 3D depth camera is controlled to capture a user image according to the operation instruction input by the user, or the user image is acquired from the picture library.
A computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor, implementing the collocation model construction method according to any one of claims 1 to 7, or the computer program being The costume recommendation method according to any one of claims 8 to 14 is implemented when the processor is executed.
A terminal comprising a memory, a processor, and a computer program stored on the memory and operable by the processor, the processor executing the computer program to implement the collocation model according to any one of claims 1 to 7. The method of constructing, or the method of recommending the clothing according to any one of claims 8 to 14 when the processor executes the computer program.