WO2020078119A1

WO2020078119A1 - Method, device and system for simulating user wearing clothing and accessories

Info

Publication number: WO2020078119A1
Application number: PCT/CN2019/103681
Authority: WO
Inventors: 潘宗涛; 王德鑫
Original assignee: 京东数字科技控股有限公司
Priority date: 2018-10-15
Filing date: 2019-08-30
Publication date: 2020-04-23
Also published as: CN109409994A

Abstract

The present invention relates to the technical field of computers, and disclosed thereby are a method, device and system for simulating a user wearing clothing and accessories. A specific embodiment of the method comprises: acquiring a full body image of a user, and performing detection on the full body image on the basis of an evaluation model so as to evaluate the pose of the user; identifying the identity of the user, and recommending clothing and accessories according to the identity of the user; identifying a gesture of the user on the basis of an identification algorithm of a geometric moment and edge detection to obtain a gesture command; and simulating the user wearing the clothing and accessories according to the pose of the user and the gesture command. The embodiment may reduce the amount of computation, and increase computation speed and identification accuracy.

Description

Method, device and system for simulating user wearing clothing accessories

Technical field

The present invention relates to the field of computer technology, and in particular, to a method, device, and system for simulating a user to wear clothing accessories.

Background technique

At present, with the development of digitalization and intelligence, various emerging technologies and applications continue to emerge, especially in the field of identification, the identification industry has successfully transitioned from the initial stage of steady development to the mature stage of high-speed development. For example: face payment, face check-in, security protection, human-computer interaction, dancing machine, etc.

Generally, when users buy clothing, the system uses existing face recognition, gender recognition, age recognition, and gesture recognition technologies to create an intelligent matching system that combines gesture recognition with real-time switching to allow users to experience different styles of matching clothes.

In the process of implementing the present invention, the inventor found that there are at least the following problems in the prior art:

1. The calculation is huge, time-consuming and costly

2. The accuracy of gesture recognition is low, requiring users to wave their hands many times.

Summary of the invention

In view of this, the embodiments of the present invention provide a method, device, and system for simulating a user wearing a clothing ornament, which can reduce the amount of calculation, improve the calculation speed, and recognition accuracy.

To achieve the above object, according to an aspect of an embodiment of the present invention, a method for simulating a user wearing a clothing accessory is provided, including: acquiring a user's full-body image, and detecting the whole-body image based on an evaluation model to evaluate the user's posture; Wherein, the evaluation model is an openpose network structure based on the MobileNet algorithm; identify the user's identity and recommend costume accessories based on the user's identity; the recognition algorithm based on geometric moments and edge detection recognizes the user's gesture and obtains a gesture command; based on the user's gesture And the gesture command simulates the user wearing a costume ornament.

To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided an apparatus for simulating a user's wearing of clothing accessories, including: an evaluation module for acquiring a user's whole-body image, and detecting the whole-body image based on an evaluation model To evaluate the user's posture; wherein, the evaluation model is an openpose network structure based on the MobileNet algorithm; the first recognition module is used to recognize the user's identity, and the clothing accessories are recommended according to the user's identity; the second recognition module is used to based on the geometry The recognition algorithm of moment and edge detection recognizes the user's gesture to obtain a gesture command; a simulation module is used to simulate the user's wearing of clothing accessories based on the user's gesture and the gesture command.

To achieve the above object, according to still another aspect of the embodiments of the present invention, a system for simulating a user's clothing accessories is provided, including an image acquisition terminal, an identification terminal, a recommendation server, a display terminal, and a data storage terminal, where: The image acquisition terminal is used to obtain the user's full-body image, facial image and gesture image; the recognition terminal is used to evaluate the user's posture, user identity and obtain gesture commands; the recommendation server is used to recommend clothing accessories based on the user identity; The display end is used to display simulated user wearing clothing accessories; the data storage end is used to store data of the image acquisition end, the recognition end, and the recommendation service end.

To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided an electronic device simulating a user's clothing accessories, including: one or more processors; a storage device, used to store one or more programs, when The one or more programs are executed by the one or more processors, so that the one or more processors implement a method for simulating a user to wear clothing accessories according to an embodiment of the present invention.

To achieve the above object, according to still another aspect of the embodiments of the present invention, a computer-readable storage medium is provided on which a computer program is stored, and when the program is executed by a processor, a simulated user of the embodiment of the present invention Ways to wear clothing accessories.

An embodiment of the above invention has the following advantages or beneficial effects: because the user's full-body image is acquired, the whole-body image is detected based on the evaluation model to evaluate the user's posture; the user's identity is identified, and clothing accessories are recommended based on the user's identity; based on the geometric moment And edge detection recognition algorithms recognize user gestures and obtain gesture commands; based on user gestures and gesture commands to simulate the technical means of wearing clothing accessories, the evaluation model is an Openpose network structure based on MobileNet algorithm, using MobileNet as a feature extraction layer to replace openpose network The ordinary convolutional layer in the structure can reduce the number of parameters and the amount of calculation. At the same time, the recognition algorithm based on geometric moments and edge detection can improve the accuracy of gesture recognition, so it overcomes the huge amount of calculation, time-consuming, and high cost; accurate gesture recognition Low degree, the technical problem that requires users to wave their hands many times, and then to achieve the technical effect of reducing the amount of calculation, increasing the calculation speed and identifying accuracy.

The further effects provided by the above-mentioned non-conventional alternatives will be described in conjunction with specific implementations below.

BRIEF DESCRIPTION

The drawings are used to better understand the present invention and do not constitute an undue limitation on the present invention. among them:

FIG. 1 is a schematic diagram of main steps of a method for simulating a user to wear clothing accessories according to an embodiment of the present invention;

2 is a schematic diagram of the main flow of a method for simulating a user to wear a clothing accessory according to a reference embodiment of the present invention;

FIG. 3 is a schematic diagram of gesture recognition for simulating a method of wearing a costume ornament by a user according to an embodiment of the present invention;

4 is a schematic diagram of main modules of a device for simulating a user to wear a clothing accessory according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a system for simulating a user to wear clothing accessories according to an embodiment of the present invention;

6 is an exemplary system architecture diagram to which embodiments of the present invention can be applied;

7 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.

detailed description

The following describes exemplary embodiments of the present invention with reference to the accompanying drawings, which includes various details of the embodiments of the present invention to facilitate understanding, and they should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Also, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

It should be noted that the embodiments of the present invention and the technical features in the embodiments can be combined with each other without conflict.

FIG. 1 is a schematic diagram of main steps of a method for simulating a user to wear clothing accessories according to an embodiment of the present invention. As shown in FIG. 1, the method for simulating a user to wear a clothing accessory according to an embodiment of the present invention mainly includes the following steps:

Step S101: Acquire the user's whole-body image, and detect the whole-body image based on the evaluation model to evaluate the user's posture.

When simulating a user wearing a costume adornment, a full-body scan can be performed on the user to obtain a full-body image, and the full-body image can be detected based on the MobileNet algorithm and the openpose network structure to identify the user's user posture. The evaluation model of the embodiment of the present invention is an Openpose network structure based on the MobileNet algorithm, wherein the evaluation model selects MobileNet as the feature extraction layer to replace the ordinary convolutional layer in the openpose network structure. Using the depth-separable convolutions of the MobileNet algorithm to extract features in the whole-body image can reduce the number of parameters and the amount of calculation. At the same time, this separation structure can core decompose the amount of compression parameters, which can increase the calculation speed for the CPU of most mobile terminals.

MobileNets is based on a streamlined architecture, which uses deep separable convolutions to build lightweight deep neural networks. The openpose network structure is a human pose estimation algorithm. The principle of openpose is: input an image, extract the features through the convolution network, get a set of feature maps, and then divide it into two parts, using ordinary convolutional layers (such as convolution) Neural network CNN) extracts partial confidence maps (Part Confidence Maps) and partial affinity fields (Part Affinity Fields); after obtaining these two pieces of information, uses even matching (Bipartite Matching) to find partial associations (Part Associations), which will be the same The joint points of the individuals are connected. Due to the vector nature of the partial affinity field, the generated even match is very correct, and finally merged into a person's overall skeleton, and then through six stages of detection (stage), each stage has two Branch detection to generate keypoint heatmap and vectmap respectively. With heatmap and vectmap, you can know all the key points in the picture. In addition, the evaluation model can be trained using the MobileNet algorithm, and the training process is similar to the evaluation process.

In the embodiment of the present invention, step S101 can be implemented in the following ways: using the MobileNet algorithm to extract bone features from the whole-body image; using the openpose network structure to extract bone line segments and key point vectors of the bone line segments from the bone features; using cosine similarity, clips The weights of angular cosine and skeletal line segments are calculated on key point vectors to evaluate user poses.

Bone features are the characteristics of the user's limbs. Bone line segments are the line segments on the limbs in the image. Generally, each limb in the image can be extracted into two bone line segments. The bone line segments represent the user's limbs, which is more in line with the human physiological structure and can be intuitively reflected The user's posture (that is, the movement of the human arm and leg).

Among them, the cosine similarity can be calculated according to the following formula:

x and y represent two vectors of skeletal line segments, i = 1, 2, 3, ..., n;

The angle cosine can be calculated according to the following formula:

cos (θ ₁ -θ ₂ ), θ ₁ -θ ₂ represents the angle between two adjacent bone segments, θ ₁ and θ ₂ are the included angle between two adjacent bone segments and the horizontal line, or θ ₁ and θ ₂ are respectively The angle between two adjacent bone segments and the vertical line;

For human behavior in special scenes, different bone segments should be given different weights. Based on historical evaluation experience, bone segment weights can be assigned corresponding weight values for bone segments corresponding to limbs, or bone segments corresponding to each part of a limb Assign corresponding weight values, etc. For example, the bone segment weights of the bone segments corresponding to the thigh and the lower leg may be the same or different.

Step S102: Identify the user's identity, and recommend costume accessories based on the user's identity.

Recognizing the user's identity can more accurately recommend suitable clothing accessories to the user. You can recommend clothing accessories to the user according to the recommendation strategy or user preferences set by the merchant, or you can recommend clothing accessories with high sales or high evaluation to the user.

It should be noted that the recommendation of the clothing accessory according to the identity of the user may also be to obtain the clothing accessory selected by the user after determining the user name of the user, that is, to recommend the clothing accessory to himself.

In the embodiments of the present invention, the user identity may include a user name, user gender, and user age.

Step S102 can be implemented by: acquiring the user's facial image when the user's posture matches the preset posture; adjusting the size of the facial image to generate facial images of different sizes to construct an image pyramid; using multi-layer neural network structure detection The image pyramid obtains the face frame, and recognizes the user name based on the face frame; the face image is detected using a classification model to determine the user's gender and user's age.

Neural networks were first proposed by psychologists and neurobiologists. Because neural networks can provide a relatively simple method when solving complex problems, image analysis and processing are commonly used. Various neural network models describe and simulate biological neural systems at different levels from different angles, which can realize functions such as function approximation, data clustering, pattern classification, and optimization calculation. The multi-layer neural network structure includes an input layer, a hidden layer, and an output layer, where the number of hidden layers depends on needs. The embodiment of the present invention uses a multi-layer neural network structure to detect the facial image to obtain the user's face frame. The face frame is composed of face feature points. The face feature points are the contours of the eyes, eyebrows, nose, mouth, and outer contour of the face. Location, you can identify the user name through the face frame. Among them, the number of layers to build the image pyramid is determined by two factors, the first is the minimum face size, the second is the scaling factor, the minimum face size (minsize) can be expressed by min (w, h), w is the face The width of the image and h is the height of the facial image. Since the human face is almost indistinguishable when the image size is less than 12, the adjusted facial image size "minL" cannot be less than 12. The number of layers of the image pyramid can be determined according to the following formula:

minL ﹥ 12, org_L is the size of the face image, minsize is the minimum face size, factor is the scaling factor, and n is the number of layers of the image pyramid. The minisize is artificially set according to the application scenario. In the case where minL is greater than 12, all n constitute the pyramid layer. Therefore, the smaller the value of minsize, the larger the range of n, and the amount of calculation will increase accordingly. The smaller the face that can be detected, you can continuously adjust minSize to determine the range of n to ensure a suitable position interval Of users can be detected, neither too close nor too far away, which can improve the user experience while optimizing computing.

In addition, the classification model can include three convolutional layers and two fully connected layers to avoid overfitting. Each convolutional layer in a convolutional neural network is composed of several convolutional units, and the parameters of each convolutional unit are optimized by a back-propagation algorithm. The purpose of the convolution operation is to extract different features of the input. The first convolutional layer may only extract some low-level features such as edges, lines, and corners. More layers of the network can iteratively extract more complex features from the low-level features. Characteristics. Each node of the fully connected layer is connected to all nodes of the previous layer, and is used to synthesize the extracted features. Since both age and gender can be divided into a limited number of categories, the detection of user gender and user age is a classification problem. A classification model can be used to detect facial images to determine the user gender and user age corresponding to the facial image.

Step S103: A recognition algorithm based on geometric moments and edge detection recognizes user gestures and obtains gesture commands.

In order to meet the needs of users to simulate wearing different clothing accessories and improve the user experience, the user can implement operations such as replacing simulated clothing accessories or collecting simulated clothing accessories through gestures.

In the embodiment of the present invention, step S101 may be implemented by: positioning and tracking the user's hand; acquiring the user's gesture image, and binarizing the gesture image; using a geometric moment and edge detection recognition algorithm Calculate the seven geometric moment feature components of the gesture image, and select four geometric moment feature components from the seven geometric moment feature components as the geometric moment feature vector; generate the gray image of the gesture image, detect the edges of the gray image, and obtain the gesture image The boundary direction feature vector of; based on the geometric moment feature vector and the boundary direction feature vector, calculate the distance between the gesture image and any gesture in the gesture library to obtain the gesture command.

In the embodiment of the present invention, various gesture commands are stored in the gesture library. When detecting the gesture commands, the gestures closest to the user's gesture image in the gesture library are directly searched to obtain the user's gesture commands. Among them, the binarization process is to set the gray value of the pixels on the image to 0 or 255, that is, the process of displaying the entire image with a clear black and white effect. The black and white image can make the calculation of the image more accurate. Geometric moment is a gesture recognition method based on statistical analysis. The translation, rotation, and scale transformation of the seven moment group images are kept unchanged. The moment group is the geometric moment feature component; edge detection is image processing and computer The basic problem in vision, the purpose of edge detection is to identify the obvious changes in brightness in digital images. The distance between images is the geometric feature distance between the input gesture image and any gesture image in the gesture library.

The gesture recognition process based on statistics and probability is easier to control and can recognize more complex and delicate gestures. In addition, based on the geometric moment feature vector and the boundary direction feature vector, when calculating the distance between the gesture image and any gesture in the gesture library, weights can be set for the geometric moment feature vector and the boundary direction feature vector according to actual needs.

Step S104: Simulate the user's wearing of clothing accessories based on the user's posture and gesture commands.

After obtaining the user's posture expressed by the skeleton line segment, it is possible to simulate the user wearing the recommended costume ornament based on the user's command (that is, the gesture command).

According to the method of simulating a user wearing a clothing accessory according to an embodiment of the present invention, it can be seen that the whole body image of the user is acquired and the whole body image is detected based on the evaluation model to evaluate the user's posture; the user identity is identified and the clothing accessory is recommended according to the user identity; Recognition algorithms based on geometric moments and edge detection recognize user gestures and obtain gesture commands; based on user gestures and gesture commands to simulate the technical means of wearing clothing accessories, so overcoming the huge amount of calculation, time-consuming, high cost; gesture recognition accuracy is low , The technical problem that requires users to wave their hands many times to achieve the technical effect of reducing the amount of calculation, increasing the calculation speed and identifying accuracy.

FIG. 2 is a schematic diagram of a main flow of a method for simulating a user to wear a clothing accessory according to a reference embodiment of the present invention. As shown in FIG. 2, the method for simulating a user wearing a clothing accessory according to an embodiment of the present invention may be implemented using the following process:

Step S201: gesture recognition: the camera can be used for continuous scanning, and only the actions matching the fixed gesture can be recognized, so as to enter the next step;

Step S202: Face recognition: the camera is also used to scan the face, and then the face is detected to identify the user, and it can also determine whether a member has been registered, etc .;

Step S203: gender identification;

Step S204: age recognition;

Step S205: Recommended strategy:

The result after recognition enters the recommendation system, intelligently recommends according to gender, age and recommendation strategy, and can also set member users to browse more preferential matching products, etc .;

Step S206: Gesture recognition: the user's gesture is recognized, so that different matching schemes are switched for display.

During the implementation of steps S201-S206, the relevant data can be stored to the data storage side, and the relevant data can also be obtained from the data storage side.

FIG. 3 is a schematic diagram of gesture recognition for simulating a method of wearing a costume ornament by a user according to an embodiment of the present invention. As shown in Figure 3, gesture recognition mainly includes the following steps:

Step S301: hand positioning and tracking;

Step S302: Hand feature extraction and preprocessing:

Gesture images are segmented, and binarization is performed on the gesture images;

Step S303: hand feature vector parameters: the geometric moment and edge detection recognition algorithm is used to calculate the seven geometric moment feature components of the gesture image, and four geometric moment feature components are selected from the seven geometric moment feature components as the geometric moment feature vector; And generating a grayscale image of the gesture image, detecting the edges of the grayscale image, and obtaining the boundary direction feature vector of the gesture image;

Step S304: Gesture recognition: based on the geometric moment feature vector and the boundary direction feature vector, calculate the distance between the gesture image and any gesture in the gesture library;

Step S305: Recognition result: obtain a gesture command according to the distance between the gesture image and any gesture in the gesture library.

In order to further illustrate the technical idea of the present invention, the classification model of the embodiment of the present invention will now be described in conjunction with specific application scenarios.

The classification model of the embodiment of the present invention includes:

The first layer: the convolution layer: 96 convolution kernels can be used, and the number of parameters of each convolution kernel is 3 * 7 * 7, which is equivalent to three 7 * 7 convolution kernels in each channel convolution. The activation function uses a linear rectification function (ReLU), the pooling uses the maximum overlapping pooling, the size of the pooling is 3 * 3, and the strides is 2; Among them, ReLU is also called a modified linear unit, which is a commonly used activation in artificial neural networks. Function (activation), usually refers to the nonlinear function represented by the ramp function and its variants;

The second layer: the convolution layer: the input of the second layer is a 96 * 28 * 28 single-channel image, because the three channels have been combined for convolution in the previous step;

The third layer: the convolution layer: the number of filters can be 384, and the size of the convolution kernel can be 3 * 3;

The fourth layer: fully connected layer: the first fully connected layer, the number of neurons can be selected 512;

The fifth layer: fully connected layer: the second fully connected layer, the number of neurons can also choose 512.

For the training of the classification model according to the embodiment of the present invention, the image processing can be directly processed with 3-channel color images, and the resized facial images can be uniformly scaled to 256 * 256, and then cropped to 227 * 227, and the training process can be random Cropping, randomly cropping multiple pictures to train helps to make the network recognition rate higher. In the verification test process, the four corners of the rectangle + the center are cropped, which means that the input of the network is a 227 * 227 3-channel color image, and the model is continuously optimized by using a small learning rate and using parameter regularization (dropout). Make it more accurate.

FIG. 4 is a schematic diagram of main modules of a device for simulating a user to wear a clothing accessory according to an embodiment of the present invention. As shown in FIG. 4, the apparatus 400 for simulating a user's wearing of clothing accessories according to an embodiment of the present invention includes: an evaluation module 401, a first recognition module 402, a second recognition module 403, and a simulation module 404. among them,

The evaluation module 401 is used to obtain a user's whole-body image and detect the whole-body image based on an evaluation model to evaluate the user's posture; wherein the evaluation model is an openpose network structure based on the MobileNet algorithm;

The first identification module 402 is used to identify the user's identity and recommend costume accessories according to the user's identity;

The second recognition module 403 is used to recognize the user's gesture based on the recognition algorithm based on geometric moment and edge detection and obtain the gesture command;

The simulation module 404 is used for simulating a user to wear a costume ornament based on the user posture and the gesture command.

In the embodiment of the present invention, the evaluation module 401 is further configured to: use the MobileNet algorithm to extract bone features from the whole-body image; use the openpose network structure to extract bone line segments and the bone line segments from the bone features The key point vector of; calculates the key point vector using the cosine similarity, the angle cosine and the weight of the bone line segment to evaluate the user's posture.

In addition, the user identity includes a user name, a user's gender, and a user's age; and the first recognition module 402 is further used to: obtain a user's facial image when the user's posture matches a preset posture; adjust the face The size of the image, generating facial images of different sizes to construct an image pyramid; wherein, the number of layers of the image pyramid is determined according to the following formula:

minL ﹥ 12, org_L is the size of the face image, minsize is the minimum face size, factor is the scaling factor, and n is the number of layers of the image pyramid; the multi-layer neural network structure is used to detect the image pyramid to obtain the face Frame, identifying the user name based on the face frame; detecting the facial image using a classification model to determine the user's gender and the user's age; wherein, the classification model includes three convolutional layers And two fully connected layers.

In the embodiment of the present invention, the second recognition module 403 is further used to: locate and track the user's hand; acquire the user's gesture image, and binarize the gesture image; use geometric moment and The edge detection recognition algorithm calculates seven geometric moment feature components of the gesture image, and selects four geometric moment feature components from the seven geometric moment feature components as geometric moment feature vectors; generates the gray of the gesture image Degree map, detecting the edge of the grayscale image to obtain the boundary direction feature vector of the gesture image; based on the geometric moment feature vector and the boundary direction feature vector, calculating any of the gesture image and the gesture library Gesture distance to get gesture commands.

It can be seen from the device for simulating the wearing of clothing accessories according to the embodiment of the present invention. The device for simulating the wearing of clothing accessories according to the embodiment of the present invention can be seen because the whole body image of the user is acquired and the whole body image is detected based on the evaluation model , To evaluate the user's posture; identify the user's identity, and recommend clothing accessories based on the user's identity; recognition algorithms based on geometric moments and edge detection recognize user gestures and obtain gesture commands; based on the user's gestures and gesture commands to simulate the technical means of wearing clothing accessories, so It overcomes the technical problem of huge calculation amount, long time consumption and high cost; low accuracy of gesture recognition, which requires users to wave their hands many times, thereby achieving the technical effects of reducing the amount of calculation, increasing the calculation speed and identifying accuracy.

FIG. 5 is a schematic diagram of a system for simulating a user to wear clothing accessories according to an embodiment of the present invention. As shown in FIG. 5, an embodiment of the present invention also provides a system 500 for simulating a user's clothing accessories. The system 500 for simulating a user's clothing accessories includes an image acquisition terminal 501, an identification terminal 502, a recommendation server 503, and a display terminal 504 和数据存端 505。 504 and data storage terminal 505. among them,

The image acquisition terminal 501 is used to acquire the user's full-body image, facial image and gesture image;

The recognition terminal 502 is used to evaluate the user's posture, user identity and obtain gesture commands;

The recommendation server 503 is used to recommend clothing accessories based on the user's identity;

The display end 504 is used to display simulated user wearing clothing accessories;

The data storage terminal 505 is used to store data of the image acquisition terminal, the recognition terminal, and the recommendation server.

According to the system for simulating a user wearing a clothing accessory according to an embodiment of the present invention, it can be seen that the whole body image of the user is acquired and the whole body image is detected based on the evaluation model to evaluate the user's posture; the user identity is identified and the clothing accessory is recommended according to the user identity; Recognition algorithms based on geometric moments and edge detection recognize user gestures and obtain gesture commands; based on user gestures and gesture commands to simulate the technical means of wearing clothing accessories, so overcoming the huge amount of calculation, time-consuming, high cost; gesture recognition accuracy is low , The technical problem that requires users to wave their hands many times to achieve the technical effect of reducing the amount of calculation, increasing the calculation speed and identifying accuracy.

FIG. 6 shows an exemplary system architecture 600 to which the method for simulating the wearing of clothing accessories or the device for simulating the wearing of clothing accessories of the embodiments of the present invention can be applied. As shown in FIG. 6, the system architecture 600 may include

terminal devices

601, 602, and 603, a network 604, and a server 605. The network 604 is used as a medium for providing communication links between the

terminal devices

601, 602, and 603 and the server 605. The network 604 may include various connection types, such as wired, wireless communication links, or fiber optic cables, and so on.

The user can use the

terminal devices

601, 602, 603 to interact with the server 605 via the network 604 to receive or send messages, etc. Various communication client applications, such as shopping applications, web browser applications, search applications, instant communication tools, email clients, and social platform software, can be installed on the

terminal devices

601, 602, and 603.

The

terminal devices

601, 602, and 603 may be various electronic devices that have a display screen and support web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and so on.

The server 605 may be a server that provides various services, for example, a background management server that provides support for shopping websites browsed by users using

terminal devices

601, 602, and 603. The background management server may perform analysis and other processing on the received product information query request and other data, and feed back the processing results (such as target push information and product information) to the terminal device.

It should be noted that the method for simulating the wearing of clothing accessories provided by the embodiment of the present invention is generally executed by the server 605, and accordingly, the device for simulating the wearing of clothing accessories by the user is generally provided in the server 605.

It should be understood that the numbers of terminal devices, networks, and servers in FIG. 6 are only schematic. According to the implementation needs, there can be any number of terminal devices, networks and servers.

7, which shows a schematic structural diagram of a computer system 700 suitable for implementing a terminal device according to an embodiment of the present invention. The terminal device shown in FIG. 7 is only an example, and should not bring any limitation to the functions and use scope of the embodiments of the present invention. As shown in FIG. 7, the computer system 700 includes a central processing unit (CPU) 701 that can be loaded into a random access memory (RAM) 703 from a program stored in a read-only memory (ROM) 702 or from a storage section 708 Instead, perform various appropriate actions and processing. In the RAM 703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701, ROM 702, and RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc .; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc .; a storage section 708 including a hard disk, etc. ; And a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. The drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is installed on the drive 710 as necessary, so that the computer program read out therefrom is installed into the storage portion 708 as needed.

In particular, according to the disclosed embodiments of the present invention, the process described above with reference to the flowchart may be implemented as a computer software program. For example, the disclosed embodiments of the present invention include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-described functions defined in the system of the present invention are executed.

It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer diskettes, hard drives, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, the computer-readable signal medium may include a data signal that is propagated in baseband or as part of a carrier wave, in which computer-readable program code is carried. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. . The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the drawings illustrate the possible implementation architecture, functions, and operations of the system, method, and computer program product according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more for implementing a specified logical function Executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession can actually be executed in parallel, and sometimes they can also be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and a combination of blocks in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be used It is realized by a combination of dedicated hardware and computer instructions.

The modules described in the embodiments of the present invention may be implemented in software or hardware. The described module may also be provided in the processor. For example, it may be described as: a processor includes an evaluation module, a first identification module, a second identification module, and an analog module. In some cases, the names of these modules do not constitute a limitation on the module itself. For example, the simulation module may also be described as "a module for simulating a user to wear a costume ornament based on the user gesture and the gesture command".

As another aspect, the present invention also provides a computer-readable medium. The computer-readable medium may be included in the device described in the foregoing embodiments; or it may exist alone without being assembled into the device. The computer-readable medium carries one or more programs. When the one or more programs are executed by a device, the device includes: Step S101: Acquire a user's whole-body image and detect the whole-body image based on an evaluation model, To evaluate the user's posture; Step S102: Recognize the user's identity and recommend clothing accessories according to the user's identity; Step S103: Recognize the user's gesture based on the recognition algorithm based on geometric moments and edge detection to obtain the gesture command; Step S104: Simulate the user based on the user's gesture and gesture command Wear clothing accessories.

According to the technical solution of the embodiment of the present invention, the whole-body image of the user is acquired and the whole-body image is detected based on the evaluation model to evaluate the user's posture; the user's identity is recognized, and clothing accessories are recommended according to the user's identity; recognition based on geometric moments and edge detection The algorithm recognizes user gestures and obtains gesture commands; based on the user's gestures and gesture commands, the technical means of simulating the user's wearing clothing accessories, so it overcomes the huge calculation, time-consuming, and high cost; the gesture recognition accuracy is low, requiring the user to wave many times. The problem is to achieve the technical effect of reducing the amount of calculation, increasing the calculation speed and identifying accuracy.

The above specific embodiments do not limit the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can occur depending on design requirements and other factors. Any modification, equivalent replacement and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

A method for simulating a user to wear clothing accessories, which is characterized by including:

Obtaining the user's whole-body image and detecting the whole-body image based on the evaluation model to evaluate the user's posture; wherein the evaluation model is an openpose network structure based on the MobileNet algorithm;

Identify the user's identity and recommend costume accessories based on the user's identity;

Recognition algorithms based on geometric moments and edge detection recognize user gestures and obtain gesture commands;

Based on the user's posture and the gesture command, the user wears a clothing accessory.
The method according to claim 1, wherein detecting the whole-body image based on the evaluation model to evaluate the user's posture includes:

Using the MobileNet algorithm to extract bone features from the whole-body image;

Extracting bone line segments and key point vectors of the bone line segments from the bone features using the openpose network structure;

The key point vector is calculated using the cosine similarity, the angle cosine and the weight of the bone line segment to evaluate the user's posture.
The method according to claim 2, wherein the user identity includes a user name, user gender, and user age; and

Identifying users includes:

When the user's posture matches the preset posture, obtain the user's facial image;

Adjust the size of the facial image to generate facial images of different sizes to construct an image pyramid; wherein, the number of layers of the image pyramid is determined according to the following formula:

minL ﹥ 12, org_L is the size of the face image, minsize is the minimum face size, factor is the scaling factor, and n is the number of layers of the image pyramid;

Detecting the image pyramid using a multi-layer neural network structure to obtain a face frame, and identifying the user name based on the face frame;

A classification model is used to detect the facial image to determine the user's gender and the user's age; wherein, the classification model includes three convolutional layers and two fully connected layers.
The method according to claim 1, wherein the recognition algorithm based on geometric moments and edge detection recognizes user gestures, and obtaining gesture commands includes:

Position and track the user's hand;

Acquiring a gesture image of a user, and performing binary processing on the gesture image;

A geometric moment and edge detection recognition algorithm is used to calculate seven geometric moment feature components of the gesture image, and four geometric moment feature components are selected from the seven geometric moment feature components as geometric moment feature vectors;

Generating a grayscale image of the gesture image, detecting edges of the grayscale image, and obtaining a boundary direction feature vector of the gesture image;

Based on the geometric moment feature vector and the boundary direction feature vector, the distance between the gesture image and any gesture in the gesture library is calculated to obtain a gesture command.
A device for simulating a user to wear a costume ornament is characterized in that it includes:

The evaluation module is used to obtain a user's whole-body image and detect the whole-body image based on an evaluation model to evaluate the user's posture; wherein the evaluation model is an openpose network structure based on the MobileNet algorithm;

The first identification module is used to identify the user's identity, and recommend costume accessories based on the user's identity;

The second recognition module is used to recognize the user's gesture based on the recognition algorithm based on geometric moment and edge detection and obtain the gesture command;

The simulation module is used for simulating a user to wear a clothing ornament based on the user posture and the gesture command.
The apparatus according to claim 5, wherein the evaluation module is further used to:

Using the MobileNet algorithm to extract bone features from the whole-body image;

Extracting bone line segments and key point vectors of the bone line segments from the bone features using the openpose network structure;

The key point vector is calculated using the cosine similarity, the angle cosine and the weight of the bone line segment to evaluate the user's posture.
The apparatus according to claim 6, wherein the user identity includes a user name, user gender, and user age; and

The first identification module is also used to:

When the user's posture matches the preset posture, obtain the user's facial image;

Adjust the size of the facial image to generate facial images of different sizes to construct an image pyramid; wherein, the number of layers of the image pyramid is determined according to the following formula:

minL ﹥ 12, org_L is the size of the face image, minsize is the minimum face size, factor is the scaling factor, and n is the number of layers of the image pyramid;

Detecting the image pyramid using a multi-layer neural network structure to obtain a face frame, and identifying the user name based on the face frame;

A classification model is used to detect the facial image to determine the user's gender and the user's age; wherein, the classification model includes three convolutional layers and two fully connected layers.
The device according to claim 5, wherein the second identification module is further used to:

Position and track the user's hand;

Acquiring a gesture image of a user, and performing binary processing on the gesture image;

A geometric moment and edge detection recognition algorithm is used to calculate seven geometric moment feature components of the gesture image, and four geometric moment feature components are selected from the seven geometric moment feature components as geometric moment feature vectors;

Generating a grayscale image of the gesture image, detecting edges of the grayscale image, and obtaining a boundary direction feature vector of the gesture image;

Based on the geometric moment feature vector and the boundary direction feature vector, the distance between the gesture image and any gesture in the gesture library is calculated to obtain a gesture command.
A system for simulating a user's clothing accessories is characterized in that it includes an image acquisition end, an identification end, a recommendation service end, a display end, and a data storage end, where:

The image acquisition terminal is used to acquire the user's full-body image, facial image and gesture image;

The recognition terminal is used to evaluate the user's posture, user identity and obtain gesture commands;

The recommendation server is used for recommending clothing accessories according to the user identity;

The display terminal is used to display simulated user wearing clothing accessories;

The data storage end is used to store data of the image acquisition end, the identification end, and the recommendation server end.
An electronic device simulating a user's clothing accessories is characterized by including:

One or more processors;

Storage device for storing one or more programs,

When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-4.
A computer-readable medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the method according to any one of claims 1-4 is implemented.