CN106557489B - A mobile terminal-based clothing search method - Google Patents

Abstract

The present invention provides a clothing search method based on a mobile terminal. A user takes a picture through a mobile terminal or selects a clothing picture from a gallery of the mobile terminal and uploads it to a server for clothing search, which specifically includes: using a web crawler to search for clothing from an e-commerce website Obtain clothing product pictures and product information, store them in the clothing database, and perform image feature extraction to form an image feature pool; in response to the clothing pictures taken or selected by the user through the mobile terminal, after the clothing pictures are cropped, a matching query is performed on the server; For the queried clothing result, the queried similar clothing pictures and corresponding clothing information are displayed to the user through the mobile terminal, and the clothing information includes the brand, price and material of the clothing. The method of the invention combines the visual characteristics of clothing commodities, provides users with a search plan for clothing commodities in the manner of "searching for pictures by image", and improves the user's experience of searching for clothing commodities.

Description

A mobile terminal-based clothing search method

technical field

The present invention relates to the technical field of mobile Internet, in particular to the field of image search based on mobile Internet, and in particular to a method for searching clothing based on mobile terminals.

Background technique

With the rapid development of the mobile Internet and the popularization of mobile terminals, e-commerce transactions are changing from traditional PC terminals to mobile terminals. Mobile terminals have the advantages of small size and easy portability. More and more users prefer mobile terminals to search for and purchase apparel products. Most of the traditional clothing search methods search for clothing products by means of plain text keywords. This method is too simple and ignores the important picture information in the e-commerce information. Moreover, users cannot accurately describe the keywords of the products. Clothing products contain rich content, and it is often difficult to fully describe them with a small amount of text. For example, it is difficult for users to accurately describe attributes such as clothing styles, shapes, and patterns. Users with good expressive ability describe the characteristics of clothing, and it is difficult to search for clothing products similar to the clothing, and the search results are difficult to satisfy users.

Based on the above problems, it is necessary for those skilled in the art to propose a new clothing search method.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a mobile terminal-based clothing commodity search method, combined with the visual characteristics of clothing commodities, to provide users with a way of "searching for pictures by image" to search for clothing commodities, and to improve the user's experience of searching for clothing commodities.

In order to achieve the above purpose, the present invention proposes a mobile terminal-based clothing search method. The client takes a photo or selects a clothing picture from the client gallery and uploads it to the server to search for clothing, including the following steps:

(1) Establishment of clothing image feature pool

Obtain clothing product pictures and product information from e-commerce websites through web crawlers, store them in the clothing database, and perform image feature extraction on the clothing pictures in the clothing database to form an image feature pool, and the clothing database is stored in the server.

(2) Search for similar products

In response to the clothing picture taken or selected by the user through the mobile terminal, the clothing picture is cropped, and then uploaded to the aforementioned server, and the feature extraction is performed on the clothing picture uploaded by the user on the server, and then matched with the image features stored in the image feature pool. Inquire

(3) Display of search results

According to the queried clothing results, the queried similar clothing pictures and corresponding clothing information are displayed to the user through the mobile terminal, and the clothing information includes the brand, price and material of the clothing.

Compared with the prior art, the clothing search method proposed by the present invention adopts the method of image search for clothing search, which changes the current method of using plain text keyword search, realizes "search for pictures by picture", and can be used anytime, anywhere, and is convenient to carry. The mobile terminal takes photos of the clothing, and has a more direct and accurate description of the actual clothing. The use of image-based clothing search effectively solves the difficulty that users cannot or are unwilling to use text to search, and reduces the difficulty of shopping. The visual characteristics meet users' higher needs for search functions, and it is convenient for users to quickly find clothing information. Improved user experience.

It is to be understood that all combinations of the foregoing concepts, as well as additional concepts described in greater detail below, are considered to be part of the inventive subject matter of the present disclosure to the extent that such concepts are not contradictory. Additionally, all combinations of the claimed subject matter are considered to be part of the inventive subject matter of this disclosure.

The foregoing and other aspects, embodiments and features of the present teachings can be more fully understood from the following description when taken in conjunction with the accompanying drawings. Other additional aspects of the invention, such as features and/or benefits of the exemplary embodiments, will be apparent from the description below, or learned by practice of specific embodiments in accordance with the teachings of this invention.

Description of drawings

The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by the same reference numeral. For clarity, not every component is labeled in every figure. Embodiments of various aspects of the present invention will now be described by way of example and with reference to the accompanying drawings, wherein:

FIG. 1 is a schematic diagram of a specific implementation manner of a mobile terminal-based clothing search method of the present invention.

FIG. 2 is a specific embodiment of the method for cutting clothing pictures according to the present invention.

FIG. 3 is a schematic diagram of cutting a garment picture by the smearing method shown in FIG. 2 .

FIG. 4 is a flow chart of a specific embodiment of the smearing method of the present invention.

FIG. 5 is a flow chart of a specific implementation manner of query matching according to the present invention.

FIG. 6 is a structural diagram of a specific implementation manner of a mobile terminal-based clothing search device of the present invention.

Detailed ways

In order to better understand the technical content of the present invention, specific embodiments are given and described below in conjunction with the accompanying drawings.

Aspects of the invention are described in this disclosure with reference to the accompanying drawings, in which a number of illustrative embodiments are shown. Embodiments of the present disclosure are not necessarily intended to include all aspects of the invention. It should be understood that the various concepts and embodiments described above, as well as those described in greater detail below, can be implemented in any of a number of ways, as the concepts and embodiments disclosed herein do not limited to any implementation. Additionally, some aspects of the present disclosure may be used alone or in any suitable combination with other aspects of the present disclosure.

With reference to Fig. 1, according to an embodiment of the present invention, a mobile terminal-based clothing commodity search method, the client takes a picture or selects a clothing picture from the client's gallery and uploads it to the server to search for clothing, including the following steps :

(1) Establishment of clothing image feature pool

Through the web crawler technology, clothing product pictures and product information are obtained from major e-commerce websites, stored in the clothing database, and image features are extracted from the clothing pictures in the clothing database to form an image feature pool. The apparel database is stored in the server.

(2) Search for similar products

The user uploads the required clothing pictures to the server through the mobile terminal. When uploading the clothing pictures, the clothing pictures should be cropped, and the features of the clothing pictures uploaded by the user are extracted on the server, and then matched with the image features stored in the image feature pool.

Uploading the clothing picture through the mobile terminal includes: taking a picture for uploading or selecting a clothing picture from the mobile terminal for uploading.

The clothing picture cropping method includes the rectangular frame method and the smear method for cropping the picture.

The mobile terminal includes a mobile phone and a Pad.

(3) Display of search results

According to the queried clothing results, the queried similar clothing pictures and corresponding clothing information are displayed to the user through the mobile terminal, and the clothing information includes the brand, price and material of the clothing.

As shown in FIG. 1, in the aforementioned clothing search method of the present invention, a picture of clothing is uploaded by a mobile terminal or selected from a mobile terminal and uploaded to the server, and then feature extraction is performed on the server, and the image features are extracted according to the extracted image features. The matching query is carried out in the pool, and the similar clothing pictures found in the query are returned to the user.

The image feature pool is to obtain clothing pictures and corresponding clothing information (such as material, price, brand, etc.) from major e-commerce websites (such as TAOBAO, Jingdong Mall, Tmall Mall, etc.) through web crawler technology and store them in the clothing database. Then, feature extraction is performed on the clothing image to form an image feature pool. When the user searches for clothing pictures, the image feature pool used is the image feature pool established above.

In general, clothing is displayed by a model, and users do not necessarily like all parts of a piece of clothing. When uploading clothing pictures, users need to crop the pictures to extract the most wanted parts for uploading. .

FIG. 2 is a specific embodiment of a method for cropping clothing pictures. In this example, a method of drawing a rectangular frame is adopted, and the part that the user is interested in is selected for uploading. The method of drawing a rectangular frame is that the user selects a part of the favorite clothing through the mobile terminal, then searches the selected part for clothing, and returns the search result to the user.

FIG. 3 is a schematic diagram of cutting a garment picture by a smear method. Combined with the example shown in the figure, this method is a smear-type cropping method. The user smears the clothing area of interest with his finger through the mobile terminal, uploads it, and returns the search result to the user. Of course, if the result of the first search is unsatisfactory, the user can cut again and perform a second search until satisfactory clothing is found.

The user needs to cut the clothing picture to search for the most satisfactory clothing. Although the method of drawing a rectangle frame can achieve the cutting effect, the method of drawing a rectangle frame may select irrelevant areas, such as selecting the background frame of the clothing picture to In the search area, the results obtained from such a search are not the most satisfactory clothing for the user. Another example is that the user may select a relatively satisfactory area after several frames, which wastes the user's time and patience. The smear method provides a more precise and lighter cutting method, and users only need to smear the favorite clothing part with their fingers to search. For example, as shown in FIG. 3 , the clothing picture on the left side of FIG. 3 is the original picture, and the right side is the clothing picture after smearing. The application can be carried out in three steps:

Apply the collar first, the body second, and the arms last. As shown in the right image of Figure 3, green represents the collar part, light blue represents the body part, and yellow represents the arm part. Then upload the smeared part to the server for feature extraction, and perform matching query in the image feature pool according to the extracted image features, and return the queried similar clothing pictures to the user.

The smearing method divides a piece of clothing into three areas for smearing and cutting, which can be the collar part, the body part and the arm part in turn. In the example of FIG. 4, a specific implementation process of clothing search given the smearing method is shown. Figure, as shown in Figure 4, the specific steps include:

Step 401 , extracting local descriptors for the three regions smeared in the clothing image, and further processing the feature points by using bone coordinates in order to process the local information of the clothing features more accurately. The bone coordinates are converted from standard coordinates.

In step 402, the local descriptors of each region are mixed and encoded. The local descriptors of the present invention are represented by adjacent visual words, and the respective positions of the local descriptors of the three parts of the smeared clothing picture are x1,...xa,xa+ 1,...,xa+b,xa+b+1,...,xa+b+c, convert it to bone coordinates: x' ₁ ,..., x' _a , x' _a+1 , ..., x' _a+b , x' _a+b+1 , ..., x' _a+b+c , where a, b, c are the three partial local descriptors of the clothing image smearing respectively number, and feature extraction is performed using bone coordinates. The visual words are obtained by clustering the local descriptors in the image feature pool using the K-means algorithm.

Step 403 , use the mixed descriptor to search for clothing, and calculate the similarity between the local descriptor of the searched clothing image and the local descriptor of the image feature pool. The feature vector of clothing matches the query with the feature vector of each image in the image feature pool.

Step 404 , returning the clothing pictures and corresponding clothing information obtained by the matching query to the user.

FIG. 5 is a specific implementation of query matching, and the specific steps include:

Step 501, uploading a clothing picture from the mobile terminal to the clothing device terminal, the clothing picture is a cropped picture, and the cropping method uses the smear method to extract local descriptors for the uploaded clothing picture.

Step 502: Calculate the similarity between the uploaded clothing picture and each picture in the clothing database, where the similarity is calculated by a spatially constrained similarity measurement method. The method explores locations using a grid voting space, which is a voting map, which is based on the positional relationship of local descriptors, has the same visual words and can simultaneously calculate similarity. Specific steps include:

1) Voting and stacking the feature point combination of the two clothing pictures with the same visual word for similarity calculation, and the voting point is the position point where the two feature points and the motion vector value are the same.

2) The area with the highest voting point represents the projection, and the combination of voting points in general represents the similarity of the two clothing pictures.

Step 503: Calculate the similarity between the uploaded clothing picture and each clothing picture in the clothing database, and return the clothing picture with high similarity (for example, meeting the set threshold standard) to the user.

Corresponding to the specific implementation of the foregoing method, the present invention also discloses a mobile terminal-based clothing search device, as shown in FIG. 6 , which may specifically include:

Clothing picture acquisition module 601, this module acquires a clothing picture from the mobile terminal, and the specific method may be to take a picture or acquire a clothing picture from the mobile terminal and upload it to the server. In addition, the user can cut the clothing picture multiple times to select the part that the user likes and upload it until the clothing that the user is satisfied with is searched. The cropping method includes a rectangular frame drawing method and a smearing method.

The local descriptor extracting module 602 is used for extracting corresponding local descriptors according to the received clothing pictures, and also extracting local descriptors on the server side for calculating similarity with the received clothing pictures.

The clothing picture retrieval module 603 is configured to perform a matching query of the local descriptors in the server-side image feature pool according to the received local descriptors of the clothing pictures, and calculate the similarity between the two clothing pictures. The image feature pool stores local descriptor information of clothing pictures obtained from e-commerce websites.

The clothing picture returning module 604 returns the retrieved clothing pictures to the user for display on the user's mobile terminal.

Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Those skilled in the art to which the present invention pertains can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be determined according to the claims.

Claims (2)

1. A dress searching method based on a mobile terminal is characterized in that a user takes a picture through the mobile terminal or selects a dress picture from a gallery of the mobile terminal and uploads the dress picture to a server side for dress searching, and the dress searching method comprises the following steps:

step 1, establishing a clothing picture feature pool

Acquiring clothing commodity pictures and commodity information from an e-commerce website through a web crawler, storing the clothing commodity pictures and the commodity information into a clothing database, and performing image feature extraction on the clothing pictures in the clothing database to form an image feature pool, wherein the clothing database is stored in a server;

step 2, searching similar commodities

Responding to a clothing picture shot or selected by a user through a mobile terminal, cutting the clothing picture, uploading the clothing picture to the server, extracting the characteristics of the clothing picture uploaded by the user on the server, and then performing matching query with the image characteristics stored in the image characteristic pool;

the clothing picture cutting method comprises at least one of a rectangular frame method and a smearing method, wherein the rectangular frame method is to select the clothing part of interest of a user from a frame and upload the selected part of the frame to a server for clothing search, and the smearing method is to smear on the clothing picture by the user and upload the selected part of the interest clothing area to the server for clothing search;

in the clothing searching process based on the smearing and cutting method, the characteristics are extracted, the codes are mixed and the matching query is carried out in the following modes:

1) extracting local descriptors of three areas coated by the clothing picture, wherein the three areas are a collar part, a body part and a hand arm part of clothing respectively;

2) mixing and coding the local descriptors of the regions, wherein each local descriptor is represented by an adjacent visual word, and the respective positions of the local descriptors of three parts of the daubed clothing picture are as follows: x is the number of₁,...x_a,x_a+1,...,x_a+b,x_a+b+1,...,x_a+b+cIt is converted into bone coordinates: x'₁,…,x'_a,x'_a+1,…,x'_a+b,x'_a+b+1,…,x'_a+b+cWherein a, b and c are respectively partial descriptions of three parts of the painting of the clothing pictureThe number of the son, the characteristic is extracted and carried on by using the skeleton coordinate, the said vision word uses the K-means algorithm to carry on the clustering to the local descriptor in the characteristic pool of the picture to get;

3) using the mixed descriptors to search clothing, calculating the similarity between the local descriptors of the searched clothing picture and the local descriptors of the image feature pool, and performing matching query on the feature vector of the clothing and the feature vector of each image in the image feature pool;

step 3, displaying search results

And displaying the inquired similar clothing pictures and corresponding clothing information to a user through a mobile terminal according to the inquired clothing results, wherein the clothing information comprises the brand, price and material of the clothing.

2. The clothing searching method based on mobile terminal as claimed in claim 1, wherein in step 2 of the aforementioned method, the similarity is calculated by a space-constrained similarity measurement method, the similarity measurement method uses a grid voting space to search for a position, the grid voting space is a voting map which is based on the position relationship of local descriptors, has the same visual words and can simultaneously calculate the similarity, and the concrete steps include:

1) voting and superposing feature point combinations with the same visual words of the two clothes pictures subjected to similarity calculation, wherein the voting points are position points with the same motion vector values of the two feature points;

2) the region with the highest voting point represents a projection, and the combination of the voting points represents the similarity of the two clothes pictures as a whole;

and carrying out similarity calculation on the uploaded clothes pictures and each clothes picture in the clothes database according to the mode, and returning the clothes pictures with the similarity meeting the set threshold value to the user in the step 3.

Images