CN106557489B - Clothing searching method based on mobile terminal - Google Patents

Clothing searching method based on mobile terminal Download PDF

Info

Publication number
CN106557489B
CN106557489B CN201510624226.3A CN201510624226A CN106557489B CN 106557489 B CN106557489 B CN 106557489B CN 201510624226 A CN201510624226 A CN 201510624226A CN 106557489 B CN106557489 B CN 106557489B
Authority
CN
China
Prior art keywords
clothing
picture
user
mobile terminal
searching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510624226.3A
Other languages
Chinese (zh)
Other versions
CN106557489A (en
Inventor
李旻先
李刚
刘媛媛
赵春霞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN201510624226.3A priority Critical patent/CN106557489B/en
Publication of CN106557489A publication Critical patent/CN106557489A/en
Application granted granted Critical
Publication of CN106557489B publication Critical patent/CN106557489B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Processing Or Creating Images (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a clothing searching method based on a mobile terminal, wherein a user takes a picture through the mobile terminal or selects a clothing picture from a gallery of the mobile terminal and uploads the clothing picture to a server side for clothing searching, and the clothing searching method specifically comprises the following steps: obtaining clothing commodity pictures and commodity information from an e-commerce website through a web crawler, storing the clothing commodity pictures and the commodity information into a clothing database, and performing image feature extraction to form an image feature pool; in response to a clothing picture shot or selected by a user through a mobile terminal, cutting the clothing picture, and then performing matching query on a server; and displaying the inquired similar clothing pictures and corresponding clothing information to a user through a mobile terminal according to the inquired clothing results, wherein the clothing information comprises the brand, price and material of the clothing. The method provided by the invention combines the visual characteristics of the clothing commodity, provides a clothing commodity searching scheme in a pattern searching manner for the user, and improves the experience of the user in searching the clothing commodity.

Description

Clothing searching method based on mobile terminal
Technical Field
The invention relates to the technical field of mobile internet, in particular to the field of picture searching based on the mobile internet, and specifically relates to a clothing searching method based on a mobile terminal.
Background
With the rapid development of the mobile internet and the popularization of the mobile terminal, the electronic commerce transaction behavior is being changed from the traditional PC terminal to the mobile terminal. The mobile terminal has the advantages of small volume, convenience in carrying and the like, and more users prefer the mobile terminal to search for and purchase clothing commodities. Most of traditional clothing searching methods search clothing commodities in a pure text keyword mode, and the method is single and ignores important picture information in electronic commerce information. Moreover, the user cannot accurately describe the keywords of the product, and the clothing product contains rich content, which is often difficult to be sufficiently described by a small amount of characters, such as: the user can not accurately describe the attributes of the style, the shape, the pattern and the like of the clothes, even if the user with good expression ability describes the clothes characteristics, the user can not search clothes commodities similar to the clothes, and the search result is not satisfactory to the user.
Based on the above problems, a person skilled in the art is required to propose a new clothing search method.
Disclosure of Invention
The invention aims to provide a clothing commodity searching method based on a mobile terminal, which combines the visual characteristic of clothing commodities, provides a mode of searching for clothing commodities by pictures for users, and improves the experience of searching for clothing commodities for users.
In order to achieve the above purpose, the invention provides a clothing searching method based on a mobile terminal, a client side carries out clothing searching by shooting or selecting a clothing picture from a client side gallery to be uploaded to a server side, and the clothing searching method comprises the following steps:
(1) establishment of clothing picture feature pool
Obtaining clothing commodity pictures and commodity information from an E-commerce website through a web crawler, storing the clothing commodity pictures and the commodity information into a clothing database, and carrying out image feature extraction on the clothing pictures in the clothing database to form an image feature pool, wherein the clothing database is stored in a server
(2) Search for similar goods
Responding to the clothing picture shot or selected by the user through the mobile terminal, cutting the clothing picture, uploading the clothing picture to the server, extracting the characteristics of the clothing picture uploaded by the user on the server, and then matching and inquiring the clothing picture with the image characteristics stored in the image characteristic pool
(3) Search result presentation
And displaying the inquired similar clothing pictures and corresponding clothing information to a user through a mobile terminal according to the inquired clothing results, wherein the clothing information comprises the brand, price and material of the clothing.
Compared with the prior art, the clothing searching method provided by the invention adopts a picture searching mode to search clothing, changes the current pure text keyword searching mode, realizes 'searching pictures with pictures', conveniently uses a portable mobile terminal to shoot clothing photos anytime and anywhere, and has more direct and accurate description on clothing objects. By adopting the clothing search based on the image, the difficulty that the user cannot or is unwilling to search by using characters is effectively solved, the shopping difficulty is reduced, the visual characteristic meets the higher requirement of the user on the search function, the user can conveniently and quickly find the clothing information, and the user experience is greatly improved.
It should be understood that all combinations of the foregoing concepts and additional concepts described in greater detail below can be considered as part of the inventive subject matter of this disclosure unless such concepts are mutually inconsistent. In addition, all combinations of claimed subject matter are considered a part of the presently disclosed subject matter.
The foregoing and other aspects, embodiments and features of the present teachings can be more fully understood from the following description taken in conjunction with the accompanying drawings. Additional aspects of the present invention, such as features and/or advantages of exemplary embodiments, will be apparent from the description which follows, or may be learned by practice of specific embodiments in accordance with the teachings of the present invention.
Drawings
The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. Embodiments of various aspects of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which:
fig. 1 is a schematic diagram of a clothing searching method based on a mobile terminal according to a specific embodiment of the invention.
Fig. 2 is a specific embodiment of the method for cutting out the clothes picture.
Fig. 3 is a schematic diagram of a picture of a garment cut by the painting method shown in fig. 2.
FIG. 4 is a flow chart of an embodiment of the application method of the present invention.
FIG. 5 is a flow diagram of one embodiment of query matching.
Fig. 6 is a structure diagram of a clothing searching device based on a mobile terminal according to an embodiment of the invention.
Detailed Description
In order to better understand the technical content of the present invention, specific embodiments are described below with reference to the accompanying drawings.
In this disclosure, aspects of the present invention are described with reference to the accompanying drawings, in which a number of illustrative embodiments are shown. Embodiments of the present disclosure are not necessarily intended to include all aspects of the invention. It should be appreciated that the various concepts and embodiments described above, as well as those described in greater detail below, may be implemented in any of numerous ways, as the disclosed concepts and embodiments are not limited to any one implementation. In addition, some aspects of the present disclosure may be used alone, or in any suitable combination with other aspects of the present disclosure.
With reference to fig. 1, according to an embodiment of the present invention, a method for searching clothing articles based on a mobile terminal, where a client performs clothing search by taking a picture or selecting a clothing image from a client gallery and uploading the selected image to a server, includes the following steps:
(1) establishment of clothing picture feature pool
The method comprises the steps of obtaining clothing commodity pictures and commodity information from various E-commerce websites through a web crawler technology, storing the clothing commodity pictures and the commodity information into a clothing database, and extracting image features of the clothing pictures in the clothing database to form an image feature pool. The apparel database is stored in a server.
(2) Search for similar goods
A user uploads a required clothes picture to a server through a mobile terminal, the clothes picture is cut when the clothes picture is uploaded, the clothes picture uploaded by the user is subjected to feature extraction on the server, and then matching query is carried out on the clothes picture and image features stored in an image feature pool.
Uploading the clothing picture through the mobile terminal comprises: and taking a picture and uploading or selecting a clothing photo from the mobile terminal to upload.
The clothing picture cutting method comprises a rectangular frame method and a smearing method for cutting pictures.
The mobile terminal comprises a mobile phone and a Pad.
(3) Search result presentation
And displaying the inquired similar clothing pictures and corresponding clothing information to a user through a mobile terminal according to the inquired clothing results, wherein the clothing information comprises the brand, price and material of the clothing.
With reference to fig. 1, in the clothing search method of the present invention, a mobile terminal is used for taking a picture and uploading the picture or selecting a clothing picture from the mobile terminal and uploading the picture to a server, then feature extraction is performed on the server, matching query is performed in an image feature pool according to the extracted image features, and the queried similar clothing picture is returned to the user.
The image feature pool is formed by obtaining the clothing pictures and corresponding clothing information (such as material, price, brand and the like) from various E-commerce websites (such as TAOBAO, Jingdong mall, Tianmao mall and the like) through a web crawler technology, storing the clothing pictures and the corresponding clothing information into a clothing database, and extracting features of the clothing pictures. And when the user searches the clothes picture, the used image feature pool is the established image feature pool.
Generally, clothes are displayed by a model, a user does not necessarily like all parts of a piece of clothes, and when the user uploads a clothes picture, the picture is cut to extract the part which is most wanted by the user for uploading. .
Fig. 2 is a specific embodiment of the method for cutting out the clothes picture. In this example, a method of drawing a rectangular box which selects a portion of interest to the user and uploads the selected portion is used. The method for drawing the rectangular frame is that the user selects the favorite part of the clothes through the mobile terminal, then the clothes of the selected part of the frame is searched, and the searched result is returned to the user.
Fig. 3 is a schematic diagram of a painting method for cutting a picture of clothes. In connection with the illustrated example, the method is a smearing type cutting, and the user smears the interested clothing area with fingers through the mobile terminal and uploads the smeared area and returns the searched result to the user. Of course, if the result of the first search is not satisfactory, the user may cut again to perform the second search until the satisfactory clothes are searched.
The user needs to cut the clothing picture to search for the most satisfactory clothing, and the rectangular frame drawing method can achieve the cutting effect, but the rectangular frame drawing method may select irrelevant area frames, for example, a background frame of the clothing picture is selected in a search area, so that the searched result is not the most satisfactory clothing for the user. And if the user can only select a relatively satisfactory area through several picture frames, the time and patience of the user are wasted. The smearing method provides a more accurate and light cutting method, and a user only needs to smear favorite clothes parts with fingers for searching. For example, as shown in fig. 3, the left garment picture in fig. 3 is the original picture, and the right garment picture after painting. The application can be carried out in three steps:
first the collar portion, second the body portion, and finally the arm portion. As shown in the right drawing of fig. 3, green indicates the collar portion, light blue indicates the body portion, and yellow indicates the arm portion. And then uploading the smeared part to a server for feature extraction, performing matching query in an image feature pool according to the extracted image features, and returning the queried similar clothing images to the user.
The smearing method divides a piece of clothing into three areas for smearing and cutting, which can be a collar part, a body part and a hand arm part in turn, and in the example of fig. 4, a flow chart of a specific implementation mode of clothing searching given to the smearing method is shown, as shown in fig. 4, the specific steps include:
step 401, extracting local descriptors from three areas painted on the clothing image, and further processing feature points by using bone coordinates in order to more accurately process local information of clothing features. The bone coordinates are converted from standard coordinates.
Step 402, mixing and coding the local descriptors of the regions, wherein the local descriptors of the invention are represented by using adjacent visual words, and the positions of the local descriptors of three parts of the daubed clothing picture are x1,. xa, xa +1,. once, xa + b, xa + b +1,. once, xa + b + c, and converting the positions into bone coordinates: x'1，...，x’a，x’a+1，...，x’a+b，x’a+b+1，...，x’a+b+cAnd a, b and c are the number of local descriptors of three parts painted on the clothing picture respectively, and the feature extraction is carried out by using bone coordinates. And clustering the local descriptors in the image feature pool by using a K-means algorithm to obtain the visual words.
And step 403, using the mixed descriptors to perform clothing searching, and calculating the similarity between the local descriptors of the searched clothing picture and the local descriptors of the image feature pool. And matching and querying the feature vector of the clothing with the feature vector of each image in the image feature pool.
And step 404, returning the clothes picture obtained by matching and querying and the corresponding clothes information to the user.
Fig. 5 is a specific implementation of query matching, which includes the following specific steps:
step 501, uploading a clothing picture to a clothing device end from a mobile end, wherein the clothing picture is a cut picture, and the cutting method uses a smearing mode to extract a local descriptor of the uploaded clothing picture.
And 502, calculating the similarity between the uploaded clothing picture and each picture in the clothing database, wherein the similarity is calculated by a spatial constraint similarity measurement method. The method uses a grid voting space to explore the position, wherein the grid voting space is a voting map which is based on the position relation of local descriptors, has the same visual words and can simultaneously calculate the similarity. The method comprises the following specific steps:
1) and voting and superposing feature point combinations with the same visual words of the two clothes pictures subjected to similarity calculation, wherein the voting points are position points with the same motion vector values of the two feature points.
2) The region with the highest voting point represents the projection, and the combination of voting points represents the similarity of the two dress pictures as a whole.
And 503, performing similarity calculation on the uploaded clothing picture and each clothing picture in the clothing database, and returning the clothing picture with higher similarity (for example, meeting a set threshold value standard) to the user.
Corresponding to the foregoing method embodiment, the present invention also discloses a clothing searching device based on a mobile terminal, as shown in fig. 6, which may specifically include:
the clothing picture obtaining module 601 obtains a clothing picture from the mobile terminal, and the specific method can be to obtain a picture by taking a picture or obtain a clothing picture from the mobile terminal and upload the clothing picture to the server. In addition, the user can cut the clothing picture for multiple times to select the favorite part of the user for uploading until the clothing which is satisfied by the user is searched. The cutting method comprises a rectangular frame drawing method and a smearing method.
The local descriptor extracting module 602 is configured to extract a corresponding local descriptor according to the received clothing image, and additionally, at the server side, extract a local descriptor for performing similarity calculation with the received clothing image.
The clothing image retrieval module 603 is configured to perform matching query on the local descriptors in the server-side image feature pool according to the received local descriptors of the clothing images, and calculate similarity between the two clothing images. The image feature pool stores local descriptor information of the clothes pictures obtained from the E-commerce website.
The clothing image returning module 604 returns the retrieved clothing image to the user for displaying in the mobile terminal of the user.
Although the present invention has been described with reference to the preferred embodiments, it is not intended to be limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the protection scope of the present invention should be determined by the appended claims.

Claims (2)

1. A dress searching method based on a mobile terminal is characterized in that a user takes a picture through the mobile terminal or selects a dress picture from a gallery of the mobile terminal and uploads the dress picture to a server side for dress searching, and the dress searching method comprises the following steps:
step 1, establishing a clothing picture feature pool
Acquiring clothing commodity pictures and commodity information from an e-commerce website through a web crawler, storing the clothing commodity pictures and the commodity information into a clothing database, and performing image feature extraction on the clothing pictures in the clothing database to form an image feature pool, wherein the clothing database is stored in a server;
step 2, searching similar commodities
Responding to a clothing picture shot or selected by a user through a mobile terminal, cutting the clothing picture, uploading the clothing picture to the server, extracting the characteristics of the clothing picture uploaded by the user on the server, and then performing matching query with the image characteristics stored in the image characteristic pool;
the clothing picture cutting method comprises at least one of a rectangular frame method and a smearing method, wherein the rectangular frame method is to select the clothing part of interest of a user from a frame and upload the selected part of the frame to a server for clothing search, and the smearing method is to smear on the clothing picture by the user and upload the selected part of the interest clothing area to the server for clothing search;
in the clothing searching process based on the smearing and cutting method, the characteristics are extracted, the codes are mixed and the matching query is carried out in the following modes:
1) extracting local descriptors of three areas coated by the clothing picture, wherein the three areas are a collar part, a body part and a hand arm part of clothing respectively;
2) mixing and coding the local descriptors of the regions, wherein each local descriptor is represented by an adjacent visual word, and the respective positions of the local descriptors of three parts of the daubed clothing picture are as follows: x is the number of1,...xa,xa+1,...,xa+b,xa+b+1,...,xa+b+cIt is converted into bone coordinates: x'1,…,x'a,x'a+1,…,x'a+b,x'a+b+1,…,x'a+b+cWherein a, b and c are respectively partial descriptions of three parts of the painting of the clothing pictureThe number of the son, the characteristic is extracted and carried on by using the skeleton coordinate, the said vision word uses the K-means algorithm to carry on the clustering to the local descriptor in the characteristic pool of the picture to get;
3) using the mixed descriptors to search clothing, calculating the similarity between the local descriptors of the searched clothing picture and the local descriptors of the image feature pool, and performing matching query on the feature vector of the clothing and the feature vector of each image in the image feature pool;
step 3, displaying search results
And displaying the inquired similar clothing pictures and corresponding clothing information to a user through a mobile terminal according to the inquired clothing results, wherein the clothing information comprises the brand, price and material of the clothing.
2. The clothing searching method based on mobile terminal as claimed in claim 1, wherein in step 2 of the aforementioned method, the similarity is calculated by a space-constrained similarity measurement method, the similarity measurement method uses a grid voting space to search for a position, the grid voting space is a voting map which is based on the position relationship of local descriptors, has the same visual words and can simultaneously calculate the similarity, and the concrete steps include:
1) voting and superposing feature point combinations with the same visual words of the two clothes pictures subjected to similarity calculation, wherein the voting points are position points with the same motion vector values of the two feature points;
2) the region with the highest voting point represents a projection, and the combination of the voting points represents the similarity of the two clothes pictures as a whole;
and carrying out similarity calculation on the uploaded clothes pictures and each clothes picture in the clothes database according to the mode, and returning the clothes pictures with the similarity meeting the set threshold value to the user in the step 3.
CN201510624226.3A 2015-09-25 2015-09-25 Clothing searching method based on mobile terminal Active CN106557489B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510624226.3A CN106557489B (en) 2015-09-25 2015-09-25 Clothing searching method based on mobile terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510624226.3A CN106557489B (en) 2015-09-25 2015-09-25 Clothing searching method based on mobile terminal

Publications (2)

Publication Number Publication Date
CN106557489A CN106557489A (en) 2017-04-05
CN106557489B true CN106557489B (en) 2020-11-06

Family

ID=58415436

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510624226.3A Active CN106557489B (en) 2015-09-25 2015-09-25 Clothing searching method based on mobile terminal

Country Status (1)

Country Link
CN (1) CN106557489B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105760999A (en) * 2016-02-17 2016-07-13 中山大学 Method and system for clothes recommendation and management
CN108021603A (en) * 2017-09-25 2018-05-11 嘉善锐琦花边有限公司 A kind of fabrics sample searches apparatus and method based on texture analysis and cloud computing
CN110427511A (en) * 2019-07-09 2019-11-08 广州搜料信息技术有限公司 It is a kind of that material method is looked for based on taking pictures for artificial intelligence picture recognition technology
CN112905889B (en) * 2021-03-03 2024-10-15 百度在线网络技术（北京）有限公司 Clothing searching method and device, electronic equipment and medium
CN113592512A (en) * 2021-07-22 2021-11-02 上海普洛斯普新数字科技有限公司 Online commodity identity uniqueness identification and confirmation system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103412938A (en) * 2013-08-22 2013-11-27 成都数之联科技有限公司 Commodity price comparing method based on picture interactive type multiple-target extraction
CN103412937A (en) * 2013-08-22 2013-11-27 成都数之联科技有限公司 Searching and shopping method based on handheld terminal

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9684719B2 (en) * 2012-12-10 2017-06-20 Verint Systems Ltd. Object search by description

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103412938A (en) * 2013-08-22 2013-11-27 成都数之联科技有限公司 Commodity price comparing method based on picture interactive type multiple-target extraction
CN103412937A (en) * 2013-08-22 2013-11-27 成都数之联科技有限公司 Searching and shopping method based on handheld terminal

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于内容的商品图像检索;方强;《中国优秀硕士学位论文全文数据库 信息科技辑》;20130615(第6期);第I138-1233页 *

Also Published As

Publication number Publication date
CN106557489A (en) 2017-04-05

Similar Documents

Publication Publication Date Title
US11599929B2 (en) Fashion preference analysis
CN107993191B (en) Image processing method and device
CN103412938B (en) A kind of commodity price-comparing method extracted based on picture interactive multiobjective
CN106557489B (en) Clothing searching method based on mobile terminal
US8718369B1 (en) Techniques for shape-based search of content
KR101955649B1 (en) Image feature data extraction and use
US9679532B2 (en) Automatic image-based recommendations using a color palette
US9135719B1 (en) Color name generation from images and color palettes
US9514543B2 (en) Color name generation from images and color palettes
US9633448B1 (en) Hue-based color naming for an image
CN106598998B (en) Information acquisition method and information acquisition device
CN102567543B (en) Clothing picture search method and clothing picture search device
CN101937549B (en) Picture guidance system for network shopping guidance
CN106933867B (en) Image query method and device
CN101950400B (en) Picture retrieving method of network shopping guiding method
US9324102B2 (en) System and method to retrieve relevant inventory using sketch-based query
CN103377287A (en) Method and device for putting in item information
CN106095884A (en) A kind of relative article information processing method based on picture and device
CN108492160A (en) Information recommendation method and device
CN103257962B (en) information providing method and device
CN105868299A (en) Data search method and device
CN104680393A (en) Interactive advertisement method based on image contents and matching
CN103279519B (en) Articles search method and apparatus
CN106250541A (en) The method for pushing of a kind of information and device
Lodkaew et al. Fashion finder: A system for locating online stores on instagram from product images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant