CN111782850A - Object searching method and device based on hand drawing - Google Patents
Object searching method and device based on hand drawing Download PDFInfo
- Publication number
- CN111782850A CN111782850A CN201911182839.0A CN201911182839A CN111782850A CN 111782850 A CN111782850 A CN 111782850A CN 201911182839 A CN201911182839 A CN 201911182839A CN 111782850 A CN111782850 A CN 111782850A
- Authority
- CN
- China
- Prior art keywords
- article
- image
- hand
- list
- item
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012163 sequencing technique Methods 0.000 claims description 8
- 230000006399 behavior Effects 0.000 claims description 7
- 239000003086 colorant Substances 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000012549 training Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000008707 rearrangement Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000010621 bar drawing Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0623—Item investigation
- G06Q30/0625—Directed, with specific intent or strategy
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0631—Item recommendations
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- General Engineering & Computer Science (AREA)
- Library & Information Science (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an article searching method and device based on a hand drawing, and relates to the technical field of computers. One embodiment of the method comprises: receiving the hand drawing, extracting a hand drawing image in the hand drawing, and identifying characteristic information in the hand drawing image; comparing the hand-drawn image with each article image in an article image library to extract the article images with the similarity exceeding a preset similarity threshold, and acquiring and arranging article information corresponding to each extracted article image to generate a first article list; converting the characteristic information into a characteristic character tag, searching for the article based on the characteristic character tag, and combining the searched article information to obtain a second article list; and merging the article information in the first article list and the second article list to obtain and display a third article list. According to the embodiment, when the user cannot provide the accurate keywords and cannot obtain the corresponding article picture, a more flexible searching mode can be provided for the user, and the searching experience of the user is improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to an article searching method and device based on a hand drawing.
Background
An item search system is an important component of electronic commerce, which makes recommendations for related items based on keywords submitted by users. Currently, there are text-based item search systems and picture-based item search systems that can be classified according to the nature of keywords.
The character-based article search system requires a user to input character keywords of purchased articles, and then the system matches the character keywords with the description information of the articles in the bottom library, and further recommends the articles according to the matching degree.
As a supplementary manner to the above manner, when the user cannot clearly describe the item information, the item search system based on the picture may recommend the item according to the picture similarity by uploading the picture of the purchased item and matching the picture with the picture of the item in the base.
In the process of implementing the invention, the inventor finds that the prior art has at least the following problems:
1. the text-based item search system requires that a user can clearly describe the name, purpose, related attributes and other characteristics of an item, otherwise, an item which is greatly different from the user's expectation is recommended. Especially under the scene that the articles need to be accurately matched, the requirements on character keywords are strict;
2. the article searching system based on the images mainly depends on the visual similarity of the article images to complete article searching, and when the user cannot provide the images, the searching cannot be conducted, so that the use scene is limited.
Disclosure of Invention
In view of this, embodiments of the present invention provide an article search method and apparatus based on a hand drawing, which can at least solve the problem that the accuracy requirement of the existing article search system for the text keywords and the image keywords is relatively strict.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided an item searching method based on a hand drawing, including:
receiving a hand drawing, extracting a hand drawing image in the hand drawing, and identifying characteristic information in the hand drawing image;
comparing the hand-drawn image with each article image in an article image library to extract the article images with similarity exceeding a preset similarity threshold, and acquiring and arranging article information corresponding to each extracted article image to generate a first article list;
converting the characteristic information into a characteristic character label, searching for an article based on the characteristic character label, and combining the searched article information to obtain a second article list;
and merging the article information in the first article list and the second article list to obtain a third article list and displaying the third article list.
Optionally, the item image further comprises an item outline image;
the method further comprises the following steps: comparing the hand-drawn image with each item contour image in the item image library to extract an item contour image with a first similarity exceeding a first preset similarity threshold; and acquiring and arranging article information corresponding to each extracted article contour image to generate a fourth article list.
Optionally, the feature information includes a category and a color attribute;
the identifying the feature information in the hand-drawn image comprises: identifying the category of the hand-drawn image by using a hand-drawn image identification mode; and
and comparing the hand-drawn image with colors in a color library to identify the color attributes in the hand-drawn image.
Optionally, the obtaining and displaying a third item list further includes: and determining the preference degrees of all the users for the articles according to the selection behaviors of the users for the articles in the third article list, sequencing the articles according to the obtained preference degrees, and obtaining and displaying a sequenced fifth article list.
To achieve the above object, according to another aspect of embodiments of the present invention, there is provided an item search apparatus based on a hand drawing, including:
the information identification module is used for receiving the hand drawing, extracting a hand drawing image in the hand drawing and identifying characteristic information in the hand drawing image;
the similarity comparison module is used for comparing the hand-drawn image with each article image in an article image library so as to extract the article image with the similarity exceeding a preset similarity threshold value, and acquiring and arranging article information corresponding to each extracted article image so as to generate a first article list;
the label conversion module is used for converting the characteristic information into a characteristic character label, searching for an article based on the characteristic character label, and combining the searched article information to obtain a second article list;
and the list merging module is used for merging the item information in the first item list and the second item list to obtain and display a third item list.
Optionally, the item image further comprises an item outline image;
the similarity comparison module is further configured to: comparing the hand-drawn image with each item contour image in the item image library to extract an item contour image with a first similarity exceeding a first preset similarity threshold; and acquiring and arranging article information corresponding to each extracted article contour image to generate a fourth article list.
Optionally, the feature information includes a category and a color attribute;
the information identification module is used for: identifying the category of the hand-drawn image by using a hand-drawn image identification mode; and
and comparing the hand-drawn image with colors in a color library to identify the color attributes in the hand-drawn image.
Optionally, the list merging module is further configured to: and determining the preference degrees of all the users for the articles according to the selection behaviors of the users for the articles in the third article list, sequencing the articles according to the obtained preference degrees, and obtaining and displaying a sequenced fifth article list.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided an item search electronic device based on a hand drawing.
The electronic device of the embodiment of the invention comprises: one or more processors; a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement any of the above-described hand-drawing based item search methods.
To achieve the above object, according to a further aspect of the embodiments of the present invention, there is provided a computer-readable medium having a computer program stored thereon, the program, when executed by a processor, implementing any one of the above-mentioned methods for searching for an item based on a hand drawing.
According to the scheme provided by the invention, one embodiment of the invention has the following advantages or beneficial effects: by using a hand drawing search technology as an article search supplement scheme beyond two modes of character search and image search, when a user cannot provide accurate keywords and cannot acquire corresponding article images, a more flexible search mode is provided for the user.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic flow chart of a method for searching for an item based on a hand drawing according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of an embodiment of the present invention;
FIG. 3 is a flow diagram of a method for item searching, particularly based on a hand drawing, according to an embodiment of the invention;
FIG. 4 is a schematic diagram of the main modules of an article search device based on a hand drawing according to an embodiment of the present invention;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
FIG. 6 is a schematic block diagram of a computer system suitable for use with a mobile device or server implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments of the present invention are not only applicable to commodity recommendation, but also applicable to recommendation of other articles, such as tools, parts, and the like.
Referring to fig. 1, a main flowchart of an item searching method based on a hand drawing according to an embodiment of the present invention is shown, including the following steps:
s101: receiving a hand drawing, extracting a hand drawing image in the hand drawing, and identifying characteristic information in the hand drawing image;
s102: comparing the hand-drawn image with each article image in an article image library to extract the article images with similarity exceeding a preset similarity threshold, and acquiring and arranging article information corresponding to each extracted article image to generate a first article list;
s103: converting the characteristic information into a characteristic character label, searching for an article based on the characteristic character label, and combining the searched article information to obtain a second article list;
s104: and merging the article information in the first article list and the second article list to obtain a third article list and displaying the third article list.
In the above embodiment, for step S101, the present invention allows the user to draw the item or the element included in the item to be searched as a picture by hand drawing, so as to avoid the risk that the user cannot describe the item well and cannot obtain the picture of the item.
A user may draw on paper, then take a picture/scan of the drawn film, and then upload to a hand-drawing search system for item searching. However, in view of the fact that the paper drawing has a single color and the photographing environment may affect the subsequent item search result, the invention provides a hand-drawing interface for the user to directly draw in the interface. And the user does not need to have a strong drawing power, and only needs to draw the outline/key elements of the article.
After the user has finished drawing and clicks the search button, the hand-drawn search system will first identify the hand-drawn image in the hand-drawn. Since the hand-drawn image is usually drawn by using line-and-bar drawing, the hand-drawn image in the invention can also be directly regarded as a line profile.
The image drawn by the user contains some feature information, which may not be visually obtained, and needs to be extracted by an image recognition means, such as recognizing the category to which the user belongs, recognizing the color used by the user, and the like.
For steps S102 and S103, the present invention mainly involves two search processes: hand-drawn image-item image search and hand-drawn image-item text label search:
1) hand-drawn image-item image search process
The method comprises the steps of calculating the similarity (actually, the similarity between image features) between a hand-drawn image provided by a user and each item image in an item image library by using a hand-drawn image-item image similarity model, and sequencing item information corresponding to each item image according to the similarity to obtain a first item list.
Further, to reduce the number of items in the list and reduce the amount of calculation, only items with similarity exceeding a certain similarity threshold, for example, items with similarity greater than 0.5, may be extracted.
The hand-drawn image-item image search is different from the conventional item search method based on pictures, and the subsequent search methods are different due to the difference of input information between the hand-drawn image and the item image.
2) Hand-drawn image-item text label search
The method comprises the step of carrying out category identification on the hand-drawn image by utilizing a hand-drawn image identification technology, wherein the identified category corresponds to a character label, and the category can also be understood as being converted into a corresponding character label.
Besides the category character label, other attribute features in the hand-drawn image, such as color, structure and the like, can be identified, and the invention mainly aims at the color attribute:
determining color attributes in a hand-drawn image according to the color of a painting brush adopted by a user for drawing;
and secondly, identifying the color attribute in the hand-drawn image of the user by using a color identification mode.
In addition, for the image drawn through the hand-drawing interface, the drawing time sequence of the user during drawing can be monitored, and the drawing sequence of each line in the drawing can be determined. Further, sequential text labels may be generated based on the resulting drawing order.
Through the operation, all the characteristic information in the hand-drawn image is converted into the character label. And then calling the existing character-based article identification system, and searching articles together based on all the obtained character labels to obtain a second article list.
In addition, for the text labels in the hand drawing, a cyclic addition process can be set (the number of additions is limited in practical operation). Specifically, the hand drawing is firstly converted into a category character label according to a hand drawing identification mode, and then the user selects whether to keep, modify and add the label again.
Taking whether to add or not as an example, if the number is added, a hand-drawing circulation flow is added (the previous flow is repeated), and if the number is not added, a plurality of attributes, such as color and category, can be added according to the selection of the user.
It should be noted that the training and testing process for each model in the present invention is the existing mode, i.e. training set-training, testing set-testing, and the training is stopped when the test index reaches the expected value, and the specific training process is not described in detail in the present invention.
The image may include a contour image thereof in addition to the entire image itself. In order to improve the accuracy of subsequent search, the present invention further includes a hand-drawn image-item outline image search process (specifically, see fig. 2) in addition to the above two search processes:
and calculating a first similarity between the hand-drawn image provided by the user and the item outline images in the library by using a hand-drawn image-item outline image similarity model, extracting the items with the first similarity exceeding a first preset similarity threshold, and sequencing according to the similarity to obtain a corresponding fourth item list.
In the present invention, the hand-drawn image is generally composed of lines, and thus can be directly defaulted as a contour image without extracting a contour. And for the outline images of the articles in the library, the outline extraction can be automatically carried out by utilizing an edge extraction algorithm.
In step S104, the three search processes respectively calculate "similar" items according to different dimension information (or feature space), so that the items in the first item list, the second item list, and the fourth item list that do not belong to the same list cannot be sorted front and back.
To obtain the item list finally recommended to the user, the three item lists may be merged and fused (i.e., the third item list) to improve the item recall rate through complementary effects between the three search processes.
Furthermore, the same article may exist in the three article lists or two article lists at the same time, and the condition that the same article appears at different positions in the same list can be avoided through the combination of the article information; meanwhile, the quantity of the subsequently calculated articles is effectively reduced, and the resource waste condition caused by repeated calculation of the same article is avoided.
According to the method provided by the embodiment, by utilizing the hand-drawing search technology, the user requirements are better met when the user cannot provide accurate keywords and cannot obtain corresponding article pictures; and the existing search system has no search mode related to hand drawing, the hand drawing search mode can make up for the defects of character search and picture search to a great extent, the three are independent and complementary, a more flexible search mode is provided for users, and the search experience of the users is improved.
Referring to fig. 3, there is shown a main flowchart of an item searching method based on a hand drawing according to an embodiment of the present invention, including the following steps:
s301: receiving the hand drawing, extracting a hand drawing image in the hand drawing, and identifying characteristic information in the hand drawing image;
s302: comparing the hand-drawn image with each article image in an article image library to extract the article images with the similarity exceeding a preset similarity threshold, and acquiring and arranging article information corresponding to each extracted article image to generate a first article list;
s303: comparing the hand-drawn image with each item contour image in an item image library to extract an item contour image with a first similarity exceeding a first preset similarity threshold, and acquiring and arranging item information corresponding to each extracted item contour image to generate a fourth item list;
s304: converting the characteristic information into a characteristic character label, searching for an article based on the characteristic character label, and combining the searched article information to obtain a second article list;
s305: merging the article information in the first article list, the second article list and the fourth article list to obtain a third article list;
s306: and determining the preference degrees of all the users for the articles according to the selection behaviors of the users for the articles in the third article list, sequencing the articles according to the obtained preference degrees, and obtaining and displaying a sequenced fifth article list.
In the above embodiment, for steps S301 to S305, reference may be made to the description of steps S101 to S104 shown in fig. 1, and details are not repeated here.
In the above embodiment, for step S306, in order to implement accuracy of recommending articles, after the fourth article list is generated from the merged list, the present invention further performs a rearrangement operation on the article information therein.
Performing CTR (Click-Through-Rate) estimation by using methods such as GBDT (Gradient Boosting Tree) or LR (logistic regression), determining preference (e.g., the possibility of being purchased) of each item in the list by the user according to user selection behavior information such as popularity, comment, user purchasing habit, and the like of the item, and then reordering the items according to the preference.
In the method provided by the above embodiment, the first item list, the second item list and the fourth item list are obtained according to certain specific dimension information (visual similarity or text semantic similarity), and the rearrangement of the items in the third item list is obtained by supplementing another dimension information, so that the item recommendation accuracy is improved, and the user experience is improved.
Referring to fig. 4, a schematic diagram of main modules of an article search apparatus 400 based on a hand drawing according to an embodiment of the present invention is shown, including:
the information identification module 401 is configured to receive a hand drawing, extract a hand drawing image in the hand drawing, and identify feature information in the hand drawing image;
a similarity comparison module 402, configured to compare the hand-drawn image with each item image in an item image library, to extract an item image with a similarity exceeding a predetermined similarity threshold, and obtain and arrange item information corresponding to each extracted item image to generate a first item list;
a tag conversion module 403, configured to convert the feature information into a feature text tag, perform item search based on the feature text tag, and combine the searched item information to obtain a second item list;
a list merging module 404, configured to merge the item information in the first item list and the second item list to obtain a third item list and display the third item list.
In the device for implementing the invention, the article image further comprises an article outline image;
the similarity alignment module 402 is further configured to:
comparing the hand-drawn image with each item contour image in the item image library to extract an item contour image with a first similarity exceeding a first preset similarity threshold;
and acquiring and arranging article information corresponding to each extracted article contour image to generate a fourth article list.
In the implementation device of the invention, the characteristic information comprises a category and a color attribute;
the information identifying module 401 is configured to: identifying the category of the hand-drawn image by using a hand-drawn image identification mode; and comparing the hand-drawn image with colors in a color library to identify color attributes in the hand-drawn image.
In the device for implementing the present invention, the list merging module 404 is further configured to: and determining the preference degrees of all the users for the articles according to the selection behaviors of the users for the articles in the third article list, sequencing the articles according to the obtained preference degrees, and obtaining and displaying a sequenced fifth article list.
In addition, the detailed implementation of the device in the embodiment of the present invention has been described in detail in the above method, so that the repeated description is not repeated here.
Fig. 5 illustrates an exemplary system architecture 500 to which embodiments of the invention may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505 (by way of example only). The network 504 serves to provide a medium for communication links between the terminal devices 501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 501, 502, 503 to interact with a server 505 over a network 504 to receive or send messages or the like. Various communication client applications may be installed on the terminal devices 501, 502, 503.
The terminal devices 501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 501, 502, 503.
It should be noted that the method provided by the embodiment of the present invention is generally executed by the server 505, and accordingly, the apparatus is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, a block diagram of a computer system 600 suitable for use with a terminal device implementing an embodiment of the invention is shown. The terminal device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes an information identification module, a similarity comparison module, a tag conversion module, and a list merging module. Where the names of these modules do not in some way constitute a limitation on the modules themselves, for example, a list merge module may also be described as a "module that merges items in a list".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise:
receiving a hand drawing, extracting a hand drawing image in the hand drawing, and identifying characteristic information in the hand drawing image;
comparing the hand-drawn image with each article image in an article image library to extract the article images with similarity exceeding a preset similarity threshold, and acquiring and arranging article information corresponding to each extracted article image to generate a first article list;
converting the characteristic information into a characteristic character label, searching for an article based on the characteristic character label, and combining the searched article information to obtain a second article list;
and merging the article information in the first article list and the second article list to obtain a third article list and displaying the third article list.
According to the technical scheme of the embodiment of the invention, by utilizing the hand-drawing search technology, when a user cannot provide accurate keywords and cannot obtain corresponding article pictures, the user requirements are better met; and the existing search system has no search mode related to hand drawing, the hand drawing search mode can make up for the defects of character search and picture search to a great extent, a more flexible search mode is provided for users, and the search experience of the users is improved.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An item searching method based on a hand drawing is characterized by comprising the following steps:
receiving a hand drawing, extracting a hand drawing image in the hand drawing, and identifying characteristic information in the hand drawing image;
comparing the hand-drawn image with each article image in an article image library to extract the article images with similarity exceeding a preset similarity threshold, and acquiring and arranging article information corresponding to each extracted article image to generate a first article list;
converting the characteristic information into a characteristic character label, searching for an article based on the characteristic character label, and combining the searched article information to obtain a second article list;
and merging the article information in the first article list and the second article list to obtain a third article list and displaying the third article list.
2. The method of claim 1, wherein the item image further comprises an item outline image;
the method further comprises the following steps:
comparing the hand-drawn image with each item contour image in the item image library to extract an item contour image with a first similarity exceeding a first preset similarity threshold;
and acquiring and arranging article information corresponding to each extracted article contour image to generate a fourth article list.
3. The method of claim 1, wherein the characteristic information includes a category and a color attribute;
the identifying the feature information in the hand-drawn image comprises:
identifying the category of the hand-drawn image by using a hand-drawn image identification mode; and
and comparing the hand-drawn image with colors in a color library to identify the color attributes in the hand-drawn image.
4. The method of any of claims 1-3, wherein the obtaining and displaying a third item list further comprises:
and determining the preference degrees of all the users for the articles according to the selection behaviors of the users for the articles in the third article list, sequencing the articles according to the obtained preference degrees, and obtaining and displaying a sequenced fifth article list.
5. An article search device based on a hand drawing, comprising:
the information identification module is used for receiving the hand drawing, extracting a hand drawing image in the hand drawing and identifying characteristic information in the hand drawing image;
the similarity comparison module is used for comparing the hand-drawn image with each article image in an article image library so as to extract the article image with the similarity exceeding a preset similarity threshold value, and acquiring and arranging article information corresponding to each extracted article image so as to generate a first article list;
the label conversion module is used for converting the characteristic information into a characteristic character label, searching for an article based on the characteristic character label, and combining the searched article information to obtain a second article list;
and the list merging module is used for merging the item information in the first item list and the second item list to obtain and display a third item list.
6. The apparatus of claim 5, wherein the item image further comprises an item outline image;
the similarity comparison module is further configured to:
comparing the hand-drawn image with each item contour image in the item image library to extract an item contour image with a first similarity exceeding a first preset similarity threshold;
and acquiring and arranging article information corresponding to each extracted article contour image to generate a fourth article list.
7. The apparatus of claim 5, wherein the characteristic information comprises a category and a color attribute;
the information identification module is used for:
identifying the category of the hand-drawn image by using a hand-drawn image identification mode; and
and comparing the hand-drawn image with colors in a color library to identify the color attributes in the hand-drawn image.
8. The apparatus of any of claims 5-7, wherein the list merge module is further configured to:
and determining the preference degrees of all the users for the articles according to the selection behaviors of the users for the articles in the third article list, sequencing the articles according to the obtained preference degrees, and obtaining and displaying a sequenced fifth article list.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-4.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911182839.0A CN111782850B (en) | 2019-11-27 | 2019-11-27 | Object searching method and device based on hand drawing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911182839.0A CN111782850B (en) | 2019-11-27 | 2019-11-27 | Object searching method and device based on hand drawing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111782850A true CN111782850A (en) | 2020-10-16 |
| CN111782850B CN111782850B (en) | 2024-06-18 |
Family
ID=72755778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911182839.0A Active CN111782850B (en) | 2019-11-27 | 2019-11-27 | Object searching method and device based on hand drawing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111782850B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112712264A (en) * | 2020-12-31 | 2021-04-27 | 无锡软美信息科技有限公司 | Smart community information sharing method and system |
| CN112969021A (en) * | 2020-10-30 | 2021-06-15 | 泰州镭昇光电科技有限公司 | Relay control platform and method using content search |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101571875A (en) * | 2009-05-05 | 2009-11-04 | 程治永 | Realization method of image searching system based on image recognition |
| US20110075950A1 (en) * | 2008-06-04 | 2011-03-31 | National Univerity Corporation Shizuoka University | Image retrieval device and computer program for image retrieval applicable to the image retrieval device |
| CN103412937A (en) * | 2013-08-22 | 2013-11-27 | 成都数之联科技有限公司 | Searching and shopping method based on handheld terminal |
| US20170262479A1 (en) * | 2016-03-08 | 2017-09-14 | Shutterstock, Inc. | User drawing based image search |
| CN107526797A (en) * | 2017-08-17 | 2017-12-29 | 广州市西美信息科技有限公司 | Substance search method and device |
| CN108804476A (en) * | 2017-05-05 | 2018-11-13 | 北京京东尚科信息技术有限公司 | Sort method, device, electronic equipment and the storage medium of image search result |
| CN109213917A (en) * | 2018-09-19 | 2019-01-15 | 百度在线网络技术（北京）有限公司 | Searching method, device, computer equipment and the storage medium of 3D content |
| CN109308324A (en) * | 2018-09-08 | 2019-02-05 | 中山大学 | A kind of image search method and system based on hand drawing style recommendation |
| CN110020185A (en) * | 2017-12-29 | 2019-07-16 | 国民技术股份有限公司 | Intelligent search method, terminal and server |
-
2019
- 2019-11-27 CN CN201911182839.0A patent/CN111782850B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110075950A1 (en) * | 2008-06-04 | 2011-03-31 | National Univerity Corporation Shizuoka University | Image retrieval device and computer program for image retrieval applicable to the image retrieval device |
| CN101571875A (en) * | 2009-05-05 | 2009-11-04 | 程治永 | Realization method of image searching system based on image recognition |
| CN103412937A (en) * | 2013-08-22 | 2013-11-27 | 成都数之联科技有限公司 | Searching and shopping method based on handheld terminal |
| US20170262479A1 (en) * | 2016-03-08 | 2017-09-14 | Shutterstock, Inc. | User drawing based image search |
| CN108804476A (en) * | 2017-05-05 | 2018-11-13 | 北京京东尚科信息技术有限公司 | Sort method, device, electronic equipment and the storage medium of image search result |
| CN107526797A (en) * | 2017-08-17 | 2017-12-29 | 广州市西美信息科技有限公司 | Substance search method and device |
| CN110020185A (en) * | 2017-12-29 | 2019-07-16 | 国民技术股份有限公司 | Intelligent search method, terminal and server |
| CN109308324A (en) * | 2018-09-08 | 2019-02-05 | 中山大学 | A kind of image search method and system based on hand drawing style recommendation |
| CN109213917A (en) * | 2018-09-19 | 2019-01-15 | 百度在线网络技术（北京）有限公司 | Searching method, device, computer equipment and the storage medium of 3D content |
Non-Patent Citations (2)
| Title |
|---|
| 李宗民;刘秀秀;刘玉杰;李华;: "结合细粒度特征与深度卷积网络的手绘图检索", 中国图象图形学报, no. 06, 30 June 2019 (2019-06-30) * |
| 高竹红;汤进;罗斌;: "基于结构图的手绘草图检索", 计算机技术与发展, no. 03, 31 March 2008 (2008-03-31) * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112969021A (en) * | 2020-10-30 | 2021-06-15 | 泰州镭昇光电科技有限公司 | Relay control platform and method using content search |
| CN112969021B (en) * | 2020-10-30 | 2022-03-15 | 上海称意包装设备有限公司 | Relay control platform and method using content search |
| CN112712264A (en) * | 2020-12-31 | 2021-04-27 | 无锡软美信息科技有限公司 | Smart community information sharing method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111782850B (en) | 2024-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106919711B (en) | Method and device for labeling information based on artificial intelligence | |
| CN110020162B (en) | User identification method and device | |
| US11741094B2 (en) | Method and system for identifying core product terms | |
| CN113592535B (en) | Advertisement recommendation method and device, electronic equipment and storage medium | |
| CN114154013A (en) | Video recommendation method, device, equipment and storage medium | |
| CN112330382B (en) | Item recommendation method, device, computing equipment and medium | |
| CN111767420A (en) | Method and device for generating clothing matching data | |
| CN111209351A (en) | Object relation prediction method and device, object recommendation method and device, electronic equipment and medium | |
| CN111782850B (en) | Object searching method and device based on hand drawing | |
| CN110059172B (en) | Method and device for recommending answers based on natural language understanding | |
| CN112749323A (en) | Method and device for constructing user portrait | |
| CN108509442B (en) | Search method and apparatus, server, and computer-readable storage medium | |
| CN112433713A (en) | Application program design graph processing method and device | |
| CN112446214A (en) | Method, device and equipment for generating advertisement keywords and storage medium | |
| US20230085684A1 (en) | Method of recommending data, electronic device, and medium | |
| CN111125502A (en) | Method and apparatus for generating information | |
| CN110807095A (en) | Article matching method and device | |
| CN107562845A (en) | Method for pushing, system and electronic equipment | |
| CN111783808B (en) | Method and device for generating information | |
| CN112861684A (en) | Article display method and device | |
| CN113744011A (en) | Article collocation method and article collocation device | |
| CN111488537A (en) | Page display method and device | |
| CN111339124A (en) | Data display method and device, electronic equipment and computer readable medium | |
| CN112860858A (en) | Method and device for answering questions | |
| CN107180037B (en) | Man-machine interaction method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |