WO2012102926A1 - Method and apparatus of identifying similar images - Google Patents

Method and apparatus of identifying similar images Download PDF

Info

Publication number
WO2012102926A1
WO2012102926A1 PCT/US2012/021722 US2012021722W WO2012102926A1 WO 2012102926 A1 WO2012102926 A1 WO 2012102926A1 US 2012021722 W US2012021722 W US 2012021722W WO 2012102926 A1 WO2012102926 A1 WO 2012102926A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
hash
signature
generate
recited
Prior art date
Application number
PCT/US2012/021722
Other languages
French (fr)
Inventor
Hui XUE
Original Assignee
Alibaba Group Holding Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Limited filed Critical Alibaba Group Holding Limited
Priority to US13/509,059 priority Critical patent/US9053386B2/en
Priority to EP12739016.9A priority patent/EP2668619A4/en
Priority to JP2013552013A priority patent/JP5926291B2/en
Publication of WO2012102926A1 publication Critical patent/WO2012102926A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5854Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using shape and object relationship
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/50Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/758Involving statistics of pixels or of feature values, e.g. histogram matching
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761Proximity, similarity or dissimilarity measures

Definitions

  • This disclosure relates to the field of multimedia image recognition. More specifically, the disclosure relates to methods and devices of image recognition.
  • Similar image retrieval as a type of multimedia recognition technologies, is booming.
  • the similar image retrieval may comprise feature extraction, index construction, inquiries, and similarities sorting.
  • conventional technologies of similar image retrieval may present some problems (e.g., poor compatibility and fault tolerance problems) because conventional image signatures do not contain content information of images.
  • two images that have identical content but saved in different formats e.g., bmp, jpeg, png, or gif
  • two images that have identical content but saved in different formats e.g., bmp, jpeg, png, or gif
  • a user may submit an image and request from a server a set of images that are similar to the submitted image.
  • the server may generate an image signature based on the content of that image.
  • the server may also conduct a Hash operation to the image signature to generate one or more Hash values.
  • the one or more values may be used to identify candidate images similar to the image.
  • the server may then determine an image similarity between each of these candidate images and the image.
  • the server may convert the image into a gray image and then divide the gray image into multiple sub-images. The server may then calculate edge histograms of each of the multiple sub-images. Based on the edge histograms, the server may generate the image signature. In some embodiments, to generate the one or more Hash values, the server may conduct a Hash operation to the image signature using Locality-sensitive hashing functions to generate one or more first Hash values. The server may then conduct a Hash operation to each of the one or more first Hash values to generate one or more second Hash values. The one or more second Hash values may be used to identify the candidate images.
  • the server may determine a similarity between the image and each of the one or more candidate images using at least one of Hamming distance or Euclidean distance.
  • FIG. 1 is a schematic diagram showing an exemplary environment for identifying similar images.
  • FIG. 2 is a schematic block diagram showing an exemplary of the image server(s) of FIG. 1.
  • FIG. 3 is a schematic block diagram showing another exemplary of image server(s) of FIG. 1.
  • FIG. 4 is a flowchart showing an exemplary process of identifying similar images.
  • FIG. 5 is a flowchart showing an exemplary process of obtaining an image signature.
  • FIG. 6 is a flowchart showing an exemplary process of creating an image index.
  • FIG. 7 is a flowchart showing another exemplary process of identifying similar images.
  • FIG. 1 is a schematic diagram showing an exemplary environment 100 for identifying similar images.
  • the environment 100 may include image servers 102 interacting with a user 104 and a user 106.
  • the image servers 102 may include upload servers 108, an image database device 110, index servers 112, image calculation servers 114, and image search front servers 116.
  • the user 104 may uploads an image to the upload servers 108 to create image index database.
  • the upload servers 108 may calculate an image signature of the image and send the image signature to the image database device 110.
  • the image may be obtained from the Internet and saved in the upload servers 108 using a crawler system.
  • the upload servers 108 may calculate the image signature of the image based on content information of the image, thus reducing probability of a conflict.
  • the content information of the image may include metadata (e.g., color, grain, etc.) instead of a non-byte stream.
  • the image database device 110 may conduct a Hash operation to the image by using Locality-sensitive hashing (LSH) after receiving the image signature.
  • LSH Locality-sensitive hashing
  • the resulting values are then sent as index information to the index servers 112.
  • the user 106 may send an image to the search front servers 116 and request the image servers to return a set of images similar to the submitted image.
  • the image search front servers 116 may transmit the image to the image calculation servers 114 and request the similar images or a list of the similar images.
  • an image signature may be generated based on the content information of the image, and then the index servers 112 search for similar images based on the image signature.
  • the image calculation servers 114 may search the similar images (e.g., IDs) based on Hash values of these similar images on the index servers 112. For example, the image calculation servers 114 may identify IDs associated with images having the same Hash value as the image.
  • the similar image or a list of similar images may be returned to the image search front server 116 and displayed to the user 106.
  • FIG. 2 is a schematic block diagram showing details of exemplary image servers 102 of FIG. 1.
  • the image servers 102 may be configured as any suitable servers.
  • the image servers 102 include processors 202, input/output interfaces 204, network interfaces 206, and memory 208.
  • the memory 208 may include computer-readable media in the form of volatile memory, such as random-access memory (RAM) and/or non-volatile memory, such as read only memory (ROM) or flash RAM.
  • RAM random-access memory
  • ROM read only memory
  • flash RAM flash random-access memory
  • Computer-readable media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk readonly memory (CD-ROM), digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device.
  • computer-readable media does not include transitory media such as modulated data signals and carrier waves.
  • the memory 208 may store any number of software modules or units, including an output unit 210, a recognition unit 212, a search unit 214, an obtaining unit 216, a Hash operation unit 218, and a Hash table creation unit 220.
  • the obtaining unit 216 may obtain an image signature of an image submitted by the user 106.
  • the Hash operation unit 218 may conduct a Hash operation to the image signature.
  • the searching unit 214 may search for candidate images in predetermined Hash tables.
  • the recognition unit 212 may identify and retrieve image(s) similar to the image from candidate images.
  • the image servers 102 may determine similarities of images based on image signatures generated from content of the images. For example, the more similar the images are, the more similar their image signatures are. In some embodiments, the similarities may be related to Hamming distance or Euclidean distance. For example, the shorter these distances, the more similar are the images. Accordingly, images that are the same but saved in different formats may have the same image signatures. Also, problems of fault tolerance with the conventional technologies may be resolved.
  • the output unit 210 may output the candidate images based on the similarities.
  • the Hash table creation unit 220 may create L Hash tables by conducting Hash operations to the image signature to obtain L Hash values by using L LSH functions, wherein L is an integer.
  • the Hash table creation unit 220 may also conduct a Hash operation to the L first Hash values to obtain L second Hash values by using an overall Hash function.
  • FIG. 3 is a schematic block diagram showing another exemplary of image servers 300 that may include, similar to the image servers 102 of FIGS. 1 and 2, the recognition unit 212, the searching unit 214, the obtaining unit 216, and the Hash operation unit 218.
  • the obtaining unit 216 may include a conversion module 302, a calculation module 304, and a generation module 306.
  • the conversion module 302 may convert an image submitted by the user 106 to a gray image.
  • the calculation module 304 may then divide the gray image into NxN sub-images, calculate edge histograms from M directions for each of the sub-images, and obtain NxNxM calculation results, wherein N and M are integers.
  • the generation module 306 may assemble the NxNxM calculation results as an NxNxM dimensional vector of the image signature.
  • non-uniform quantitative technologies based on human eye sensitivity can be applied to the NxNxM dimensional vector.
  • statistic data of edge histograms may be used as basic features of the image signature.
  • the statistic data combined with the image division and the non-uniform quantitative technologies based on human eye sensitivity may be resolved problems caused by different colors, scales, or partial fuzzy & distortion.
  • the gray images may be divided into NxP sub-images.
  • the calculation module 304 may, for each image block included in a sub-image, calculate a gradient value on each of the M directions. The calculation module 304 may then select one of the M directions that has the greatest gradient value as the direction of the corresponding image block for statistics. The module 304 may further calculate the number of the selected directions for each of the M directions and obtain the statistic value as histograms of the sub-image.
  • direction A for statistic data are 100 image blocks
  • direction B has 200 image blocks
  • direction C has 300 image blocks
  • direction D has 400 image blocks
  • direction E has no image blocks.
  • the statistics value (i.e., histograms) of the sub-image may be represented by a vector (100, 200, 300, 400, 0).
  • the Hash operation unit 218 may include a first Hash operation module 308 and a second Hash operation module 310.
  • the first Hash operation module 308 may conduct Hash operations to the image signature, and obtain L first Hash values by using L LSH functions, wherein L is an integer.
  • the second Hash operation module 310 may conduct Hash operations to the L first Hash values, and obtain L second Hash values by using an overall Hash function.
  • a method of mass similar image index based on the LSH technology may be used to reduce complexity of search time to a non-linear level.
  • the searching unit 214 may include a searching module 312 and an addition module 314.
  • the searching module 312 may search an entry recording the second Hash value in the Hash tables.
  • Each entry of the Hash tables records a Hash value and an image identification of an image.
  • Each entry may further include an image signature.
  • the addition module 314 may add an image to the candidate images upon identifying an entry having the second Hash value in the Hash tables.
  • the first Hash operation module 308 may convert an image signature to a R dimensional binary vector, wherein R is an integer.
  • the first Hash operation module 308 may then generate L LSH functions by using the R dimensional binary vector. For example, each of the LSH functions is generated based on a one-dimensional or multiple-dimensional binary vector of the R dimensional binary vector.
  • the first Hash operation module 308 may assign an input parameter of a LSH function as K, randomly select K dimensional binary vector from the R dimensional binary vector, and combine the K dimensional binary vector as a return value of the LSH function, wherein K ⁇ R.
  • the recognition unit 212 may include a calculation module 316 and a recognition module 318.
  • the calculation module 316 may calculate a spatial distance between an image signature of a submitted image and an image signature of a candidate image (e.g., Hamming distance or Euclidean distance).
  • the recognition module 318 may then determine a similarity between the candidate image and the submitted image based on the spatial distance. For example, the smaller the spatial distance is, the more similar the candidate image is to the submitted image.
  • FIGS. 4-7 are flowcharts showing exemplary processes of identifying similar images.
  • the processes are illustrated as a collection of blocks in logical flowcharts, which represent a sequence of operations that can be implemented in hardware, software, or a combination.
  • the processes are described with reference to the image servers 102 shown in FIGS. 1-3. However, the processes may be performed using different environments and devices. Moreover, the environments and devices described herein may be used to perform different processes.
  • FIG. 4 is a flowchart showing an exemplary process 400 of identifying similar images.
  • the process 400 may be performed by one or more processors of the image servers, which may be inclusive of one or more microcontrollers (MCU) or one or more field-programmable gate arrays (FPGAs).
  • MCU microcontrollers
  • FPGAs field-programmable gate arrays
  • the image servers may receive an image submitted by the user 106 via, for example, a USB transmission interface, a Bluetooth transmission interface, or an Ethernet transmission interface.
  • the obtaining unit 216 may generate or otherwise obtain an image signature of the submitted image.
  • the obtaining unit 216 may involve a MCU or a FPGA to calculate the image signature.
  • the Hash operation unit 218 may conduct a Hash operation to the image signature.
  • the Hash operation unit 218 may include a MCU or a FPGA configured to perform the Hash operations.
  • the functions performed by the Hash operation unit 218 and the obtaining unit 216 may be implemented by the same processor(s).
  • the search unit 214 may search entries corresponding to a result of the Hash operation in predetermined Hash tables to identify candidate images.
  • the recognition unit 212 may recognize an image similar to the submitted image from candidate images corresponding to the entries.
  • the image calculation server 108 obtains, locally or from third-party equipment, the candidate images based on the IDs of similar images returned by the index server 106, and recognizes the image similar to the submitted image from the candidate images based on the entries.
  • the recognition unit 212 may obtain, locally or from third-party equipment, the candidate image based on the ID of the similar images returned by the searching unit 214, and recognizes the image similar to the submitted image from the candidate images based on the entries.
  • the output unit 210 may include a Bluetooth transmission module, an infrared transmission module and/or an Ethernet transmission module to output the identified similar image and to display to the user 106.
  • FIG. 5 is a flowchart showing an exemplary process 500 of obtaining an image signature.
  • the process 500 may generate an image signature based on the grain features of edge histograms.
  • the image servers convert an image submitted by the user 106 to a gray image to make final results not sensitive to color and/or light.
  • the calculation module 304 may divide the gray image into N xN sub-images.
  • the calculation module 304 may further divide each of the sub-images into a fix number of image blocks. The area of each image block depends on the area of the submitted image.
  • the calculation module 304 may then calculate edge histograms from one or more directions for each of the sub-images. For example, the calculation module 304 may calculate a gradient value of an image block on five (5) directions to select a direction having the greatest gradient value to be the direction of the image block for statistics.
  • the calculation module 304 may calculate the frequency that each of the 5 directions is selected as the direction for statistics in one sub-image. The frequency may be used as histograms of the sub-image.
  • a sub-image is divided into 1000 image blocks, and the one or more directions include A, B, C, D and E, 5 directions. Further, suppose that direction A is selected as the directions of 100 image blocks for statistics, direction B had 200 image blocks, direction C has 300 image blocks, direction D has 400 image blocks, and direction E has 0 image blocks.
  • the statistics value (i.e., corresponding histograms) of the sub-image is a vector (100, 200, 300, 400, 0).
  • the generation module 306 may conduct quantitative compression to, for example, the 80 dimensional vector to achieve better space utilization. For example, an image signature would be approximately 240 bits (i.e., 30 bytes) if 8 digitals between 0-7 for quantitative process are used. The process may save 90% of storage.
  • the image servers 102 may obtain the image signature after the compression process.
  • the conversion module 302, the calculation module 304 and the generation module 306 may be implemented in a MCU.
  • FIG. 6 is a flowchart showing an exemplary process 600 of creating an image index.
  • the first Hash operation module 308 may convert a vector obtained in the process 500 to a high-dimensional binary vector (i.e., each dimension is 1 or 0) in a Hamming space. For example, suppose that a one-dimensional vector value is X and the biggest value is C. As a result, the vector presented in a Hamming space is C- dimensional binary vector including X consecutive Is right after C-X consecutive 0s.
  • the first Hash operation module 308 may define a Hash function G by randomly selecting a K dimensional binary vector from vectors obtained in the operation 602, and return a value after combining the results. In these instances, the greater the similarity between the vectors is, the greater the probability of generating the same Hash value is.
  • the first Hash operation module 308 may conduct L Hash functions on the high dimensional vector obtained in the operation 602 to reduce errors of the similarity search, wherein L is an integer.
  • the second Hash operation module 310 may conduct Hash operations to the results from the operation 606 with the traditional Hash function (e.g., Message-Digest Algorithm 5 (MD5)).
  • the second Hash operation module 310 may save Hash results of the operation 608 as keys and unique IDs of image in corresponding L Hash tables, wherein L is an integer.
  • images with the same image signatures may be saved in the same bin, while images with different image signatures are more likely to be saved in different bins.
  • the first Hash operation module 308 and the second Hash operation module 310 may be implemented by a same encoding chip or a same MCU.
  • a simulation may be conducted for prediction in advance.
  • the image index supported by appropriate hardware, may be stored in the memory to increase efficiency in searching.
  • an image index document may be stored in local disks or processed in a way of distribution for a mass sample database.
  • FIG. 7 is a flowchart showing an exemplary process 700 of identifying similar images.
  • the obtaining unit 210 may calculate an image signature of the image submitted by the user 106.
  • the Hash operation unit 218 may define a LSH function based on the image signature.
  • a Hash code of the image signature may be calculated using multiple LSH functions that are randomly selected.
  • a Hash operation (e.g., MD5) may be conducted to obtained results.
  • the searching unit 214 may search L Hash tables using the results obtained in the operation 708 as keys.
  • the searching unit 214 may add candidate images based on the searched results to a candidate image array.
  • the recognition unit 212 may calculate a spatial distance (e.g., Hamming distance or Euclidean distance) between each image signature in the candidate image array and the image signature of the image submitted by the user 106. In some embodiments, the distance may indicate similarity between the candidate image and the submitted image.
  • the output unit 210 may sort and output candidate images based on the distance.
  • a number of the image signatures in the candidate array is much smaller than a number of the image signatures in database. Accordingly, the cost of calculation can be greatly reduced while compared with conventional technologies.
  • the Hash operation unit 218 may be a MCU or a FPGA. In some embodiments, the functions operated by the Hash operation unit 218 and the obtaining unit 216 can be implemented by a same processor. In some embodiments, the searching unit 214 may communicate with database through an internal bus, and the searching unit 214 and the recognition unit 212 may be implemented by a same MCU.
  • an image signature may be determined based on content of the image. Similarity of image signatures depends on similarity of image content. For example, the more similar two images are, the more similar their image signatures are.
  • the similarity may be represented by a spatial distance (e.g., Hamming distance or Euclidean distance). Therefore, two identical images but saved in different formats may have the same image signatures, resolving the fault tolerance problem existing in the conventional similar image recognition technologies.
  • the disclosure also introduces methods of creating a mass similar image index based on a LSH Hash technology. The methods may reduce complexity of search, save time to a non-linear level, and sort and output results based on similarities.
  • the disclosure also provides edge histogram statistics as a basic feature of the image signature, an image division and the non-uniform quantitative technology based on human eye sensitivity.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Library & Information Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Image Analysis (AREA)

Abstract

A user may submit an image and request from a server one or more images that are similar to the submitted image. The server may generate an image signature based on the content of the submitted image. The server may conduct a Hash operation to the image signature to generate one or more Hash values. These Hash values may be used to identify one or more candidate images similar to the image in a Hash table. These candidate images may be sorted and outputted to the user based on similarity. The similarity between each of the candidate images and the image may be determined using at least one of Hamming distance or Euclidean distance.

Description

METHOD AND APPARATUS OF IDENTIFYING SIMILAR IMAGES
CROSS REFERENCE TO RELATED PATENT APPLICATIONS
This application claims priority to Chinese Patent Application No. 201110031701.8, filed on January 28, 2011, entitled "METHOD AND DEVICE OF SIMILAR IMAGE RECOGNITION," which is hereby incorporated by reference in its entirety.
TECHN ICAL FIELD
This disclosure relates to the field of multimedia image recognition. More specifically, the disclosure relates to methods and devices of image recognition.
BACKGROUND
Similar image retrieval, as a type of multimedia recognition technologies, is booming. The similar image retrieval may comprise feature extraction, index construction, inquiries, and similarities sorting. However, conventional technologies of similar image retrieval may present some problems (e.g., poor compatibility and fault tolerance problems) because conventional image signatures do not contain content information of images. For example, two images that have identical content but saved in different formats (e.g., bmp, jpeg, png, or gif) may be considered as different images. SUMMARY
This disclosure provides methods and devices for identifying similar images. In some embodiments, a user may submit an image and request from a server a set of images that are similar to the submitted image. Upon receiving the submitted image, the server may generate an image signature based on the content of that image. The server may also conduct a Hash operation to the image signature to generate one or more Hash values. The one or more values may be used to identify candidate images similar to the image. The server may then determine an image similarity between each of these candidate images and the image.
In some embodiments, to generate the image signature, the server may convert the image into a gray image and then divide the gray image into multiple sub-images. The server may then calculate edge histograms of each of the multiple sub-images. Based on the edge histograms, the server may generate the image signature. In some embodiments, to generate the one or more Hash values, the server may conduct a Hash operation to the image signature using Locality-sensitive hashing functions to generate one or more first Hash values. The server may then conduct a Hash operation to each of the one or more first Hash values to generate one or more second Hash values. The one or more second Hash values may be used to identify the candidate images. In some embodiments, to determine the image similarity, the server may determine a similarity between the image and each of the one or more candidate images using at least one of Hamming distance or Euclidean distance. BRIEF DESCRI PTION OF TH E DRAWI NGS
The Detailed Description is described with reference to the accompanying figures. The use of the same reference numbers in different figures indicates similar or identical items.
FIG. 1 is a schematic diagram showing an exemplary environment for identifying similar images.
FIG. 2 is a schematic block diagram showing an exemplary of the image server(s) of FIG. 1.
FIG. 3 is a schematic block diagram showing another exemplary of image server(s) of FIG. 1.
FIG. 4 is a flowchart showing an exemplary process of identifying similar images. FIG. 5 is a flowchart showing an exemplary process of obtaining an image signature.
FIG. 6 is a flowchart showing an exemplary process of creating an image index. FIG. 7 is a flowchart showing another exemplary process of identifying similar images.
DETAILED DESCRIPTION
The detailed description of a method of and server for TCP is set forth with reference to the accompanying figures.
FIG. 1 is a schematic diagram showing an exemplary environment 100 for identifying similar images. The environment 100 may include image servers 102 interacting with a user 104 and a user 106. The image servers 102 may include upload servers 108, an image database device 110, index servers 112, image calculation servers 114, and image search front servers 116.
The user 104 may uploads an image to the upload servers 108 to create image index database. The upload servers 108 may calculate an image signature of the image and send the image signature to the image database device 110. In some embodiments, the image may be obtained from the Internet and saved in the upload servers 108 using a crawler system. In some embodiments, the upload servers 108 may calculate the image signature of the image based on content information of the image, thus reducing probability of a conflict. For example, the content information of the image may include metadata (e.g., color, grain, etc.) instead of a non-byte stream.
The image database device 110 may conduct a Hash operation to the image by using Locality-sensitive hashing (LSH) after receiving the image signature. The resulting values are then sent as index information to the index servers 112.
The user 106 may send an image to the search front servers 116 and request the image servers to return a set of images similar to the submitted image. The image search front servers 116 may transmit the image to the image calculation servers 114 and request the similar images or a list of the similar images. In some embodiments, an image signature may be generated based on the content information of the image, and then the index servers 112 search for similar images based on the image signature. In some embodiments, the image calculation servers 114 may search the similar images (e.g., IDs) based on Hash values of these similar images on the index servers 112. For example, the image calculation servers 114 may identify IDs associated with images having the same Hash value as the image. The similar image or a list of similar images may be returned to the image search front server 116 and displayed to the user 106.
FIG. 2 is a schematic block diagram showing details of exemplary image servers 102 of FIG. 1. The image servers 102 may be configured as any suitable servers. In one exemplary configuration, the image servers 102 include processors 202, input/output interfaces 204, network interfaces 206, and memory 208.
The memory 208 may include computer-readable media in the form of volatile memory, such as random-access memory (RAM) and/or non-volatile memory, such as read only memory (ROM) or flash RAM. The memory 208 is an example of computer-readable media.
Computer-readable media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk readonly memory (CD-ROM), digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.
Turning to the memory 208 in more detail, the memory 208 may store any number of software modules or units, including an output unit 210, a recognition unit 212, a search unit 214, an obtaining unit 216, a Hash operation unit 218, and a Hash table creation unit 220. The obtaining unit 216 may obtain an image signature of an image submitted by the user 106. The Hash operation unit 218 may conduct a Hash operation to the image signature. The searching unit 214 may search for candidate images in predetermined Hash tables. The recognition unit 212 may identify and retrieve image(s) similar to the image from candidate images.
In some embodiments, the image servers 102 may determine similarities of images based on image signatures generated from content of the images. For example, the more similar the images are, the more similar their image signatures are. In some embodiments, the similarities may be related to Hamming distance or Euclidean distance. For example, the shorter these distances, the more similar are the images. Accordingly, images that are the same but saved in different formats may have the same image signatures. Also, problems of fault tolerance with the conventional technologies may be resolved.
In some embodiments, the output unit 210 may output the candidate images based on the similarities. In some embodiments, the Hash table creation unit 220 may create L Hash tables by conducting Hash operations to the image signature to obtain L Hash values by using L LSH functions, wherein L is an integer. The Hash table creation unit 220 may also conduct a Hash operation to the L first Hash values to obtain L second Hash values by using an overall Hash function. The Hash table creation unit 220 may record a jth second Hash value, an image identification and an image signature in the jth Hash table, wherein j=l...L.
FIG. 3 is a schematic block diagram showing another exemplary of image servers 300 that may include, similar to the image servers 102 of FIGS. 1 and 2, the recognition unit 212, the searching unit 214, the obtaining unit 216, and the Hash operation unit 218.
The obtaining unit 216 may include a conversion module 302, a calculation module 304, and a generation module 306. The conversion module 302 may convert an image submitted by the user 106 to a gray image. The calculation module 304 may then divide the gray image into NxN sub-images, calculate edge histograms from M directions for each of the sub-images, and obtain NxNxM calculation results, wherein N and M are integers. The generation module 306 may assemble the NxNxM calculation results as an NxNxM dimensional vector of the image signature. In some embodiments, non-uniform quantitative technologies based on human eye sensitivity can be applied to the NxNxM dimensional vector. In some embodiments, statistic data of edge histograms may be used as basic features of the image signature. The statistic data combined with the image division and the non-uniform quantitative technologies based on human eye sensitivity may be resolved problems caused by different colors, scales, or partial fuzzy & distortion. In some embodiments, the gray images may be divided into NxP sub-images.
In some embodiments, the calculation module 304 may, for each image block included in a sub-image, calculate a gradient value on each of the M directions. The calculation module 304 may then select one of the M directions that has the greatest gradient value as the direction of the corresponding image block for statistics. The module 304 may further calculate the number of the selected directions for each of the M directions and obtain the statistic value as histograms of the sub-image.
Suppose that a sub-image has 1000 image blocks, and M has 5 directions including A, B, C, D and E (i.e., M=5). Further, suppose that direction A for statistic data are 100 image blocks, direction B has 200 image blocks, direction C has 300 image blocks, direction D has 400 image blocks, and direction E has no image blocks. As a result, the statistics value (i.e., histograms) of the sub-image may be represented by a vector (100, 200, 300, 400, 0).
In some embodiments, the Hash operation unit 218 may include a first Hash operation module 308 and a second Hash operation module 310. The first Hash operation module 308 may conduct Hash operations to the image signature, and obtain L first Hash values by using L LSH functions, wherein L is an integer. The second Hash operation module 310 may conduct Hash operations to the L first Hash values, and obtain L second Hash values by using an overall Hash function. In some embodiments, a method of mass similar image index based on the LSH technology may be used to reduce complexity of search time to a non-linear level.
In some embodiments, the searching unit 214 may include a searching module 312 and an addition module 314. The searching module 312 may search an entry recording the second Hash value in the Hash tables. Each entry of the Hash tables records a Hash value and an image identification of an image. Each entry may further include an image signature. The addition module 314 may add an image to the candidate images upon identifying an entry having the second Hash value in the Hash tables. In some embodiments, an ith second Hash value may correspond to an ith Hash table, and i=l ... L.
In some embodiments, the first Hash operation module 308 may convert an image signature to a R dimensional binary vector, wherein R is an integer. The first Hash operation module 308 may then generate L LSH functions by using the R dimensional binary vector. For example, each of the LSH functions is generated based on a one-dimensional or multiple-dimensional binary vector of the R dimensional binary vector.
In some embodiments, the first Hash operation module 308 may assign an input parameter of a LSH function as K, randomly select K dimensional binary vector from the R dimensional binary vector, and combine the K dimensional binary vector as a return value of the LSH function, wherein K<R.
In some embodiments, the recognition unit 212 may include a calculation module 316 and a recognition module 318. The calculation module 316 may calculate a spatial distance between an image signature of a submitted image and an image signature of a candidate image (e.g., Hamming distance or Euclidean distance). The recognition module 318 may then determine a similarity between the candidate image and the submitted image based on the spatial distance. For example, the smaller the spatial distance is, the more similar the candidate image is to the submitted image.
FIGS. 4-7 are flowcharts showing exemplary processes of identifying similar images. The processes are illustrated as a collection of blocks in logical flowcharts, which represent a sequence of operations that can be implemented in hardware, software, or a combination. For discussion purposes, the processes are described with reference to the image servers 102 shown in FIGS. 1-3. However, the processes may be performed using different environments and devices. Moreover, the environments and devices described herein may be used to perform different processes.
FIG. 4 is a flowchart showing an exemplary process 400 of identifying similar images. The process 400 may be performed by one or more processors of the image servers, which may be inclusive of one or more microcontrollers (MCU) or one or more field-programmable gate arrays (FPGAs).
At 402, the image servers may receive an image submitted by the user 106 via, for example, a USB transmission interface, a Bluetooth transmission interface, or an Ethernet transmission interface. At 404, the obtaining unit 216 may generate or otherwise obtain an image signature of the submitted image. For example, the obtaining unit 216 may involve a MCU or a FPGA to calculate the image signature.
At 406, the Hash operation unit 218 may conduct a Hash operation to the image signature. In some embodiments, the Hash operation unit 218 may include a MCU or a FPGA configured to perform the Hash operations. In some embodiments, the functions performed by the Hash operation unit 218 and the obtaining unit 216 may be implemented by the same processor(s).
At 408, the search unit 214 may search entries corresponding to a result of the Hash operation in predetermined Hash tables to identify candidate images. At 410, the recognition unit 212 may recognize an image similar to the submitted image from candidate images corresponding to the entries. In some embodiments, the image calculation server 108 obtains, locally or from third-party equipment, the candidate images based on the IDs of similar images returned by the index server 106, and recognizes the image similar to the submitted image from the candidate images based on the entries. In some embodiments, the recognition unit 212 may obtain, locally or from third-party equipment, the candidate image based on the ID of the similar images returned by the searching unit 214, and recognizes the image similar to the submitted image from the candidate images based on the entries. In some embodiment, the output unit 210 may include a Bluetooth transmission module, an infrared transmission module and/or an Ethernet transmission module to output the identified similar image and to display to the user 106.
FIG. 5 is a flowchart showing an exemplary process 500 of obtaining an image signature. The process 500 may generate an image signature based on the grain features of edge histograms.
At 502, the image servers convert an image submitted by the user 106 to a gray image to make final results not sensitive to color and/or light. At 504, the calculation module 304 may divide the gray image into N xN sub-images. At 506, the calculation module 304 may further divide each of the sub-images into a fix number of image blocks. The area of each image block depends on the area of the submitted image. At 508, the calculation module 304 may then calculate edge histograms from one or more directions for each of the sub-images. For example, the calculation module 304 may calculate a gradient value of an image block on five (5) directions to select a direction having the greatest gradient value to be the direction of the image block for statistics. The calculation module 304 may calculate the frequency that each of the 5 directions is selected as the direction for statistics in one sub-image. The frequency may be used as histograms of the sub-image.
Suppose that a sub-image is divided into 1000 image blocks, and the one or more directions include A, B, C, D and E, 5 directions. Further, suppose that direction A is selected as the directions of 100 image blocks for statistics, direction B had 200 image blocks, direction C has 300 image blocks, direction D has 400 image blocks, and direction E has 0 image blocks. As a result, the statistics value (i.e., corresponding histograms) of the sub-image is a vector (100, 200, 300, 400, 0).
At 510, the generation module 306 may combine the vector of each sub- image to a multiple dimensional vector as the image signature of the image submitted by the user 106. If N=4, an image signature may be presented as a 4x4x5=80 dimensional vector. In some embodiments, at 512, a non-uniform quantitative method may be adopted given non-uniform characteristics of human eye sensitivity. The generation module 306 may conduct quantitative compression to, for example, the 80 dimensional vector to achieve better space utilization. For example, an image signature would be approximately 240 bits (i.e., 30 bytes) if 8 digitals between 0-7 for quantitative process are used. The process may save 90% of storage.
At 514, the image servers 102 may obtain the image signature after the compression process. In some embodiments, the conversion module 302, the calculation module 304 and the generation module 306 may be implemented in a MCU.
FIG. 6 is a flowchart showing an exemplary process 600 of creating an image index. At 602, the first Hash operation module 308 may convert a vector obtained in the process 500 to a high-dimensional binary vector (i.e., each dimension is 1 or 0) in a Hamming space. For example, suppose that a one-dimensional vector value is X and the biggest value is C. As a result, the vector presented in a Hamming space is C- dimensional binary vector including X consecutive Is right after C-X consecutive 0s.
At 604, the first Hash operation module 308 may define a Hash function G by randomly selecting a K dimensional binary vector from vectors obtained in the operation 602, and return a value after combining the results. In these instances, the greater the similarity between the vectors is, the greater the probability of generating the same Hash value is.
At 606, the first Hash operation module 308 may conduct L Hash functions on the high dimensional vector obtained in the operation 602 to reduce errors of the similarity search, wherein L is an integer. At 608, the second Hash operation module 310 may conduct Hash operations to the results from the operation 606 with the traditional Hash function (e.g., Message-Digest Algorithm 5 (MD5)). At 610, the second Hash operation module 310 may save Hash results of the operation 608 as keys and unique IDs of image in corresponding L Hash tables, wherein L is an integer. In some embodiments, images with the same image signatures may be saved in the same bin, while images with different image signatures are more likely to be saved in different bins. In some embodiments, the first Hash operation module 308 and the second Hash operation module 310 may be implemented by a same encoding chip or a same MCU.
In some embodiments, because accuracy and recall rates may be dramatically affected by different K and L, a simulation may be conducted for prediction in advance. The image index, supported by appropriate hardware, may be stored in the memory to increase efficiency in searching. In some embodiments, an image index document may be stored in local disks or processed in a way of distribution for a mass sample database.
FIG. 7 is a flowchart showing an exemplary process 700 of identifying similar images. At 702, the obtaining unit 210 may calculate an image signature of the image submitted by the user 106. At 704, the Hash operation unit 218 may define a LSH function based on the image signature. At 706, a Hash code of the image signature may be calculated using multiple LSH functions that are randomly selected. At 708, a Hash operation (e.g., MD5) may be conducted to obtained results. At 710, the searching unit 214 may search L Hash tables using the results obtained in the operation 708 as keys.
At 712, the searching unit 214 may add candidate images based on the searched results to a candidate image array. At 714, the recognition unit 212 may calculate a spatial distance (e.g., Hamming distance or Euclidean distance) between each image signature in the candidate image array and the image signature of the image submitted by the user 106. In some embodiments, the distance may indicate similarity between the candidate image and the submitted image.
At 716, the output unit 210 may sort and output candidate images based on the distance. In these instances, a number of the image signatures in the candidate array is much smaller than a number of the image signatures in database. Accordingly, the cost of calculation can be greatly reduced while compared with conventional technologies.
In some embodiments, the Hash operation unit 218 may be a MCU or a FPGA. In some embodiments, the functions operated by the Hash operation unit 218 and the obtaining unit 216 can be implemented by a same processor. In some embodiments, the searching unit 214 may communicate with database through an internal bus, and the searching unit 214 and the recognition unit 212 may be implemented by a same MCU.
In this disclosure, an image signature may be determined based on content of the image. Similarity of image signatures depends on similarity of image content. For example, the more similar two images are, the more similar their image signatures are. The similarity may be represented by a spatial distance (e.g., Hamming distance or Euclidean distance). Therefore, two identical images but saved in different formats may have the same image signatures, resolving the fault tolerance problem existing in the conventional similar image recognition technologies. The disclosure also introduces methods of creating a mass similar image index based on a LSH Hash technology. The methods may reduce complexity of search, save time to a non-linear level, and sort and output results based on similarities. The disclosure also provides edge histogram statistics as a basic feature of the image signature, an image division and the non-uniform quantitative technology based on human eye sensitivity.
The em bodiments in this disclosure are merely for illustrating purposes and are not intended to limit the scope of this disclosure. A person having ordinary skill in the art would be able to make changes and alterations to embodiments provided in this disclosure. Any changes and alterations that persons with ordinary skill in the art would appreciate fall within the scope of this disclosure.

Claims

CLAIMS What is claimed is:
1. A computer-implemented method for identifying similar images, the method comprising:
receiving an image;
generating an image signature of the image based on content of the image; conducting a Hash operation to the image signature to generate a value; identifying, in one or more Hash tables, one or more candidate images based on the value; and
determining an image similarity between the image and each of the one or more candidate images.
2. The computer-implemented method as recited in claim 1, wherein the generating of the image signature comprises:
converting the image into a gray image;
dividing the gray image into one or more sub-images;
calculating edge histograms from multiple directions for each of the one or more sub-images; and
generating the image signature based on the edge histograms.
3. The computer-implemented method as recited in claim 1, wherein the conducting of the Hash operation comprises:
conducting one or more Hash operations to the image signature using one or more Locality-sensitive hashing (LSH) functions to generate one or more first Hash values; and
conducting another Hash operation to each of the one or more first Hash values to generate one or more second Hash values.
4. The computer-implemented method as recited in claim 3, wherein the identifying of the one or more candidate images comprises identifying based on the one or more second Hash values.
5. The computer-implemented method as recited in claim 4, wherein the one or more Hash tables included one or more entries, each entry including at least one of a Hash value, an image identification of one image, or an image signature of one image, and each of the one or more second Hash values corresponds to one of the one or more Hash tables.
6. The computer-implemented method as recited in claim 3, wherein the conducting of the one or more Hash operations comprises:
converting the image signature to a binary vector having a plurality of dimensions;
generating one or more LSH functions based on the binary vector such that each of the one or more LSH functions is generated based on at least one of a single dimension binary vector of the binary, or a multiple dimension binary vector of the binary vector; and
conducting the one or more Hash operations to the binary vector by using the one or more LSH functions.
7. The method as recited in claim 6, wherein the generating of the one or more LSH functions comprises:
assigning a certain number to each input parameter of the one or more LSH function, the certain number being lower than a number of the plurality of dimensions; and
selecting a binary vector having the certain number of dimensions from the binary vector to generate a return value for each of the one or more LSH functions.
8. The method as recited in any one of claims 1, wherein the determining of the image similarity comprises:
calculating a spatial distance between the image signature of the image and an image signature of each of the one or more candidate images; and
determining the image similarity based on the spatial distance.
9. The method as recited in claim 8, further comprising:
outputting each of the candidate images based on the spatial distance.
10. The method as recited in claim 3, further comprising:
conducting the one or more Hash operations to an image signature of each of the one or more candidate images to generate one or more third Hash values based on the one or more LSH functions;
conducting the other Hash operation to each of the one or more third Hash values to generate one or more fourth Hash value; and recording each of the one or more fourth Hash values, a corresponding image identification, and a corresponding image signature in a corresponding Hash table.
11. One or more servers comprising:
memory;
one or more processors coupled to the memory;
an obtaining unit configured to:
receive an image, and
generate an image signature of the image based on content of the image;
a Hash operation unit configured to conduct a Hash operation to the image signature to generate a value;
a searching unit configured to identify, in one or more Hash tables, one or more candidate images based on the value; and
a recognition unit configured to determine an image similarity between the image and each of the one or more candidate images.
12. The one or more servers as recited in claim 11, wherein the obtaining unit is configured to generate the image signature by executing operation to:
convert the image into a gray image;
divide the gray image one or more sub-images, and
calculate edge histograms from multiple directions for each of the one or more sub-images; and
generate the image signature based on the edge histograms.
13. The one or more servers as recited in claim 11, wherein the Hash operation unit is configured to conduct of the Hash operation by executing operations to:
conduct one or more Hash operations to the image signature using one or more LSH functions to generate one or more first Hash values; and
conduct another Hash operation to each of the one or more first Hash values to generate one or more second Hash values.
14. The one or more servers as recited in claim 13, wherein the searching unit is configured to identify the one or more candidate images by identifying, based on the one or more second Hash values, the one or more Hash tables including one or more records that each include at least one of a Hash value, an image identification of one image, or an image signature of one image.
15. The one or more servers as recited in claim 13, wherein the Hash operation unit is configured to conduct of the one or more Hash operations by executing operations to:
convert the image signature to a R dimension binary vector, the R being a fixed integer;
generate one or more LSH functions based on the R dimension binary vector, each of the one or more LSH functions being generated based on at least one of a single dimension binary vector of the R dimension binary vector, or a multiple dimension binary vector of the R dimension binary vector; and conduct the one or more Hash operations by using the one or more LSH functions.
16. The one or more servers as recited in claim 15, wherein the Hash operation unit is further configured to generate the one or more LSH functions by: assigning K to each input parameter of the one or more LSH function, the K being another fixed integer and lower than R; and
selecting K dimension binary vector from R dimension binary vector to generate a return value for each of the one or more LSH functions.
17. The one or more servers as recited in any one of claims 11, wherein the recognition unit is configured to determine the image similarity by executing operations to:
calculate a spatial distance between the image signature of the image and an image signature of each of the one or more candidate images; and
determine the image similarity based on the spatial distance.
18. The one or more servers as recited in claim 17, further comprising an output unit configured to output each of the candidate images based on the spatial distance.
19. The one or more servers as recited in claim 13, further comprising a Hash ta ble creation unit configured to:
conduct one or more third Hash operations to an image signature of each of the one or more candidate images to obtain one or more third Hash values based on the one or more LSH functions; and
conducting a fourth Hash operation to each of the one or more third Hash values to obtain one or more fourth Hash value; and
recording each of the one or more fourth Hash values, a corresponding image identification, and a corresponding image signature in a corresponding Hash table.
20. One or more computer-readable media encoded with instructions that, when executed by a processor of a server, perform acts comprising:
receiving an image;
generating a multiple dimension vector indicating an image signature of the image based on edge histograms associated with the image;
conducting a Hash operation to the image signature to generate a first Hash value using a LSH function that is defined based on the multiple dimension vector; conducting another Hash operation to the first Hash value using Message- Digest Algorithm to generate a second Hash value;
retrieving from a Hash table one or more candidate images based on the second Hash value; and
determining a similarity between the image and each of the one or more candidate images using at least one of Hamming distance or Euclidean distance.
PCT/US2012/021722 2011-01-28 2012-01-18 Method and apparatus of identifying similar images WO2012102926A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US13/509,059 US9053386B2 (en) 2011-01-28 2012-01-18 Method and apparatus of identifying similar images
EP12739016.9A EP2668619A4 (en) 2011-01-28 2012-01-18 Method and apparatus of identifying similar images
JP2013552013A JP5926291B2 (en) 2011-01-28 2012-01-18 Method and apparatus for identifying similar images

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201110031701.8A CN102622366B (en) 2011-01-28 2011-01-28 Similar picture identification method and similar picture identification device
CN201110031701.8 2011-01-28

Publications (1)

Publication Number Publication Date
WO2012102926A1 true WO2012102926A1 (en) 2012-08-02

Family

ID=46562289

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2012/021722 WO2012102926A1 (en) 2011-01-28 2012-01-18 Method and apparatus of identifying similar images

Country Status (7)

Country Link
US (1) US9053386B2 (en)
EP (1) EP2668619A4 (en)
JP (1) JP5926291B2 (en)
CN (1) CN102622366B (en)
HK (1) HK1168919A1 (en)
TW (1) TWI528196B (en)
WO (1) WO2012102926A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11556581B2 (en) * 2018-09-04 2023-01-17 Inception Institute of Artificial Intelligence, Ltd. Sketch-based image retrieval techniques using generative domain migration hashing

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI470549B (en) * 2012-04-20 2015-01-21 Insyde Software Corp A method of using an image recognition guide to install an application, and an electronic device
CN103324650A (en) * 2012-10-23 2013-09-25 深圳市宜搜科技发展有限公司 Image retrieval method and system
CN103902595B (en) * 2012-12-27 2017-02-08 北京新媒传信科技有限公司 Method and device for establishing database
CN103761483A (en) * 2014-01-27 2014-04-30 百度在线网络技术(北京)有限公司 Method and device for detecting malicious codes
CN103955914A (en) * 2014-02-27 2014-07-30 西安电子科技大学 SAR image segmentation method based on random projection and Signature/EMD framework
CN103984776B (en) * 2014-06-05 2017-05-03 北京奇虎科技有限公司 Repeated image identification method and image search duplicate removal method and device
CN105224569B (en) 2014-06-30 2018-09-07 华为技术有限公司 A kind of data filtering, the method and device for constructing data filter
CN104199827B (en) * 2014-07-24 2017-08-04 北京大学 The high dimensional indexing method of large scale multimedia data based on local sensitivity Hash
KR101713197B1 (en) * 2015-04-01 2017-03-09 주식회사 씨케이앤비 Server computing device and system for searching image based contents cognition using the same
CN106302202B (en) * 2015-05-15 2020-07-28 阿里巴巴集团控股有限公司 Data current limiting method and device
WO2017099806A1 (en) * 2015-12-11 2017-06-15 Hewlett Packard Enterprise Development Lp Hash suppression
CN105589938A (en) * 2015-12-13 2016-05-18 公安部第三研究所 Image retrieval system and retrieval method based on FPGA
CN106991095B (en) * 2016-01-21 2021-09-28 阿里巴巴集团控股有限公司 Machine exception handling method, learning rate adjusting method and device
CN107464268A (en) * 2016-06-02 2017-12-12 国家计算机网络与信息安全管理中心 A kind of joint coding method using global and local feature
CN108154072A (en) * 2016-12-02 2018-06-12 天津工业大学 Insulator breakdown of taking photo by plane based on depth convolutional neural networks detects automatically
CN108011956A (en) * 2017-12-06 2018-05-08 易居(中国)企业集团股份有限公司 Distributed storage method based on file content cryptographic Hash
CN108121806A (en) * 2017-12-26 2018-06-05 湖北工业大学 One kind is based on the matched image search method of local feature and system
WO2019198850A1 (en) * 2018-04-13 2019-10-17 라인플러스 주식회사 Method and system for generating animal-shaped avatar by using human face
CN108629049A (en) * 2018-05-14 2018-10-09 芜湖岭上信息科技有限公司 A kind of image real-time storage and lookup device and method based on hash algorithm
US11126649B2 (en) 2018-07-11 2021-09-21 Google Llc Similar image search for radiology
KR101998200B1 (en) * 2018-11-26 2019-07-10 주식회사 아임클라우드 Same images identification information generation system
US11977959B2 (en) * 2019-05-15 2024-05-07 EMC IP Holding Company LLC Data compression using nearest neighbor cluster
CN113383314B (en) * 2019-06-26 2023-01-10 深圳市欢太科技有限公司 User similarity calculation method and device, server and storage medium
CN110909791B (en) * 2019-11-20 2023-02-28 车智互联(北京)科技有限公司 Similar image identification method and computing device
US11645323B2 (en) 2020-02-26 2023-05-09 Samsung Electronics Co.. Ltd. Coarse-to-fine multimodal gallery search system with attention-based neural network models
CN111414527B (en) * 2020-03-16 2023-10-10 腾讯音乐娱乐科技(深圳)有限公司 Query method, device and storage medium for similar items
CN111818236A (en) * 2020-07-08 2020-10-23 山东省肿瘤防治研究院(山东省肿瘤医院) Synchronous acquisition and playback method of nuclear magnetic resonance image
CN112581516A (en) * 2020-11-30 2021-03-30 北京迈格威科技有限公司 Image matching method and device, electronic equipment and storage medium
CN112614187B (en) * 2020-12-31 2024-03-26 深圳市优必选科技股份有限公司 Loop detection method, loop detection device, terminal equipment and readable storage medium
US20240220083A1 (en) * 2022-12-29 2024-07-04 Automation Anywhere, Inc. Identifying user interfaces of an application

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080313140A1 (en) * 2007-06-18 2008-12-18 Zeitera, Llc Method and Apparatus for Multi-Dimensional Content Search and Video Identification
US20090324100A1 (en) * 2008-06-27 2009-12-31 Palo Alto Research Center Incorporated Method and system for finding a document image in a document collection using localized two-dimensional visual fingerprints
US20100166339A1 (en) * 2005-05-09 2010-07-01 Salih Burak Gokturk System and method for enabling image recognition and searching of images

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6671407B1 (en) 1999-10-19 2003-12-30 Microsoft Corporation System and method for hashing digital images
US8037310B2 (en) * 2004-11-30 2011-10-11 Ricoh Co., Ltd. Document authentication combining digital signature verification and visual comparison
JP4284288B2 (en) 2005-03-10 2009-06-24 株式会社東芝 Pattern recognition apparatus and method
US20060242418A1 (en) * 2005-04-25 2006-10-26 Xerox Corporation Method for ensuring the integrity of image sets
US7519200B2 (en) * 2005-05-09 2009-04-14 Like.Com System and method for enabling the use of captured images through recognition
US7783135B2 (en) * 2005-05-09 2010-08-24 Like.Com System and method for providing objectified image renderings using recognition information from images
JP4968882B2 (en) * 2005-06-03 2012-07-04 キヤノン株式会社 Image search apparatus, image search method, and program
US7457825B2 (en) * 2005-09-21 2008-11-25 Microsoft Corporation Generating search requests from multimodal queries
US7647331B2 (en) 2006-03-28 2010-01-12 Microsoft Corporation Detecting duplicate images using hash code grouping
US20080059461A1 (en) * 2006-08-29 2008-03-06 Attributor Corporation Content search using a provided interface
EP1912160B1 (en) * 2006-10-11 2012-05-16 Mitsubishi Electric R&D Centre Europe B.V. Image descriptor for image recognition
US8265345B2 (en) 2006-11-20 2012-09-11 Sharp Kabushiki Kaisha Image processing method, image processing apparatus, image forming apparatus, and image reading apparatus
US8055079B2 (en) 2007-03-06 2011-11-08 Sharp Kabushiki Kaisha Image processing method, image processing apparatus, and image forming apparatus
JP4898531B2 (en) * 2007-04-12 2012-03-14 キヤノン株式会社 Image processing apparatus, control method therefor, and computer program
JP2009031876A (en) 2007-07-24 2009-02-12 Sharp Corp Image processor, image forming device and image reader therewith, image processing method, image processing program and recording medium recording image processing program
JP5098504B2 (en) 2007-08-09 2012-12-12 富士通株式会社 Character recognition program, character recognition device, and character recognition method
US20100174707A1 (en) * 2007-09-07 2010-07-08 Daiki Kudo Image searching apparatus, image sorting apparatus and method and program
GB2471036B (en) * 2008-03-03 2012-08-22 Videoiq Inc Object matching for tracking, indexing, and search
JP4538507B2 (en) 2008-05-02 2010-09-08 シャープ株式会社 Image collation method, image collation apparatus, image data output processing apparatus, program, and storage medium
JP4565015B2 (en) * 2008-05-15 2010-10-20 シャープ株式会社 Image processing apparatus, image forming apparatus, image processing system, image processing program, and recording medium thereof
US8144947B2 (en) * 2008-06-27 2012-03-27 Palo Alto Research Center Incorporated System and method for finding a picture image in an image collection using localized two-dimensional visual fingerprints
CN101354728B (en) * 2008-09-26 2010-06-09 中国传媒大学 Method for measuring similarity based on interval right weight
US8533848B2 (en) * 2009-02-18 2013-09-10 Korea Advanced Institute Of Science And Technology Method and system for producing multimedia fingerprint based on quantum hashing
US20100268628A1 (en) * 2009-04-15 2010-10-21 Attributor Corporation Managing controlled content on a web page having revenue-generating code
JP5224537B2 (en) * 2009-06-01 2013-07-03 日本電信電話株式会社 Locality-detectable hash construction device, similar neighborhood search processing device, and program
US8467631B2 (en) * 2009-06-30 2013-06-18 Red Hat Israel, Ltd. Method and apparatus for identification of image uniqueness
US9233399B2 (en) * 2010-02-09 2016-01-12 Xerox Corporation Document separation by document sequence reconstruction based on information capture
US9183173B2 (en) * 2010-03-02 2015-11-10 Microsoft Technology Licensing, Llc Learning element weighting for similarity measures
US8452106B2 (en) 2010-03-23 2013-05-28 Microsoft Corporation Partition min-hash for partial-duplicate image determination
CN101887457B (en) * 2010-07-02 2012-10-03 杭州电子科技大学 Content-based copy image detection method
US8498473B2 (en) 2010-08-24 2013-07-30 Compuware Corporation System for computationally quantifying similarities between images

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100166339A1 (en) * 2005-05-09 2010-07-01 Salih Burak Gokturk System and method for enabling image recognition and searching of images
US20080313140A1 (en) * 2007-06-18 2008-12-18 Zeitera, Llc Method and Apparatus for Multi-Dimensional Content Search and Video Identification
US20090324100A1 (en) * 2008-06-27 2009-12-31 Palo Alto Research Center Incorporated Method and system for finding a document image in a document collection using localized two-dimensional visual fingerprints

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2668619A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11556581B2 (en) * 2018-09-04 2023-01-17 Inception Institute of Artificial Intelligence, Ltd. Sketch-based image retrieval techniques using generative domain migration hashing

Also Published As

Publication number Publication date
TW201232302A (en) 2012-08-01
CN102622366A (en) 2012-08-01
EP2668619A4 (en) 2016-12-14
JP2014505313A (en) 2014-02-27
US9053386B2 (en) 2015-06-09
CN102622366B (en) 2014-07-30
TWI528196B (en) 2016-04-01
US20130301935A1 (en) 2013-11-14
JP5926291B2 (en) 2016-05-25
HK1168919A1 (en) 2013-01-11
EP2668619A1 (en) 2013-12-04

Similar Documents

Publication Publication Date Title
US9053386B2 (en) Method and apparatus of identifying similar images
EP3709184B1 (en) Sample set processing method and apparatus, and sample querying method and apparatus
Zhou et al. Towards codebook-free: Scalable cascaded hashing for mobile image search
Zhou et al. Scalar quantization for large scale image search
US9589208B2 (en) Retrieval of similar images to a query image
WO2017020741A1 (en) Method, apparatus and system for retrieving image, acquiring image information and identifying image
Neyshabur et al. The power of asymmetry in binary hashing
US9009149B2 (en) Systems and methods for mobile search using Bag of Hash Bits and boundary reranking
CN113918753B (en) Image retrieval method based on artificial intelligence and related equipment
EP2657884B1 (en) Identifying multimedia objects based on multimedia fingerprint
CN105095435A (en) Similarity comparison method and device for high-dimensional image features
Valsesia et al. Large-scale image retrieval based on compressed camera identification
US20190272341A1 (en) Layered locality sensitive hashing (lsh) partition indexing for big data applications
US9619501B2 (en) Index scan device and index scan method
TW201828109A (en) Image search, image information acquisition and image recognition methods, apparatuses and systems effectively improving the image search accuracy, reducing the rearrangement filtering workload, and improving the search efficiency
Zhang et al. Large‐scale video retrieval via deep local convolutional features
Chen et al. Real-time, large-scale duplicate image detection method based on multi-feature fusion
CN110083731B (en) Image retrieval method, device, computer equipment and storage medium
CN115129915A (en) Repeated image retrieval method, device, equipment and storage medium
JP6017277B2 (en) Program, apparatus and method for calculating similarity between contents represented by set of feature vectors
CN107430633B (en) System and method for data storage and computer readable medium
Liu et al. Video copy detection by conducting fast searching of inverted files
Yin et al. Content‐Based Image Retrial Based on Hadoop
US10824811B2 (en) Machine learning data extraction algorithms
CN111090743B (en) Thesis recommendation method and device based on word embedding and multi-value form concept analysis

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 13509059

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12739016

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2013552013

Country of ref document: JP

Kind code of ref document: A

REEP Request for entry into the european phase

Ref document number: 2012739016

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2012739016

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE