CN112182262B

CN112182262B - Image query method based on feature classification

Info

Publication number: CN112182262B
Application number: CN202011367078.9A
Authority: CN
Inventors: 马勇; 钱辉; 李辉; 李汉曦; 戴梦轩; 赵家乐; 郭诗辉
Original assignee: Jiangxi Normal University
Current assignee: Jiangxi Normal University
Priority date: 2020-11-30
Filing date: 2020-11-30
Publication date: 2021-03-19
Anticipated expiration: 2040-11-30
Also published as: CN112182262A

Abstract

The invention discloses an image query method based on feature classification, which comprises the following steps: extracting the coding features and image features of the initial image uploaded by the image owner and storing the coding features and image features in a database; extracting coding features and image features of a query image uploaded by a user; comparing the coding features of the initial image and the query image to obtain a first query result; comparing the image characteristics of the initial image and the query image to obtain image similarity data; combining the first query result and the image similarity data, and obtaining a classification result through a feature classifier; processing the classification result by using a decision tree to obtain a second query result; and returning the initial image corresponding to the second query result to the user as a final query result. By the aid of the method, safe and quick retrieval and matching for massive image data can be realized, accuracy of results can be guaranteed, and key effects on further transferring client computing overhead to a cloud end and improving image data safe retrieval efficiency are achieved.

Description

Image query method based on feature classification

Technical Field

The invention relates to the technical field of internet, in particular to an image query method based on feature classification.

Background

Image data is a special data format, and is one of the most commonly used data formats in daily life, and is widely used in the fields of medical treatment, education, resident information management, design, social contact and the like. In order to store the image data, the size of the constructed image database is huge, and in view of the particularity of the image data, the image is generally queried by using the features of the image. After we extract the features of the image, the image can be represented by a multi-dimensional feature vector. The comparison and calculation of the high-dimensional data are the most important element calculation in the safe approximate nearest neighbor query, and are effective means for improving the efficiency of the approximate nearest neighbor query on the high-dimensional data. In the existing research, the relevant research is mainly carried out on low-dimensional structured data and a specific measurement method, the practical defects of difficult popularization from low dimension to high dimension, great influence of dimension disaster phenomenon, lack of universality and the like exist, the safety measurement of high-dimensional data approximation cannot be efficiently realized, and the method is not suitable for safe and rapid retrieval and matching for massive image data. Therefore, a universal and efficient high-dimensional data approximation security measurement mechanism is an important problem, and the solution of the problem is crucial to further migrating the client computing overhead to the cloud and improving the security and fast retrieval efficiency of image data.

The existing approximate nearest neighbor query method mainly has three main defects of fragile security model, low query efficiency and dimension disaster. The patent document with the publication number of CN 103810252B, named as an image retrieval method based on group sparse feature selection, proposes a scheme for forming a feature matrix for image similar pair features and dissimilar pair features, which can improve the accuracy of content-based image retrieval; however, the problem of low query efficiency exists because the complete image library is queried during query. The patent document with publication number CN 111125416 a entitled image retrieval method based on multi-feature fusion proposes a scheme for extracting and re-fusing global features and local features respectively for query, which fully extracts feature points of images to improve retrieval accuracy, but has the problem of overloading the system when the data size is large.

The noun explains:

PHA: perceptual hashing, which uses DCT (discrete cosine transform) to reduce the frequency, can achieve more accurate results compared to AHA.

DHA: the differential value hashing, based on gradual implementation, is superior to PHA in speed and superior to AHA in accuracy.

AHA: average hashing, which is implemented based on comparing each pixel of the gray map with the average value, is most suitable for thumbnail and enlarged image searching.

PCA: the principal component analysis method has the main idea that n-dimensional features are mapped to k-dimensions, the k-dimensions are brand new orthogonal features and are also called principal components, and the k-dimensional features are reconstructed on the basis of original n-dimensional features.

LBP: the local binary pattern is an operator used for describing local texture features of the image; the method has the advantages of rotation invariance, gray scale invariance and the like, and is used for extracting local texture features of the image.

SVD: singular value decomposition, for any matrix of M N, find a set of orthogonal bases so that after it is transformed, it is also an orthogonal base.

LDA: a document theme generation model is a three-layer Bayesian probability model and comprises three layers of structures of words, themes and documents.

Resnet: in the residual error network, one or more layers are directly skipped by introducing an identity shortcut connection (identity shortcut connection) structure to solve the problem that the accuracy of a training set is reduced along with the deepening of the network in the CNN (convolutional neural network).

Resnet 101: including Resnet variants of layer 101 networks.

K-MEANS clustering: a group of feature matrixes X of N samples are divided into K non-intersecting clusters, the clusters are visually a group of data gathered together, the data in one cluster are considered to be the same class, and the clusters are the result expression of the clustering.

Clustering by DBSCAN: a clustering algorithm based on density is characterized in that a sample set which is derived from a density reachable relation and is connected with the maximum density is a category of final clustering.

Expectation-maximization (EM) clustering of Gaussian Mixture Models (GMMs): and describing the cluster class of each sample by adopting a probability model, namely a soft division method.

Disclosure of Invention

The invention mainly solves the technical problem of providing an image query method based on feature classification, which can solve the practical defects of difficult popularization from low dimension to high dimension, great influence of dimension disaster phenomenon, lack of universality, excessive system burden and the like in the prior art, can not efficiently realize the safety measurement of high-dimensional data approximation, and is not suitable for safe and rapid retrieval and matching for massive image data.

In order to solve the technical problems, the invention adopts a technical scheme that: the image query method based on the feature classification specifically comprises the following steps:

step 1, extracting coding features and image features of an initial image uploaded by an image owner and storing the coding features and the image features in a database, wherein the initial image is stored in an image library;

and the image library is a data set of the image to be put in storage.

Step 2, extracting the coding features and image features of the image to be inquired;

step 3, comparing the coding features of the image to be queried with the coding features of the initial image, and preferably selecting a plurality of items as a first query result, wherein the method comprises the following steps:

step 3-1, extracting the initial image coding features stored in the database;

step 3-2, respectively calculating the Hamming distance between the image coding feature to be inquired and the initial image coding feature to obtain a Hamming distance calculation result set;

3-3, sorting the Hamming distance calculation results from small to large to obtain an ordered Hamming distance calculation result set;

and 3-4, selecting the first 15% of data from the ordered Hamming distance calculation result set as a first query result.

Step 4, calculating the cosine distance between the image feature of the image to be inquired and the image feature of the initial image corresponding to the first inquiry result to obtain image similarity data, including:

step 4-1, extracting image characteristics of the initial image corresponding to the first query result stored in the database;

step 4-2, respectively calculating cosine distances between the image features of the image to be inquired and the extracted image features of the initial image to obtain a cosine distance calculation result set;

and 4-3, sequencing the cosine distance calculation results from small to large to obtain an ordered cosine distance calculation result set serving as image similarity data.

And 5, combining the first query result and the image similarity data, classifying the first query result through a feature classifier to obtain a classification result, wherein the classification result comprises the following steps:

step 5-1, normalizing the hamming distance calculation result in the step 3 and the cosine distance calculation result in the step 4 to obtain a corresponding normalization result;

step 5-2, calculating difference distance of the normalization result;

step 5-3, dividing the special value in the calculation result of the difference distance as an edge;

and 5-4, taking the first two types of data in the division result as classification results.

And calculating the difference distance by taking each item of normalized data as source data, and comparing the items in the ordered set front and back to obtain the difference distance between the items, wherein a formula D = (next-this)/(Last-First) is calculated, wherein the next represents the next item of data, the this represents the item of data, the Last represents the Last item of data value in the ordered set, and the First represents the First item of data value in the ordered set.

The special value is a term in which the difference distance is greater than the average difference distance.

Step 6, processing the classification result by using the decision tree, and preferably selecting a plurality of items from the classification result as a second query result, wherein the method comprises the following steps:

step 6-1, performing pairwise comparison on image similarity data of the initial images corresponding to the classification results, taking a special value in the comparison results as an edge, and dividing the classification results into a plurality of classes;

6-2, taking the first two types of data in the division result as contrast classification results;

and 6-3, comparing every two coding features of the corresponding images in the contrast classification results, and taking the ten smallest data in the comparison results as second query results.

And 7, returning the initial image corresponding to the second query result as a final query result to the user.

The invention has the beneficial effects that:

1. the image and the characteristic information are stored separately, the image is stored in the image library, and only the characteristic information stored in the database is operated during query, so that the safety of image storage is ensured;

2. according to the method, the accuracy of approximate image query is ensured by carrying out multiple queries and classifications;

3. according to the method, most of images with insufficient similarity can be screened out through first query, so that the workload during second query is remarkably reduced, and the high efficiency of image approximate query is ensured;

4. the method can realize quick and safe retrieval and matching for massive image data, and plays a key role in further transferring the calculation overhead of the client to the cloud and improving the safe retrieval efficiency of the image data.

Drawings

FIG. 1 is a flow chart of a preferred embodiment of a method for image query based on feature classification according to the present invention;

Detailed Description

The following detailed description of the preferred embodiments of the present invention, taken in conjunction with the accompanying drawings, will make the advantages and features of the invention easier to understand by those skilled in the art, and thus will clearly and clearly define the scope of the invention.

Referring to fig. 1, an embodiment of the present invention includes:

example 1:

an image query method based on feature classification can efficiently query image information and can be used for safely retrieving and matching massive ciphertext images.

An image query method based on feature classification specifically comprises the following processes:

step 1, extracting and storing coding features and image features of an initial image uploaded by an image owner in a database, wherein the initial image is stored in an image library.

The specific process is as follows:

1-1, generating a hash value for an image by using a DHA algorithm, wherein the hash value is used as an encoding characteristic of the image;

step 1-2, extracting a neural network model by using the feature vector to process the image to obtain a high-dimensional feature vector;

and 1-3, reducing the dimension of the high-dimension feature vector by using a feature vector dimension reduction neural network model to obtain a low-dimension feature vector which is used as the image feature of the image.

And the image library is a data set of the image to be put in storage.

The hash value is generated mainly by using algorithms such as DHA, AHA, PHA, and the like, and the DHA algorithm is adopted in the embodiment, so that relatively good operation speed and accuracy can be obtained.

The extraction and dimension reduction of the feature vector are realized by PCA, LBP and other algorithms at present, and the dimension reduction of the high-dimensional feature vector is realized by SVD, LDA and other algorithms.

The low-dimensional feature vector dimension is 2048 dimensions.

And 2, extracting the coding features and the image features of the image to be inquired.

And 3, comparing the coding features of the image to be queried with the coding features of the initial image, and preferably selecting a plurality of items from the coding features as a first query result. The specific process is as follows:

step 3-1, extracting the initial image coding features stored in the database;

And 4, calculating the cosine distance between the image characteristics of the image to be inquired and the image characteristics of the initial image corresponding to the first inquiry result to obtain image similarity data. The specific process is as follows:

And 5, combining the first query result and the image similarity data, and classifying the first query result through a feature classifier to obtain a classification result. The specific process is as follows:

step 5-2, calculating difference distance of the normalization result;

The feature classifier uses a clustering method for classification, and the current clustering methods include K-MEANS clustering, mean shift clustering, DBSCAN clustering, Expectation Maximization (EM) clustering of Gaussian Mixture Model (GMM), and hierarchical clustering. The embodiment adopts the mean shift clustering algorithm, and has the advantages that the mean shift does not need to select the number of clusters, can be very intuitively understood and is suitable for natural data driving.

The normalization is a dimensionless processing means, and the absolute value of the physical system value is changed into a certain relative value relation. Simplifying the calculation and reducing the magnitude. The current normalization methods are (0,1) normalization, Z-score normalization, Sigmoid function, min-max normalization. The Z-score standardization algorithm is adopted in the embodiment, and the method has the advantages that the algorithm is simple and convenient, the result is convenient to compare, the method can be applied to numerical data, and the method is not influenced by the data magnitude.

The calculation of the difference distance takes each item of data after normalization as source data, and obtains the difference distance between each item through comparing the front and the back in the ordered set, and the specific calculation process is as follows: d = (next-this)/(Last-First), where next represents the next data item, this represents the previous data item, Last represents the Last data value in the sorted set, and First represents the First data value in the sorted set.

And 6, processing the classification result by using the decision tree, and preferably selecting a plurality of items from the classification result as a second query result. The specific process is as follows:

And 7, acquiring an initial image as a final query result according to the second query result and returning the final query result to the user.

Example 2:

the present embodiment has the same technical solution as embodiment 1, and more specifically:

processing the image to be queried in step 2:

inputting: an image to be queried;

and (3) outputting: the image query method comprises the steps of obtaining the image to be queried, the coding characteristic Img _ HASH _ search of the image to be queried and the image characteristic Img _ E _ search of the image to be queried.

More specifically:

Img_HASH_seach=ab23abab9398d4d0；

Img_E_seach=（0.90802705,1.0070337,0.94805366,…,1.0490991,1.0155762）。

comparing the coding characteristics in the step 3:

inputting: the image characteristic Img _ HASH _ ImgID and the image characteristic Img _ HASH _ seach of the image to be inquired are stored in the database;

and (3) outputting: hamming distance data distance _ Hamming.

More specifically, in this embodiment:

Img_HASH_000017029=79569a72722320af；

Img_HASH_000151962=8c829586c2a2a686；

Img_HASH_000369541=7a562364a202940b；

……

distance_000017029=4；

distance_000151962=12；

distance_000369541=12；

……

comparing the image characteristics in the step 4:

inputting: the image characteristics Img _ E _ ImgID and the image characteristics Img _ E _ search of the image to be inquired are stored in the database;

and (3) outputting: image similarity data distance _ cos.

More specifically, in this embodiment:

and 6, specific contents of the second query result:

{Img_Path,Img_HASH,distance_Hamming,distance_feature}

where Img _ Path represents the initial image storage Path.

More specifically, in this embodiment, the result of the second query is:

{(000017029.jpg,79569a72722320af,4,149.91662),

(000151962.jpg,8c829586c2a2a686,12,151.04811),

(000369541.jpg,7a562364a202940b,12,154.05436)}

the above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. An image query method based on feature classification is characterized by comprising the following steps:

step 3, comparing the coding features of the image to be queried with the coding features of the initial image, and preferably selecting a plurality of items from the coding features as a first query result;

step 4, calculating the cosine distance between the image characteristics of the image to be inquired and the image characteristics of the initial image corresponding to the first inquiry result to obtain image similarity data;

step 5, combining the first query result and the image similarity data, and classifying the first query result through a feature classifier to obtain a classification result;

step 6, processing the classification result by using a decision tree, and preferably selecting a plurality of items from the classification result as a second query result;

step 7, returning the initial image corresponding to the second query result to the user as a final query result;

the image library is a data set of the images to be put in storage;

the step 5 comprises the following steps:

step 5-2, calculating difference distance of the normalization result;

step 5-4, taking the first two types of data in the division result as classification results;

calculating the difference distance by taking each item of data after normalization as source data and comparing the front and back in the ordered set to obtain the difference distance between each item, wherein the calculation formula is as follows: d = (next-this)/(Last-First), where next represents the next data item, this represents the data item, Last represents the Last data value in the sorted set, and First represents the First data value in the sorted set;

2. The method as claimed in claim 1, wherein the step 1 and the step 2 extract the coding features of the image by using a difference hash algorithm.

3. The method for querying an image based on feature classification as claimed in claim 1, wherein the image features extracted in step 1 and step 2 comprise:

extracting a neural network model by using the feature vector to process the image to obtain a high-dimensional feature vector;

and then, reducing the dimension of the high-dimension feature vector by using a feature vector dimension reduction neural network model to obtain a low-dimension feature vector which is used as the image feature of the image.

4. The method as claimed in claim 1, wherein the step 3 of comparing the coding features of the image to be queried with the coding features of the initial image, and preferring several items from the comparison as the first query result comprises:

step 3-1, extracting the initial image coding features stored in the database;

5. The image query method based on feature classification as claimed in claim 1, wherein the step 4 of calculating a cosine distance between the image feature of the image to be queried and the image feature of the initial image corresponding to the first query result to obtain image similarity data comprises:

6. The method for querying an image based on feature classification as claimed in claim 1, wherein the step 6 comprises: