CN113515661A - Image retrieval method based on filtering depth convolution characteristics - Google Patents

Image retrieval method based on filtering depth convolution characteristics Download PDF

Info

Publication number
CN113515661A
CN113515661A CN202110805566.1A CN202110805566A CN113515661A CN 113515661 A CN113515661 A CN 113515661A CN 202110805566 A CN202110805566 A CN 202110805566A CN 113515661 A CN113515661 A CN 113515661A
Authority
CN
China
Prior art keywords
image
depth convolution
characteristic
data image
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110805566.1A
Other languages
Chinese (zh)
Other versions
CN113515661B (en
Inventor
张伯健
卢奋
刘广海
孔令杰
陆周
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangxi Normal University
Original Assignee
Guangxi Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangxi Normal University filed Critical Guangxi Normal University
Priority to CN202110805566.1A priority Critical patent/CN113515661B/en
Publication of CN113515661A publication Critical patent/CN113515661A/en
Application granted granted Critical
Publication of CN113515661B publication Critical patent/CN113515661B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/53—Querying
    • G06F16/538—Presentation of query results
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Databases & Information Systems (AREA)
  • Library & Information Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an image retrieval method based on filtering depth convolution characteristics, which comprises the steps of firstly inputting an image of a data set into a pre-trained depth convolution neural network model and extracting depth convolution characteristics; secondly, filtering the depth convolution characteristics to remove background noise; then, designing a space weight for enhancing the response of the target object; secondly, enhancing the channel by using the channel weight to generate a representation vector of the image; and finally, normalizing and reducing the dimension of the expression vector of the image to obtain a final feature vector for similarity matching, thereby returning the result of image retrieval. The method simulates the depth convolution characteristic of the image obtained by the depth convolution neural network model to execute the image retrieval task, and the generated image representation can effectively describe the target object in the image and can improve the image retrieval accuracy.

Description

Image retrieval method based on filtering depth convolution characteristics
Technical Field
The invention relates to the technical field of image retrieval, in particular to an image retrieval method based on filtering depth convolution characteristics.
Background
With the rapid development and wide application of network communication technology, people prefer to share their daily lives through the internet, and thus a large amount of image data is uploaded to the network. Image data on networks has exhibited explosive growth. Meanwhile, the internet is an effective technical means for acquiring information by human beings, and in the face of massive images, how to query the images required by the human beings faces huge difficulties and challenges. Early image retrieval techniques were text-based, which required labeling of each image, which varied with human subjectivity, and at the same time, a single text did not effectively represent the content of the image, which limited the development of text-based image retrieval techniques. Accordingly, content-based image retrieval techniques have begun to emerge. In early developments of image retrieval, researchers often represented image content using global features such as color and texture. However, under certain lighting, occlusion, and deformation conditions, global features perform poorly, and are therefore difficult to apply to image retrieval tasks in certain scenes. In recent years, the application of deep convolutional neural networks to the field of image retrieval shows very excellent performance. The method mainly comprises the steps of extracting depth convolution characteristics from a depth convolution neural network, forming an image characteristic representation vector with differentiability by aggregating the depth convolution characteristics, carrying out characteristic matching by using the characteristic vector and returning the most similar image. Image retrieval technology based on a deep convolutional neural network has been a hotspot of current research, however, since background noise in deep convolutional features affects retrieval results, how to construct a more distinguishable image representation by using the deep convolutional features is a main difficulty and challenge facing currently.
Disclosure of Invention
The invention provides an image retrieval method based on a filtering depth convolution characteristic, aiming at the problem that the background noise in the depth convolution characteristic influences the retrieval result.
In order to solve the problems, the invention is realized by the following technical scheme:
an image retrieval method based on filtering depth convolution characteristics comprises the following steps:
step 1, inputting each data image in a data set into a deep convolution neural network model respectively, and extracting N deep convolution feature maps X of each data imagemn(p, q); wherein each depth convolution feature map corresponds to a channel;
step 2, calculating a filter graph F of each data imagem(p,q)；
Step 2.1, calculating the variance E of N depth convolution characteristics of each data imagemn；
2.2, selecting k depth convolution feature maps with larger variance from the N depth convolution feature maps of each data image as a filtering selected depth convolution feature map of each data image;
step 2.3, adding the characteristic values of pixel points at the same positions of the k filtering selected depth convolution characteristic graphs of each data image to obtain a superposition depth convolution characteristic graph of the data image; dividing the characteristic value of each pixel point of the superimposed depth convolution characteristic graph by k to obtain a filter graph F of each data imagem(p,q)；
Step 3, for each data image, filtering picture F of the data image obtained in the step 2m(p, q) and the N depth convolution feature maps X of the data image obtained in the step 1mn(p, q) carrying out dot multiplication to obtain N filtering depth convolution characteristic maps X 'of each data image'mn(p,q)；
Step 4, calculating the space weight graph S of each data imagem(p,q)；
Step 4.1, convolving N filtering depth of each data image into a feature map X'mnAdding the characteristic values of all pixel points of (p, q) to obtain the comprehensive characteristic value h of N filtering depth convolution characteristic graphs of each data imagemn；
Step 4.2, adding the comprehensive characteristic values of the filtering depth convolution characteristics of the corresponding channels of all the data images to obtain N channel characteristic values h'n；
Step 4.3, firstly, carrying out comparison on N channel characteristic values h'nSorting and recording a channel characteristic value h'nThe serial number of the larger first b channel characteristic values is used as the serial number of the selected channel; respectively selecting a depth convolution feature map corresponding to the selected channel serial number from the N depth convolution feature maps of each data image as a space selected depth convolution feature map of each data image;
step 4.4, carrying out square addition on the feature values of the pixel points at the same position on the b space selection depth convolution feature maps of each data image to obtain a space superposition depth convolution feature map S 'of each data image'm(p,q)；
Step 4.5, superposing the space of each data image with a depth convolution characteristic map S'm(p, q) normalizing to obtain a spatial weight map S of each data imagem(p,q)；
Step 5, for each data image, the space weight map S of the data image obtained in the step 4m(p, q) and the N depth convolution feature maps X of the data image obtained in the step 1mn(p, q) performing dot multiplication to obtain N space weighted depth convolution characteristic maps X' of each data imagemn(p,q)；
Step 6, channel weight values P of N space weighted depth convolution characteristic graphs of each data imagemnAnd the integrated characteristic value phimnMultiplying to obtain N channel weighted depth convolution characteristic values G of each data imagemn；
Step 7, weighting depth convolution characteristic value G of N channels of each data imagemnL2 standardization and PCA whitening dimensionality reduction processing are carried out to obtain N characteristic representations G 'of each data image'mnAnd using N per data imageIs characterized by being G'mnConstructing a feature representation vector G 'of each data image'm；
Step 8, inputting the image to be retrieved into the depth convolution neural network model, and extracting N depth convolution characteristic graphs X of the image to be retrieved*n(p,q)；
Step 9, firstly, performing square addition of feature values of pixel points at the same position on b depth convolution feature maps of the image to be retrieved corresponding to the selected channel serial number obtained in the process of calculating the spatial weight map in the step 4.3 to obtain a spatial superposition depth convolution feature map of the image to be retrieved; then, normalizing the spatial superposition depth convolution characteristic graph of the image to be retrieved to obtain a spatial weight graph S of the image to be retrieved*(p,q)；
Step 10, a space weight graph S of the image to be retrieved*(p, q) N depth convolution feature maps X with the image to be retrieved*n(p, q) performing dot multiplication to obtain N space weighted depth convolution characteristic graphs X' of the image to be retrieved*n(p,q)；
Step 11, channel weight values P of N space weighted depth convolution characteristic graphs of the image to be retrieved*NAnd the integrated characteristic value phi*nMultiplying to obtain N channel weighted depth convolution characteristic values G of the image to be retrieved*n；
Step 12, weighting depth convolution characteristic value G of N channels of image to be retrieved*nL2 standardization and PCA whitening dimensionality reduction processing are carried out to obtain N characteristic representations G 'of the image to be retrieved'*nAnd representing G 'by using N features of the image to be retrieved'mnConstructing a feature representation vector G 'of an image to be retrieved'*；
Step 13, calculating a feature representation vector G 'of the image to be retrieved'*And a feature representation vector G 'of each data image in the data set'mL2, and returning the final retrieval result in the order of the distance from small to large;
where M is 1, 2., M denotes the number of data images in the data set, N is 1, 2., N denotes the number of channels, p is 1, 2., W denotes the height of the depth convolution feature map, q is 1, 2., H denotes the height of the depth convolution feature map, k, b are set values, and e is a set constant.
In the above step 6, the nth channel weight value P of the mth data imagemnComprises the following steps:
Figure BDA0003166396070000031
where M1, 2., M denotes the number of data images in the data set, N1, 2., N denotes the number of depth convolution feature maps, p 1, 2., W denotes the height of the depth convolution feature maps, q 1, 2., H denotes the height of the depth convolution feature maps, Z denotes the height of the depth convolution feature maps, and M denotes the number of data images in the data setmnA non-zero ratio, beta, of characteristic values of pixels of an nth spatially weighted depth convolution characteristic map representing an mth data imagemnAnd the response intensity value of the nth space weighted depth convolution characteristic map which represents the mth data image is a set constant value.
In the above step 6, the nth overall characteristic value Φ of the mth data imagemnComprises the following steps:
Figure BDA0003166396070000032
where M1, 2., M denotes the number of data images in the data set, N1, 2., N denotes the number of depth convolution feature maps, p 1, 2., W denotes the height of the depth convolution feature maps, q 1, 2., H denotes the height of the depth convolution feature maps, X ″, andmn(p, q) represents the nth spatially weighted depth convolution feature map of the mth data image.
In the above step 11, the nth channel weight value P of the image to be retrieved*nComprises the following steps:
Figure BDA0003166396070000033
where N1, 2., N denotes the number of deep convolution feature maps, and p 1, 2., W denotes deepHeight of degree convolution feature map, q 1, 2., H denotes height of depth convolution feature map, Z*nNon-zero ratio, beta, of characteristic values of pixels of an nth spatially weighted depth convolution characteristic map representing an image to be retrieved*nAnd the response intensity value of the nth space weighted depth convolution characteristic graph representing the image to be retrieved is a set constant.
In the above step 11, the nth comprehensive characteristic value Φ of the image to be retrieved*nComprises the following steps:
Figure BDA0003166396070000041
where N ═ 1, 2., N denote the number of deep convolution feature maps, p ═ 1, 2., W denote the height of the deep convolution feature maps, q ═ 1, 2., H denote the height of the deep convolution feature maps, X ″, and*nand (p, q) represents the nth spatial weighted depth convolution characteristic map of the image to be retrieved.
Compared with the prior art, the invention provides an algorithm for filtering the depth convolution characteristics on the basis of the visual application based on the depth convolution characteristics of the image to obtain a new image representation vector based on the depth convolution characteristics, which can remarkably describe a target object in the image, can effectively inhibit background noise in the depth convolution characteristics, can form distinguishable image representation, and the experimental result proves that the image retrieval accuracy can be effectively improved by the method.
Drawings
FIG. 1 is a flow chart of an image retrieval method based on a filtered depth convolution feature.
Fig. 2 is an exemplary diagram of a filter map for calculating a depth convolution feature map for each data image.
FIG. 3 is an exemplary diagram of computing a spatial weight map.
Fig. 4 is an exemplary diagram of computing channel weighted depth convolution eigenvalues.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to specific examples.
The invention provides an image retrieval method based on filtering depth convolution characteristics, as shown in figure 1, firstly, inputting an image of a data set into a pre-trained depth convolution neural network model, and extracting a depth convolution characteristic graph; secondly, filtering the depth convolution characteristics to remove background noise; then, designing a space weight for enhancing the response of the target object; secondly, enhancing the channel by using the channel weight to generate a representation vector of the image; and finally, normalizing and reducing the dimension of the expression vector of the image to obtain a final feature vector for similarity matching, thereby returning the result of image retrieval. The method simulates the depth convolution neural network model to acquire the depth convolution characteristics of the image to execute the image retrieval task, the generated image representation can effectively describe a target object in the image, the image retrieval accuracy can be improved, higher weights are respectively given to the characteristic graphs comprising the key semantic information through the space weight and the channel weight so as to improve the distinguishable performance of the image, and the method comprises the following specific steps:
1) obtaining feature representation vectors for data maps in a dataset
Step 1, inputting each data image in a data set into a deep convolution neural network model respectively, and extracting a deep convolution characteristic map of each data image.
Inputting an image in the image data set into the deep convolution neural network model, and extracting a deep convolution characteristic image X of the imagemn(p, q), where M is 1, 2., M denotes the number of data images in the dataset, N is 1, 2., N denotes the number of depth convolution feature maps (also the number of value channels, since each channel corresponds to one depth convolution feature map) per data image, p is 1, 2., W denotes the width of the depth convolution feature maps, and q is 1, 2., H denotes the height of the depth convolution feature maps.
And 2, calculating a filter map of the depth convolution characteristic map of each data image, as shown in FIG. 2.
Step 2.1, calculating the variance of N depth convolution characteristic images of each data imageEmn：
Figure BDA0003166396070000051
In the formula, symbol
Figure BDA0003166396070000059
Indicating averaging.
Figure BDA0003166396070000052
Represents XmnThe mean value of the (p, q) depth convolution characteristic map refers to the depth convolution characteristic map XmnAnd (q, p) adding the characteristic values of all the pixel points and dividing the sum by the number of the pixel points.
Figure BDA0003166396070000053
Representing a deep convolution feature map XmnAnd (5) sequentially subtracting the mean value from the characteristic value of each pixel point in the (p, q).
And 2.2, selecting k depth convolution feature maps with larger variance from the N depth convolution feature maps of each data image as the selected filtering depth convolution feature map of each data image.
Step 2.3, adding the characteristic values of pixel points at the same positions of the k selected filtering depth convolution characteristic graphs of each data image to obtain a superposition depth convolution characteristic graph of the data image; dividing the characteristic value of each pixel point of the superimposed depth convolution characteristic graph by k to obtain a filter graph F of the data imagem(p,q)：
Figure BDA0003166396070000054
Step 3, for each data image, utilizing the filter image F of the data image obtained in the step 2m(p, q) are respectively convolved with the N depth convolution feature maps X of the data image obtained in the step 1mn(p, q) carrying out dot multiplication to obtain N filtering depth convolution characteristic maps X 'of each data image'mn(p,q)：
Figure BDA0003166396070000055
In the formula, symbol
Figure BDA0003166396070000056
Representing element dot multiplication, dot multiplication referring to the filter graph Fm(p, q) are respectively convolved with N depth convolution feature maps Xmn(p, q) multiplying the characteristic values of the pixel points at the same position.
And 4, calculating a space weight map as shown in FIG. 3.
Step 4.1, convolving N filtering depth of each data image into a feature map X'mnAdding the characteristic values of all pixel points of (p, q) to obtain the comprehensive characteristic value h of N filtering depth convolution characteristic graphs of each data imagemn：
Figure BDA0003166396070000057
Step 4.2, adding the comprehensive characteristic values of the filtering depth convolution characteristic graphs of the corresponding channels of all the data images to obtain N channel characteristic values h'n：
Figure BDA0003166396070000058
Step 4.3, firstly, starting from h'nAnd selecting larger b channel characteristic values from every N channel characteristic values, and recording channel serial numbers with the larger b channel characteristic values, wherein the channel serial numbers are used as the uniform selected channel serial numbers of all the data images (namely, each data image selects the depth convolution characteristic images on the channels corresponding to the serial numbers as the space selected depth convolution characteristic images). And then using the depth convolution feature maps corresponding to the selected channel serial numbers of each data image as the space selected depth convolution feature map of each data image, wherein the selected depth convolution feature maps are used for constructing each data imageThe spatial weight map of (1).
And 4.4, performing square superposition on the b selected depth convolution feature maps of each data image to obtain a spatial superposition depth convolution feature map S 'of each data image'm(p,q)；
Figure BDA0003166396070000061
Step 4.5, superposing the space of each data image with a depth convolution characteristic map S'm(p, q) normalizing to obtain a spatial weight map S of each data imagem(p,q)。
Step 5, for each data image, utilizing the space weight map S of the data image obtained in step 4m(p, q) are respectively convolved with the N depth convolution feature maps X of the data image obtained in the step 1mn(p, q) performing dot multiplication to obtain N space weighted depth convolution characteristic maps X' of each data imagemn(p,q)：
Figure BDA0003166396070000062
Step 6, channel weight values P of N space weighted depth convolution characteristic graphs of each data imagemnAnd the integrated characteristic value phimnMultiplying to obtain N channel weighted depth convolution characteristic values G of each data imagenmAs shown in fig. 4.
Channel weight value Pmn：
Figure BDA0003166396070000063
Wherein Z ismnNon-zero fraction (i.e., X ″) of feature values representing pixels of a spatially weighted depth convolution feature mapmnAnd (p, q) the ratio of the number of the characteristic values of the pixel points in the (p, q) to the total number of the pixel points is greater than zero). Beta is amnRepresenting response intensity values of the spatially weighted depth convolved feature map,
Figure BDA0003166396070000064
epsilon is a small constant set to 0.0001, which is intended to ensure that the denominator and numerator are not 0.
The comprehensive characteristic value is obtained by adding the characteristic values of all pixel points of the space weighted depth convolution characteristic graph, wherein the comprehensive characteristic value phimnComprises the following steps:
Figure BDA0003166396070000065
where M1, 2., M denotes the number of data images in the data set, N1, 2., N denotes the number of depth convolution feature maps, p 1, 2., W denotes the height of the depth convolution feature maps, q 1, 2., H denotes the height of the depth convolution feature maps, X ″, andmn(p, q) represents a spatially weighted depth convolution signature.
Then N channel weighted depth convolution eigenvalues G of each data imagemn：
Gmn＝Pmn×Φmn
Step 7, weighting depth convolution characteristic value G of N channels of each data imagemnL2 standardization and PCA whitening dimensionality reduction processing are carried out to obtain N characteristic representations G 'of each data image'mnAnd G 'is expressed by N features of each data image'mnConstructing a feature representation vector G 'of each data image'm＝{G′mn，n＝1,2,...,N}。
2) Obtaining a feature representation vector of an image to be retrieved
Step 8, inputting the image to be retrieved into the depth convolution neural network model, and extracting N depth convolution characteristic graphs X of the image to be retrieved*n(p,q)。
Step 9, firstly, performing square addition of feature values of corresponding pixel points on b depth convolution feature maps of the image to be retrieved corresponding to the selected channel serial number obtained in the process of calculating the space weight map to obtain a space superposition depth convolution feature map S 'of the image to be retrieved'*(p,q)(ii) a And then, overlaying a depth convolution feature map S 'in space of the image to be retrieved'*(p, q) normalizing to obtain a space weight map S of the image to be retrieved*(p,q)。
Step 10, a space weight graph S of the image to be retrieved*(p, q) N depth convolution feature maps X with the image to be retrieved*n(p, q) performing dot multiplication to obtain N space weighted depth convolution characteristic graphs X' of the image to be retrieved*n(P,Q)。
Step 11, channel weight values P of N space weighted depth convolution characteristic graphs of the image to be retrieved*nAnd the integrated characteristic value phi*nMultiplying to obtain N channel weighted depth convolution characteristic values G of the image to be retrieved*n。
Step 12, weighting depth convolution characteristic value G of N channels of image to be retrieved*nL2 standardization and PCA whitening dimensionality reduction processing are carried out to obtain N characteristic representations G 'of the image to be retrieved'*nAnd representing G 'by using N features of the image to be retrieved'*nConstructing a feature representation vector G 'of an image to be retrieved'*＝{G′*n，n＝1,2,...,N}。
3) Retrieval by feature representation vectors
Step 13, calculating a feature representation vector G 'of the image to be retrieved'*And a feature representation vector G 'of each data image in the data set'mL2, smaller distance indicates more similar images, and the final search results are returned in order of smaller distance to larger distance.
The invention provides an image retrieval method based on a filtering depth convolution characteristic, which aims to eliminate background noise and highlight a target object in a deep convolution characteristic, provides a novel method for weighting a space and a channel of the depth convolution characteristic, and applies the method to image retrieval. We propose a new image retrieval framework. The key technology mainly comprises a filter, a space weight and a channel weight. When we focus on the channel, the filter can eliminate the noise interference, which is beneficial to select more distinctive channels. Spatial weighting can enhance the response of target objects in a spatial location, playing a role in highlighting key features, further suppressing background noise. Channel weights may enhance certain channels that do not respond prominently but contain critical functions, and may also play a role in suppressing visual abruptness. Experimental results show that the method can improve the performance of image retrieval.
It should be noted that, although the above-mentioned embodiments of the present invention are illustrative, the present invention is not limited thereto, and thus the present invention is not limited to the above-mentioned embodiments. Other embodiments, which can be made by those skilled in the art in light of the teachings of the present invention, are considered to be within the scope of the present invention without departing from its principles.

Claims (5)

1. An image retrieval method based on filtering depth convolution characteristics is characterized by comprising the following steps:
step 1, inputting each data image in a data set into a deep convolution neural network model respectively, and extracting N deep convolution feature maps X of each data imagemn(p, q); wherein each depth convolution feature map corresponds to a channel;
step 2, calculating a filter graph F of each data imagem(p，q)；
Step 2.1, calculating the variance E of N depth convolution characteristics of each data imagemn；
2.2, selecting k depth convolution feature maps with larger variance from the N depth convolution feature maps of each data image as a filtering selected depth convolution feature map of each data image;
step 2.3, adding the characteristic values of pixel points at the same positions of the k filtering selected depth convolution characteristic graphs of each data image to obtain a superposition depth convolution characteristic graph of the data image; dividing the characteristic value of each pixel point of the superimposed depth convolution characteristic graph by k to obtain a filter graph F of each data imagem(p，q)；
Step 3, for each data image, filtering picture F of the data image obtained in the step 2m(p, q) and N of the data image obtained in step 1Depth convolution feature map Xmn(p, q) carrying out dot multiplication to obtain N filtering depth convolution characteristic maps X 'of each data image'mn(p，q)；
Step 4, calculating the space weight graph S of each data imagem(p，q)；
Step 4.1, convolving N filtering depth of each data image into a feature map X'mnAdding the characteristic values of all pixel points of (p, q) to obtain the comprehensive characteristic value h of N filtering depth convolution characteristic graphs of each data imagemn；
Step 4.2, adding the comprehensive characteristic values of the filtering depth convolution characteristics of the corresponding channels of all the data images to obtain N channel characteristic values h'n；
Step 4.3, firstly, carrying out comparison on N channel characteristic values h'nSorting and recording a channel characteristic value h'nThe serial number of the larger first b channel characteristic values is used as the serial number of the selected channel; respectively selecting a depth convolution feature map corresponding to the selected channel serial number from the N depth convolution feature maps of each data image as a space selected depth convolution feature map of each data image;
step 4.4, carrying out square addition on the feature values of the pixel points at the same position on the b space selection depth convolution feature maps of each data image to obtain a space superposition depth convolution feature map S 'of each data image'm(p，q)；
Step 4.5, superposing the space of each data image with a depth convolution characteristic map S'm(p, q) normalizing to obtain a spatial weight map S of each data imagem(p，q)；
Step 5, for each data image, the space weight map S of the data image obtained in the step 4m(p, q) and the N depth convolution feature maps X of the data image obtained in the step 1mn(p, q) performing dot multiplication to obtain N space weighted depth convolution characteristic maps X' of each data imagemn(p，q)；
Step 6, channel weight values P of N space weighted depth convolution characteristic graphs of each data imagemnAnd the integrated characteristic value phimnMultiplying to obtain N channel weighted depth convolution characteristic values G of each data imagemn；
Step 7, weighting depth convolution characteristic value G of N channels of each data imagemnL2 standardization and PCA whitening dimensionality reduction processing are carried out to obtain N characteristic representations G 'of each data image'mnAnd G 'is expressed by N features of each data image'mnConstructing a feature representation vector G 'of each data image'm；
Step 8, inputting the image to be retrieved into the depth convolution neural network model, and extracting N depth convolution characteristic graphs X of the image to be retrieved*n(p，q)；
Step 9, firstly, performing square addition of feature values of pixel points at the same position on b depth convolution feature maps of the image to be retrieved corresponding to the selected channel serial number obtained in the process of calculating the spatial weight map in the step 4.3 to obtain a spatial superposition depth convolution feature map of the image to be retrieved; then, normalizing the spatial superposition depth convolution characteristic graph of the image to be retrieved to obtain a spatial weight graph S of the image to be retrieved*(p，q)；
Step 10, a space weight graph S of the image to be retrieved*(p, q) N depth convolution feature maps X with the image to be retrieved*n(p, q) performing dot multiplication to obtain N space weighted depth convolution characteristic graphs X' of the image to be retrieved*n(p，q)；
Step 11, channel weight values P of N space weighted depth convolution characteristic graphs of the image to be retrieved*nAnd the integrated characteristic value phi*nMultiplying to obtain N channel weighted depth convolution characteristic values G of the image to be retrieved*n；
Step 12, weighting depth convolution characteristic value G of N channels of image to be retrieved*nL2 standardization and PCA whitening dimensionality reduction processing are carried out to obtain N characteristic representations G 'of the image to be retrieved'*nAnd representing G 'by using N features of the image to be retrieved'mnConstructing a feature representation vector G 'of an image to be retrieved'*；
Step 13 and step 12 are used for calculating a feature representation vector G 'of the image to be retrieved'*And the feature expression vector G 'of each data image in the data set obtained in the step 7'mL2, and returning the final retrieval result in the order of the distance from small to large;
where M is 1, 2., M denotes the number of data images in the data set, N is 1, 2., N denotes the number of channels, p is 1, 2., W denotes the height of the depth convolution feature map, q is 1, 2., H denotes the height of the depth convolution feature map, k, b are set values, and e is a set constant.
2. The method as claimed in claim 1, wherein in step 6, the nth channel weight value P of the mth data imagemnComprises the following steps:
Figure FDA0003166396060000021
where M1, 2., M denotes the number of data images in the data set, N1, 2., N denotes the number of depth convolution feature maps, p 1, 2., W denotes the height of the depth convolution feature maps, q 1, 2., H denotes the height of the depth convolution feature maps, Z denotes the height of the depth convolution feature maps, and M denotes the number of data images in the data setmnA non-zero ratio, beta, of characteristic values of pixels of an nth spatially weighted depth convolution characteristic map representing an mth data imagemnAnd the response intensity value of the nth space weighted depth convolution characteristic map which represents the mth data image is a set constant value.
3. The method as claimed in claim 1, wherein in step 6, the nth integrated feature value Φ of the mth data image ismnComprises the following steps:
Figure FDA0003166396060000031
where M1, 2., M denotes the number of data images in the data set, N1, 2., N denotes the number of depth convolution feature maps, p 1, 2., W denotes the height of the depth convolution feature maps, q 1, 2., H denotes the height of the depth convolution feature maps, X ″, andmn(p, q) represents the nth spatially weighted depth convolution feature map of the mth data image.
4. The image retrieval method based on the filter depth convolution characteristic as claimed in claim 1, wherein in step 11, the nth channel weight value P of the image to be retrieved*nComprises the following steps:
Figure FDA0003166396060000032
wherein N is 1,2, the., N represents the number of depth convolution feature maps, p is 1,2, the., W represents the height of the depth convolution feature maps, q is 1,2, the., H represents the height of the depth convolution feature maps, Z represents the height of the depth convolution feature maps, and N represents the number of depth convolution feature maps*nNon-zero ratio, beta, of characteristic values of pixels of an nth spatially weighted depth convolution characteristic map representing an image to be retrieved*nAnd the response intensity value of the nth space weighted depth convolution characteristic graph representing the image to be retrieved is a set constant.
5. The image retrieval method based on the filter depth convolution characteristic as claimed in claim 1, wherein in step 11, the nth comprehensive characteristic value Φ of the image to be retrieved*nComprises the following steps:
Figure FDA0003166396060000033
where N ═ 1, 2., N denote the number of deep convolution feature maps, p ═ 1, 2., W denote the height of the deep convolution feature maps, q ═ 1, 2., H denote the height of the deep convolution feature maps, X ″, and*n(p, q) indicates suspectThe nth spatially weighted depth convolution feature map of the cable image.
CN202110805566.1A 2021-07-16 2021-07-16 Image retrieval method based on filtering depth convolution characteristics Active CN113515661B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110805566.1A CN113515661B (en) 2021-07-16 2021-07-16 Image retrieval method based on filtering depth convolution characteristics

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110805566.1A CN113515661B (en) 2021-07-16 2021-07-16 Image retrieval method based on filtering depth convolution characteristics

Publications (2)

Publication Number Publication Date
CN113515661A true CN113515661A (en) 2021-10-19
CN113515661B CN113515661B (en) 2022-03-11

Family

ID=78067814

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110805566.1A Active CN113515661B (en) 2021-07-16 2021-07-16 Image retrieval method based on filtering depth convolution characteristics

Country Status (1)

Country Link
CN (1) CN113515661B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117576105A (en) * 2024-01-17 2024-02-20 高科建材（咸阳）管道科技有限公司 Pipeline production control method and system based on artificial intelligence

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106156748A (en) * 2016-07-22 2016-11-23 浙江零跑科技有限公司 Traffic scene participant's recognition methods based on vehicle-mounted binocular camera
CN107577758A (en) * 2017-08-31 2018-01-12 桂林电子科技大学 A kind of generation method for the image convolution feature for intersecting weights based on multizone
CN109858496A (en) * 2019-01-17 2019-06-07 广东工业大学 A kind of image characteristic extracting method based on weighting depth characteristic
CN110297931A (en) * 2019-04-23 2019-10-01 西北大学 A kind of image search method
US20210089807A1 (en) * 2019-09-25 2021-03-25 Samsung Electronics Co., Ltd. System and method for boundary aware semantic segmentation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106156748A (en) * 2016-07-22 2016-11-23 浙江零跑科技有限公司 Traffic scene participant's recognition methods based on vehicle-mounted binocular camera
CN107577758A (en) * 2017-08-31 2018-01-12 桂林电子科技大学 A kind of generation method for the image convolution feature for intersecting weights based on multizone
CN109858496A (en) * 2019-01-17 2019-06-07 广东工业大学 A kind of image characteristic extracting method based on weighting depth characteristic
CN110297931A (en) * 2019-04-23 2019-10-01 西北大学 A kind of image search method
US20210089807A1 (en) * 2019-09-25 2021-03-25 Samsung Electronics Co., Ltd. System and method for boundary aware semantic segmentation

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
YU SHEN ET AL.: "A Spectral-Spatial Domain-Specific Convolutional Deep Extreme Learning Machine for Supervised Hyperspectral Image Classification", 《IEEE ACCESS》 *
李振东 等: "基于深度特征的快速人脸图像检索方法", 《光学学报》 *
袁晖: "基于特征性能增强和目标定位的图像检索研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117576105A (en) * 2024-01-17 2024-02-20 高科建材（咸阳）管道科技有限公司 Pipeline production control method and system based on artificial intelligence
CN117576105B (en) * 2024-01-17 2024-03-29 高科建材（咸阳）管道科技有限公司 Pipeline production control method and system based on artificial intelligence

Also Published As

Publication number Publication date
CN113515661B (en) 2022-03-11

Similar Documents

Publication Publication Date Title
CN111695467B (en) Spatial spectrum full convolution hyperspectral image classification method based on super-pixel sample expansion
CN110866896B (en) Image saliency target detection method based on k-means and level set super-pixel segmentation
CN109086405B (en) Remote sensing image retrieval method and system based on significance and convolutional neural network
CN111126385A (en) Deep learning intelligent identification method for deformable living body small target
CN113870124B (en) Weak supervision-based double-network mutual excitation learning shadow removing method
CN116612288B (en) Multi-scale lightweight real-time semantic segmentation method and system
CN109215003B (en) Image fusion method and device
CN111667019A (en) Hyperspectral image classification method based on deformable separation convolution
CN113392244A (en) Three-dimensional model retrieval method and system based on depth measurement learning
Liu et al. Research on super-resolution reconstruction of remote sensing images: A comprehensive review
CN114510594A (en) Traditional pattern subgraph retrieval method based on self-attention mechanism
CN113515661B (en) Image retrieval method based on filtering depth convolution characteristics
CN115861076A (en) Unsupervised hyperspectral image super-resolution method based on matrix decomposition network
CN117576402B (en) Deep learning-based multi-scale aggregation transducer remote sensing image semantic segmentation method
CN112329818B (en) Hyperspectral image non-supervision classification method based on graph convolution network embedded characterization
CN114037770A (en) Discrete Fourier transform-based attention mechanism image generation method
Jing et al. Image generation step by step: animation generation-image translation
CN116543021A (en) Siamese network video single-target tracking method based on feature fusion
Tu et al. Texture pattern separation for hyperspectral image classification
CN117011655A (en) Adaptive region selection feature fusion based method, target tracking method and system
CN115311550B (en) Remote sensing image semantic change detection method and device, electronic equipment and storage medium
CN113780305B (en) Significance target detection method based on interaction of two clues
CN115661754A (en) Pedestrian re-identification method based on dimension fusion attention
CN114202694A (en) Small sample remote sensing scene image classification method based on manifold mixed interpolation and contrast learning
Zhang et al. Deep photographic style transfer guided by semantic correspondence

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant