CN117218720B - Footprint identification method, system and related device of composite attention mechanism - Google Patents
Footprint identification method, system and related device of composite attention mechanism Download PDFInfo
- Publication number
- CN117218720B CN117218720B CN202311089272.9A CN202311089272A CN117218720B CN 117218720 B CN117218720 B CN 117218720B CN 202311089272 A CN202311089272 A CN 202311089272A CN 117218720 B CN117218720 B CN 117218720B
- Authority
- CN
- China
- Prior art keywords
- feature
- footprint
- characteristic
- module
- attention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000007246 mechanism Effects 0.000 title claims abstract description 56
- 239000002131 composite material Substances 0.000 title claims abstract description 34
- 238000000605 extraction Methods 0.000 claims abstract description 33
- 238000012545 processing Methods 0.000 claims abstract description 5
- 230000002087 whitening effect Effects 0.000 claims description 25
- 230000004927 fusion Effects 0.000 claims description 23
- 239000011159 matrix material Substances 0.000 claims description 22
- 238000012549 training Methods 0.000 claims description 17
- 150000001875 compounds Chemical class 0.000 claims description 13
- 238000004891 communication Methods 0.000 claims description 9
- 238000011176 pooling Methods 0.000 claims description 8
- 238000013526 transfer learning Methods 0.000 claims description 8
- 238000001228 spectrum Methods 0.000 claims description 7
- 238000012935 Averaging Methods 0.000 claims 1
- 238000013527 convolutional neural network Methods 0.000 description 17
- 230000009466 transformation Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a footprint identification method, a system and a related device of a composite attention mechanism, wherein the identification method comprises the following steps: s1, acquiring a template footprint image, and establishing a feature database according to the features of the template footprint image; s2, extracting features of the footprint image to be identified by using a convolution layer to obtain basic features; s3, inputting the basic features into a global branch for feature extraction to obtain first features; s4, inputting the basic features into a composite attention mechanism, and processing the basic features through a decoupling enhancement spatial attention module and a grouping channel attention module to obtain second features; s5, splicing the first feature and the second feature to obtain an actual feature; s6, comparing the actual features with a feature database to obtain the recognition result of the footprint image to be recognized.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a footprint identification method, a footprint identification system and a footprint identification related device for a composite attention mechanism.
Background
The footprint recognition task aims to identify the person through the shoe-wearing footprint sequence. However, the single shoe footprint image provides too little information and is susceptible to interference, which results in high difficulty and inadaptability of the task.
To solve this problem, many footprint recognition methods based on deep learning have been proposed and have achieved far beyond the effect of the conventional algorithm, and are currently the mainstream methods. However, these deep learning-based footprint identification methods have the following drawbacks:
(1) In complex crime scenes, most methods have weak ability to extract effective shoe print texture information. In the methods, the whole footprint picture is often used as input information to operate, and when the whole footprint picture is faced with a complex field background, the network is easily interfered for all the input information to be the same, so that effective local texture information of the shoe print cannot be extracted.
(2) When the shoe print image of the crime scene is collected, the collected shoe print image can be subjected to visual angle, light and scale impression. On such a premise, the convolution operation may cause some difference in the features corresponding to the pixels under the same label.
Chinese patent CN111382629a discloses a footprint recognition and information mining method and system based on a neural network, which adopts the technical scheme that the acquired footprint image is preprocessed, and is input into a prediction model for feature extraction, the identity information and the predicted biological information of the footprint image can be obtained through the convolutional neural network, and the input footprint image is convolved by using a convolution template generated by other functions with direction and scale information such as Gabor function.
In the technical scheme, the footprint recognition is to process the whole footprint image and extract the characteristics, when the background of the footprint image is complex, the texture information in the footprint image cannot be effectively extracted, and the interference caused by external factors during the acquisition of the footprint image cannot be overcome.
In view of this, there is an urgent need to improve the existing visual target tracking algorithm, and propose a footprint recognition algorithm capable of overcoming the interference of various environmental factors and extracting the texture information of the shoe print with identification degree.
Disclosure of Invention
In view of this, the invention provides a footprint recognition method of a composite attention mechanism, which adds a grouping channel attention mechanism module into a composite attention mechanism module to enhance the association degree of channel attention to local information, and simultaneously uses hole convolution to extract global information, thereby improving the accuracy of extracting the characteristics of footprint images to be recognized.
The technical scheme of the invention is realized as follows:
in a first aspect, the present invention provides a method for identifying a footprint of a composite attention mechanism, the method comprising the steps of:
s1, acquiring a template footprint image, and establishing a feature database according to the features of the template footprint image;
s2, footprint image I to be identified 0 Extracting features by using N convolution layers to obtain basic features F 4 The method comprises the steps of carrying out a first treatment on the surface of the Wherein N convolution layers are sequentially arranged, N is a non-zero natural number;
s3, the basic characteristic F 4 Inputting to the (n+1) th convolution layer for global feature extraction to obtain a first feature C 1 ；
S4, the basic characteristic F 4 Inputting the second characteristic C into a composite attention mechanism, and extracting local characteristics by decoupling the space attention enhancement module and the group channel attention module to obtain the second characteristic C 2 The method comprises the steps of carrying out a first treatment on the surface of the Wherein the composite attention mechanism comprises a decoupling enhancement spatial attention module and a grouping channel attention module;
s5, the first characteristic C 1 And second feature C 2 Splicing to obtain the actual characteristic C f ；
S6, the actual characteristic C f Comparing with a characteristic database to obtain the footprint image I to be identified 0 Is a result of the recognition of (a).
On the basis of the above technical solution, preferably, the compound attention mechanism further includes an n+2th convolution layer and a spectrum value difference orthogonality regularization layer, and the step S4 specifically includes:
s41, the basic characteristic F 4 Inputting into the (n+2) th convolution layer for feature extraction to obtain convolution feature F j ；
S43, inputting the third feature S to a decoupling enhancement spatial attention module for feature extraction to obtain a fourth feature D;
s44, inputting the third feature S to a grouping channel attention module for feature extraction to obtain a fifth feature G;
s45, fusing the fourth feature D and the fifth feature G to obtain a first fused feature
S46, integrating the first fusion characteristicGlobal average pooling is performed to obtain the second feature C 2 。
On the basis of the above technical solution, preferably, the step S44 specifically includes:
dividing the third features S into a plurality of groups according to the number of channels to obtain a plurality of first feature groups Z;
sequentially inputting the plurality of first feature groups Z into a grouping channel attention module to obtain a plurality of second feature groups Z';
connecting the plurality of second feature groups Z' to obtain a channel attention feature F u ；
Extracting the third feature S through cavity convolution with a receptive field of 5 multiplied by 5 to obtain a cavity convolution feature H;
will be spentChannel attention feature F u Selectively fusing the cavity convolution characteristic H to obtain a fifth characteristic G:
G＝a c H+b c F u ，a c +b c ＝1，
wherein element a and element b represent a hole convolution feature H and a channel attention feature F, respectively u Soft attention vector of element S c Line c, element a, representing the third feature S c C element representing element a, element H c Line c, element b, representing the hole convolution characteristic H c The c-th element, F, representing element b, F is the channel attention feature F u And a selection weight vector generated when the cavity convolution characteristic H is selectively fused.
On the basis of the above technical solution, preferably, the selectively fusing the channel attention feature and the cavity convolution feature specifically includes:
directing the channel attention feature F u Adding the second fusion characteristic with the cavity convolution characteristic H to obtain a second fusion characteristic
Characterizing the second fusionPerforming feature dimension mapping through a global average pooling layer to obtain a selection weight vector f;
the channel attention feature F is selected according to the weight vector F u The fifth feature G is obtained by assigning values with a soft attention mechanism across channels with the hole convolution feature H.
On the basis of the above technical solution, preferably, the identification method further includes:
training by using the characteristics of the template footprint image in the database to obtain a trained footprint recognition model;
when the actual characteristics are different from the database, performing cross-domain learning identification on the characteristics of the footprint image to be identified by adopting transfer learning, and adjusting the footprint identification model;
and identifying the footprint image to be identified by adopting the adjusted footprint identification model to obtain a footprint image identification result.
On the basis of the above technical solution, preferably, the step S43 specifically includes:
extracting the third features S by utilizing an extracting boundary information module to obtain a pixel-by-pixel saliency boundary information matrix A;
extracting the third characteristic S by using a whitening region similarity information extraction module to obtain a whitening region similarity information matrix B;
adding the pixel-by-pixel saliency boundary information matrix A and the whitening region similarity information matrix B to obtain a third fusion feature U;
performing feature processing according to the third feature S and the third fusion feature U, and scaling through a weight coefficient to obtain a sixth feature W;
and adding the characteristic W and the third characteristic S to obtain a fourth characteristic D.
In a second aspect, the present invention provides a footprint recognition system of a compound attention mechanism, and a footprint recognition method adopting the compound attention mechanism as described in any one of the above, including:
the acquisition module is used for acquiring the template footprint image and establishing a characteristic database according to the characteristics of the template image;
the characteristic extraction module is used for extracting characteristics of the footprint image to be identified to obtain actual characteristics;
and the comparison module is used for comparing the actual characteristics of the footprint image to be identified with the characteristic database to obtain the identification result of the footprint image to be identified.
Still further preferred, further comprises a training module and a learning module, wherein,
the training module is used for training according to the characteristics of the template footprint image in the characteristic database to obtain a trained footprint recognition model;
and the learning module is used for performing cross-domain learning identification by adopting transfer learning when the actual characteristics and the database are different, and adjusting the trained footprint identification model.
In a third aspect, the present invention provides an electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus; wherein,
the processor, the memory and the communication interface complete the communication with each other through the bus;
the memory stores program instructions executable by the processor that the processor invokes to implement a footprint identification method of the composite attention mechanism as described in any of the above.
In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions that cause the computer to implement a footprint identification method of a compound attention mechanism as described in any one of the above.
Compared with the prior art, the footprint identification method of the composite attention mechanism has the following beneficial effects:
(1) By setting a composite attention mechanism in the convolutional neural network, attention of local features of the footprint image to be identified is enhanced by using the attention mechanism, and the identification accuracy is improved; the composite attention mechanism comprises a decoupling enhancement spatial attention module and a grouping channel attention module, wherein the sensitivity of correlation information among characteristic channels is improved through the grouping channel attention module, and the expression capability of image characteristics is enhanced;
(2) Decoupling basic features of the footprint image to be identified through a decoupling enhancement spatial attention module to obtain saliency boundary information and whitening region similarity information of the footprint image to be identified, enhancing learning capacity between texture information and region pixels in the footprint image to be identified, reducing computational complexity of a convolutional neural network, and enhancing robustness of the features of the footprint image to be identified;
(3) When the actual characteristics of the footprint images to be identified are different from the comparison of the database, the transfer learning is used for carrying out cross-domain learning identification, the footprint identification model is adjusted, and the neural network is enabled to be more quickly suitable for the footprint images to be identified by utilizing the grouping channel attention module, so that the accuracy rate of footprint identification is further improved, and the performance of the footprint identification model is greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a footprint identification method of the composite attention mechanism of the present invention;
FIG. 2 is a schematic diagram of a footprint recognition method of the composite attention mechanism of the present invention;
FIG. 3 is a schematic diagram of a packet channel attention module of the footprint recognition method of the composite attention mechanism of the present invention;
FIG. 4 is a schematic diagram of a channel attention module architecture of a grouped channel attention module of the footprint recognition method of the composite attention mechanism of the present invention;
FIG. 5 is a schematic diagram of a decoupling enhanced spatial attention module of the footprint recognition method of the composite attention mechanism of the present invention;
FIG. 6 is a block diagram of a footprint recognition system of the composite attention mechanism of the present invention.
Detailed Description
The following description of the embodiments of the present invention will clearly and fully describe the technical aspects of the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, are intended to fall within the scope of the present invention.
As shown in fig. 1, the present invention provides a footprint identification method of a composite attention mechanism, the identification method comprising the steps of:
s1, acquiring a template footprint image, and establishing a feature database according to the features of the template footprint image.
S2, footprint image I to be identified 0 Extracting features by using N convolution layers to obtain basic features F 4 The method comprises the steps of carrying out a first treatment on the surface of the Wherein, N convolution layers are arranged in sequence, N is a non-zero natural number.
In a preferred embodiment of the present application, N is 4, the footprint image I to be identified 0 And sequentially inputting the features into the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer to extract the features.
Specifically, the first convolution layer performs feature extraction by using a convolution kernel of 7×7 to obtain a feature F 1 The method comprises the steps of carrying out a first treatment on the surface of the The second convolution layer adopts 3 groups of convolution kernels of 1 multiplied by 1, 3 multiplied by 3 and 1 multiplied by 1 to extract the characteristics to obtain the characteristics F 2 The method comprises the steps of carrying out a first treatment on the surface of the The third convolution layer adopts 4 groups of convolution kernels of 1×1, 3×3 and 1×1 to extract the features to obtain the features F 3 The method comprises the steps of carrying out a first treatment on the surface of the In the fourth convolution layer, 6 groups of convolution kernels of 1×1, 3×3 and 1×1 are adopted to perform feature extraction to obtain F 4 I.e. the basic features.
S3, the basic characteristic F 4 Inputting to the (n+1) th convolution layer for global feature extraction to obtain a first feature C 1 . I.e. when N is 4, the basic feature F 4 Inputting to fifth convolution layer, and performing feature extraction by 3 groups of convolution kernels of 1×1, 3×3 and 1×1 to obtain feature F 5 Feature F 5 After being input into the global average pooling layer for processing, the global feature, namely the first feature C, is obtained 1 。
S4, the basic characteristic F 4 Inputting the second characteristic C into a composite attention mechanism, and extracting local characteristics by decoupling the space attention enhancement module and the group channel attention module to obtain the second characteristic C 2 The method comprises the steps of carrying out a first treatment on the surface of the Wherein the complexThe attention combining mechanism includes a decoupling enhancement spatial attention module and a grouping channel attention module. For basic characteristics F through a composite attention mechanism 4 Channel aggregation and position sensing are carried out to obtain a second characteristic C 2 。
S5, the first characteristic C 1 And second feature C 2 Splicing to obtain the actual characteristic C f ；
S6, the actual characteristic C f And comparing the footprint images with a feature database to obtain the identification result of the footprint images to be identified.
According to the method, the characteristic extraction is carried out on the footprint image to be identified through the convolution layer, the basic characteristic of the footprint image to be identified is obtained, the characteristic extraction is carried out on the basic characteristic through the composite attention mechanism, the correlation between the characteristics is reduced through the orthogonalization regularization of the characteristic space in the composite attention mechanism, the accuracy of the characteristic extraction is improved, the attention of the local characteristic of the footprint image to be identified is enhanced through the attention mechanism, meanwhile, long-distance dependence is established, and the identification accuracy of the footprint image to be identified is improved.
As shown in fig. 2, as a preferred embodiment of the present application, the composite attention mechanism further includes an n+2th convolution layer and a spectrum value difference orthogonality regularization layer, and the step S4 specifically includes:
s41, the basic characteristic F 4 Inputting into the (n+2) th convolution layer for feature extraction to obtain convolution feature F j ；
S42, the convolution characteristic F j Inputting a spectrum value difference orthogonality regularization layer to obtain a third characteristic S;
s43, inputting the third feature S to a decoupling enhancement spatial attention module for feature extraction to obtain a fourth feature D;
s44, inputting the third feature S to a grouping channel attention module for feature extraction to obtain a fifth feature G;
s45, the fourth characteristic C 4 Fusing the first fusion characteristic with the fifth characteristic G to obtain a first fusion characteristic
S46, integrating the first fusion characteristicGlobal average pooling is performed to obtain the second feature C 2 。
Wherein, when N is 4, the basic feature F is checked in the sixth convolution layer by 3 groups of 1×1, 3×3 and 1×1 convolutions, respectively 4 Extracting features to obtain convolution feature F j The method comprises the steps of carrying out a first treatment on the surface of the Will convolve feature F j Orthogonalization regularization to reduce convolution feature F j And the correlation between the two features is obtained to obtain a third feature S, wherein the orthogonalization regularization (SVDO) method can enable the convolutional neural network to pay more attention to the detailed information in the footprint image to be identified, improve the identification accuracy and avoid the phenomenon of fitting of the convolutional neural network. The third feature S is decoupled through the decoupling enhancement spatial attention module, the learning capacity between texture information and regional pixels in the footprint image to be identified is enhanced, the robustness of the characteristics of the footprint image to be identified is improved, meanwhile, the sensitivity of the convolutional neural network to the related information between characteristic channels is improved through the grouping channel attention module, the expression capacity of the characteristics in the footprint image to be identified is enhanced, and the identification effect of the footprint image to be identified is improved.
As shown in fig. 3, as a preferred embodiment of the present application, the step S44 specifically includes:
dividing the third features S into a plurality of groups according to the number of channels to obtain a plurality of first feature groups Z;
sequentially inputting the plurality of first feature groups Z into a grouping channel attention module to obtain a plurality of second feature groups Z';
connecting the plurality of second feature groups Z' to obtain a channel attention feature F u ；
Extracting the third feature S through cavity convolution with a receptive field of 5 multiplied by 5 to obtain a cavity convolution feature H;
directing the channel attention feature F u Selectively fusing with the cavity convolution characteristic H to obtainFifth feature G.
Specifically, dividing the third feature S into G groups according to the number of channels, wherein the number of feature channels in each group is g=c/G, obtaining a plurality of first feature groups Z with single feature sizes of gxhxw, wherein C is the number of channels in a convolution layer, H is the height of a convolution kernel, and W is the width of the convolution kernel, sequentially inputting the plurality of first feature groups Z into a grouping channel attention module (CAM attention module) to obtain a plurality of features Q with single feature sizes of gxhxw, and sequentially connecting the features Q to obtain a channel attention feature F with single feature sizes of c×h×w u Extracting the third feature S through a cavity convolution with a receptive field of 5×5 to obtain a cavity convolution feature H, and obtaining the attention feature F of each group of channels through the cavity convolution u Correlation between them, channel attention feature F u And fusing the cavity convolution characteristic H to obtain a fifth characteristic G.
With further reference to fig. 4, the CAM attention module specifically includes:
group g features Z of the first feature group Z i ，i∈[1，g]Sequentially inputting into 1×1 convolution to obtain characteristic F with size of KXH×W i Wherein K is the number of convolved channels, H is the height of the convolution kernel, and W is the width of the convolution kernel;
will feature F i Dimension transformation is carried out to obtain a characteristic F with the size of KXN i1 Wherein N is the number of convolution kernels;
feature Z i Performing dimension transformation, and transposing the dimension transformation result to obtain a characteristic P with the size of NxG i ；
Will feature P i And feature F i1 Multiplying and calculating the multiplied result by using a softmax layer to finally obtain a channel attention matrix X with the size of K multiplied by G i ；
Will feature F i Performing dimension transformation again to obtain characteristic F with the size of KXN i2 ；
Matrix X of channel attention i Transpose of (a) and the feature F i2 Multiplying, and performing dimension transformation to obtain characteristic Y with size of GXH×W i ；
Feature Y i And feature Z i Adding elements by element to obtain a characteristic Z' i ；
Characterization of group g Z' i ，i∈[1，g]Sequentially connected, a local attention feature of size C x H x W is obtained, i.e. channel attention feature F u 。
It can be understood that the building channel attention module can model the dependency relationship between different channels, further enhance the relevance between channels with similar semantics, and dynamically adjust the weights of different channels of the convolutional neural network, thereby improving the expressive power of the features.
As a preferred embodiment of the present application, said directing the channel attention feature F u Selectively fusing the cavity convolution characteristic H, which specifically comprises the following steps:
directing the channel attention feature F u Adding the second fusion characteristic with the cavity convolution characteristic H to obtain a second fusion characteristic
Characterizing the second fusionPerforming feature dimension mapping through a global average pooling layer to obtain a selection weight vector f;
the channel attention feature F is selected according to the weight vector F u The fifth feature G is obtained by assigning values with a soft attention mechanism across channels with the hole convolution feature H.
It will be appreciated that channel attention feature F u Adding the second fusion characteristic with the cavity convolution characteristic H to obtain a second fusion characteristicSecond fusion feature->After global average pooling operation, mapping the feature dimension to d dimension to obtain a selection weight direction with the size of d multiplied by 1Quantity F, according to the selected weight vector F, channel attention feature F u Assigning values with the cavity convolution characteristic H by using a cross-channel soft attention mechanism, and fusing global information and local attention information of footprint images to be identified by different weights to obtain grouped channel attention information, namely a fifth characteristic G:
G＝a c H+b c F u ，a c +b c ＝1，
wherein the third feature S and the hole convolution feature H are both transformed to C×d, and element a and element b represent the hole convolution feature H and the channel attention feature F, respectively u Soft attention vector of element S c Line c, element a, representing the third feature S c C element representing element a, element H c Line c, element b, representing the hole convolution characteristic H c The c-th element, F, representing element b, F is the channel attention feature F u And a selection weight vector generated when the cavity convolution characteristic H is selectively fused.
As shown in fig. 5, as a preferred embodiment of the present application, the decoupling enhancement spatial attention module includes a boundary information extraction module and a whitening region similarity extraction module, and the decoupling enhancement spatial attention module enables better learning and utilization of importance degree and similarity between pixels of each pixel of the footprint image to be identified, enhances learning ability between texture information and region pixels in the footprint image to be identified, and reduces computation complexity of the convolutional neural network.
As a preferred embodiment of the present application, the step S43 specifically includes:
extracting the third features S by utilizing a boundary information module to obtain a pixel-by-pixel saliency boundary information matrix A;
extracting the third characteristic S by using a whitening region similarity information module to obtain a whitening region similarity information matrix B;
adding the pixel-by-pixel saliency boundary information matrix A and the whitening region similarity information matrix B to obtain a third fusion feature U;
performing feature processing according to the third feature S and the third fusion feature U, and scaling through a weight coefficient to obtain a sixth feature W;
and adding the sixth feature W and the third feature S to obtain a fourth feature D. The weight coefficient can be set according to actual requirements and can be learned and adjusted gradually along with training of the convolutional neural network.
Specifically, the third feature S of size c×h×w is subjected to feature extraction by convolution of 1×1 to obtain the boundary information feature F of size 1×h×w b The method comprises the steps of carrying out a first treatment on the surface of the Will boundary information feature F b Dimension transformation is carried out to obtain boundary information characteristic F 'with the size of 1 XN' b The method comprises the steps of carrying out a first treatment on the surface of the Boundary information feature F' b Inputting a Softmax layer to perform nonlinear mapping to obtain significance boundary information F distributed in the range of 0-1 sbi The method comprises the steps of carrying out a first treatment on the surface of the Saliency boundary information F sbi And performing dimension expansion to obtain a pixel-by-pixel saliency boundary information matrix A with the size of N multiplied by N, and well modeling the region boundary of the footprint image to be identified by extracting the boundary information of the third feature S.
Further, feature extraction is performed on the third feature S with the size of C×H×W through a BN (Batch Normalization) layer and a ReLU activation function to obtain a feature S with the size of C×H×W BR The method comprises the steps of carrying out a first treatment on the surface of the Will feature S BR Input into a whitening module to obtain whitening feature F w . Therefore, the third characteristic S can be better corrected according to the similarity of the whitened areas, so that the convolutional neural network can learn the relationship in the whitened areas well.
Specifically, whiten feature F w Dimension transformation is carried out to obtain whitening characteristic F 'with the size of C multiplied by N' w The method comprises the steps of carrying out a first treatment on the surface of the Will whiten the character F' w Performing transposition operation to obtain whitening characteristic F', with size of N×C w The method comprises the steps of carrying out a first treatment on the surface of the Whitening feature F w And F' w Multiplying to obtain a whitening region matrix F with the size of N multiplied by N z The method comprises the steps of carrying out a first treatment on the surface of the Whitening region matrix F z Input toIn the softmax layer, a whitening region similarity information matrix B having a size of nxn is obtained, and the relationship between pixels in the whitening region can be learned by extracting the whitening region similarity information of the third feature S. And the third feature S is decoupled into saliency boundary information and whitening region similarity information through the decoupling enhancement spatial attention module, so that the utilization rate of each pixel is improved, and the recognition effect of the footprint image to be recognized is further improved.
Further, the saliency boundary information matrix A and the whitening region similarity information matrix B are simply added to obtain a third fusion characteristic U with the size of N multiplied by N, and the effects of region boundary modeling and relationship learning in the whitening region of the footprint image to be identified can be simultaneously improved by fusing the saliency boundary information matrix A and the whitening region similarity information matrix B; will feature S BR Performing dimension transformation to obtain a feature S' with the size of C multiplied by N; multiplying the feature B' by the transpose of the third fusion feature U, and then carrying out dimensional change on the multiplied result to obtain a feature F with the size of C multiplied by H multiplied by W; the feature F is scaled by a weight coefficient to obtain a sixth feature W, the sixth feature W and the third feature S are added according to elements to obtain a final decoupling enhancement spatial attention module output feature with the size of C multiplied by H multiplied by W, namely a fourth feature D, and the formula is as follows:
wherein, alpha is a weight coefficient, which is set to 0 in an initial state, and is gradually learned and adjusted along with training so as to more accurately represent the importance among the features; d (D) k Kth line, S, representing a fourth feature k Kth line, U, representing third feature S jk Represents the kth element, S 'of the jth column of the third fusion feature U' k Kth line, k.epsilon.1, C, representing feature S]C is the number of channels of the convolutional neural network.
In an embodiment of the present application, inputting the basic feature into the composite attention mechanism in step S4 further includes:
inputting the fifth characteristic G into the orthogonalization of the spectrum value differenceThen the layer is converted to obtain the characteristic G S ；
Inputting the fourth feature D into the spectrum value difference orthogonality regularization layer to obtain a feature D S ；
The characteristic G S Feature D S Adding the third feature S element by element, and obtaining a 1024-dimensional second feature C by a global average pooling layer 2 。
As a preferred embodiment of the present application, the identification method further includes:
training by using the characteristics of the template footprint image in the database to obtain a trained footprint recognition model;
when the actual characteristics are different from the database, performing cross-domain learning identification on the characteristics of the footprint image to be identified by adopting transfer learning, and adjusting the footprint identification model;
and identifying the footprint image to be identified by adopting the adjusted footprint identification model to obtain a footprint image identification result.
As can be appreciated by those skilled in the art, the template footprint image is collected to form a dataset, and the convolutional neural network is trained using the dataset, thereby obtaining a trained footprint recognition model. The data sets are divided into a same-domain data set and a cross-domain data set, wherein the same-domain data set is that sole patterns corresponding to footprint images to be identified appear in a training set and a verification set; the cross-domain data set is that the sole pattern corresponding to the footprint image to be identified does not appear in the training set and the verification set.
When the sole patterns corresponding to the footprint images to be identified are in the training set, the footprint images to be identified can be identified by using a trained footprint identification model, and the method is same-domain identification;
when the sole patterns corresponding to the footprint image to be identified do not appear in the training set, namely the sole patterns of the footprint image to be identified are not trained by the convolutional neural network, the transfer learning is needed to carry out the cross-domain learning identification, so that the trained footprint identification model can quickly learn and adapt to the new sole patterns when facing the new sole patterns, and the method is the cross-domain identification.
Specifically, the specific steps of cross-domain identification include:
preprocessing the footprint image to be identified;
extracting features of the footprint image to be identified by using a convolutional neural network, storing the extracted features into a feature database, and adjusting a footprint identification model;
and identifying the footprint image to be identified by using the adjusted footprint identification model, and outputting an identification result.
By adjusting the trained footprint recognition model, the time consumption is short, the recognition accuracy of the footprint recognition model can be improved, and the performance of the footprint recognition model is greatly improved.
According to the method and the device, the characteristic extraction is carried out on the footprint image to be identified through the grouping channel attention mechanism module, the sensitivity of the convolutional neural network to the correlation information among characteristic channels is improved, the characteristic expression capability of the footprint image to be identified is enhanced, meanwhile, the decoupling enhancement spatial attention module is used for decoupling basic characteristics of the footprint image to be identified, the learning capability among texture information and regional pixels in the footprint image to be identified is enhanced, the calculation complexity of the convolutional neural network is reduced, and the robustness of the characteristics is improved. The method and the device can also adjust the trained footprint recognition model, and improve the recognition accuracy of the footprint recognition model.
As shown in fig. 6, the present application further discloses a footprint identification system of a compound attention mechanism, and the footprint identification method adopting the compound attention mechanism as described in any one of the above, including:
the acquisition module is used for acquiring the template footprint image and establishing a characteristic database according to the characteristics of the template image;
the characteristic extraction module is used for extracting characteristics of the footprint image to be identified to obtain actual characteristics;
and the comparison module is used for comparing the actual characteristics of the footprint image to be identified with the characteristic database to obtain the identification result of the footprint image to be identified.
According to the method, the template footprint image is collected through the collecting module, the feature database is built, the feature extraction module performs feature extraction on the footprint image to be identified, the actual features of the footprint image to be identified are compared with the feature database through the comparison module, and then the identification result of the footprint image to be identified is obtained, wherein the feature extraction module comprises a decoupling enhancement spatial attention unit and a grouping channel attention unit, the association degree of local information of the footprint image to be identified is enhanced through the grouping channel attention unit, the identification performance is improved, meanwhile, the learning capability between texture information and regional pixels in the footprint image to be identified is enhanced through the decoupling enhancement spatial attention unit, and the calculation complexity of a convolutional neural network is reduced.
In a further embodiment of the present application, the footprint recognition system further includes a training module and a learning module, where the training module is configured to train according to features of the template footprint image in the feature database, to obtain a trained footprint recognition model; and the learning module is used for performing cross-domain learning identification by adopting transfer learning when the actual characteristics and the database are different, and adjusting the trained footprint identification model.
By adjusting the trained footprint recognition model, the recognition accuracy of the footprint recognition model is improved, and the performance of the footprint recognition model is greatly improved.
The application also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus; the processor, the memory and the communication interface complete communication with each other through the bus; the memory stores program instructions executable by the processor that the processor invokes to implement a footprint identification method of the composite attention mechanism as described in any of the above.
The application also discloses a computer readable storage medium storing computer instructions that cause the computer to implement a footprint identification method of a composite attention mechanism as described in any of the above.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.
Claims (8)
1. A footprint identification method of a composite attention mechanism is characterized by comprising the following steps of: the identification method comprises the following steps:
s1, acquiring a template footprint image, and establishing a feature database according to the features of the template footprint image;
s2, footprint image I to be identified 0 Extracting features by using N convolution layers to obtain basic features F 4 The method comprises the steps of carrying out a first treatment on the surface of the Wherein N convolution layers are sequentially arranged, N is a non-zero natural number;
s3, the basic characteristic F 4 Inputting to the (n+1) th convolution layer for global feature extraction to obtain a first feature C 1 ；
S4, the basic characteristic F 4 Inputting the second characteristic C into a composite attention mechanism, and extracting local characteristics by decoupling the space attention enhancement module and the group channel attention module to obtain the second characteristic C 2 The method comprises the steps of carrying out a first treatment on the surface of the Wherein the composite attention mechanism comprises a decoupling enhancement spatial attention module and a grouping channel attention module;
s5, the first characteristic C 1 And second feature C 2 Splicing to obtain the actual characteristic C f ；
S6, the actual characteristic C f Comparing with a characteristic database to obtain the footprint image I to be identified 0 Is a result of the recognition of (a);
the composite attention mechanism further comprises an n+2th convolution layer and a spectrum value difference orthogonality regularization layer, and the step S4 specifically comprises:
s41, the basic characteristic F 4 Inputting into the (n+2) th convolution layer for feature extraction to obtain convolution feature F j ；
S42, the convolution characteristic F j Inputting a spectrum value difference orthogonality regularization layer to obtain a third characteristic S;
s43, inputting the third feature S to a decoupling enhancement spatial attention module for feature extraction to obtain a fourth feature D;
s44, inputting the third feature S to a grouping channel attention module for feature extraction to obtain a fifth feature G;
s45, fusing the fourth feature D and the fifth feature G to obtain a first fused feature
S46, integrating the first fusion characteristicGlobal average pooling is performed to obtain the second feature C 2 ；
The step S43 specifically includes:
extracting the third features S by utilizing an extracting boundary information module to obtain a pixel-by-pixel saliency boundary information matrix A;
extracting the third characteristic S by using a whitening region similarity information extraction module to obtain a whitening region similarity information matrix B;
adding the pixel-by-pixel saliency boundary information matrix A and the whitening region similarity information matrix B to obtain a third fusion feature U;
performing feature processing according to the third feature S and the third fusion feature U, and scaling through a weight coefficient to obtain a sixth feature W;
and adding the characteristic W and the third characteristic S to obtain a fourth characteristic D.
2. A method of footprint recognition of a compound attention mechanism as claimed in claim 1, wherein: the step S44 specifically includes:
dividing the third features S into a plurality of groups according to the number of channels to obtain a plurality of first feature groups Z;
sequentially inputting the plurality of first feature groups Z into a grouping channel attention module to obtain a plurality of second feature groups Z';
connecting the plurality of second feature groups Z' to obtain a channel attention feature F u ；
Extracting the third feature S through cavity convolution with a receptive field of 5 multiplied by 5 to obtain a cavity convolution feature H;
directing the channel attention feature F u Selectively fusing the cavity convolution characteristic H to obtain a fifth characteristic G:
G＝a c H+b c F u ,a c +b c ＝1，
wherein element a and element b represent a hole convolution feature H and a channel attention feature F, respectively u Soft attention vector of element S c Line c, element a, representing the third feature S c C element representing element a, element H c Line c, element b, representing the hole convolution characteristic H c The c-th element, F, representing element b, F is the channel attention feature F u And a selection weight vector generated when the cavity convolution characteristic H is selectively fused.
3. A method of footprint recognition of a compound attention mechanism as claimed in claim 2, wherein: the selectively fusing the channel attention feature and the cavity convolution feature specifically comprises the following steps:
directing the channel attention feature F u Adding the second fusion characteristic with the cavity convolution characteristic H to obtain a second fusion characteristic
Characterizing the second fusionLayer feed through global averaging poolingMapping the row characteristic dimension to obtain a selection weight vector f;
the channel attention feature F is selected according to the weight vector F u The fifth feature G is obtained by assigning values with a soft attention mechanism across channels with the hole convolution feature H.
4. A method of footprint recognition of a compound attention mechanism as claimed in claim 1, wherein: the identification method further comprises the following steps:
training by using the characteristics of the template footprint image in the database to obtain a trained footprint recognition model;
when the actual characteristics are different from the database, performing cross-domain learning identification on the characteristics of the footprint image to be identified by adopting transfer learning, and adjusting the footprint identification model;
and identifying the footprint image to be identified by adopting the adjusted footprint identification model to obtain a footprint image identification result.
5. A footprint recognition system of a compound attention mechanism, employing a footprint recognition method of a compound attention mechanism as claimed in any one of claims 1-4, characterized in that: comprising the following steps:
the acquisition module is used for acquiring the template footprint image and establishing a characteristic database according to the characteristics of the template image;
the characteristic extraction module is used for extracting characteristics of the footprint image to be identified to obtain actual characteristics;
and the comparison module is used for comparing the actual characteristics of the footprint image to be identified with the characteristic database to obtain the identification result of the footprint image to be identified.
6. A footprint recognition system of a compound attention mechanism as claimed in claim 5, wherein: the training module and the learning module are also included, wherein,
the training module is used for training according to the characteristics of the template footprint image in the characteristic database to obtain a trained footprint recognition model;
and the learning module is used for performing cross-domain learning identification by adopting transfer learning when the actual characteristics and the database are different, and adjusting the trained footprint identification model.
7. An electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus; wherein,
the processor, the memory and the communication interface complete the communication with each other through the bus;
the memory stores program instructions executable by the processor, the processor invoking the program instructions to implement the footprint identification method of the composite attention mechanism of any of claims 1-4.
8. A computer readable storage medium storing computer instructions for causing a computer to implement the footprint identification method of the compound attention mechanism of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311089272.9A CN117218720B (en) | 2023-08-25 | 2023-08-25 | Footprint identification method, system and related device of composite attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311089272.9A CN117218720B (en) | 2023-08-25 | 2023-08-25 | Footprint identification method, system and related device of composite attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117218720A CN117218720A (en) | 2023-12-12 |
| CN117218720B true CN117218720B (en) | 2024-04-16 |
Family
ID=89043355
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311089272.9A Active CN117218720B (en) | 2023-08-25 | 2023-08-25 | Footprint identification method, system and related device of composite attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117218720B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070073A (en) * | 2019-05-07 | 2019-07-30 | 国家广播电视总局广播电视科学研究院 | Pedestrian's recognition methods again of global characteristics and local feature based on attention mechanism |
| WO2019153908A1 (en) * | 2018-02-11 | 2019-08-15 | 北京达佳互联信息技术有限公司 | Image recognition method and system based on attention model |
| CN112580694A (en) * | 2020-12-01 | 2021-03-30 | 中国船舶重工集团公司第七0九研究所 | Small sample image target identification method and system based on joint attention mechanism |
| CN112767273A (en) * | 2021-01-21 | 2021-05-07 | 中山大学 | Low-dose CT image restoration method and system applying feature decoupling |
| CN113220926A (en) * | 2021-05-06 | 2021-08-06 | 安徽大学 | Footprint image retrieval method based on multi-scale local attention enhancement network |
| CN113255821A (en) * | 2021-06-15 | 2021-08-13 | 中国人民解放军国防科技大学 | Attention-based image recognition method, attention-based image recognition system, electronic device and storage medium |
| CN113822209A (en) * | 2021-09-27 | 2021-12-21 | 海南长光卫星信息技术有限公司 | Hyperspectral image recognition method and device, electronic equipment and readable storage medium |
| CN115795081A (en) * | 2023-01-20 | 2023-03-14 | 安徽大学 | Cross-domain incomplete footprint image retrieval system based on multi-channel fusion |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111428699B (en) * | 2020-06-10 | 2020-09-22 | 南京理工大学 | Driving fatigue detection method and system combining pseudo-3D convolutional neural network and attention mechanism |
-
2023
- 2023-08-25 CN CN202311089272.9A patent/CN117218720B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019153908A1 (en) * | 2018-02-11 | 2019-08-15 | 北京达佳互联信息技术有限公司 | Image recognition method and system based on attention model |
| CN110070073A (en) * | 2019-05-07 | 2019-07-30 | 国家广播电视总局广播电视科学研究院 | Pedestrian's recognition methods again of global characteristics and local feature based on attention mechanism |
| CN112580694A (en) * | 2020-12-01 | 2021-03-30 | 中国船舶重工集团公司第七0九研究所 | Small sample image target identification method and system based on joint attention mechanism |
| CN112767273A (en) * | 2021-01-21 | 2021-05-07 | 中山大学 | Low-dose CT image restoration method and system applying feature decoupling |
| CN113220926A (en) * | 2021-05-06 | 2021-08-06 | 安徽大学 | Footprint image retrieval method based on multi-scale local attention enhancement network |
| CN113255821A (en) * | 2021-06-15 | 2021-08-13 | 中国人民解放军国防科技大学 | Attention-based image recognition method, attention-based image recognition system, electronic device and storage medium |
| CN113822209A (en) * | 2021-09-27 | 2021-12-21 | 海南长光卫星信息技术有限公司 | Hyperspectral image recognition method and device, electronic equipment and readable storage medium |
| CN115795081A (en) * | 2023-01-20 | 2023-03-14 | 安徽大学 | Cross-domain incomplete footprint image retrieval system based on multi-channel fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117218720A (en) | 2023-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111582044B (en) | Face recognition method based on convolutional neural network and attention model | |
| WO2022036777A1 (en) | Method and device for intelligent estimation of human body movement posture based on convolutional neural network | |
| CN109657595B (en) | Key feature region matching face recognition method based on stacked hourglass network | |
| CN113077471A (en) | Medical image segmentation method based on U-shaped network | |
| CN111325111A (en) | Pedestrian re-identification method integrating inverse attention and multi-scale deep supervision | |
| CN112926396A (en) | Action identification method based on double-current convolution attention | |
| CN115496928B (en) | Multi-modal image feature matching method based on multi-feature matching | |
| CN111652273B (en) | Deep learning-based RGB-D image classification method | |
| CN113554032B (en) | Remote sensing image segmentation method based on multi-path parallel network of high perception | |
| CN113947814B (en) | Cross-view gait recognition method based on space-time information enhancement and multi-scale saliency feature extraction | |
| CN104077742B (en) | Human face sketch synthetic method and system based on Gabor characteristic | |
| CN111931603B (en) | Human body action recognition system and method of double-flow convolution network based on competitive network | |
| CN114972976B (en) | Night target detection and training method and device based on frequency domain self-attention mechanism | |
| Lu et al. | Rethinking prior-guided face super-resolution: A new paradigm with facial component prior | |
| CN113610046B (en) | Behavior recognition method based on depth video linkage characteristics | |
| CN114170286B (en) | Monocular depth estimation method based on unsupervised deep learning | |
| CN112580480A (en) | Hyperspectral remote sensing image classification method and device | |
| CN116030498A (en) | Virtual garment running and showing oriented three-dimensional human body posture estimation method | |
| CN116311353A (en) | Intensive pedestrian multi-target tracking method based on feature fusion, computer equipment and storage medium | |
| CN110135435B (en) | Saliency detection method and device based on breadth learning system | |
| CN109447147B (en) | Image clustering method based on depth matrix decomposition of double-image sparsity | |
| CN111209873A (en) | High-precision face key point positioning method and system based on deep learning | |
| CN112949500A (en) | Improved YOLOv3 lane line detection method based on spatial feature coding | |
| CN117218720B (en) | Footprint identification method, system and related device of composite attention mechanism | |
| CN110400250A (en) | Intelligent image preprocess method and system based on AI |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |