CN116994002B - Image feature extraction method, device, equipment and storage medium - Google Patents

Image feature extraction method, device, equipment and storage medium Download PDF

Info

Publication number
CN116994002B
CN116994002B CN202311236293.9A CN202311236293A CN116994002B CN 116994002 B CN116994002 B CN 116994002B CN 202311236293 A CN202311236293 A CN 202311236293A CN 116994002 B CN116994002 B CN 116994002B
Authority
CN
China
Prior art keywords
feature
image group
loop
group
initial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311236293.9A
Other languages
Chinese (zh)
Other versions
CN116994002A (en
Inventor
邹磊
韩雪超
卢天华
倪军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou AIMS Intelligent Technology Co Ltd
Original Assignee
Hangzhou AIMS Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou AIMS Intelligent Technology Co Ltd filed Critical Hangzhou AIMS Intelligent Technology Co Ltd
Priority to CN202311236293.9A priority Critical patent/CN116994002B/en
Publication of CN116994002A publication Critical patent/CN116994002A/en
Application granted granted Critical
Publication of CN116994002B publication Critical patent/CN116994002B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an image feature extraction method, an image feature extraction device, image feature extraction equipment and a storage medium, wherein the image feature extraction method comprises the following steps: acquiring an initial feature map group, wherein the initial feature map group comprises a specified number of feature maps; performing feature extraction on the initial feature image group by adopting a loop-back differential pair to obtain a first feature image group, wherein the pixel points in each feature image in the first feature image group have linkage; and carrying out random order scrambling treatment on the first feature image group, and fusing the first feature image group subjected to order scrambling with the initial feature image group to obtain image features. The initial feature image group is subjected to feature extraction in a loop-back differential mode, so that the linkage of each pixel point is established, random order-disturbing operation is used, the information circulation capacity between channels is improved, prediction fluctuation caused by abnormal values in a region is eliminated through loop-back differential, the adaptability in a complex environment is effectively improved, and the robustness of feature extraction is improved.

Description

Image feature extraction method, device, equipment and storage medium
Technical Field
The present invention relates to the field of deep learning technologies, and in particular, to an image feature extraction method, apparatus, device, and storage medium.
Background
With the development of deep learning technology, neural network technology is increasingly widely applied, and at present, it is more common to use neural network to extract image features.
However, when the existing neural network performs feature extraction, the gradient coding limitation which is not displayed in the initialization process of the convolution kernel makes it difficult to focus the extraction of the image gradient information in the circulation process, so that the extraction capability for edge features is poor; in addition, the abnormal value of the image is not optimized on the convolution operator during feature extraction, so that a stable effect cannot be exerted in some complex scenes, and the robustness in the feature extraction process is poor.
Disclosure of Invention
The invention provides an image feature extraction method, an image feature extraction device, image feature extraction equipment and a storage medium, so as to accurately extract image features.
According to an aspect of the present invention, there is provided an image feature extraction method including: acquiring an initial feature map group, wherein the initial feature map group comprises a specified number of feature maps;
performing feature extraction on the initial feature image group by using a loop-back differential pair to obtain a first feature image group, wherein pixels in each feature image in the first feature image group have linkage;
and carrying out random order scrambling treatment on the first feature image group, and fusing the first feature image group subjected to order scrambling with the initial feature image group to obtain image features.
According to another aspect of the present invention, there is provided an image feature extraction apparatus including: the device comprises an initial feature map group acquisition module, a feature map generation module and a feature map generation module, wherein the initial feature map group acquisition module is used for acquiring an initial feature map group, and the initial feature map group comprises a specified number of feature maps;
the loop difference processing module is used for carrying out feature extraction on the initial feature image group by adopting a loop difference to obtain a first feature image group, wherein the pixel point in each feature image in the first feature image group has linkage;
and the feature fusion module is used for carrying out random order scrambling on the first feature image group and fusing the first feature image group subjected to order scrambling with the initial feature image group to obtain image features.
According to another aspect of the present invention, there is provided a computer apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of any one of the embodiments of the present invention.
According to another aspect of the invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to perform the method according to any of the embodiments of the invention.
According to the technical scheme, the initial feature image group is subjected to feature extraction in a loop-back differential mode, so that the linkage of each pixel point is established, random order-disturbing operation is used, the information circulation capacity among channels is improved, prediction fluctuation caused by abnormal values in a region is eliminated through loop-back differential, the adaptability in a complex environment is effectively improved, and the robustness of feature extraction is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of an image feature extraction method according to a first embodiment of the present invention;
FIG. 2 is a schematic diagram of a feature extraction block according to a first embodiment of the invention;
FIG. 3 is a schematic diagram illustrating a loop-back differential processing according to a first embodiment of the present invention;
fig. 4 is a flowchart of an image feature extraction method according to a second embodiment of the present invention;
fig. 5 is a schematic structural view of an image feature extraction device according to a third embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, "comprises," "comprising," and "having" and any variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or device that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or device.
Example 1
Fig. 1 is a flowchart of an image feature extraction method according to an embodiment of the present invention, where the method may be performed by an image feature extraction device, and the image feature extraction device may be implemented in hardware and/or software, and the image feature extraction device may be integrally configured in an electronic device. As shown in fig. 1, the method includes:
step S101, an initial feature map set is acquired.
Specifically, in this embodiment, the residual structure of the residual neural network res net is mainly optimized. The loop difference mode is introduced, so that the problem of poor robustness of a general feature extraction method is solved on a convolution operator layer, meanwhile, the problem that a network cannot focus image gradients in a training process is solved, the effect of the network on image edge feature extraction is better, a structural schematic diagram of a feature extraction block related to the application is shown in fig. 2, and the feature extraction block comprises an input layer, a channel compressed 1*1 convolution layer, a clockwise loop difference layer, a counterclockwise loop difference layer, a channel splicing layer, a channel expanded 1*1 convolution layer and an output layer.
In this embodiment, an initial feature map set is obtained through the input layer, where the initial feature map set includes a specified number of feature maps, for example, 32 feature maps of 128×128 are included in the initial feature map set.
Step S102, feature extraction is carried out on the initial feature map set by adopting a loop-back differential pair to obtain a first feature map set.
Optionally, performing feature extraction on the initial feature map set by using a loop differential pair to obtain a first feature map set, including: inputting the initial feature image group into a first convolution layer, and carrying out channel compression on the initial feature image group according to a designated multiple through the first convolution layer to obtain a compressed feature image group, wherein the number of feature images in the compressed feature image group is smaller than that of the initial feature image group; and carrying out feature extraction on the compressed feature image group by adopting a loop-back differential mode to obtain a first feature image group.
Optionally, performing feature extraction on the compressed feature map set by using a loop-back differential mode to obtain a first feature map set, including: equally dividing the compressed characteristic image group to obtain two sub-characteristic image groups, and inputting each sub-characteristic image group into a clockwise loop-back difference layer and a counterclockwise loop-back difference layer respectively; performing clockwise loop difference processing on the input sub-feature image group through the clockwise loop difference layer to obtain a first loop difference processing feature image group; performing anticlockwise loop-back differential processing on the input sub-feature image group through an anticlockwise loop-back differential layer to obtain a second loop-back differential processing feature image group; and performing channel splicing on the first loop differential processing characteristic image group and the second loop differential processing characteristic image group to obtain a first characteristic image group, wherein the number of channels of the first characteristic image group is the same as that of channels of the compressed characteristic image group.
Specifically, in this embodiment, after the initial feature map set is acquired through the input layer, the input is split into two branches, path 1 and path 2. For the path 1, an initial feature map set, for example, 32 feature maps of 128×128, is passed through a first convolution layer, for example, a channel compressed 1*1 convolution layer, and the convolution layer is used to perform channel compression according to a specified multiple to obtain a compressed feature map set, for example, compressed to 1/4 of the original feature map set, so as to obtain 8 feature maps. The output of the 1*1 convolution layer is equally divided into two sub-feature image groups, namely each sub-feature image group comprises 4 feature images, each sub-feature image group is respectively input into a clockwise loop-back difference layer and a counterclockwise loop-back difference layer, namely 4 feature images are input into the clockwise loop-back difference layer to be subjected to clockwise loop-back difference processing to obtain a first loop-back difference processing feature image group, the other 4 feature images are input into the counterclockwise loop-back difference layer to be subjected to counterclockwise loop-back difference processing to obtain a second loop-back difference processing feature image group, and the result of the completion of the two-part difference operation is subjected to channel splicing to obtain a first feature image group, so that 8 feature images are contained in the first feature image group, but the obtained feature images are richer than the feature images in the initial feature image group.
Optionally, performing the clockwise loop difference processing on the input sub-feature image group through the clockwise loop difference layer to obtain a first loop difference processing feature image group, including: sliding on each feature map in the sub-feature map group according to a specified step length by adopting a convolution kernel with a specified size through a clockwise loop difference layer; determining a coverage area corresponding to each sliding, and subtracting the outer ring edge pixels of the coverage area in a clockwise direction to obtain a first processing result; respectively carrying out dot multiplication processing on the first processing result corresponding to each sliding and the convolution kernel to obtain a sub-feature diagram after clockwise feature extraction; and combining the sub-feature graphs after the clockwise feature extraction to obtain a first loop differential processing feature graph group.
Optionally, performing the counterclockwise loop-back differential processing on the input sub-feature image group through the counterclockwise loop-back differential layer to obtain a second loop-back differential processing feature image group, including: sliding on each feature map in the sub-feature map group according to a specified step length by adopting a convolution kernel with a specified size through a anticlockwise loop difference layer; determining a coverage area corresponding to each sliding, and subtracting the outer ring edge pixels of the coverage area in the anticlockwise direction to obtain a second processing result; respectively carrying out dot multiplication processing on the second processing result corresponding to each sliding and the convolution kernel to obtain a sub-feature diagram after the anticlockwise feature extraction; and combining the sub-feature graphs after the anticlockwise feature extraction to obtain a second loop differential processing feature graph group.
In this embodiment, the clockwise loop difference processing in the clockwise loop difference layer and the counterclockwise loop difference processing in the counterclockwise loop difference layer are taken as an example for illustration, as shown in fig. 3. When it is determined that the sub-feature map group includes four 128×128 feature maps, the following operations are performed for each feature map in the input clockwise loop differential layer: sliding the convolution kernel of 3*3 on the feature map according to the step length 1, subtracting adjacent pixels on the outer ring in a clockwise direction for a 3*3 coverage area corresponding to each sliding, where x1 and x 2..x 9 in fig. 3 represent pixel points before being processed, w1 and w2...w 9 represent weight values of the convolution kernel, and the pixel point x5 does not perform subtraction operation, so that a new set of pixel values is obtained after loop subtraction, and the obtained new pixel values are used as a first processing result. And then, performing dot multiplication operation on the first processing result corresponding to the sliding and the weight value of the 3*3 convolution kernel to finish the feature extraction operation of the clockwise loop difference part. The following formula (1) is a calculation formula of the clockwise loop difference:
y=w1*(x1-x2)+w2*(x2-x3)+w3*(x3-x6)+w4*(x4-x1)+w5*x5+w6*(x6-x9)+w7*(x7-x4)+w8*(x8-x7)+w9*(x9-x8) (1)
in this embodiment, only sliding is taken as an example to describe, a sub-feature map for each feature map is obtained according to a first processing result obtained by sliding each time, and the obtained four sub-feature maps are combined to obtain a first loop differential processing feature map group.
Note that, in this embodiment, the counterclockwise loop difference operation is similar to the clockwise loop difference operation, but the mode of subtracting the adjacent pixels in the clockwise direction is replaced by the counterclockwise subtraction, and the following formula (2) is a calculation formula of the counterclockwise loop difference:
y=w1*(x1-x4)+w2*(x2-x1)+w3*(x3-x2)+w4*(x4-x7)+w5*x5+w6*(x6-x3)+w7*(x7-x8)+w8*(x8-x9)+w9*(x9-x6) (2)
in this embodiment, the first loop differential processing feature image group and the second loop differential processing feature image group are subjected to channel stitching to obtain a first feature image group, so that the first feature image group contains 8 processed feature images, and the number of channels of the first feature image group and the compressed feature image group is the same as 8. As can be known from the above loop-back differential processing operation, the pixel points in each feature map in the first feature map set have linkage, for example, the pixel points x1 and x2 in the original feature map are independent, and the pixel points in the feature map processed in the present application are differences, for example, x1-x2, of the original pixel points. The image gradient information is thus better focused in case the number of feature extraction blocks in the network is sufficiently large.
Step S103, carrying out random order scrambling treatment on the first feature image group, and fusing the first feature image group subjected to order scrambling with the initial feature image group to obtain image features.
Specifically, in this embodiment, after the first feature map set is obtained, the first feature map set is subjected to random order scrambling, so that the model is prevented from focusing on only the information of the clockwise loop difference layer or the counterclockwise loop difference layer, for example, the first feature map set includes feature map 1', feature map 2', feature map 3', feature map 4', feature map 5', feature map 6', feature map 7', and feature map 8', wherein feature map 1', feature map 2', feature map 3', and feature map 4' are output results of the clockwise loop difference layer, and feature map 5', feature map 6', feature map 7', and feature map 8' are output results of the counterclockwise loop difference layer, and then the first feature map set is subjected to random order scrambling, for example, the feature map 8', feature map 2', feature map 6', feature map 7', feature map 4', feature map 5', feature map 1', and feature map 3' are only illustrated in this embodiment, and the specific form of the first feature map set is not limited.
Optionally, fusing the first feature map set and the initial feature map set after the disordered sequence to obtain image features includes: inputting the first feature image group after the order is disordered into a second convolution layer, and carrying out channel expansion on the first feature image group after the order is disordered through the second convolution layer according to a designated multiple to obtain an expanded feature image group; and fusing the expansion characteristic image group and the initial characteristic image group to obtain image characteristics.
Specifically, in this embodiment, after the first feature map set is obtained through the path 1, the first feature map set after the order is disturbed is input into the second convolution layer, and the first feature map after the order is disturbed is subjected to channel expansion by the second convolution layer according to a specified multiple to obtain the expanded feature map set, where the expansion multiple of the second convolution layer corresponds to the compression multiple of the first convolution layer, for example, the expansion multiple of the second convolution layer is 4, so that expansion of 8 feature maps into 32 feature maps is achieved, and an expanded feature map set is constructed according to the expanded feature maps.
In this embodiment, the number of channels of the obtained expansion feature map set and the initial feature map set is the same, so in this embodiment, the initial feature map set obtained by the input layer is directly obtained from the path 2, and the initial feature map set and the expansion feature map set are added point by point, so as to be used for fusion of shallow layer information and deep layer information, and the final image feature is obtained through the output layer.
In the embodiment, only the feature extraction of one feature extraction block is taken as an example for description, and the feature extraction block uses a calculation mode of clockwise loop difference and anticlockwise loop difference to enable the model to better focus image gradient information in the training process, and meanwhile, in order to prevent the model from excessively still carrying out clockwise loop difference or anticlockwise loop difference information, the method and the device use random order-disturbing operation, improve the information circulation capacity among channels, and reduce the occurrence of an overfitting phenomenon in the model training process. In addition, the pixel values in the smooth areas of the clockwise loop difference and the anticlockwise loop difference are used, prediction fluctuation caused by abnormal values in the areas is eliminated, the adaptability of the model in complex environments such as severe brightness change can be effectively improved, and the method has important significance for downstream tasks such as semantic segmentation.
It should be noted that, in this embodiment, the above-mentioned multiple feature extraction blocks may be used to construct a feature extraction network by using different arrangements, and the following table 1 shows an example of structural parameters of the constructed feature extraction network:
TABLE 1
Wherein, the feature extraction blocks in table 1 refer to the feature extraction blocks shown in fig. 2, and x in the layer name represents the serial number of repeated stacking, and if the feature extraction blocks 2_x are stacked 3 times, the layer name is the feature extraction block 2_1, the feature extraction block 2_2 and the feature extraction block 2_3. The last stacking of the current module requires 2 times downsampling, for example, the feature extraction block 2_3 in the feature extraction block 2_x requires 2 times downsampling in resolution, and the output channel N in the softmax layer represents the number of categories to be classified. The above table 1 is exemplified by the input image resolution 224x224, and the output size is different for different input resolutions, but the same form as in the above table 1 is adopted, and the number of output channels of the next module is 2 times the number of output channels of the previous module. Of course, the present embodiment is merely illustrative, and the specific form of the feature extraction network constructed by using the above-described feature extraction block is not limited.
According to the method and the device, the initial feature image group is subjected to feature extraction in a loop-back differential mode, so that the linkage among all pixel points is established, random order-disturbing operation is used, the information circulation capacity among channels is improved, prediction fluctuation caused by abnormal values in a region is eliminated through loop-back differential, the adaptability in a complex environment is effectively improved, and the robustness of feature extraction is improved.
Example two
Fig. 4 is a flowchart of an image feature extraction method according to a second embodiment of the present invention, where after the fused image features are obtained, the fused image features are detected. As shown in fig. 4, the method includes:
step S201, an initial feature map set is acquired.
Step S202, performing feature extraction on the initial feature map set by using a loop-back differential pair to obtain a first feature map set.
Optionally, performing feature extraction on the initial feature map set by using a loop differential pair to obtain a first feature map set, including: inputting the initial feature image group into a first convolution layer, and carrying out channel compression on the initial feature image group according to a designated multiple through the first convolution layer to obtain a compressed feature image group, wherein the number of feature images in the compressed feature image group is smaller than that of the initial feature image group; and carrying out feature extraction on the compressed feature image group by adopting a loop-back differential mode to obtain a first feature image group.
Optionally, performing feature extraction on the compressed feature map set by using a loop-back differential mode to obtain a first feature map set, including: equally dividing the compressed characteristic image group to obtain two sub-characteristic image groups, and inputting each sub-characteristic image group into a clockwise loop-back difference layer and a counterclockwise loop-back difference layer respectively; performing clockwise loop difference processing on the input sub-feature image group through the clockwise loop difference layer to obtain a first loop difference processing feature image group; performing anticlockwise loop-back differential processing on the input sub-feature image group through an anticlockwise loop-back differential layer to obtain a second loop-back differential processing feature image group; and performing channel splicing on the first loop differential processing characteristic image group and the second loop differential processing characteristic image group to obtain a first characteristic image group, wherein the number of channels of the first characteristic image group is the same as that of channels of the compressed characteristic image group.
Optionally, performing the clockwise loop difference processing on the input sub-feature image group through the clockwise loop difference layer to obtain a first loop difference processing feature image group, including: sliding on each feature map in the sub-feature map group according to a specified step length by adopting a convolution kernel with a specified size through a clockwise loop difference layer; determining a coverage area corresponding to each sliding, and subtracting the outer ring edge pixels of the coverage area in a clockwise direction to obtain a first processing result; respectively carrying out dot multiplication processing on the first processing result corresponding to each sliding and the convolution kernel to obtain a sub-feature diagram after clockwise feature extraction; and combining the sub-feature graphs after the clockwise feature extraction to obtain a first loop differential processing feature graph group.
Optionally, performing the counterclockwise loop-back differential processing on the input sub-feature image group through the counterclockwise loop-back differential layer to obtain a second loop-back differential processing feature image group, including: sliding on each feature map in the sub-feature map group according to a specified step length by adopting a convolution kernel with a specified size through a anticlockwise loop difference layer; determining a coverage area corresponding to each sliding, and subtracting the outer ring edge pixels of the coverage area in the anticlockwise direction to obtain a second processing result; respectively carrying out dot multiplication processing on the second processing result corresponding to each sliding and the convolution kernel to obtain a sub-feature diagram after the anticlockwise feature extraction; and combining the sub-feature graphs after the anticlockwise feature extraction to obtain a second loop differential processing feature graph group.
And step S203, carrying out random order scrambling treatment on the first feature image group, and fusing the first feature image group subjected to order scrambling with the initial feature image group to obtain image features.
Optionally, fusing the first feature map set and the initial feature map set after the disordered sequence to obtain image features includes: inputting the first feature image group after the order is disordered into a second convolution layer, and carrying out channel expansion on the first feature image group after the order is disordered through the second convolution layer according to a designated multiple to obtain an expanded feature image group; and fusing the expansion characteristic image group and the initial characteristic image group to obtain image characteristics.
Step S204, judging whether the feature value of the image feature is larger than the initial feature map set, if yes, executing step S205, otherwise, executing step S206.
Specifically, in this embodiment, after the image features are obtained through image fusion, since the feature extraction method of loop difference is adopted in this embodiment, the feature value actually extracted by the feature extraction block is correspondingly increased, so that in this embodiment, the finally obtained image feature value may be compared with the initial feature map set, and the success condition of the feature extraction is determined according to the increase condition of the feature value.
In step S205, it is determined that feature extraction is successful.
When the feature value of the image feature is detected to be larger than the feature value of the initial feature image group, the description accords with the actual situation, so that the feature extraction process is successful, prompt information of successful feature extraction can be generated, and a user can know the process of feature extraction in time conveniently.
Step S206, determining that the feature extraction fails.
When the detected characteristic value of the image characteristic is smaller than or equal to the characteristic value of the initial characteristic image group, the description is not consistent with the actual situation, so that the characteristic extraction process is failed, and prompt information of failure in characteristic extraction is generated, thereby facilitating corresponding checking and debugging of equipment or software by a user in time according to the prompt information of failure in characteristic extraction, and further improving the efficiency and accuracy of characteristic extraction.
According to the method and the device, the initial feature image group is subjected to feature extraction in a loop-back differential mode, so that the linkage among all pixel points is established, random order-disturbing operation is used, the information circulation capacity among channels is improved, prediction fluctuation caused by abnormal values in a region is eliminated through loop-back differential, the adaptability in a complex environment is effectively improved, and the robustness of feature extraction is improved.
Example III
Fig. 5 is a schematic structural diagram of an image feature extraction device according to a third embodiment of the present invention. As shown in fig. 5, the apparatus includes: an initial feature map set acquisition module 310, a loop-back differential processing module 320, and a feature fusion module 330.
The initial feature map set obtaining module 310 is configured to obtain an initial feature map set, where the initial feature map set includes a specified number of feature maps;
the loop differential processing module 320 is configured to perform feature extraction on the initial feature map set by using a loop differential pair to obtain a first feature map set, where a pixel point in each feature map in the first feature map set has linkage;
the feature fusion module 330 is configured to randomly shuffle the first feature map set, and fuse the shuffled first feature map set with the initial feature map set to obtain image features.
Optionally, the loop differential processing module includes a channel compression unit, configured to input an initial feature image group into a first convolution layer, and perform channel compression on the initial feature image group by using the first convolution layer according to a specified multiple to obtain a compressed feature image group, where the number of feature images in the compressed feature image group is smaller than that of the initial feature image group;
and the loop difference unit is used for carrying out feature extraction on the compressed feature map set by adopting a loop difference mode to obtain a first feature map set.
Optionally, the loop-back differential unit includes: the compression characteristic diagram equally dividing sub-unit is used for equally dividing the compression characteristic diagram group to obtain two sub-characteristic diagram groups, and inputting each sub-characteristic diagram group into a clockwise loop-back difference layer and a counterclockwise loop-back difference layer respectively;
the clockwise loop difference processing subunit is used for carrying out clockwise loop difference processing on the input sub-feature image group through the clockwise loop difference layer to obtain a first loop difference processing feature image group;
the anticlockwise loop differential processing subunit is used for obtaining a second loop differential processing characteristic diagram set through anticlockwise loop differential processing of the anticlockwise loop differential layer pair input sub-characteristic diagram set;
and the splicing subunit is used for carrying out channel splicing on the first loop differential processing characteristic image group and the second loop differential processing characteristic image group to obtain a first characteristic image group, wherein the number of channels of the first characteristic image group is the same as that of channels of the compressed characteristic image group.
Optionally, the clockwise loop differential processing subunit is configured to slide on each feature map in the sub-feature map group according to a specified step size by using a convolution kernel with a specified size through the clockwise loop differential layer;
determining a coverage area corresponding to each sliding, and subtracting the outer ring edge pixels of the coverage area in a clockwise direction to obtain a first processing result;
respectively carrying out dot multiplication processing on the first processing result corresponding to each sliding and the convolution kernel to obtain a sub-feature diagram after clockwise feature extraction;
and combining the sub-feature graphs after the clockwise feature extraction to obtain a first loop differential processing feature graph group.
Optionally, the anticlockwise loop differential processing subunit is configured to slide, by using an anticlockwise loop differential layer, on each feature map in the sub-feature map group by using a convolution kernel with a specified size according to a specified step length;
determining a coverage area corresponding to each sliding, and subtracting the outer ring edge pixels of the coverage area in the anticlockwise direction to obtain a second processing result;
respectively carrying out dot multiplication processing on the second processing result corresponding to each sliding and the convolution kernel to obtain a sub-feature diagram after the anticlockwise feature extraction;
and combining the sub-feature graphs after the anticlockwise feature extraction to obtain a second loop differential processing feature graph group.
Optionally, the feature fusion module is used for inputting the first feature map group after the order is disturbed into the second convolution layer, and expanding the channel of the first feature map after the order is disturbed by the second convolution layer according to a specified multiple to obtain an expanded feature map group;
and fusing the expansion characteristic image group and the initial characteristic image group to obtain image characteristics.
Optionally, the device further comprises an image feature detection module for judging whether the feature value of the image feature is larger than the initial feature image group, if so, determining that the feature extraction is successful,
otherwise, determining that the feature extraction fails.
The image feature extraction device provided by the embodiment of the invention can execute the image feature extraction method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 6 shows a schematic diagram of the structure of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 6, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the respective methods and processes described above, for example, the image feature extraction method.
In some embodiments, the image feature extraction method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the image feature extraction method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image feature extraction method in any other suitable way (e.g. by means of firmware).
Various implementations of the apparatus and techniques described here above may be implemented in digital electronic circuit devices, integrated circuit devices, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), on-chip device devices (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on programmable devices including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, operable to receive data and instructions from, and to transmit data and instructions to, a storage device, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable crown block work warning device such that the computer programs, when executed by the processor, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution apparatus, device, or apparatus. The computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor apparatus, device, or apparatus, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the apparatus and techniques described herein may be implemented on a device having: a display device (e.g., a touch screen) for displaying information to a user; and keys, the user may provide input to the device through a touch screen or keys. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (8)

1. An image feature extraction method, characterized by comprising:
acquiring an initial feature map group, wherein the initial feature map group comprises a specified number of feature maps;
performing feature extraction on the initial feature image group by using a loop-back differential pair to obtain a first feature image group, wherein pixels in each feature image in the first feature image group have linkage;
randomly scrambling the first feature image group, and fusing the first feature image group with the initial feature image group to obtain image features;
the step of extracting features of the initial feature map set by using loop-back differential pair to obtain a first feature map set includes: inputting the initial feature image group into a first convolution layer, and carrying out channel compression on the initial feature image group through the first convolution layer according to a specified multiple to obtain a compressed feature image group, wherein the number of feature images in the compressed feature image group is smaller than that of the initial feature image group;
performing feature extraction on the compressed feature image group by adopting a loop-back differential mode to obtain the first feature image group;
the feature extraction of the compressed feature image set by adopting a loop-back differential mode to obtain the first feature image set includes: equally dividing the compressed characteristic image group to obtain two sub-characteristic image groups, and respectively inputting each sub-characteristic image group into a clockwise loop difference layer and a counterclockwise loop difference layer;
performing clockwise loop difference processing on the input sub-feature image group through the clockwise loop difference layer to obtain a first loop difference processing feature image group;
performing anticlockwise loop-back differential processing on the input sub-feature image group through the anticlockwise loop-back differential layer to obtain a second loop-back differential processing feature image group;
and carrying out channel splicing on the first loop differential processing characteristic image group and the second loop differential processing characteristic image group to obtain the first characteristic image group, wherein the number of channels of the first characteristic image group is the same as that of channels of the compression characteristic image group.
2. The method according to claim 1, wherein said performing the clockwise loop-back differential processing on the input sub-feature map set by the clockwise loop-back differential layer to obtain a first loop-back differential processing feature map set includes:
sliding on each feature map in the sub-feature map group according to a specified step length by adopting a convolution kernel with a specified size through the clockwise loop differential layer;
determining a coverage area corresponding to each sliding, and subtracting outer ring edge pixels of the coverage area in a clockwise direction to obtain a first processing result;
respectively carrying out dot multiplication processing on a first processing result corresponding to each sliding and the convolution kernel to obtain a sub-feature diagram after clockwise feature extraction;
and combining the sub-feature graphs after the clockwise feature extraction to obtain the first loop differential processing feature graph group.
3. The method according to claim 1, wherein the performing the counterclockwise loop-back differential processing on the input sub-feature map set by the counterclockwise loop-back differential layer to obtain a second loop-back differential processing feature map set includes:
sliding on each feature map in the sub-feature map group according to a specified step length by adopting a convolution kernel with a specified size through the anticlockwise loop difference layer;
determining a coverage area corresponding to each sliding, and subtracting outer ring edge pixels of the coverage area in a counter-clockwise direction to obtain a second processing result;
respectively carrying out dot multiplication on a second processing result corresponding to each sliding and the convolution kernel to obtain a sub-feature diagram after anticlockwise feature extraction;
and combining the sub-feature graphs after the anticlockwise feature extraction to obtain the second loop differential processing feature graph group.
4. The method of claim 1, wherein fusing the unordered first feature map set with the initial feature map set to obtain image features comprises:
inputting the first feature map group after the disorder into a second convolution layer, and performing channel expansion on the first feature map after the disorder through the second convolution layer according to a designated multiple to obtain an expanded feature map group;
and fusing the expansion characteristic image group and the initial characteristic image group to obtain the image characteristics.
5. The method of claim 1, wherein after fusing the first out-of-order feature map set with the initial feature map set to obtain image features, further comprising:
judging whether the characteristic value of the image characteristic is larger than the initial characteristic image group, if so, determining that the characteristic extraction is successful,
otherwise, determining that the feature extraction fails.
6. An image feature extraction device, characterized by comprising:
the device comprises an initial feature map group acquisition module, a feature map generation module and a feature map generation module, wherein the initial feature map group acquisition module is used for acquiring an initial feature map group, and the initial feature map group comprises a specified number of feature maps;
the loop difference processing module is used for carrying out feature extraction on the initial feature image group by adopting a loop difference to obtain a first feature image group, wherein the pixel point in each feature image in the first feature image group has linkage;
the feature fusion module is used for carrying out random order scrambling on the first feature image group and fusing the first feature image group subjected to order scrambling with the initial feature image group to obtain image features;
the loop-back differential processing module comprises: the channel compression unit is used for inputting the initial characteristic image group into a first convolution layer, and carrying out channel compression on the initial characteristic image group through the first convolution layer according to a designated multiple to obtain a compressed characteristic image group, wherein the number of characteristic images in the compressed characteristic image group is smaller than that of the initial characteristic image group;
the loop difference unit is used for carrying out feature extraction on the compressed feature image group by adopting a loop difference mode to obtain the first feature image group;
the loop difference unit is further used for equally dividing the compressed characteristic image group to obtain two sub-characteristic image groups, and inputting each sub-characteristic image group into a clockwise loop difference layer and a counterclockwise loop difference layer respectively;
performing clockwise loop difference processing on the input sub-feature image group through the clockwise loop difference layer to obtain a first loop difference processing feature image group;
performing anticlockwise loop-back differential processing on the input sub-feature image group through the anticlockwise loop-back differential layer to obtain a second loop-back differential processing feature image group;
and carrying out channel splicing on the first loop differential processing characteristic image group and the second loop differential processing characteristic image group to obtain the first characteristic image group, wherein the number of channels of the first characteristic image group is the same as that of channels of the compression characteristic image group.
7. A computer device, the device comprising:
one or more processors;
storage means for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-5.
8. A storage medium having stored thereon computer program, characterized in that the program when executed by a processor implements the method according to any of claims 1-5.
CN202311236293.9A 2023-09-25 2023-09-25 Image feature extraction method, device, equipment and storage medium Active CN116994002B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311236293.9A CN116994002B (en) 2023-09-25 2023-09-25 Image feature extraction method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311236293.9A CN116994002B (en) 2023-09-25 2023-09-25 Image feature extraction method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN116994002A CN116994002A (en) 2023-11-03
CN116994002B true CN116994002B (en) 2023-12-19

Family

ID=88528601

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311236293.9A Active CN116994002B (en) 2023-09-25 2023-09-25 Image feature extraction method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116994002B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110163193A (en) * 2019-03-25 2019-08-23 腾讯科技(深圳)有限公司 Image processing method, device, computer readable storage medium and computer equipment
CN115937537A (en) * 2022-12-08 2023-04-07 京北方信息技术股份有限公司 Intelligent identification method, device and equipment for target image and storage medium
CN116152523A (en) * 2022-12-06 2023-05-23 马上消费金融股份有限公司 Image detection method, device, electronic equipment and readable storage medium
CN116740355A (en) * 2023-06-15 2023-09-12 中国第一汽车股份有限公司 Automatic driving image segmentation method, device, equipment and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111145177B (en) * 2020-04-08 2020-07-31 浙江啄云智能科技有限公司 Image sample generation method, specific scene target detection method and system thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110163193A (en) * 2019-03-25 2019-08-23 腾讯科技(深圳)有限公司 Image processing method, device, computer readable storage medium and computer equipment
CN116152523A (en) * 2022-12-06 2023-05-23 马上消费金融股份有限公司 Image detection method, device, electronic equipment and readable storage medium
CN115937537A (en) * 2022-12-08 2023-04-07 京北方信息技术股份有限公司 Intelligent identification method, device and equipment for target image and storage medium
CN116740355A (en) * 2023-06-15 2023-09-12 中国第一汽车股份有限公司 Automatic driving image segmentation method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN116994002A (en) 2023-11-03

Similar Documents

Publication Publication Date Title
CN110020620B (en) Face recognition method, device and equipment under large posture
CN111480169B (en) Method, system and device for pattern recognition
CN112434721A (en) Image classification method, system, storage medium and terminal based on small sample learning
WO2019075130A1 (en) Image processing method and processing device
KR20220038475A (en) Video content recognition method and apparatus, storage medium, and computer device
CN108596944A (en) A kind of method, apparatus and terminal device of extraction moving target
CN111881804B (en) Posture estimation model training method, system, medium and terminal based on joint training
CN113704531A (en) Image processing method, image processing device, electronic equipment and computer readable storage medium
Huang et al. RGB-D salient object detection by a CNN with multiple layers fusion
US10769784B2 (en) Image analyzing method and electrical device
JP7282474B2 (en) Encryption mask determination method, encryption mask determination device, electronic device, storage medium, and computer program
CN111160225B (en) Human body analysis method and device based on deep learning
CN114821096A (en) Image processing method, neural network training method and related equipment
CN116778935A (en) Watermark generation, information processing and audio watermark generation model training method and device
CN113222813A (en) Image super-resolution reconstruction method and device, electronic equipment and storage medium
CN108520532B (en) Method and device for identifying motion direction of object in video
CN117952941A (en) Circuit board defect detection method and device
CN116994002B (en) Image feature extraction method, device, equipment and storage medium
CN117315758A (en) Facial expression detection method and device, electronic equipment and storage medium
CN111967478A (en) Feature map reconstruction method and system based on weight inversion, storage medium and terminal
CN116824609A (en) Document format detection method and device and electronic equipment
CN116226852A (en) Mobile platform malicious software detection method and device based on multi-mode information fusion
CN113610856B (en) Method and device for training image segmentation model and image segmentation
CN115810152A (en) Remote sensing image change detection method and device based on graph convolution and computer equipment
CN112801045B (en) Text region detection method, electronic equipment and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant