CN116129198B - Multi-domain tire pattern image classification method, system, medium and equipment - Google Patents
Multi-domain tire pattern image classification method, system, medium and equipment Download PDFInfo
- Publication number
- CN116129198B CN116129198B CN202310382056.7A CN202310382056A CN116129198B CN 116129198 B CN116129198 B CN 116129198B CN 202310382056 A CN202310382056 A CN 202310382056A CN 116129198 B CN116129198 B CN 116129198B
- Authority
- CN
- China
- Prior art keywords
- features
- domain
- memory
- tire pattern
- holographic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000013145 classification model Methods 0.000 claims abstract description 15
- 230000000007 visual effect Effects 0.000 claims abstract description 14
- 238000009826 distribution Methods 0.000 claims abstract description 12
- 230000005540 biological transmission Effects 0.000 claims abstract description 10
- 238000010276 construction Methods 0.000 claims abstract description 10
- 230000008569 process Effects 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims abstract description 7
- 230000015654 memory Effects 0.000 claims description 88
- 230000006870 function Effects 0.000 claims description 11
- 230000007246 mechanism Effects 0.000 claims description 11
- 238000000354 decomposition reaction Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000009977 dual effect Effects 0.000 claims description 6
- 208000000044 Amnesia Diseases 0.000 claims description 5
- 208000026139 Memory disease Diseases 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 5
- 230000006984 memory degeneration Effects 0.000 claims description 5
- 208000023060 memory loss Diseases 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 238000003860 storage Methods 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 2
- 238000005457 optimization Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of image processing, and provides a multi-domain tire pattern image classification method, a system, a medium and equipment, aiming at the problem of multi-scene tire pattern image classification, the proposal is as follows: classifying the tire pattern image data set and the trained tire pattern classification model to obtain classification probability distribution to obtain tire pattern image classification results; the construction process of the tire pattern classification model comprises the following steps: dividing the tire pattern image data set into a source domain and a target domain; extracting features of the tire pattern image data to obtain global features and local features of the tire pattern image; based on the global features and the local features, a double-memory module is constructed, and visual knowledge and cross-domain transmission knowledge of the local features in two domains are stored through the double-memory module, so that holographic features containing information of the two domains are constructed by utilizing the features of one domain. The universality and the noise resistance are improved.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a multi-domain tire pattern image classification method, a multi-domain tire pattern image classification system, a multi-domain tire pattern image classification medium and multi-domain tire pattern image classification equipment.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
The tire pattern image classification task is a task of extracting features of tire pattern images by using a deep learning technology and classifying the tire pattern images by using a classifier according to probability distribution calculated by the extracted features, and is a basic task in the field of computer vision. The task also has a wide range of application scenarios in production and life, such as the need to classify tire patterns left on site, which is definitely inefficient and laborious if the pictures are classified manually due to the wide variety of vehicles. The task can effectively alleviate the problems, and greatly saves manpower and material resources on the basis of guaranteeing classification accuracy.
However, there are also certain difficulties in the investigation of tire pattern classification tasks, particularly for the following reasons:
1) There is a great difference in the tire pictures taken in different time domains. For the tire pictures shot in the same time period, such as the pictures shot in noon, the displayed pattern features are clear, the textures are clear, the negative shadow areas represented in the tire pictures are fewer, good classification effects can be obtained only by relying on the existing classical deep learning classification model, but the tire pictures shot in the timing period have no generality. For tire pattern pictures in actual application scenes, the shooting time is often very random, the brightness requirements of the tire pattern images on shooting environments are very high, different brightness can cause the texture features of the tire pattern images to show very large differences, and different background noise can be generated due to the influences of the shooting environments and the brightness, so that great difficulty is brought to classification of the tire pattern images.
2) The tire patterns photographed in different spatial domains are greatly different. Under the actual application scene, the acquisition place of the tire pattern pictures has extremely high instability, and the angles of the pictures acquired by photographers are different, so that a large number of tire pattern pictures with extremely high difference are generated, and when the tire pattern pictures are classified, the model is required to accurately identify the tire pattern pictures shot in different environments, and the pictures shot in different angles are required to have extremely high identification degree, so that the model is required to have extremely high noise resistance.
3) A single tire image contains multiple types of patterns. Because the road sections of the image acquisition are different, multiple vehicles can possibly pass through the acquisition points, which causes another problem of classifying the multiple images, the image often contains more complex characteristics, the noise resistance requirement on the model is very high, and the classifying performance of the current classifying model on the image still needs to be improved.
In view of the above challenges, existing methods propose a multi-period tire pattern image classification method that successfully improves the performance of one picture (e.g., a tire pattern picture taken in dusk) by transferring knowledge from a picture in another scene (e.g., a tire pattern picture taken in noon), but they have limitations in general because the source and target fields are taken at fixed angles. On the other hand, the detection performance of the multi-tire pattern image cannot reach the expected performance, which proves that the anti-noise performance of the model is not good, and the complicated tire image cannot be correctly classified.
Disclosure of Invention
In order to solve at least one technical problem in the background technology, the invention provides a multi-domain tire pattern image classification method, a multi-domain tire pattern image classification system, a multi-domain tire pattern image classification medium and multi-domain tire pattern image classification equipment, which can greatly inhibit background noise of pictures through multi-loss optimization by learning characteristics of target images from multiple angles, so that visual characteristics learned by a model are more comprehensive.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
a first aspect of the present invention provides a multi-domain tire pattern image classification method comprising the steps of:
acquiring a tire pattern image dataset;
classifying the tire pattern image data set and the trained tire pattern classification model to obtain classification probability distribution to obtain tire pattern image classification results;
the construction process of the tire pattern classification model comprises the following steps:
dividing the tire pattern image data set into a source domain and a target domain; extracting features of the tire pattern image data to obtain global features and local features of the tire pattern image; based on the global features and the local features, a double-memory module is constructed, and visual knowledge and cross-domain transmission knowledge of the local features in two domains are stored through the double-memory module, so that holographic features containing information of the two domains are constructed by utilizing the features of one domain.
Further, the feature extraction of the tire pattern image data to obtain the global feature and the local feature of the tire pattern image includes:
carrying out feature extraction on the tire pattern picture data by adopting a pre-training neural network to obtain integral image features;
carrying out average value pool processing on the integral image characteristics;
introducing a self-attention mechanism to encode the image characteristics processed by the mean value pool to obtain image encoding characteristics;
based on the image coding features, factorization is carried out by adopting a depth residual mapping method, and global features and local features of the tire pattern image are obtained.
Further, the process of constructing the dual memory module includes:
the mass center of the distinguishing characteristics of different tire pictures is used as a basic component of the memory;
in the basic component of each memory, each centroid is initialized to a mean vector of the local features of all tread image samples belonging to the respective class.
Further, the method for storing the visual knowledge of the local features in two domains and transmitting the knowledge across domains by using the dual memory module, so as to construct the holographic features containing the information of the two domains by using the features of one domain specifically comprises the following steps:
constructing and obtaining heterogeneous holographic features based on local features of a source domain memory module and global image features of a target domain;
based on the local characteristics of the memory module of the source domain or the target domain and the global characteristics thereof, constructing and obtaining homogeneous holographic characteristics;
and splicing the heterogeneous holographic features and the homogeneous holographic features to obtain the holographic features.
Further, the constructing to obtain the heterogeneous holographic feature based on the local feature of the source domain memory module and the global image feature of the target domain includes:
based on the local characteristics of a given domain, obtaining the memory characteristics of the opposite domain by retrieving the memory;
adopting a memory selector based on an attention mechanism to obtain the participated heterogeneous memory characteristics by self-adaptive selection in a soft mode;
and combining the memory characteristics of the opposite domains with the participating heterogeneous memory characteristics to obtain the heterogeneous holographic characteristics.
Further, the constructing the homogeneous holographic feature based on the local feature and the global feature of the memory module of the source domain or the target domain includes:
based on the given global feature, obtaining global memory features by retrieving the memory;
adopting a memory selector based on an attention mechanism to obtain the participated homogeneous memory characteristics in a soft mode in a self-adaptive mode;
and combining the global memory characteristics with the participated homogeneous memory characteristics to obtain homogeneous holographic characteristics.
Further, the loss function when the tire pattern classification model is trained is as follows:
wherein,,to counter the balance weight of the loss, +.>Balance weight for memory loss, +.>For the discrimination loss of holographic features, < >>For discriminating loss of local features, < >>To optimize the contrast loss of the domain arbiter and the dynamic holographic module,is good atDissolving the fight loss function of the decomposition module +.>Is a memory penalty.
A second aspect of the present invention provides a multi-domain tire pattern image classification system comprising:
a data acquisition module configured to: acquiring a tire pattern image dataset;
an image classification module configured to: classifying the tire pattern image data set and the trained tire pattern classification model to obtain classification probability distribution to obtain tire pattern image classification results;
the construction process of the tire pattern classification model comprises the following steps:
dividing the tire pattern image data set into a source domain and a target domain; extracting features of the tire pattern image data to obtain global features and local features of the tire pattern image; based on the global features and the local features, a double-memory module is constructed, and visual knowledge and cross-domain transmission knowledge of the local features in two domains are stored through the double-memory module, so that holographic features containing information of the two domains are constructed by utilizing the features of one domain.
A third aspect of the present invention provides a computer-readable storage medium.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of a multi-domain tire pattern image classification method according to the first aspect.
A fourth aspect of the invention provides an electronic device.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in a multi-domain tire pattern image classification method according to the first aspect when the program is executed.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention obtains the global feature and the local feature of the tire pattern image by carrying out feature extraction on the tire pattern image data; based on the global features and the local features, a double-memory module is constructed, visual knowledge and cross-domain transmission knowledge of the local features in two domains are stored through the double-memory module, and therefore holographic features containing information of the two domains are constructed by utilizing the features of one domain; compared with the previous method, the tire pattern classification method has the advantages that the accuracy of tire pattern classification tasks is improved to a certain extent, and the universality and the noise resistance are improved.
2. Aiming at the problem of classifying multi-scene tire pattern pictures, the invention provides a multi-domain tire pattern image classifying method which can learn the characteristics of a target image from multiple angles, and can greatly inhibit the background noise of the pictures through multi-loss optimization, so that the visual characteristics learned by a model are more comprehensive.
3. Aiming at the problem of single-picture multi-pattern, the invention can learn by setting a single-pattern tire image as a source domain and a multi-pattern image as a target domain, thereby greatly increasing the intake of useful information and interfering the influence of patterns.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
FIG. 1 is a flow chart of a multi-domain tire pattern image classification method provided by an embodiment of the invention.
Detailed Description
The invention will be further described with reference to the drawings and examples.
It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
Interpretation of the terms
Source domain: (source domain) represents a different domain than the test sample, but has rich supervisory information;
target domain: (target domain) indicates the domain in which the test sample is located, with no label or only a small number of labels.
The source domain and the target domain often belong to the same class of tasks, but are distributed differently.
Example 1
As shown in fig. 1, the present embodiment provides a multi-domain tire pattern image classification method, which includes the following steps:
step 1: acquiring tire pattern image data of different scenes and shooting angles, so that tire pattern pictures under different scenes and different times are respectively classified;
and randomly selecting two types of tire pattern picture data of different scenes and shooting angles to be placed in a source domain and a target domain.
Tire pattern picture assuming target areaAnd the picture set of the source domain->。
Wherein,,representing the +.>Picture (or->Representation->Corresponding pattern types of tiresTo represent one of two data sets, wherein +.>Representing the picture domain index.
Step 2: extracting features of tire pattern image data by using ResNet-152 pre-trained on ImageNet to obtain integral image features, and carrying out average value pool processing on images processed by a pre-training network to obtain feature embedding dimension as followsImage characteristics of->，/>Is a CNN feature dimension.
Wherein,,for integral image features->And->For the features of the ith and jth image, -/->And->Respectively 3 1 x 1 convolution layers, < >>For the attention score, the relationship between the jth image and the ith image is expressed, and T represents processing T pictures at a time.
Using trainable parametersThe participating features are added back to the original features and then pooling is used to compute a representation of the input features:
in order to obtain fine grain characteristic representation of the texture characteristics of the tire pattern picture, a self-attention mechanism is introduced to the preprocessed image characteristics, and the image characteristics after the mean value pool processing are encoded to obtain image encoding characteristics. Thus, the final image coding feature is obtained.
Step 3: and factorizing the final overall image features by adopting a depth residual mapping method to obtain global features and local texture features of the tire pattern image so as to transmit specific knowledge among different features.
In order to pass knowledge between different domains, it is necessary to put the knowledge in a stateDecomposition into local features->And global features->This allows domain-specific knowledge to be passed between different domains to construct holographic features.
The depth residual mapping method is adopted to decompose the characteristics, and a residual decomposition module is as follows:
wherein,,representing local features->Representing global features->Is made up of parameters->A nonlinear transducer composed of a fully attached layer and a tanh activation layer.
Step 4: building dual memory modulesAnd->And respectively storing the local characteristics of all tire pattern categories in the two domains through the double memory modules.
The mass center of the distinguishing characteristics of different tire pictures is used as a basic component of the memory, so thatVisual feature memory representing target domain training data;
in a similar manner to that described above,visual characteristics memory representing the source domain.
Wherein K is the tire pattern number, each centroidKnowledge of local features representing the ith tire class held in the target domain, each centroid +.>Knowledge of the local features of the ith tire class stored in the source domain is shown.
At the position ofAnd->Each centroid being initialized to a mean vector of local features of all tire tread image samples belonging to the respective class:
wherein,,is a support set containing training samples of all markers in field v of the ith tire tread category,/->Is thatLocal features of the j-th picture in (a).
When constructing the holographic features of the source domain, the corresponding features of the target domain are required to be supplemented, and then the realization is proposed to save the local features of each domain through a memory block, wherein the dual memory module comprises two memory blocks for saving the visual knowledge of the local features of the image and the cross-domain transmission knowledge in the two domains.
Step 5: the method for constructing the holographic feature containing the information of the two domains by using the characteristics of one domain comprises the steps of:
step 501: combining knowledge from local features of a source domain memory module and global image features of a target domain to construct heterogeneous holographic features, wherein the method specifically comprises the following steps:
step 5011: given local featuresBy retrieving memory->Obtain memory characteristics of opposite domains->The operation is as follows:
Norm(/>),/>
wherein,,is a learnable linear transducer constructed based on a fully connected layer and is used for calculating holographic coefficients +.>By retrieving the memory of the opposite domain +.>Norm is a regularization function, a Softmax function.
Step 5012: a memory selector based on an attention mechanism is designed, and important memories are adaptively selected in a soft mode:
wherein,,is a learnable linear transducer constructed based on fully connected layers>Representing multiplication of elements>Is a participating heterogeneous memory feature.
Step 5013: bonding ofAnd->Construction of heterogeneous holographic features->The following is shown:
step 502: combining knowledge from local features of a memory module of a source domain or a target domain and global features thereof to construct homogeneous holographic features, wherein the method specifically comprises the following steps:
step 5021: given global featuresBy retrieving memory->Obtain global memory feature->The operation is as follows:
Norm(/>),/>
wherein,,is a learnable linear transducer constructed based on a fully connected layer and is used for calculating holographic coefficients +.>By retrieving domain memory->Norm is a regularization function, a Softmax function.
Step 5022: a memory selector based on an attention mechanism is designed, and important memories are adaptively selected in a soft mode:
wherein,,is a learnable linear transducer constructed based on fully connected layers>Representing multiplication of elements>Is a participated homogeneous memory feature.
Step 5023: bonding ofAnd->Construction of homogeneous holographic features->The following is shown:
step 503: and splicing the heterogeneous holographic features and the homogeneous holographic features to obtain holographic features, wherein the holographic features comprise image features from a source domain and a target domain.
The holographic features were constructed as follows:
wherein,,representing vector concatenation, wherein +_in the above formula>And->Image information containing a target field is provided,and->Image information containing a source field.
Step 6: based on holographic features, a tire pattern classification module is designed, which generates a final probability distribution through a Softmax full-connection layer.
Based on holographic featuresA classification module is created>The module can generate a final classification probability distribution through the fully connected layer with Softmax.
In the multi-classification recognition task, the same classification module is used to respectively base onAnd->Calculating classification scores of the source domain and the target domain, < ->And->Share the same holographic feature space.
Step 7: using a weighted sum of multiple loss functions as the total loss for the method, the losses involved are as follows:
wherein,,to counter the balance weight of the loss, +.>Balance weight for memory loss, +.>For the discrimination loss of holographic features, < >>For discriminating loss of local features, < >>To optimize the contrast loss of the domain arbiter and the dynamic holographic module,to optimize the fight loss function of the decomposition module +.>Is a memory penalty.
Application ofAnd->Make holographic feature->And local features->The discrimination is provided for different tire types. The dynamic holographic module is connected with the classification module to construct holographic characteristics and predict the probability distribution y of final classification.
Wherein,,constraint using cross entropy loss:
wherein,,is a group-trunk type tag +.>Is encoded by one-hot->Representation->。
Discrimination loss: discrimination loss makes the holographic and local features of different domains distinguishable.
For both domains, the local features of the domain are specified with negative log likelihood of the group-trunk tire class:
wherein,,by being based on local features->Classification module of->Generate (I)>。
Countering losses: the global features of different domains share a global feature space, and the holographic features of different domains share a holographic feature space.
By limiting the dynamic holographic block so that a trained arbiter cannot reliably predict domain labels of holographic features. This is achieved by gradient inversion layer (GRL) and domain discriminatorsRealized on the basis of holographic features->Predictive domain label->。
Optimizing a domain discriminator and a dynamic hologram module by using cross entropy loss:
wherein,,is a group-trunk domain label +.>Is described.
Creating another GRL layer and view discriminatorIt is based on->Predictive domain label->。
The fight loss function of the optimization decomposition module is defined as follows:
wherein,,is a group-trunk domain label +.>Is described.
Memory loss: to optimize the dual memory module, local feature information is updated by minimizing intra-class distances and inter-class distances between view-specific features and each memory centroid to ensure intra-class compactness and inter-class differentiation.
Specifically, there are two components of memory loss:
wherein,,and->The similarity loss and the inter-class large marginal loss are respectively. Wherein:
wherein,,for manually defined boundary factors +.>Is based on image->Local features calculated by feature coding and decomposition module, < >>For specific domain->Middle->The characteristic memory centroid of each tire class.
In this embodiment, the final classification results include "dandulp 1956016", "dongfeng 165R13", "kantai 2056516", and the like.
Example two
The present embodiment provides a multi-domain tire pattern image classification system, comprising:
a data acquisition module configured to: acquiring a tire pattern image dataset;
an image classification module configured to: classifying the tire pattern image data set and the trained tire pattern classification model to obtain classification probability distribution to obtain tire pattern image classification results;
the construction process of the tire pattern classification model comprises the following steps:
dividing the tire pattern image data set into a source domain and a target domain; extracting features of the tire pattern image data to obtain global features and local features of the tire pattern image; based on the global features and the local features, a double-memory module is constructed, and visual knowledge and cross-domain transmission knowledge of the local features in two domains are stored through the double-memory module, so that holographic features containing information of the two domains are constructed by utilizing the features of one domain.
Example III
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a multi-domain tire pattern image classification method of the first embodiment.
Example IV
The present embodiment provides an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps in the multi-domain tire pattern image classification method of the first embodiment when executing the program.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. The multi-domain tire pattern image classification method is characterized by comprising the following steps of:
acquiring a tire pattern image dataset;
classifying the tire pattern image data set and the trained tire pattern classification model to obtain classification probability distribution to obtain tire pattern image classification results;
the construction process of the tire pattern classification model comprises the following steps:
dividing the tire pattern image data set into a source domain and a target domain; extracting features of the tire pattern image data to obtain global features and local features of the tire pattern image; based on the global features and the local features, a double-memory module is constructed, visual knowledge and cross-domain transmission knowledge of the local features in two domains are stored through the double-memory module, and therefore holographic features containing information of the two domains are constructed by utilizing the features of one domain;
the method for saving the visual knowledge and the cross-domain transmission knowledge of the local features in two domains by the double memory module, thereby constructing the holographic features containing the information of the two domains by utilizing the features of one domain comprises the following steps:
constructing and obtaining heterogeneous holographic features based on local features of a source domain memory module and global image features of a target domain;
based on the local characteristics of the memory module of the source domain or the target domain and the global characteristics thereof, constructing and obtaining homogeneous holographic characteristics; splicing the heterogeneous holographic features and the homogeneous holographic features to obtain holographic features;
the heterogeneous holographic feature is constructed based on the local feature of the source domain memory module and the global image feature of the target domain, and comprises the following steps:
based on the local characteristics of a given domain, obtaining the memory characteristics of the opposite domain by retrieving the memory; adopting a memory selector based on an attention mechanism to obtain the participated heterogeneous memory characteristics by self-adaptive selection in a soft mode; combining the memory characteristics of the opposite domains with the participating heterogeneous memory characteristics to obtain heterogeneous holographic characteristics;
the construction of the local feature and the global feature of the memory module based on the source domain or the target domain to obtain the homogeneous holographic feature comprises the following steps:
based on the given global feature, obtaining global memory features by retrieving the memory; adopting a memory selector based on an attention mechanism to obtain the participated homogeneous memory characteristics in a soft mode in a self-adaptive mode; combining global memory characteristics and participated homogeneous memory characteristics to obtain homogeneous holographic characteristics;
based on holographic features, a tire pattern classification module is designed that generates a final probability distribution through a Softmax full tie layer, comprising:
in the multi-classification recognition task, the same classification module is used for calculating classification scores of a source domain and a target domain based on holographic features respectively, and the holographic features of different domains share the same holographic feature space.
2. A multi-domain tire pattern image classification method as in claim 1, wherein said feature extraction of tire pattern image data to obtain global features and local features of tire pattern images comprises:
carrying out feature extraction on the tire pattern picture data by adopting a pre-training neural network to obtain integral image features;
carrying out average value pool processing on the integral image characteristics;
introducing a self-attention mechanism to encode the image characteristics processed by the mean value pool to obtain image encoding characteristics;
based on the image coding features, factorization is carried out by adopting a depth residual mapping method, and global features and local features of the tire pattern image are obtained.
3. A multi-domain tire tread image classification method as in claim 1, wherein said process of constructing a dual memory module comprises:
the mass center of the distinguishing characteristics of different tire pictures is used as a basic component of the memory;
in the basic component of each memory, each centroid is initialized to a mean vector of the local features of all tread image samples belonging to the respective class.
4. A multi-domain tire pattern image classification method as claimed in claim 1, wherein the loss function of the tire pattern classification model when trained is:
wherein,,to counter the balance weight of the loss, +.>Balance weight for memory loss, +.>For the discrimination loss of holographic features, < >>For discriminating loss of local features, < >>To optimize the contrast loss of the domain arbiter and the dynamic holographic module +.>To optimize the fight loss function of the decomposition module +.>Is a memory penalty.
5. A multi-domain tire pattern image classification system, comprising:
a data acquisition module configured to: acquiring a tire pattern image dataset;
an image classification module configured to: classifying the tire pattern image data set and the trained tire pattern classification model to obtain classification probability distribution to obtain tire pattern image classification results;
the construction process of the tire pattern classification model comprises the following steps:
dividing the tire pattern image data set into a source domain and a target domain; extracting features of the tire pattern image data to obtain global features and local features of the tire pattern image; based on the global features and the local features, a double-memory module is constructed, visual knowledge and cross-domain transmission knowledge of the local features in two domains are stored through the double-memory module, and therefore holographic features containing information of the two domains are constructed by utilizing the features of one domain;
the method for saving the visual knowledge and the cross-domain transmission knowledge of the local features in two domains by the double memory module, thereby constructing the holographic features containing the information of the two domains by utilizing the features of one domain comprises the following steps:
constructing and obtaining heterogeneous holographic features based on local features of a source domain memory module and global image features of a target domain; based on the local characteristics of the memory module of the source domain or the target domain and the global characteristics thereof, constructing and obtaining homogeneous holographic characteristics; splicing the heterogeneous holographic features and the homogeneous holographic features to obtain holographic features;
the heterogeneous holographic feature is constructed based on the local feature of the source domain memory module and the global image feature of the target domain, and comprises the following steps:
based on the local characteristics of a given domain, obtaining the memory characteristics of the opposite domain by retrieving the memory; adopting a memory selector based on an attention mechanism to obtain the participated heterogeneous memory characteristics by self-adaptive selection in a soft mode; combining the memory characteristics of the opposite domains with the participating heterogeneous memory characteristics to obtain heterogeneous holographic characteristics;
the construction of the local feature and the global feature of the memory module based on the source domain or the target domain to obtain the homogeneous holographic feature comprises the following steps:
based on the given global feature, obtaining global memory features by retrieving the memory; adopting a memory selector based on an attention mechanism to obtain the participated homogeneous memory characteristics in a soft mode in a self-adaptive mode; combining global memory characteristics and participated homogeneous memory characteristics to obtain homogeneous holographic characteristics;
based on holographic features, a tire pattern classification module is designed that generates a final probability distribution through a Softmax full tie layer, comprising:
in the multi-classification recognition task, the same classification module is used for calculating classification scores of a source domain and a target domain based on holographic features respectively, and the holographic features of different domains share the same holographic feature space.
6. A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of a multi-domain tire pattern image classification method as claimed in any one of claims 1-4.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor performs the steps of a multi-domain tire pattern image classification method as claimed in any one of claims 1-4 when the program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310382056.7A CN116129198B (en) | 2023-04-12 | 2023-04-12 | Multi-domain tire pattern image classification method, system, medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310382056.7A CN116129198B (en) | 2023-04-12 | 2023-04-12 | Multi-domain tire pattern image classification method, system, medium and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116129198A CN116129198A (en) | 2023-05-16 |
| CN116129198B true CN116129198B (en) | 2023-07-18 |
Family
ID=86310317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310382056.7A Active CN116129198B (en) | 2023-04-12 | 2023-04-12 | Multi-domain tire pattern image classification method, system, medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116129198B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114387207A (en) * | 2021-12-02 | 2022-04-22 | 浙江大学 | Tire flaw detection method and model based on self-attention mechanism and dual-field self-adaptation |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20230071240A1 (en) * | 2021-09-03 | 2023-03-09 | Gopi Krishnan RAJBAHADUR | Methods, systems, and media for robust classification using active learning and domain knowledge |
| CN114511521A (en) * | 2022-01-21 | 2022-05-17 | 浙江大学 | Tire defect detection method based on multiple representations and multiple sub-field self-adaption |
| CN114882278A (en) * | 2022-05-09 | 2022-08-09 | 浙江理工大学 | Tire pattern classification method and device based on attention mechanism and transfer learning |
| CN114821282B (en) * | 2022-06-28 | 2022-11-04 | 苏州立创致恒电子科技有限公司 | Image detection device and method based on domain antagonistic neural network |
| CN115493511A (en) * | 2022-08-09 | 2022-12-20 | 北京理工大学 | Three-dimensional imaging method based on super-surface holographic projection |
| CN115661484A (en) * | 2022-12-16 | 2023-01-31 | 山东建筑大学 | Lightweight tire pattern classification method and system |
-
2023
- 2023-04-12 CN CN202310382056.7A patent/CN116129198B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114387207A (en) * | 2021-12-02 | 2022-04-22 | 浙江大学 | Tire flaw detection method and model based on self-attention mechanism and dual-field self-adaptation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116129198A (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110414432B (en) | Training method of object recognition model, object recognition method and corresponding device | |
| CN111709311B (en) | Pedestrian re-identification method based on multi-scale convolution feature fusion | |
| CN112069896B (en) | Video target tracking method based on twin network fusion multi-template features | |
| CN110598029A (en) | Fine-grained image classification method based on attention transfer mechanism | |
| CN113761259A (en) | Image processing method and device and computer equipment | |
| CN109919073B (en) | Pedestrian re-identification method with illumination robustness | |
| CN114358188A (en) | Feature extraction model processing method, feature extraction model processing device, sample retrieval method, sample retrieval device and computer equipment | |
| CN109871892A (en) | A kind of robot vision cognitive system based on small sample metric learning | |
| SG171858A1 (en) | A method for updating a 2 dimensional linear discriminant analysis (2dlda) classifier engine | |
| WO2021243947A1 (en) | Object re-identification method and apparatus, and terminal and storage medium | |
| CN113095370A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN112733602B (en) | Relation-guided pedestrian attribute identification method | |
| CN114842343A (en) | ViT-based aerial image identification method | |
| CN108960270A (en) | A kind of data scaling method and system based on manifold transfer learning | |
| CN115830531A (en) | Pedestrian re-identification method based on residual multi-channel attention multi-feature fusion | |
| CN111126155B (en) | Pedestrian re-identification method for generating countermeasure network based on semantic constraint | |
| CN114780767A (en) | Large-scale image retrieval method and system based on deep convolutional neural network | |
| Bi et al. | Critical direction projection networks for few-shot learning | |
| CN113283320A (en) | Pedestrian re-identification method based on channel feature aggregation | |
| CN116129198B (en) | Multi-domain tire pattern image classification method, system, medium and equipment | |
| CN115527064A (en) | Toxic mushroom fine-grained image classification method based on multi-stage ViT and contrast learning | |
| CN117011219A (en) | Method, apparatus, device, storage medium and program product for detecting quality of article | |
| CN116958624A (en) | Method, device, equipment, medium and program product for identifying appointed material | |
| Da et al. | Remote sensing image ship detection based on improved YOLOv3 | |
| CN114998604A (en) | Point cloud feature extraction method based on local point cloud position relation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |