CN113128441A - System and method for identifying vehicle weight by embedding structure of attribute and state guidance - Google Patents

System and method for identifying vehicle weight by embedding structure of attribute and state guidance Download PDF

Info

Publication number
CN113128441A
CN113128441A CN202110465670.0A CN202110465670A CN113128441A CN 113128441 A CN113128441 A CN 113128441A CN 202110465670 A CN202110465670 A CN 202110465670A CN 113128441 A CN113128441 A CN 113128441A
Authority
CN
China
Prior art keywords
attribute
vehicle
state
feature
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110465670.0A
Other languages
Chinese (zh)
Other versions
CN113128441B (en
Inventor
郑爱华
高亚飞
李洪潮
李成龙
汤进
罗斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui University
Original Assignee
Anhui University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui University filed Critical Anhui University
Priority to CN202110465670.0A priority Critical patent/CN113128441B/en
Publication of CN113128441A publication Critical patent/CN113128441A/en
Application granted granted Critical
Publication of CN113128441B publication Critical patent/CN113128441B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

An attribute and state guided structure embedded vehicle weight recognition system and method belongs to the technical field of computer vision, solves the problems of intra-class difference between the same vehicles and inter-class difference between different vehicles in vehicle weight recognition, and learns the characteristics with discrimination in vehicle weight recognition through attribute-based enhancement and state-based weakening, and comprises the following steps: a residual network module, an attribute-based enhancement and expansion module, a state-based attenuation and contraction module and a global structure embedding module; the method comprises two stages: a training stage and a testing stage; the characteristic spacing between different vehicles is increased by the enhancement and expansion of the attributes, and the characteristic spacing between the same vehicles is reduced by the weakening and contraction of the state, so that the characteristics of the vehicles are better learned by a network.

Description

System and method for identifying vehicle weight by embedding structure of attribute and state guidance
Technical Field
The invention belongs to the technical field of computer vision, and relates to an attribute and state guided structure embedded vehicle weight recognition system and method.
Background
The task of vehicle weight recognition is to match vehicle images in non-overlapping surveillance cameras. Because of its wide application in the fields of video surveillance, social security, smart cities and smart traffic, it is an active and challenging computer vision task and has attracted extensive attention. Despite recent breakthroughs in vehicle re-identification, it still faces two serious challenges.
As shown in fig. 1(a) and (b), there is a large intra-class difference between the same vehicle pictures in different states (e.g., different camera angles, vehicle viewing angles, and shooting times). As shown in fig. 1(b), (c) and (d), there are slight inter-class differences between different vehicles, and especially two vehicles have the same or similar attributes, such as color, model, manufacturer, etc. Therefore, solving the intra-class differences between the same vehicles and the inter-class differences between different vehicles has become an important task in vehicle weight recognition.
Existing solutions present different approaches to address the two challenges described above. Representative methods are mainly classified into four categories: 1) extracting global manual/depth features of the vehicle image through a specific metric learning method based on a global feature method; 2) the method based on the path generally adopts space-time information to remove unreasonable vehicles in the inference stage, and refines the retrieval result; 3) perspective-based methods, the objective of which is to handle perspective changes through metric learning and learn the multi-perspective features of the vehicle Re-ID; 4) the method based on local information enhancement enhances the inter-class difference in vehicle weight recognition by providing some stable and discriminative clues.
However, in the two major challenges of the vehicle weight recognition that there is a large intra-class difference between the same vehicles and a small inter-class difference between different vehicles, the existing solutions have the following disadvantages:
1) global feature-based approaches only consider the appearance of the vehicle image, and it is often difficult to capture intra-class similarities and inter-class differences;
2) the route-based method ignores appearance changes of the vehicle caused by space-time changes in the learning stage of vehicle characteristics;
3) perspective-based approaches, while significantly reducing intra-class variation, they ignore intrinsic state factors of the vehicle (such as camera perspective and capture time), ignoring the challenge of subtle inter-class variation;
4) the method based on local information enhancement needs local region extraction operation, however, the local region extraction model usually needs a large amount of labeled data, and is time-consuming and labor-consuming.
In the prior art, a document 'vehicle re-identification method research based on multiple attributes' (Xiamen university, Like) with a publication date of 2019 discloses a vehicle re-identification algorithm fusing vehicle angles, provides a vehicle re-identification algorithm fusing vehicle colors and vehicle types, and constructs a vehicle re-identification demonstration system based on a webpage platform; however, although this document considers the attribute information of the vehicle, it only uses the attribute information directly spliced to the vehicle feature as the auxiliary information, and thus obtains a good vehicle weight recognition performance, but does not consider optimizing the attribute information and the vehicle feature at the same time. The document ' research on a multi-view sparse fusion and multi-scale attention vehicle re-identification method ' (university of Anhui, Dong's King), published in 2020, designs a vehicle re-identification network based on a multi-scale attention mechanism, extracts a vehicle feature map by using a skeleton network, generates vehicle feature maps of other scales by using a quadratic interpolation method, and sends the vehicle feature maps to corresponding sub-networks containing spatial channel attention modules respectively. After training the sub-networks separately, the multi-scale attention feature maps are fused using the joining layer, and the entire network is fine-tuned. The network obtains more robust features by fusing multi-scale complementary information and mining local details of discriminability by using an attention mechanism; but this document does not take into account the problem of information of attributes and states.
Disclosure of Invention
The invention aims to design a structure embedded vehicle weight recognition method and system for attribute and state guidance, and solve the problems of intra-class difference between the same vehicles and inter-class difference between different vehicles in vehicle weight recognition.
The invention solves the technical problems through the following technical scheme:
an attribute and state guided structure embedded vehicle weight recognition system that learns a feature having discriminative power in vehicle weight recognition through attribute-based enhancement and state-based weakening, comprising: a residual network module, an attribute-based enhancement and expansion module, a state-based attenuation and contraction module and a global structure embedding module;
the residual error network module is used for extracting a visible light characteristic diagram, copying the characteristic diagram into three parts, inputting one part of the characteristic diagram into the attribute-based enhancement and expansion module, inputting the other part of the characteristic diagram into the state-based weakening and contraction module, and obtaining category scores through global average pooling and full connection operation;
the attribute-based enhancement and expansion module is used for enhancing the recognition of vehicle features through attribute information related to identities and increasing feature distances among different vehicles by setting an attribute-based expansion loss function;
the state-based weakening and shrinking module is used for weakening state information interfering identification and reducing the inter-class feature distance through a state-based shrinking loss function;
the global structure embedding module is used for obtaining a final vehicle feature vector based on the enhancement of the attribute and the weakening operation based on the state, calculating a category score through the final feature vector, transmitting the category score to a search library and judging whether the vehicle appears in other cameras.
The technical scheme of the invention provides a method for identifying the weight of the vehicle by embedding an attribute and state-guided structure into the vehicle, which increases the characteristic distance between different vehicles by using the enhancement and expansion of the attribute, and reduces the characteristic distance between the same vehicles by using the weakening and contraction of the state, thereby better helping the network to learn the characteristics of the vehicle.
A method for embedding a structure for guiding the attribute and the state into a vehicle weight recognition system is characterized by comprising two stages: a training stage and a testing stage;
the training phase comprises the following steps:
step 1): acquiring a visible light image of a vehicle, and inputting the visible light image into the system;
step 2): extracting a visible light characteristic diagram through a residual error network, copying the characteristic diagram into three parts, inputting one part of the characteristic diagram into an enhancement and expansion module based on attributes, inputting the other part of the characteristic diagram into a weakening and contraction module based on states, and obtaining category scores through global average pooling and full connection operation; calculating the category constraint loss values of the predicted category and the true value by using a cross entropy function, sharing the category loss values of three constraints of attributes, states and identities, and adding the three according to the proportion to form a multi-label category loss function to predict the categories of the three;
step 3): the feature distribution of the attributes is expanded through the expansion loss function of the attributes, so that the difference of the attributes among the classes is increased; narrowing the distribution of states by a shrinkage loss function of the states, thereby reducing the difference of the states in the class;
step 4): using a global structure embedding loss function, training all samples through heterogeneous sample separation and homogeneous sample disturbance, so that the vehicle characteristic distance of the heterogeneous sample is increased along with the expansion of the attribute until the upper boundary is reached, and the vehicle characteristic distance of the homogeneous sample is disturbed along with the contraction of the state until the lower boundary is smaller than the lower boundary;
step 5): the final loss of the model is defined as the sum of multi-label classification loss, attribute-based expansion loss, state-based contraction loss and global structure embedding loss; in each training, the attribute and state-guided structure is embedded into the system to reversely transmit a final loss value, the loss value is reduced as a target, model parameters of the residual block and parameters of the attribute enhancing module and the state weakening module are updated by a self-adaptive momentum random gradient descent method, after multiple iterations, the loss value of the system is not reduced any more, all the parameters are optimal, and the training of the system is finished;
the testing stage comprises the following steps:
step a): loading a visible light image of a vehicle and inputting the visible light image into a trained system;
step b): extracting a visible light characteristic diagram through a trained residual error network;
step c): respectively passing the visible light characteristic diagram through a trained attribute-based enhancing and expanding module and a state-based weakening and contracting module, and obtaining an attribute enhanced characteristic tensor and a state weakened characteristic tensor;
step d): obtaining a final vehicle feature vector through attribute-based enhancement and state-based weakening operation;
step e): and calculating a category score through the final feature vector, transmitting the category score to a search library, and judging whether the vehicle appears in other cameras.
As a further improvement of the technical solution of the present invention, the method for increasing the difference between the attributes in the classes by expanding the feature distribution of the attributes through the attribute expansion loss function in step 3) comprises:
inputting a vehicle image I with an image size of 256 × 256, and acquiring a feature map T of the vehicle image through a 50-layer residual network, where T is ResNet50 (I);
copying three parts of a feature map T of the vehicle image, wherein the first part is transmitted into an attribute-based enhancement and expansion module;
then, inputting the feature map T into 1 × 1 volume blocks corresponding to different attributes to obtain a feature map related to the attributes:
Figure BDA0003043816540000041
and (3) carrying out batch regularization BN operation and modified linear unit ReLU operation on the feature graph related to the attribute:
Figure BDA0003043816540000042
carrying out global average pooling operation on the feature graph related to the attribute to obtain the feature related to the attribute:
Figure BDA0003043816540000043
introducing attribute related tags
Figure BDA0003043816540000044
Adding a full connection layer FC, so that the characteristics related to the attributes are constrained by the labels in the end-to-end training, and the constraint of the labels related to the attributes is formulated as:
Figure BDA0003043816540000045
then, the average value of the attributes of the current vehicle relative to all the vehicles is calculated
Figure BDA0003043816540000046
Is distributed over a plurality of features, | |)2Represents l2Norm square root of sum of squared features:
Figure BDA0003043816540000047
designing an extended loss function based on attributes to force the network to continuously extend the feature distribution related to the attributes in end-to-end training, wherein the extended formula is as follows:
Figure BDA0003043816540000048
wherein i represents the type of attribute, and in this embodiment, the following are respectively: a color attribute of the vehicle, a type attribute of the vehicle, a manufacturer attribute of the vehicle;
the system is trained to perform back transmission loss function towards the descending direction of the expansion loss, so that the attribute feature distribution is continuously expanded, and the feature distances of different vehicles are driven to be continuously far away;
using sigmoid functions to correlate feature maps with attributes
Figure BDA0003043816540000049
Normalizing to 0-1, and performing element-level product operation with the vehicle characteristic diagram T to obtain an enhanced diagram T related to the attributeseThe formula is as follows:
Figure BDA0003043816540000051
in the iterative process of the system, the expansion of the attribute is continuously executed, and the feature diagram related to the attribute is utilized
Figure BDA0003043816540000052
The calculation formula of the enhanced vehicle characteristic diagram T' is as follows:
T′＝T+Te
wherein, TeAnd (3) showing an enhancement map related to the attribute, T showing a characteristic map of the vehicle image, and T' showing a characteristic map of vehicle enhancement.
As a further improvement of the technical solution of the present invention, the method for reducing the difference of the intra-class states by narrowing down the distribution of the states through the shrinkage loss function of the states in step 3) comprises:
inputting a vehicle image I with an image size of 256 × 256, and acquiring a feature map T of the vehicle image through a 50-layer residual network, where T is ResNet50 (I);
copying three parts of a characteristic diagram T of the vehicle image, wherein the first part is transmitted into a weakening and shrinking module based on the state;
then, inputting the feature map T into the 1 × 1 volume blocks corresponding to different states to obtain a feature map independent of the states:
Figure BDA0003043816540000053
performing batch regularization BN operation and modified linear unit ReLU operation on state-independent feature maps
Figure BDA0003043816540000054
And carrying out global average pooling operation on the feature graph irrelevant to the state to obtain features irrelevant to the state:
Figure BDA0003043816540000055
introducing state independent tags
Figure BDA0003043816540000056
Adding a full connectivity layer (FC) so that the state-independent features are constrained by labels in the end-to-end training, the state-independent label constraint being formulated as:
Figure BDA0003043816540000057
then calculating the different state mean value of the current vehicle relative to all vehicles
Figure BDA0003043816540000058
Is distributed over a plurality of features, | |)2Represents l2The norm calculates the square root of the sum of squares of the features;
Figure BDA0003043816540000059
designing a contraction loss function based on the state to force the network to make different state feature distributions contract continuously in end-to-end training, wherein the contraction formula of the state is as follows:
Figure BDA0003043816540000061
the training of the network can carry out back transmission loss function towards the descending direction of shrinkage loss, so that the state characteristic distribution is continuously shrunk, and the characteristic distance of the same vehicle is driven to be continuously shrunk;
using sigmoid functions to make state-independent profiles
Figure BDA0003043816540000062
Normalizing to 0-1, and multiplying element level with the vehicle characteristic diagram T to obtain a weakening diagram T independent of statewThe formula is as follows:
Figure BDA0003043816540000063
in the iterative process of the system, the contraction of the state is continuously executed, and the characteristic diagram which is irrelevant to the state is utilized
Figure BDA0003043816540000064
Weakening the characteristic diagram T 'of vehicle reinforcement to obtain a final vehicle characteristic diagram T', wherein the calculation formula is as follows:
T″＝T′-Tw
wherein, T 'represents the final vehicle characteristic diagram, and T' represents the enhanced characteristic diagram of the vehicle.
As a further improvement of the technical scheme of the present invention, the method for training all samples by heterogeneous sample separation and homogeneous samples in step 4) comprises:
for the input image I, obtaining a final vehicle feature map T 'through an attribute-based enhancement and expansion module and a state-based weakening and contraction module, and performing global average pooling operation on the final vehicle feature map T' to obtain a feature vector of the image I:
f＝GAP(T″)
for any one in the training setA pair of heterogeneous samples (I)i，Ij) Corresponding vehicle characteristics (f) can be obtainedi，fj) Corresponding attribute features
Figure BDA0003043816540000065
The designed attribute-driven heterogeneous sample separation constraint is as follows:
Figure BDA0003043816540000066
wherein y isij0 means that this is a heterogeneous sample pair,
Figure BDA0003043816540000067
represents a heterogeneous sample pair (I)i，Ij) Of the attribute features of (1), dijRepresenting the Euclidean distance of the vehicle characteristics of the heterogeneous sample pairs, wherein the heterogeneous sample separation driven by the attributes enables the characteristics of the heterogeneous samples to be learned to have a weight related to the attributes in the metric learning stage, and the vehicle characteristic distance of the heterogeneous samples is increased along with the expansion of the attributes until an upper boundary 1 is reached;
for any one homogeneous sample pair (I) in the training seti，Ij) Corresponding vehicle characteristics (f) can be obtainedi，fj) Corresponding attribute features
Figure BDA0003043816540000071
The designed attribute-driven heterogeneous sample separation constraint is as follows:
Figure BDA0003043816540000072
wherein y isijMeaning that this is a heterogeneous sample pair,
Figure BDA0003043816540000073
represents a heterogeneous sample pair (I)i，Ij) Is characteristic of the state of (a), dijRepresenting pairs of homogeneous samplesThe characteristic Euclidean distance of the vehicle and the similar samples driven by the states are close together, so that in the measurement learning stage, the characteristic learning of the similar samples has a weight related to the states; so that the vehicle characteristic distance of the same and different samples is disturbed along with the contraction of the state until the distance is less than the lower boundary 0.3.
As a further improvement of the technical solution of the present invention, the function of the global structure embedding loss in step 5) is:
Figure BDA0003043816540000074
the training of the system carries out back propagation loss function towards the descending direction of the embedding loss of the global structure, so that the vehicle characteristic distance of the heterogeneous sample is increased along with the expansion of the attribute until the distance reaches an upper boundary 1, and the vehicle characteristic distance of the homogeneous sample is disturbed along with the contraction of the state until the distance is less than a lower boundary 0.3; and finally, reversely transmitting the multi-label classification loss value, updating and iterating 120 times to obtain the optimal network parameter based on the expansion loss value of the attribute, the weakening loss value of the state and the global structured embedding loss value.
The invention has the advantages that:
(1) the technical scheme of the invention increases the feature distance between different vehicles by enhancing and expanding the attributes, and reduces the feature distance between the same vehicles by weakening and contracting the state, thereby better helping the network to learn the features of the vehicles.
(2) According to the technical scheme, when the vehicle re-identification task is executed in the testing stage, the attribute information of the vehicle is directly acquired by using the attribute label in the training stage, most of the vehicles without the same attribute can be filtered out in the re-identification process by enhancing the attribute information of the vehicle, and the vehicles with the same identity under the same attribute can be better excavated in the re-identification process of the vehicle by expanding the attribute information of the vehicle; meanwhile, the camera environment, the visual angle information and the time information of the tested vehicle are judged by utilizing the state label in the training stage, the vehicles with the same identity can be more easily distinguished by the re-recognition model by weakening the state information of the vehicle, and the vehicles with the same identity can be better retrieved when the vehicles are re-recognized by contracting the state information of the vehicles.
(3) The technical scheme of the invention starts from a single vehicle test image, associates the attribute information and the state information of the vehicle, is easier to find the vehicle with the same identity in a huge vehicle search library, and provides a simple and effective strategy for deploying the rapid re-identification system for intelligent transportation.
Drawings
FIG. 1 is a block diagram of the system of the present invention;
FIG. 2 is a block diagram of the system of the present invention;
FIG. 3 is a block diagram of the global structure embedded module of the present invention;
FIG. 4 is a flow chart of a training phase of the method of the present invention;
FIG. 5 is a flow chart of a testing phase of the method of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The technical scheme of the invention is further described by combining the drawings and the specific embodiments in the specification:
as shown in fig. 2, an attribute and state-guided structure embedded in a vehicle weight recognition system for learning a feature having discriminative power in vehicle weight recognition by attribute-based enhancement and state-based weakening includes: a residual network module, an attribute-based enhancement and expansion module, a state-based attenuation and contraction module and a global structure embedding module;
the residual error network module is used for extracting a visible light characteristic diagram, copying the characteristic diagram into three parts, inputting one part of the characteristic diagram into the attribute-based enhancement and expansion module, inputting the other part of the characteristic diagram into the state-based weakening and contraction module, and obtaining category scores through global average pooling and full connection operation;
the attribute-based enhancement and expansion module is used for enhancing the recognition of vehicle features through attribute information related to identities and increasing feature distances among different vehicles by setting an attribute-based expansion loss function;
the state-based weakening and shrinking module is used for weakening state information interfering identification and reducing the inter-class feature distance through a state-based shrinking loss function;
as shown in fig. 3, the global structure embedding module is configured to obtain a final vehicle feature vector based on the attribute enhancement and the state-based weakening operation, calculate a category score through the final feature vector, transmit the category score to the search library, and determine whether the vehicle appears in another camera.
The calculating the category score through the final feature vector specifically includes: and calculating the category constraint loss values of the predicted category and the true value by using a cross entropy function, sharing the category loss values of three constraints of attributes, states and identities, and adding the three according to a proportion to form a multi-label category loss function to predict the categories of the three.
The characteristic spacing between different vehicles is increased by the enhancement and expansion of the attributes, and the characteristic spacing between the same vehicles is reduced by the weakening and contraction of the state, so that the characteristics of the vehicles are better learned by a network.
As shown in fig. 4-5, a structure-embedded vehicle weight identification method for attribute and status guidance includes two stages: a training phase and a testing phase.
1. The training phase comprises the following steps
Step 1: a visible light image of a vehicle is acquired and transmitted as input to the network.
Step 2: extracting a visible light characteristic diagram through a residual error network commonly used in the field of vehicle weight identification, copying the characteristic diagram into three parts, inputting one part of the characteristic diagram into an enhancement and expansion module based on attributes, inputting the other part of the characteristic diagram into a weakening and contraction module based on states, and obtaining category scores through global average pooling and full connection operation.
And calculating the category constraint loss values of the predicted category and the true value by using a cross entropy function, sharing the category loss values of three constraints of attributes, states and identities, and adding the three according to a proportion to form a multi-label category loss function to predict the categories of the three.
And step 3: in addition to attribute-based enhancement and state-based weakening, attribute-based expansion loss and state-based contraction loss are further proposed. Attribute-based expansion loss is to expand the feature distribution of attributes, thereby increasing the differences in attributes between classes. State-based shrinkage loss is to narrow the distribution of states and thereby reduce the variance of states within a class.
And 4, step 4: it is proposed that the use of state and attribute guided global structure embedding loss allows negative samples with less attribute difference and positive samples with larger state to obtain larger gradient amplitude.
And 5: the final penalty of the model is defined as the sum of the multi-label classification penalty, the attribute-based expansion penalty, the state-based contraction penalty, and the global embedding penalty. In each training, the structure embedded network guided by the attributes and the states reversely transmits a final loss value, and updates model parameters of the residual block, the attribute enhancing module and the state weakening module parameters by using a self-adaptive momentum random gradient descent method with the loss value reduced as a target. After a certain number of iterations, the loss value of the network is not reduced any more, and the training of the network is finished when all the parameters reach the optimal values.
The specific training mode comprises the following steps:
(1) the network batch size was set to 64 in each training, i.e., the network read 64 images at a time. Each image was resized to 256 × 256 pixels and the 10 pixels filling the periphery of the image were 0. It is then randomly cropped to a 256 × 256 rectangular image. Each image is flipped with a probability level of 0.5 and the R, G, B channels of the image are normalized with a mean of 0.485, 0.456, 0.406 and a standard deviation of 0.229, 0.224, 0.225, respectively, and each image is decoded as 32-bit floating-point raw data, i.e., floating-point numbers with pixel values between [0, 1] of the image.
(2) For a vehicle image input into the network, firstly, a corresponding feature map is extracted through a trained 50-layer residual error network. The feature map is copied into three parts, one part is input into an enhancement and expansion module based on attributes, the other part is input into a weakening and contraction module based on states, and the other part obtains category scores through global average pooling and full connection operation. In one aspect, multi-label classification penalties are obtained by combining classification penalties for attributes, states, and identities to ensure that the prediction class from each module is consistent with a true value. At the same time, differences in intra-class states are reduced by increasing the differences in attributes between classes based on extended penalties of attributes and by reducing the differences in states based on contracted penalties of states. On the other hand, larger gradient amplitudes are obtained for negative samples with smaller attribute differences and positive samples with larger states using attribute and state-guided global structure embedding loss. The final penalty value for the attribute-and state-directed structure embedding network model is the sum of the multi-label classification penalty, the attribute-based expansion penalty, the state-based contraction penalty, and the attribute-and state-directed global structure embedding penalty.
(3) The loss value is transmitted back and the parameters are updated. And repeating the iteration until the network converges. And transmitting the loss obtained by the last step back to the network, and updating model parameters by using an optimizer, wherein the optimizer adopts a random gradient descent method, the learning rate is dynamically set along with the network training times, the initial learning rate is set to be 0.00035, and the initial learning rate is reduced by 10 times in the 40 th iteration and the 70 th iteration respectively. There were a total of 120 training sessions.
2. The testing phase comprises the following steps
Step 1: a visible light image of the vehicle is loaded. The network that has been trained is input.
Step 2: and extracting a visible light characteristic diagram through a trained residual error network.
And step 3: the visible light characteristic diagram is respectively subjected to a trained attribute-based enhancing and expanding module and a state-based weakening and contracting module, and an attribute enhanced characteristic tensor and a state weakened characteristic tensor are obtained.
And 4, step 4: and obtaining a final vehicle feature vector through attribute-based enhancement and state-based weakening operation.
And 5: and calculating a category score through the final feature vector, transmitting the category score to a search library, and judging whether the vehicle appears in other cameras.
The specific test mode comprises the following steps:
1) a vehicle image is input. Each image was resized to 256 x 256 pixels into the trained network.
2) And the network calculates to obtain the category scores after the attribute enhancement and the state reduction.
3) And after the category score of the vehicle is obtained, the vehicle characteristics are put into an image retrieval library for comparison, and whether the vehicle appears in other cameras or not is judged.
3. Specific workflow of extension phase
(1) A vehicle image I with an image size of 256 × 256 is input, and a feature map T of the vehicle image T, which is ResNet50(I), can be obtained through a 50-layer residual network.
(2) The feature map T of the vehicle image is copied in triplicate, with the first copy passing into the attribute-based enhancement and expansion module.
(3) Then, inputting the feature map T into a 1 × 1 volume block related to the color attribute to obtain a feature map related to the color attribute:
Figure BDA0003043816540000111
(4) performing batch regularization BN operation and modified linear unit ReLU operation on feature graph related to color attribute
Figure BDA0003043816540000112
(5) Carrying out global average pooling operation on the feature map related to the color attribute to obtain the features related to the color attribute
Figure BDA0003043816540000113
(6) Introducing color attribute dependent labels
Figure BDA0003043816540000114
Adding a full connection layer (FC) so that the color attribute related features are constrained by the label in end-to-end training, wherein the label constraint of the color attribute is formulated as:
Figure BDA0003043816540000115
(7) label constraint of multiple (M) attributes is performed simultaneously, and the formula is as follows:
Figure BDA0003043816540000116
(8) then, the color attribute mean value of the current vehicle relative to all vehicles is calculated
Figure BDA0003043816540000117
Is distributed over a plurality of features, | |)2Represents l2The norm takes the square root of the sum of the squares of the features.
Figure BDA0003043816540000118
(9) An extended loss function based on attributes is designed to force the network to continuously extend the attribute-related feature distribution in end-to-end training, and the extended formula of the attributes is as follows:
Figure BDA0003043816540000119
(10) simultaneously extending various attributes (M), wherein the formula is as follows:
Figure BDA00030438165400001110
the training of the network can carry out back transmission loss function towards the descending direction of the expansion loss, so that the distribution of the correlation characteristics is continuously expanded (the black and gray attribute characteristics are far away), the characteristic distances of different vehicles are driven to be continuously far away, and the images of the different vehicles are more easily distinguished by the network.
4. Detailed workflow of enhancement phase
Since the expansion of the color properties is performed continuously in iterations of the network. Will obtain a more discriminative color feature map
Figure BDA00030438165400001111
It is desirable to utilize a more discriminating color profile
Figure BDA00030438165400001112
Enhancing the vehicle characteristic map T;
(1) and normalizing the color characteristic diagram into a product operation of 0-1 and the vehicle characteristic diagram at an element level by using an S-type function, wherein the formula is as follows:
Figure BDA0003043816540000121
(2) the enhancement map of the plurality of attributes (M) may be formulated as:
Figure BDA0003043816540000122
(3) and (3) performing element level accumulation on the enhanced graph with various attributes and the original vehicle characteristic graph:
T′＝T+Te
5. specific workflow of the contraction phase
(1) A vehicle image I with an image size of 256 × 256 is input, and a feature map T of the vehicle image T, which is ResNet50(I), can be obtained through a 50-layer residual network.
(2) The feature map T of the vehicle image is copied in triplicate, with the first copy passing into the attribute-based enhancement and expansion module.
(3) Then inputting the feature map T into a 1 × 1 volume block related to the camera state to obtain a feature map related to the camera state:
Figure BDA0003043816540000123
(4) performing batch regularization BN operation and modified linear unit ReLU operation on feature graph related to color attribute
Figure BDA0003043816540000124
(5) Carrying out global average pooling operation on the feature map related to the color attribute to obtain the features related to the camera state
Figure BDA0003043816540000125
(6) Introducing camera state dependent tags
Figure BDA0003043816540000126
Adding a full connection layer (FC) so that the camera state related features are constrained by the label in the end-to-end training, wherein the label constraint of the camera state is formulated as:
Figure BDA0003043816540000127
(7) label constraint of multiple (N) states is performed simultaneously, the formula is:
Figure BDA0003043816540000128
(8) then calculating the mean value of the camera states of the current vehicle relative to all vehicles
Figure BDA0003043816540000129
Is distributed over a plurality of features, | |)2Represents l2The norm takes the square root of the sum of the squares of the features.
Figure BDA0003043816540000131
(9) Designing a contraction loss function based on a state to force the network to make the state-related feature distribution continuously contract in end-to-end training, wherein a contraction formula of the state is as follows:
Figure BDA0003043816540000132
(10) the contraction of the various states (N) is performed simultaneously, and the formula is as follows:
Figure BDA0003043816540000133
the training of the network carries out a back transmission loss function towards the descending direction of contraction loss, so that the distribution of state-related features (camera 139 and camera 79) is continuously contracted, the feature distance of the same vehicle is driven to be continuously contracted, and images of the same vehicle in different states are more easily distinguished by the network.
6. Specific workflow of the weakening phase
Since the shrinking of the camera state is performed continuously in iterations of the network. Will obtain the camera feature map irrelevant to the vehicle discrimination
Figure BDA0003043816540000134
It is therefore desirable to utilize camera profiles that are irrelevant for vehicle identification
Figure BDA0003043816540000135
Weakening vehicle characteristic diagram T
(1) And normalizing the color characteristic diagram into a product operation of 0-1 and the vehicle characteristic diagram at an element level by using an S-type function, wherein the formula is as follows:
Figure BDA0003043816540000136
(2) the de-emphasis graph for multiple states (N) can be formulated as:
Figure BDA0003043816540000137
(3) performing element level cumulative difference on the enhanced graphs with various attributes and the enhanced vehicle characteristic graph to obtain a final vehicle characteristic graph:
T″＝T′-Tw
7. attribute-driven heterogeneous sample separation
(1) For the input image I, a final vehicle feature map T 'is obtained through an attribute-based enhancement and expansion module and a state-based weakening and contraction module, a global average pooling operation is carried out on the vehicle feature map T', and a feature vector of the image I can be obtained:
f＝GAP(T″)
(2) for any one heterogeneous sample pair (I) in the training seti，Ij) Corresponding vehicle characteristics (f) can be obtainedi，fj) Corresponding attribute features
Figure BDA0003043816540000141
The designed attribute-driven heterogeneous sample separation constraint is as follows:
Figure BDA0003043816540000142
wherein y isij0 means that this is a heterogeneous sample pair,
Figure BDA0003043816540000143
represents a heterogeneous sample pair (I)i，Ij) Of the attribute features of (1), dijVehicle characteristic Euclidean distance representing heterogeneous sample pair and attribute-driven heterogeneous sampleThe separation is such that in the metric learning phase, the features of the heterogeneous samples are learned with a weight associated with the attribute. Such that the vehicle feature distance of the heterogeneous sample increases as the attribute expands until the upper boundary 1 is reached.
8. State-driven homogeneous sample closedown
(1) For the input image I, a final vehicle feature map T 'is obtained through an attribute-based enhancement and expansion module and a state-based weakening and contraction module, a global average pooling operation is carried out on the vehicle feature map T', and a feature vector of the image I can be obtained:
f＝GAP(T″)
(2) for any one homogeneous sample pair (I) in the training seti，Ij) Corresponding vehicle characteristics (f) can be obtainedi，fj) Corresponding attribute features
Figure BDA0003043816540000144
The designed attribute-driven heterogeneous sample separation constraint is as follows:
Figure BDA0003043816540000145
wherein y isijMeaning that this is a heterogeneous sample pair,
Figure BDA0003043816540000146
represents a heterogeneous sample pair (I)i，Ij) Is characteristic of the state of (a), dijAnd representing the Euclidean distance of the vehicle characteristics of the similar sample pairs, and enabling the similar samples driven by the states to be close together, so that the characteristic learning of the similar samples has a weight related to the states in the metric learning stage. So that the vehicle characteristic distance of the same and different samples is disturbed along with the contraction of the state until the distance is less than the lower boundary 0.3.
9. Global structure embedding
(1) Since heterogeneous sample separation and homogeneous sample perturbation are operations on all samples of the training set, this whole constraint is named as global structure embedding module, and the global structure embedding loss function is:
Figure BDA0003043816540000147
the training of the network carries out back propagation loss function towards the descending direction of the embedding loss of the global structure, so that the vehicle characteristic distance of the heterogeneous sample is increased along with the expansion of the attribute until the upper boundary 1 is reached, and the vehicle characteristic distance of the homogeneous sample is disturbed along with the contraction of the state until the lower boundary is less than 0.3. Images of different vehicles are easier to distinguish by the network, and images of the same vehicle in different states are easier to distinguish by the network.
(2) And finally, reversely transmitting the multi-label classification loss value, updating and iterating 120 times to obtain the optimal network parameter based on the expansion loss value of the attribute, the weakening loss value of the state and the global structured embedding loss value.
In the embodiment, feature learning and metric learning are trained by using an end-to-end deep neural network, and attribute information (color, vehicle type and manufacturer) and state information (the number of a camera usually means the arrangement place of the camera and the number of how many shooting places there will be) and the embedding of the viewpoints of a vehicle (five viewpoints of the head, the tail, the side, the front side and the rear side) and shooting time (0-23, including 24 hours of a day) are considered in both stages of the feature learning and the metric learning.
The attribute information of the vehicle is optimized, so that the characteristic distances of different vehicles are far away due to the fact that different attribute information is continuously expanded in end-to-end learning, and the fact that the characteristic distances of different vehicles are far away is more meaningful compared with direct characteristic splicing, and better vehicle re-identification performance can be obtained.
In addition to this, the status information of the vehicle is also taken into account, which is also original, and the information of the vehicle is divided into two groups, one group being considered helpful for the vehicle weight identification (attribute of the vehicle) and one group being considered to have an influence on the vehicle discrimination (status of the vehicle).
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (6)

1. An attribute and state guided structure embedded vehicle weight recognition system characterized by learning features having discriminative power in vehicle weight recognition by attribute-based enhancement and state-based weakening, comprising: a residual network module, an attribute-based enhancement and expansion module, a state-based attenuation and contraction module and a global structure embedding module;
the residual error network module is used for extracting a visible light characteristic diagram, copying the characteristic diagram into three parts, inputting one part of the characteristic diagram into the attribute-based enhancement and expansion module, inputting the other part of the characteristic diagram into the state-based weakening and contraction module, and obtaining category scores through global average pooling and full connection operation;
the attribute-based enhancement and expansion module is used for enhancing the recognition of vehicle features through attribute information related to identities and increasing feature distances among different vehicles by setting an attribute-based expansion loss function;
the state-based weakening and shrinking module is used for weakening state information interfering identification and reducing the inter-class feature distance through a state-based shrinking loss function;
the global structure embedding module is used for obtaining a final vehicle feature vector based on the enhancement of the attribute and the weakening operation based on the state, calculating a category score through the final feature vector, transmitting the category score to a search library and judging whether the vehicle appears in other cameras.
2. A method for application of the attribute and state guided structure embedded vehicle weight recognition system of claim 1, characterized by two stages: a training stage and a testing stage;
the training phase comprises the following steps:
step 1): acquiring a visible light image of a vehicle, and inputting the visible light image into the system;
step 2): extracting a visible light characteristic diagram through a residual error network, copying the characteristic diagram into three parts, inputting one part of the characteristic diagram into an enhancement and expansion module based on attributes, inputting the other part of the characteristic diagram into a weakening and contraction module based on states, and obtaining category scores through global average pooling and full connection operation; calculating the category constraint loss values of the predicted category and the true value by using a cross entropy function, sharing the category loss values of three constraints of attributes, states and identities, and adding the three according to the proportion to form a multi-label category loss function to predict the categories of the three;
step 3): the feature distribution of the attributes is expanded through the expansion loss function of the attributes, so that the difference of the attributes among the classes is increased; narrowing the distribution of states by a shrinkage loss function of the states, thereby reducing the difference of the states in the class;
step 4): using a global structure embedding loss function, training all samples through heterogeneous sample separation and homogeneous sample disturbance, so that the vehicle characteristic distance of the heterogeneous sample is increased along with the expansion of the attribute until the upper boundary is reached, and the vehicle characteristic distance of the homogeneous sample is disturbed along with the contraction of the state until the lower boundary is smaller than the lower boundary;
step 5): the final loss of the model is defined as the sum of multi-label classification loss, attribute-based expansion loss, state-based contraction loss and global structure embedding loss; in each training, the attribute and state-guided structure is embedded into the system to reversely transmit a final loss value, the loss value is reduced as a target, model parameters of the residual block and parameters of the attribute enhancing module and the state weakening module are updated by a self-adaptive momentum random gradient descent method, after multiple iterations, the loss value of the system is not reduced any more, all the parameters are optimal, and the training of the system is finished;
the testing stage comprises the following steps:
step a): loading a visible light image of a vehicle and inputting the visible light image into a trained system;
step b): extracting a visible light characteristic diagram through a trained residual error network;
step c): respectively passing the visible light characteristic diagram through a trained attribute-based enhancing and expanding module and a state-based weakening and contracting module, and obtaining an attribute enhanced characteristic tensor and a state weakened characteristic tensor;
step d): obtaining a final vehicle feature vector through attribute-based enhancement and state-based weakening operation;
step e): and calculating a category score through the final feature vector, transmitting the category score to a search library, and judging whether the vehicle appears in other cameras.
3. The method for embedding an attribute and state guided structure in a vehicle re-identification system according to claim 2, wherein the method for expanding the feature distribution of the attribute by the expansion loss function of the attribute in the step 3) so as to increase the difference of the attribute between the classes is as follows:
inputting a vehicle image I with an image size of 256 × 256, and acquiring a feature map T of the vehicle image through a 50-layer residual network, where T is ResNet50 (I);
copying three parts of a feature map T of the vehicle image, wherein the first part is transmitted into an attribute-based enhancement and expansion module;
then, inputting the feature map T into 1 × 1 volume blocks corresponding to different attributes to obtain a feature map related to the attributes:
Figure FDA0003043816530000021
and (3) carrying out batch regularization BN operation and modified linear unit ReLU operation on the feature graph related to the attribute:
Figure FDA0003043816530000022
carrying out global average pooling operation on the feature graph related to the attribute to obtain the feature related to the attribute:
Figure FDA0003043816530000023
introducing attribute related tags
Figure FDA0003043816530000024
Adding a full connection layer FC, so that the characteristics related to the attributes are constrained by the labels in the end-to-end training, and the constraint of the labels related to the attributes is formulated as:
Figure FDA0003043816530000025
then, the average value of the attributes of the current vehicle relative to all the vehicles is calculated
Figure FDA0003043816530000026
Is distributed over a plurality of features, | |)2Represents l2Norm square root of sum of squared features:
Figure FDA0003043816530000027
designing an extended loss function based on attributes to force the network to continuously extend the feature distribution related to the attributes in end-to-end training, wherein the extended formula is as follows:
Figure FDA0003043816530000031
wherein i represents the type of attribute, and in this embodiment, the following are respectively: a color attribute of the vehicle, a type attribute of the vehicle, a manufacturer attribute of the vehicle;
the system is trained to perform back transmission loss function towards the descending direction of the expansion loss, so that the attribute feature distribution is continuously expanded, and the feature distances of different vehicles are driven to be continuously far away;
using sigmoid functions to correlate feature maps with attributes
Figure FDA0003043816530000034
Normalizing to 0-1, and performing element-level product operation with the vehicle characteristic diagram T to obtain an enhanced diagram T related to the attributeseThe formula is as follows:
Figure FDA0003043816530000032
in the iterative process of the system, the expansion of the attribute is continuously executed, and the feature diagram related to the attribute is utilized
Figure FDA0003043816530000035
The calculation formula of the enhanced vehicle characteristic diagram T' is as follows:
T′＝T+Te
wherein, TeAnd (3) showing an enhancement map related to the attribute, T showing a characteristic map of the vehicle image, and T' showing a characteristic map of vehicle enhancement.
4. The method for embedding attribute and state guided structure in vehicle re-identification system according to claim 2, wherein the method for narrowing the distribution of the states by the shrinkage loss function of the states in step 3) so as to reduce the difference of the states in the class is:
inputting a vehicle image I with an image size of 256 × 256, and acquiring a feature map T of the vehicle image through a 50-layer residual network, where T is ResNet50 (I);
copying three parts of a characteristic diagram T of the vehicle image, wherein the first part is transmitted into a weakening and shrinking module based on the state;
then, inputting the feature map T into the 1 × 1 volume blocks corresponding to different states to obtain a feature map independent of the states:
Figure FDA0003043816530000036
performing batch regularization BN operation and modified linear unit ReLU operation on state-independent feature maps
Tj st＝ReLU(BN(Tj st))
And carrying out global average pooling operation on the feature graph irrelevant to the state to obtain features irrelevant to the state:
fj st＝GAP(Tj st)
introducing state independent tags
Figure FDA0003043816530000033
Adding a full connectivity layer (FC) so that the state-independent features are constrained by labels in the end-to-end training, the state-independent label constraint being formulated as:
Figure FDA0003043816530000041
then calculating the different state mean value of the current vehicle relative to all vehicles
Figure FDA0003043816530000042
Is distributed over a plurality of features, | |)2Represents l2The norm calculates the square root of the sum of squares of the features;
Figure FDA0003043816530000043
designing a contraction loss function based on the state to force the network to make different state feature distributions contract continuously in end-to-end training, wherein the contraction formula of the state is as follows:
Figure FDA0003043816530000044
the training of the network can carry out back transmission loss function towards the descending direction of shrinkage loss, so that the state characteristic distribution is continuously shrunk, and the characteristic distance of the same vehicle is driven to be continuously shrunk;
using sigmoid functions to make the characteristic diagram T independent of statej stNormalizing to 0-1, and multiplying element level with the vehicle characteristic diagram T to obtain a weakening diagram T independent of statewThe formula is as follows:
Figure FDA0003043816530000045
in the iterative process of the system, the contraction of the state is continuously executed, and the characteristic diagram T which is irrelevant to the state is utilizedj stWeakening the characteristic diagram T 'of vehicle reinforcement to obtain a final vehicle characteristic diagram T', wherein the calculation formula is as follows:
T″＝T′-Tw
wherein, T 'represents the final vehicle characteristic diagram, and T' represents the enhanced characteristic diagram of the vehicle.
5. The method for embedding the attribute-and-state-guided structure in the vehicle re-identification system according to claim 2, wherein the method for training all the samples by heterogeneous sample separation and homogeneous sample perturbation in the step 4) comprises the following steps:
for the input image I, obtaining a final vehicle feature map T 'through an attribute-based enhancement and expansion module and a state-based weakening and contraction module, and performing global average pooling operation on the final vehicle feature map T' to obtain a feature vector of the image I:
f＝GAP(T″)
for any one heterogeneous sample pair (I) in the training seti，Ij) Corresponding vehicle characteristics (f) can be obtainedi，fj) Corresponding attribute features
Figure FDA0003043816530000051
The designed attribute-driven heterogeneous sample separation constraint is as follows:
Figure FDA0003043816530000052
wherein y isij0 means that this is a heterogeneous sample pair,
Figure FDA0003043816530000053
represents a heterogeneous sample pair (I)i，Ij) Of the attribute features of (1), dijRepresenting the Euclidean distance of the vehicle characteristics of the heterogeneous sample pairs, wherein the heterogeneous sample separation driven by the attributes enables the characteristics of the heterogeneous samples to be learned to have a weight related to the attributes in the metric learning stage, and the vehicle characteristic distance of the heterogeneous samples is increased along with the expansion of the attributes until an upper boundary 1 is reached;
for any one homogeneous sample pair (I) in the training seti，Ij) Corresponding vehicle characteristics (f) can be obtainedi，fj) Corresponding attribute features
Figure FDA0003043816530000054
The designed attribute-driven heterogeneous sample separation constraint is as follows:
Figure FDA0003043816530000055
wherein y isijMeaning that this is a heterogeneous sample pair,
Figure FDA0003043816530000056
represents a heterogeneous sample pair (I)i，Ij) Is characteristic of the state of (a), dijRepresenting the characteristic Euclidean distance of the vehicle of the same type sample pair, the state-driven same type samples are closed to makeIn the metric learning stage, the feature learning of the similar sample has a weight related to the state; so that the vehicle characteristic distance of the same and different samples is disturbed along with the contraction of the state until the distance is less than the lower boundary 0.3.
6. The method for attribute-and-state-guided structure-embedded vehicle re-identification system according to claim 2, wherein the function of global structure embedding loss in step 5) is:
Figure FDA0003043816530000057
the training of the system carries out back propagation loss function towards the descending direction of the embedding loss of the global structure, so that the vehicle characteristic distance of the heterogeneous sample is increased along with the expansion of the attribute until the distance reaches an upper boundary 1, and the vehicle characteristic distance of the homogeneous sample is disturbed along with the contraction of the state until the distance is less than a lower boundary 0.3; and finally, reversely transmitting the multi-label classification loss value, updating and iterating 120 times to obtain the optimal network parameter based on the expansion loss value of the attribute, the weakening loss value of the state and the global structured embedding loss value.
CN202110465670.0A 2021-04-28 2021-04-28 System and method for identifying vehicle weight by embedding structure of attribute and state guidance Active CN113128441B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110465670.0A CN113128441B (en) 2021-04-28 2021-04-28 System and method for identifying vehicle weight by embedding structure of attribute and state guidance

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110465670.0A CN113128441B (en) 2021-04-28 2021-04-28 System and method for identifying vehicle weight by embedding structure of attribute and state guidance

Publications (2)

Publication Number Publication Date
CN113128441A true CN113128441A (en) 2021-07-16
CN113128441B CN113128441B (en) 2022-10-14

Family

ID=76780792

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110465670.0A Active CN113128441B (en) 2021-04-28 2021-04-28 System and method for identifying vehicle weight by embedding structure of attribute and state guidance

Country Status (1)

Country Link
CN (1) CN113128441B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114694171A (en) * 2022-02-22 2022-07-01 电子科技大学 Pedestrian attribute identification method based on self-supervision mode feature enhancement

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108921064A (en) * 2018-06-21 2018-11-30 西安理工大学 Pedestrian based on multi-feature fusion recognition methods again
CN109635728A (en) * 2018-12-12 2019-04-16 中山大学 A kind of isomery pedestrian recognition methods again based on asymmetric metric learning
CN110084108A (en) * 2019-03-19 2019-08-02 华东计算技术研究所（中国电子科技集团公司第三十二研究所） Pedestrian re-identification system and method based on GAN neural network
CN111652293A (en) * 2020-05-20 2020-09-11 西安交通大学苏州研究院 Vehicle weight recognition method for multi-task joint discrimination learning
CN111931627A (en) * 2020-08-05 2020-11-13 智慧互通科技有限公司 Vehicle re-identification method and device based on multi-mode information fusion
CN112149538A (en) * 2020-09-14 2020-12-29 南京邮电大学 Pedestrian re-identification method based on multi-task learning

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108921064A (en) * 2018-06-21 2018-11-30 西安理工大学 Pedestrian based on multi-feature fusion recognition methods again
CN109635728A (en) * 2018-12-12 2019-04-16 中山大学 A kind of isomery pedestrian recognition methods again based on asymmetric metric learning
CN110084108A (en) * 2019-03-19 2019-08-02 华东计算技术研究所（中国电子科技集团公司第三十二研究所） Pedestrian re-identification system and method based on GAN neural network
CN111652293A (en) * 2020-05-20 2020-09-11 西安交通大学苏州研究院 Vehicle weight recognition method for multi-task joint discrimination learning
CN111931627A (en) * 2020-08-05 2020-11-13 智慧互通科技有限公司 Vehicle re-identification method and device based on multi-mode information fusion
CN112149538A (en) * 2020-09-14 2020-12-29 南京邮电大学 Pedestrian re-identification method based on multi-task learning

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
CHI SU 等: "Multi-type attributes driven multi-camera person re-identification", 《PATTERN RECOGNITIONVOLUME》 *
XIAODONG CHEN等: "AttriMeter: An Attribute-guided Metric Interpreter for Person Re-Identification", 《ARXIV 》 *
ZHENG A等: "Attributes Guided Feature Learning for Vehicle Re-identification", 《ARXIV 》 *
ZHUGE C 第: "Attribute-guided Feature Extraction and Augmentation Robust Learning for Vehicle Re-identification", 《2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW)》 *
林伟鹏: "基于多视角学习的车辆再识别方法研究", 《中国优秀硕士学位论文全文数据库》 *
林贤敏: "基于属性引导与判别特征挖掘的车辆重识别方法研究", 《中国优秀硕士学位论文全文数据库》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114694171A (en) * 2022-02-22 2022-07-01 电子科技大学 Pedestrian attribute identification method based on self-supervision mode feature enhancement
CN114694171B (en) * 2022-02-22 2023-10-10 电子科技大学 Pedestrian attribute identification method based on self-supervision mode feature enhancement

Also Published As

Publication number Publication date
CN113128441B (en) 2022-10-14

Similar Documents

Publication Publication Date Title
CN109255364B (en) Scene recognition method for generating countermeasure network based on deep convolution
CN110633632A (en) Weak supervision combined target detection and semantic segmentation method based on loop guidance
CN111767882A (en) Multi-mode pedestrian detection method based on improved YOLO model
CN107239730B (en) Quaternion deep neural network model method for intelligent automobile traffic sign recognition
CN106127197B (en) Image saliency target detection method and device based on saliency label sorting
CN110796026A (en) Pedestrian re-identification method based on global feature stitching
CN112233124A (en) Point cloud semantic segmentation method and system based on countermeasure learning and multi-modal learning
CN110349179B (en) Visible light infrared vision tracking method and device based on multiple adapters
CN113139470A (en) Glass identification method based on Transformer
CN104281572A (en) Target matching method and system based on mutual information
CN103617413A (en) Method for identifying object in image
KR20210151773A (en) Target re-recognition method and apparatus, terminal and storage medium
CN105574545A (en) Environment image multi-view-angle meaning cutting method and device
Li et al. An aerial image segmentation approach based on enhanced multi-scale convolutional neural network
CN113128441B (en) System and method for identifying vehicle weight by embedding structure of attribute and state guidance
CN114358133B (en) Method for detecting looped frames based on semantic-assisted binocular vision SLAM
Tsutsui et al. Distantly supervised road segmentation
CN114596548A (en) Target detection method, target detection device, computer equipment and computer-readable storage medium
CN113723356B (en) Vehicle re-identification method and device with complementary heterogeneous characteristic relationships
CN110751005B (en) Pedestrian detection method integrating depth perception features and kernel extreme learning machine
CN111695460A (en) Pedestrian re-identification method based on local graph convolution network
CN115797884A (en) Vehicle weight identification method based on human-like visual attention weighting
CN116168418A (en) Multi-mode target perception and re-identification method for image
CN115098646A (en) Multilevel relation analysis and mining method for image-text data
Lukac et al. An algorithm selection based platform for image understanding using high-level symbolic feedback and machine learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant