CN112990048A - Vehicle pattern recognition method and device - Google Patents
Vehicle pattern recognition method and device Download PDFInfo
- Publication number
- CN112990048A CN112990048A CN202110326344.1A CN202110326344A CN112990048A CN 112990048 A CN112990048 A CN 112990048A CN 202110326344 A CN202110326344 A CN 202110326344A CN 112990048 A CN112990048 A CN 112990048A
- Authority
- CN
- China
- Prior art keywords
- granularity
- feature
- image
- level
- hierarchical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The method comprises the steps of constructing a multi-granularity level feature coupling learning framework, obtaining an image to be recognized, inputting the image to be recognized into the trained multi-granularity level feature coupling learning framework to obtain a moire feature and level feature classification result, screening image data in a moire recognition data set according to the level feature classification result, calculating the Euclidean distance between the moire feature of the image to be recognized and the moire feature of the screened image, and obtaining the moire recognition result according to the Euclidean distance calculation result. According to the method and the device, the characteristic learning of each granularity is improved in a common promotion mode through a multi-granularity level characteristic coupling learning framework, and the efficiency and the accuracy of vehicle pattern recognition are improved.
Description
Technical Field
The application belongs to the technical field of computer vision and pattern recognition, and particularly relates to a method and a device for recognizing a vehicle pattern.
Background
In recent years, because the car pattern recognition method has great application value, the car pattern recognition method has attracted wide attention in academic fields and industrial fields, and has very important significance in applying the car pattern recognition method to intelligent traffic management and illegal crime tracking. For example, in searching for a suspect criminal vehicle, if the suspect vehicle has no license plate, a fake license plate, or other license plate information, the suspect vehicle has to be searched for by the vehicle pattern recognition technology. Secondly, in the aspects of vehicle violation penalty, unmanned driving, automatic toll collection systems and the like of traffic police, the problem of low recognition precision caused by the method for recognizing the traffic lines based on the license plate information can be solved, and the method has important theoretical significance and important practical value. In the related technology, the traditional convolutional neural network method is used for identifying and calculating the vehicle pattern, the identification accuracy is low, and the vehicle pattern identification speed is low and the efficiency is low due to more data information and larger data volume.
Disclosure of Invention
In order to overcome the problems that the traditional convolutional neural network method is used for identifying and calculating the vehicle pattern, the identification accuracy rate is low, and the vehicle pattern identification speed is low and the efficiency is low due to the fact that more data information and larger data amount exist, the method and the device for identifying the vehicle pattern are provided.
In a first aspect, the present application provides a method for identifying a vehicle pattern, including:
constructing a multi-granularity level feature coupling learning framework;
acquiring an image to be recognized, and inputting the image to be recognized into a trained multi-granularity hierarchical feature coupling learning framework to obtain a classification result of the vehicle pattern feature and the hierarchical feature;
screening image data in the pattern recognition data set according to the hierarchical feature classification result;
and calculating the Euclidean distance between the vehicle pattern features of the image to be recognized and the vehicle pattern features of the screened image, and obtaining a vehicle pattern recognition result according to the Euclidean distance calculation result.
Further, the constructing a multi-granularity level feature coupling learning framework includes:
acquiring a plurality of hierarchical features of an image to be identified;
inputting the multiple hierarchical features into a multi-granularity feature learning model based on a graph network to obtain multi-granularity hierarchical embedded features;
calculating classification loss and sequencing loss according to the multi-granularity level characteristics;
and optimizing model parameters of a multi-granularity level feature coupling learning framework according to the classification loss and the sequencing loss to construct the multi-granularity level feature coupling learning framework.
Further, the acquiring a plurality of hierarchical features of the image to be recognized includes:
constructing a vehicle pattern recognition feature extraction network;
extracting a plurality of hierarchical features of an image to be identified through the feature extraction network, the plurality of hierarchical features comprising: a brand level, a sub-level, and an identity level;
the plurality of hierarchical features comprises brand features corresponding to a brand hierarchy;
the vehicle year money characteristics and the vehicle sub money characteristics corresponding to the sub money levels;
and (4) the identity level corresponds to the vehicle-line characteristic.
Further, the constructing the fingerprint identification feature extraction network includes:
constructing a vehicle pattern recognition feature extraction network based on Lip-ResNet 50;
initializing the vehicle pattern recognition feature extraction network using pre-trained network parameters on ImageNet;
and training the car pattern recognition feature extraction network by using a car pattern recognition data set.
Further, the network for extracting the identification features of the car-line comprises a trunk network and a branch network, and further comprises:
parameters of the backbone network are commonly used by a plurality of levels in the branch network;
the parameters of the branching network are used independently by each hierarchy, the branching network including brand hierarchy branches, sub-fund hierarchy branches, and identity hierarchy branches.
Further, the calculating the classification loss and the sorting loss according to the multi-granularity hierarchical features comprises:
performing global average pooling on the brand hierarchy branches, the sub-fund hierarchy branches and the identity hierarchy branches, and inputting the branches into a multi-granularity feature learning model based on a graph network to obtain multi-granularity hierarchy embedded features;
inputting the multi-granularity level embedded features into a classifier, and calculating classification losses on a brand level and a sub-cost level;
inputting the multi-granular hierarchy embedded features into a multi-granular ordering loss module based on metric learning to calculate an ordering loss.
Further, the method also comprises the following steps: constructing a multi-granularity feature learning model based on a graph network, comprising the following steps:
constructing a hierarchical relation graph of a plurality of hierarchical characteristics;
training the graph convolutional neural network model according to the hierarchical relationship graph to obtain the weight and the bias of the full connection layer of the graph convolutional neural network model;
and constructing a multi-granularity feature learning model based on the graph network according to the weight and the bias of the full connection layer of the graph convolution neural network model.
Further, the multi-granularity level embedding feature input metric learning-based multi-granularity ranking loss module calculates ranking loss, including:
acquiring quintuple samples from the pattern identification dataset;
dividing the quintuple sample into image pairs, and classifying the image pairs;
calculating the spatial distance of each image pair in the feature space;
and constructing a multi-granularity sorting loss function according to the space distance.
Further, the constructing a multi-granularity sorting loss function according to the spatial distance includes:
calculating a loss term according to the spatial distance;
carrying out weighted combination on the losses to obtain a multi-granularity sorting loss function;
the loss term includes:
the coarsest granularity sorting loss item is used for restricting the distance between the image corresponding to the brand level and the target image in the feature space to be larger than the distance between the image corresponding to the sub-money level and the target image so as to increase the discrimination between coarsest granularity levels;
the coarse-grained sequencing loss item is used for restricting the distance between the image corresponding to the sub-money level and the target image in the feature space to be larger than the distance between the image corresponding to the identity level and the target image so as to increase the discrimination between coarse-grained levels;
the fine-grained sequencing loss items are used for increasing the distinguishability among different identity levels under the same sub-level and reducing the difference of identity classes in the same identity level;
the sample pair loss item is used for restricting the distance of the target images belonging to the same fine-grained category in the feature space to be as small as possible;
and classifying loss items by fine-grained categories, which are cross entropy losses of the identity levels, are used for reducing differences in the levels and simultaneously accelerating the convergence speed of the loss function.
In a second aspect, the present application provides a car-line recognition apparatus, including:
the construction module is used for constructing a multi-granularity level feature coupling learning framework;
the input module is used for acquiring an image to be recognized and inputting the image to be recognized into a trained multi-granularity hierarchical feature coupling learning framework to obtain a classification result of the vehicle pattern feature and the hierarchical feature;
the screening module is used for screening the image data in the pattern recognition data set according to the hierarchical feature classification result;
and the calculation module is used for calculating the Euclidean distance between the vehicle pattern features of the image to be identified and the vehicle pattern features of the screened image, and obtaining a vehicle pattern identification result according to the Euclidean distance calculation result.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
according to the method and the device for identifying the moire pattern, the image to be identified is obtained by constructing the multi-granularity hierarchical feature coupling learning framework, the image to be identified is input into the trained multi-granularity hierarchical feature coupling learning framework to obtain the moire pattern feature and the hierarchical feature classification result, image data in the moire pattern identification data set are screened according to the hierarchical feature classification result, the Euclidean distance between the moire feature of the image to be identified and the moire feature of the screened image is calculated, the moire identification result is obtained according to the Euclidean distance calculation result, feature learning of each granularity is improved in a common promotion mode through the multi-granularity hierarchical feature coupling learning framework, and efficiency and accuracy of moire identification are improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
Fig. 1 is a flowchart of a method for identifying a car-pattern according to an embodiment of the present application.
Fig. 2 is a flowchart of a method for identifying a car-pattern according to another embodiment of the present application.
Fig. 3 is a flowchart of a method for identifying a car-pattern according to another embodiment of the present application.
Fig. 4 is a hierarchical relationship diagram according to an embodiment of the present application.
Fig. 5 is a flowchart of a method for identifying a car-pattern according to another embodiment of the present application.
Fig. 6 is a schematic diagram of a multi-granularity level feature coupling learning framework according to an embodiment of the present application.
Fig. 7 is a schematic diagram of a multi-granularity hierarchical ranking penalty based on metric learning according to an embodiment of the present application.
Fig. 8 is a functional block diagram of a car-pattern recognition apparatus according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail below. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a flowchart of a method for identifying a vehicle print according to an embodiment of the present application, and as shown in fig. 1, the method for identifying a vehicle print includes:
s11: constructing a multi-granularity level feature coupling learning framework;
s12: acquiring an image to be recognized, and inputting the image to be recognized into a trained multi-granularity hierarchical feature coupling learning framework to obtain a classification result of the vehicle pattern feature and the hierarchical feature;
s13: screening the image data in the pattern recognition data set according to the hierarchical feature classification result;
s14: and calculating the Euclidean distance between the vehicle pattern features of the image to be recognized and the vehicle pattern features of the screened image, and obtaining a vehicle pattern recognition result according to the Euclidean distance calculation result.
The traditional convolutional neural network method is used for identifying and calculating the vehicle pattern, the identification accuracy is low, and the vehicle pattern identification speed is low and the efficiency is low due to the fact that more data information and larger data volume are provided.
In the embodiment, the characteristic learning of each granularity is improved in a common promotion mode through a multi-granularity hierarchical characteristic coupling learning framework, the image data in the fingerprint identification data set is screened according to the hierarchical characteristic classification result, the fingerprint identification data set can be screened firstly, the fingerprint characteristic retrieval range and the data volume are reduced, and the accuracy and the identification efficiency of fingerprint identification are improved.
In the embodiment, the image to be recognized is acquired by constructing a multi-granularity hierarchical feature coupling learning framework, the image to be recognized is input into the trained multi-granularity hierarchical feature coupling learning framework to obtain the moire feature and the hierarchical feature classification result, image data in the moire recognition data set is screened according to the hierarchical feature classification result, the Euclidean distance between the moire feature of the image to be recognized and the moire feature of the screened image is calculated, the moire recognition result is obtained according to the Euclidean distance calculation result, feature learning of each granularity is improved in a common promotion mode through the multi-granularity hierarchical feature coupling learning framework, and the efficiency and the accuracy of moire recognition are improved.
Fig. 2 is a flowchart of a method for identifying a vehicle print according to another embodiment of the present application, and as shown in fig. 2, the method for identifying a vehicle print includes:
s21: acquiring a plurality of hierarchical features of an image to be identified;
s22: inputting the multiple hierarchical features into a multi-granularity feature learning model based on a graph network to obtain multi-granularity hierarchical embedded features;
s23: calculating classification loss and sequencing loss according to the multi-granularity level characteristics;
s24: and optimizing model parameters of the multi-granularity level feature coupling learning framework according to the classification loss and the sequencing loss to construct the multi-granularity level feature coupling learning framework.
In some embodiments, obtaining a plurality of hierarchical features of an image to be identified includes:
s211: constructing a vehicle pattern recognition feature extraction network;
in some embodiments, constructing the fingerprint identification feature extraction network comprises:
constructing a vehicle pattern recognition feature extraction network based on Lip-ResNet 50;
initializing the vehicle pattern recognition feature extraction network using pre-trained network parameters on ImageNet;
a vehicle pattern recognition feature extraction network is trained using a vehicle pattern recognition dataset.
S212: extracting a plurality of hierarchical features of an image to be identified through a feature extraction network, wherein the plurality of hierarchical features comprise: a brand level, a sub-level, and an identity level;
the plurality of hierarchical features includes brand features corresponding to a brand hierarchy;
the vehicle year money characteristics and the vehicle sub money characteristics corresponding to the sub money levels;
and (4) the identity level corresponds to the vehicle-line characteristic.
In some embodiments, the network for extracting the fingerprint identification features includes a backbone network and a branch network, and further includes:
parameters of the backbone network are commonly used by a plurality of levels in the branch network;
parameters of a branching network are used independently by each hierarchy, the branching network including brand-level branches, sub-level branches, and identity-level branches.
In some embodiments, the backbone network is comprised of conv _1, conv _2x, the brand hierarchy is comprised of conv _3x, conv _4x, and the vehicle sub-hierarchy and identity hierarchy are comprised of conv _3x, conv _4x, conv _5 x.
In the embodiment, the vehicle characteristic information can be extracted by constructing the vehicle pattern recognition characteristic extraction network, accurate input reduction learning sample data is provided for subsequent characteristic learning and vehicle pattern recognition, and the recognition efficiency and accuracy are further improved.
Fig. 3 is a flowchart of a method for identifying a vehicle print according to another embodiment of the present application, and as shown in fig. 3, the method for identifying a vehicle print includes:
s31: performing global average pooling on the brand hierarchy branches, the sub-fund hierarchy branches and the identity hierarchy branches, and inputting the branches into a multi-granularity feature learning model based on a graph network to obtain multi-granularity hierarchy embedded features;
s32: embedding features into multiple granularity levels, inputting the features into a classifier, and calculating classification losses on a brand level and a sub-money level;
s33: the multi-granularity level embedded features are input into a multi-granularity ranking loss module based on metric learning to calculate the ranking loss.
In some embodiments, further comprising: the method for constructing the multi-granularity feature learning model based on the graph network specifically comprises the following steps:
s311: constructing a hierarchical relation graph of a plurality of hierarchical characteristics;
the hierarchical relationship diagram is shown in fig. 4, and includes:
l1，l2，...，lLis L levels of granularity, arranged from high to low, level LkHas ckA category; f. of1，f2，...，fLRepresenting l by a feature extraction network1，l2，...，lLThe characteristics of the granularity level.
The whole hierarchical relation graph is represented as G ═ V, E }, wherein V is a characteristic node set and consists of all classes of L granularity levels; e is a collection of edges representing the relationship of classes between any two different levels of granularity in V. In particular, V can be expressed as
VkIs a subset of V, represented by level lLThe point composition of (2).
Representing point viAnd point vjWith no directional edge in between.
S312: training the graph convolutional neural network model according to the hierarchical relationship graph to obtain the weight and the bias of the full connection layer of the graph convolutional neural network model;
node vi∈VkLevel l for status characterization ofkIs initialized with the logic value of class i, level lkBy fkIs calculated to be Zk＝Wkfk+bk。
Is level lkOf logits vector, zi∈ZkIs corresponding to viThe location of (1). Wk，bkIs the weight and bias of the fully connected layer. For initializing viIs expressed as
Wherein, 0*Representing a 0 vector. x is the number ofiIs a c1+c2+…+ckThe dimension vector has all values of 0 for the dimensions other than the ith dimension.
Multi-granular feature learning is performed by stacking N linear convolutional layers and non-linear layers. After the node is initialized, the learning process formula is
Wherein the content of the first and second substances,
consisting of | V | points each having a feature vector of
And (5) maintaining.
Is a transformation matrix of the image data to be transformed,
is | V | dimensions of
Is used to generate the vector.
A is the adjacency matrix derived from E.
Will output MnSplitting into V vectors to obtain a characteristic vector of each node
The level lkIs concatenated with the node vector of fkThe phases are concatenated to obtain the vector f 'that is ultimately used for classification'kFinal level lkThe categories of (2) are pre-measured as:
pk＝Softmax(W′kf′k+b′k)
the weight and bias of the fully connected layer can be obtained by the above formula.
S313: and constructing a multi-granularity feature learning model based on the graph network according to the weight and the bias of the full connection layer of the graph convolution neural network model.
In some embodiments, as shown in fig. 5, the method for identifying the car-line includes the following steps:
step 1: inputting a picture, obtaining the characteristics of three levels of a vehicle brand, a sub-payment and an identity by using a characteristic extraction network, then inputting the characteristics into a multi-granularity characteristic learning model based on the graph network to obtain the characteristics of the three levels embedded through a multi-granularity hierarchical relation, then classifying the vehicle brand and the sub-payment granularity levels by using a Softmax classifier, calculating Euclidean distances between the characteristics and targets on the vehicle identity level, sequencing the calculation results, and taking the target with the nearest Euclidean distance as the vehicle fingerprint identification result.
Step 2: during training, cross entropy loss is utilized on a vehicle brand and sub-granularity level, multi-granularity sequencing loss is utilized on a vehicle identity level, and a vehicle pattern recognition data set is utilized to carry out end-to-end training on a feature extraction network and a multi-granularity feature learning model based on a graph network.
And step 3: during testing, inputting pictures, outputting classification results on the vehicle brand and the sub-payment granularity level by the feature extraction network, performing data screening on the vehicle pattern recognition data set according to the classification results, and retrieving the vehicle pattern features in the screened vehicle pattern recognition data set to obtain a vehicle pattern recognition result.
The traditional method of using the traditional convolutional neural network to identify and calculate the vehicle pattern of the vehicle has low identification accuracy and single identification characteristic, for example, only the brand or the model of the vehicle can be identified, and the requirement that a user wants to identify more information of the vehicle at the same time cannot be met.
In this embodiment, vehicle line characteristic information, vehicle brand information and vehicle money information are discerned simultaneously, not only can promote the discernment rate of accuracy, still satisfy the demand that the user wants to discern more characteristic information of vehicle simultaneously, and further, through many granularities level characteristic coupling learning frame, improve the characteristic learning of each granularity with the mode of promoting jointly, provide an effective constraint for the division of semantic space, thereby help the algorithm focus on more subtle characteristic accurate recognition vehicle.
In the embodiment, the graph neural network is used for learning the semantic relation among the multiple granularity layers for the first time, the feature learning of each granularity is improved in a common promotion mode, and the graph convolution neural network is embedded into the model, so that end-to-end training is realized.
Fig. 6 is a flowchart of a method for identifying a vehicle print according to another embodiment of the present application, and as shown in fig. 6, the method for identifying a vehicle print includes:
s61: acquiring quintuple samples from the pattern identification dataset;
s62: dividing the quintuple sample into image pairs, and classifying the image pairs;
s63: calculating the spatial distance of each image pair in the feature space;
s64: and constructing a multi-granularity sorting loss function according to the spatial distance.
In some embodiments, constructing a multi-granular ordering loss function from distance comprises:
calculating a loss term according to the spatial distance;
weighting and combining the losses to obtain a multi-granularity sorting loss function;
the loss term includes:
the coarsest granularity sorting loss item is used for restricting the distance between the image corresponding to the brand level and the target image in the feature space to be larger than the distance between the image corresponding to the sub-money level and the target image so as to increase the discrimination between coarsest granularity levels;
the coarse-grained sequencing loss item is used for restricting the distance between the image corresponding to the sub-money level and the target image in the feature space to be larger than the distance between the image corresponding to the identity level and the target image so as to increase the discrimination between coarse-grained levels;
the fine-grained sequencing loss items are used for increasing the distinguishability among different identity levels under the same sub-level and reducing the difference of identity classes in the same identity level;
the sample pair loss item is used for restricting the distance of the target images belonging to the same fine-grained category in the feature space to be as small as possible;
and classifying the loss items by fine granularity categories, namely cross entropy loss, and reducing the difference in the levels and accelerating the convergence speed of the loss function.
As shown in FIG. 7, assume a training set
There are N pictures corresponding to V vehicle identity categories and M vehicle sub-money categories, T vehicle brand categories. All the image pairs are classified into four categories according to whether the two target images belong to a certain granularity or not:
p: belonging to the same vehicle identity.
Nv: belong to different vehicle identities, but belong to the same vehicle fund.
Nm: belonging to different vehicle sub-types and the same vehicle brand.
Nt: belonging to different vehicle brands.
The four classes respectively correspond to four semantic similarity relations between targets, and for any quintuple sample { i, j, l, k, o } in the training set, let { i, l } belong to P and { i, j } belong to NvThe { i, k } belongs to NmThe { i, o } belongs to NtIn the feature space, the following distance relationship is as follows:
D(i，o)＞D(i，k)＞D(i，j)＞D(i，l)
the multi-granularity ordering loss function is constructed as follows:
L＝Rt+ε1Rc+ε2Rf+ε3P+ε4C
wherein R istRepresenting a coarsest-grained ordering penalty term by constraining objects from different vehicle type classes to be one interval M further in feature space than objects belonging to the same annuity classtTo increase the discrimination between the coarsest granularity class classes as much as possible. The specific form is expressed as follows:
Rcrepresenting coarse-grained ordering penalty terms by constraining objects from different annuity classes to be more in feature space than objects belonging to the same vehicle identityOne interval M is marked far awaycTo increase the discrimination between coarse-grained classes as much as possible. The specific form is expressed as follows:
Rfrepresenting fine-grained ordering penalty terms aimed at further increasing the separation M between different vehicle identity classes belonging to the same yearfWhile reducing variability within the same vehicle identity class. It is written in detail as follows:
p is a sample pair penalty to enhance the intra-class compactness of the same fine-grained class as much as possible, so as to map two target images of the same fine-grained class onto one point in the feature space. It is written in detail as follows:
and C is a fine-grained classification loss term, so that intra-class difference can be further reduced, and the convergence speed of a loss function is accelerated.
In the embodiment, by constructing a multi-granularity hierarchical relationship diagram and using a diagram convolution neural network and multi-granularity sorting loss, the characteristics of each granularity level are interacted, so that the purposes of mutual constraint, common promotion and improvement of the identification precision of the vehicle pattern are achieved.
An embodiment of the present invention provides a car-pattern recognition apparatus, as shown in a functional structure diagram of fig. 8, where the car-pattern recognition apparatus includes:
the building module 81 is used for building a multi-granularity level feature coupling learning framework;
the input module 82 is used for acquiring an image to be recognized, and inputting the image to be recognized into a trained multi-granularity hierarchical feature coupling learning framework to obtain a classification result of the vehicle pattern feature and the hierarchical feature;
the screening module 83 is used for screening the image data in the pattern recognition data set according to the hierarchical feature classification result;
and the calculating module 84 is configured to calculate an euclidean distance between the moire feature of the image to be identified and the moire feature of the screened image, and obtain a moire identification result according to a euclidean distance calculation result.
In the embodiment, a multi-granularity hierarchical feature coupling learning framework is constructed through a construction module, an input module acquires an image to be recognized, the image to be recognized is input into the trained multi-granularity hierarchical feature coupling learning framework to obtain a moire feature and a hierarchical feature classification result, a screening module screens image data in a moire recognition data set according to the hierarchical feature classification result, a calculation module calculates Euclidean distances between the moire features of the image to be recognized and the moire features of the screened image, a moire recognition result is obtained according to the Euclidean distance calculation result, feature learning of each granularity is improved through the multi-granularity hierarchical feature coupling learning framework in a common promotion mode, and efficiency and accuracy of moire recognition are improved.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that, in the description of the present application, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present application, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional component mode. The integrated module, if implemented in the form of a software functional component and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
It should be noted that the present invention is not limited to the above-mentioned preferred embodiments, and those skilled in the art can obtain other products in various forms without departing from the spirit of the present invention, but any changes in shape or structure can be made within the scope of the present invention with the same or similar technical solutions as those of the present invention.
Claims (10)
1. A method for recognizing a vehicle pattern is characterized by comprising the following steps:
constructing a multi-granularity level feature coupling learning framework;
acquiring an image to be recognized, and inputting the image to be recognized into a trained multi-granularity hierarchical feature coupling learning framework to obtain a classification result of the vehicle pattern feature and the hierarchical feature;
screening image data in the pattern recognition data set according to the hierarchical feature classification result;
and calculating the Euclidean distance between the vehicle pattern features of the image to be recognized and the vehicle pattern features of the screened image, and obtaining a vehicle pattern recognition result according to the Euclidean distance calculation result.
2. The method according to claim 1, wherein the constructing a multi-granularity level feature coupling learning framework comprises:
acquiring a plurality of hierarchical features of an image to be identified;
inputting the multiple hierarchical features into a multi-granularity feature learning model based on a graph network to obtain multi-granularity hierarchical embedded features;
calculating classification loss and sequencing loss according to the multi-granularity level characteristics;
and optimizing model parameters of a multi-granularity level feature coupling learning framework according to the classification loss and the sequencing loss to construct the multi-granularity level feature coupling learning framework.
3. The method according to claim 2, wherein the obtaining of the plurality of hierarchical features of the image to be recognized comprises:
constructing a vehicle pattern recognition feature extraction network;
extracting a plurality of hierarchical features of an image to be identified through the feature extraction network, the plurality of hierarchical features comprising: a brand level, a sub-level, and an identity level;
the plurality of hierarchical features comprises brand features corresponding to a brand hierarchy;
the vehicle year money characteristics and the vehicle sub money characteristics corresponding to the sub money levels;
and (4) the identity level corresponds to the vehicle-line characteristic.
4. The method according to claim 3, wherein the constructing the network for extracting the moire identification features comprises:
constructing a vehicle pattern recognition feature extraction network based on Lip-ResNet 50;
initializing the vehicle pattern recognition feature extraction network using pre-trained network parameters on ImageNet;
and training the car pattern recognition feature extraction network by using a car pattern recognition data set.
5. The method according to claim 4, wherein the network for extracting the moire identification features comprises a trunk network and a branch network, and further comprises:
parameters of the backbone network are commonly used by a plurality of levels in the branch network;
the parameters of the branching network are used independently by each hierarchy, the branching network including brand hierarchy branches, sub-fund hierarchy branches, and identity hierarchy branches.
6. The method according to claim 5, wherein the calculating classification loss and sorting loss according to multi-granularity hierarchical features comprises:
performing global average pooling on the brand hierarchy branches, the sub-fund hierarchy branches and the identity hierarchy branches, and inputting the branches into a multi-granularity feature learning model based on a graph network to obtain multi-granularity hierarchy embedded features;
inputting the multi-granularity level embedded features into a classifier, and calculating classification losses on a brand level and a sub-cost level;
inputting the multi-granular hierarchy embedded features into a multi-granular ordering loss module based on metric learning to calculate an ordering loss.
7. The car-line recognition method according to claim 2, further comprising: constructing a multi-granularity feature learning model based on a graph network, comprising the following steps:
constructing a hierarchical relation graph of a plurality of hierarchical characteristics;
training the graph convolutional neural network model according to the hierarchical relationship graph to obtain the weight and the bias of the full connection layer of the graph convolutional neural network model;
and constructing a multi-granularity feature learning model based on the graph network according to the weight and the bias of the full connection layer of the graph convolution neural network model.
8. The method according to claim 6, wherein said inputting the multi-granular level embedded features into a multi-granular ranking loss module based on metric learning calculates ranking losses, comprising:
acquiring quintuple samples from the pattern identification dataset;
dividing the quintuple sample into image pairs, and classifying the image pairs;
calculating the spatial distance of each image pair in the feature space;
and constructing a multi-granularity sorting loss function according to the space distance.
9. The method according to claim 8, wherein the constructing a multi-granularity ranking loss function according to the spatial distance comprises:
calculating a loss term according to the spatial distance;
carrying out weighted combination on the losses to obtain a multi-granularity sorting loss function;
the loss term includes:
the coarsest granularity sorting loss item is used for restricting the distance between the image corresponding to the brand level and the target image in the feature space to be larger than the distance between the image corresponding to the sub-money level and the target image so as to increase the discrimination between coarsest granularity levels;
the coarse-grained sequencing loss item is used for restricting the distance between the image corresponding to the sub-money level and the target image in the feature space to be larger than the distance between the image corresponding to the identity level and the target image so as to increase the discrimination between coarse-grained levels;
the fine-grained sequencing loss items are used for increasing the distinguishability among different identity levels under the same sub-level and reducing the difference of identity classes in the same identity level;
the sample pair loss item is used for restricting the distance of the target images belonging to the same fine-grained category in the feature space to be as small as possible;
and classifying loss items by fine-grained categories, which are cross entropy losses of the identity levels, are used for reducing differences in the levels and simultaneously accelerating the convergence speed of the loss function.
10. A car line recognition device, comprising:
the construction module is used for constructing a multi-granularity level feature coupling learning framework;
the input module is used for acquiring an image to be recognized and inputting the image to be recognized into a trained multi-granularity hierarchical feature coupling learning framework to obtain a classification result of the vehicle pattern feature and the hierarchical feature;
the screening module is used for screening the image data in the pattern recognition data set according to the hierarchical feature classification result;
and the calculation module is used for calculating the Euclidean distance between the vehicle pattern features of the image to be identified and the vehicle pattern features of the screened image, and obtaining a vehicle pattern identification result according to the Euclidean distance calculation result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110326344.1A CN112990048B (en) | 2021-03-26 | 2021-03-26 | Vehicle pattern recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110326344.1A CN112990048B (en) | 2021-03-26 | 2021-03-26 | Vehicle pattern recognition method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112990048A true CN112990048A (en) | 2021-06-18 |
| CN112990048B CN112990048B (en) | 2021-11-23 |
Family
ID=76333871
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110326344.1A Active CN112990048B (en) | 2021-03-26 | 2021-03-26 | Vehicle pattern recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112990048B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107346328A (en) * | 2017-05-25 | 2017-11-14 | 北京大学 | A kind of cross-module state association learning method based on more granularity hierarchical networks |
| CN109359696A (en) * | 2018-10-29 | 2019-02-19 | 重庆中科云丛科技有限公司 | A kind of vehicle money recognition methods, system and storage medium |
| CN110659374A (en) * | 2019-09-19 | 2020-01-07 | 江苏鸿信系统集成有限公司 | Method for searching images by images based on neural network extraction of vehicle characteristic values and attributes |
| CN111178432A (en) * | 2019-12-30 | 2020-05-19 | 武汉科技大学 | Weak supervision fine-grained image classification method of multi-branch neural network model |
| CN111814920A (en) * | 2020-09-04 | 2020-10-23 | 中国科学院自动化研究所 | Fine classification method and system for multi-granularity feature learning based on graph network |
| CN112489436A (en) * | 2020-10-29 | 2021-03-12 | 浙江预策科技有限公司 | Vehicle identity recognition method, device and system and electronic device |
-
2021
- 2021-03-26 CN CN202110326344.1A patent/CN112990048B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107346328A (en) * | 2017-05-25 | 2017-11-14 | 北京大学 | A kind of cross-module state association learning method based on more granularity hierarchical networks |
| CN109359696A (en) * | 2018-10-29 | 2019-02-19 | 重庆中科云丛科技有限公司 | A kind of vehicle money recognition methods, system and storage medium |
| CN110659374A (en) * | 2019-09-19 | 2020-01-07 | 江苏鸿信系统集成有限公司 | Method for searching images by images based on neural network extraction of vehicle characteristic values and attributes |
| CN111178432A (en) * | 2019-12-30 | 2020-05-19 | 武汉科技大学 | Weak supervision fine-grained image classification method of multi-branch neural network model |
| CN111814920A (en) * | 2020-09-04 | 2020-10-23 | 中国科学院自动化研究所 | Fine classification method and system for multi-granularity feature learning based on graph network |
| CN112489436A (en) * | 2020-10-29 | 2021-03-12 | 浙江预策科技有限公司 | Vehicle identity recognition method, device and system and electronic device |
Non-Patent Citations (2)
| Title |
|---|
| H. GUO 等: "Two-Level Attention Network With Multi-Grain Ranking Loss for Vehicle Re-Identification", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| X. MA 等: "Vehicle Re-Identification with Refined Part Model", 《2019 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA & EXPO WORKSHOPS (ICMEW)》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112990048B (en) | 2021-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111539370B (en) | Image pedestrian re-identification method and system based on multi-attention joint learning | |
| Huttunen et al. | Car type recognition with deep neural networks | |
| US9129524B2 (en) | Method of determining parking lot occupancy from digital camera images | |
| CN111767882A (en) | Multi-mode pedestrian detection method based on improved YOLO model | |
| Derpanis et al. | Classification of traffic video based on a spatiotemporal orientation analysis | |
| Huo et al. | Vehicle type classification and attribute prediction using multi-task RCNN | |
| Tesfaye et al. | Multi-target tracking in multiple non-overlapping cameras using fast-constrained dominant sets | |
| CN109784392A (en) | A kind of high spectrum image semisupervised classification method based on comprehensive confidence | |
| CN103679187B (en) | Image-recognizing method and system | |
| CN102236675A (en) | Method for processing matched pairs of characteristic points of images, image retrieval method and image retrieval equipment | |
| CN110147707B (en) | High-precision vehicle identification method and system | |
| CN110751027B (en) | Pedestrian re-identification method based on deep multi-instance learning | |
| CN108647595A (en) | Vehicle recognition methods again based on more attribute depth characteristics | |
| CN112990282B (en) | Classification method and device for fine-granularity small sample images | |
| Arth et al. | Object reacquisition and tracking in large-scale smart camera networks | |
| Ji et al. | A hybrid model of convolutional neural networks and deep regression forests for crowd counting | |
| Wang et al. | Small vehicle classification in the wild using generative adversarial network | |
| Sahli et al. | Robust vehicle detection in low-resolution aerial imagery | |
| Tripathi et al. | Object detection using YOLO: A survey | |
| CN110619280A (en) | Vehicle heavy identification method and device based on deep joint discrimination learning | |
| Bhattacharyya et al. | JUVDsi v1: developing and benchmarking a new still image database in Indian scenario for automatic vehicle detection | |
| Yuan et al. | A multi-image Joint Re-ranking framework with updateable Image Pool for person re-identification | |
| CN112990048B (en) | Vehicle pattern recognition method and device | |
| CN115050044B (en) | Cross-modal pedestrian re-identification method based on MLP-Mixer | |
| Ebren et al. | Determining the occupancy of Vehicle Parking Areas by deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |