CN113159115A - Vehicle fine-grained identification method, system and device based on neural architecture search - Google Patents
Vehicle fine-grained identification method, system and device based on neural architecture search Download PDFInfo
- Publication number
- CN113159115A CN113159115A CN202110262167.5A CN202110262167A CN113159115A CN 113159115 A CN113159115 A CN 113159115A CN 202110262167 A CN202110262167 A CN 202110262167A CN 113159115 A CN113159115 A CN 113159115A
- Authority
- CN
- China
- Prior art keywords
- network
- fine
- vehicle
- grained identification
- grained
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a vehicle fine-grained identification method based on neural architecture search, which comprises the steps of carrying out vehicle fine-grained identification on an acquired image by utilizing a predetermined multi-path network, and fusing a plurality of classification results output by the multi-path network to obtain a final identification result; the determining of the multipath network comprises: acquiring a source domain vehicle fine-grained identification data set and a target domain vehicle fine-grained identification data set; identifying a data set in a fine granularity of a vehicle in a source domain, and searching an internal connection structure of a network architecture unit when the preset initial network performance is optimal based on a preset loss function to obtain an optimal network architecture unit; repeatedly stacking the optimal network architecture unit to obtain an untrained multipath network; and training parameters of the multipath network by adopting a preset loss function based on the target domain vehicle fine-grained identification data set to obtain the trained multipath network. The invention can solve the defect that the neural architecture searching method is difficult to expand to a fine-grained identification task.
Description
Technical Field
The invention relates to a vehicle fine-grained identification method, a system and a device based on neural architecture search, and belongs to the technical field of computer vision.
Background
The neural architecture search is a technology for automatically searching a network architecture through a machine, and usually selects an optimal operation combination from a set constructed by a large number of atomic operations such as convolution, pooling and the like to form an optimal network architecture unit, so as to construct a new network. The neural architecture search abandons the idea of manually designing a network architecture, and the task of designing the neural network architecture is completed by automatic search of a machine.
The vehicle fine-grained recognition is an extension of a classical classification task, is different from the classical general target recognition in which the target categories are different greatly, and the vehicle fine-grained recognition needs to distinguish vehicles of different subclasses under the large category. Because the vehicles to be classified belong to the same large class, the difference between the classes is relatively small, and meanwhile, because different individual vehicles of the same type have the influence of various factors such as a shooting visual angle, a distance, illumination, shielding and the like, the intra-class difference phase pair is relatively large, so that the difficulty of vehicle fine-grained identification is much higher than that of a general target identification task. At the present stage, neural architecture search is mainly used for a general target identification task to search an optimal network architecture, and is difficult to directly expand to a fine-grained identification task due to the lack of consideration of target function design, network skeleton construction and the like from the fine-grained identification angle.
Disclosure of Invention
The invention aims to overcome the defects in the prior art, provides a vehicle fine-grained identification method, a system and a device based on neural architecture search, and can solve the defect that the neural architecture search method in the prior art is difficult to directly and effectively expand to a fine-grained identification task. In order to achieve the purpose, the invention is realized by adopting the following technical scheme:
in a first aspect, the invention provides a vehicle fine-grained identification method based on neural architecture search, which comprises the following steps:
acquiring an image containing a target to be identified;
carrying out vehicle fine-grained identification on the image by using a predetermined multi-path network, and fusing a plurality of classification results output by the multi-path network by using a preset fusion method to obtain a final identification result;
wherein the multipath network is obtained by:
acquiring a source domain vehicle fine-grained identification data set and a target domain vehicle fine-grained identification data set;
identifying a data set in a fine granularity of a vehicle in a source domain, and searching an internal connection structure of a network architecture unit when the preset initial network performance is optimal based on a preset loss function to obtain an optimal network architecture unit;
repeatedly stacking the optimal network architecture unit to obtain an untrained multipath network;
and training parameters of the multipath network by adopting a preset loss function based on the target domain vehicle fine-grained identification data set to obtain the trained multipath network.
With reference to the first aspect, preferably, the source domain vehicle fine-grained identification dataset is obtained by one or more of the following means: constructing one or more general vehicle fine-grained identification data sets or general vehicle re-identification data sets which can be obtained by starting; shooting and recording the vehicle by adopting shooting equipment and marking the vehicle; and searching, downloading and labeling from the Internet by adopting a search engine.
With reference to the first aspect, preferably, the target domain vehicle fine-grained identification dataset is obtained by one or more of the following means: shooting and recording the target domain vehicle by adopting shooting equipment and marking to obtain the target domain vehicle; and searching, downloading and labeling from the Internet by adopting a search engine.
With reference to the first aspect, in particular, the target area vehicle refers to industry-specific vehicles, including military armored vehicles and engineering construction vehicles.
In combination with the first aspect, preferably, the method of searching is DARTS or PDARTS.
With reference to the first aspect, further, the preset initial network is formed by stacking network architecture units in several neural architecture searches, and the internal connection structure of each network architecture unit is the same and determined through searching.
With reference to the first aspect, preferably, when the preset initial network performance is optimal, the internal connection structure of the corresponding network architecture unit is optimal, so as to obtain an optimal network architecture unit.
With reference to the first aspect, further, the preset loss function is:
L＝LPM+λLCE (1)
(1) wherein λ represents LCEAnd LPMWeight between, LPMA class-centered metric loss function is expressed by the following formula:
(2) in the formula (f)i kRepresenting the kth sample feature in the ith vehicle,
representing the centre of a sample feature in a class i vehicle, the function g representing the metric fi kAnd
a distance function of (d);
(1) in the formula, LCEA classification loss function is expressed by the following formula:
(3) in the formula (f)kTo representCharacteristic of the kth sample, ykA true label representing the kth sample, and yk∈{1，2，...，C}，
Representing a feature fkMapping to a genuine tag class ykWeight parameter of WjRepresenting a feature fkAnd C is the total number of the categories.
With reference to the first aspect, further, the center-like metric loss function needs to configure the following data sampling method:
randomly selecting P categories using a no-put-back sample; for each selected category, Q samples are randomly selected again using the sample-not-put-back, and finally a batch of sample data totaling PQ samples is constructed.
With reference to the first aspect, it is preferable that the center of the sample feature in the i-th vehicle is expressed
And calculating the average value of the ith class Q samples in the batch data.
With reference to the first aspect, preferably, the multi-path network is composed of a plurality of branch paths and is divided into N computation stages, and each computation stage in the multi-path network is obtained by repeatedly stacking optimal network architecture units.
With reference to the first aspect, preferably, after the image data of the source-domain vehicle fine-grained identification dataset is input into the network, the output of each calculation stage is input into the next stage, or input into another branch path, and N feature outputs and the classification prediction probabilities are generated in total at stage N.
In combination with the first aspect, preferably, the preset loss function is used to guide the calculation.
With reference to the first aspect, preferably, the optimal network architecture unit is repeatedly stacked according to a certain rule, where the certain rule includes that a plurality of neuron cells are linearly connected in series at each stage, and have input and output sizes with the same dimension.
With reference to the first aspect, further, the parameters of the initial network of the trained multipath network are trained by using a batch stochastic gradient descent algorithm, where a parameter update rule is as follows:
(4) in the formula, wtIs the parameter value at the t-th iteration,
for the target loss vs. parameter wtPartial derivative of (1), learning rate etatUpdating according to a cosine simulated annealing strategy with preheating, wherein the calculation rule is as follows:
(5) in the formula eta0To maximum learning rate, t0Number of iterations for preheating, ttotalIs the total number of iterations.
With reference to the first aspect, further, the preset fusion method includes weighted fusion and product fusion:
wherein the weighted fusion is achieved by the following formula:
(6) in the formula (I), the compound is shown in the specification,
to weight the prediction probability of the fused ith class,
the probability of the ith class output for the mth path, wherein M is the number of classifiers, namely the number of branch paths, and C is the number of classes;
wherein the product fusion is achieved by the following formula:
(7) in the formula (I), the compound is shown in the specification,
to weight the prediction probability of the fused ith class,
the probability of the ith class output for the mth path, M is the number of classifiers, i.e., the number of branch paths, and C is the number of classes.
With reference to the first aspect, further, when the multi-path network is used to perform fine-grained identification on the vehicle and fuse a plurality of classification results output by the multi-path network, any one of weighted fusion and product fusion in a preset fusion method is adopted.
In a second aspect, the present invention provides a vehicle fine-grained identification system based on neural architecture search, including:
an acquisition module: the method comprises the steps of obtaining an image containing a target to be identified;
a fusion recognition module: the system is used for carrying out vehicle fine-grained identification on the image by utilizing a predetermined multi-path network, fusing a plurality of classification results output by the multi-path network by adopting a preset fusion method, and obtaining a final identification result;
wherein, still include the network construction module used for confirming the said multipath network, including:
a dataset acquisition module: the method comprises the steps of acquiring a source domain vehicle fine-grained identification data set and a target domain vehicle fine-grained identification data set;
a search module: the method comprises the steps that a data set is identified in a fine-grained mode of a vehicle in a source domain, and an internal connection structure of a network architecture unit is searched for when the preset initial network performance is optimal based on a preset loss function, so that the optimal network architecture unit is obtained;
stacking the building blocks: the optimal network architecture unit is used for repeatedly stacking the optimal network architecture unit to obtain an untrained multipath network;
a training module: the method is used for training parameters of the multipath network by adopting a preset loss function based on a target domain vehicle fine-grained identification data set to obtain the trained multipath network.
With reference to the first aspect, the method further includes an initial network building module, where the initial network is built by stacking network architecture units in the neural architecture search, and the initial network is an initial network for fine-grained vehicle identification.
In a third aspect, the invention provides a vehicle fine-grained identification device based on neural architecture search, comprising a processor and a storage medium;
the storage medium is used for storing instructions;
the processor is configured to operate in accordance with the instructions to perform the steps of the method of the first aspect.
In a fourth aspect, the invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of the first aspect.
Compared with the prior art, the fine-grained identification method for the vehicle based on the neural architecture search has the beneficial effects that:
the method utilizes the multi-path network to identify the fine granularity of the vehicle of the image containing the identification target, and integrates a plurality of classification results output by the multi-path network to obtain a final identification result; the fusion step integrates the prediction probabilities of a plurality of paths to carry out final classification decision, and even if a single path has classification decision errors, the decision result of multi-path fusion can also obtain a correct decision result; compared with a single path network, the multi-path network has robustness, reliability and strong generalization capability;
the multipath network is obtained by repeatedly stacking the optimal network architecture units, wherein the internal connection structure of the optimal network architecture units is obtained by guiding search in a source domain vehicle fine-grained identification data set through a loss function, parameters of the multipath network are trained through the loss function based on a target domain vehicle fine-grained identification data set, the multipath network more suitable for vehicle fine-grained identification can be obtained, the defect that a neural architecture search method in the prior art is difficult to directly and effectively expand to a fine-grained identification task can be overcome, and the accuracy and the robustness of a fine-grained identification result can be improved.
Drawings
Fig. 1 is a flowchart of a fine-grained identification method for a vehicle based on neural architecture search according to an embodiment of the present invention;
fig. 2 is a structural diagram of a multipath network in a fine-grained identification method for a vehicle based on neural architecture search according to an embodiment of the present invention;
fig. 3 is a comparison graph of a learning rate change curve in a neural-architecture-search-based fine-grained vehicle identification method according to an embodiment of the present invention;
fig. 4 is a structural diagram of a fine-grained identification system of a vehicle based on a neural architecture search according to a second embodiment of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
The first embodiment is as follows:
as shown in fig. 1, an embodiment of the present invention provides a fine-grained identification method for a vehicle based on a neural architecture search, including:
acquiring an image containing a target to be identified;
carrying out vehicle fine-grained identification on the image by using a predetermined multi-path network, and fusing a plurality of classification results output by the multi-path network by using a preset fusion method to obtain a final identification result;
wherein the multipath network is obtained by:
acquiring a source domain vehicle fine-grained identification data set and a target domain vehicle fine-grained identification data set;
identifying a data set in a fine granularity of a vehicle in a source domain, and searching an internal connection structure of a network architecture unit when the preset initial network performance is optimal based on a preset loss function to obtain an optimal network architecture unit;
repeatedly stacking the optimal network architecture unit to obtain an untrained multipath network;
and training parameters of the multipath network by adopting a preset loss function based on the target domain vehicle fine-grained identification data set to obtain the trained multipath network.
As shown in fig. 1, the specific steps are as follows:
step 1: an image containing an object to be recognized is acquired.
Step 2: and acquiring a source domain vehicle fine-grained identification data set and a target domain vehicle fine-grained identification data set.
It should be noted that the source domain vehicle fine-grained identification data set is larger in data size than the target domain vehicle fine-grained identification data set, and may include various general or non-general vehicles, for example: cars, buses, trucks, machineshop cars, armored cars, and the like. And the target domain vehicle mainly refers to vehicles of a specific industry, such as: tanks, walking and fighting vehicles and communication vehicles in military armored vehicles, or cranes, earth moving vehicles, road rollers and the like in engineering construction vehicles.
The source domain vehicle fine-grained identification dataset is obtained by one or more of the following means: constructing one or more general vehicle fine-grained identification data sets or general vehicle re-identification data sets which can be obtained by starting; shooting and recording the vehicle by adopting shooting equipment and marking the vehicle; and searching, downloading and labeling from the Internet by adopting a search engine. The target domain vehicle fine-grained identification data set is acquired by one or more of the following ways: shooting and recording the target domain vehicle by adopting shooting equipment and marking to obtain the target domain vehicle; and searching, downloading and labeling from the Internet by adopting a search engine.
When the source domain vehicle fine-grained identification dataset is constructed by one or more source-accessible general vehicle fine-grained identification datasets or general vehicle re-identification datasets, the source-accessible general vehicle fine-grained identification dataset includes StanfordCars and the like, the general vehicle re-identification dataset includes VeRi776 and the like, and similar data in different datasets needs to be merged, for example: if the data set a includes the vehicle type S and the data set B also includes the vehicle type S, the data set a is classified as one after the contract, and the classification is S.
When the camera device is used for shooting and recording the vehicle and labeling the vehicle to obtain a source domain/target domain vehicle fine-grained identification data set, the vehicle data of the same category should contain data under different angles, scenes, weather and other conditions as much as possible, for example, a 99-type main battle tank is shot, the data shot at angles such as forward view, backward view, side view, overlook and the like should be included, the data shot at scenes such as jungles, plains, hills, deserts, cities and the like should be included, and the data shot under different weather conditions such as sunny days, rainy days, foggy days, snowy days and the like should be included.
When a search engine is adopted to search, download and label from the Internet to obtain a source domain/target domain vehicle fine-grained identification data set, a query interface of the search engine can be used for carrying out batch downloading and then manually screening, and a crawler interface of the search engine can also be used for downloading data from a professional vehicle website and manually screening and labeling.
And randomly dividing a source domain vehicle fine-grained identification data set into a training set and a testing set according to a certain proportion, such as 1: 1 or 2: 1.
And step 3: and identifying a data set in a fine granularity of the vehicle in a source domain, and searching an internal connection structure of the network architecture unit when the preset initial network performance is optimal based on a preset loss function to obtain the optimal network architecture unit.
The network architecture unit is also called a neuron Cell (Cell), and an initial network for vehicle fine-grained identification can be constructed by stacking a plurality of cells. The Cell is composed of a plurality of nodes and operation edges, and the optimal network architecture unit is the network architecture unit when the initial network obtains the optimal performance through a certain search technology.
The Cell is composed of several computing operations, and specific computing operations may include 3 × 3, 5 × 5, and 7 × 7 depth separable convolutions, 3 × 3 and 5 × 5 dilated convolutions, 7 × 1 by 1 × 7 combined convolutions, 3 × 3 average pooling, 3 × 3 maximum pooling, jump join and zero operations, and so on.
Depending on whether the resolution is reduced or not, the neuron cells are classified into two types: normal cells and descending cells. The division reduction unit cell is executed to reduce the image characteristic resolution and increase the number of characteristic channels; the normal cells, which are executed without changing the size of the input and output, may be repeatedly stacked to increase the depth of the network.
Specifically, the preset initial network is formed by stacking a plurality of network architecture units in the neural architecture search, and the internal connection structure of each network architecture unit is the same and determined through the search. When the preset initial network performance is optimal, the internal connection structure of the corresponding network architecture unit is optimal, and the optimal network architecture unit is obtained.
The preset loss function is a weighted function of the class-center measurement loss function and the classification loss function and is expressed by the following formula:
L＝LPM+λLCE (1)
(1) wherein λ represents LCEAnd LPMWeight between, LPMA class-centered metric loss function is expressed by the following formula:
(2) in the formula (f)i kRepresenting the kth sample feature in the ith vehicle,
representing the centre of a sample feature in a class i vehicle, the function g representing the metric fi kAnd
a distance function of (d);
(1) in the formula, LCEA classification loss function is expressed by the following formula:
(3) in the formula (f)kDenotes the kth sample feature, ykA true label representing the kth sample, and yk∈{1，2，...，C}，
Representing a feature fkMapping to a genuine tag class ykWeight parameter of WjRepresenting a feature fkAnd C is the total number of the categories.
Matching with the sampling method requirement of the class center measurement loss function: firstly, randomly selecting P categories by using non-replacement sampling; then, for each selected category, Q samples are randomly selected again by using the non-return sampling, and finally a batch of sample data with the total number of PQ samples is constructed. Wherein, the selection of P and Q is set according to experience and equipment capability, for example: q is 4, P is 16.
Center of sample feature
And calculating the average value of the ith class Q samples in the batch data.
Searching for an optimal network architecture element: under the guidance of a preset loss function, DARTS technology or PDARTS technology is adopted to search the internal connection structure of a network architecture unit in the initial network in the fine-grained identification data set of the source domain vehicle, so that the initial network obtains the best performance. It should be noted that the number of internal nodes of the network architecture unit (i.e., the neuron cell) is fixed, and the search process is equivalent to selecting 1 from several calculation operations for any two nodes as an operation edge, thereby obtaining an optimal network architecture unit.
Specifically, the DARTS technology is described in Hanxiao Liu, Karen simony, Yiming Yang, DARTS: difference Architecture Search [ C ]// International Conference on Learning retrieval (ICLR 2019), May 6-9, 2019, New Orleanes, LA, USA.
In particular, the methods related to the PDARTS technology are described in Xin Chen, Lingxixie, Jun Wu, Qi Tian, Progressive DARTS: bridging the Optimization Gap for NAS in the Wild [ J ] int.J.Compout Vis., 2020.
The initial network, the loss function and the searching method can search the optimal internal connection structure of the network architecture unit through continuous iterative optimization, train a model on the training set and evaluate the performance of the model on the verification set. The process is repeated for a plurality of times, and the network architecture unit with the best evaluation performance is selected as the optimal network architecture unit.
And 4, step 4: and repeatedly stacking the optimal network architecture unit according to a certain rule to obtain an untrained multipath network, and training parameters of the multipath network by adopting a preset loss function based on a target domain vehicle fine-grained identification data set to obtain the trained multipath network.
Specifically, a rule includes that each stage linearly connects a plurality of neuron cells in series, and has the input and output sizes with the same dimension.
For example: stage 1 includes neuronal cells A, B, C connected in series, where the output magnitude of A is equal to the output magnitude of B and equal to the output magnitude of C. Stage 2 includes neuronal cells D, E, F, G connected in series, where the output magnitude of D is equal to the output magnitude of E, equal to the output magnitude of F, and equal to the output magnitude of G. Stage 3, …, stage N, similar to ….
The certain rules further include a number of neuron cell hopping connections per stage.
For example: stage 2 includes neuronal cells D, E, F, G, connected in sequence, where D, in addition to direct connection E, spans E, connection F, connection G; similarly, E connects G across F in addition to connecting F directly.
The certain rules also include that different stages may have different input-output sizes.
For example: the stage 1 output dimension is W1 × H1 × C1, the stage 2 output dimension is W2 × H2 × C2, and the stage 3 output dimension is W3 × H3 × C3, …, where W1 is not equal to W2 and not equal to W3, …, H1 is not equal to H2 and not equal to H3, …, C1 is not equal to C2 and not equal to C3, ….
The different stages may have different input and output sizes, and specifically, the following strategies may be adopted: in the multipath network, after data is input into the next stage, the first neuron cell of each stage is the descending cell; or after inputting the next path, the first neuron cell is the descending cell.
The certain rules in turn include one or more phases with different numbers per path.
For example: path 1 comprises stages 1, 2, 3, 4, 5, path 2 comprises stages 2, 3, 4, 5, path 3 comprises stages 3, 4, 5, path 4 comprises stages 4, 5, path 5 comprises stage 5.
The certain rule comprises different path numbers and stage numbers.
For example: as shown in fig. 2(b), the input of path 2 is the output of stage 2 of path 1, the number of paths is 3, and the number of stages is 4.
The preset loss function is a weighted function of a class-center measurement loss function and a classification loss function, the class-center loss function acts on the extracted features of each path, and the features are obtained by certain mapping operation of a result output by the last stage. Wherein the certain operation may be any one of the following operations: (a) maximum pooling operation; (b) performing mean value pooling operation; (c) maximum pooling and mean pooling; (d) and (5) tensor straightening operation. The feature acted by the center-like loss function can be subjected to a batch normalization operation, and the feature normalization can improve the consistency of measurement.
The parameters of the initial network of the training multipath network are trained by adopting a batch stochastic gradient descent algorithm, wherein the parameter updating rule is as follows:
(4) in the formula, wtIs the parameter value at the t-th iteration,
for the target loss vs. parameter wtPartial derivative of (1), learning rate etatUpdating according to a cosine simulated annealing strategy with preheating, wherein the calculation rule is as follows:
(5) in the formula eta0To maximum learning rate, t0Number of iterations for preheating, ttotalIs the total number of iterations.
For example: an effective setting is eta0＝0.0035，t0＝5，ttotal＝200。
FIG. 3 shows ttotal＝200，η0Under the condition of 0.0035, the iteration times t of different preheating0Learning rate variation curves of 5, 10 and 20. Meanwhile, a change curve of the cosine simulated annealing learning rate without preheating is given.
According to fig. 3, it can be found that the learning rate of the cosine simulated annealing with preheating is gradually increased from 0 at the beginning, so that the learning has a slow adaptation process to the data, the optimization process can be accelerated by maintaining a larger value at the early stage, the target is locally optimized by quickly decreasing at the middle stage, and the learning rate is gradually decreased at the later stage, so that the optimization is gradually converged. The initial linear change process reflects the preheating process of the model, and the learning result can be better improved.
And the multi-path network can learn to obtain model parameters capable of executing a prediction task under the condition that the batch random gradient descent optimization strategy is executed by the preheated cosine simulated annealing learning rate.
And 5: and carrying out vehicle fine-grained identification on the image by using a predetermined multipath network, and fusing a plurality of classification results output by the multipath network by using a preset fusion method to obtain a final identification result.
The preset fusion method comprises weighted fusion and product fusion, the multipath network is used for carrying out fine-grained identification on the vehicle, and any one of weighted fusion and product fusion in the preset fusion method is adopted when a plurality of classification results output by the multipath network are fused.
Wherein the weighted fusion is achieved by the following formula:
(6) in the formula (I), the compound is shown in the specification,
to weight the prediction probability of the fused ith class,
the probability of the ith class output for the mth path, wherein M is the number of classifiers, namely the number of branch paths, and C is the number of classes;
wherein the product fusion is achieved by the following formula:
(7) in the formula (I), the compound is shown in the specification,
to weight the prediction probability of the fused ith class,
the probability of the ith class output for the mth path, M is the number of classifiers, i.e., the number of branch paths, and C is the number of classes.
When the multi-path network executes the fine-grained identification task, the probability of the ith class output from the mth path is
If the total number of branch paths is M, namely M classifiers exist, and the number of the classes classified by each classifier is C, performing weighted fusion or product fusion to perform multiple prediction nodesAnd (6) combining fruits.
Example two:
as shown in fig. 4, an embodiment of the present invention provides a vehicle fine-grained identification system based on neural architecture search, including:
an acquisition module: the method comprises the steps of obtaining an image containing a target to be identified;
a fusion recognition module: carrying out vehicle fine-grained identification on the image by using a predetermined multi-path network, and fusing a plurality of classification results output by the multi-path network by using a preset fusion method to obtain a final identification result;
wherein, still include the network construction module used for confirming the said multipath network, including:
a dataset acquisition module: the method comprises the steps of acquiring a source domain vehicle fine-grained identification data set and a target domain vehicle fine-grained identification data set;
a search module: the method comprises the steps that a data set is identified in a fine-grained mode of a vehicle in a source domain, and an internal connection structure of a network architecture unit is searched for when the preset initial network performance is optimal based on a preset loss function, so that the optimal network architecture unit is obtained;
stacking the building blocks: the optimal network architecture unit is used for repeatedly stacking the optimal network architecture unit to obtain an untrained multipath network;
a training module: the method is used for training parameters of the multipath network by adopting a preset loss function based on a target domain vehicle fine-grained identification data set to obtain the trained multipath network.
Specifically, the method further comprises an initial network construction module, wherein the initial network is obtained by utilizing the network architecture unit stacking construction in the neural architecture search, and the initial network is an initial network for vehicle fine-grained identification.
Example three:
the embodiment of the invention provides a vehicle fine-grained identification device based on neural architecture search, which comprises a processor and a storage medium, wherein the processor is used for processing a vehicle fine-grained identification;
the storage medium is used for storing instructions;
the processor is configured to operate in accordance with the instructions to perform the steps of the method of embodiment one.
Example four:
embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the method according to one embodiment.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A vehicle fine-grained identification method based on neural architecture search is characterized by comprising the following steps:
acquiring an image containing a target to be identified;
carrying out vehicle fine-grained identification on the image by using a predetermined multi-path network, and fusing a plurality of classification results output by the multi-path network by using a preset fusion method to obtain a final identification result;
wherein the multipath network is obtained by:
acquiring a source domain vehicle fine-grained identification data set and a target domain vehicle fine-grained identification data set;
identifying a data set in a fine granularity of a vehicle in a source domain, and searching an internal connection structure of a network architecture unit when the preset initial network performance is optimal based on a preset loss function to obtain an optimal network architecture unit;
repeatedly stacking the optimal network architecture unit to obtain an untrained multipath network;
and training parameters of the multipath network by adopting a preset loss function based on the target domain vehicle fine-grained identification data set to obtain the trained multipath network.
2. The fine-grained identification method for vehicles based on neural architecture search according to claim 1, characterized in that the preset initial network is composed of network architecture units stacked in several neural architecture searches, and the internal connection structure of each network architecture unit is the same and determined through search.
3. The fine-grained identification method for vehicles based on neural architecture search according to claim 1, characterized in that the preset loss function is:
L＝LPM+λLCE (1)
(1) wherein λ represents LCEAnd LPMWeight between, LPMA class-centered metric loss function is expressed by the following formula:
(2) in the formula (f)i kRepresenting the kth sample feature in the ith vehicle,
representing the centre of a sample feature in a class i vehicle, the function g representing the metric fi kAnd
a distance function of (d);
(1) in the formula, LCEA classification loss function is expressed by the following formula:
(3) in the formula (f)kDenotes the kth sample feature, ykA true label representing the kth sample, and yk∈{1,2,…,C}，
Representing a feature fkMapping to a genuine tag class ykWeight parameter of WjRepresenting a feature fkAnd C is the total number of the categories.
4. The fine-grained identification method for vehicles based on neural architecture search according to claim 3, characterized in that the class-centered metric loss function requires configuring the following data sampling method:
randomly selecting P categories using a no-put-back sample; for each selected category, Q samples are randomly selected again using the sample-not-put-back, and finally a batch of sample data totaling PQ samples is constructed.
5. The fine-grained identification method for vehicles based on neural architecture search according to claim 1, characterized in that the parameters of the trained multipath network are trained by using a batch stochastic gradient descent algorithm, wherein the parameter update rule is as follows:
(4) in the formula, wtIs the parameter value at the t-th iteration,
for the target loss vs. parameter wtPartial derivative of (1), learning rate etatUpdating according to a cosine simulated annealing strategy with preheating, wherein the calculation rule is as follows:
(5) in the formula eta0To maximum learning rate, t0Number of iterations for preheating, ttotalIs the total number of iterations.
6. The neural-architecture-search-based fine-grained identification method for vehicles according to claim 1, wherein the preset fusion method comprises weighted fusion and product fusion:
wherein the weighted fusion is achieved by the following formula:
(6) in the formula (I), the compound is shown in the specification,
to weight the prediction probability of the fused ith class,
the probability of the ith class output for the mth path, wherein M is the number of classifiers, namely the number of branch paths, and C is the number of classes;
wherein the product fusion is achieved by the following formula:
(7) in the formula (I), the compound is shown in the specification,
to multiply the prediction probability of the fused ith class,
the probability of the ith class output for the mth path, M is the number of classifiers, i.e., the number of branch paths, and C is the number of classes.
7. The fine-grained identification method for vehicles based on neural architecture search according to claim 6, wherein when the vehicles are subjected to fine-grained identification by using the multi-path network and a plurality of classification results output by the multi-path network are fused, any one of weighted fusion and product fusion in a preset fusion method is adopted.
8. A vehicle fine-grained identification system based on neural architecture search, comprising:
an acquisition module: the method comprises the steps of obtaining an image containing a target to be identified;
a fusion recognition module: the system is used for carrying out vehicle fine-grained identification on the image by utilizing a predetermined multi-path network, fusing a plurality of classification results output by the multi-path network by adopting a preset fusion method, and obtaining a final identification result;
wherein, still include the network construction module used for confirming the said multipath network, including:
a dataset acquisition module: the method comprises the steps of acquiring a source domain vehicle fine-grained identification data set and a target domain vehicle fine-grained identification data set;
a search module: the method comprises the steps that a data set is identified in a fine-grained mode of a vehicle in a source domain, and an internal connection structure of a network architecture unit is searched for when the preset initial network performance is optimal based on a preset loss function, so that the optimal network architecture unit is obtained;
stacking the building blocks: the optimal network architecture unit is used for repeatedly stacking the optimal network architecture unit to obtain an untrained multipath network;
a training module: the method is used for training parameters of the multipath network by adopting a preset loss function based on a target domain vehicle fine-grained identification data set to obtain the trained multipath network.
9. The fine-grained identification device of the vehicle based on neural architecture search is characterized by comprising a processor and a storage medium;
the storage medium is used for storing instructions;
the processor is configured to operate in accordance with the instructions to perform the steps of the method of any of claims 1 to 7.
10. Computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110262167.5A CN113159115B (en) | 2021-03-10 | 2021-03-10 | Vehicle fine granularity identification method, system and device based on neural architecture search |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110262167.5A CN113159115B (en) | 2021-03-10 | 2021-03-10 | Vehicle fine granularity identification method, system and device based on neural architecture search |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113159115A true CN113159115A (en) | 2021-07-23 |
| CN113159115B CN113159115B (en) | 2023-09-19 |
Family
ID=76886729
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110262167.5A Active CN113159115B (en) | 2021-03-10 | 2021-03-10 | Vehicle fine granularity identification method, system and device based on neural architecture search |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113159115B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114495027A (en) * | 2022-01-11 | 2022-05-13 | 北京科技大学 | Vehicle type fine-grained identification method and device based on network data |
| CN115424086A (en) * | 2022-07-26 | 2022-12-02 | 北京邮电大学 | Multi-view fine-granularity identification method and device, electronic equipment and medium |
| CN117036911A (en) * | 2023-10-10 | 2023-11-10 | 华侨大学 | Vehicle re-identification light-weight method and system based on neural architecture search |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109214441A (en) * | 2018-08-23 | 2019-01-15 | 桂林电子科技大学 | A kind of fine granularity model recognition system and method |
| CN110852168A (en) * | 2019-10-11 | 2020-02-28 | 西北大学 | Pedestrian re-recognition model construction method and device based on neural framework search |
| US20200082275A1 (en) * | 2018-09-10 | 2020-03-12 | Fujitsu Limited | Neural network architecture search apparatus and method and computer readable recording medium |
| CN110956260A (en) * | 2018-09-27 | 2020-04-03 | 瑞士电信公司 | System and method for neural architecture search |
| WO2020221200A1 (en) * | 2019-04-28 | 2020-11-05 | 华为技术有限公司 | Neural network construction method, image processing method and devices |
-
2021
- 2021-03-10 CN CN202110262167.5A patent/CN113159115B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109214441A (en) * | 2018-08-23 | 2019-01-15 | 桂林电子科技大学 | A kind of fine granularity model recognition system and method |
| US20200082275A1 (en) * | 2018-09-10 | 2020-03-12 | Fujitsu Limited | Neural network architecture search apparatus and method and computer readable recording medium |
| CN110956260A (en) * | 2018-09-27 | 2020-04-03 | 瑞士电信公司 | System and method for neural architecture search |
| WO2020221200A1 (en) * | 2019-04-28 | 2020-11-05 | 华为技术有限公司 | Neural network construction method, image processing method and devices |
| CN110852168A (en) * | 2019-10-11 | 2020-02-28 | 西北大学 | Pedestrian re-recognition model construction method and device based on neural framework search |
Non-Patent Citations (1)
| Title |
|---|
| 王耀玮;唐伦;刘云龙;陈前斌;: "基于多任务卷积神经网络的车辆多属性识别", 计算机工程与应用, no. 08 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114495027A (en) * | 2022-01-11 | 2022-05-13 | 北京科技大学 | Vehicle type fine-grained identification method and device based on network data |
| CN115424086A (en) * | 2022-07-26 | 2022-12-02 | 北京邮电大学 | Multi-view fine-granularity identification method and device, electronic equipment and medium |
| CN117036911A (en) * | 2023-10-10 | 2023-11-10 | 华侨大学 | Vehicle re-identification light-weight method and system based on neural architecture search |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113159115B (en) | 2023-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113159115B (en) | Vehicle fine granularity identification method, system and device based on neural architecture search | |
| Liu et al. | Progressive neural architecture search | |
| CN111723915B (en) | Target detection method based on deep convolutional neural network | |
| CN114037844A (en) | Global rank perception neural network model compression method based on filter characteristic diagram | |
| CN111967594A (en) | Neural network compression method, device, equipment and storage medium | |
| CN109034086B (en) | Vehicle weight identification method, device and system | |
| CN111931505A (en) | Cross-language entity alignment method based on subgraph embedding | |
| CN110442143B (en) | Unmanned aerial vehicle situation data clustering method based on combined multi-target pigeon swarm optimization | |
| CN114556248A (en) | Method for determining a sensor configuration | |
| CN112381227B (en) | Neural network generation method and device, electronic equipment and storage medium | |
| CN112308825A (en) | SqueezeNet-based crop leaf disease identification method | |
| CN116244647A (en) | Unmanned aerial vehicle cluster running state estimation method | |
| CN111352926A (en) | Data processing method, device, equipment and readable storage medium | |
| CN117636298A (en) | Vehicle re-identification method, system and storage medium based on multi-scale feature learning | |
| CN114556364A (en) | Neural architecture search based on similarity operator ordering | |
| CN115018884B (en) | Visible light infrared visual tracking method based on multi-strategy fusion tree | |
| Yamada et al. | Weight Features for Predicting Future Model Performance of Deep Neural Networks. | |
| CN113032612B (en) | Construction method of multi-target image retrieval model, retrieval method and device | |
| CN114742199A (en) | Attention mechanism-based neural network macro architecture searching method and system | |
| CN114202671A (en) | Image prediction optimization processing method and device | |
| CN114792397A (en) | SAR image urban road extraction method, system and storage medium | |
| CN114627076A (en) | Industrial detection method combining active learning and deep learning technologies | |
| CN114997366A (en) | Protein structure model quality evaluation method based on graph neural network | |
| Toubeh et al. | Risk-aware planning by confidence estimation using deep learning-based perception | |
| CN111626373A (en) | Multi-scale widening residual error network, small target identification detection network and optimization method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |