CN110378236B - Vehicle identity recognition model construction and recognition method and system based on deep learning - Google Patents
Vehicle identity recognition model construction and recognition method and system based on deep learning Download PDFInfo
- Publication number
- CN110378236B CN110378236B CN201910537407.0A CN201910537407A CN110378236B CN 110378236 B CN110378236 B CN 110378236B CN 201910537407 A CN201910537407 A CN 201910537407A CN 110378236 B CN110378236 B CN 110378236B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- network
- picture
- identity recognition
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of computer vision, and relates to a vehicle identity recognition model building and recognizing method and system based on deep learning. Firstly, the method utilizes large-scale road monitoring pictures to carry out model training of vehicle detection, and training adopts a multi-loss function staged combined training strategy. Then, the detected car face image is subjected to component extraction, and classification is carried out by utilizing feature extraction and a fusion network or a common classification network according to the car face component extraction condition. And finally, extracting and filtering the identity characteristic vector of the vehicle face by using a multitask network, and performing similarity measurement on the image characteristics to be analyzed and the characteristic vector of the image in the vehicle information base to obtain a vehicle identity recognition result. The deep learning network framework provided by the invention can improve the feature extraction capability of the network model in different aspects according to the requirements, thereby realizing the optimal model expression capability and facilitating the extraction of the identification feature vector with obvious discrimination.
Description
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a vehicle identity recognition model building and recognizing method and system based on deep learning.
Background
In recent years, the computer vision theory based on neural network and machine learning is redefined and qualitatively developed, the traditional image feature extraction scheme with limited manual design effect and insufficient robustness is broken through, high-dimensional features are extracted through multilayer convolution pooling to fit a large amount of training data, and a feature scheme most suitable for solving the problem is regressed. Therefore, a large number of application examples with high efficiency, strong applicability and easy adaptation emerge in the field of computer vision, license plate recognition is an example in which the development is mature and the application value is high, and the license plate recognition method is widely applied to violation fine at video monitoring places of traffic checkpoints, vehicle screening of garages of enterprises and the like. However, the license plate as an important feature of the appearance of the vehicle is easily altered, shielded, disassembled and forged by illegal molecules, so that the recognition of the vehicle cannot only depend on the recognition of the license plate.
At present, AI services for detecting vehicle types and vehicle colors appear, and feature extraction of other key shapes of vehicles can be used as important bases for vehicle identification. However, for vehicles of the same color and vehicle model, there is little distinguishing feature other than the license plate for a wide range of detection, screening and identification. The identity recognition of the individuals of the vehicles with the modified appearances can not be carried out, such as the screening and re-recognition of the vehicles with similar appearances in a mass target, the identity recognition of the unlicensed or fake-licensed vehicles and the like.
Disclosure of Invention
Aiming at the problems of insufficient fine granularity of vehicle feature extraction and inaccurate vehicle identity identification in the prior art, the invention provides a vehicle identity identification model construction and identification method and system based on deep learning, which are realized by adopting the following technical scheme:
the vehicle identity recognition model construction method based on deep learning comprises the following steps:
step 1: acquiring a plurality of scene pictures, and preprocessing the scene pictures to obtain a preprocessed picture set;
step 2: labeling the preprocessed picture set to obtain a label set, wherein labels of the label set comprise vehicle, vehicle faces, vehicle types and vehicle identity information;
and step 3: training the network by utilizing the preprocessed picture set and the label set;
the network comprises a vehicle detection network, a vehicle face component extraction network, a vehicle type classification network and a vehicle identity identification network;
the vehicle detection network is used for outputting a vehicle target picture;
the car face component extraction network is used for outputting a car face component picture by using a vehicle target picture;
the vehicle type classification network is used for outputting a vehicle classification result by utilizing a vehicle target picture and a vehicle face component picture;
the vehicle identity recognition network is used for obtaining the same type of vehicle face component pictures according to the vehicle classification result and then outputting vehicle identity characteristic vectors by using the same type of vehicle face component pictures;
and obtaining a vehicle identity recognition model.
Further, the vehicle detection network obtaining method includes:
and marking and preprocessing the vehicles in each picture of the preprocessed picture set to obtain a vehicle detection data set, and training a Mobilene-SSD network by using the vehicle detection data set to obtain a vehicle detection network.
Further, the method for obtaining the vehicle type classification network comprises the following steps:
step a: labeling the car face in the car target picture to obtain a car face component detection data set, training an FPN-SSD network by using the car face component detection data set to obtain a car face component extraction network, and outputting a car face component picture;
step b: marking vehicle types contained in the vehicle target picture and the vehicle face part picture to obtain a vehicle classification data set, establishing a multi-path convolutional neural network, and training the multi-path convolutional neural network by using the vehicle classification data set to obtain a vehicle identity recognition network;
the multipath convolutional neural network comprises five convolutional layers and two full-connection layers for a vehicle target picture, three convolutional layers for four car face component pictures and five independent convolutional layers for two full-connection networks.
Further, the vehicle identification network obtaining method comprises the following steps:
according to the vehicle classification result, labeling vehicle identity information in the vehicle face part pictures of the same vehicle type to obtain a vehicle front windshield image segmentation data set, and training a multitask convolution neural network Mask R-CNN by using the vehicle front windshield image segmentation data set to obtain a vehicle identity recognition network.
The vehicle identity recognition method based on deep learning comprises the following steps:
step 1: acquiring vehicle identity characteristic vectors output by a plurality of scene pictures, corresponding vehicle labels and vehicle pictures to establish a vehicle information base;
step 2: acquiring a picture to be recognized, and acquiring a vehicle target picture of the picture to be recognized by utilizing a vehicle detection network in a staged vehicle identity recognition model construction method based on deep learning;
and step 3: obtaining a car face part picture through a car face part extraction network in a staged car identity recognition model building method based on deep learning, and inputting the car face part picture of the picture to be recognized into a car type classification network in the staged car identity recognition model building method based on deep learning to obtain a car classification result of the picture to be recognized;
and 4, step 4: obtaining a vehicle identity characteristic vector of an image to be recognized through a vehicle identity recognition network in a staged vehicle identity recognition model construction method based on deep learning;
and 5: and (3) carrying out similarity matching on the vehicle identity characteristic vector of the image to be recognized and the vehicle information base obtained in the step (1) to finish vehicle identity recognition.
The vehicle identity recognition model building and recognition system based on deep learning comprises a vehicle face detection module, a vehicle type classification module and a vehicle identity recognition module;
the car face detection module is used for marking a car by using the preprocessed scene picture and outputting a car target picture;
the vehicle type classification module comprises a vehicle face component extraction sub-module and a vehicle type classification sub-module, and the vehicle face component extraction sub-module is used for labeling a vehicle face by using a vehicle target picture obtained by the vehicle face detection module and outputting a vehicle face component picture; the vehicle type classification submodule is used for labeling vehicle type information by using a vehicle target picture and a vehicle face component picture and outputting a vehicle classification result;
the vehicle identity recognition module is used for labeling vehicle identity information in the images of the vehicle face parts of the same vehicle type according to vehicle classification results, outputting vehicle identity characteristic vectors, obtaining images to be recognized, and completing vehicle identity recognition according to a vehicle identity recognition method based on deep learning.
Further, the car face detection module is used for marking and preprocessing a car in each picture of the preprocessed scene picture to obtain a car detection data set, and the car detection data set is used for training a mobilene-SSD network to obtain the car detection data set.
Further, the car face component extraction submodule labels a car face in a car target picture to obtain a car face component detection data set, and then the car face component detection data set is used for training the FPN-SSD network to obtain the car face component detection data set.
Further, the vehicle type classification sub-module obtains a vehicle classification data set by labeling vehicle target pictures and vehicle types contained in the vehicle face component pictures, establishes a multi-path convolutional neural network, and trains the multi-path convolutional neural network by using the vehicle classification data set, wherein the multi-path convolutional neural network comprises five convolutional layers and two full-connection layers for the vehicle target pictures and five independent convolutional layers for three convolutional layers and two full-connection networks for four vehicle face component pictures.
Further, in the vehicle identity recognition module, vehicle identity information in the vehicle face part pictures of the same vehicle type is labeled by using vehicle classification results to obtain a vehicle front windshield image segmentation data set, a multitask convolutional neural network Mask R-CNN is trained by using the vehicle front windshield image segmentation data set to obtain all vehicle identity feature vectors, a picture to be recognized is obtained, and vehicle identity recognition is completed according to a vehicle identity recognition method based on deep learning.
The invention also has the following beneficial effects:
(1) The invention realizes staged feature extraction and fusion, utilizes global and vehicle local features to the maximum extent, and compared with the existing method which only utilizes local features or global features to realize vehicle identification, the invention is a more complete extraction process and identification method, and can also independently apply the functions of different modules.
(2) The deep learning network framework provided by the invention can improve the feature extraction capability of the network model in different aspects according to the requirements, thereby realizing the optimal model expression capability and facilitating the extraction of the identification feature vector with remarkable discrimination. Such as car face components suitable for extracting multi-scale targets, and a vehicle type classification network that can achieve better utilization of extraction and fusion of global and local features of the car face. Meanwhile, the different-deep learning network frameworks used in different stages are matched with an accurate target, and the identification of the vehicle identity can be better realized.
(3) The invention provides vehicle identity recognition, which bypasses the traditional method for recognizing and confirming the vehicle identity by license plates, can screen out vehicles with different appearances by utilizing vehicle detection and vehicle type classification, and realizes the recognition of the unlicensed fake-licensed vehicles by fully utilizing vehicle annual inspection marks and vehicle interior ornaments to extract vehicle identity characteristics in the same subdivided vehicle type.
Drawings
FIG. 1 is a functional schematic diagram of the present invention applied to staged vehicle identification in a surveillance scenario;
FIG. 2 is a functional block diagram of the present invention applied to staged vehicle identification for surveillance scenarios;
FIG. 3 is a schematic diagram of the classification results of the vehicle classification processed according to the present invention;
FIG. 4 is a schematic diagram of a structure for extracting a vehicle identity feature vector according to the present invention;
FIG. 5 is a schematic flow chart of vehicle identification according to the present invention;
FIG. 6 is a schematic diagram of the recognition matching result of the present invention for processing the recognized vehicle image.
Detailed Description
The following embodiments of the present invention are provided, and it should be noted that the present invention is not limited to the following embodiments, and all equivalent changes based on the technical solutions of the present invention are within the protection scope of the present invention.
The following explains the terms appearing in this patent:
labeling: and aiming at the visual content of the image, adding text characteristic information reflecting the content of the image to the image by a machine learning method.
Pretreatment: histogram equalization and filtering noise reduction are performed.
Scene picture: the road pictures shot by the camera comprise road surface information, environment information and vehicle information.
Vehicle target picture: the picture contains the coordinates, length and width of the vehicle and score information, the score representing the probability of a detected vehicle target.
Vehicle face: a vehicle front face image acquired from a vehicle frontal angle.
Vehicle identity information: including interior trim, window front seat trim, vehicle annual inspection identification, and the like.
Car face component picture: the method comprises pictures of target components such as a front windshield window, a rearview mirror, a license plate, a car lamp, a car logo and the like of the car.
All kinds of target components: comprises a front windshield window of a vehicle, a rearview mirror, a license plate, a vehicle lamp and a vehicle logo.
Vehicle-required information: the method comprises a unique vehicle label, a vehicle picture and a feature vector corresponding to the vehicle.
Example 1
The embodiment discloses a vehicle identity recognition model construction method based on deep learning, which comprises the following steps:
step 1: acquiring a plurality of scene pictures, and preprocessing the scene pictures to obtain a preprocessed picture set;
step 2: labeling the preprocessed picture set to obtain a label set, wherein labels of the label set comprise vehicle, vehicle face, vehicle type and vehicle identity information;
and step 3: training the network by utilizing the preprocessed picture set and the label set;
the network comprises a vehicle detection network, a vehicle face component extraction network, a vehicle type classification network and a vehicle identity identification network;
the vehicle detection network is used for outputting a vehicle target picture;
the car face component extraction network is used for outputting a car face component picture by using a vehicle target picture;
the vehicle type classification network is used for outputting a vehicle classification result by utilizing a vehicle target picture and a vehicle face component picture;
the vehicle identity recognition network is used for obtaining the same type of vehicle face component pictures according to the vehicle classification result and then outputting vehicle identity characteristic vectors by using the same type of vehicle face component pictures;
and obtaining a vehicle identity recognition model.
The invention provides a vehicle identity recognition model construction method based on deep learning, the existing vehicle face recognition technology is provided to make up the defects of license plate recognition and vehicle type recognition in vehicle recognition, security checkpoints are often carried out on vehicles suspected of illegal crimes in the process of public security investigation, and the requirements of positioning, screening and tracking of road monitoring are met. The invention aims at the recognition of the vehicle face part, can obviously distinguish the appearances of different brands and different styles, can best reflect the extracted characteristics of the vehicle face part of design elements, and becomes an important help for recognizing the vehicle appearance and the vehicle type. The attribute information of the vehicle is concerned, and the attribute information is applied to the vehicle identification analysis process in stages, so that the vehicle detection, the vehicle type classification and the extraction of the vehicle identification characteristics are independently realized, and the coarse-to-fine vehicle identification model is constructed by combining the attribute information, the vehicle type classification and the vehicle identification characteristics.
Specifically, the method for obtaining the vehicle detection network includes the following substeps:
step 1a: marking the position, the size and the category of a vehicle target on each picture of the preprocessed picture set to manufacture a vehicle detection data set, and dividing the vehicle detection data set into a training set, a testing set and a verification set according to the proportion of 8;
step 1b: training a Mobilene-SSD network by using the vehicle detection data set obtained in the step 1a to obtain a vehicle detection network;
step 1c: and inputting the preprocessed image into a vehicle detection network, and outputting a vehicle target picture.
Specifically, the method for acquiring the vehicle type classification network comprises the following substeps:
step 2a: performing car face labeling on a car target picture obtained by a car detection network, wherein the labeling is to reserve the car target picture which comprises the most comprehensive car face and the least irrelevant car face part to obtain a car face part detection data set;
and step 2b: replacing a VGG-16 network in an SSD network with an FPN-SSD network model to obtain an FPN-SSD network model, training the FPN-SSD network by using a vehicle face component detection data set to obtain a vehicle face component extraction network, and outputting a vehicle face component picture;
and step 2c: and carrying out vehicle type labeling on the vehicle target picture and the vehicle face part picture to obtain a vehicle type classification data set, and dividing the vehicle type classification data set into a training set, a testing set and a verification set according to the proportion of 8.
Step 2d: establishing a multi-path convolutional neural network structure, training by utilizing the vehicle classification data set obtained in the step 2, realizing independent convolution of the input vehicle face picture and the vehicle face part picture and then fusing the characteristics to obtain a vehicle type classification network with the vehicle full-face characteristics fused, and outputting a vehicle classification result;
specifically, in step 2d, if the vehicle component picture output in step 2b does not include all kinds of target components, the vehicle classification data set is trained to the existing ResNet classification network to obtain the vehicle classification result.
Preferably, the multipath convolutional neural network comprises five convolutional layers and two fully-connected layers for the vehicle target picture, three convolutional layers for four car face component pictures and five independent convolutional layers of a two-layer fully-connected network.
Specifically, the obtaining method of the vehicle identification network comprises the following steps:
according to the vehicle classification result, vehicle identity information in the vehicle face part pictures of the same vehicle type is labeled to obtain a vehicle front windshield image segmentation data set, and a multitask convolutional neural network Mask R-CNN is trained by the vehicle front windshield image segmentation data set to obtain a vehicle identity recognition network.
And filtering the full-image feature map and the ROI feature map by extracting the feature map and using a mask obtained by masking branches, and splicing after regularized whitening treatment to obtain an output vehicle identity feature vector.
In order to increase the weight of important parts in the car face, the local features and the overall features are combined, the full-face features of the car are extracted as much as possible, each part is cut and then is convolved with a complete car face picture, and a car type identification network integrating the car face features is designed, as shown in fig. 4. Parameters are not shared between convolution of the part picture and the part picture, and parameters are not shared between the part picture and the full-face picture, because information is lost due to the fact that convolution kernels are shared by different car face regions. And finally, directly splicing the convolution result of the local area where the part is located and the convolution result of the vehicle full-face picture, inputting two full-connection layers of 4096 neurons and one full-connection layer of 228 neurons for fusion, and finally classifying the 228-dimensional result through a softmax algorithm to obtain the type of the input vehicle face.
Example 2
The embodiment discloses a vehicle identity recognition method based on deep learning, which comprises the following steps:
step 1: collecting vehicle identity characteristic vectors output by a plurality of scene pictures to establish a vehicle information base;
and 2, step: obtaining a picture to be recognized, and obtaining a vehicle target picture of the picture to be recognized by using a vehicle detection network in any one of the deep learning-based staged vehicle identity recognition model construction methods in the embodiment 1;
and step 3: obtaining a car face part picture through a car face part extraction network in a staged car identity recognition model building method based on deep learning, and inputting the car face part picture of the picture to be recognized into a car type classification network in the staged car identity recognition model building method based on deep learning to obtain a car classification result of the picture to be recognized; as shown in fig. 3, the picture classification result is KIA kx5.
And 4, step 4: obtaining a vehicle identity characteristic vector of an image to be recognized through a vehicle identity recognition network in a staged vehicle identity recognition model construction method based on deep learning;
and 5: and (3) carrying out similarity matching on the vehicle identity characteristic vector of the image to be recognized and the vehicle information base obtained in the step (1) to finish vehicle identity recognition.
Fig. 1 is a functional schematic diagram of a staged vehicle identification applied to a monitoring scene, the staged identification process conforms to the normal cognitive process of human beings, the targets of different stages are realized by making full use of various structures of a convolutional neural network in a targeted manner, and the method is suitable for realizing the vehicle identification at the present stage.
The method comprises the steps of firstly detecting the vehicle position of an input picture, then classifying the vehicle types of the vehicle face picture to obtain the specific models of the input picture, realizing screening and filtering, finally realizing vehicle identification by utilizing a front gear glass window area of a vehicle, and matching the vehicle identification with the vehicle information registered in a database.
The invention provides a vehicle identity recognition method based on deep learning, which designs a method for classifying by extracting and fusing global and local characteristics of a vehicle face in vehicle type recognition, designs a vehicle face part extraction network suitable for extracting multi-scale targets and a vehicle type classification network capable of fully utilizing vehicle face characteristics; the characteristics containing vehicle identity information are extracted in vehicle identity recognition, a more accurate vehicle identity characteristic vector is obtained by filtering through a multi-task network, and meanwhile matching of vehicle identities can be progressively retrieved by splicing the characteristics, so that the vehicle identity recognition efficiency is improved.
Specifically, if the matching is successful, returning the corresponding vehicle information in the vehicle information base, and if the matching is failed, returning that the vehicle is not in the vehicle information base; when matching is performed, a certain number of car faces need to be tested, so that each car face can obtain a vector distance from all car faces in the information base, and a threshold value is selected, which enables a value with the largest sens [ sens = TP/P ] or a value with the largest acc [ acc = (TP + TN)/total test sample number ], as shown in fig. 5, in this embodiment, the threshold value is 0.82. Wherein, TP (True positive) represents True positive example, namely correctly predicting the positive class as the number of the positive class; TN (True negative) indicates True negative, i.e. correctly predicts a negative class as a negative class number; acc (accuracy) represents the accuracy and is the most common machine learning evaluation index, and generally speaking, the higher the accuracy, the better the classifier; sens (positive) represents sensitivity, represents the proportion of all positive examples which are paired, and measures the recognition capability of the classifier on the positive examples.
Example 3
The embodiment discloses a vehicle identity recognition model building and recognition system based on deep learning, which comprises a vehicle face detection module, a vehicle type classification module and a vehicle identity recognition module, as shown in fig. 2;
the car face detection module is used for marking a car by using the preprocessed scene picture and outputting a car target picture; the module is one of three modules of the system and is the basis of the functions of the subsequent modules. The method mainly realizes the detection of the vehicle in the input large graph and the cutting and storing work of the vehicle, wherein the pre-starting module is responsible for reading in the picture and loading the model and is a necessary part in each functional module.
The vehicle type classification module comprises a vehicle face component extraction sub-module and a vehicle type classification sub-module, and the vehicle face component extraction sub-module is used for labeling a vehicle face by using a vehicle target picture obtained by the vehicle face detection module and outputting a vehicle face component picture; the vehicle type classification submodule is used for labeling vehicle type information by using a vehicle target picture and a vehicle face component picture and outputting a vehicle classification result; the car face part is stored for application in vehicle type classification and is also an essential material for providing a picture of a front windshield of a vehicle as vehicle identification.
The vehicle identity recognition module is used for labeling the vehicle identity information in the images of the vehicle face parts of the same vehicle type according to the vehicle classification result, outputting a vehicle identity characteristic vector, obtaining the image to be recognized, and completing vehicle identity recognition according to any one of the vehicle identity recognition methods based on deep learning in the embodiment 2. The main sub-module of the module comprises two modules of vehicle information registration and individual identification. The vehicle information registration module realizes persistence of necessary information corresponding to each vehicle, wherein the necessary information comprises a unique vehicle mark number, a vehicle picture and a feature vector corresponding to the vehicle. And the individual recognition is responsible for calling the deep learning platform to obtain the feature vector of the detection picture by using the trained network model, and the feature vector is compared with the registered vehicle features in the database to return the similarity and the corresponding vehicle number.
Specifically, the car face detection module is used for obtaining a car detection data set by labeling and preprocessing a car in each picture of the preprocessed scene pictures, and obtaining the car detection data set by training a mobilene-SSD network by using the car detection data set.
Specifically, the car face component extraction submodule labels a car face in a car target picture to obtain a car face component detection data set, and then the car face component detection data set is used for training the FPN-SSD network to obtain the car face component detection data set.
Specifically, the vehicle type classification sub-module obtains a vehicle classification data set by labeling vehicle target pictures and vehicle types contained in the vehicle face component pictures, establishes a multi-path convolutional neural network, and trains the multi-path convolutional neural network by using the vehicle classification data set, wherein the multi-path convolutional neural network comprises five convolutional layers and two full-connection layers for the vehicle target pictures and five independent convolutional layers of three convolutional layers and two full-connection networks for four vehicle face component pictures.
Specifically, in the vehicle identity recognition module, the vehicle identity information in the vehicle face part pictures of the same vehicle type is marked by using the vehicle classification result, a vehicle front windshield image segmentation data set is obtained, a multitask convolutional neural network Mask R-CNN is trained by using the vehicle front windshield image segmentation data set, all vehicle identity feature vectors are obtained, the picture to be recognized is obtained, and vehicle identity recognition is completed according to a vehicle identity recognition method based on deep learning.
As shown in fig. 6, in the embodiment, a schematic diagram of the recognition and matching result of the recognized vehicle image is processed, and from the recognition result, the vehicle identification can be correctly matched with the identity information of the corresponding vehicle.
Claims (8)
1. The method for constructing the vehicle identity recognition model based on deep learning is characterized by comprising the following steps of:
step 1: acquiring a plurality of scene pictures, and preprocessing the scene pictures to obtain a preprocessed picture set;
step 2: labeling the preprocessed picture set to obtain a label set, wherein labels of the label set comprise vehicle, vehicle faces, vehicle types and vehicle identity information;
and step 3: training the network by utilizing the preprocessed picture set and the label set;
the network comprises a vehicle detection network, a vehicle face component extraction network, a vehicle type classification network and a vehicle identity identification network;
the vehicle detection network is used for outputting a vehicle target picture;
the car face component extraction network is used for outputting a car face component picture by using a vehicle target picture;
the vehicle type classification network is used for outputting a vehicle classification result by utilizing a vehicle target picture and a vehicle face component picture;
the vehicle identity recognition network is used for obtaining the same type of vehicle face component pictures according to the vehicle classification result and then outputting vehicle identity characteristic vectors by using the same type of vehicle face component pictures;
obtaining a vehicle identity recognition model;
the method for obtaining the vehicle type classification network comprises the following steps:
a, step a: labeling the car face in the car target picture to obtain a car face component detection data set, training a FPN-SSD network by using the car face component detection data set to obtain a car face component extraction network, and outputting a car face component picture;
step b: marking vehicle types contained in the vehicle target picture and the vehicle face part picture to obtain a vehicle classification data set, establishing a multi-path convolutional neural network, and training the multi-path convolutional neural network by using the vehicle classification data set to obtain a vehicle identity recognition network;
the multipath convolutional neural network comprises five convolutional layers and two full-connection layers for vehicle target pictures, three convolutional layers for four car face component pictures and five independent convolutional layers of two full-connection networks.
2. The method for constructing the vehicle identification model based on the deep learning of claim 1, wherein the vehicle detection network is obtained by the following steps:
and marking and preprocessing the vehicles in each picture of the preprocessed picture set to obtain a vehicle detection data set, and training a Mobilene-SSD network by using the vehicle detection data set to obtain a vehicle detection network.
3. The method for constructing the vehicle identification model based on the deep learning of claim 1, wherein the vehicle identification network is obtained by the following steps:
according to the vehicle classification result, vehicle identity information in the vehicle face part pictures of the same vehicle type is labeled to obtain a vehicle front windshield image segmentation data set, and a multitask convolutional neural network Mask R-CNN is trained by the vehicle front windshield image segmentation data set to obtain a vehicle identity recognition network.
4. The vehicle identity recognition method based on deep learning is characterized by comprising the following steps:
step 1: acquiring vehicle identity characteristic vectors output by a plurality of scene pictures, corresponding vehicle labels and vehicle pictures to establish a vehicle information base;
step 2: acquiring a picture to be recognized, and acquiring a vehicle target picture of the picture to be recognized by utilizing a vehicle detection network in the deep learning-based vehicle identity recognition model construction method of any one of claims 1 to 3;
and step 3: obtaining a car face part picture through a car face part extraction network in any one of claims 1 to 3 based on the deep learning vehicle identity recognition model construction method, inputting the car face part picture of the picture to be recognized into a vehicle type classification network in any one of claims 1 to 3 based on the deep learning vehicle identity recognition model construction method to obtain a vehicle classification result of the picture to be recognized;
and 4, step 4: obtaining a vehicle identity feature vector of an image to be recognized through a vehicle identity recognition network in the vehicle identity recognition model construction method based on deep learning according to any one of claims 1 to 3;
and 5: and (3) carrying out similarity matching on the vehicle identity characteristic vector of the image to be recognized and the vehicle information base obtained in the step (1) to finish vehicle identity recognition.
5. The vehicle identity recognition model building and recognition system based on deep learning is characterized by comprising a vehicle face detection module, a vehicle type classification module and a vehicle identity recognition module;
the car face detection module is used for marking a car by using the preprocessed scene picture and outputting a car target picture;
the vehicle type classification module comprises a vehicle face component extraction sub-module and a vehicle type classification sub-module, and the vehicle face component extraction sub-module is used for labeling a vehicle face by using a vehicle target picture obtained by the vehicle face detection module and outputting a vehicle face component picture; the vehicle type classification submodule is used for labeling vehicle type information by using a vehicle target picture and a vehicle face component picture and outputting a vehicle classification result; the vehicle type classification submodule is used for obtaining a vehicle classification data set by marking vehicle target pictures and vehicle types contained in the vehicle face component pictures, establishing a multipath convolutional neural network, and training the multipath convolutional neural network by using the vehicle classification data set to obtain the vehicle classification data set, wherein the multipath convolutional neural network comprises five convolution layers and two full-connection layers aiming at the vehicle target pictures and five independent convolution layers aiming at three convolution layers and two full-connection layers of four vehicle face component pictures;
the vehicle identity recognition module is used for labeling the vehicle identity information in the images of the vehicle face parts of the same vehicle type according to the vehicle classification result, outputting a vehicle identity characteristic vector, acquiring the image to be recognized, and completing vehicle identity recognition according to the vehicle identity recognition method based on deep learning as claimed in claim 4.
6. The deep learning-based vehicle identification model building and recognizing system according to claim 5, wherein the vehicle face detection module is obtained by labeling and preprocessing vehicles in each of the preprocessed scene pictures to obtain a vehicle detection data set, and training a mobilene-SSD network by using the vehicle detection data set.
7. The deep learning-based vehicle identification model building and recognizing system according to claim 5, wherein the vehicle face component extracting sub-module is obtained by labeling a vehicle face in a vehicle target picture to obtain a vehicle face component detection data set, and then training an FPN-SSD network by using the vehicle face component detection data set.
8. The vehicle identity recognition model building and recognition system based on deep learning of claim 5, wherein in the vehicle identity recognition module, the vehicle identity information in the images of the vehicle face parts of the same vehicle type is labeled by using vehicle classification results to obtain a vehicle front windshield image segmentation data set, a multitask convolutional neural network Mask R-CNN is trained by using the vehicle front windshield image segmentation data set to obtain all vehicle identity feature vectors, the images to be recognized are obtained, and vehicle identity recognition is completed according to the vehicle identity recognition method based on deep learning of claim 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910537407.0A CN110378236B (en) | 2019-06-20 | 2019-06-20 | Vehicle identity recognition model construction and recognition method and system based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910537407.0A CN110378236B (en) | 2019-06-20 | 2019-06-20 | Vehicle identity recognition model construction and recognition method and system based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110378236A CN110378236A (en) | 2019-10-25 |
| CN110378236B true CN110378236B (en) | 2023-04-07 |
Family
ID=68249055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910537407.0A Active CN110378236B (en) | 2019-06-20 | 2019-06-20 | Vehicle identity recognition model construction and recognition method and system based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378236B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111079543B (en) * | 2019-11-20 | 2022-02-15 | 浙江工业大学 | Efficient vehicle color identification method based on deep learning |
| CN110991506B (en) * | 2019-11-22 | 2023-05-23 | 高新兴科技集团股份有限公司 | Vehicle brand identification method, device, equipment and storage medium |
| CN111126224A (en) * | 2019-12-17 | 2020-05-08 | 成都通甲优博科技有限责任公司 | Vehicle detection method and classification recognition model training method |
| CN111126393A (en) * | 2019-12-22 | 2020-05-08 | 上海眼控科技股份有限公司 | Vehicle appearance refitting judgment method and device, computer equipment and storage medium |
| CN111144372A (en) * | 2019-12-31 | 2020-05-12 | 上海眼控科技股份有限公司 | Vehicle detection method, device, computer equipment and storage medium |
| CN111339834B (en) * | 2020-02-04 | 2023-06-02 | 浙江大华技术股份有限公司 | Method for identifying vehicle driving direction, computer device and storage medium |
| CN111310844B (en) * | 2020-02-26 | 2022-09-16 | 广州华工邦元信息技术有限公司 | Vehicle identification model construction method and device and identification method and device |
| CN111340796B (en) * | 2020-03-10 | 2023-07-21 | 创新奇智(成都)科技有限公司 | Defect detection method and device, electronic equipment and storage medium |
| CN112087420B (en) * | 2020-07-24 | 2022-06-14 | 西安电子科技大学 | Network killing chain detection method, prediction method and system |
| CN112016433A (en) * | 2020-08-24 | 2020-12-01 | 高新兴科技集团股份有限公司 | Vehicle color identification method based on deep neural network |
| CN112115800A (en) * | 2020-08-25 | 2020-12-22 | 南京航空航天大学 | Vehicle combination recognition system and method based on deep learning target detection |
| CN112257869A (en) * | 2020-09-29 | 2021-01-22 | 北京北大千方科技有限公司 | Fake-licensed car analysis method and system based on random forest and computer medium |
| CN112257625B (en) * | 2020-10-29 | 2022-12-16 | 上海工程技术大学 | Vehicle weight recognition method based on vehicle front face features |
| CN112528921A (en) * | 2020-12-21 | 2021-03-19 | 山东雨润环保机械设备有限公司 | Construction site dust identification system and method based on machine vision |
| CN113239836A (en) * | 2021-05-20 | 2021-08-10 | 广州广电运通金融电子股份有限公司 | Vehicle body color identification method, storage medium and terminal |
| CN113298021A (en) * | 2021-06-11 | 2021-08-24 | 宿州学院 | Mining area transport vehicle head and tail identification method and system based on convolutional neural network |
| CN113780130A (en) * | 2021-08-31 | 2021-12-10 | 东南大学 | Vehicle type identification method based on magnetic field data |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107729818A (en) * | 2017-09-21 | 2018-02-23 | 北京航空航天大学 | A kind of multiple features fusion vehicle recognition methods again based on deep learning |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9710729B2 (en) * | 2014-09-04 | 2017-07-18 | Xerox Corporation | Domain adaptation for image classification with class priors |
| CN108319952B (en) * | 2017-01-16 | 2021-02-02 | 浙江宇视科技有限公司 | Vehicle feature extraction method and device |
| CN107092876A (en) * | 2017-04-12 | 2017-08-25 | 湖南源信光电科技股份有限公司 | The low-light (level) model recognizing method combined based on Retinex with S SIFT features |
| CN108460328A (en) * | 2018-01-15 | 2018-08-28 | 浙江工业大学 | A kind of fake-licensed car detection method based on multitask convolutional neural networks |
| CN109033175A (en) * | 2018-06-25 | 2018-12-18 | 高新兴科技集团股份有限公司 | A kind of method and system to scheme to search vehicle |
-
2019
- 2019-06-20 CN CN201910537407.0A patent/CN110378236B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107729818A (en) * | 2017-09-21 | 2018-02-23 | 北京航空航天大学 | A kind of multiple features fusion vehicle recognition methods again based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110378236A (en) | 2019-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110378236B (en) | Vehicle identity recognition model construction and recognition method and system based on deep learning | |
| US11887064B2 (en) | Deep learning-based system and method for automatically determining degree of damage to each area of vehicle | |
| CN108596277B (en) | Vehicle identity recognition method and device and storage medium | |
| CN109711370B (en) | Data fusion method based on WIFI detection and face clustering | |
| CN104166841B (en) | The quick detection recognition methods of pedestrian or vehicle is specified in a kind of video surveillance network | |
| TWI384408B (en) | Method and system for identifying image and outputting identification result | |
| CN101142584B (en) | Method for facial features detection | |
| CN111126224A (en) | Vehicle detection method and classification recognition model training method | |
| CN111860291A (en) | Multi-mode pedestrian identity recognition method and system based on pedestrian appearance and gait information | |
| CN110222596B (en) | Driver behavior analysis anti-cheating method based on vision | |
| CN106650731A (en) | Robust license plate and logo recognition method | |
| CN109214336A (en) | A kind of vehicle window marker detection method and device | |
| CN109033175A (en) | A kind of method and system to scheme to search vehicle | |
| KR102105954B1 (en) | System and method for accident risk detection | |
| CN110826484A (en) | Vehicle weight recognition method and device, computer equipment and model training method | |
| CN106934374A (en) | The recognition methods of traffic signboard and system in a kind of haze scene | |
| CN109657599B (en) | Picture identification method of distance-adaptive vehicle appearance part | |
| CN108846269A (en) | One kind is towards manifold identity identifying method and identification authentication system | |
| CN111814751A (en) | Vehicle attribute analysis method and system based on deep learning target detection and image recognition | |
| CN115375991A (en) | Strong/weak illumination and fog environment self-adaptive target detection method | |
| CN111444843A (en) | Multimode driver and vehicle illegal behavior monitoring method and system | |
| CN115984830A (en) | Safety belt wearing detection method, device, equipment and storage medium | |
| Santos et al. | Car recognition based on back lights and rear view features | |
| Xu et al. | A machine learning approach to vehicle occupancy detection | |
| Amin et al. | An automatic number plate recognition of Bangladeshi vehicles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |