CN110895692B - Vehicle brand identification method and device and readable storage medium - Google Patents
Vehicle brand identification method and device and readable storage medium Download PDFInfo
- Publication number
- CN110895692B CN110895692B CN201811070622.6A CN201811070622A CN110895692B CN 110895692 B CN110895692 B CN 110895692B CN 201811070622 A CN201811070622 A CN 201811070622A CN 110895692 B CN110895692 B CN 110895692B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- brand
- feature
- region
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image processing, and provides a vehicle brand identification method, a device and a readable storage medium, wherein the method comprises the following steps: firstly, obtaining a vehicle picture, inputting the vehicle picture into a preset convolutional neural network, and extracting features by using a first network to obtain a first feature map; then, carrying out area detection on the first characteristic diagram to obtain a first vehicle characteristic area, and carrying out secondary characteristic extraction on the first characteristic diagram by utilizing a second network to obtain a second characteristic diagram; and finally, extracting a plurality of vehicle brand features in the second feature map according to the first vehicle feature area and identifying the vehicle brand in the vehicle image according to the plurality of vehicle brand features. Compared with the prior art, the invention improves the vehicle brand identification efficiency.
Description
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to a vehicle brand identification method and device and a readable storage medium.
Background
With the development of science and technology, intelligent traffic systems are rapidly developed, and more vehicles cause problems of road congestion, frequent accidents and the like, so that the traffic systems need to be managed more strictly. Vehicle brand identification is an important item for extracting the whole vehicle structural information, and in order to realize stricter traffic management, the vehicle brand identification is required.
In the prior art, brand recognition is performed according to accurate position information of a license plate. When the vehicle has no license plate, can not position the license plate, and has inaccurate license plate positioning, the vehicle brand recognition efficiency is low.
Disclosure of Invention
The embodiment of the invention aims to provide a vehicle brand identification method, a vehicle brand identification device and a readable storage medium, so as to solve the problem of low vehicle brand identification efficiency.
In order to achieve the above object, the embodiments of the present invention adopt the following technical solutions:
in a first aspect, an embodiment of the present invention provides a vehicle brand identification method, which is applied to an electronic device, and the method includes: acquiring a vehicle picture, inputting the vehicle picture into a preset convolutional neural network, and extracting features by using a first network of the convolutional neural network to obtain a first feature map; carrying out area detection on the first characteristic diagram to obtain a first vehicle characteristic area in the first characteristic diagram; performing secondary feature extraction on the first feature map by using a second network of the convolutional neural network to obtain a second feature map; extracting a plurality of vehicle brand features in the second feature map according to the first vehicle feature area, wherein the second feature map comprises a plurality of pieces of vehicle brand feature information; identifying a vehicle brand in the vehicle picture according to the plurality of vehicle brand features.
In a second aspect, an embodiment of the present invention provides a vehicle brand identification apparatus, which is applied to an electronic device. The first feature extraction module is used for acquiring a vehicle picture, inputting the vehicle picture into a preset convolutional neural network, and extracting features by using a first network of the convolutional neural network to obtain a first feature map; the area detection module is used for carrying out area detection on the first characteristic diagram to obtain a first vehicle characteristic area in the first characteristic diagram; the second feature extraction module is used for performing secondary feature extraction on the first feature map by using a second network of the convolutional neural network to obtain a second feature map; the execution module is used for extracting a plurality of vehicle brand features in the second feature map according to the first vehicle feature area, wherein the second feature map comprises a plurality of pieces of vehicle brand feature information; the brand identification module is used for identifying the vehicle brand in the vehicle picture according to the plurality of vehicle brand features. In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the above-mentioned vehicle brand identification method.
Compared with the prior art, the vehicle brand identification method, the vehicle brand identification device and the readable storage medium provided by the embodiments of the present invention acquire a vehicle picture, input the vehicle picture into a first network of a convolutional neural network to obtain a first feature map, perform region detection on the first feature map to obtain a first vehicle feature region, extract a plurality of vehicle brand features in the second feature map according to the first vehicle feature region, and identify a vehicle brand in the vehicle picture according to the plurality of vehicle brand features. Compared with the prior art, the vehicle brand identification method provided by the embodiment of the invention can be used for carrying out area detection on the first characteristic diagram to obtain the first vehicle characteristic area, and identifying the vehicle brand not only by depending on the accurate position information of the license plate, so that the vehicle brand identification efficiency is improved.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 shows a block schematic diagram of an electronic device provided by an embodiment of the present invention.
FIG. 2 is a flow chart of a brand identification method for a vehicle according to an embodiment of the present invention.
Fig. 3 is a flowchart illustrating sub-steps of step S104 shown in fig. 2.
Fig. 4 is a flowchart illustrating sub-steps of step S105 shown in fig. 2.
FIG. 5 is a block diagram of a brand identification device for a vehicle according to an embodiment of the present invention.
Icon: 100-an electronic device; 101-a memory; 102-a memory controller; 103-a processor; 104-peripheral interfaces; 105-a communication module; 200-a vehicle brand identification device; 201-a first feature extraction module; 202-area detection module; 203-a second feature extraction module; 204-an execution module; 205-brand identification module; 300-a camera; 400-display screen.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Referring to fig. 1, fig. 1 is a block diagram illustrating an electronic device 100 according to an embodiment of the invention. The electronic device 100 may be, but is not limited to, a laptop portable computer, a vehicle-mounted computer, a Personal Digital Assistant (PDA), a server, and the like. The electronic device 100 includes a vehicle brand identification apparatus 200, a memory 101, a memory controller 102, a processor 103, a peripheral interface 104, and a communication module 105.
The memory 101, the memory controller 102, the processor 103, the peripheral interface 104, the communication module 105, the camera 300 and the display screen 400 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The vehicle brand identification apparatus 200 includes at least one software function module that may be stored in the memory 101 in the form of software or firmware (firmware) or solidified in an Operating System (OS) of the electronic device 100. Processor 103 is configured to execute executable modules stored in memory 101, such as software functional modules or computer programs included in vehicle brand identification device 200.
The Memory 101 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 101 is used for storing a program, and the processor 103 executes the program after receiving an execution instruction, and the method executed by the server defined by the process disclosed in any embodiment of the present invention may be applied to the processor 103, or implemented by the processor 103.
The processor 103 may be an integrated circuit chip having signal processing capabilities. The Processor 103 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), a voice Processor, a video Processor, and so on; but also be a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor 103 may be any conventional processor or the like.
The peripheral interface 104 is used to couple various input/output devices to the processor 103 as well as to the memory 101. In some embodiments, the peripheral interface 104, the processor 103, and the memory controller 102 may be implemented in a single chip. In other examples, they may be implemented separately from the individual chips.
The communication module 105 is used for receiving the vehicle picture shot by the camera 300 and sending the vehicle picture to the processor 103. The communication module 105 may be, but is not limited to, a DSP (Digital Signal Processing) chip and a semiconductor chip.
The display screen 400 is used for realizing interaction between a user and the electronic device 100, and may be, but is not limited to, displaying a brand of a vehicle or a picture of the vehicle, which needs to be photographed by the display screen 400.
First embodiment
Referring to fig. 2, fig. 2 is a flowchart illustrating a method for identifying a brand of a vehicle according to an embodiment of the present invention. The vehicle brand identification method comprises the following steps:
and S101, obtaining a vehicle picture, inputting the vehicle picture into a preset convolutional neural network, and extracting features by using a first network of the convolutional neural network to obtain a first feature map.
In the embodiment of the present invention, the vehicle picture may be a picture including a front face of a vehicle, and the vehicle picture may include regions such as a left headlight, a right headlight, a vehicle logo, a left rearview mirror, a right rearview mirror, a left fog light, a right fog light, and a grille. The vehicle picture can be obtained by real-time shooting through the camera 300, or can be obtained by downloading from the network in advance. The convolutional neural network is used for feature extraction and vehicle brand identification and comprises a first network, a second network and a third network, a vehicle picture is input into the first network to obtain a first feature map, feature extraction is carried out on the first feature map to obtain a first vehicle feature area, the first feature map passes through the second network to obtain a second feature map, a plurality of vehicle brand features in the second feature map are extracted according to the first vehicle feature area, and the plurality of vehicle brand features are identified through the third network. The first network comprises a plurality of first convolution layers and at least one first pooling layer, and the total number of layers of the first convolution layers is greater than the total number of layers of the at least one first pooling layer, which can be understood as that the first network comprises at least one first convolution pooling group, each first convolution pooling group comprises at least one first convolution layer and one first pooling layer, the first pooling layer of each first convolution pooling group is connected with the first convolution layer of the next first convolution pooling group, when only one first convolution pooling group exists in the first network, the first convolution pooling group comprises a plurality of first convolution layers and one first pooling layer, and the total number of layers of the first convolution layers in the first network is greater than the total number of layers of the first pooling layers. The first characteristic map may be a characteristic map obtained by passing the vehicle picture through the plurality of first convolution layers and the at least one first pooling layer. The method for inputting the vehicle picture into the preset convolutional neural network and extracting the features by using the first network of the convolutional neural network to obtain the first feature map may include: firstly, inputting a vehicle picture into a first network; and then, carrying out convolution operation and at least one pooling process on the vehicle picture for multiple times, wherein the output of the first pooling layer in each first convolution pooling group is the input of the first convolution layer in the next first convolution pooling group, and the output of the last first pooling layer is the first characteristic diagram.
And S102, carrying out area detection on the first characteristic diagram to obtain a first vehicle characteristic area in the first characteristic diagram. In the embodiment of the present invention, the first vehicle feature region may be a plurality of regions obtained by region detection of the first feature map, and may be obtained by performing region detection on the first feature map by using a target detection algorithm based on SSD (Single Shot multi box Detector). The first vehicle characteristic region comprises 9 regions such as a front face region, a left headlamp region, a right headlamp region, a logo region, a left rearview mirror region, a right rearview mirror region, a left fog lamp region, a right fog lamp region and a grille region. Each area is a rectangle, and can be represented by two diagonal coordinates, or can be represented by the length in the horizontal direction, the length in the vertical direction and one coordinate. For example, the vehicle front face region may be represented by coordinates (0,0) and coordinates (10,8), that is, the vehicle front face region is a rectangular region having a length of 10 units and a width of 8 units in the first feature map, or may be represented by coordinates (0,0) and a length in the horizontal direction of 10 and a length in the vertical direction of 8.
In other embodiments of the present invention, a target detection algorithm based on YOLO (young Only Look Once) may be further collected to perform area detection on the first feature map to obtain the first vehicle feature area.
After the first vehicle characteristic region is obtained, counting the number of regions in the first vehicle characteristic region, when the number of regions is smaller than a first preset number (for example, 4), repeatedly executing the steps S101-S102 until the number of regions is larger than the first preset number, and when the number of regions is larger than a second preset number (for example, 3), continuing the following steps. After step S105 is executed, comparing the number of the regions with a first preset number and a second preset number, when the number of the regions is smaller than the first preset number (e.g., 4), repeating steps S101-S105 until the number of the regions is larger than the first preset number, and when the number of the regions is larger than the second preset number (e.g., 3), the electronic device 100 controls the display screen 400 to display the vehicle brand of the vehicle picture.
And step S103, performing secondary feature extraction on the first feature map by using a second network of the convolutional neural network to obtain a second feature map.
In an embodiment of the present invention, the second network includes a plurality of second convolutional layers and at least one second pooling layer, and the total number of layers of the plurality of second convolutional layers is greater than the total number of layers of the at least one second pooling layer, it is understood that the second network includes a plurality of second convolutional pooling groups, each of the second convolutional pooling groups includes at least one second convolutional layer and one second pooling layer, the second pooling layer of each second convolutional pooling group is connected to the second convolutional layer of the next second convolutional pooling group, and the total number of layers of the second convolutional layers in the second network is greater than the total number of layers of the second pooling layers. The second profile may be a profile of the first profile obtained by passing through a plurality of second convolutional layers and at least one second pooling layer. The method for performing secondary feature extraction on the first feature map by using the second network in the convolutional neural network can comprise the following steps: firstly, inputting the first feature map into a second network, then carrying out convolution and pooling processing on the first feature map for multiple times, wherein the output of a second pooling layer in each second convolution pooling group is the input of a first second convolution layer in the next second convolution pooling group, and the output of the last second pooling layer is the second feature map.
It should be noted that, in other embodiments of the present invention, the execution order of step S103 and step S102 may be exchanged, or step S102 and step S103 may be executed simultaneously.
And step S104, extracting a plurality of vehicle brand characteristics in a second characteristic diagram according to the first vehicle characteristic area, wherein the second characteristic diagram comprises a plurality of vehicle brand characteristic information.
In an embodiment of the present invention, the vehicle brand feature may be an area in the second feature map that contains a plurality of pieces of vehicle brand feature information. The plurality of vehicle brand features can include a vehicle brand feature corresponding to a vehicle front face region, a vehicle brand feature corresponding to the vehicle front face region, a vehicle brand feature corresponding to a left headlamp region, a vehicle brand feature corresponding to a right headlamp region, a vehicle brand feature corresponding to a vehicle logo region, a vehicle brand feature corresponding to a left rearview mirror region, a vehicle brand feature corresponding to a right rearview mirror region, a vehicle brand feature corresponding to a left fog lamp region, and a vehicle brand feature corresponding to a right fog lamp region. The plurality of pieces of vehicle brand feature information may be features obtained by performing secondary feature extraction on the first feature map via the second network. The step of extracting a plurality of vehicle brand features in the second feature map according to the first vehicle feature area comprises the following steps: firstly, determining the proportional relation between the first characteristic diagram and the second characteristic diagram, then obtaining a second vehicle characteristic region according to the proportional relation and the second characteristic diagram, and finally extracting a plurality of pieces of vehicle brand characteristic information in the second characteristic diagram corresponding to the second vehicle characteristic region to obtain a plurality of vehicle brand characteristics.
Referring to fig. 2, step S104 may further include the following sub-steps:
and a substep S1041 of determining a proportional relationship between the first characteristic diagram and the second characteristic diagram according to the total number of layers of the second pooling layer in the second network.
In an embodiment of the present invention, the total number of second pooled layers in the second network is the total number of second pooled layers in all second convolutional pooled groups in the second network. Since the second network includes a plurality of second pooling layers, the size of the picture is reduced each time the pooling process is performed, and the second feature map is obtained by inputting the first feature map into the second network for secondary feature extraction, the size of the second feature map is not consistent with the size of the first feature map, and the proportional relationship between the first feature map and the second feature map needs to be known.
The second network may be determined based on a total number of second pooling layersA ratio of the first characteristic diagram to the second characteristic diagram, the second network comprises a plurality of second pooling layers, and if the total number of the second pooling layers in the second network is N and the size of the picture is half of the original size after each pooling process, the first characteristic diagram/the second characteristic diagram =2 N . For example, the total number of the second pooling layers N =3 in the second network, and the ratio of the size of the first feature map to the size of the second feature map is obtained as the first feature map/second feature map =2 N ＝2 3 ＝8。
And a substep S1042, mapping the first vehicle characteristic region to the second characteristic map according to the proportional relation, so as to obtain a second vehicle characteristic region.
In the embodiment of the present invention, the second vehicle feature region may be a plurality of regions obtained by mapping the first vehicle feature region to the second feature map. The second vehicle characteristic region comprises 9 regions, such as a vehicle front face region, a left headlamp region, a right headlamp region, a vehicle logo region, a left rearview mirror region, a right rearview mirror region, a left fog lamp region, a right fog lamp region, a grille region and the like, which are in one-to-one correspondence with the first vehicle characteristic region. It should be noted that the relationship between 9 regions in the second vehicle feature region and 9 regions in the first feature region is related to the proportional relationship between the first feature map and the second feature map. The first vehicle feature area is directly obtained from the first feature map after area detection, so the first vehicle feature area is also selected according to the size of the first feature map, and therefore the mapping of the first vehicle feature area to the second feature map also needs to be in accordance with a proportional relationship. In one embodiment, the proportional relationship is first inverted, and then the first vehicle feature region is multiplied by the inverse before the first vehicle feature region is mapped to the second feature map. For example, the ratio of the size of the first map to the size of the second map is 8:1, the reciprocal is found to be 0.125, and the first vehicle feature area is multiplied by the reciprocal and then mapped to the second map. When the vehicle front face region can be represented by coordinates (0,0) and coordinates (10,8), the coordinates (0,0) and the coordinates (1.25,1) are obtained by multiplying 0.125, and rectangles of the coordinates (0,0) and the coordinates (1.25,1) are mapped to the second feature map, so that a second vehicle sub-feature region corresponding to the vehicle front face region is obtained. The extraction method of the second vehicle sub-feature region corresponding to the other regions is the same, and is not described herein again.
In the substep S1043, a plurality of pieces of vehicle brand feature information corresponding to the second vehicle feature region are extracted to obtain a plurality of vehicle brand features.
In the embodiment of the present invention, the second feature map includes a plurality of pieces of vehicle brand feature information, for example, vehicle brand feature information corresponding to a rectangular area in which coordinates of a front face area in the second vehicle feature area are coordinates (0,0) and coordinates (1.25,1) is extracted, and a vehicle brand feature corresponding to the front face area is obtained. The extraction method of the brand features of the vehicles corresponding to other areas is the same, and is not described herein again.
And step S105, identifying the vehicle brand in the vehicle picture according to the plurality of vehicle brand features.
In embodiments of the present invention, the vehicle brand may be, but is not limited to, popular, ford, liberation, and the like. The step of identifying the vehicle brand in the vehicle picture according to the plurality of vehicle brand features may be to classify the vehicle brand features by using a support vector machine to identify the vehicle brand in the vehicle picture, and may also be to identify the plurality of vehicle brand features by using a third network of a convolutional neural network to obtain the vehicle brand in the vehicle picture. As one implementation, the vehicle brand features are classified by a support vector machine to identify the vehicle brand in the vehicle picture. Extracting shape feature vectors Of each brand feature Of the vehicle by using HOG (Histogram Of gradient), extracting appearance feature vectors by using LBP (Local Binary Pattern), connecting the shape feature vectors and the appearance feature vectors Of all the brand features Of the vehicle into a total feature vector, matching the total feature vector with the stored standard feature vectors by using a support vector machine, calculating the confidence coefficient Of the brand Of the vehicle based on the shape feature vectors if the total feature vectors are matched with the stored standard feature vectors, and determining the brand with the highest confidence coefficient as the brand Of the vehicle picture.
In another embodiment, a third network of convolutional neural networks is used to identify a plurality of vehicle brand features to obtain the vehicle brand in the vehicle picture. The third network may be a pyramid pooling layer and a 1 × 1 convolutional layer, or a pyramid pooling layer and a fully-connected layer. The pyramid pooling layer of the third network is connected to the second network. The step of identifying a plurality of vehicle brand features by using a third network of the convolutional neural network to obtain the vehicle brand in the vehicle picture comprises the following steps: firstly, inputting a plurality of vehicle brand features into a pyramid layer of a third network for maximum pooling processing to obtain a plurality of brand feature vectors, arranging the brand feature vectors according to a preset sequence to obtain a feature vector sequence, then inputting the feature vector sequence into a 1 × 1 convolution layer or a full connection layer of the third network to obtain a one-dimensional feature vector and determine a maximum feature value therein, and finally determining the vehicle brand of a vehicle picture according to the vehicle brand list and the maximum feature value.
Referring to fig. 3, the step of identifying a plurality of vehicle brand features by using a third network of the convolutional neural network in step S105 to obtain the vehicle brand in the vehicle picture may further include the following sub-steps:
and a substep S1051 of performing pyramid pooling on the plurality of vehicle brand features to obtain a plurality of brand feature vectors.
In the embodiment of the invention, a plurality of vehicle brand features are input into the pyramid pooling layer, and the plurality of vehicle brand features are processed by utilizing a maximum pooling method to obtain a plurality of brand feature vectors. The brand feature vector may be a vector relating to vehicle brand features that is derived from a plurality of vehicle brand features via a pyramid pooling layer. Assume that the size of the vehicle brand feature of the front face is a × a, and the vehicle brand feature of the front face is divided into sub-areas of size n × n. Pyramid pooling can be seen as a convolution operator with window size a/n and step size a/n in sliding window mode. Alternatively, three levels of spatial pyramid pooling configurations are used, with nxn being 1 × 1, 2 × 2, and 4 × 4, respectively. The final output of the pyramid pooling is to connect the three levels of pooling results into a vector, namely the brand feature vector of the front vehicle face, and the brand feature vector of the front vehicle face is of a fixed length, so that the size of the input brand feature of the vehicle does not need to be considered. The extraction method of the brand feature vectors of other regions is the same, and is not described herein again. The sub-step S1051 obtains a brand feature vector of the front face, a brand feature vector of the left headlight, a brand feature vector of the right headlight, a brand feature vector of the emblem, a brand feature vector of the left rearview mirror, a brand feature vector of the right rearview mirror, a brand feature vector of the left fog lamp, a brand feature vector of the fog lamp, and a brand feature vector of the grille.
And a substep S1052, arranging the plurality of brand feature vectors according to a preset sequence to obtain a feature vector sequence.
In an embodiment of the present invention, the preset sequence may be, but is not limited to, front car-left headlight-right headlight-emblem-left rearview mirror-right rearview mirror-left fog light-grille. The feature vector sequence may be obtained by sequentially arranging a plurality of brand feature vectors. And (4) sequentially refining the plurality of brand feature vectors obtained in the substep S1051 according to a preset sequence to obtain a feature vector sequence.
And a substep S1053 of performing convolution operation on the feature vector sequence to obtain a one-dimensional feature vector containing a plurality of feature values.
In the embodiment of the present invention, the one-dimensional feature vector may be a one-dimensional vector obtained by performing convolution operation on a feature vector sequence, and the feature value may be a value included in the one-dimensional feature vector. For example, the one-dimensional feature vector may be (3,5,7,9, … …), and the inner 3,5,7,9,3 is the feature value.
And a substep S1054 of determining a maximum eigenvalue from the plurality of eigenvalues.
In the embodiment of the present invention, the maximum eigenvalue is the maximum of all eigenvalues. The largest eigenvalue of the plurality of eigenvalues may be determined using a bubble method, an interpolated ranking method, a selective ranking method, and the like. For example, the bubble method is selected to determine the maximum eigenvalue, and the adjacent eigenvalues are compared. If the first eigenvalue is greater than the second eigenvalue, swapping the positions of the two; the same is done for each pair of adjacent eigenvalues, from the beginning to the end, so that at the end the eigenvalue is the maximum eigenvalue.
The electronic device 100 stores a vehicle brand table in advance, where the vehicle brand table includes a plurality of sequentially arranged vehicle brand names, and the vehicle brand table is a one-dimensional vector. The vehicle brand name may be, but is not limited to, popular, galloping, BMW, etc. For example, the one-dimensional feature vector may be (Volkswagen, buick, chevrolet, modern, BYD, benz, BMW … …)
And the substep S1055, using the vehicle brand name corresponding to the maximum characteristic value in the vehicle brand table as the vehicle brand of the vehicle picture.
In the embodiment of the invention, the one-dimensional feature vector and the vehicle brand table are both one-dimensional vectors, the one-dimensional feature vector is aligned with the vehicle brand table, the feature value in each one-dimensional feature vector corresponds to one vehicle brand name, and the vehicle brand name corresponding to the maximum feature value in the vehicle brand table is the vehicle brand of the vehicle picture. For example, the one-dimensional feature vector is (2,4,7,8,5), the vehicle brand table is (popular, buck, schofland, modern, biddie), and it can be understood that the vehicle brand name corresponding to the feature value 2 is popular, the vehicle brand name corresponding to the feature value 4 is buck, the vehicle brand name corresponding to the feature value 7 is schofland, the vehicle brand name corresponding to the feature value 8 is modern, and the vehicle brand name corresponding to the feature value 5 is biddie. Since 8 is the largest feature value among all feature values, the brand name of the vehicle corresponding to the feature value 8 is used as the brand of the vehicle in the vehicle picture.
Compared with the prior art, the embodiment of the invention has the following advantages:
firstly, 9 areas of a front face, a left headlamp, a right headlamp, a logo, a left rearview mirror, a right rearview mirror, a left fog lamp, a right fog lamp and a grille are detected to be used for brand recognition, so that richer characteristic information is obtained, the robustness on the interference of factors such as fouling, inclination, small pixels, surrounding textures and partial shielding is higher, the accuracy of the brand recognition of the vehicle is improved, and the brand recognition of the vehicle is also carried out when the conditions that the vehicle has no license plate, the license plate cannot be positioned, the license plate is not accurately positioned and the like occur, so that the efficiency of the brand recognition of the vehicle is improved.
And secondly, the first characteristic diagram in the area detection is obtained by extracting the characteristics of the vehicle picture through a first network in the convolutional neural network, so that the sharing of partial network parameters and the first characteristic diagram is realized, the video memory occupation is reduced, and the speed of identifying the vehicle brand is improved.
Finally, the vehicle region feature detection and the vehicle brand identification belong to different learning tasks, so that the end-to-end vehicle region feature detection and the vehicle brand identification multi-task learning are realized. Therefore, sharing of network parameters and the first feature map of the vehicle area detection network and the vehicle brand identification network is achieved, video memory occupation of the model is reduced, meanwhile, regularization is carried out on vehicle brand learning through vehicle area feature detection learning, and generalization capability of the model is improved.
Second embodiment
Referring to FIG. 4, FIG. 4 is a block diagram illustrating a brand identification device 200 according to an embodiment of the present invention. The vehicle brand identification apparatus 200 includes a first feature extraction module 201, an area detection module 202, a second feature extraction module 203, an execution module 204, and a brand identification module 205.
The first feature extraction module 201 is configured to acquire a vehicle picture, input the vehicle picture into a preset convolutional neural network, and perform feature extraction by using a first network of the convolutional neural network to obtain a first feature map.
An area detection module 202, configured to perform area detection on the first feature map to obtain a first vehicle feature area in the first feature map.
And the second feature extraction module 203 is configured to perform secondary feature extraction on the first feature map by using a second network of the convolutional neural network to obtain a second feature map.
In an embodiment of the present invention, the first network includes a plurality of first convolutional layers and at least one first pooling layer, and a total number of layers of the plurality of first convolutional layers is greater than a total number of layers of the at least one first pooling layer; the second network comprises a plurality of second convolutional layers and at least one second pooling layer, and the total number of layers of the plurality of second convolutional layers is greater than that of the at least one second pooling layer.
The executing module 204 is configured to extract a plurality of vehicle brand features in the second feature map according to the first vehicle feature area, where the second feature map includes a plurality of pieces of vehicle brand feature information.
In this embodiment of the present invention, the executing module 204 is specifically configured to: determining the proportional relation between the first characteristic diagram and the second characteristic diagram according to the total number of the second pooling layers in the second network; mapping the first vehicle characteristic region to a second characteristic map according to the proportional relation to obtain a second vehicle characteristic region; and extracting a plurality of pieces of vehicle brand feature information corresponding to the second vehicle feature area to obtain a plurality of vehicle brand features.
The brand identification module 205 is configured to identify a vehicle brand in the vehicle picture according to a plurality of vehicle brand features.
In an embodiment of the present invention, the brand identification module 205 may be configured to classify the brand features of the vehicle by using a support vector machine to identify the brand of the vehicle in the vehicle picture, and may be further configured to identify the brand features of the vehicle by using a third network of the convolutional neural network to obtain the brand of the vehicle in the vehicle picture.
In this embodiment of the present invention, brand identification module 205 is specifically configured to: carrying out pyramid pooling on the plurality of vehicle brand features to obtain a plurality of brand feature vectors; arranging the plurality of brand feature vectors according to a preset sequence to obtain a feature vector sequence; performing convolution operation on the feature vector sequence to obtain a one-dimensional feature vector containing a plurality of feature values; determining a maximum eigenvalue from the plurality of eigenvalues; and taking the corresponding vehicle brand name of the maximum characteristic numerical value in the vehicle brand table as the vehicle brand of the vehicle picture.
The embodiment of the invention also provides a computer-readable storage medium, which has computer-executable instructions, and the computer-executable instructions are used for realizing the vehicle brand identification method.
In summary, the present invention provides a method, an apparatus and a readable storage medium for identifying a vehicle brand, wherein the method includes: acquiring a vehicle picture, inputting the vehicle picture into a preset convolutional neural network, and extracting features by using a first network of the convolutional neural network to obtain a first feature map; carrying out area detection on the first characteristic diagram to obtain a first vehicle characteristic area in the first characteristic diagram; performing secondary feature extraction on the first feature map by using a second network of the convolutional neural network to obtain a second feature map; extracting a plurality of vehicle brand features in a second feature map according to the first vehicle feature area, wherein the second feature map comprises a plurality of vehicle brand feature information; and identifying the vehicle brand in the vehicle picture according to the plurality of vehicle brand characteristics. Compared with the prior method for identifying the brand of the vehicle according to the accurate position information of the license plate, the method for identifying the brand of the vehicle provided by the invention has the advantages that 9 areas including the front face, the left headlamp, the right headlamp, the logo, the left rearview mirror, the right rearview mirror, the left fog lamp, the right fog lamp and the grille are detected to be used for identifying the brand of the vehicle, so that richer characteristic information is obtained, the method has stronger robustness on the interference of factors such as fouling, tilting, small pixels, surrounding textures, partial shielding and the like, the accuracy of identifying the brand of the vehicle is improved, and the brand of the vehicle is also identified when the vehicle has no license plate, the license plate cannot be positioned, the license plate is not accurately positioned and the like, so that the brand efficiency of the vehicle is improved; and secondly, the first characteristic diagram in the area detection is obtained by extracting the characteristics of the vehicle picture through a first network in the convolutional neural network, so that the sharing of partial network parameters and the first characteristic diagram is realized, the video memory occupation is reduced, and the speed of identifying the vehicle brand is improved. Finally, the vehicle region feature detection and the vehicle brand identification belong to different learning tasks, so that the end-to-end vehicle region feature detection and the vehicle brand identification multi-task learning are realized. Therefore, sharing of network parameters and the first feature map of the vehicle area detection network and the vehicle brand identification network is achieved, video memory occupation of the model is reduced, meanwhile, regularization is carried out on vehicle brand learning through vehicle area feature detection learning, and generalization capability of the model is improved.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative and, for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes. It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
Claims (10)
1. A vehicle brand identification method applied to an electronic device, the method comprising:
acquiring a vehicle picture, inputting the vehicle picture into a preset convolutional neural network, and extracting features by using a first network of the convolutional neural network to obtain a first feature map;
carrying out area detection on the first characteristic diagram to obtain a first vehicle characteristic area in the first characteristic diagram; the first vehicle characteristic region comprises a vehicle front face region, a left headlamp region, a right headlamp region, a vehicle logo region, a left rearview mirror region, a right rearview mirror region, a left fog lamp region, a right fog lamp region and a grille region;
performing secondary feature extraction on the first feature map by using a second network of the convolutional neural network to obtain a second feature map;
extracting a plurality of vehicle brand features in the second feature map according to the first vehicle feature area, wherein the second feature map comprises a plurality of pieces of vehicle brand feature information; the plurality of vehicle brand features comprise a vehicle brand feature corresponding to the front face area of the vehicle, a vehicle brand feature corresponding to the left headlamp area, a vehicle brand feature corresponding to the right headlamp area, a vehicle brand feature corresponding to the vehicle logo area, a vehicle brand feature corresponding to the left rearview mirror area, a vehicle brand feature corresponding to the right rearview mirror area, a vehicle brand feature corresponding to the left fog lamp area, a vehicle brand feature corresponding to the right fog lamp area and a vehicle brand feature corresponding to the grille area;
identifying a vehicle brand in the vehicle picture according to the plurality of vehicle brand features.
2. The method of claim 1, wherein the first network comprises a plurality of first convolutional layers and at least one first pooling layer, and a total number of layers of the plurality of first convolutional layers is greater than a total number of layers of the at least one first pooling layer;
the second network comprises a plurality of second convolutional layers and at least one second pooling layer, and the total number of layers of the second convolutional layers is greater than that of the at least one second pooling layer.
3. The method of claim 2, wherein the step of extracting a plurality of vehicle brand features in the second feature map according to the first vehicle feature region comprises:
determining a proportional relation between the first characteristic diagram and the second characteristic diagram according to the total number of the second pooling layers in the second network;
mapping the first vehicle characteristic region to the second characteristic map according to the proportional relation to obtain a second vehicle characteristic region;
extracting the plurality of pieces of vehicle brand feature information corresponding to the second vehicle feature area to obtain the plurality of vehicle brand features.
4. The method of claim 1, wherein the step of identifying the vehicle brand in the vehicle picture from the plurality of vehicle brands comprises:
and identifying the plurality of vehicle brand features by utilizing a third network of the convolutional neural network to obtain the vehicle brand in the vehicle picture.
5. The method of claim 4, wherein a vehicle brand table is pre-stored in the electronic device, the vehicle brand table includes a plurality of sequentially arranged vehicle brand names, the vehicle brand table is a one-dimensional vector, and the step of identifying the plurality of vehicle brand features by using a third network of the convolutional neural network to obtain the vehicle brand in the vehicle picture comprises:
performing pyramid pooling on the plurality of vehicle brand features to obtain a plurality of brand feature vectors;
arranging the plurality of brand feature vectors according to a preset sequence to obtain a feature vector sequence;
performing convolution operation on the feature vector sequence to obtain a one-dimensional feature vector containing a plurality of feature numerical values;
determining a maximum eigenvalue from the plurality of eigenvalues;
and taking the corresponding vehicle brand name of the maximum characteristic numerical value in the vehicle brand table as the vehicle brand of the vehicle picture.
6. The method of claim 5, wherein the third network comprises a 1 x 1 convolutional layer or a fully connected layer.
7. A vehicle brand recognition apparatus applied to an electronic device, the apparatus comprising:
the first feature extraction module is used for acquiring a vehicle picture, inputting the vehicle picture into a preset convolutional neural network, and extracting features by using a first network of the convolutional neural network to obtain a first feature map;
the area detection module is used for carrying out area detection on the first characteristic diagram to obtain a first vehicle characteristic area in the first characteristic diagram; the first vehicle characteristic region comprises a vehicle front face region, a left headlamp region, a right headlamp region, a vehicle logo region, a left rearview mirror region, a right rearview mirror region, a left fog lamp region, a right fog lamp region and a grille region;
the second feature extraction module is used for performing secondary feature extraction on the first feature map by using a second network of the convolutional neural network to obtain a second feature map;
the execution module is used for extracting a plurality of vehicle brand features in the second feature map according to the first vehicle feature area, wherein the second feature map comprises a plurality of pieces of vehicle brand feature information; the plurality of pieces of vehicle brand feature information comprise brand feature vectors of a front face of the vehicle, brand feature vectors of left headlamps, brand feature vectors of right headlamps, brand feature vectors of a logo, brand feature vectors of left rearview mirrors, brand feature vectors of right rearview mirrors, brand feature vectors of left fog lamps, brand feature vectors of right fog lamps and brand feature vectors of grids;
and the brand identification module is used for identifying the vehicle brand in the vehicle picture according to the plurality of vehicle brand characteristics.
8. The apparatus of claim 7, wherein the first network comprises a plurality of first convolutional layers and at least one first pooling layer, and a total number of layers of the plurality of first convolutional layers is greater than a total number of layers of the at least one first pooling layer;
the second network comprises a plurality of second convolutional layers and at least one second pooling layer, and the total number of layers of the second convolutional layers is greater than that of the at least one second pooling layer.
9. The apparatus of claim 8, wherein the execution module is specifically configured to:
determining a proportional relation between the first characteristic diagram and the second characteristic diagram according to the total number of the second pooling layers in the second network;
mapping the first vehicle characteristic region into the second characteristic map according to the proportional relation to obtain a second vehicle characteristic region;
extracting the plurality of vehicle brand feature information corresponding to the second vehicle feature area to obtain the plurality of vehicle brand features.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811070622.6A CN110895692B (en) | 2018-09-13 | 2018-09-13 | Vehicle brand identification method and device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811070622.6A CN110895692B (en) | 2018-09-13 | 2018-09-13 | Vehicle brand identification method and device and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110895692A CN110895692A (en) | 2020-03-20 |
| CN110895692B true CN110895692B (en) | 2023-04-07 |
Family
ID=69785534
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811070622.6A Active CN110895692B (en) | 2018-09-13 | 2018-09-13 | Vehicle brand identification method and device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110895692B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111754784B (en) * | 2020-06-23 | 2022-05-24 | 高新兴科技集团股份有限公司 | Method for identifying main and sub brands of vehicle based on multi-layer network of attention mechanism |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103488973A (en) * | 2013-09-12 | 2014-01-01 | 上海依图网络科技有限公司 | Method and system for recognizing vehicle brand based on image |
| CN105894045A (en) * | 2016-05-06 | 2016-08-24 | 电子科技大学 | Vehicle type recognition method with deep network model based on spatial pyramid pooling |
| CN107274451A (en) * | 2017-05-17 | 2017-10-20 | 北京工业大学 | Isolator detecting method and device based on shared convolutional neural networks |
| CN108334892A (en) * | 2017-12-26 | 2018-07-27 | 新智数字科技有限公司 | A kind of model recognizing method, device and equipment based on convolutional neural networks |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170124409A1 (en) * | 2015-11-04 | 2017-05-04 | Nec Laboratories America, Inc. | Cascaded neural network with scale dependent pooling for object detection |
| CN105654066A (en) * | 2016-02-02 | 2016-06-08 | 北京格灵深瞳信息技术有限公司 | Vehicle identification method and device |
-
2018
- 2018-09-13 CN CN201811070622.6A patent/CN110895692B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103488973A (en) * | 2013-09-12 | 2014-01-01 | 上海依图网络科技有限公司 | Method and system for recognizing vehicle brand based on image |
| CN105894045A (en) * | 2016-05-06 | 2016-08-24 | 电子科技大学 | Vehicle type recognition method with deep network model based on spatial pyramid pooling |
| CN107274451A (en) * | 2017-05-17 | 2017-10-20 | 北京工业大学 | Isolator detecting method and device based on shared convolutional neural networks |
| CN108334892A (en) * | 2017-12-26 | 2018-07-27 | 新智数字科技有限公司 | A kind of model recognizing method, device and equipment based on convolutional neural networks |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110895692A (en) | 2020-03-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3806064B1 (en) | Method and apparatus for detecting parking space usage condition, electronic device, and storage medium | |
| CN109658454B (en) | Pose information determination method, related device and storage medium | |
| CN110909712B (en) | Moving object detection method and device, electronic equipment and storage medium | |
| CN104361359A (en) | Vehicle recognition method based on image detection | |
| CN109348731A (en) | A kind of method and device of images match | |
| CN107748882B (en) | Lane line detection method and device | |
| CN115100450B (en) | Intelligent traffic brand automobile big data detection method and system based on artificial intelligence | |
| CN110853085B (en) | Semantic SLAM-based mapping method and device and electronic equipment | |
| CN111860219A (en) | High-speed road occupation judging method and device and electronic equipment | |
| CN110705338A (en) | Vehicle detection method and device and monitoring equipment | |
| CN110544268A (en) | Multi-target tracking method based on structured light and SiamMask network | |
| CN110895692B (en) | Vehicle brand identification method and device and readable storage medium | |
| CN110097108B (en) | Method, device, equipment and storage medium for identifying non-motor vehicle | |
| JP2021532449A (en) | Lane attribute detection | |
| CN117218622A (en) | Road condition detection method, electronic equipment and storage medium | |
| CN116863124B (en) | Vehicle attitude determination method, controller and storage medium | |
| CN111678488B (en) | Distance measuring method and device, computer readable storage medium and electronic equipment | |
| CN112287905A (en) | Vehicle damage identification method, device, equipment and storage medium | |
| CN111709377A (en) | Feature extraction method, target re-identification method and device and electronic equipment | |
| WO2023044656A1 (en) | Vehicle passage warning method and apparatus, and vehicle-mounted terminal | |
| CN109213515A (en) | Normalizing method and device and an electronic equipment are buried under multi-platform | |
| KR20190081910A (en) | Method for auto-conversion of multi-depth image | |
| CN111126271B (en) | Bayonet snap image vehicle detection method, computer storage medium and electronic equipment | |
| CN109409247B (en) | Traffic sign identification method and device | |
| CN111639640A (en) | License plate recognition method, device and equipment based on artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |