WO2023155483A1

WO2023155483A1 - Vehicle type identification method, device, and system

Info

Publication number: WO2023155483A1
Application number: PCT/CN2022/128972
Authority: WO
Inventors: 潘新生; 黄宇恒; 金晓峰; 杨振; 徐天适
Original assignee: 广州广电运通金融电子股份有限公司
Priority date: 2022-02-17
Filing date: 2022-11-01
Publication date: 2023-08-24
Also published as: CN114639078A

Abstract

The present application relates to vehicle type identification, and in particular relates to a vehicle type identification method, a device, and a system. The vehicle type identification method comprises: acquiring a vehicle front image, a vehicle body image, and a vehicle back image of a vehicle; obtaining width, height, and length data of the vehicle, the width, height, and length data comprising first width-height data, second width-height data and third width-height data; obtaining vehicle type data of the vehicle, the vehicle type data comprising vehicle type information and axle information; and combining the width, height, and length data and the vehicle type data to output vehicle type identification data. After the vehicle front image, the vehicle body image, and the vehicle back image are obtained, license plate evaluation is carried out, and then a first proportionality coefficient is obtained according to a license plate image size and a license plate physical size; subsequently, accurate first width-height data of the front of the vehicle, second width-height data of the back of the vehicle, and third width-height data of the vehicle body are obtained, and the vehicle type and the axle type are also accurately identified; data of a vehicle that has undergone vehicle type identification can be accurately obtained, and identification accuracy can be improved.

Description

A vehicle type identification method, device and system

This disclosure claims the priority of the Chinese patent application with the application number 202210147600.5 and the title of the invention "a vehicle identification method, device and system" submitted to the China Patent Office on February 17, 2022, the entire contents of which are incorporated herein by reference. In public.

technical field

The present disclosure relates to vehicle type identification, and in particular to a vehicle type identification method, device and system.

Background technique

Vehicle length, width and height, number of vehicle axles, vehicle axle type, license plate type, license plate color, license plate number, vehicle shape and other information are important basis for vehicle type classification. Judgment, traffic police road management, road traffic billing and other applications play an important role. Because the vehicle type classification is based on many information items and complement each other, a single sensor device such as monocular camera, binocular camera, weighing, laser scanning, and infrared scanning cannot obtain relevant information comprehensively and accurately. Therefore, the vehicle type classification It is difficult to obtain a high accuracy rate in recognition accuracy.

Patent CN111783638A proposes a system and method for detecting the number of vehicle axles and vehicle type identification. The system includes a distance measuring sensor device, a high-frequency parallel signal acquisition device, and a calculation processing device. Through the distance measuring sensor device, a wheel depth map can be obtained And the data sequence of the vehicle body depth map to detect the number of vehicle axles, and obtain the vehicle model recognition result based on the license plate information of the acquisition device, but the information items based on the vehicle are only the vehicle wheel axle number and license plate information, which is not enough to realize the accurate identification of the vehicle model.

Patent CN111523579A proposes a vehicle type recognition method and system based on improved deep learning. This method is based on deep learning methods and requires a large number of vehicle image data sets at traffic checkpoints for cutting, sorting, classification, and training to realize the classification and recognition of vehicle types. Without the shape information of the vehicle image, the accuracy of vehicle model recognition is difficult to guarantee.

Therefore, there are still deficiencies in the existing vehicle type identification technology, which needs to be improved and improved.

Contents of the invention

(1) Technical problems to be solved

The accuracy rate of the existing vehicle type identification technology for vehicle type identification is not high.

(2) Technical solution

In view of the shortcomings of the above-mentioned prior art, the purpose of the present disclosure is to provide a vehicle type identification method, device and system, which can improve the accuracy of the length, width, and height of vehicle identification, and at the same time fully identify the vehicle type, and comprehensively improve the vehicle type. recognition accuracy.

In order to achieve the above object, the present disclosure adopts the following technical solutions:

On the one hand, the present disclosure provides a vehicle type identification method, including:

Obtain the front image, body image and rear image of the vehicle;

Obtain the width, height and length data of the vehicle; the width, height and length data include the first width and height data, the second width and height data, and the third width and height data; wherein, license plate detection is performed on the front image and the rear image, Obtain the first proportional coefficient between the image size of the license plate and the physical size of the license plate, and then obtain the first width and height data of the front of the car and the second width and height data of the rear of the car respectively according to the image size of the front of the car and the image of the rear of the car; Or perform feature value extraction and matching between the rear image and the vehicle body image, and obtain the third width and height data of the vehicle body according to the first scale factor and the size of the vehicle body image;

Obtain vehicle model data; the vehicle model data includes vehicle model information and axle information; wherein, the vehicle head image is recognized by the first recognition model to obtain the vehicle model information; the vehicle body image is recognized by the second recognition model to obtain said axle information;

Integrating the width, height and length data and the vehicle type data to output vehicle type identification data.

Preferably, the step of obtaining the first width and height data or the second width and height includes:

Perform license plate image recognition on the first image through a third recognition model to obtain license plate position data and license plate type data; the first image is the front image or the rear image;

adjusting the aspect ratio of the license plate image to be the same as the actual physical aspect ratio of the corresponding license plate type through affine transformation, and synchronously adjusting the aspect ratio of the first image based on the same adjustment ratio to obtain a first adjusted image;

Obtain the first scale factor between the image size of the license plate image and the physical size of the license plate;

The image size of the first adjusted image is acquired, and the first width and height data or the second width and height data are obtained based on the first proportional coefficient.

Preferably, obtaining the proportionality coefficient specifically includes:

Obtain the character image size of each character in the license plate image;

According to the physical size of the characters of the license plate, the first proportional coefficient is obtained.

Preferably, the step of obtaining the third width and height data is:

Obtain the feature matching points of the first adjustment image and the vehicle body image, and then obtain the corresponding second scale factor;

performing normalization processing on the body image according to the second scale factor to obtain a first normalized image;

Acquire the image size of the first normalized image, and obtain third width and height data based on the first scale factor.

Preferably, the first adjustment image includes a front adjustment image and a rear adjustment image;

The steps for obtaining the third width and height data are:

Obtain the feature matching points of the adjusted image of the front of the car and the image of the body, and then obtain the corresponding third proportional coefficient;

performing normalization processing on the body image according to the third scale factor to obtain a second normalized image;

Obtain the feature matching points of the rear adjustment image and the body image, and then obtain the corresponding fourth proportional coefficient;

performing normalization processing on the body image according to the fourth scale factor to obtain a third normalized image;

Acquire the image size of the second normalized image, and obtain the first vehicle body width and height data based on the first scale factor;

Acquire the image size of the third normalized image, and obtain second vehicle body width and height data based on the first scale factor;

Weighted calculation is performed based on the first vehicle body width and height data and the second vehicle body width and height data to obtain the third width and height data.

Preferably, the axle information includes axle number and axle type;

The vehicle type identification data further includes: the vehicle front image, the vehicle body image, the vehicle rear image, and license plate type data.

Preferably, the step of acquiring the front image or the rear image includes:

Acquire the second image, use the fourth recognition model to identify the front or rear of the second image, and obtain rough rectangular frame position information; the second image is an image with front or rear features;

Image edge detection is performed on the rough rectangular frame to obtain a front image or a rear image.

Preferably, the step of acquiring the body image includes:

Acquire continuously generated multi-frame third images; the third images are images with vehicle body features;

Calculate the feature vector of the third image described in each frame, and calculate the matching similarity with the third image described in the previous frame respectively, and then obtain the corresponding offset;

The plurality of frames of the third image are stitched according to the translation transformation corresponding to the offset to obtain the vehicle body image.

In another aspect, the present disclosure provides a vehicle type identification device, including:

The obtaining module is used to obtain the front image, the body image and the rear image of the vehicle;

The processing module is used to obtain the width, height and length data of the vehicle; the width, height and length data include the first width and height data, the second width and height data, and the third width and height data; wherein, the image of the front of the vehicle and the rear of the vehicle Carry out license plate detection on the image to obtain the first proportional coefficient between the license plate image size and the physical size of the license plate, and then obtain the first width and height data of the front of the car and the second width and height data of the rear of the car according to the image size of the front of the car and the image size of the rear of the car respectively; Extracting and matching the feature values of the front image and/or the rear image and the vehicle body image, obtaining the third width and height data of the vehicle body according to the first scale factor and the size of the vehicle body image; obtaining the vehicle type data; the vehicle type The data includes vehicle type information and axle information; wherein, the vehicle type information is obtained by identifying the vehicle head image through the first recognition model; the vehicle body image is recognized by the second recognition model to obtain the axle information; the width is integrated The height data and the vehicle type data are used to output vehicle type identification data.

In another aspect, the present disclosure provides a vehicle type identification system, including:

Multi-camera with multiple cameras with different orientations;

The vehicle type recognition device is connected with the multi-eye camera, and is used to receive the images transmitted by multiple cameras, and then obtain the front image, body image and rear image of the vehicle; and obtain the width, height and length data of the vehicle; the width, height and length The data includes the first width and height data, the second width and height data, and the third width and height data; wherein, the license plate detection is performed on the front image and the rear image to obtain the first distance between the image size of the license plate and the physical size of the license plate. Scale coefficient, and then obtain the first width and height data of the front of the car and the second width and height data of the rear according to the size of the front image and the size of the rear image respectively; perform feature value extraction on the front image and/or the rear image and the body image and matching, obtain the third width and height data of the vehicle body according to the first scale factor and the image size of the vehicle body; obtain the vehicle model data; the vehicle model data includes vehicle model information and axle information; Recognizing the vehicle head image to obtain the vehicle type information; using a second recognition model to recognize the vehicle body image to obtain the axle information; integrating the width, height and length data with the vehicle type data to output vehicle type identification data.

(3) Beneficial effects

Compared with the prior art, the vehicle type identification method, device and system provided by the present disclosure have the following beneficial effects:

Using the vehicle type identification method provided by the present disclosure, after obtaining the front image, body image, and rear image, the license plate detection is performed, and then the first proportional coefficient is obtained according to the license plate image size and the license plate physical size, and then the accurate first width of the front of the car is obtained. Height data, the second width and height data of the rear, and the third width and height data of the body, and at the same time accurately identify the vehicle type and axle type, can accurately obtain the vehicle type identification data of passing vehicles, and improve the identification accuracy.

Description of drawings

Fig. 1 is a flow chart of the vehicle type identification method provided by the present disclosure;

Fig. 2 is a schematic diagram of multi-eye camera installation provided by the present disclosure;

Fig. 3 is a flow chart of steps for acquiring width and height data of the front or rear of the vehicle provided by the present disclosure;

Fig. 4 is a flow chart of obtaining steps in another embodiment of the first proportional coefficient provided by the present disclosure;

Fig. 5 is a flowchart of an acquisition method of the third width and height data provided by the present disclosure;

Fig. 6 is a flow chart of another way of obtaining the third width and height data provided by the present disclosure;

FIG. 7 is a flow chart of the body image acquisition steps provided by the present disclosure;

FIG. 8 is a schematic diagram of vehicle body image acquisition provided by the present disclosure;

FIG. 9 is a flow chart of an embodiment of a vehicle type identification method provided by the present disclosure;

Fig. 10 is a structural block diagram of a vehicle type identification device provided by the present disclosure;

Fig. 11 is a structural block diagram of the vehicle type identification system provided by the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and effects of the present disclosure more clear and definite, the present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described here are only used to explain the present disclosure, not to limit the present disclosure.

It is to be understood by those of ordinary skill in the art that the foregoing general description and the following detailed description are exemplary and illustrative specific embodiments of the present disclosure and are not intended to limit the present disclosure.

Herein the terms "comprises", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion such that a process or method comprising a list of steps includes not only those steps but may also include steps not expressly listed or inherent in such process or method other steps. Likewise, one or more equipment or subsystems, elements or structures or assemblies beginning with "consisting of ... a" do not, without more limitations, exclude the presence of other equipment or other sub- system or other element or other structure or other component or other device or other subsystem or other element or other structure or other component. Throughout this specification, appearances of the phrases "in one embodiment," "in another embodiment," and similar language may, but do not necessarily, all refer to the same embodiment.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

Please refer to FIG. 1 , FIG. 2 and FIG. 9 together. The present disclosure provides a vehicle identification method, which is applied to a vehicle identification device and connected with a multi-camera. In this embodiment, according to the actual application scenario, the multi-camera is installed at position 1 or position 2 (as shown in FIG. 2 ) beside the road, and the multi-camera has at least a plurality of cameras, and each camera has a different orientation. to obtain image data of roads in different directions.

In a further preferred solution, the number of the cameras is 3, and the respective orientations facing the road are respectively facing the road (acquiring the second image used to generate the image of the vehicle body), and the direction of the vehicle (acquiring the second image used to generate the image of the vehicle head). image of the vehicle), the direction of vehicle departure (the image used to generate the image of the rear of the vehicle is acquired). In a further preferred solution, the number of the cameras is more than 3, and the number of cameras in each direction is one or more, so that for the same vehicle, the preferred image data can be selected to generate the corresponding front image and rear image , Body image. The requirements for camera installation are shown in the figure: 1) There is a certain degree of overlap between the field of view angles between camera 1 and camera 2, and between camera 2 and camera 3; 2) Each camera can adjust the angle up and down, left and right; 3) Vehicles pass by In the camera group, camera 1 can completely capture the image of the front of the vehicle, camera 2 can completely capture the image of the whole body height of the vehicle body and the image screen should be parallel to the road as far as possible, and camera 3 can completely capture the image of the rear of the vehicle, so that it can be conveniently carried out. Feature point matching between images.

Further, the working process between the multiple cameras of the multi-eye camera is preferably: when the vehicle enters the area 1, the camera 1 collects the image sequence of the front of the vehicle, and the processing module detects and recognizes the image of the front of the vehicle according to the image sequence, and synchronizes the image acquisition signal to the camera 2 ; When the vehicle enters the area 2, the camera 2 collects the image sequence of the vehicle body, and simultaneously sends the image acquisition signal to the camera 3; , and then obtain the front image, body image, and rear image, and the camera acquires images in sequence, which can effectively save power, storage resources, and computing resources.

The vehicle identification method includes:

S1. Acquire the front image, the body image and the rear image of the vehicle; in this embodiment, the front image, the body image, and the rear image are all generated by a sequence of images captured by a camera. Of course, in When using some cameras with automatic recognition function and capable of acquiring cleaned target images, the images transmitted by such cameras can be used directly.

S2. Obtain the width, height and length data of the vehicle; the width, height and length data include the first width and height data, the second width and height data, and the third width and height data; wherein, the license plate is performed on the front image and the rear image Detect to obtain the first proportional coefficient between the image size of the license plate and the physical size of the license plate, and then obtain the first width and height data of the front of the car and the second width and height data of the rear of the car according to the image size of the front of the car and the image of the rear of the car respectively; And/or carry out feature value extraction and matching between the rear image and the vehicle body image, and obtain the third width and height data of the vehicle body according to the first scale factor and the size of the vehicle body image; in this embodiment, the first scale factor Including the first width scaling factor and the first height scaling factor, which are simply calculated separately. For example, if the license plate image size is 400*150 pixels, and the physical size of the license plate is 440mm×140mm, then the first width scaling factor is 0.91 pixels/mm , the first height scale factor is 1.07 pixels/mm, and then after obtaining the front image size or the rear image size, perform fast calculation of the first width and height data and the second width and height data, and the third width and height data can be calculated similarly .

S3. Obtain vehicle model data; the vehicle model data includes vehicle model information and axle information; wherein, the vehicle head image is recognized by the first recognition model to obtain the vehicle model information; the vehicle body image is processed by the second recognition model The axle information is obtained through recognition; specifically, both the first recognition model and the second recognition model are obtained based on a deep neural network model.

Specifically, the training process of the first recognition model includes:

Obtain the first training set, the first training set includes several first training images, the first training image is an image with vehicle characteristics, and labels of various vehicle types are marked; the vehicle types include large cars, Trailer small cars, police cars, etc.

The first recognition model is obtained by using the first training set to train the initialized neural network model.

The training process of the second recognition model includes:

Obtain a second training set, the second training set includes several second training images, the second training image is an image with axle characteristics, and labels of various axle types are marked;

Using the second training set to train the initialized neural network model to obtain the second recognition model.

S4. Integrate the width, height and length data with the vehicle type data, and output vehicle type identification data. Specifically, the data integration process is to perform fusion processing on the identified data to obtain accurate identification of the vehicle model information.

Further, please refer to FIG. 3. As a preferred solution, in this embodiment, the step of obtaining the first width and height data or the second width and height includes:

S21. Perform license plate image recognition on the first image through the third recognition model to obtain license plate position data and license plate type data; the first image is the front image or the rear image; specifically, the third recognition The model is trained based on the neural network model, which can automatically recognize the first image and quickly obtain the corresponding recognition results while ensuring accuracy.

Specifically, the license plate types include: 1. Large-scale car license plate-front: 440mm×140mm-rear: 440mm×220mm-black frame with black characters on yellow background; 2. Trailer license plate-440mm×220mm-black characters on yellow background Black frame line; 3. Small car license plate-440mm×140mm-white frame line with white characters on blue background; 4. License plate of embassy car-440mm×140mm-white characters on black background, red “shi” and “collar” characters and white frame line; 5. Consulate car license plate-440mm×140mm-black background with white characters, red “shi” and “collar” and white frame lines; 6. Hong Kong and Macao entry and exit vehicle license plate-440mm×140mm- black background and white characters, white “Hong Kong” , "Australia" word white frame; 7. Coach car license plate - 440mm × 140mm - yellow background with black characters, black "Xue" character with black frame line; 8. Police car license plate - 440mm × 140mm-white background with black characters , red "police" word black frame.

The training process of the third recognition model includes:

Obtain a third training set, the third training set includes some third training images, the third training images are images with license plate features, and are marked with labels of various license plate types;

Using the third training set to train the initialized neural network model to obtain the third recognition model.

S22. Adjust the aspect ratio of the license plate image to be the same as the actual physical aspect ratio of the corresponding license plate type through affine transformation, and synchronously adjust the aspect ratio of the first image based on the same adjustment ratio to obtain a first adjusted image;

S23. Obtain a first proportionality coefficient between the image size of the license plate image and the physical size of the license plate;

S24. Acquire the image size of the first adjusted image, and obtain first width and height data or second width and height data based on the first scale factor.

In this embodiment, the license plate detection, positioning, correction, type classification, character segmentation and recognition processing are mainly performed on the front image or the rear image.

In a further embodiment, after obtaining the license plate position data, the positions of the four vertices of the license plate rectangular frame are obtained by target detection and feature point extraction, and meanwhile, the third recognition model can also obtain license plate color and license plate type data, Then combine the license plate type information to obtain the license plate image with the same aspect ratio as the actual physical license plate through affine transformation, and then combine the semantic segmentation method to obtain the precise rectangular frame position of each character, and finally recognize the license plate number information, you can quickly know the license plate. Basic information, including number, color, type, etc.

Further, please refer to FIG. 4. As a preferred solution, in this embodiment, obtaining the proportional coefficient specifically includes:

Obtain the character image size of each character in the license plate image;

According to the physical size of the characters of the license plate, the first proportional coefficient is obtained. Even if the physical size and image size of each character are used to generate the first scale factor, it will be more accurate, so that the width and height data of the front or rear of the vehicle can be calculated more accurately. Realize accurate estimation of the physical size of the license plate characters, so as to accurately estimate the physical size of the vehicle's height, width, and length.

In the specific implementation process, after the affine transformation, first obtain the transformation parameter M (M is a 3x3 matrix), and combine the precise position information of the front of the car to obtain the frontal image of the front of the car through the method of affine transformation, and the image size of the obtained front image is W _{img_h} *H _{img_h} .

Obtain the precise rectangular frame position of each character, and obtain the image size W _{img_h_ch_i} *H _{img_h_ch_i} (wherein i is the character serial number) of each character by the transformation parameter M through the method of affine transformation.

According to the image size of the characters and combined with the imaging conditions of each character, the character "1" or other data whose character image size is too large or too small due to defacement and other reasons are eliminated by cluster analysis, and then the average image size of the character is calculated. :

Where N ₁ is the number of valid characters for the width image size, and N ₂ is the number of valid characters for the height image size.

Combining the single-character physical size W _{phy_ch} *H _{phy_h} stipulated in the latest industry standard of "Motor Vehicle Number Plate of the People's Republic of China", the proportional coefficient of the physical character size and image size (ie the first proportional coefficient, wherein the first proportional coefficient includes the first proportional coefficient) is calculated. a width scale factor, a first height scale factor):

The first width scale factor is

The first height scale factor is

Combined with the proportional imaging characteristics of the camera image, the physical width and height of the front of the car (that is, the first width and height data of the front of the car) can be calculated:

In a further embodiment, before acquiring the first scale factor, distortion correction is performed on the first image to make the image closer to the actual shape of the front of the vehicle.

Further, please refer to FIG. 5. As a preferred solution, in this embodiment, the step of obtaining the third width and height data is:

The feature matching points of the first adjusted image and the vehicle body image are obtained, and then the corresponding second proportional coefficient is obtained; the first adjusted image is the first adjusted image of the front image or the first adjusted image of the rear image.

Acquire the image size of the first normalized image, and obtain third width and height data based on the first scale factor. In this embodiment, only the front image or the rear image is used to normalize the vehicle body image, so that the image size of the vehicle body image is adaptively adjusted with the adjusted front image or rear image, and then the first adjusted The normalized image adapted to the image can further improve the calculation accuracy of the width and height of the vehicle body.

Further, please refer to FIG. 6. As a preferred solution, in this embodiment, the first adjustment image includes an adjustment image for the front of the vehicle and an adjustment image for the rear of the vehicle;

The steps for obtaining the third width and height data are:

Weighted calculation is performed based on the first vehicle body width and height data and the second vehicle body width and height data to obtain the third width and height data. In this embodiment, after feature point matching is performed on the vehicle body image and the adjusted image of the front of the vehicle or the adjusted image of the rear of the vehicle, a three-dimensional stereo image of the vehicle can be obtained. At the same time, the front image and the rear image are used to match the body image, which can effectively improve the calculation accuracy of the third width and height data of the body.

In the specific implementation, the feature matching points of the two images are calculated through the complete image of the front of the car and the image of the body, and the third proportional coefficient f ₁ of the height transformation of the two images is obtained:

Wherein H _{img_s} is the height of the vehicle body image, H _{img_h} is the height of the head image, F _s (y _i ) is the y coordinate of the feature point of the head image, F _h (y _i ) is the y coordinate of the feature point of the body image, and K is the number of feature point pairs;

Calculate the second normalized image according to the third scale factor, and obtain the normalized length W _{img0_s} and height H _{img0_s} of the complete image of the side of the vehicle body:

W _{img0_s} =f ₁ *W _{img_s} , where W _{img_s} is the length of the body image,

H _{img0_s} =f ₁ *H _{img_s} , where H _{img_s} is the height of the vehicle body image

According to the first scale factor, combined with the proportional imaging characteristics of the camera image, the length and height information of the first body width and height data of the body can be estimated:

Similarly, the length W" _{phy_s} and the height H" _{phy_s} information of the second body width and height data of the body can be obtained by using the feature point matching of the rear adjustment image and the body image and the height information of the rear body;

In order to more accurately estimate the third width and height data of the body, use the first body width and height data and the second body width and height data to find the weighted value as the length and height information in the third width and height data of the body:

W _{phy_s} = W' _{phy_s} *α+W" _{phy_s} *β, where α∈(0,1) and β∈(0,1) are weights, α+β=1;

where δ∈(0,1) and

is the weight,

Generally, take α=0.5, β=0.5, δ=0.5,

That is, to find the average value.

Further, please refer to FIG. 7 and FIG. 8. As a preferred solution, in this embodiment, the axle information includes the number of axles and the type of axles;

The vehicle type identification data further includes: the vehicle front image, the vehicle body image, the vehicle rear image, and license plate type data. Specifically, after multi-data information fusion processing, the front view images of the complete front, body, and rear surfaces, as well as the corresponding three-dimensional images, can be obtained, and the length, width, and height of the vehicle, the number of wheel axles, and the vehicle Axle type, license plate type, license plate color, license plate number, vehicle shape and other information to achieve accurate vehicle identification.

Combining the information obtained in all the above steps, the multi-data information fusion processing method is used to obtain the front view images of the complete three faces of the front, body and rear, and the length, width and height of the vehicle, the number of vehicle axles, the vehicle axle type, License plate type, license plate color, license plate number, vehicle shape and other information to achieve accurate vehicle identification.

Further, as a preferred solution, in this embodiment, the step of acquiring the front image or the rear image includes:

Acquire the second image, use the fourth recognition model to identify the front or rear of the second image to obtain rough rectangular frame position information; the second image is an image with front or rear features; preferably, the first The four recognition models are obtained by training the initialized neural network model. The image with the features of the front or the rear of the car is captured by a preset first camera (such as camera 1 or camera 3). The first camera is installed on the side of the road and takes a certain angle (not 90° ), when a vehicle appears in the field of view of the camera, the photograph taken has the characteristics of the front or rear of the vehicle.

Image edge detection is performed on the rough rectangular frame to obtain a front image or a rear image. Using the neural network-based recognition model to identify the front or rear of the car can quickly determine whether there is a front or rear image in the second image, and quickly mark it while ensuring accuracy.

Based on the deep learning target detection method, the rough rectangular frame position information of the vehicle front or rear is obtained, combined with the image edge detection method to obtain the edge image of the front image, and then the precise left, right, up and down positions of the front are determined according to the edge information, and the precise positioning of the front is realized. , combined with the detection confidence and the precise position information of the front of the car to capture the complete image of the front of the car. The training process of the fourth recognition model includes:

Obtain the fourth training set, the fourth training set includes some fourth training images, the fourth training image is an image with the characteristics of the front or the rear of the vehicle, and the label of the front or the rear of the vehicle is marked;

The fourth recognition model is obtained by using the fourth training set to train the initialized neural network model.

Further, as a preferred solution, in this embodiment, the step of acquiring the vehicle body image includes:

Acquire continuously generated multiple frames of third images; the third images are images with vehicle body characteristics; preferably, the continuously generated multiple frames of third images may be obtained by screening a segment of video per unit time. The image with the characteristics of the vehicle body is captured by a preset second camera (such as camera 2), and the shooting direction of the second camera is perpendicular to the edge of the road. When a vehicle appears in the field of view, the photograph taken has the characteristics of the vehicle body .

Calculate the feature vector of the third image in each frame, and calculate the matching similarity with the third image in the previous frame respectively, and then obtain the corresponding offset;

The plurality of frames of the third image are stitched according to the translation transformation corresponding to the offset to obtain the vehicle body image. In this embodiment, whether it is a small car or a large car (especially a vehicle with an ultra-long or wide width), a complete vehicle image can be obtained.

Specifically in the implementation, please refer to FIG. 8. According to the third multi-frame image sequence formed by the vehicle body image sequence, it is detected whether the vehicle (body) appears in the camera field of view area 2, and if it is recorded as flag=1, the splicing starts.

Calculate the set of feature points of the third image of the current frame i

Where i is the serial number of the third image frame, j is the serial number of the feature point, N is the number of valid feature points, (x _j , y _j ) is the image position corresponding to the feature point, then the last frame is the third image of the i-1th frame The corresponding set of matching feature points is

According to the set of feature points of the third image of the i-th frame

and the set of feature points of the third image of i-1 in the previous frame

Calculate the matching similarity of the feature points, filter out the points that fail to match through the threshold, then calculate the average (x _i0 , y _i0 ) displacement of the matching feature pair, that is, the third image of the i-th frame and the third image of the i-1th frame The image body target is a translation transformation, its offset:

Where M is the number of effective feature pairs;

Combined with the actual situation of the vehicle passing through the position of the camera 2 area 2 normally, the vertical displacement y _i0 of the vehicle body image is approximately zero and can be ignored. Therefore, the i-th frame can be calculated according to the horizontal displacement x _i0 of the vehicle body image x i0 The mosaic diagram of the image and the body target in the third image of frame i-1 is shown in Fig. 8 .

Operate each frame of the third image in accordance with the aforementioned steps until the vehicle (body) disappears at position 2 of the field of view of the camera, that is, flag=0. After the last third image is processed, the stitching of the complete body image of the vehicle is completed. Enter the preparation stage for splicing of the next vehicle.

Please refer to FIG. 10 , the present disclosure also provides a vehicle type identification device, including:

Please refer to FIG. 11 , the present disclosure also provides a vehicle type identification system, including:

Multi-camera with multiple cameras with different orientations;

It can be understood that those skilled in the art can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present disclosure, and all these changes or replacements should fall within the protection scope of the appended claims of the present disclosure.

Industrial Applicability

The vehicle type identification method provided in the present disclosure can obtain the first proportional coefficient according to the size of the license plate image and the physical size of the license plate by acquiring the front image, the body image, and the rear image, and detecting the license plate, and then obtain the accurate first width of the front of the vehicle. Height data, the second width and height data of the rear, and the third width and height data of the body, and at the same time accurately identify the vehicle type and axle type, can accurately obtain the vehicle type identification data of passing vehicles, improve the identification accuracy, and has a strong industrial practicality.

Claims

A vehicle identification method, characterized in that, comprising:

Obtain the front image, body image and rear image of the vehicle;

Obtain the width, height and length data of the vehicle; the width, height and length data include the first width and height data, the second width and height data, and the third width and height data; wherein, license plate detection is performed on the front image and the rear image, Obtain the first proportional coefficient between the image size of the license plate and the physical size of the license plate, and then obtain the first width and height data of the front of the car and the second width and height data of the rear of the car respectively according to the image size of the front of the car and the image of the rear of the car; Or perform feature value extraction and matching between the rear image and the vehicle body image, and obtain the third width and height data of the vehicle body according to the first scale factor and the size of the vehicle body image;

Obtain vehicle model data; the vehicle model data includes vehicle model information and axle information; wherein, the vehicle head image is recognized by the first recognition model to obtain the vehicle model information; the vehicle body image is recognized by the second recognition model to obtain said axle information;

Integrating the width, height and length data and the vehicle type data to output vehicle type identification data.
The vehicle type identification method according to claim 1, wherein the step of acquiring the first width and height data or the second width and height comprises:

Perform license plate image recognition on the first image through a third recognition model to obtain license plate position data and license plate type data; the first image is the front image or the rear image;

adjusting the aspect ratio of the license plate image to be the same as the actual physical aspect ratio of the corresponding license plate type through affine transformation, and synchronously adjusting the aspect ratio of the first image based on the same adjustment ratio to obtain a first adjusted image;

Obtain the first scale factor between the image size of the license plate image and the physical size of the license plate;

The image size of the first adjusted image is acquired, and the first width and height data or the second width and height data are obtained based on the first proportional coefficient.
The vehicle type identification method according to claim 2, wherein obtaining the proportional coefficient specifically comprises:

Obtain the character image size of each character in the license plate image;

According to the physical size of the characters of the license plate, the first proportional coefficient is obtained.
The vehicle type identification method according to claim 2, wherein the step of obtaining the third width and height data is:

Obtain feature matching points of the first adjustment image and the body image, and then obtain a corresponding second scale factor;

performing normalization processing on the body image according to the second scale factor to obtain a first normalized image;

Acquire the image size of the first normalized image, and obtain third width and height data based on the first scale factor.
The vehicle type identification method according to claim 2, wherein the first adjustment image includes a front adjustment image and a rear adjustment image;

The steps for obtaining the third width and height data are:

Obtain the feature matching points of the adjusted image of the front of the car and the image of the body, and then obtain the corresponding third proportional coefficient;

performing normalization processing on the body image according to the third scale factor to obtain a second normalized image;

Obtain the feature matching points of the rear adjustment image and the body image, and then obtain the corresponding fourth proportional coefficient;

performing normalization processing on the body image according to the fourth scale factor to obtain a third normalized image;

Acquire the image size of the second normalized image, and obtain the first vehicle body width and height data based on the first scale factor;

Acquire the image size of the third normalized image, and obtain second vehicle body width and height data based on the first scale factor;

Weighted calculation is performed based on the first vehicle body width and height data and the second vehicle body width and height data to obtain the third width and height data.
The vehicle type identification method according to claim 2, wherein the axle information includes axle number and axle type;

The vehicle type identification data further includes: the vehicle front image, the vehicle body image, the vehicle rear image, and license plate type data.
The vehicle type identification method according to claim 1, wherein the acquiring step of the front image or the rear image comprises:

Acquire the second image, use the fourth recognition model to identify the front or rear of the second image, and obtain rough rectangular frame position information; the second image is an image with front or rear features;

Image edge detection is performed on the rough rectangular frame to obtain a front image or a rear image.
The vehicle type identification method according to claim 1, wherein the step of acquiring the vehicle body image comprises:

Acquire continuously generated multi-frame third images; the third images are images with vehicle body features;

Calculate the feature vector of the third image in each frame, and calculate the matching similarity with the third image in the previous frame respectively, and then obtain the corresponding offset;

The plurality of frames of the third image are stitched according to the translation transformation corresponding to the offset to obtain the vehicle body image.
A vehicle type identification device is characterized in that it comprises:

The obtaining module is used to obtain the front image, the body image and the rear image of the vehicle;

The processing module is used to obtain the width, height and length data of the vehicle; the width, height and length data include the first width and height data, the second width and height data, and the third width and height data; wherein, the image of the front of the vehicle and the rear of the vehicle Carry out license plate detection on the image to obtain the first proportional coefficient between the license plate image size and the physical size of the license plate, and then obtain the first width and height data of the front of the car and the second width and height data of the rear according to the image size of the front of the car and the image size of the rear of the car respectively; Extracting and matching the feature values of the front image and/or the rear image and the vehicle body image, obtaining the third width and height data of the vehicle body according to the first scale factor and the size of the vehicle body image; obtaining the vehicle type data; the vehicle type The data includes vehicle type information and axle information; wherein, the vehicle type information is obtained by recognizing the vehicle head image through the first recognition model; the vehicle body image is recognized through the second recognition model to obtain the axle information; the width is integrated The height data and the vehicle type data are used to output vehicle type identification data.
A vehicle identification system, characterized in that it comprises:

Multi-camera with multiple cameras with different orientations;

The vehicle type recognition device is connected with the multi-eye camera, and is used to receive the images transmitted by multiple cameras, and then obtain the front image, body image and rear image of the vehicle; and obtain the width, height and length data of the vehicle; the width, height and length The data includes the first width and height data, the second width and height data, and the third width and height data; wherein, the license plate detection is performed on the front image and the rear image to obtain the first distance between the image size of the license plate and the physical size of the license plate. Scale coefficient, and then obtain the first width and height data of the front of the car and the second width and height data of the rear according to the size of the front image and the size of the rear image respectively; perform feature value extraction on the front image and/or the rear image and the body image and matching, obtain the third width and height data of the vehicle body according to the first scale factor and the image size of the vehicle body; obtain the vehicle type data; the vehicle type data includes vehicle type information and axle information; The vehicle head image is recognized to obtain the vehicle type information; the vehicle body image is recognized by a second recognition model to obtain the axle information; the width, height and length data and the vehicle type data are integrated to output vehicle type identification data.