CN114639078A - Vehicle type recognition method, device and system - Google Patents
Vehicle type recognition method, device and system Download PDFInfo
- Publication number
- CN114639078A CN114639078A CN202210147600.5A CN202210147600A CN114639078A CN 114639078 A CN114639078 A CN 114639078A CN 202210147600 A CN202210147600 A CN 202210147600A CN 114639078 A CN114639078 A CN 114639078A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- image
- width
- data
- height
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention relates to vehicle type recognition, in particular to a vehicle type recognition method, a vehicle type recognition device and a vehicle type recognition system. A vehicle type recognition method, comprising: acquiring a head image, a body image and a tail image of a vehicle; acquiring width, height and length data of a vehicle; the width, height and length data comprise first width, height and width data, second width and height data and third width and height data; acquiring vehicle type data of a vehicle; the vehicle type data comprises vehicle type information and axle information; and integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data. After a vehicle head image, a vehicle body image and a vehicle tail image are obtained, license plate detection is carried out, a first proportional coefficient is obtained according to the size of the license plate image and the physical size of the license plate, accurate first width and height data of the vehicle head, accurate second width and height data of the vehicle tail and accurate third width and height data of the vehicle body are obtained, meanwhile, accurate recognition is carried out on the vehicle type and the type of the vehicle axle, vehicle type recognition data passing through the vehicle can be accurately obtained, and recognition accuracy is improved.
Description
Technical Field
The invention relates to vehicle type recognition, in particular to a vehicle type recognition method, a vehicle type recognition device and a vehicle type recognition system.
Background
The information of the length, width and height of the vehicle, the number of wheel axles of the vehicle, the axle type of the vehicle, the type of the license plate, the color of the license plate, the license plate number, the appearance of the vehicle and the like is an important basis for classifying the types of the vehicle, and the type of the vehicle is used as important information of a motor vehicle and plays an important role in the applications of automatic driving of the vehicle, police criminal investigation case judgment, traffic police road management, road traffic charging and the like. Because the information items of the vehicle type classification bases are multiple and mutually complementary, and the single sensor devices such as a monocular camera, a binocular camera, weighing, laser scanning, infrared scanning and the like cannot comprehensively and accurately acquire related information, the high accuracy rate of the vehicle type classification on the identification precision is difficult to obtain.
The patent CN111783638A provides a system and a method for detecting the number of vehicle axles and vehicle type identification, where the system includes a distance measurement sensing device, a high-frequency parallel signal acquisition device, and a calculation processing device, and through the distance measurement sensing device, data sequences of a wheel depth map and a vehicle body depth map can be obtained, so as to realize vehicle axle number detection, and obtain a vehicle type identification result according to license plate information of the acquisition device, but the information items according to the vehicle axle number and the license plate information are only vehicle axle number and vehicle type identification, and are not enough to realize accurate vehicle type identification.
The patent CN111523579A proposes a vehicle type recognition method and system based on improved deep learning, the method is based on the deep learning method, a large number of vehicle image data sets at traffic gates are required to be cut, sorted, classified and trained, and classification and recognition of vehicle types are realized, only the shape information of the vehicle images is utilized, and the accuracy of vehicle type recognition is difficult to guarantee.
Therefore, the existing vehicle type recognition technology has defects and needs to be improved and enhanced.
Disclosure of Invention
In view of the defects of the prior art, the invention aims to provide a vehicle type identification method, a vehicle type identification device and a vehicle type identification system, which can improve the accuracy of length, width and height of vehicle identification, comprehensively identify vehicle types and comprehensively improve the accuracy of vehicle type identification.
In order to achieve the purpose, the invention adopts the following technical scheme:
in one aspect, the present invention provides a vehicle type identification method, including:
acquiring a head image, a body image and a tail image of a vehicle;
acquiring width, height and length data of a vehicle; the width, height and length data comprise first width, height and width data, second width and height data and third width and height data; the method comprises the steps that vehicle head images and vehicle tail images are subjected to license plate detection, a first proportion coefficient between the size of the license plate images and the physical size of the license plate is obtained, and then first width-height data of a vehicle head and second width-height data of a vehicle tail are obtained according to the size of the vehicle head images and the size of the vehicle tail images; extracting and matching feature values of the vehicle head image and/or the vehicle tail image with the vehicle body image, and obtaining third width and height data of the vehicle body according to the first scale coefficient and the vehicle body image size;
acquiring vehicle type data of a vehicle; the vehicle type data comprises vehicle type information and axle information; the vehicle head image is identified through a first identification model to obtain vehicle type information; identifying the vehicle body image through a second identification model to obtain the axle information;
and integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data.
Preferably, the step of acquiring the first width and height data or the second width and height data comprises:
performing license plate image recognition on the first image through a third recognition model to obtain license plate position data and license plate type data; the first image is the image of the vehicle head or the image of the vehicle tail;
adjusting the aspect ratio of the license plate image to be the same as the actual physical aspect ratio of the corresponding license plate type through affine transformation, and synchronously adjusting the aspect ratio of the first image based on the same adjustment ratio to obtain a first adjustment image;
acquiring a first proportional coefficient between the image size of the license plate image and the physical size of the license plate;
and acquiring the image size of the first adjustment image, and obtaining first width and height data or second width and height data based on the first scale coefficient.
Preferably, the obtaining of the scaling factor specifically includes:
acquiring the character image size of each character in the license plate image;
and obtaining the first scale coefficient according to the physical size of the characters of the license plate.
Preferably, the step of acquiring the third width and height data is:
acquiring feature matching points of the first adjustment image and the vehicle body image so as to obtain a corresponding second proportionality coefficient;
normalizing the vehicle body image according to the second proportionality coefficient to obtain a first normalized image;
and acquiring the image size of the first normalized image, and obtaining third width and height data based on the first scale coefficient.
Preferably, the first adjustment image comprises a vehicle head adjustment image and a vehicle tail adjustment image;
the third width and height data acquisition step comprises:
acquiring feature matching points of the vehicle head adjusting image and the vehicle body image so as to obtain a corresponding third proportionality coefficient;
normalizing the vehicle body image according to the third proportionality coefficient to obtain a second normalized image;
acquiring feature matching points of the vehicle tail adjustment image and the vehicle body image so as to obtain a corresponding fourth proportionality coefficient;
normalizing the vehicle body image according to the fourth scale coefficient to obtain a third normalized image;
acquiring the image size of the second normalized image, and obtaining first vehicle body width and height data based on the first proportional coefficient;
acquiring the image size of the third normalized image, and obtaining second body width and height data based on the first scale coefficient;
and performing weighting calculation based on the first body width and height data and the second body width and height data to obtain third width and height data.
Preferably, the axle information includes the number of axles and the type of axle;
the vehicle type identification data further includes: the vehicle head image, the vehicle body image, the vehicle tail image and the license plate type data.
Preferably, the step of acquiring the vehicle head image or the vehicle tail image comprises:
acquiring a second image, and identifying the head or tail of the vehicle on the second image by using a fourth identification model to obtain the position information of the rough rectangular frame; the second image is an image with a vehicle head characteristic or a vehicle tail characteristic;
and carrying out image edge detection on the rough rectangular frame to obtain a vehicle head image or a vehicle tail image.
Preferably, the step of acquiring the vehicle body image includes:
acquiring a plurality of continuously generated third images; the third image is an image with vehicle body characteristics;
calculating a feature vector of the third image of each frame, and calculating matching similarity of the third image of the previous frame respectively to obtain corresponding offset;
and carrying out translation transformation splicing on the plurality of frames of the third images according to the corresponding offset to obtain the vehicle body image.
In another aspect, the present invention provides a vehicle type recognition apparatus including:
the acquisition module is used for acquiring a head image, a body image and a tail image of the vehicle;
the processing module is used for acquiring width, height and length data of the vehicle; the width, height and length data comprise first width, height and width data, second width and height data and third width and height data; the method comprises the steps that vehicle head images and vehicle tail images are subjected to license plate detection, a first proportion coefficient between the size of the license plate images and the physical size of the license plate is obtained, and then first width-height data of a vehicle head and second width-height data of a vehicle tail are obtained according to the size of the vehicle head images and the size of the vehicle tail images; extracting and matching feature values of the vehicle head image and/or the vehicle tail image with the vehicle body image, and obtaining third width and height data of the vehicle body according to the first scale coefficient and the vehicle body image size; acquiring vehicle type data of a vehicle; the vehicle type data comprises vehicle type information and axle information; the vehicle head image is identified through a first identification model to obtain vehicle type information; identifying the vehicle body image through a second identification model to obtain the axle information; and integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data.
In another aspect, the present invention provides a vehicle type recognition system, including:
a multi-view camera having a plurality of cameras oriented differently;
the vehicle type recognition device is connected with the multi-view camera and is used for receiving the images transmitted by the cameras so as to obtain a head image, a body image and a tail image of the vehicle; acquiring width, height and length data of the vehicle; the width, height and length data comprise first width, height and width data, second width and height data and third width and height data; the method comprises the steps that vehicle head images and vehicle tail images are subjected to license plate detection, a first proportion coefficient between the size of the license plate images and the physical size of the license plate is obtained, and then first width-height data of a vehicle head and second width-height data of a vehicle tail are obtained according to the size of the vehicle head images and the size of the vehicle tail images; extracting and matching feature values of the vehicle head image and/or the vehicle tail image and the vehicle body image, and obtaining third width and height data of the vehicle body according to the first scale coefficient and the vehicle body image size; acquiring vehicle type data of a vehicle; the vehicle type data comprises vehicle type information and axle information; the vehicle head image is identified through a first identification model to obtain vehicle type information; identifying the vehicle body image through a second identification model to obtain the axle information; and integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data.
Compared with the prior art, the vehicle type identification method, the vehicle type identification device and the vehicle type identification system have the following beneficial effects:
by using the vehicle type identification method provided by the invention, after a vehicle head image, a vehicle body image and a vehicle tail image are obtained, the vehicle license plate detection is carried out, a first proportional coefficient is obtained according to the size of the vehicle license plate image and the physical size of the vehicle license plate, accurate first width and height data of the vehicle head, accurate second width and height data of the vehicle tail and accurate third width and height data of the vehicle body are obtained, meanwhile, the vehicle type identification data of the passing vehicle can be accurately obtained, and the identification accuracy is improved.
Drawings
Fig. 1 is a flowchart of a vehicle type recognition method provided by the present invention.
Fig. 2 is a schematic view of the installation of the multi-view camera provided by the invention.
Fig. 3 is a flow chart of the steps for acquiring width and height data of the vehicle head or the vehicle tail provided by the invention.
FIG. 4 is a flowchart of the steps for obtaining the first scaling factor according to another embodiment of the present invention.
Fig. 5 is a flow chart of a method for acquiring the third aspect data according to the present invention.
Fig. 6 is a flow chart of another method for acquiring the third width and height data according to the present invention.
FIG. 7 is a flow chart of the vehicle body image acquisition steps provided by the present invention.
FIG. 8 is a schematic view of the present invention providing for the acquisition of an image of a vehicle body.
Fig. 9 is a flowchart of an embodiment of a vehicle type recognition method provided by the present invention.
Fig. 10 is a block diagram showing the structure of a vehicle type recognition apparatus according to the present invention.
Fig. 11 is a block diagram showing a structure of a vehicle type recognition system according to the present invention.
Detailed Description
In order to make the objects, technical solutions and effects of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It is to be understood by those skilled in the art that the foregoing general description and the following detailed description are exemplary and explanatory of specific embodiments of the invention, and are not intended to limit the invention.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of steps does not include only those steps, but may include other steps not expressly listed or inherent to such process or method. Also, without further limitation, one or more devices or subsystems, elements or structures or components beginning with "comprise. The appearances of the phrases "in one embodiment," "in another embodiment," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
Referring to fig. 1, 2 and 9, the present invention provides a vehicle type recognition method, which is applied to a vehicle type recognition device connected to a multi-view camera. In this embodiment, according to an actual application scenario, the multi-view camera is installed at a position 1 or a position 2 (as shown in fig. 2) beside a road, the multi-view camera at least has a plurality of cameras, and each camera faces to a different direction to respectively acquire image data of the road in the different direction.
In a further preferred embodiment, the number of the cameras is 3, and the respective facing directions of the facing roads are a facing road (acquiring the second image for generating the image of the vehicle body), a vehicle coming direction (acquiring the image for generating the image of the vehicle head), and a vehicle departing direction (acquiring the image for generating the image of the vehicle tail). In a further preferred scheme, the number of the cameras exceeds 3, and the number of the cameras in each direction is one or more, so that the corresponding head images, tail images and body images of the same vehicle can be generated according to the preferred image data. The requirements for camera installation are as shown in the figure: 1) the field angles between the cameras 1 and 2 and between the cameras 2 and 3 are overlapped to a certain degree; 2) the angle of each camera can be adjusted up and down, left and right; 3) when a vehicle runs through the camera set, the camera 1 can completely acquire images of the head of the vehicle, the camera 2 can completely acquire images of the height of the whole body of the vehicle, the image pictures are horizontally parallel to the road surface as much as possible, and the camera 3 can completely acquire images of the tail of the vehicle, so that the characteristic point matching between the images can be conveniently carried out.
Further, the working process among the multiple cameras of the multi-view camera is preferably as follows: when a vehicle enters the area 1, the camera 1 acquires a vehicle head image sequence, the processing module detects and identifies a vehicle head image according to the image sequence, and synchronizes an image acquisition signal to the camera 2; when a vehicle enters the area 2, the camera 2 collects a vehicle body image sequence, and simultaneously synchronizes image collection signals to the camera 3; when a vehicle enters the area 3, the camera 3 collects a vehicle tail image sequence, so that all image sequences of the vehicle passing through the area 1-the area 3 are obtained, a vehicle head image, a vehicle body image and a vehicle tail image are further obtained, the camera sequentially obtains the images, and electric energy, storage resources, calculation resources and the like can be effectively saved.
The vehicle type identification method comprises the following steps:
s1, acquiring a head image, a body image and a tail image of the vehicle; in this embodiment, the images of the vehicle head, the images of the vehicle body, and the images of the vehicle tail are generated by capturing image sequences with cameras, and certainly, when some cameras having an automatic recognition function and capable of acquiring a target image for cleaning are used, the images transmitted by the cameras can be directly used.
S2, acquiring width, height and length data of the vehicle; the width, height and length data comprise first width, height and width data, second width and height data and third width and height data; the method comprises the steps that vehicle head images and vehicle tail images are subjected to license plate detection, a first proportion coefficient between the size of the license plate images and the physical size of the license plate is obtained, and then first width-height data of a vehicle head and second width-height data of a vehicle tail are obtained according to the size of the vehicle head images and the size of the vehicle tail images; extracting and matching feature values of the vehicle head image and/or the vehicle tail image and the vehicle body image, and obtaining third width and height data of the vehicle body according to the first scale coefficient and the vehicle body image size; in this embodiment, the first scaling factor includes a first width scaling factor and a first height scaling factor, which are respectively calculated simply, for example, the size of the license plate image is 400 × 150 pixels, the physical size of the license plate is 440mm × 140mm, the first width scaling factor is 0.91 pixels/mm, the first height scaling factor is 1.07 pixels/mm, and then after the size of the front image or the size of the rear image is obtained, the first width and height data and the second width and height data are calculated quickly, and similarly, the third width and height data can be calculated.
S3, obtaining vehicle type data of the vehicle; the vehicle type data comprises vehicle type information and axle information; the vehicle head image is identified through a first identification model to obtain vehicle type information; identifying the vehicle body image through a second identification model to obtain the axle information; specifically, the first recognition model and the second recognition model are both obtained based on a deep neural network model.
Specifically, the training process of the first recognition model includes:
acquiring a first training set, wherein the first training set comprises a plurality of first training images, the first training images are images with vehicle type characteristics, and labels of various vehicle types are marked; the vehicle types include large cars, trailer cars, police cars, and the like.
And training the initialized neural network model by using a first training set to obtain the first recognition model.
The training process of the second recognition model comprises the following steps:
acquiring a second training set, wherein the second training set comprises a plurality of second training images, the second training images are images with axle characteristics, and labels of various axle types are labeled;
and training the initialized neural network model by using a second training set to obtain the second recognition model.
And S4, integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data. Specifically, the data to be identified is integrated in the data integrating process, so that accurate identification of the vehicle type information of the vehicle is obtained.
By using the vehicle type identification method provided by the invention, after a vehicle head image, a vehicle body image and a vehicle tail image are obtained, the vehicle license plate detection is carried out, a first proportional coefficient is obtained according to the size of the vehicle license plate image and the physical size of the vehicle license plate, accurate first width and height data of the vehicle head, accurate second width and height data of the vehicle tail and accurate third width and height data of the vehicle body are obtained, meanwhile, the vehicle type identification data of the passing vehicle can be accurately obtained, and the identification accuracy is improved.
Further, referring to fig. 3, as a preferred solution, in this embodiment, the step of obtaining the first width and height data or the second width and height data includes:
s21, carrying out license plate image recognition on the first image through a third recognition model to obtain license plate position data and license plate type data; the first image is the image of the vehicle head or the image of the vehicle tail; specifically, the third recognition model is obtained based on neural network model training, the first image can be automatically recognized, the corresponding recognition result can be quickly obtained, and meanwhile, the accuracy is guaranteed.
Specifically, the license plate types include: 1. large car number-front: 440mm × 140 mm-rear: 440mm x 220 mm-yellow black frame line; 2. the number plate of the trailer is-440 mm multiplied by 220 mm-yellow bottom black frame line; 3. the small car license plate is-440 mm multiplied by 140 mm-blue bottom white frame line; 4. the license plate of the Shichan automobile is-440 mm multiplied by 140 mm-white with black background, and the white frame lines of the words of red messenger and collar; 5. the license plate of the front-end museum automobile is-440 mm multiplied by 140 mm-white with black bottom, and the white frame lines of the red lead and the collar lead are marked; 6. the number plate of the vehicles entering and leaving the border of hong Kong and Macao is-440 mm multiplied by 140 mm-white characters with black background, white frame lines of Chinese character ' hong Kong ' and Macao '; 7. coach car license plate-440 mm x 140 mm-yellow bottom black character, black learning character black frame line; 8. police car license plate-440 mm x 140 mm-white bottom black character, red alarm character black frame line.
The training process of the third recognition model comprises the following steps:
acquiring a third training set, wherein the third training set comprises a plurality of third training images, the third training images are images with license plate characteristics, and labels of various license plate types are labeled;
and training the initialized neural network model by using a third training set to obtain the third recognition model.
S22, adjusting the aspect ratio of the license plate image to be the same as the actual physical aspect ratio of the corresponding license plate type through affine transformation, and synchronously adjusting the aspect ratio of the first image based on the same adjustment ratio to obtain a first adjustment image;
s23, acquiring a first proportional coefficient between the image size of the license plate image and the physical size of the license plate;
and S24, acquiring the image size of the first adjustment image, and obtaining first width and height data or second width and height data based on the first scale coefficient.
The embodiment mainly performs detection, positioning, correction, type classification, character segmentation and recognition processing on the vehicle head image or the vehicle tail image.
In a further embodiment, after obtaining the license plate position data, the 4 vertex positions of the rectangular frame of the license plate are obtained by a target detection and feature point extraction method, meanwhile, the third recognition model can also obtain the license plate color and the license plate type data, then a license plate image consistent with the actual physical license plate aspect ratio is obtained through affine transformation by combining the license plate type information, then the accurate rectangular frame position of each character is obtained by combining a semantic segmentation method, and finally the license plate number information is recognized, so that the basic information of the license plate, including the number, the color, the type and the like, can be rapidly obtained.
Further, referring to fig. 4, as a preferred scheme, in this embodiment, the acquiring the scaling factor specifically includes:
acquiring the character image size of each character in the license plate image;
and obtaining the first scale coefficient according to the physical size of the characters of the license plate. That is, the first scale factor is generated using the physical size of each character and the image size, which is more accurate, so that it is more accurate in calculating the width and height data of the head or the tail of the vehicle. The method and the device realize accurate estimation of the physical size of the characters of the license plate, thereby accurately estimating the physical size of the height, the width and the length of the vehicle.
In the specific implementation process, after affine transformation, firstly obtaining transformation parameters M (M is a 3x3 matrix), and obtaining a front image of a locomotive by an affine transformation method by combining with accurate locomotive position information to obtain an image size W of the locomotive imageimg_h*Himg_h。
Obtaining the accurate rectangular frame position of each character, and obtaining the image size W of each character by the affine transformation method through the transformation parameter Mimg_h_ch_i*Himg_h_ch_i(where i is the character number).
According to the image size of the character and the imaging condition of each character, eliminating the character '1' or other data with overlarge or undersize character images caused by dirt and the like by a clustering analysis method, and then calculating to obtain the average image size of the character:
Physical dimension W of single character combined with latest industry standard of motor vehicle number plate of people's republic of Chinaphy_ch*Hphy_hAnd calculating to obtain the scaling coefficients of the physical size of the character and the image size (namely a first scaling coefficient, wherein the first scaling coefficient comprises a first width scaling coefficient and a first height scaling coefficient)：
A first width proportionality coefficient of
The first height proportionality coefficient is
By combining the equal-scale imaging characteristic of the camera image, the physical width and height of the vehicle head (namely the first width and height data of the vehicle head) can be calculated:
in a further embodiment, before the first scale coefficient is obtained, distortion correction is performed on the first image, so that the image is closer to the actual shape of the vehicle head.
Further, referring to fig. 5, as a preferred embodiment, in the present embodiment, the step of acquiring the third width and height data includes:
acquiring feature matching points of the first adjustment image and the vehicle body image so as to obtain a corresponding second proportionality coefficient; the first adjusting image is a first adjusting image of a head image or a first adjusting image of a tail image.
Normalizing the vehicle body image according to the second proportionality coefficient to obtain a first normalized image;
and acquiring the image size of the first normalized image, and obtaining third width and height data based on the first scale coefficient. In this embodiment, only the front image or the rear image is used to perform normalization adjustment on the vehicle body image, so that the image size of the vehicle body image is adaptively adjusted with the adjusted front image or rear image, and a normalized image adapted to the first adjusted image is obtained, thereby further improving the width and height calculation accuracy of the vehicle body.
Further, please refer to fig. 6, as a preferred scheme, in this embodiment, the first adjustment image includes a vehicle head adjustment image and a vehicle tail adjustment image;
the third width and height data acquisition step comprises:
acquiring feature matching points of the vehicle head adjusting image and the vehicle body image so as to obtain a corresponding third proportionality coefficient;
normalizing the vehicle body image according to the third proportionality coefficient to obtain a second normalized image;
acquiring feature matching points of the vehicle tail adjustment image and the vehicle body image so as to obtain a corresponding fourth proportionality coefficient;
normalizing the vehicle body image according to the fourth scale coefficient to obtain a third normalized image;
acquiring the image size of the second normalized image, and obtaining first vehicle body width and height data based on the first proportional coefficient;
acquiring the image size of the third normalized image, and obtaining second body width and height data based on the first scale coefficient;
and performing weighting calculation based on the first body width and height data and the second body width and height data to obtain third width and height data. In this embodiment, after the feature point matching is performed on the vehicle body image and the vehicle head adjustment image or the vehicle tail adjustment image, a three-dimensional image of the vehicle can be obtained. And meanwhile, the vehicle head image and the vehicle tail image are used for matching the vehicle body image, so that the calculation accuracy of the third width and height data of the vehicle body can be effectively improved.
In specific implementation, through a complete image of the front of a vehicle head and a vehicle body image, the characteristic matching points of the two images are calculated, and a third proportionality coefficient f for height conversion of the two images is solved1：
calculating a second normalized image according to the third proportional coefficient to obtain the normalized length W of the complete image of the side surface of the vehicle bodyimg0_sAnd high Himg0_s：
Wimg0_s＝f1*Wimg_sWherein W isimg_sAs the length of the image of the vehicle body,
Himg0_s＝f1*Himg_sin which H isimg_sFor height of vehicle body image
According to the first scale coefficient and the camera image equal-scale imaging characteristic, the length and height information of the first vehicle width and height data of the vehicle body can be estimated:
similarly, the length W of the second body width and height data of the vehicle body can be obtained by using the feature point matching of the vehicle tail adjustment image and the vehicle body image and the vehicle tail height information "phy_sAnd a height H'phy_sInformation;
in order to estimate the third width and height data of the vehicle body more accurately, a weighting value is calculated using the first width and height data and the second width and height data as length and height information in the third width and height data of the vehicle body:
Wphy_s＝W'phy_s*α+W"phy_sβ, where α ∈ (0,1) and β ∈ (0,1) are weights, α + β ═ 1;
in general, when α is 0.5, β is 0.5, δ is 0.5,
i.e. averaging.
Further, please refer to fig. 7 and 8, as a preferred scheme, in this embodiment, the axle information includes the number of axles and the type of axle;
the vehicle type identification data further includes: the vehicle head image, the vehicle body image, the vehicle tail image and the license plate type data. Specifically, after multi-data information fusion processing is carried out, front view images of 3 surfaces of a complete vehicle head, a complete vehicle body and a complete vehicle tail and corresponding three-dimensional stereo images can be obtained, and information such as the length, the width and the height of a vehicle, the number of vehicle axles, the vehicle axle type, the vehicle plate color, the vehicle license plate number, the vehicle appearance and the like is obtained through fusion calculation, so that accurate vehicle type recognition is realized.
And (3) integrating the information obtained in all the steps, acquiring front view images of 3 surfaces of the complete vehicle head, the complete vehicle body and the complete vehicle tail by adopting a multi-data information fusion processing method, and fusing and calculating the information such as the length, the width and the height of the vehicle, the number of vehicle wheel shafts, the vehicle shaft type, the license plate color, the license plate number, the vehicle appearance and the like to realize accurate vehicle type identification.
Further, as a preferred scheme, in this embodiment, the acquiring step of the vehicle head image or the vehicle tail image includes:
acquiring a second image, and identifying the head or tail of the vehicle on the second image by using a fourth identification model to obtain the position information of the rough rectangular frame; the second image is an image with a vehicle head characteristic or a vehicle tail characteristic; preferably, the fourth recognition model is obtained by training an initialized neural network model. The image with the head feature or the tail feature is obtained by shooting through a preset first camera (such as the camera 1 or the camera 3), the first camera is arranged on one side of the road and forms a certain angle (not 90 degrees) with the edge of the road, and when a vehicle appears in the visual field of the camera, the shot picture has the head feature or the tail feature.
And carrying out image edge detection on the rough rectangular frame to obtain a vehicle head image or a vehicle tail image. The recognition model based on the neural network is used for recognizing the vehicle head or the vehicle tail, whether the image of the vehicle head or the vehicle tail exists in the second image can be rapidly determined, the labeling is rapidly carried out, and meanwhile, the accuracy is guaranteed.
The method comprises the steps of obtaining position information of a rough rectangular frame of a vehicle head or a vehicle tail based on a deep learning target detection method, obtaining edge images of a vehicle head image by combining an image edge detection method, determining left, right, upper and lower accurate positions of the vehicle head according to the edge information, achieving accurate positioning of the vehicle head, and capturing a complete image of the vehicle head by combining detection confidence and accurate position information of the vehicle head.
The training process of the fourth recognition model comprises the following steps:
acquiring a fourth training set, wherein the fourth training set comprises a plurality of fourth training images, and the fourth training images are images with the characteristics of the vehicle head or the vehicle tail and are labeled with labels of the vehicle head or the vehicle tail;
and training the initialized neural network model by using a fourth training set to obtain the fourth recognition model.
Further, as a preferable aspect, in this embodiment, the step of acquiring the vehicle body image includes:
acquiring a plurality of continuously generated third images; the third image is an image with vehicle body characteristics; preferably, the continuously generated frames of third images may be obtained by screening a video segment in a unit time. The image with the vehicle body characteristics is obtained by shooting through a preset second camera (such as the camera 2), the shooting direction of the second camera is perpendicular to the road edge, and when a vehicle appears in the field of view, the shot picture has the vehicle body characteristics.
Calculating a feature vector of the third image of each frame, and respectively calculating matching similarity with the third image of the previous frame to obtain corresponding offset;
and carrying out translation transformation splicing on the plurality of frames of the third images according to the corresponding offset to obtain the vehicle body image. In the embodiment, a complete vehicle image can be acquired no matter whether the vehicle is a small vehicle or a large vehicle (particularly, a vehicle with an ultra-long and ultra-wide vehicle).
In an implementation, please refer to fig. 8, whether a vehicle (vehicle body) appears at the position of the camera view field range area 2 is detected according to a plurality of frames of third images formed by a vehicle body image sequence, if yes, the flag is 1, and the stitching starts.
Calculating the feature point set of the current ith frame third image
Where i is the third image frame number, j is the feature point number, N is the number of valid feature points, (x)j,yj) The image position corresponding to the feature point is the matching feature point set corresponding to the third image of the previous frame, i.e. the i-1 th frame
Feature point set of third image according to ith frame
And the feature point set of the previous frame i-1 third image
Calculating the matching similarity of the feature points, filtering out the points failing to be matched through a threshold value, and respectively calculating the average (x) of the matched feature pairsi0,yi0) Displacement, namely the body object of the third image of the ith frame and the third image of the ith-1 frame is subjected to translation transformation, and the displacement is as follows:
combined with normal running of vehicleThe actual position of the area 2 passing through the camera 2, the vertical displacement y of the vehicle body imagei0Is approximately zero and can be ignored, therefore, the horizontal displacement x can be determined according to the vehicle body imagei0And calculating a splicing map of the third image of the ith frame and the third image of the ith-1 frame of the vehicle body object, as shown in FIG. 8.
And (3) operating each frame of third image according to the steps until the position of the field range area 2 of the camera for the disappearance of the vehicle (vehicle body), namely the flag is 0, completing the splicing of the complete vehicle body images of the vehicle after the last third image is processed, and entering the splicing preparation stage of the next vehicle.
Referring to fig. 10, the present invention further provides a vehicle type recognition apparatus, including:
the acquisition module is used for acquiring a head image, a body image and a tail image of the vehicle;
the processing module is used for acquiring width, height and length data of the vehicle; the width, height and length data comprise first width and height data, second width and height data and third width and height data; the method comprises the steps that vehicle head images and vehicle tail images are subjected to license plate detection, a first proportion coefficient between the size of the license plate images and the physical size of the license plate is obtained, and then first width-height data of a vehicle head and second width-height data of a vehicle tail are obtained according to the size of the vehicle head images and the size of the vehicle tail images; extracting and matching feature values of the vehicle head image and/or the vehicle tail image with the vehicle body image, and obtaining third width and height data of the vehicle body according to the first scale coefficient and the vehicle body image size; acquiring vehicle type data of a vehicle; the vehicle type data comprises vehicle type information and axle information; the vehicle head image is identified through a first identification model to obtain vehicle type information; identifying the vehicle body image through a second identification model to obtain the axle information; and integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data.
Referring to fig. 11, the present invention further provides a vehicle type recognition system, including:
a multi-view camera having a plurality of cameras oriented differently;
the vehicle type recognition device is connected with the multi-view camera and used for receiving the images transmitted by the cameras so as to obtain a head image, a body image and a tail image of the vehicle; acquiring width, height and length data of the vehicle; the width, height and length data comprise first width, height and width data, second width and height data and third width and height data; the method comprises the steps that vehicle head images and vehicle tail images are subjected to license plate detection, a first proportion coefficient between the size of the license plate images and the physical size of the license plate is obtained, and then first width-height data of a vehicle head and second width-height data of a vehicle tail are obtained according to the size of the vehicle head images and the size of the vehicle tail images; extracting and matching feature values of the vehicle head image and/or the vehicle tail image with the vehicle body image, and obtaining third width and height data of the vehicle body according to the first scale coefficient and the vehicle body image size; acquiring vehicle type data of a vehicle; the vehicle type data comprises vehicle type information and axle information; the vehicle head image is identified through a first identification model to obtain vehicle type information; identifying the vehicle body image through a second identification model to obtain the axle information; and integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data.
It should be understood that equivalents and modifications of the technical solution and inventive concept thereof may occur to those skilled in the art, and all such modifications and alterations should fall within the scope of the appended claims.
Claims (10)
1. A vehicle type recognition method is characterized by comprising the following steps:
acquiring a head image, a body image and a tail image of a vehicle;
acquiring width, height and length data of a vehicle; the width, height and length data comprise first width, height and width data, second width and height data and third width and height data; the method comprises the steps that vehicle head images and vehicle tail images are subjected to license plate detection, a first proportion coefficient between the size of the license plate images and the physical size of the license plate is obtained, and then first width-height data of a vehicle head and second width-height data of a vehicle tail are obtained according to the size of the vehicle head images and the size of the vehicle tail images; extracting and matching feature values of the vehicle head image and/or the vehicle tail image with the vehicle body image, and obtaining third width and height data of the vehicle body according to the first scale coefficient and the vehicle body image size;
acquiring vehicle type data of a vehicle; the vehicle type data comprises vehicle type information and axle information; the vehicle head image is identified through a first identification model to obtain vehicle type information; identifying the vehicle body image through a second identification model to obtain the axle information;
and integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data.
2. The vehicle type recognition method according to claim 1, wherein the obtaining of the first width and height data or the second width and height data includes:
performing license plate image recognition on the first image through a third recognition model to obtain license plate position data and license plate type data; the first image is the image of the vehicle head or the image of the vehicle tail;
adjusting the aspect ratio of the license plate image to be the same as the actual physical aspect ratio of the corresponding license plate type through affine transformation, and synchronously adjusting the aspect ratio of the first image based on the same adjustment ratio to obtain a first adjustment image;
acquiring a first proportional coefficient between the image size of the license plate image and the physical size of the license plate;
and acquiring the image size of the first adjustment image, and obtaining first width and height data or second width and height data based on the first scale coefficient.
3. The vehicle type identification method according to claim 2, wherein the obtaining of the scaling factor specifically includes:
acquiring the character image size of each character in the license plate image;
and obtaining the first scale coefficient according to the physical size of the characters of the license plate.
4. The vehicle type recognition method according to claim 2, wherein the third width and height data is obtained by:
acquiring feature matching points of the first adjustment image and the vehicle body image so as to obtain a corresponding second proportionality coefficient;
normalizing the vehicle body image according to the second proportionality coefficient to obtain a first normalized image;
and acquiring the image size of the first normalized image, and obtaining third width and height data based on the first scale coefficient.
5. The vehicle type recognition method according to claim 2, wherein the first adjustment image includes a nose adjustment image and a tail adjustment image;
the third width and height data acquisition step comprises:
acquiring feature matching points of the vehicle head adjusting image and the vehicle body image so as to obtain a corresponding third proportionality coefficient;
normalizing the vehicle body image according to the third proportionality coefficient to obtain a second normalized image;
acquiring feature matching points of the vehicle tail adjustment image and the vehicle body image so as to obtain a corresponding fourth proportionality coefficient;
normalizing the vehicle body image according to the fourth scale coefficient to obtain a third normalized image;
acquiring the image size of the second normalized image, and obtaining first vehicle body width and height data based on the first proportional coefficient;
acquiring the image size of the third normalized image, and obtaining second body width and height data based on the first scale coefficient;
and performing weighting calculation based on the first body width and height data and the second body width and height data to obtain third width and height data.
6. The vehicle type recognition method according to claim 2, wherein the axle information includes the number of axles, the type of axle;
the vehicle type identification data further includes: the vehicle head image, the vehicle body image, the vehicle tail image and the license plate type data.
7. The vehicle type identification method according to claim 1, wherein the vehicle head image or the vehicle tail image acquiring step includes:
acquiring a second image, and identifying the head or tail of the vehicle on the second image by using a fourth identification model to obtain the position information of the rough rectangular frame; the second image is an image with a vehicle head characteristic or a vehicle tail characteristic;
and carrying out image edge detection on the rough rectangular frame to obtain a vehicle head image or a vehicle tail image.
8. The vehicle type recognition method according to claim 1, wherein the step of acquiring the vehicle body image includes:
acquiring a plurality of continuously generated third images; the third image is an image with vehicle body characteristics;
calculating a feature vector of the third image of each frame, and calculating matching similarity of the third image of the previous frame respectively to obtain corresponding offset;
and carrying out translation transformation splicing on the plurality of frames of the third images according to the corresponding offset to obtain the vehicle body image.
9. A vehicle type recognition apparatus characterized by comprising:
the acquisition module is used for acquiring a head image, a body image and a tail image of the vehicle;
the processing module is used for acquiring width, height and length data of the vehicle; the width, height and length data comprise first width, height and width data, second width and height data and third width and height data; the method comprises the steps that vehicle head images and vehicle tail images are subjected to license plate detection, a first proportion coefficient between the size of the license plate images and the physical size of the license plate is obtained, and then first width-height data of a vehicle head and second width-height data of a vehicle tail are obtained according to the size of the vehicle head images and the size of the vehicle tail images; extracting and matching feature values of the vehicle head image and/or the vehicle tail image with the vehicle body image, and obtaining third width and height data of the vehicle body according to the first scale coefficient and the vehicle body image size; acquiring vehicle type data of a vehicle; the vehicle type data comprises vehicle type information and axle information; the vehicle head image is identified through a first identification model to obtain vehicle type information; identifying the vehicle body image through a second identification model to obtain the axle information; and integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data.
10. A vehicle type recognition system, characterized by comprising:
a multi-view camera having a plurality of cameras oriented differently;
the vehicle type recognition device is connected with the multi-view camera and used for receiving the images transmitted by the cameras so as to obtain a head image, a body image and a tail image of the vehicle; acquiring width, height and length data of the vehicle; the width, height and length data comprise first width and height data, second width and height data and third width and height data; the method comprises the steps that vehicle head images and vehicle tail images are subjected to license plate detection, a first proportion coefficient between the size of the license plate images and the physical size of the license plate is obtained, and then first width-height data of a vehicle head and second width-height data of a vehicle tail are obtained according to the size of the vehicle head images and the size of the vehicle tail images; extracting and matching feature values of the vehicle head image and/or the vehicle tail image with the vehicle body image, and obtaining third width and height data of the vehicle body according to the first scale coefficient and the vehicle body image size; acquiring vehicle type data of a vehicle; the vehicle type data comprises vehicle type information and axle information; the vehicle head image is identified through a first identification model to obtain vehicle type information; identifying the vehicle body image through a second identification model to obtain the axle information; and integrating the width, height and length data and the vehicle type data and outputting vehicle type identification data.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210147600.5A CN114639078A (en) | 2022-02-17 | 2022-02-17 | Vehicle type recognition method, device and system |
| PCT/CN2022/128972 WO2023155483A1 (en) | 2022-02-17 | 2022-11-01 | Vehicle type identification method, device, and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210147600.5A CN114639078A (en) | 2022-02-17 | 2022-02-17 | Vehicle type recognition method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114639078A true CN114639078A (en) | 2022-06-17 |
Family
ID=81945756
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210147600.5A Pending CN114639078A (en) | 2022-02-17 | 2022-02-17 | Vehicle type recognition method, device and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114639078A (en) |
| WO (1) | WO2023155483A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023155483A1 (en) * | 2022-02-17 | 2023-08-24 | 广州广电运通金融电子股份有限公司 | Vehicle type identification method, device, and system |
| CN116740662A (en) * | 2023-08-15 | 2023-09-12 | 贵州中南锦天科技有限责任公司 | Axle recognition method and system based on laser radar |
| CN117553695A (en) * | 2024-01-11 | 2024-02-13 | 摩斯智联科技有限公司 | Method, apparatus and computer storage medium for calculating vehicle height based on aspect ratio suppression noise |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117087790A (en) * | 2023-10-19 | 2023-11-21 | 江西江铃集团晶马汽车有限公司 | Full-automatic bus door installation method and system |
| CN117351439B (en) * | 2023-12-06 | 2024-02-20 | 山东博安智能科技股份有限公司 | Dynamic monitoring management system for intelligent expressway overrun vehicle |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107545239B (en) * | 2017-07-06 | 2021-01-12 | 南京理工大学 | Fake plate detection method based on license plate recognition and vehicle characteristic matching |
| US10286908B1 (en) * | 2018-11-01 | 2019-05-14 | Eric John Wengreen | Self-driving vehicle systems and methods |
| CN112652173B (en) * | 2019-10-11 | 2022-05-03 | 深圳富泰宏精密工业有限公司 | Driving safety prompting method, vehicle and storage medium |
| CN111582255A (en) * | 2020-06-19 | 2020-08-25 | 上海眼控科技股份有限公司 | Vehicle overrun detection method and device, computer equipment and storage medium |
| CN112966582A (en) * | 2021-02-26 | 2021-06-15 | 北京卓视智通科技有限责任公司 | Vehicle type three-dimensional recognition method, device and system, electronic equipment and storage medium |
| CN114639078A (en) * | 2022-02-17 | 2022-06-17 | 广州广电运通金融电子股份有限公司 | Vehicle type recognition method, device and system |
-
2022
- 2022-02-17 CN CN202210147600.5A patent/CN114639078A/en active Pending
- 2022-11-01 WO PCT/CN2022/128972 patent/WO2023155483A1/en unknown
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023155483A1 (en) * | 2022-02-17 | 2023-08-24 | 广州广电运通金融电子股份有限公司 | Vehicle type identification method, device, and system |
| CN116740662A (en) * | 2023-08-15 | 2023-09-12 | 贵州中南锦天科技有限责任公司 | Axle recognition method and system based on laser radar |
| CN116740662B (en) * | 2023-08-15 | 2023-11-21 | 贵州中南锦天科技有限责任公司 | Axle recognition method and system based on laser radar |
| CN117553695A (en) * | 2024-01-11 | 2024-02-13 | 摩斯智联科技有限公司 | Method, apparatus and computer storage medium for calculating vehicle height based on aspect ratio suppression noise |
| CN117553695B (en) * | 2024-01-11 | 2024-05-03 | 摩斯智联科技有限公司 | Method and device for calculating vehicle height and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023155483A1 (en) | 2023-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114639078A (en) | Vehicle type recognition method, device and system | |
| CN106919915B (en) | Map road marking and road quality acquisition device and method based on ADAS system | |
| US7366325B2 (en) | Moving object detection using low illumination depth capable computer vision | |
| CN109190523B (en) | Vehicle detection tracking early warning method based on vision | |
| CN102765365B (en) | Pedestrian detection method based on machine vision and pedestrian anti-collision warning system based on machine vision | |
| CN104143098B (en) | Night pedestrian recognition method based on far-infrared camera head | |
| CN111369541A (en) | Vehicle detection method for intelligent automobile under severe weather condition | |
| CN115717894B (en) | Vehicle high-precision positioning method based on GPS and common navigation map | |
| CN111160220B (en) | Deep learning-based parcel detection method and device and storage medium | |
| CN108189757B (en) | Driving safety prompt system | |
| CN114495064A (en) | Monocular depth estimation-based vehicle surrounding obstacle early warning method | |
| CN110717445A (en) | Front vehicle distance tracking system and method for automatic driving | |
| CN105512641B (en) | A method of dynamic pedestrian and vehicle under calibration sleet state in video | |
| CN110675442B (en) | Local stereo matching method and system combined with target recognition technology | |
| CN109740486B (en) | Method and system for identifying number of human beings contained in image | |
| JP2002197588A (en) | Method for discriminating tire type of traveling vehicle and method and device for discriminating vehicle model | |
| Rahaman et al. | Lane detection for autonomous vehicle management: PHT approach | |
| Yun et al. | Video-based detection and analysis of driver distraction and inattention | |
| CN110765877B (en) | Pedestrian detection method and system based on thermal imager and binocular camera | |
| CN102722724B (en) | Vehicle-mounted night view system having target identification function and target identification method thereof | |
| CN110414357B (en) | Front vehicle positioning method based on vehicle type identification | |
| CN109190591A (en) | A kind of front truck identification prior-warning device and identification method for early warning based on camera | |
| WO2014050285A1 (en) | Stereo camera device | |
| Damian et al. | Using mono and stereo camera system for static and moving objects detection | |
| CN113781578A (en) | Gate passing behavior identification and control method combining target detection and binocular vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |