CN109726717B - A vehicle comprehensive information detection system - Google Patents
A vehicle comprehensive information detection system Download PDFInfo
- Publication number
- CN109726717B CN109726717B CN201910002648.5A CN201910002648A CN109726717B CN 109726717 B CN109726717 B CN 109726717B CN 201910002648 A CN201910002648 A CN 201910002648A CN 109726717 B CN109726717 B CN 109726717B
- Authority
- CN
- China
- Prior art keywords
- license plate
- vehicle
- detection
- image
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 63
- 238000013135 deep learning Methods 0.000 claims abstract description 9
- 238000000034 method Methods 0.000 claims description 48
- 230000011218 segmentation Effects 0.000 claims description 22
- 238000010586 diagram Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 12
- 238000013527 convolutional neural network Methods 0.000 claims description 8
- 230000000877 morphologic effect Effects 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000013461 design Methods 0.000 claims description 4
- 238000003708 edge detection Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000012549 training Methods 0.000 claims description 2
- 238000012937 correction Methods 0.000 description 7
- 229910052704 radon Inorganic materials 0.000 description 7
- SYUHGPGVQRZVTB-UHFFFAOYSA-N radon atom Chemical compound [Rn] SYUHGPGVQRZVTB-UHFFFAOYSA-N 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 230000009191 jumping Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
- Character Discrimination (AREA)
- Traffic Control Systems (AREA)
Abstract
一种车辆综合信息检测系统，可对车辆的车型型号以及车牌信息进行检测识别，具有较高的准确性和较快的检测速度。传统检测方法只能对车辆的车牌信息进行检测，无法得到车辆的整体信息，在实际应用中无法有效的解决套牌车、肇事逃逸等交通问题，同时传统检测方法只能解决较为简单场景下的车牌识别问题，在复杂的实际应用中不具有鲁棒性。而本发明结合了深度学习等理论，能快速、准确地实现车辆定位和车牌识别，可适用于各种环境下的检测对象，同时在车型识别的应用中也表现出了高效性和准确性。将该系统与现有交通系统结合可有效地解决各种交通问题，同时提高整个交通系统的车辆检测效率。
The utility model relates to a vehicle comprehensive information detection system, which can detect and identify the vehicle model type and license plate information, and has higher accuracy and faster detection speed. The traditional detection method can only detect the license plate information of the vehicle, and cannot obtain the overall information of the vehicle. In practical applications, it cannot effectively solve the traffic problems such as fake license plates and hit-and-run. At the same time, the traditional detection method can only solve the problems in relatively simple scenarios The license plate recognition problem is not robust in complex practical applications. The invention combines deep learning and other theories, can quickly and accurately realize vehicle positioning and license plate recognition, can be applied to detection objects in various environments, and also shows high efficiency and accuracy in the application of vehicle recognition. Combining this system with the existing traffic system can effectively solve various traffic problems while improving the vehicle detection efficiency of the entire traffic system.
Description
Technical Field
The invention designs a high-precision vehicle comprehensive information detection system based on a deep learning theory, which can be used in an intelligent traffic system to detect and query vehicle information.
Background
China has already stepped into the automobile society as early as 2012, and under the condition that the automobile keeping amount is continuously and rapidly increased, huge challenges are formed on the traffic, energy, ecological environment and the like of China, such as a series of problems of traffic jam, parking difficulty, traffic accidents and the like commonly existing in various big cities. The intelligent transportation system plays a role of a core pillar in the whole transportation management system, and is an accurate, real-time and efficient comprehensive transportation and management system which can play a vital role in a wide range and various aspects. The intelligent traffic system can effectively reduce traffic jam, save energy and reduce accident rate and tail gas emission. Therefore, the intelligent transportation system has a very important promotion effect on the development of China at present. The vehicle detection of the natural scene image is an important component of an intelligent traffic system, is also an important current research content, and has wide application prospects, such as highway charging, social security, hit-and-run, illegal lane occupation, purposeful vehicle accurate retrieval and the like. The real-time performance and the accuracy of vehicle detection are the precondition for the function of an intelligent traffic system, and the license plate is the same as the identity card of people in China for the automobile, has the function of unique identification marks, and can further mine the relevant information of the automobile and an automobile owner by combining with the license plate detection, so that the accurate and fast identification of the license plate information in a complex real scene is very important. Therefore, the license plate recognition technology is fused on the basis of vehicle positioning and vehicle type recognition, and a vehicle type recognition system is finally formed, so that more complex conditions can be processed, for example, the problems of fake-licensed vehicles, traffic diversion and the like can be rapidly solved. The vehicle detection realized based on the deep learning theory, the computer vision and the neural network provided by the invention can realize the on-line high-precision rapid vehicle detection only by being fused with the existing traffic video and monitoring system. Meanwhile, the system does not need to damage the road surface, is low in cost, can operate only by installing a camera and a corresponding data acquisition and transmission system at a traffic intersection, and can be used for other traffic supervision or big data general survey and other aspects.
Disclosure of Invention
The invention designs a vehicle comprehensive information detection system which can quickly and accurately obtain the comprehensive information of a vehicle by directly detecting the photo or video information containing the vehicle. The method is characterized in that:
1. a vehicle detection algorithm based on SSD target detection is adopted to realize vehicle positioning rapidly and accurately; and identifying and classifying the vehicle type by adopting a convolutional neural network structure inclusion-AB-Full. The method solves the two problems of judging whether the image contains the vehicle and positioning the vehicle, can realize accurate positioning of a plurality of vehicle targets with any size and position in the natural scene image, and has good robustness to noise such as illumination change and the like.
2. The license plate positioning method based on SSD target detection is adopted to position the license plate, the Radon algorithm is used to perform inclined correction on the license plate, and meanwhile, the noise background and the license plate frame caused by license plate correction are removed in the process of detecting the license plate, so that the license plate information is accurately positioned. In the process of identifying the license plate information, the SSD target detection method is still adopted to directly identify the whole license plate, and on the basis, the system is combined with the traditional method to identify the license plate and is suitable for different detection environments.
The invention combines the vehicle positioning function, the vehicle type recognition function and the license plate recognition function together to form a vehicle comprehensive information detection system. The invention can be combined in the existing traffic system, can effectively reduce the error information identification caused by human factors, and simultaneously improves the vehicle detection efficiency of the whole traffic system.
Drawings
FIG. 1 is a system image processing flow diagram, 2 is a network structure diagram of SSD _ VGG, 3 is a vehicle detection schematic diagram, 4 is an inclusion-AB-Net deep learning model, 5 is an inclusion-A structure diagram, 6 is an inclusion-B structure diagram, 7 is a license plate recognition flow diagram, 8 is a license plate detection network structure diagram, 9 is a frame, 10 is a license plate recognition network structure diagram, 11 is a license plate positioning algorithm flow diagram based on morphology and color features, 12 is a vertical projection convolution front and back broken line diagram, 13 is a vertical projection convolution front and back broken line diagram, 14 is a license plate character segmentation effect diagram, 15 is an inclusion-Small-Net part structure diagram, 16 is a vehicle type recognition system software interface structure diagram
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clear, the following description of the drawings describes the implementation of each function of the present invention by using three modules, namely, vehicle positioning, vehicle type recognition and license plate recognition. The general flow of the system is shown in fig. 1, and is specifically described as follows:
1 vehicle positioning
As shown in FIG. 2, the present invention constructs an SSD target detection network, denoted as SSD _ VGG, based on the convolutional neural network VGG-16 structure. Fig. 3 is a flow of identifying and locating a vehicle by the SSD _ VGG network, and when detecting the vehicle, the network performs feature extraction on a photo, and determines and locates a vehicle position from an image with a complex background.
2 vehicle type recognition
In vehicle type identification, the invention adopts an inclusion-AB-Net deep learning model which is constructed by taking a convolutional neural network inclusion-v 2 structure as a core, and a network structure is shown in FIG. 4. The inclusion-AB-Net is mainly composed of an inclusion-A network and an inclusion-B network. The convolution layer part is composed of 13 mixed layers, wherein the inclusion-A structure is shown in FIG. 5, and the step size of all convolution kernels in the inclusion-A is 1. The inclusion-B structure is shown in fig. 6 with step sizes of all convolution kernels of 2. Specific parameter configurations of inclusion-AB-Net are shown in table 1, where the convolution kernel of the first convolution layer of the first stage of the network model is 7 × 7, step size is 2, and the function is to remove the edge background of the vehicle image, and the convolution kernels of the remaining convolution blending layer and pooling layer are both 3 × 3 or 1 × 1. The convolution kernel size of the global pooling layer is 7 × 7, and the step size is 1, which aims to provide a one-dimensional feature vector for the classifier of the next stage. The feature extraction method can fully extract the features of the automobile image so as to accurately classify the subclasses of various automobile types.
The invention still uses the network structures of the inclusion-A and the inclusion-B on the basis of the inclusion-AB-Net network, increases the number of the inclusion-A and the inclusion-B convolution kernels under the condition of not changing the overall structure, further excavates the potential of the structure so as to obtain the inclusion-AB-Full model, and the specific parameter configuration is shown in the table 2. Compared with the Inception-AB-Net model, the Inception-AB-Full model has the advantages of faster and more stable convergence speed and higher accuracy.
3 license plate recognition
Fig. 7 is a license plate detection flow, and the license plate identification is realized by using an SSD-based target detection method in the present invention, and simultaneously combining with a conventional license plate identification method. The invention makes corresponding optimization improvement on the basis of the traditional method.
3.1 SSD-based target detection method
3.1.1 license plate location
The invention also uses the SSD _ VGG network to construct the license plate detection network. Because the image to be detected is positioned by vehicle detection, only one license plate or no license plate is generally available. Meanwhile, the size of the image to be detected is normalized to 300 x 300, and the size of the license plate is between 48 x 25 and 79 x 36, so that the original SSD network structure is modified and optimized according to the size range. Fig. 8 is a network structure after adjustment and optimization, and the adjustment is mainly to remove convolutional layers such as Conv9, Conv10, and Conv11 from the original SSD network structure, because default boxes predicted from feature maps of these layers are generally large, and are suitable for detecting large targets and not suitable for license plate detection tasks.
3.1.2 license plate correction
And the license plate is corrected by adopting a Radon transformation method, and the method not only can simultaneously obtain the inclination angles of the license plate in the horizontal direction and the vertical direction, but also has high calculation speed.
For Radon transform in general, an original function is subjected to a spatial transform, for example, points originally on an M-N plane are mapped onto a C-D plane, and all points originally on the M-N plane and belonging to the same straight line correspond to the same point on the C-D plane. It is only necessary to count the cumulative degree of each point on the C-D plane to determine whether there is a straight line on the M-N plane from this information. For an image, the Radon transform can be obtained by performing line integration along different straight lines in one plane. For a two-dimensional image f (x, y), the Radon transform form is shown as formula (1):
wherein
Radon(f(x-x',y-y'))＝Rf((λ-x'cosθ-y'sinθ),θ) (2)
Equation (2) is the translation of Radon, where the coordinate system x '-y' is transformed with the coordinate system x-y by the equation:
3.1.3 removing the background and frame of license plate
(1) The horizontal border is first removed. For a binary image of a license plate, if the value of a certain pixel point in a certain row is 0 (or 255) and the value of the next pixel point is 255 (or 0), the situation is called one-time jump, and since a general complete license plate has 7 characters, theoretically, each row of a license plate character region has at least 14 times of jumps, and the jump times of parts except the license plate characters are less. If the width of the license plate is width and the height of the license plate is height, scanning upwards from the height/2 of the binary image of the license plate by using the jumping information, if the jumping times of three continuous lines are all smaller than a set threshold value threshold, determining the image height corresponding to the line of which the jumping time of the first line is smaller than the threshold value as an upper boundary, and scanning downwards from the height/2 of the binary image of the license plate in the same way to obtain the lower boundary of the image of the license plate.
(2) Then, the background frames at the left and right sides are removed. Since the studied object is a license plate with blue background and white characters, the color characteristics of the license plate are utilized to remove the backgrounds on the two sides of the license plate. Converting the license plate RGB image with the horizontal frame removed into an HSV image, extracting a blue component in the HSV image to obtain a binary image of a blue bottom plate of the license plate, performing morphological operation to form a connected domain, performing outline detection, screening out a license plate region by using prior knowledge that the license plate area ratio, the rectangular shape and the aspect ratio of the license plate, the license plate character region necessarily contain white components and the like, scanning the screened region from left to right in a row, counting the number of pixels with the pixel value of each row being 255 (white), determining the row with the number of white pixels in the first row being less than threshold value threshold _2 as a left boundary of the license plate if three continuous rows are less than the threshold value threshold _2, and similarly scanning from right to left to obtain the right boundary of the license plate. Meanwhile, after the horizontal frame is removed, the height of the image is basically the height of the character, the image subjected to left and right boundary division can be verified by using the ratio of the width of the standard license plate to the character, and the license plate area is ensured to be complete after the left and right backgrounds are removed.
The effect of removing the border and the background of the license plate is shown in fig. 9.
3.1.4 license plate character detection
The license plate recognition method established based on SSD target detection can directly recognize the whole license plate, the network structure is shown in FIG. 10, the main detection process is similar to the previous vehicle detection and license plate detection, only the binary target detection is not needed at the moment, the marking of the training data is basically consistent with the method used in the license plate detection, and only the positions of all license plate characters in each license plate image are marked and the corresponding categories are filled.
3.2 conventional methods
3.2.1 license plate location
The invention provides a license plate positioning method based on morphology and HSV color characteristics on the basis of combining a traditional method, and FIG. 11 is a flow chart of the algorithm. Firstly, preprocessing an image and converting the image into a gray-scale image, aiming at reducing the data volume and facilitating the subsequent edge detection; then, using a sobel operator to carry out edge detection in the vertical direction, and then carrying out filtering and binarization processing on the obtained vertical edge image; and then, performing two continuous morphological closing operations on the image to enable the license plate region to form a connected domain, so that detailed screening can be performed subsequently according to the contour detection and the prior knowledge of the color, the length-width ratio and the like of the license plate.
The method adopts the contour detection function in the computer vision library Opencv to detect each contour in the connected domain of the binary image, and can calculate the area of each contour and the minimum rectangular frame wrapping the contour. Therefore, the detected outline is screened by using the prior information such as the length-width ratio of the license plate and the like to obtain a candidate area of the license plate. The number of the candidate areas is generally 1-5, and then coordinates of the candidate areas are used for confirming the original color image through HSV color feature one-to-one comparison, and finally the position of the license plate is determined.
3.2.2 license plate correction and license plate background and frame removal
The license plate correction and the license plate background removal method are the same as the frame correction and frame removal method.
3.2.3 character segmentation
The method is characterized in that the license plate characters are segmented by combining a projection method and a template matching method on the basis of a connected domain license plate character segmentation method, and the method specifically comprises the following steps:
(1) and searching the position of the segmentation point "·". As long as the segmentation points of the second character and the third character in the license plate are found, 5 characters can be segmented rightwards, and 2 characters can be segmented leftwards, so that the segmentation difficulty is reduced. Firstly, the license plate image is projected in the vertical direction, then the projection result is subjected to convolution calculation, the number values in convolution kernels are all 1, for the license plate image with the Width of Width, the size of the convolution kernel is 1 × N, and N is Width (34/440), wherein a parameter 34/440 is determined according to the interval between a second character and a third character in the standard license plate, as shown in fig. 11, a is the original binary license plate image, b is the vertical projection result, and c is the result after convolution. And (4) taking the abscissa X-left corresponding to the minimum value of the left half of the license plate in the graph c as the left boundary of the second character and the third character, and then taking the coordinate of the separator "·" as X-left + N/2.
(2) After the segmentation point is determined, the two characters on the left side of the segmentation point are segmented. And cutting the binary image of the original license plate according to the segmentation point to obtain a binary image of the first two characters, performing morphological opening operation on the binary image to avoid the two characters from being adhered together, forming a connected domain by the second English character, and determining the left boundary of the second English character by utilizing contour detection and a minimum rectangular frame containing the contour. If the contour detection fails, scanning the lines from right to left by using a projection method, if the number of white pixel points in a certain line is greater than a set threshold value threshold _3 for the first time, marking the white pixel points as the right boundary of the second character, and then if the number of the white pixel points in the certain line is less than the set threshold value threshold _3, marking the white pixel points as the left boundary of the second character. The effect is shown in fig. 13.
(3) A segmentation of the right 5 characters is performed. And performing morphological opening operation on the remaining 5 character areas, then using contour detection, recording the left and right boundaries of the areas to facilitate subsequent character sequencing if the contour meets the requirement, then cutting the image, judging whether the image needs to be segmented again according to the width of the remaining image, and repeating the previous process until 5 character blocks are finally obtained. If no contour meeting the requirement exists in contour detection, pre-segmentation is carried out according to the size and the interval of 5 characters behind the standard license plate in proportion, and then correction of the segmentation line is carried out according to continuous zero values or minimum values in the neighborhood (-omega, omega) of the pre-segmentation line by using a projection method.
Fig. 14 shows a character segmentation effect.
3.2.4 license plate character recognition
The work of character recognition is to process and analyze the segmented single character image to recognize the license plate number therein. Generally, the license plate characters obtained after segmentation are all small, so that the large-scale network structure adopted in the prior art is not applicable. The invention designs an inclusion-Small-Net network based on the structure of the inclusion-AB-Net, the table 3 shows the parameter configuration of the inclusion-Small-Net, and the size of input data received by the network is 28 multiplied by 28, so that the network is convenient for processing license plate characters.
The structure of Simple a, Simple B and their basic convolution units in table 3 is shown in fig. 15. Wherein the basic convolution unit Conv is composed of a convolution layer, a Batch Norm layer and a Relu activation layer; the Simple A is mainly used for extracting features and is formed by connecting a convolution kernel with the size of 1 multiplied by 1 and a convolution kernel with the size of 3 multiplied by 3 in parallel; simple B, which may be referred to as a downsampled layer, consists of a convolution kernel of size 3 x 3 in parallel with a Pooling of size 3 x 3.
TABLE 1 Inception-AB-Net model network parameters
TABLE 2 Inception-AB-Full model network parameters
TABLE 3 Inception-Small-Net parameter configuration
Claims (8)
1. The utility model provides a vehicle integrated information detecting system, has vehicle location, vehicle classification, three functions of license plate discernment, can detect the discernment according to image or video information to the vehicle model and the license plate information of vehicle which characterized in that:
a. adopting a vehicle detection algorithm based on deep learning to locate the position of the vehicle in the picture;
b. identifying and classifying the vehicle type by adopting a convolutional neural network structure inclusion-AB-Full;
the license plate is positioned and identified by any one of the following methods:
the method comprises the following steps:
c. positioning the license plate by adopting a license plate detection method based on deep learning;
d. recognizing the license plate by adopting a license plate recognition method based on deep learning;
the second method comprises the following steps:
e. providing a license plate positioning method based on morphology and HSV color characteristics;
f. on the basis of a connected domain-based license plate character segmentation method, a projection method and a template matching method are combined to segment license plate characters;
g. identifying the segmented license plate characters by utilizing an increment-Small-Net network in the traditional detection;
the Incep-AB-Full model is obtained by still using the network structures of the Incep-A and the Incep-B on the basis of the Incep-AB-Net network and increasing the number of convolution kernels of the Incep-A and the Incep-B under the condition of not changing the overall structure;
the convolution layer part consists of 13 mixed layers, the step length of all convolution kernels in the convolution-A is 1, and the step length of all convolution kernels in the convolution-B is 2; the convolution kernel of the first convolution layer in the first stage of the inclusion-AB-Net network model is 7 x 7, the step length is 2, the function of the convolution kernel is to remove the edge background of the vehicle image, the convolution kernels of the remaining convolution mixed layer and the pooling layer are both 3 x 3 or 1 x 1, the convolution kernel size of the global pooling layer is 7 x 7, and the step length is 1;
the network comprises an inclusion-Small-Net network, a base station, a network control unit and a base station, wherein the inclusion-Small-Net network is based on the inclusion-AB-Net structural design, the size of input data received by the network is 28 x 28, and the network comprises a Simple A, a Simple B and a basic convolution unit; wherein the basic convolution unit consists of a convolution layer, a Batch Norm layer and a Relu activation layer; the Simple A is used for extracting features and is formed by connecting a convolution kernel with the size of 1 multiplied by 1 and a convolution kernel with the size of 3 multiplied by 3 in parallel; simple B is called a downsampled layer and consists of a convolution kernel of 3 x 3 size in parallel with a Pooling of 3 x 3 size.
2. The vehicle integrated information detecting system according to claim 1, characterized in that: the position of the vehicle in the map is located by adopting a deep learning-based vehicle detection algorithm:
an SSD target detection network is constructed on the basis of a convolutional neural network VGG-16 structure, and the position of a vehicle can be judged and located from an image with a complex background by using the SSD target detection network to perform feature extraction on the picture.
3. The vehicle integrated information detecting system according to claim 1, characterized in that: the method for identifying and classifying the vehicle type by adopting the convolutional neural network structure inclusion-AB-Full comprises the following steps: the Incep-AB-Full network is constructed on the basis of the convolutional neural network Incep-AB-Net network.
4. The vehicle integrated information detecting system according to claim 2, characterized in that: the license plate detection method based on deep learning is adopted to position the license plate:
the license plate detection network is constructed on the basis of the SSD target detection network, Conv9, Conv10 and Conv11 convolution layers are removed from the original SSD target detection network structure, and because default boxes predicted from characteristic diagrams of the layers are generally large, the license plate detection network is suitable for detecting large targets and is not suitable for license plate detection tasks.
5. The vehicle integrated information detecting system according to claim 1, characterized in that: adopting a license plate recognition method based on deep learning to recognize the license plate:
the license plate identification is multi-classification target detection, and model training can be carried out only by marking the positions of all license plate characters in each license plate image and filling corresponding classes.
6. The vehicle integrated information detecting system according to claim 1, characterized in that:
the license plate positioning method based on morphology and HSV color characteristics comprises the following steps:
the method comprises the following steps: preprocessing an image, and converting the image into a gray scale image;
step two: using a sobel operator to carry out edge detection in the vertical direction;
step three: filtering and binarizing the obtained vertical edge image;
step four: performing two continuous morphological closing operations on the image to enable the license plate region to form a connected domain;
step five: detecting each contour in a connected domain of the binary image by adopting a contour detection function in a computer vision library Opencv, and screening the detected contour by utilizing aspect ratio prior information of the license plate to obtain a candidate region of the license plate;
step six: and confirming the original color image through HSV color characteristic one-to-one comparison by utilizing the coordinates of the candidate region, and finally determining the position of the license plate.
7. The vehicle integrated information detecting system according to claim 1, characterized in that:
on the basis of a connected domain-based license plate character segmentation method, a projection method and a template matching method are combined to segment license plate characters, and the method comprises the following steps:
the method comprises the following steps: projecting the license plate image in the vertical direction, performing convolution calculation on the projection result, and finding out the position of a segmentation point "·" by using the calculation result;
step two: cutting a binary image of an original license plate according to a segmentation point to obtain a binary image of the first two characters, performing morphological opening operation on the binary image to avoid the two characters from being adhered together, simultaneously enabling the second English character to form a connected domain, then determining a left boundary of the second English character by utilizing contour detection and a minimum rectangular frame containing the contour, scanning in a row from right to left by using a projection method if the contour detection fails, marking the right boundary of the second character if the number of white pixel points of a certain row is greater than a set threshold for the first time, and then marking the left boundary of the second character if the number of the white pixel points of the certain row is less than the set threshold;
step three: the method comprises the steps of segmenting 5 characters on the right side, detecting outlines of the remaining 5 character areas after morphological opening operation is carried out, recording left and right boundaries of the left and right characters to facilitate subsequent character sequencing if the outlines meet requirements, then cutting an image, judging whether segmentation needs to be carried out again according to the width of the remaining image, repeating the previous process until 5 character blocks are totally arranged at last, if the outlines do not meet the requirements in outline detection, pre-segmenting the left and right characters according to the size and the interval of the 5 characters behind a standard license plate in proportion, and then correcting a segmentation line according to continuous zero values or minimum values in a neighborhood (-omega, omega) of the pre-segmentation line by using a projection method.
8. The vehicle integrated information detecting system according to claim 1, characterized in that: in the traditional detection, an inclusion-Small-Net network is used for identifying the segmented license plate characters, and the inclusion-Small-Net network is designed based on a convolutional neural network inclusion-AB-Net structure to process the segmented license plate characters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910002648.5A CN109726717B (en) | 2019-01-02 | 2019-01-02 | A vehicle comprehensive information detection system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910002648.5A CN109726717B (en) | 2019-01-02 | 2019-01-02 | A vehicle comprehensive information detection system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109726717A CN109726717A (en) | 2019-05-07 |
| CN109726717B true CN109726717B (en) | 2022-03-01 |
Family
ID=66298021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910002648.5A Expired - Fee Related CN109726717B (en) | 2019-01-02 | 2019-01-02 | A vehicle comprehensive information detection system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109726717B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110334709B (en) * | 2019-07-09 | 2022-11-11 | 西北工业大学 | License plate detection method based on end-to-end multi-task deep learning |
| CN110503831A (en) * | 2019-08-28 | 2019-11-26 | 上海眼控科技股份有限公司 | A kind of method and apparatus identifying driver's illegal activities |
| CN110765861A (en) * | 2019-09-17 | 2020-02-07 | 中控智慧科技股份有限公司 | Unlicensed vehicle type identification method and device and terminal equipment |
| CN111027558A (en) * | 2019-10-23 | 2020-04-17 | 中电科新型智慧城市研究院有限公司 | A kind of illegal vehicle type and license plate recognition method |
| KR102814354B1 (en) * | 2019-12-05 | 2025-05-30 | 차나안 브라이트 사이트 컴퍼니 리미티드 | Character segmentation method, device and computer-readable storage medium |
| CN111860317A (en) * | 2020-07-20 | 2020-10-30 | 青岛特利尔环保集团股份有限公司 | Boiler operation data acquisition method, system, equipment and computer medium |
| CN112712012B (en) * | 2020-12-29 | 2024-09-13 | 中通服公众信息产业股份有限公司 | Road gate vehicle position detection method |
| CN113642412B (en) * | 2021-07-16 | 2023-12-26 | 盛视科技股份有限公司 | Method, device and equipment for detecting vehicle occupying bus lane |
| CN113869196B (en) * | 2021-09-27 | 2022-04-19 | 中远海运科技股份有限公司 | Vehicle type classification method and device based on laser point cloud data multi-feature analysis |
| CN114999167A (en) * | 2021-11-19 | 2022-09-02 | 深圳市智泊云科技有限公司 | High-definition license plate recognition system based on artificial intelligence |
| CN114999051B (en) * | 2022-06-16 | 2024-07-05 | 广州晟烨信息科技股份有限公司 | Security monitoring system for intelligent community platform |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105975941A (en) * | 2016-05-31 | 2016-09-28 | 电子科技大学 | Multidirectional vehicle model detection recognition system based on deep learning |
| CN106650553A (en) * | 2015-10-30 | 2017-05-10 | 比亚迪股份有限公司 | License plate recognition method and system |
| KR101888683B1 (en) * | 2017-07-28 | 2018-08-14 | 펜타시큐리티시스템 주식회사 | Method and apparatus for detecting anomaly traffic |
| CN108734189A (en) * | 2017-04-20 | 2018-11-02 | 天津工业大学 | Vehicle License Plate Recognition System based on atmospherical scattering model and deep learning under thick fog weather |
| CN108805121A (en) * | 2017-05-02 | 2018-11-13 | 优信拍（北京）信息科技有限公司 | A kind of car plate detection localization method, device, equipment and computer-readable medium |
| CN108921083A (en) * | 2018-06-28 | 2018-11-30 | 浙江工业大学 | Illegal flowing street pedlar recognition methods based on deep learning target detection |
-
2019
- 2019-01-02 CN CN201910002648.5A patent/CN109726717B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650553A (en) * | 2015-10-30 | 2017-05-10 | 比亚迪股份有限公司 | License plate recognition method and system |
| CN105975941A (en) * | 2016-05-31 | 2016-09-28 | 电子科技大学 | Multidirectional vehicle model detection recognition system based on deep learning |
| CN108734189A (en) * | 2017-04-20 | 2018-11-02 | 天津工业大学 | Vehicle License Plate Recognition System based on atmospherical scattering model and deep learning under thick fog weather |
| CN108805121A (en) * | 2017-05-02 | 2018-11-13 | 优信拍（北京）信息科技有限公司 | A kind of car plate detection localization method, device, equipment and computer-readable medium |
| KR101888683B1 (en) * | 2017-07-28 | 2018-08-14 | 펜타시큐리티시스템 주식회사 | Method and apparatus for detecting anomaly traffic |
| CN108921083A (en) * | 2018-06-28 | 2018-11-30 | 浙江工业大学 | Illegal flowing street pedlar recognition methods based on deep learning target detection |
Non-Patent Citations (3)
| Title |
|---|
| Christian Szegedy ; Vincent Vanhoucke ; Sergey Ioffe ; Jon Shlens.Rethinking the Inception Architecture for Computer Vision.《2016 IEEE Conference on Computer Vision and Pattern Recognition》.2016, * |
| Looking beyond appearances: Synthetic training data for deep CNNs in reidentification;Igor Barros Barbosa,Marco Cristani,Barbara Caputo,Aleksander Rog;《Computer Vision and Image Understanding》;20171212;50-62 * |
| 基于深度学习的车牌识别研究;江永爱;《中国优秀硕士论文全文数据库工程科技Ⅱ辑》;20180715;C034-380 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109726717A (en) | 2019-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109726717B (en) | A vehicle comprehensive information detection system | |
| Wei et al. | Multi-vehicle detection algorithm through combining Harr and HOG features | |
| CN101334836B (en) | License plate positioning method incorporating color, size and texture characteristic | |
| CN109886896B (en) | A blue license plate segmentation and correction method | |
| CN101872416B (en) | Vehicle license plate recognition method and system of road image | |
| CN105373794B (en) | A kind of licence plate recognition method | |
| CN103324930B (en) | A license plate character segmentation method based on gray histogram binarization | |
| CN103116751B (en) | A kind of Method of Automatic Recognition for Character of Lcecse Plate | |
| CN109190444B (en) | An implementation method of a video-based toll lane vehicle feature recognition system | |
| CN103136528B (en) | A kind of licence plate recognition method based on dual edge detection | |
| CN103824081B (en) | Method for detecting rapid robustness traffic signs on outdoor bad illumination condition | |
| CN110069986A (en) | A kind of traffic lights recognition methods and system based on mixed model | |
| CN106203433A (en) | In a kind of vehicle monitoring image, car plate position automatically extracts and the method for perspective correction | |
| CN107895492A (en) | A kind of express highway intelligent analysis method based on conventional video | |
| CN111860509B (en) | A two-stage method for accurate extraction of unconstrained license plate regions from coarse to fine | |
| CN105160309A (en) | Three-lane detection method based on image morphological segmentation and region growing | |
| CN105005989B (en) | A kind of vehicle target dividing method under weak contrast | |
| CN111695373B (en) | Zebra stripes positioning method, system, medium and equipment | |
| Zhang et al. | A multi-feature fusion based traffic light recognition algorithm for intelligent vehicles | |
| CN111259796A (en) | A Lane Line Detection Method Based on Image Geometric Features | |
| CN106709530A (en) | License plate recognition method based on video | |
| CN104598912A (en) | Traffic light detection and recognition method based CPU and GPU cooperative computing | |
| CN100414561C (en) | A License Plate Extraction Method Based on Projection and Mathematical Morphology | |
| CN112101108B (en) | Left-right pass-through sign recognition method based on graph pole position characteristics | |
| CN100385452C (en) | A license plate character segmentation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Xu Haiying Inventor after: Wan Min Inventor after: Bao Hailong Inventor after: Zhang Qiang Inventor after: Li Zhonglin Inventor after: Zeng Tao Inventor after: Binzechuan Inventor after: Ning Yuhan Inventor after: Chen Yunsheng Inventor before: Wan Min Inventor before: Bao Hailong Inventor before: Zhang Qiang Inventor before: Li Zhonglin Inventor before: Zeng Tao Inventor before: Binzechuan Inventor before: Ning Yuhan Inventor before: Chen Yunsheng |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220301 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |