CN116977949A - Vehicle parking inspection method, device and equipment - Google Patents
Vehicle parking inspection method, device and equipment Download PDFInfo
- Publication number
- CN116977949A CN116977949A CN202311076318.3A CN202311076318A CN116977949A CN 116977949 A CN116977949 A CN 116977949A CN 202311076318 A CN202311076318 A CN 202311076318A CN 116977949 A CN116977949 A CN 116977949A
- Authority
- CN
- China
- Prior art keywords
- license plate
- inspection
- vehicle
- image
- plate number
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 153
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012545 processing Methods 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000012216 screening Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 11
- 238000012549 training Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 4
- 230000009467 reduction Effects 0.000 description 3
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 229910052744 lithium Inorganic materials 0.000 description 2
- 230000006641 stabilisation Effects 0.000 description 2
- 238000011105 stabilization Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Traffic Control Systems (AREA)
Abstract
The present disclosure provides a vehicle parking inspection method, device and equipment, including: acquiring a patrol video; aiming at each frame of inspection image in the inspection video, extracting a target image of the current inspection vehicle from the inspection image, and identifying the initial license plate number of the current inspection vehicle and the confidence of each character in the initial license plate number based on the target image; and storing a plurality of initial license plates which are identified in the continuous inspection image frames and have the similarity larger than a preset threshold value and the confidence coefficient of each character in each initial license plate number into a statistics queue, and determining the license plate number of the current inspection vehicle based on the statistics queue. Because the license plate number of the current patrol vehicle is comprehensively determined based on the continuous image frames in the patrol video, the license plate number of the parked vehicle which is currently patrol can be accurately extracted.
Description
Technical Field
The disclosure relates to the technical field of image processing, and in particular relates to a vehicle parking inspection method, device and equipment.
Background
With the continuous rise of the vehicle holding quantity, the demand for parking spaces is increasingly increased, and the parking pressure is continuously increased. In order to standardize the parking order, the parking behavior is better managed, the video of the vehicle in the parking area is generally acquired by adopting a camera device, then the acquired video is analyzed and processed to obtain the license plate number and the parking state of the parked vehicle, and finally the parking management is performed according to the license plate number and the parking state of the parked vehicle. However, it is difficult to accurately extract the license plate number of the currently-inspected parked vehicle by using the current video analysis and processing method, thereby affecting the subsequent parking management efficiency.
Disclosure of Invention
In view of the above, the present disclosure provides a method, an apparatus and a device for parking and inspecting a vehicle, which can accurately extract the license plate number of the currently inspected parked vehicle.
According to a first aspect of the present disclosure, there is provided a vehicle parking inspection method comprising:
acquiring a patrol video;
extracting a target image of a current patrol vehicle from each frame of patrol image in the patrol video, and identifying an initial license plate number of the current patrol vehicle and confidence of each character in the initial license plate number based on the target image;
and storing a plurality of initial license plates which are identified in the continuous inspection image frames and have the similarity larger than a preset threshold value and the confidence coefficient of each character in each initial license plate number into a statistics queue, and determining the license plate number of the current inspection vehicle based on the statistics queue.
In one possible implementation, the inspection video is generated by a parking inspection system;
the parking inspection system when generating the inspection video comprises the following steps:
at least two initial videos are respectively acquired by adopting at least one camera;
and splicing the image frames in at least two initial videos to obtain the inspection video.
In one possible implementation manner, after obtaining the inspection video, the parking inspection system further includes: and carrying out image preprocessing on each image frame in the inspection video.
In one possible implementation, when the target image of the current patrol vehicle is extracted from the patrol image, the method is implemented based on a pre-built vehicle identification model.
In one possible implementation manner, when identifying the initial license plate number of the current patrol vehicle and the confidence of each character in the initial license plate number based on the target image, the method includes:
based on the target image, acquiring a license plate image of the current patrol vehicle by adopting a pre-constructed license plate recognition model;
based on the license plate image, acquiring each character image in the license plate image by adopting a pre-constructed character segmentation model;
based on each character image, acquiring an initial license plate number of the current patrol vehicle and the confidence coefficient of each character in the initial license plate number by adopting a pre-constructed character recognition model.
In one possible implementation, when determining the license plate number of the current patrol vehicle based on the statistical queue, the confidence and mode of each character in the statistical queue are implemented based.
In one possible implementation, when determining the license plate number of the current patrol vehicle based on the confidence and mode of each character in the statistics queue, the method includes:
traversing each character forming each initial license plate number in the statistic queue;
aiming at the traversed current character, screening out the characters with the highest confidence coefficient and the set number from the initial license plates as preferred characters, and calculating the mode of the preferred characters;
and after the traversal is finished, determining the license plate number of the current patrol vehicle based on each mode.
In one possible implementation, the method further includes:
and acquiring the parking state of the current patrol vehicle by adopting a pre-built parking state identification model based on the target image, wherein the parking state comprises at least one of normal parking, line-crossing parking and cross-position parking.
According to a second aspect of the present disclosure, there is provided a vehicle parking inspection device comprising:
the video acquisition module is used for acquiring the inspection video;
the image frame processing module is used for extracting a target image of a current patrol vehicle from each frame of patrol image in the patrol video, and identifying an initial license plate number of the current patrol vehicle and the confidence coefficient of each character in the initial license plate number based on the target image;
the license plate number recognition module is used for storing a plurality of initial license plates which are recognized in the continuous inspection image frame and have similarity larger than a preset threshold value and the confidence coefficient of each character in each initial license plate number into the statistics queue, and determining the license plate number of the current inspection vehicle based on the statistics queue.
According to a third aspect of the present disclosure, there is provided a vehicle parking inspection apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the method of the first aspect of the present disclosure.
The present disclosure provides a vehicle parking inspection method, comprising: acquiring a patrol video; aiming at each frame of inspection image in the inspection video, extracting a target image of the current inspection vehicle from the inspection image, and identifying the initial license plate number of the current inspection vehicle and the confidence of each character in the initial license plate number based on the target image; and storing a plurality of initial license plates which are identified in the continuous inspection image frames and have the similarity larger than a preset threshold value and the confidence coefficient of each character in each initial license plate number into a statistics queue, and determining the license plate number of the current inspection vehicle based on the statistics queue. Because the license plate number of the current patrol vehicle is comprehensively determined based on the continuous image frames in the patrol video, the license plate number of the parked vehicle which is currently patrol can be accurately extracted.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 illustrates a flow chart of a vehicle parking inspection method in accordance with an embodiment of the present disclosure;
FIG. 2 illustrates a system frame diagram of a parking inspection system in accordance with an embodiment of the present disclosure;
FIG. 3 illustrates a hardware layout of a parking inspection system according to an embodiment of the present disclosure;
FIG. 4 shows a schematic block diagram of a vehicle parking inspection device in accordance with an embodiment of the present disclosure;
fig. 5 shows a schematic block diagram of a vehicle parking lot inspection device according to an embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
In addition, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.
< method example >
Fig. 1 shows a flowchart of a vehicle parking inspection method performed by a cloud server according to an embodiment of the present disclosure. As shown in fig. 1, the method includes steps S1100-S1300.
S1100, acquiring a patrol video. Specifically, in the process of parking and inspecting vehicles, video acquisition is performed on each vehicle parked in a parking area through a parking and inspecting system, and the video acquired through the parking and inspecting system is an inspection video. After the parking inspection system acquires the inspection video, the inspection video is sent to the cloud server, and therefore the cloud server can acquire the inspection video.
In one possible implementation, the parking inspection system is shown in fig. 2, and includes a power management system 110, an embedded terminal motherboard 120, an ethernet switch 130, a 5G ethernet module 140, a camera module 150, and a cloud server 160. The power management system 110 is configured to provide an operating power for the parking inspection system. The camera module 150 is configured to perform video acquisition on each vehicle parked in the parking area, and send the acquired video to the embedded terminal motherboard 120 through the ethernet switch 130. After acquiring the video sent by the camera module, the embedded terminal motherboard 120 processes the acquired video into a patrol video, and sends the patrol video to the cloud server 160 through the ethernet switch 130 and the 5G ethernet module 140.
In this embodiment, the power management system 110 is composed of a 12v lithium battery, a battery protection and voltage stabilization power supply module and a power supply charging interface, and is used for performing voltage reduction processing on a power supply to meet the power supply requirement of equipment, so that the current equipment power can be displayed, and the system is powered when no external power supply exists. The capacity of the lithium battery is not lower than 8000mah, the battery protection and voltage stabilization power supply module provides input charging current not lower than 1.5A, supports power supply output current not lower than 6A and supports 12 v-30 v wide voltage input. The power interface adopts 5521 type circular DC interface.
The embedded terminal motherboard 120 may employ a high-performance ARM architecture master control, and an integrated hardware acceleration platform for accelerating video data processing. The embedded terminal motherboard 120 is further configured with multiple interfaces such as a gigabit ethernet port, a USB interface, a TF card interface, and the like, and has rich expansion capability.
The Ethernet switch 130 provides no less than 5 10/100/1000Mbps adaptive RJ45 ports, supports Auto-rollover (Auto-MDI/MDIX), conforms to the IEEE 802.3, IEEE 802.3u and IEEE 802.3ab standards, employs a non-blocking switching architecture, and forwards and filters data packets at full line speeds.
The 5G Ethernet module 140 satisfies SRRC/NAL/CCC forced authentication, supports SA and NSA networking, supports various operators, and has a maximum downlink bandwidth of 2Gbps and a maximum uplink bandwidth of 1Gbps of 5G SA Sub-6.
In one possible implementation, the camera module 150 includes at least two cameras, each of which is circumferentially distributed. In this way, when the video capturing is performed on the vehicle located in the parking area, at least two cameras may be used to capture at least two initial videos respectively, and the two initial videos may be sent to the embedded terminal motherboard 120. After at least two initial videos are acquired, the embedded terminal motherboard 120 splices image frames in the at least two initial videos, so as to obtain the inspection video. When the image frames in at least two initial videos are spliced, the image frames collected at the same time in the at least two initial videos are spliced to obtain spliced image frames with wider visual field range. Specifically, image frames collected at the same time in each initial video can be spliced into a panoramic picture; image frames acquired at the same time in each initial video may be spliced in an n×n window, which is not specifically limited herein. When the image frames collected at the same time in each initial video are spliced in an N x N window, the method specifically comprises the following steps: after the window is established, the picture of each camera occupies one frame of the window, and then the picture of the whole window is output, so that the collected image frames of the multiple cameras are combined into one frame of inspection image frame. In this embodiment, since each frame of inspection image in the inspection video is obtained by splicing at least two initial image frames with different visual field ranges, each frame of inspection image has a larger visual field range, so that the accuracy of the subsequent license plate number identification can be improved.
Example 1, referring to the first layer in fig. 3, the camera module 150 includes three cameras, where the three cameras are circumferentially distributed (i.e., the included angle between the three cameras is 120 °), for each moment in the inspection process: respectively adopting 3 cameras to acquire image frames to obtain 3-frame inspection images; transmitting the acquired 3-frame inspection image to the embedded terminal motherboard 120; after acquiring 3 frames of inspection images, the embedded terminal motherboard 120 splices the three frames of inspection images in a 2 x 2 window (for example, splices the three frames of images in three areas of the upper left, the upper right and the lower left of the window respectively) so as to generate an inspection image frame at the current moment, thereby overcoming the blind area of the visual field of a single camera, repeating the above operation, and combining the inspection image frames in sequence to obtain the inspection frequency. The size of the whole window is 2 times of the single frame size, for example, the single frame size is 1280×960, and the actual window size is 2560×1920.
Example 2, referring to fig. 3, the camera module 150 includes three cameras, where the three cameras are circumferentially distributed (i.e., the included angle between the three cameras is 120 °), for each moment in the inspection process: respectively adopting 3 cameras to acquire image frames to obtain 3-frame inspection images; transmitting the acquired 3-frame inspection image to the embedded terminal motherboard 120; after the embedded terminal motherboard 120 acquires 3 frames of inspection images, the three frames of inspection images are spliced into a panoramic image, so that the vision blind area of a single camera is overcome, the above operation is repeated, and the inspection images are sequentially combined to obtain the inspection video.
In one possible implementation, after the patrol video is generated, the embedded terminal motherboard 120 further includes an operation of performing image preprocessing on each frame of the patrol image in the patrol video. The image preprocessing operation may include at least one of noise reduction processing and contrast enhancement processing. Specifically, the median filtering algorithm or the Gaussian filtering algorithm is adopted to carry out noise reduction treatment on each frame of inspection image, and then the histogram equalization algorithm or the self-adaptive histogram equalization algorithm is adopted to carry out contrast enhancement treatment on each frame of inspection image after denoising, so that the identifiability of license plates in each frame of inspection image is improved.
In one possible implementation manner, after performing an image preprocessing operation on each frame of the inspection image in the inspection video, the embedded terminal motherboard 120 compresses the inspection video, and pushes the compressed inspection video to the cloud server in real time through the ethernet switch 130 and the 5G ethernet module 140. In this embodiment, after receiving the compressed inspection video, the cloud server decompresses the compressed inspection video to obtain the inspection video.
S1200, extracting a target image of the current patrol vehicle from the patrol image aiming at each frame of the patrol image in the patrol video, and identifying the initial license plate number of the current patrol vehicle and the confidence of each character in the initial license plate number based on the target image. The current inspection vehicle is a vehicle which is positioned on the right side of the parking inspection system in the inspection process and is nearest to the parking inspection system.
It should be noted that, in the parking area, a plurality of parked vehicles are often included, and thus, when video capturing is performed for the current patrol vehicle, other vehicles will inevitably be captured. In order to improve accuracy of identification of the license plate number of the current patrol vehicle, when each frame of patrol image in the patrol video is processed, a target image of the current patrol vehicle is identified from the patrol image, and then subsequent license plate identification operation is carried out based on the target image of the current patrol vehicle.
In one possible implementation, the method is implemented based on a pre-built vehicle identification model when the target image of the current patrol vehicle is extracted from the patrol image.
The vehicle identification model needs to be built before the target image of the current patrol vehicle is extracted from the patrol image based on the pre-built vehicle identification model. The vehicle identification model may be constructed by the steps of: firstly, a parking inspection system is adopted to collect videos of vehicle parking conditions in a plurality of parking areas. And extracting each image frame from the acquired video, labeling a boundary frame of the current inspection vehicle, the confidence of the boundary frame and the probability of the current inspection vehicle in the boundary frame for each image frame, and taking each labeled image frame as first training data of a vehicle identification model. And finally, training the first YOLO model based on the first training data and a preset first loss function to obtain a vehicle identification model. Specifically, for each first training data, it is input into a first YOLO model that will make predictions of the bounding box of the current patrol vehicle, the bounding box confidence, and the probability of the current patrol vehicle within the bounding box. And inputting the predicted data into a preset first loss function, and calculating a loss value. Based on the calculated loss values, back propagation is employed and the network parameters of the first YOLO model are continuously optimized in a gradient descent manner. And after training is finished, obtaining the vehicle identification model. The preset first loss function may be as follows:
wherein:
: is super-parameter and is used for increasing the importance degree of the accuracy of the object center coordinate prediction.
: for the summation operation, all grids are traversed.
: for sum operation, a bounding box for all predictions in each grid is traversed.
: to indicate a function, the bounding box j in grid i has a value of 1 if and only if it contains an object.
: is the error in the x and y directions of the predicted and actual bounding box centers.
: bounding box for predictionErrors in width and height from the actual bounding box width and height. In order to make the coordinate error of the smaller bounding box equally important as the coordinate error of the larger bounding box, square root is used.
: representing that the predicted bounding box contains a confidence level for the current patrol vehicle; />Indicating that the actual bounding box contains the confidence level of the current patrol vehicle.
: is a super parameter for reducing the degree of importance of the boundary box prediction that does not include the current inspection vehicle.
: to indicate a function, the bounding box j in grid i has a value of 1 if and only if it does not contain the current patrol vehicle.
: to indicate a function, the value of grid i is 1 if and only if it contains the current patrol vehicle.
: a conditional probability representing a predicted class c; />The conditional probability of the actual class C is represented, wherein class C is the current patrol vehicle.
After the vehicle identification model is obtained, the vehicle identification model can be adopted to extract the target image of the current patrol vehicle from the patrol image. Specifically, after the inspection image is input to the vehicle recognition model, the vehicle recognition model divides the input/output inspection image into a network of s×s, preferably, S takes a value of 7, that is, the input/output inspection image is divided into a grid of 7*7, and when there is a center of the object falling into the grid, the boundary box of the object, the confidence coefficient C of the boundary box, and the probability P of the object in the boundary box belonging to each category are predicted. For each predicted bounding box, calculating the product of the confidence coefficient C of the bounding box and the probability P that the object class in the bounding box is the vehicle, and taking the calculated product as the comprehensive confidence coefficient of the bounding box. And extracting an image in the boundary box with the highest comprehensive confidence as a target image of the current inspection vehicle.
Further, after the target image of the current patrol vehicle is extracted from the patrol image, the initial license plate number of the current patrol vehicle and the confidence of each character in the initial license plate number can be identified based on the target image of the current patrol vehicle. Specifically, the method may include the steps of:
first, based on a target image of a current patrol vehicle, a license plate image of the current patrol vehicle is obtained by adopting a pre-constructed license plate recognition model.
Before this step is performed, a license plate recognition model needs to be built. The specific construction process comprises the following steps: firstly, a large number of images containing license plates are obtained, boundary frames of the license plates in the images, confidence of the boundary frames and probability of the license plates in the boundary frames are marked, and the marked images are used as second training data of a license plate recognition model. And finally, training the second YOLO model based on the second training data and a preset second loss function to obtain the license plate recognition model. Specifically, for each second training data, it is input into a second YOLO model that will make predictions of the bounding box of the license plate, the bounding box confidence, and the probability of being a license plate within the bounding box. And inputting the predicted data into a preset second loss function, and calculating a loss value. Based on the calculated loss values, back propagation is employed and the network parameters of the second YOLO model are continuously optimized in a gradient descent manner. And obtaining the license plate recognition model after training is finished. The predetermined second loss function may be the same as the first loss function, which is not described herein.
After the license plate recognition model is obtained, a license plate image of the current patrol vehicle can be obtained by adopting a pre-constructed license plate recognition model based on the target image of the current patrol vehicle. Specifically, after a target image of a current inspection vehicle is input into a license plate recognition model, the license plate recognition model divides the input target image into a network of s×s, and predicts a bounding box of an object, a bounding box confidence C and a probability P that the object in the bounding box belongs to each category when the center of the object falls into a grid. For each predicted bounding box, calculating the product of the confidence coefficient C of the bounding box and the probability P that the object category in the bounding box is the license plate, and taking the calculated product as the comprehensive confidence coefficient of the bounding box. And extracting the image in the boundary box with the highest comprehensive confidence as a license plate image of the current inspection vehicle.
Secondly, based on the license plate image, a character segmentation model constructed in advance is adopted to obtain each character image in the license plate image. The character segmentation model can adopt the existing character segmentation model. Specifically, after the character segmentation model is built, the license plate image can be input into the character segmentation model, and then the image of each character output by the character segmentation model is obtained.
Thirdly, based on each character image in the license plate, acquiring an initial license plate number of the current patrol vehicle and confidence coefficient of each character in the initial license plate number by adopting a pre-constructed character recognition model. Wherein the character recognition model can adopt the existing character recognition model. Specifically, after the character recognition model is built, each character image segmented in the license plate image can be sequentially input into the character recognition model, so that each character in the initial license plate number and the confidence level of each character can be obtained, and the recognized characters are sequentially combined, so that the initial license plate number of the current patrol vehicle can be obtained.
S1300, storing a plurality of initial license plates which are identified in the continuous inspection image frames and have the similarity larger than a preset threshold value and the confidence coefficient of each character in each initial license plate into a statistics queue, and determining the license plate number of the current inspection vehicle based on the statistics queue.
In one possible implementation manner, the inspection image frames in the inspection video are acquired frame by frame, and for the inspection image frames acquired at the current time t, the initial license plate number of the current inspection vehicle at the time t and the confidence of each character in the initial license plate number are identified by executing the step S1200. Further, calculating the similarity between the initial license plate number of the current patrol vehicle at the time t and the initial license plate number at the time t-1, and when the similarity is larger than or equal to a set threshold value, indicating that the current patrol vehicle is the same vehicle, and storing the initial license plate number at the time t and the confidence coefficient of each character in the initial license plate number into a statistics queue. The above operation is repeated until the similarity between the initial license plate number at the present time and the initial license plate number at the previous time is smaller than the set threshold, and no license plate number is detected in the continuous F frames, indicating that the next vehicle inspection video collection has been entered, at which time the license plate number of the present inspection vehicle can be determined based on the data stored in the statistical queue. Wherein, the set threshold value can be set to 0.9, and the value of F can be set to 3 or more.
In one possible implementation, when determining the license plate number of the current patrol vehicle based on the statistical queue, the confidence and mode of each character in the statistical queue are implemented based. Specifically, traversing each character forming each initial license plate number in the statistics queue; aiming at the traversed current character, screening out the characters with the highest confidence coefficient and the set number from the initial license plates as preferred characters, and calculating the mode of the preferred characters; and after the traversal is finished, determining the license plate number of the current patrol vehicle based on the modes. Wherein, the set number can be determined by the following formula:
M2=M1-M1/4
wherein M1 is the total initial number of license plates stored in the statistic queue, M1/4 is the number of characters with the lowest confidence, and M2 is the number of characters with the highest confidence, namely the set number.
For example, the data stored in the statistics queue is shown in Table 1:
table 1:
when determining the license plate number of the current inspection vehicle based on the statistics queue, for each character sequence (one column in table 1) in the statistics queue, for example, "Beijing Guangdong", deleting 1/4 records with the lowest confidence, namely deleting Guangdong with the confidence of 0.2, and obtaining the mode Beijing with the highest appearance frequency by the rest queue content. The other 6 characters are processed in the same way. After all the characters are processed, the mode characters corresponding to each character are sequentially combined, so that the final license plate number 'Beijing A11B 20' can be obtained, the confidence coefficient of each character is the average value '0.8, 0.9, 0.8 and 0.9' of each queue, and the overall confidence coefficient is the average value of each character and 0.83.
In one possible implementation manner, when the license plate image is identified, the license plate color is also identified, the license plate colors are counted together into a statistics queue, and then the license plate color of the current patrol vehicle is determined by calculating the mode of the license plate colors in the statistics queue. The license plate color of the current patrol vehicle was determined to be blue after the license plate color was subjected to mode calculation as shown in table 1.
In one possible implementation manner, for each frame of inspection image, the comprehensive confidence coefficient of the license plate image obtained in the processing process is recorded in the processing process, and the comprehensive confidence coefficient is recorded in a statistics queue, so that when the license plate number is determined based on the statistics queue, the time with the highest comprehensive confidence coefficient can be selected as the inspection time of the current inspection vehicle.
In one possible implementation, the method further includes: based on the target image, a pre-built parking state identification model is adopted to obtain the parking state of the current patrol vehicle, wherein the parking state comprises at least one of normal parking, line-crossing parking and cross-position parking. The parking state recognition model can adopt the existing parking state recognition model. Specifically, the target image is input into a parking state identification model, and the parking state of the current patrol vehicle can be obtained.
Through the processing, at least one of the inspection time, license plate color, license plate number and parking state information of the current inspection vehicle can be obtained.
Further, after the inspection time, license plate color, license plate number and parking state of the current inspection vehicle are acquired, parking management can be performed based on the acquired information. For example: when the license plate number and the license plate color of the current inspection vehicle are identified, the management system enters a database, wherein the database has already entered basic information of a plurality of vehicles in advance, such as the license plate number, the license plate color, the name of an owner, the telephone of the owner, whether ETC exists or not, and the like. Other information of the current patrol vehicle can be obtained by comparing the license plate number and the license plate color of the current patrol vehicle with those of each vehicle in the database, for example, the blue license plate Beijing A11111 is Zhang San, the mobile phone number is 12345678900, and ETC is available. And then, corresponding parking management is carried out according to the current parking state of the patrol vehicle. Specifically, if the parking state of the current patrol vehicle is normal, only recording the current parking time; if the parking state of the current patrol vehicle is the line-crossing parking or the position-crossing parking, the vehicle belongs to illegal parking, at the moment, a short message is sent to a vehicle owner of the current patrol vehicle through a management system, the vehicle owner is informed of illegal parking of the vehicle, the parking position is adjusted in time, and the current parking time is recorded.
The present disclosure provides a vehicle parking inspection method, comprising obtaining inspection video; aiming at each frame of inspection image in the inspection video, extracting a target image of the current inspection vehicle from the inspection image, and identifying the initial license plate number of the current inspection vehicle and the confidence of each character in the initial license plate number based on the target image; and storing a plurality of initial license plates which are identified in the continuous inspection image frames and have the similarity larger than a preset threshold value and the confidence coefficient of each character in each initial license plate number into a statistics queue, and determining the license plate number of the current inspection vehicle based on the statistics queue. Because the license plate number of the current patrol vehicle is comprehensively determined based on the continuous image frames in the patrol video, the license plate number of the parked vehicle which is currently patrol can be accurately extracted.
< device example >
Fig. 4 shows a schematic block diagram of a vehicle parking inspection device according to an embodiment of the present disclosure. As shown in fig. 4, the vehicle parking inspection device 100 includes:
a video acquisition module 110, configured to acquire a patrol video;
the image frame processing module 120 is configured to extract, for each frame of inspection image in the inspection video, a target image of the current inspection vehicle from the inspection image, and identify, based on the target image, an initial license plate number of the current inspection vehicle and a confidence level of each character in the initial license plate number;
the license plate number recognition module 130 is configured to store, in the statistics queue, a plurality of initial license plates with similarity greater than a preset threshold value and confidence levels of characters in the initial license plates, which are recognized in the continuous inspection image frame, and determine a license plate number of the current inspection vehicle based on the statistics queue.
< device example >
Fig. 5 shows a schematic block diagram of a vehicle parking lot inspection device according to an embodiment of the present disclosure. As shown in fig. 5, the vehicle parking lot inspection apparatus 200 includes: processor 210 and memory 220 for storing instructions executable by processor 210. Wherein the processor 210 is configured to implement any of the vehicle parking inspection methods described above when executing the executable instructions.
Here, it should be noted that the number of processors 210 may be one or more. Meanwhile, in the vehicle parking lot inspection apparatus 200 of the embodiment of the present disclosure, an input device 230 and an output device 240 may be further included. The processor 210, the memory 220, the input device 230, and the output device 240 may be connected by a bus, or may be connected by other means, which is not specifically limited herein.
The memory 220 is a computer-readable storage medium that can be used to store software programs, computer-executable programs, and various modules, such as: the program or the module corresponding to the vehicle parking inspection method in the embodiment of the disclosure. The processor 210 executes various functional applications and data processing of the vehicle parking inspection device 200 by running software programs or modules stored in the memory 220.
The input device 230 may be used to receive an input digital or signal. Wherein the signal may be a key signal generated in connection with user settings of the device/terminal/server and function control. The output means 240 may comprise a display device such as a display screen.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (10)
1. The vehicle parking inspection method is implemented by a cloud server and is characterized by comprising the following steps of:
acquiring a patrol video;
extracting a target image of a current patrol vehicle from each frame of patrol image in the patrol video, and identifying an initial license plate number of the current patrol vehicle and confidence of each character in the initial license plate number based on the target image;
and storing a plurality of initial license plates which are identified in the continuous inspection image frames and have the similarity larger than a preset threshold value and the confidence coefficient of each character in each initial license plate number into a statistics queue, and determining the license plate number of the current inspection vehicle based on the statistics queue.
2. The method of claim 1, wherein the inspection video is generated by a parking inspection system;
the parking inspection system when generating the inspection video comprises the following steps:
at least two initial videos are respectively acquired by adopting at least one camera;
and splicing the image frames in at least two initial videos to obtain the inspection video.
3. The method of claim 2, wherein the parking inspection system, after obtaining the inspection video, further comprises: and carrying out image preprocessing on each image frame in the inspection video.
4. The method of claim 1, wherein the extracting of the target image of the current inspection vehicle from the inspection image is based on a pre-constructed vehicle identification model.
5. The method of claim 1, wherein upon identifying an initial license plate number of the current patrol vehicle and a confidence level of each character in the initial license plate number based on the target image, comprising:
based on the target image, acquiring a license plate image of the current patrol vehicle by adopting a pre-constructed license plate recognition model;
based on the license plate image, acquiring each character image in the license plate image by adopting a pre-constructed character segmentation model;
based on each character image, acquiring an initial license plate number of the current patrol vehicle and the confidence coefficient of each character in the initial license plate number by adopting a pre-constructed character recognition model.
6. The method of claim 1, wherein, when determining the license plate number of the current inspection vehicle based on the statistical queue, based on a confidence and a mode of each character in the statistical queue.
7. The method of claim 6, wherein determining the license plate number of the current inspection vehicle based on the confidence and mode of each character in the statistical queue comprises:
traversing each character forming each initial license plate number in the statistic queue;
aiming at the traversed current character, screening out the characters with the highest confidence coefficient and the set number from the initial license plates as preferred characters, and calculating the mode of the preferred characters;
and after the traversal is finished, determining the license plate number of the current patrol vehicle based on each mode.
8. The method as recited in claim 1, further comprising:
and acquiring the parking state of the current patrol vehicle by adopting a pre-built parking state identification model based on the target image, wherein the parking state comprises at least one of normal parking, line-crossing parking and cross-position parking.
9. A vehicle parking inspection device, comprising:
the video acquisition module is used for acquiring the inspection video;
the image frame processing module is used for extracting a target image of a current patrol vehicle from each frame of patrol image in the patrol video, and identifying an initial license plate number of the current patrol vehicle and the confidence coefficient of each character in the initial license plate number based on the target image;
the license plate number recognition module is used for storing a plurality of initial license plates which are recognized in the continuous inspection image frame and have similarity larger than a preset threshold value and the confidence coefficient of each character in each initial license plate number into the statistics queue, and determining the license plate number of the current inspection vehicle based on the statistics queue.
10. A vehicle parking inspection apparatus, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method of any one of claims 1 to 8 when executing the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311076318.3A CN116977949A (en) | 2023-08-24 | 2023-08-24 | Vehicle parking inspection method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311076318.3A CN116977949A (en) | 2023-08-24 | 2023-08-24 | Vehicle parking inspection method, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116977949A true CN116977949A (en) | 2023-10-31 |
Family
ID=88476655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311076318.3A Pending CN116977949A (en) | 2023-08-24 | 2023-08-24 | Vehicle parking inspection method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116977949A (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103699905A (en) * | 2013-12-27 | 2014-04-02 | 深圳市捷顺科技实业股份有限公司 | Method and device for positioning license plate |
| CN103824066A (en) * | 2014-03-18 | 2014-05-28 | 厦门翼歌软件科技有限公司 | Video stream-based license plate recognition method |
| CN104951784A (en) * | 2015-06-03 | 2015-09-30 | 杨英仓 | Method of detecting absence and coverage of license plate in real time |
| CN108052931A (en) * | 2018-01-05 | 2018-05-18 | 北京智芯原动科技有限公司 | A kind of license plate recognition result fusion method and device |
| CN110706261A (en) * | 2019-10-22 | 2020-01-17 | 上海眼控科技股份有限公司 | Vehicle violation detection method and device, computer equipment and storage medium |
| US20200090506A1 (en) * | 2018-09-19 | 2020-03-19 | National Chung-Shan Institute Of Science And Technology | License plate recognition system and license plate recognition method |
| CN110930729A (en) * | 2019-11-13 | 2020-03-27 | 北京智芯原动科技有限公司 | Low-pole road side parking detection method and device based on video |
| CN111339949A (en) * | 2020-02-26 | 2020-06-26 | 北京停简单信息技术有限公司 | License plate recognition method and device and inspection vehicle |
| CN112115939A (en) * | 2020-08-26 | 2020-12-22 | 深圳市金溢科技股份有限公司 | Vehicle license plate recognition method and device |
| CN112836683A (en) * | 2021-03-04 | 2021-05-25 | 广东建邦计算机软件股份有限公司 | License plate recognition method, device, equipment and medium for portable camera equipment |
| CN114446059A (en) * | 2021-12-29 | 2022-05-06 | 北京智联云海科技有限公司 | System and method for vehicle-mounted monitoring of roadside parking vehicles |
| CN114898353A (en) * | 2022-07-13 | 2022-08-12 | 松立控股集团股份有限公司 | License plate identification method based on video sequence image characteristics and information |
| CN115298705A (en) * | 2020-01-10 | 2022-11-04 | 顺丰科技有限公司 | License plate recognition method and device, electronic equipment and storage medium |
| CN115294558A (en) * | 2022-08-09 | 2022-11-04 | 杭州中威电子股份有限公司 | Large-angle license plate recognition system and method thereof |
| CN116152691A (en) * | 2021-11-19 | 2023-05-23 | 成都鼎桥通信技术有限公司 | Image detection method, device, equipment and storage medium |
-
2023
- 2023-08-24 CN CN202311076318.3A patent/CN116977949A/en active Pending
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103699905A (en) * | 2013-12-27 | 2014-04-02 | 深圳市捷顺科技实业股份有限公司 | Method and device for positioning license plate |
| CN103824066A (en) * | 2014-03-18 | 2014-05-28 | 厦门翼歌软件科技有限公司 | Video stream-based license plate recognition method |
| CN104951784A (en) * | 2015-06-03 | 2015-09-30 | 杨英仓 | Method of detecting absence and coverage of license plate in real time |
| CN108052931A (en) * | 2018-01-05 | 2018-05-18 | 北京智芯原动科技有限公司 | A kind of license plate recognition result fusion method and device |
| US20200090506A1 (en) * | 2018-09-19 | 2020-03-19 | National Chung-Shan Institute Of Science And Technology | License plate recognition system and license plate recognition method |
| CN110706261A (en) * | 2019-10-22 | 2020-01-17 | 上海眼控科技股份有限公司 | Vehicle violation detection method and device, computer equipment and storage medium |
| CN110930729A (en) * | 2019-11-13 | 2020-03-27 | 北京智芯原动科技有限公司 | Low-pole road side parking detection method and device based on video |
| CN115298705A (en) * | 2020-01-10 | 2022-11-04 | 顺丰科技有限公司 | License plate recognition method and device, electronic equipment and storage medium |
| CN111339949A (en) * | 2020-02-26 | 2020-06-26 | 北京停简单信息技术有限公司 | License plate recognition method and device and inspection vehicle |
| CN112115939A (en) * | 2020-08-26 | 2020-12-22 | 深圳市金溢科技股份有限公司 | Vehicle license plate recognition method and device |
| CN112836683A (en) * | 2021-03-04 | 2021-05-25 | 广东建邦计算机软件股份有限公司 | License plate recognition method, device, equipment and medium for portable camera equipment |
| CN116152691A (en) * | 2021-11-19 | 2023-05-23 | 成都鼎桥通信技术有限公司 | Image detection method, device, equipment and storage medium |
| CN114446059A (en) * | 2021-12-29 | 2022-05-06 | 北京智联云海科技有限公司 | System and method for vehicle-mounted monitoring of roadside parking vehicles |
| CN114898353A (en) * | 2022-07-13 | 2022-08-12 | 松立控股集团股份有限公司 | License plate identification method based on video sequence image characteristics and information |
| CN115294558A (en) * | 2022-08-09 | 2022-11-04 | 杭州中威电子股份有限公司 | Large-angle license plate recognition system and method thereof |
Non-Patent Citations (3)
| Title |
|---|
| RAYSON LAROCA等: "A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector", 《IJCNN》, 14 October 2018 (2018-10-14), pages 1 - 10 * |
| 余以春等: "临时停车场自动收费系统", 《计算机系统应用》, vol. 30, no. 5, 28 April 2021 (2021-04-28), pages 76 - 82 * |
| 田爱军等: "无人机端路面车辆违停检测及取证系统", 《测控技术》, vol. 40, no. 5, 31 May 2021 (2021-05-31), pages 67 - 74 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109858371B (en) | Face recognition method and device | |
| KR102217253B1 (en) | Apparatus and method for analyzing behavior pattern | |
| CN105631418A (en) | People counting method and device | |
| WO2019114145A1 (en) | Head count detection method and device in surveillance video | |
| CN111723656B (en) | Smog detection method and device based on YOLO v3 and self-optimization | |
| US20220301317A1 (en) | Method and device for constructing object motion trajectory, and computer storage medium | |
| CN111723773A (en) | Remnant detection method, device, electronic equipment and readable storage medium | |
| CN112800825A (en) | Key point-based association method, system and medium | |
| CN115131714A (en) | Intelligent detection and analysis method and system for video image | |
| CN111160107B (en) | Dynamic region detection method based on feature matching | |
| WO2017177559A1 (en) | Image management method and apparatus | |
| US9256789B2 (en) | Estimating motion of an event captured using a digital video camera | |
| CN112163544A (en) | Method and system for judging random placement of non-motor vehicles | |
| CN111428740A (en) | Detection method and device for network-shot photo, computer equipment and storage medium | |
| CN113139419B (en) | Unmanned aerial vehicle detection method and device | |
| CN110769262A (en) | Video image compression method and system | |
| CN110956093A (en) | Big data-based model identification method, device, equipment and medium | |
| CN116977949A (en) | Vehicle parking inspection method, device and equipment | |
| CN114449362B (en) | Video cover selection method, device, equipment and storage medium | |
| CN113486788A (en) | Video similarity determination method and device, electronic equipment and storage medium | |
| CN114445788A (en) | Vehicle parking detection method and device, terminal equipment and readable storage medium | |
| CN115082326A (en) | Processing method for deblurring video, edge computing equipment and central processor | |
| CN112347996A (en) | Scene state judgment method, device, equipment and storage medium | |
| CN113361426A (en) | Vehicle loss assessment image acquisition method, medium, device and electronic equipment | |
| CN113191210A (en) | Image processing method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |