CN116030452A - License plate recognition method, device, equipment and computer storage medium - Google Patents
License plate recognition method, device, equipment and computer storage medium Download PDFInfo
- Publication number
- CN116030452A CN116030452A CN202211721745.8A CN202211721745A CN116030452A CN 116030452 A CN116030452 A CN 116030452A CN 202211721745 A CN202211721745 A CN 202211721745A CN 116030452 A CN116030452 A CN 116030452A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- frame
- license plate
- target
- effective
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The application provides a license plate recognition method, device, equipment and computer storage medium, wherein the method comprises the following steps: carrying out rough vehicle identification on the current image frame, and carrying out vehicle tracking according to the identification result; determining an effective vehicle appearing in the current frame according to the vehicle tracking result, the vehicle rough identification result and the effective vehicle screening condition; selecting at least one effective vehicle matched with the resources to be scheduled as the vehicle to be scheduled according to the vehicle scheduling priority of the effective vehicle appearing in the current frame and the current resources to be scheduled; detecting the license plate frame of the vehicle to be dispatched again, identifying the license plate frame detected again, associating the license plate frame detected again with the identified license plate and the corresponding vehicle to be dispatched, and obtaining a license plate identification result; determining a target vehicle according to a license plate recognition result, and scheduling the target vehicle by utilizing resources to be scheduled; according to the license plate recognition method and device, the accuracy of license plate recognition can be improved under a natural scene, and important technical support is provided for tracking vehicles.
Description
Technical Field
The present disclosure relates to the field of computer vision, and in particular, to a license plate recognition method, device, apparatus, and computer storage medium.
Background
In a video monitoring scene, the license plate recognition technology of the vehicle is an important technical means when the vehicle is traced, the content in the license plate can be recognized and stored, and important technical support can be provided for technicians when the vehicle is traced; however, in a natural scene, due to factors such as vehicle angle, illumination, equipment performance and the like, the problem of inaccurate identification can occur with high probability, the work is affected, and especially when law enforcement personnel trace the illegal behaviors of the vehicles, the situation that the vehicles are wrong or cannot be identified can occur, so that the work is greatly affected. Based on the above-mentioned problems, there is a need for a license plate recognition method that can recognize the content of a license plate as much as possible in a natural environment.
Disclosure of Invention
The application provides a license plate recognition method, device, equipment and computer storage medium, which can ensure that the accuracy of license plate recognition is improved in a natural scene and provide important technical support for tracking vehicles.
In a first aspect, the present application provides a license plate recognition method, including:
Acquiring a current image frame from a video stream, performing vehicle rough identification on the current image frame, and performing vehicle tracking according to an identification result, wherein the vehicle rough identification comprises vehicle frame detection and license plate frame detection of a vehicle;
determining an effective vehicle appearing in the current frame according to the vehicle tracking result, the vehicle rough identification result and the effective vehicle screening condition;
according to the vehicle dispatching priority of the effective vehicle appearing in the current frame and the current resources to be dispatched, selecting at least one effective vehicle matched with the resources to be dispatched as the vehicle to be dispatched;
carrying out license plate frame detection again on the position area of the vehicle frame of the vehicle to be dispatched, carrying out license plate recognition on the license plate frame detected again, and associating the license plate frame detected again with the recognized license plate and the corresponding vehicle to be dispatched to obtain a license plate recognition result;
and determining a target vehicle according to the license plate recognition result, and scheduling the target vehicle by utilizing resources to be scheduled.
In one or more possible embodiments, performing vehicle coarse recognition on the current image frame includes:
performing vehicle frame detection and license plate frame detection on the current image frame by using a deep learning detection method;
calculating the association degree of the detected vehicle frame and the license plate frame according to the position areas of the detected vehicle frame and the license plate frame of the current frame;
And determining the associated vehicle frames and license plate frames belonging to the same vehicle according to the calculated association degree, wherein the vehicle frames without the associated license plate frames and the license plate frames without the associated vehicle frames are obtained, and obtaining the recognition result.
In one or more possible embodiments, the vehicle tracking based on the recognition result includes:
acquiring a detected vehicle frame and a detected license plate frame;
when a tracking algorithm is utilized to determine and identify a frame or a license plate frame of a new vehicle, an identity corresponding to the frame of the new vehicle is added, and the frame and the license plate frame of the new vehicle with association relations adopt the same identity;
when the frame or license plate frame of the tracked vehicle is identified by utilizing the tracking algorithm, the identity of the frame or license plate frame of the tracked vehicle is currently identified according to the identity of the frame or license plate frame of the tracked vehicle, and the frame and license plate frame of the tracked vehicle with association relations adopt the same identity.
In one or more possible embodiments, determining the valid vehicle in which the current frame appears according to the target tracking result, the vehicle coarse recognition result, and the valid target screening condition includes:
acquiring vehicles in a current target cache pool;
and updating the vehicles in the current target cache pool according to the target tracking result, the vehicle coarse recognition result and the effective target screening condition, wherein the vehicles in the updated target cache pool are effective vehicles appearing in the current frame.
In one or more possible embodiments, updating the vehicles in the current target cache pool according to the target tracking result, the vehicle coarse recognition result and the effective target screening condition comprises at least one of the following steps:
according to the target tracking result, the vehicle coarse recognition result and the effective target screening condition, when any effective vehicle appearing in the current frame is determined not to be in the target cache pool, the effective vehicle is placed in the target cache pool;
according to a target tracking result, when any vehicle in the target cache pool exceeds a set time period and is not present in an image frame, deleting the vehicle from the target cache pool;
determining that the size of a position area of a detected vehicle frame does not meet the requirement according to a rough vehicle identification result, determining the vehicle corresponding to the detected vehicle frame as an invalid vehicle, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool;
according to a rough vehicle identification result, determining that a vehicle corresponding to a vehicle frame or a license plate frame with detection confidence coefficient lower than a set threshold value is an invalid vehicle, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool;
According to a rough vehicle identification result, determining an identified vehicle frame or license plate frame outside a set detection area as an invalid vehicle, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool;
determining displacement of a vehicle frame or a license plate frame according to a rough vehicle identification result and a target tracking result, determining that a vehicle in a static state is an invalid vehicle according to the displacement, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool;
and according to the rough identification result of the vehicle, determining that the vehicle which exists in the cache pool and does not appear in the current frame is an invalid vehicle, and deleting the invalid vehicle from the target cache pool.
In one or more possible embodiments, the vehicle dispatch priority of the active vehicle in which the current frame appears is determined in the following manner:
if the effective vehicle is a vehicle which newly appears in the current frame, determining that the vehicle scheduling priority of the effective vehicle is a set default value;
if the effective vehicle is not scheduled in the previous frame, the scheduling priority of the effective vehicle is increased by one level;
and if the effective vehicle is the effective vehicle scheduled in the previous frame, determining that the scheduling priority of the effective vehicle is the lowest priority.
In one or more possible embodiments, associating the redetected license plate frame with the identified license plate, the corresponding vehicle to be scheduled, includes:
for the same vehicle to be dispatched, if a license plate frame associated with the detection frame exists in the rough recognition of the vehicle, and the associated license plate frame is inconsistent with the redetected license plate frame, replacing the associated license plate frame by utilizing the redetected license plate frame;
and if the license plate frame associated with the detection frame does not exist in the rough identification of the vehicle, associating the re-detected license plate frame with the detection frame of the vehicle to be dispatched.
In one or more possible embodiments, determining a target vehicle according to a license plate recognition result, and scheduling the target vehicle by using resources to be scheduled, including:
for the same vehicle to be scheduled, acquiring license plate recognition results of a plurality of latest frames and carrying out statistics;
if the license plate recognition results of the vehicles to be scheduled in multiple frames are different, determining the vehicle corresponding to the license plate recognition result with the largest frame number as a target vehicle according to the frame number of the same license plate recognition result;
if a plurality of different license plate recognition junctions with the largest number of frames exist, determining the vehicle corresponding to the license plate recognition result with the highest confidence as the target vehicle.
In a second aspect, the present application provides a license plate recognition device, the device comprising: the vehicle coarse recognition module is used for acquiring a current image frame from the video stream, carrying out vehicle coarse recognition on the current image frame, and carrying out vehicle tracking according to a recognition result, wherein the vehicle coarse recognition comprises vehicle frame detection and license plate frame detection of a vehicle;
the effective vehicle determining module is used for determining an effective vehicle appearing in the current frame according to the vehicle tracking result, the vehicle rough identification result and the effective vehicle screening condition;
the vehicle to be scheduled determining module is used for selecting at least one effective vehicle matched with the resources to be scheduled as a vehicle to be scheduled according to the vehicle scheduling priority of the effective vehicle appearing in the current frame and the current resources to be scheduled;
the license plate recognition module is used for detecting the license plate frame again in the position area of the vehicle frame of the vehicle to be dispatched, recognizing the license plate frame detected again, and associating the detected license plate frame with the recognized license plate and the corresponding vehicle to be dispatched to obtain a license plate recognition result;
and the license plate result determining module is used for determining a target vehicle according to a license plate recognition result and scheduling the target vehicle by utilizing resources to be scheduled.
In a third aspect, the present application provides a license plate recognition apparatus, the apparatus comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the license plate recognition method of any one of the first aspects.
In a fourth aspect, the present application also provides a computer storage medium storing a computer program for causing a computer to execute the license plate recognition method according to any one of the first aspects.
The application provides a license plate recognition method, device, equipment and computer storage medium, which can ensure that the accuracy of license plate recognition is improved in a natural scene and provide important technical support for tracking vehicles; the scheme illustrates a license plate recognition strategy of how to poll a motor vehicle tracking target through a resource scheduling strategy under the condition of limited resources, and a more accurate license plate recognition result can be obtained through multi-frame license plate recognition voting while controlling single-frame peak time consumption.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application and do not constitute an undue limitation on the application.
FIG. 1 is a flow chart provided in accordance with one embodiment of the present application;
FIG. 2 is a flow chart provided in accordance with one embodiment of the present application;
FIG. 3 is a flow chart provided in accordance with one embodiment of the present application;
FIG. 4 is a schematic illustration of an apparatus provided according to one embodiment of the present application;
FIG. 5 is a schematic diagram of an apparatus provided according to one embodiment of the present application;
fig. 6 is a schematic diagram of a computer-readable storage medium according to one embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In a video monitoring scene, the license plate recognition technology of the vehicle is an important technical means when the vehicle is traced, the content in the license plate can be recognized and stored, and important technical support can be provided for technicians when the vehicle is traced; however, in a natural scene, due to factors such as vehicle angle, illumination, equipment performance and the like, the problem of inaccurate identification can occur with high probability, the work is affected, and especially when law enforcement personnel trace the illegal behaviors of the vehicles, the situation that the vehicles are wrong or cannot be identified can occur, so that the work is greatly affected. Therefore, the license plate recognition method provided by the application based on the problems can ensure that the accuracy of license plate recognition is improved in a natural scene, and provides important technical support for tracking vehicles.
The license plate recognition method provided by the application, as shown in fig. 1, comprises the following steps:
in one or more possible embodiments, first, image frame data in a video stream needs to be acquired, processing is performed for each image frame, and rough recognition of a vehicle including frame detection and license plate frame detection of the vehicle is performed on a current frame, so that frame detection information D can be obtained v ＝{d i I=1..m }, indicating that the current frame has M vehicles in total, and obtaining license plate frame detection information D p ＝{d i I=1..n }, indicating that N license plates are detected in total in the current frame, and tracking the vehicle according to the obtained M vehicles and the N license plates.
In one or more possible embodiments, the M vehicles and the N license plates are identified in the current frame, and the vehicle meeting the conditions is selected as the valid vehicle of the current frame according to the vehicle tracking result, the vehicle rough identification result and the valid vehicle screening condition; the vehicle tracking results comprise new vehicles identified by the current frame, vehicles in a lost state, vehicles in a deleted state exceeding the life cycle, vehicles in tracking and the like; the life cycle is defined as a period from the start of the first frame to the last frame that disappears to a predetermined period, and a vehicle exceeding the life cycle is defined as a vehicle in a deleted state.
And 103, selecting at least one effective vehicle matched with the resources to be scheduled as the vehicle to be scheduled according to the vehicle scheduling priority of the effective vehicle appearing in the current frame and the current resources to be scheduled.
In one or more possible embodiments, the vehicle scheduling priority of the current frame that is newly appeared is a default value, and the scheduling priority of the valid vehicle is related to the time that is not to be scheduled, that is, the longer the time that is not to be scheduled, the higher the vehicle scheduling priority; the effective vehicles can be ordered according to the vehicle dispatching priority, and the set number of effective vehicles are selected as vehicles to be dispatched according to the ordering of the vehicles and the resources to be dispatched, so that the next operation is carried out.
And 104, detecting the license plate frame again in the position area of the vehicle frame of the vehicle to be dispatched, recognizing the license plate frame detected again, and associating the detected license plate frame with the recognized license plate and the corresponding vehicle to be dispatched to obtain a license plate recognition result.
In one or more possible embodiments, after the vehicle to be dispatched is output, the vehicle to be dispatched is subjected to vehicle detection and license plate frame detection again, license plate recognition is performed, the redetected license plate frame is associated with the corresponding vehicle to be dispatched, and then the corresponding license plate recognition result can be obtained by the vehicle to be dispatched.
And 105, determining a target vehicle according to the license plate recognition result, and scheduling the target vehicle by utilizing resources to be scheduled.
In one or more possible embodiments, a target vehicle is determined from the vehicles to be scheduled, and the target vehicle is scheduled by using the resources to be scheduled to perform other required tasks.
According to the license plate recognition method provided by the application, the accuracy of license plate recognition can be guaranteed to be improved in a natural scene; under the condition of limited resources, a license plate recognition strategy for polling a vehicle tracking target through a resource scheduling strategy can acquire a more accurate license plate recognition result through multi-frame license plate recognition voting while controlling single-frame peak time consumption, and provides important technical support for tracking vehicles; and according to the two correlations between the vehicle and the license plate, the error license plate can be filtered, and the accuracy of identifying the license plate is improved.
In one or more possible embodiments, the performing the vehicle coarse recognition on the current image frame, as shown in fig. 2, includes:
in one or more possible embodiments, the deep learning detection algorithm may be YOLO series or CNN-characteristic region fast RCNN series deep learning detection method, and perform frame detection and license plate frame detection on the current image frame.
In one or more possible embodiments, according to the detected position areas of the vehicle frame and the license plate frame, calculating the association degree of the detected vehicle frame and the license plate frame through a hungarian algorithm, wherein a specific association degree calculation formula is as follows:
wherein x is v And y v Respectively an abscissa and an ordinate of a center point of the vehicle frame; x is x p And y p Respectively an abscissa and an ordinate of a central point of the license plate frame; w (w) frame And h frame The width and the height of the image of the current frame are respectively; according to the calculation of the degree of association, the degree of association can be calculated by settingA threshold value is used for determining whether an association relationship exists between the vehicle and the license plate; for example, there are M vehicles and N license plates, calculating the association degrees of the M vehicles and the N license plates, obtaining a matrix of M rows and N columns, setting a threshold to determine whether there is an association relationship between the vehicle and the license plate, determining that the corresponding license plate has no association relationship with the vehicle when the association relationship exceeds the set threshold, and determining that the corresponding vehicle is associated with the license plate according to the calculated bidirectional matching result of the vehicle and the license plate and smaller than the set threshold. Therefore, after the above-mentioned association calculation, three situations can appear, namely, the associated vehicle frame and license plate frame, the vehicle frame without the associated license plate frame, and the license plate frame without the associated vehicle frame, so as to obtain three recognition results.
In one or more possible embodiments, the vehicle tracking according to the recognition result, as shown in fig. 3, includes:
In one or more possible embodiments, the tracking algorithm may track the vehicle for a tracking algorithm such as Sort, deepSort; when the tracking algorithm is used for tracking the vehicle, if the frame or license plate frame of the new vehicle is identified in the current frame, a new identity is set for the new frame or license plate frame, and if the new frame and license plate frame have an association relationship, the same identity can be adopted, for example (T v ,T p )＝{(t v ,t p ) i I=1..p }, indicating that the vehicle and license plate association was successful, where t v And t p Representing successfully associated vehicles and license plates, and P represents the logarithm of successfully associated vehicles and license plates; (T) v )＝{(t v ) i I=1,..q } means that there are Q vehicles alone; (T) p )＝{(t p ) i I=1,..r } means that there are R license plates alone; when the frame or license plate frame of the tracked vehicle is identified by other frames, the identity of the currently identified frame or license plate frame is determined according to the identity of the frame or license plate frame of the tracked vehicle, and the frame and license plate frame of the tracked vehicle with association relationship adopt the same identity; therefore, the identification corresponding to the associated vehicle frame and the license plate frame, the vehicle frame without the associated license plate frame and the license plate frame without the associated vehicle frame can be obtained.
In one or more possible embodiments, determining the valid vehicle in which the current frame appears according to the target tracking result, the vehicle coarse recognition result and the valid target screening condition includes at least one of the following steps:
1) Acquiring vehicles in a current target cache pool; the current target cache pool comprises the effective vehicles determined by the previous frame;
2) According to the target tracking result, the rough vehicle identification result and the effective target screening condition, updating the vehicles in the current target cache pool, wherein the vehicles in the updated target cache pool are effective vehicles appearing in the current frame; because new effective vehicles, lost vehicles, vehicles exceeding the life cycle, vehicles with poor accuracy of recognition results and the like appear in the current frame, the vehicles in the current target cache pool need to be updated to acquire the latest target cache pool.
3) The target cache pool can be established before the data acquisition of the image frames of the video stream is carried out, and the target cache pool can be directly called when the data of the current frame is used according to the updating of the current frame step by step; and the three different identity identifications are stored in the target cache pool, so that the method is more convenient and quick to call.
In one or more possible embodiments, updating the vehicles in the current target cache pool according to the target tracking result, the vehicle coarse recognition result and the effective target screening condition comprises at least one of the following steps:
1) According to the target tracking result, the vehicle coarse recognition result and the effective target screening condition, when any effective vehicle appearing in the current frame is determined not to be in the target cache pool, the effective vehicle is placed in the target cache pool; and taking the effective vehicle appearing in the current frame as a new effective vehicle, and putting the new effective vehicle into the target cache pool for updating.
2) According to a target tracking result, when any vehicle in the target cache pool exceeds a set time period and is not present in an image frame, deleting the vehicle from the target cache pool; and determining whether the number of frames from the last frame to the current frame exceeds a set number threshold or not according to the number of frames from the last frame to the current frame, and deleting the vehicle from the target cache pool when the set time is not exceeded, wherein the vehicle is in a deleting state.
3) Determining that the size of a position area of a detected vehicle frame does not meet the requirement according to a rough vehicle identification result, determining the vehicle corresponding to the detected vehicle frame as an invalid vehicle, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool; and when the height of the frame at the current frame is smaller than the set height threshold and the width is smaller than the set width threshold, determining that the size of the position area of the detected frame does not meet the requirement, defining the vehicle corresponding to the frame as an invalid vehicle and deleting the vehicle from the target cache pool.
4) According to a rough vehicle identification result, determining that a vehicle corresponding to a vehicle frame or a license plate frame with detection confidence coefficient lower than a set threshold value is an invalid vehicle, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool; in the vehicle detection process, not only the vehicle frame and the license plate frame are detected, but also the confidence coefficient of the detection result is obtained, if the confidence coefficient is lower than a set threshold value, the accuracy of the detection result is poor, the corresponding vehicle is defined as an invalid vehicle, and the invalid vehicle is deleted from the target cache pool. 5) According to a rough vehicle identification result, determining an identified vehicle frame or license plate frame positioned in a set detection area as an invalid vehicle, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool; for example, the set detection area of the license plate frame is generally the lower half part of the vehicle frame, the upper half part of the vehicle frame is outside the set detection area, if the license plate frame is recognized outside the set detection area, the license plate frame is directly determined to be recognized as an invalid license plate, and the corresponding vehicle is defined as an invalid vehicle and is deleted from the target cache pool.
6) Determining displacement of a vehicle frame or a license plate frame according to a rough vehicle identification result and a target tracking result, determining that a vehicle in a static state is an invalid vehicle according to the displacement, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool; the stationary state can be calculated by displacement of the center point of the frame, and if the abscissa displacement is less than half the width of the frame and the ordinate displacement is less than half the height of the frame, the motor vehicle is in the stationary state.
7) And according to the rough identification result of the vehicle, determining that the vehicle which exists in the cache pool and does not appear in the current frame is an invalid vehicle, and deleting the invalid vehicle from the target cache pool.
In one placeIn one or more possible embodiments, the vehicle scheduling priority of the valid vehicle in which the current frame appears is determined in the following manner: if the effective vehicle is a vehicle which newly appears in the current frame, determining that the vehicle scheduling priority of the effective vehicle is a set default value; if the effective vehicle is an unscheduled effective vehicle in the previous frame, the scheduling priority of the effective vehicle is increased by one level; and if the effective vehicle is the vehicle scheduled in the previous frame, determining that the scheduling priority of the effective vehicle is the lowest priority. In this way, the vehicle dispatching priority of the newly-appearing vehicle in the current frame can be set as a default value and added into the target cache pool, if the effective vehicle in the target cache pool is also the effective vehicle in the previous frame but not dispatched, the dispatching priority of the corresponding effective vehicle is increased by one level, so that the vehicles can be dispatched as soon as possible in the following frame, if the effective vehicle is dispatched in the previous frame, the dispatching priority of the corresponding effective vehicle is set as the lowest priority, and the repeated dispatching in the following frame is ensured; for example, the valid vehicle is a new joining vehicle for the current frame, and the scheduling priority of the vehicle is a default priority d The method comprises the steps of carrying out a first treatment on the surface of the The effective vehicles in the target cache pool are also effective vehicles in the last frame but are not scheduled, the scheduling priority of the effective vehicles is updated, and the original scheduling priority p is updated to p=p+1; if the effective vehicle is already scheduled in the previous frame, the corresponding vehicle scheduling priority is set to be the lowest priority 0, and repeated scheduling of the effective vehicle is avoided.
In one or more possible embodiments, associating the redetected license plate frame with the identified license plate, the corresponding vehicle to be scheduled, includes: for the same vehicle to be dispatched, if a license plate frame associated with the detection frame exists in the rough recognition of the vehicle, and the associated license plate frame is inconsistent with the redetected license plate frame, replacing the associated license plate frame by utilizing the redetected license plate frame; and if the license plate frame associated with the detection frame does not exist in the rough identification of the vehicle, associating the re-detected license plate frame with the detection frame of the vehicle to be dispatched. Because the first coarse recognition process is based on the whole frame of image for recognition, the situation that the association between the vehicle and the license plate is wrong or the situation that the vehicle is missed is likely to occur, the second recognition of the vehicle to be scheduled is to accurately detect fewer vehicles independently, and the detection accuracy can be effectively improved; the corresponding vehicles to be dispatched are output and then are subjected to vehicle frame detection and license plate frame detection again, license plate recognition is carried out according to the new license plate frame detection at this time, and the obtained license plate recognition result is more accurate; when the license plate of the vehicle to be dispatched is associated, if the license plate frame associated with the detection frame exists in the rough recognition of the vehicle, but the associated license plate frame is inconsistent with the redetected license plate frame due to different detection thickness, the redetected license plate frame needs to be replaced for re-associating with the vehicle to be dispatched; and if the license plate frame associated with the detection frame does not exist in the rough identification of the vehicle, associating the re-detected license plate frame with the detection frame of the vehicle to be dispatched.
In one or more possible embodiments, determining a target vehicle according to a license plate recognition result, and scheduling the target vehicle by using resources to be scheduled, including: for the same vehicle to be scheduled, acquiring license plate recognition results of a plurality of latest frames and carrying out statistics; if the license plate recognition results of the vehicles to be scheduled in multiple frames are different, for example, 10 frames of license plate recognition results are shared, wherein 8 frames are "SuD 3Q521", and the other two frames of license plate recognition results are "SuD 30521", the vehicle corresponding to the license plate recognition result with the largest frame number is required to be determined as the target vehicle according to the frame number of the same license plate recognition result, so that the license plate of the target vehicle should be selected as the final recognition result by using "SuD 3Q 521"; if there are a plurality of different license plate recognition junctions with the largest number of frames, for example, 10 license plate recognition results are shared, wherein 5 frames are "SuD 3Q521", and the other 5 frames are "SuD 30521", but the confidence of the algorithm with the recognition result of "SuD 3Q521" is higher, and the vehicle corresponding to the license plate recognition result with the highest confidence is determined to be the target vehicle, so that the license plate of the target vehicle should be selected as the final recognition result by "SuD 3Q 521". Thus, license plate recognition results corresponding to the scheduled vehicles can be obtained; the license plate recognition result with the highest frame number and the license plate recognition result with the highest confidence coefficient are used as the final license plate recognition result, so that the accuracy of the license plate recognition result can be effectively improved.
According to the license plate recognition method, the accuracy of license plate recognition can be improved under a natural scene, and important technical support is provided for tracking vehicles. The scheme illustrates a license plate recognition strategy of how to poll a motor vehicle tracking target through a resource scheduling strategy under the condition of limited resources, and a more accurate license plate recognition result can be obtained through multi-frame license plate recognition voting while controlling single-frame peak time consumption.
Based on the same inventive concept, the present application further provides a license plate recognition device, as shown in fig. 4, including:
the vehicle coarse recognition module 401 is configured to obtain a current image frame from a video stream, perform vehicle coarse recognition on the current image frame, and perform vehicle tracking according to a recognition result, where the vehicle coarse recognition includes vehicle frame detection and license plate frame detection of a vehicle;
an effective vehicle determining module 402, configured to determine an effective vehicle that appears in the current frame according to the vehicle tracking result, the vehicle rough recognition result, and the effective vehicle screening condition;
the to-be-scheduled vehicle determining module 403 is configured to select, as a to-be-scheduled vehicle, at least one valid vehicle that matches with a to-be-scheduled resource according to a vehicle scheduling priority of the valid vehicle that appears in the current frame and a current to-be-scheduled resource;
The license plate recognition module 404 is configured to perform license plate frame detection again on the location area of the vehicle frame of the vehicle to be dispatched, perform license plate recognition on the re-detected license plate frame, and associate the re-detected license plate frame with the recognized license plate and the corresponding vehicle to be dispatched to obtain a license plate recognition result;
and the license plate result determining module 405 is configured to determine a target vehicle according to a license plate recognition result, and schedule the target vehicle by using resources to be scheduled.
Based on the same inventive concept, the application also provides license plate recognition equipment, which comprises at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the license plate recognition method.
As shown in fig. 5, the apparatus includes a processor 501, a memory 502, a communication interface 503, and a bus 504. Wherein the processor 501, the memory 502 and the communication interface 503 are interconnected by a bus 504.
The processor 501 is configured to read and execute the instructions in the memory 502, so that at least one processor can execute the license plate recognition method provided in the foregoing embodiment.
The memory 502 is used for storing various instructions and programs of the license plate recognition method provided in the above embodiment.
The processor 501 may be any combination of a central processor (central processing unit, CPU for short), a network processor (network processor, NP for short), an image processor (Graphic Processing Unit, GPU for short) or CPU, NP, GPU. But also a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (programmable logic device, PLD), or a combination thereof. The PLD may be a complex programmable logic device (complex programmable logic device, CPLD for short), a field-programmable gate array (field-programmable gate array, FPGA for short), general-purpose array logic (generic array logic, GAL for short), or any combination thereof.
In addition, the present application also provides a computer-readable storage medium, as shown in fig. 6, in which a computer program is stored, where the computer program is configured to make a computer execute any one of the methods in the foregoing embodiments.
The memory may include readable media in the form of volatile memory, such as Random Access Memory (RAM) 1321 and/or cache memory 1322, and may further include Read Only Memory (ROM) 1323.
The memory may also include a program/utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.
Claims (11)
1. A license plate recognition method, comprising:
acquiring a current image frame from a video stream, performing vehicle rough identification on the current image frame, and performing vehicle tracking according to an identification result, wherein the vehicle rough identification comprises vehicle frame detection and license plate frame detection of a vehicle;
determining an effective vehicle appearing in the current frame according to the vehicle tracking result, the vehicle rough identification result and the effective vehicle screening condition;
According to the vehicle dispatching priority of the effective vehicle appearing in the current frame and the current resources to be dispatched, selecting at least one effective vehicle matched with the resources to be dispatched as the vehicle to be dispatched;
carrying out license plate frame detection again on the position area of the vehicle frame of the vehicle to be dispatched, carrying out license plate recognition on the license plate frame detected again, and associating the license plate frame detected again with the recognized license plate and the corresponding vehicle to be dispatched to obtain a license plate recognition result;
and determining a target vehicle according to the license plate recognition result, and scheduling the target vehicle by utilizing resources to be scheduled.
2. The method of claim 1, wherein performing vehicle coarse recognition on the current image frame comprises:
performing vehicle frame detection and license plate frame detection on the current image frame by using a deep learning detection method;
calculating the association degree of the detected vehicle frame and the license plate frame according to the position areas of the detected vehicle frame and the license plate frame of the current frame;
and determining the associated vehicle frames and license plate frames belonging to the same vehicle according to the calculated association degree, wherein the vehicle frames without the associated license plate frames and the license plate frames without the associated vehicle frames are obtained, and obtaining the recognition result.
3. The method of claim 2, wherein tracking the vehicle based on the recognition result comprises:
acquiring a detected vehicle frame and a detected license plate frame;
when a tracking algorithm is utilized to determine and identify a frame or a license plate frame of a new vehicle, an identity corresponding to the frame of the new vehicle is added, and the frame and the license plate frame of the new vehicle with association relations adopt the same identity;
when the frame or license plate frame of the tracked vehicle is identified by utilizing the tracking algorithm, the identity of the frame or license plate frame of the tracked vehicle is currently identified according to the identity of the frame or license plate frame of the tracked vehicle, and the frame and license plate frame of the tracked vehicle with association relations adopt the same identity.
4. A method according to any one of claims 1 to 3, wherein determining the valid vehicle for which the current frame is present based on the target tracking result, the vehicle coarse identification result, and the valid target screening condition comprises:
acquiring vehicles in a current target cache pool;
and updating the vehicles in the current target cache pool according to the target tracking result, the vehicle coarse recognition result and the effective target screening condition, wherein the vehicles in the updated target cache pool are effective vehicles appearing in the current frame.
5. The method of claim 4, wherein updating the vehicles in the current target cache pool based on the target tracking result, the vehicle coarse recognition result, and the effective target screening condition, comprises at least one of:
according to the target tracking result, the vehicle coarse recognition result and the effective target screening condition, when any effective vehicle appearing in the current frame is determined not to be in the target cache pool, the effective vehicle is placed in the target cache pool;
according to a target tracking result, when any vehicle in the target cache pool exceeds a set time period and is not present in an image frame, deleting the vehicle from the target cache pool;
determining that the size of a position area of a detected vehicle frame does not meet the requirement according to a rough vehicle identification result, determining the vehicle corresponding to the detected vehicle frame as an invalid vehicle, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool;
according to a rough vehicle identification result, determining that a vehicle corresponding to a vehicle frame or a license plate frame with detection confidence coefficient lower than a set threshold value is an invalid vehicle, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool;
According to a rough vehicle identification result, determining an identified vehicle frame or license plate frame outside a set detection area as an invalid vehicle, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool;
determining displacement of a vehicle frame or a license plate frame according to a rough vehicle identification result and a target tracking result, determining that a vehicle in a static state is an invalid vehicle according to the displacement, and deleting the invalid vehicle from the target cache pool if the invalid vehicle is in the target cache pool;
and according to the rough identification result of the vehicle, determining that the vehicle which exists in the cache pool and does not appear in the current frame is an invalid vehicle, and deleting the invalid vehicle from the target cache pool.
6. The method of claim 1, wherein the vehicle dispatch priority of the active vehicle for which the current frame is present is determined by:
if the effective vehicle is a vehicle which newly appears in the current frame, determining that the vehicle scheduling priority of the effective vehicle is a set default value;
if the effective vehicle is an effective vehicle which is not scheduled in the previous frame, the scheduling priority of the effective vehicle is increased by one level;
And if the effective vehicle is the effective vehicle scheduled in the previous frame, determining that the scheduling priority of the effective vehicle is the lowest priority.
7. The method of claim 2, wherein associating the redetected license plate frame with the identified license plate, corresponding vehicle to be scheduled, comprises:
for the same vehicle to be dispatched, if a license plate frame associated with the detection frame exists in the rough recognition of the vehicle, and the associated license plate frame is inconsistent with the redetected license plate frame, replacing the associated license plate frame by utilizing the redetected license plate frame;
and if the license plate frame associated with the detection frame does not exist in the rough identification of the vehicle, associating the re-detected license plate frame with the detection frame of the vehicle to be dispatched.
8. A method according to any one of claims 1 to 3, wherein determining a target vehicle according to a license plate recognition result, and scheduling the target vehicle using resources to be scheduled, comprises:
for the same vehicle to be scheduled, acquiring license plate recognition results of a plurality of latest frames and carrying out statistics;
if the license plate recognition results of the vehicles to be scheduled in multiple frames are different, determining the vehicle corresponding to the license plate recognition result with the largest frame number as a target vehicle according to the frame number of the same license plate recognition result;
If a plurality of different license plate recognition junctions with the largest number of frames exist, determining the vehicle corresponding to the license plate recognition result with the highest confidence as the target vehicle.
9. A license plate recognition device, the device comprising:
the vehicle coarse recognition module is used for acquiring a current image frame from the video stream, carrying out vehicle coarse recognition on the current image frame, and carrying out vehicle tracking according to a recognition result, wherein the vehicle coarse recognition comprises vehicle frame detection and license plate frame detection of a vehicle;
the effective vehicle determining module is used for determining an effective vehicle appearing in the current frame according to the vehicle tracking result, the vehicle rough identification result and the effective vehicle screening condition;
the vehicle to be scheduled determining module is used for selecting at least one effective vehicle matched with the resources to be scheduled as a vehicle to be scheduled according to the vehicle scheduling priority of the effective vehicle appearing in the current frame and the current resources to be scheduled;
the license plate recognition module is used for detecting the license plate frame again in the position area of the vehicle frame of the vehicle to be dispatched, recognizing the license plate frame detected again, and associating the detected license plate frame with the recognized license plate and the corresponding vehicle to be dispatched to obtain a license plate recognition result;
And the license plate result determining module is used for determining a target vehicle according to a license plate recognition result and scheduling the target vehicle by utilizing resources to be scheduled.
10. A license plate recognition apparatus, characterized in that the apparatus comprises: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the license plate recognition method of any one of claims 1-8.
11. A computer storage medium storing a computer program for causing a computer to execute the license plate recognition method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211721745.8A CN116030452A (en) | 2022-12-30 | 2022-12-30 | License plate recognition method, device, equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211721745.8A CN116030452A (en) | 2022-12-30 | 2022-12-30 | License plate recognition method, device, equipment and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116030452A true CN116030452A (en) | 2023-04-28 |
Family
ID=86075381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211721745.8A Pending CN116030452A (en) | 2022-12-30 | 2022-12-30 | License plate recognition method, device, equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116030452A (en) |
-
2022
- 2022-12-30 CN CN202211721745.8A patent/CN116030452A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110060237B (en) | Fault detection method, device, equipment and system | |
| CN110619333B (en) | Text line segmentation method, text line segmentation device and electronic equipment | |
| CN108564579B (en) | Concrete crack detection method and detection device based on time-space correlation | |
| CN110533654A (en) | The method for detecting abnormality and device of components | |
| CN110659645B (en) | Character recognition method for digital instrument | |
| CN112052850A (en) | License plate recognition method and device, electronic equipment and storage medium | |
| CN111582032A (en) | Pedestrian detection method and device, terminal equipment and storage medium | |
| CN111369801A (en) | Vehicle identification method, device, equipment and storage medium | |
| CN109919154B (en) | Intelligent character recognition method and device | |
| CN112966687B (en) | Image segmentation model training method and device and communication equipment | |
| CN113723467A (en) | Sample collection method, device and equipment for defect detection | |
| CN112434585A (en) | Method, system, electronic device and storage medium for identifying virtual reality of lane line | |
| CN116137061B (en) | Training method and device for quantity statistical model, electronic equipment and storage medium | |
| CN113065607A (en) | Image detection method, image detection device, electronic device, and medium | |
| CN115661131B (en) | Image identification method and device, electronic equipment and storage medium | |
| CN116030452A (en) | License plate recognition method, device, equipment and computer storage medium | |
| CN110390224B (en) | Traffic sign recognition method and device | |
| CN111126286A (en) | Vehicle dynamic detection method and device, computer equipment and storage medium | |
| CN114037700A (en) | Panel defect detection method and device, electronic equipment and storage medium | |
| CN112629828B (en) | Optical information detection method, device and equipment | |
| CN115331193A (en) | Parking space identification method, parking space identification system, electronic equipment and storage medium | |
| CN114240924A (en) | Power grid equipment quality evaluation method based on digitization technology | |
| CN113902999A (en) | Tracking method, device, equipment and medium | |
| CN111639640A (en) | License plate recognition method, device and equipment based on artificial intelligence | |
| CN116542998B (en) | Contour detection method, device, equipment and medium for photoetching film inductance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |