CN115690767A - License plate recognition method and device, unmanned aerial vehicle and storage medium - Google Patents
License plate recognition method and device, unmanned aerial vehicle and storage medium Download PDFInfo
- Publication number
- CN115690767A CN115690767A CN202211315086.8A CN202211315086A CN115690767A CN 115690767 A CN115690767 A CN 115690767A CN 202211315086 A CN202211315086 A CN 202211315086A CN 115690767 A CN115690767 A CN 115690767A
- Authority
- CN
- China
- Prior art keywords
- license plate
- detection frame
- vehicle
- image
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
Abstract
The application provides a license plate recognition method and device, an unmanned aerial vehicle and a storage medium. The license plate recognition method comprises the following steps: detecting an image to be identified containing a vehicle, which is acquired by an image acquisition device, to obtain a detection frame corresponding to the vehicle; adjusting the position and the size of the detection frame to obtain a target frame containing a license plate; carrying out target tracking processing on the target frame, and increasing the optical zoom multiple of the image acquisition equipment in the process of carrying out target tracking processing on the target frame; and identifying the license plate content according to the image acquired by the image acquisition equipment after the optical zoom multiple is increased. According to the method and the device, the license plate content can be prevented from being identified unsuccessfully after the image acquisition equipment zooms and amplifies, and the license plate content can be identified successfully.
Description
Technical Field
The application relates to the technical field of intelligent transportation, in particular to a license plate recognition method and device, an unmanned aerial vehicle and a storage medium.
Background
In the field of intelligent transportation, images can be collected through image collection equipment, and license plate content in the images can be identified. The low image resolution can cause the license plate content to be unrecognizable, so that the image acquisition equipment can be zoomed and amplified to ensure that the license plate resolution is enough for license plate recognition. However, after the image acquisition device zooms and magnifies, the shot picture is limited, and the license plate may appear, so that the license plate content recognition fails.
Disclosure of Invention
One objective of the present application is to provide a license plate recognition method, a license plate recognition device, an unmanned aerial vehicle, and a storage medium, which can avoid failure of license plate content recognition after zooming and amplifying of an image acquisition device.
According to an aspect of the embodiments of the present application, a license plate recognition method is provided, including:
detecting an image to be identified, which is acquired by image acquisition equipment and contains a vehicle, to obtain a detection frame corresponding to the vehicle;
adjusting the position and the size of the detection frame to obtain a target frame containing a license plate;
carrying out target tracking processing on the target frame, and increasing the optical zoom multiple of the image acquisition equipment in the process of carrying out target tracking processing on the target frame;
and recognizing the license plate content according to the image acquired by the image acquisition equipment after the optical zoom multiple is increased.
According to an aspect of an embodiment of the present application, there is disclosed a license plate recognition apparatus including:
the detection module is used for detecting the image to be identified which is acquired by the image acquisition equipment and contains the vehicle to obtain a detection frame corresponding to the vehicle;
the adjusting module is used for adjusting the position and the size of the detection frame to obtain a target frame containing a license plate;
the tracking module is used for carrying out target tracking processing on the target frame and increasing the optical zoom multiple of the image acquisition equipment in the process of carrying out target tracking processing on the target frame;
and the recognition module is used for recognizing the license plate content according to the image collected by the image collection equipment after the optical zoom multiple is increased.
In some embodiments of the present application, based on the above technical solutions, the adjusting module includes:
the target detection module is used for carrying out target detection on the detection frame to obtain a license plate position detection result;
the position adjusting module is used for adjusting the position of the detection frame according to the license plate position detection result;
and the size adjusting module is used for adjusting the size of the detection frame according to a preset proportion to obtain a target frame containing the license plate.
In some embodiments of the present application, based on the above technical solutions, the position adjustment module includes:
and the first moving module is used for acquiring the pixel coordinate of the license plate and moving the central point of the detection frame to the position of the pixel coordinate of the license plate if the license plate in the detection frame is successfully detected as the license plate position detection result.
In some embodiments of the present application, based on the above technical solutions, the position adjustment module includes:
and the second moving module is used for moving the position of the detection frame according to a preset moving amount if the license plate position detection result indicates that the license plate in the detection frame is not detected.
In some embodiments of the present application, based on the above technical solutions, the second moving module includes:
the central point acquisition module is used for acquiring the central point of the detection frame;
and the longitudinal and transverse moving module is used for moving the central point of the detection frame to a position which is moved by a preset moving amount in the longitudinal direction and keeping the position of the detection frame in the transverse direction unchanged.
In some embodiments of the present application, based on the above technical solutions, the tracking module includes:
the current height acquisition module is used for acquiring the current height of the unmanned aerial vehicle in the process of carrying out target tracking processing on the target frame;
the zoom factor acquisition module is used for acquiring a preset zoom factor corresponding to the height; the preset zoom multiple is a preset zoom multiple which is determined in advance that the acquired image completely contains the license plate and meets the definition requirement under the height;
and the zooming multiple adjusting module is used for adjusting the optical zooming multiple of the pod to the preset zooming multiple.
In some embodiments of the present application, based on the above technical solutions, the detection module includes:
and the violation vehicle detection module is used for identifying the violation vehicle in the image to be identified, which contains the vehicle and is acquired by the image acquisition equipment, and obtaining a detection frame corresponding to the violation vehicle.
According to an aspect of the embodiments of the present application, there is provided an unmanned aerial vehicle, including: one or more processors; a storage device for storing one or more programs that, when executed by the one or more processors, cause the drone to implement the methods provided in the various alternative implementations described above.
According to an aspect of embodiments of the present application, there is provided a computer program medium having stored thereon computer readable instructions, which, when executed by a processor of a computer, cause the computer to perform the method provided in the above various alternative implementations.
According to an aspect of embodiments herein, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the various alternative implementations described above.
In the embodiment of the application, the target frame containing the license plate is obtained by adjusting the position and the size of the detection frame corresponding to the vehicle in the image to be recognized. And then in the process of tracking the target frame, the optical zoom multiple of the image acquisition equipment is increased, and the license plate content is identified through the image acquired after the optical zoom multiple is adjusted. The moving target frame containing the license plate is tracked, so that the license plate picture is prevented from being lost after the original detection frame is tracked and optically zoomed and amplified; and the size of the detection frame is adjusted to avoid license plate picture loss caused by target tracking failure due to excessive interference pictures contained in the collected images after the license plate is moved. Therefore, in the tracking process, the images collected by the image collection equipment after optical zooming and amplification comprise the images of the license plate, the license plate content can be further successfully identified according to the images comprising the license plate, and the license plate content can be successfully identified after the image collection equipment is subjected to optical zooming and amplification.
Other features and advantages of the present application will be apparent from the following detailed description, or may be learned by practice of the application.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
Fig. 1 is a schematic flow chart illustrating a license plate recognition method according to an embodiment of the present application.
Fig. 2 shows a schematic diagram of a detection frame in an image to be recognized according to an embodiment of the application.
Fig. 3 shows a schematic diagram of the detection frame according to the embodiment of the present application after moving downward.
Fig. 4 shows a schematic diagram of a detection frame according to an embodiment of the present application after moving down and shrinking.
Fig. 5 is a schematic diagram illustrating an image collected after an optical zoom factor is enlarged in a target tracking process according to an embodiment of the present application.
Fig. 6 shows a detailed flow chart of unmanned detection of license plate number according to the embodiment of the application.
Fig. 7 is a schematic structural diagram of a license plate recognition device according to an embodiment of the present application.
Fig. 8 shows a schematic structural diagram of a drone according to one embodiment of the present application.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The drawings are merely schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more example embodiments. In the following description, numerous specific details are provided to give a thorough understanding of example embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, steps, and so forth. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.
Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
Fig. 1 is a schematic flow chart illustrating a license plate recognition method according to an embodiment of the present application. The license plate recognition method includes steps S101 to S104. The following describes steps S101 to S104 in detail.
Step S101: and detecting the image to be identified containing the vehicle, which is acquired by the image acquisition equipment, to obtain a detection frame corresponding to the vehicle.
In this embodiment, the execution subject may be various license plate recognition devices capable of performing target tracking on a vehicle and adjusting an optical zoom factor in the process of tracking the vehicle target. A license plate recognition device such as a drone.
In the field of intelligent transportation, images collected by a camera used for license plate recognition have resolution limit, the size of the license plate in the images is small, the region of the license plate in the images is not easy to recognize, and the recognition is possible to succeed or fail. In case of failed recognition, the related art cannot recognize the license plate content.
In addition, even if the license plate is successfully recognized, the existence of the license plate in the image is often detected only based on a classification algorithm. And because the resolution of the license plate region in the image is limited, the license plate content is still difficult to identify.
Therefore, in the embodiment, the optical zoom factor of the image acquisition equipment is amplified no matter whether the license plate is successfully detected or not, so that a clearer license plate picture is obtained. And even if the license plate is detected, the size of the license plate is small, and the tracking loss is easily caused by directly tracking the license plate. Therefore, in this embodiment, the optical zoom factor is enlarged while tracking the target frame.
In the process of enlarging the optical zoom factor, since the vehicle is moving, the optical zoom factor is also enlarged while tracking the vehicle. If the optical zoom factor is directly amplified without adjusting the detection frame, the acquired picture is a part close to the central area of the vehicle because the visual field is reduced after the picture is amplified, and at the moment, the picture of the license plate is not acquired because the license plate is positioned below the vehicle in the shot image. And under the condition that the license plate is displayed, the content of the license plate cannot be identified. Therefore, the embodiment provides a method for successfully identifying the license plate content.
In one embodiment of the application, the license plate recognition method can be applied to a violation license plate recognition scene to recognize the license plate content of the violation vehicle. The image to be identified who contains the vehicle that gathers to image acquisition equipment detects, obtains the detection frame that the vehicle corresponds, includes: and identifying the violation vehicles in the to-be-identified image containing the vehicles, which is acquired by the image acquisition equipment, and obtaining the detection frames corresponding to the violation vehicles. Therefore, the license plate content of the violation vehicle can be further identified by combining the following steps.
Step S102: and adjusting the position and the size of the detection frame to obtain a target frame containing the license plate.
In order to successfully recognize the license plate content, the position and the size of the detection frame corresponding to the vehicle are adjusted and processed.
The reason for adjusting the position of the detection frame is: the detection frame is a detection frame of a vehicle, but license plate content is actually required to be acquired, and in order to acquire the license plate content, a picture of a license plate needs to be amplified. If the position of the detection frame is directly used for tracking and amplifying the image, the center of the vehicle is located at the center of the image, and when the optical zoom multiple of the image acquisition equipment is amplified, the license plate of the vehicle is far away from the center of the image, and the license plate may appear on the image. Therefore, the position of the detection frame needs to be moved to the position of the license plate.
The reason for adjusting the size of the detection frame is as follows:
on one hand, after the detection frame is moved, the detection frame comprises a picture below a vehicle picture, the part of the picture is mainly a ground background, the ground background is used as an interference factor to cause interference on subsequent target tracking, tracking loss is easily caused, and the license plate picture cannot be detected after tracking loss, so that license plate content identification failure is caused.
On the other hand, the occupation ratio of the license plate area in the detection frame is too small to be easily recognized, and only the size of the detection frame is adjusted. In the process of adjusting the size of the detection frame, the adjustment of the zoom multiple is not involved, so that the size of the vehicle and the size of the license plate in the shot picture are not changed, and the occupation ratio of the license plate in the detection frame is actually adjusted.
Therefore, on the basis of moving the position of the detection frame, the size of the detection frame needs to be adjusted, so that the license plate content can be successfully identified. The size of the detection frame is adjusted to be mainly reduced. In an embodiment of the present application, adjusting the position and size of the detection frame to obtain a target frame including a license plate includes: and moving the position of the detection frame according to the preset movement amount, and adjusting the size of the detection frame according to a preset proportion to obtain a target frame containing the license plate.
The preset movement amount is a position movement amount of the detection frame set in advance. In the process of shooting, there are two shooting modes: the method comprises the steps that tracking shooting is carried out on a vehicle head and tracking shooting is carried out on a vehicle tail, and no matter a license plate of the vehicle head is shot or the license plate of the vehicle tail is shot, the license plate is located below the vehicle in a shot image containing the vehicle. Although the vehicle head and the vehicle tail of the vehicle are provided with the license plates and the vehicle head or the vehicle tail can be shot, the license plates close to one side of the image acquisition equipment are distributed below the vehicle in the image because the visual angle of the image acquisition equipment faces the vehicle; the license plate that keeps away from image acquisition equipment one side correspondence can distribute in the top of vehicle in the image, and the license plate is the back towards the one side of image acquisition equipment this moment to the license plate often can be sheltered from by the automobile body, is difficult to normally shoot the license plate that is arranged in the image above the vehicle. Therefore, in the captured image including the vehicle, the license plate is located below the vehicle. Therefore, the position of the detection frame of the vehicle can be moved downward by the preset movement amount, which can be obtained by previously measuring the relative distance between the license plate positions of a large number of samples and the position of the detection frame of the vehicle. The preset proportion is a ratio of a preset size of the target frame to a preset size of the detection frame.
By adopting the mode, the position and the size of the detection frame are directly adjusted without detecting whether the license plate is successfully detected in the image to be recognized, so that the target frame containing the license plate is obtained, the efficiency is high, and the target frame containing the license plate can be quickly obtained.
In an embodiment of the present application, adjusting the position and size of the detection frame to obtain a target frame including a license plate includes: performing target detection on the detection frame to obtain a license plate position detection result; adjusting the position of the detection frame according to the detection result of the license plate position; and adjusting the size of the detection frame according to a preset proportion to obtain a target frame containing the license plate.
And the detection frame is subjected to target detection, the license plate is mainly used as a target to be detected in the detection frame, and the license plate position detection result comprises the successful detection of the license plate and the unsuccessful detection of the license plate. The position adjustment mode of the detection frame corresponding to the successfully detected license plate is different from the position adjustment mode of the detection frame corresponding to the unsuccessfully detected license plate. By adopting the mode, the position of the detection frame can be further adjusted in a targeted manner, and the position adjustment requirements corresponding to different license plate position detection results are matched.
In an embodiment of the present application, adjusting the size of the detection frame according to a preset ratio includes: and reducing the size of the detection frame according to a preset proportion. The reduction is relative to the original size of the detection frame. If the ratio obtained by dividing the size of the target frame by the size of the detection frame is taken as a preset ratio, the preset ratio is smaller than 1. If the ratio obtained by dividing the size of the detection frame by the size of the target frame is taken as a preset ratio, the preset ratio is larger than 1. By adopting the method, the interference of backgrounds such as the ground and the like on target tracking is reduced, the occupation ratio of the license plate area in the target frame is increased, and the tracking accuracy is improved.
In an embodiment of the present application, reducing the size of the detection frame according to a preset ratio includes: and reducing the height of the detection frame according to a preset proportion, and keeping the width of the detection frame unchanged.
In an embodiment of the present application, adjusting the size of the detection frame according to a preset ratio to obtain a target frame including a license plate includes: and acquiring a preset proportion matched with the license plate position result, and adjusting the size of the detection frame according to the preset proportion matched with the license plate position result to obtain a target frame containing the license plate. The preset proportion matched with the successfully detected license plate position is smaller than the preset proportion matched with the unsuccessfully detected license plate position. By the method, more accurate target tracking can be performed under the condition that the license plate is successfully detected, and the license plate can be contained in the target frame under the condition that the license plate is not successfully detected.
In an embodiment of the present application, adjusting the position of the detection frame according to the detection result of the license plate position includes: and if the license plate position detection result is that the license plate in the detection frame is successfully detected, acquiring the pixel coordinates of the license plate, and moving the central point of the detection frame to the position of the pixel coordinates of the license plate.
And under the condition that the license plate is successfully detected, the position of the license plate can be further determined, so that the position of the license plate can be directly obtained, and the position of the detection frame is moved to the position of the license plate. The license plate position is a pixel coordinate of the license plate, which may be specifically a pixel coordinate selected as a reference point from the license plate, for example, a pixel coordinate of a center point of the license plate. And acquiring the central point of the detection frame, and moving the central point of the detection frame to the position of the pixel coordinate of the license plate, namely, in the moving process, the position of the detection frame can be moved by taking the central point of the detection frame as a reference point.
In an embodiment of the application, in the case that the license plate position is successfully detected, in the moving process, the positions of the center point of the detection frame and the pixel coordinates of the license plate position in the transverse direction are the same or slightly changed, and the movement is mainly reflected in the position movement in the longitudinal direction.
The detection frame and the target frame are generally rectangular in shape. However, if other shapes of target frames are adopted, the position and size of the detection frame can be adjusted in a similar manner to obtain the target frame. In addition, the reference point for indicating the position of the detection frame during the movement may be a point other than the center point. In addition, the detection frame can be moved properly in the transverse direction and the longitudinal direction so as to adapt to the situation that the ground background exists on the left side and the right side of the detection frame when the vehicle inclination degree is relatively high.
In an embodiment of the present application, adjusting the position of the detection frame according to the detection result of the license plate position includes: and if the license plate position detection result indicates that the license plate in the detection frame is not detected, moving the position of the detection frame according to the preset movement amount. By adopting the method, even if the license plate is not successfully detected, the target frame containing the license plate can be obtained.
Under the condition that the license plate is not successfully detected, the position of the license plate cannot be directly determined, and the target frame containing the license plate is obtained mainly based on the following principle: in the collected picture, the license plate is positioned below the vehicle. The position of the license plate relative to the vehicle is measured in various vehicle types and in an unmanned aerial vehicle shooting mode, so that the relative position information of the license plate and the vehicle under the common condition can be obtained, the relative position information is stored, and the relative position information is the preset movement amount. After the detection frame is obtained, the vehicle type of the vehicle can be obtained, if the vehicle head is tracked or the vehicle tail is tracked and shot according to the shooting mode of the unmanned aerial vehicle, the vehicle type and the corresponding preset movement amount under the shooting mode can be obtained (it needs to be pointed out that the preset movement amount when the vehicle head is tracked and the vehicle tail is tracked can be the same or different, the vehicle head can be moved according to the preset movement amount without determining the shooting mode of the unmanned aerial vehicle when the vehicle head is the same), the detection frame is moved according to the preset movement amount, and the detection frame after the position is adjusted can be obtained. In one embodiment of the present application, moving the position of the detection frame according to a preset movement amount includes: acquiring a central point of the detection frame; and moving the central point of the detection frame to a position which is moved by a preset movement amount in the longitudinal direction, and keeping the position of the detection frame in the transverse direction unchanged. By adopting the method, the position of the detection frame can be adjusted under the condition that the license plate is not detected, and the target frame containing the license plate is obtained.
In an embodiment of the present application, the preset moving amount may be a distance obtained by proportionally adjusting a vertical coordinate of a center point of the detection frame. For example, if the coordinates (x, y) of the center point of the frame are detected, the predetermined movement amount is ay, and a is the ratio. The direction of movement is downward. This ratio may be adjusted on a case-by-case basis, for example, but is not limited thereto, and may be set to about 35% as measured most preferably in the downward movement.
In an embodiment of the present application, the degree of reducing the detection frame when the license plate is successfully detected may be slightly greater than the degree of reducing the detection frame when the license plate is not detected. The reason for this is that when a license plate is successfully detected, the position of the license plate is relatively more accurate, so that the picture can be greatly reduced to reduce other interference pictures as much as possible, and when a license plate is not successfully detected, the picture can be reduced to a relatively smaller extent so that the reduced picture contains the license plate. For example, the horizontal distance of the detection frame may be reduced to 50% of the original horizontal distance when the license plate is successfully detected, and the horizontal distance of the detection frame may be reduced to 70% of the original horizontal distance when the license plate is not successfully detected.
In one embodiment of the present application, the preset moving amount may also be a fixed moving distance. For example, the size of the vehicle may be estimated according to the width to height ratio of the detection frame, and a moving distance matching the size may be selected from a plurality of preset moving distances according to the size of the vehicle, so as to move directly according to the moving distance.
It should be noted that, in the process of adjusting the position and size of the detection frame, the size of the detection frame may be adjusted first, and after the size of the detection frame is adjusted, the position of the detection frame may be adjusted.
Step S103: and carrying out target tracking processing on the target frame, and increasing the optical zoom multiple of the image acquisition equipment in the process of carrying out the target tracking processing on the target frame.
Since the vehicle is moving, it is necessary to adjust the optical zoom factor while tracking the target. And in the target tracking process, amplifying the optical zoom multiple of the image acquisition equipment.
In one embodiment of the present application, the image capture device is a pod of the drone. The unmanned aerial vehicle can shoot images through road cruising, whether the images contain vehicles or not is identified, and the images containing the vehicles are used as images to be identified.
As an optional implementation manner, in the process of performing target tracking processing on the target frame, adjusting the optical zoom multiple of the image capturing device to be larger includes: and in the process of carrying out target tracking processing on the target frame, adjusting the optical zoom multiple of the image acquisition equipment to a preset zoom multiple. The preset zoom factor may be a maximum zoom factor that the image capturing apparatus can support.
In the above embodiment, the flying height of the unmanned aerial vehicle during cruising on the road is fixed, that is, the flying height of the unmanned aerial vehicle for license plate recognition is fixed. The height which is most suitable for license plate recognition of the unmanned aerial vehicle can be obtained in advance through measurement, and the unmanned aerial vehicle is kept at the height. In this case, when the optical zoom factor is adjusted, the optical zoom factor can be directly adjusted to the maximum optical zoom factor that can be supported by the image capture device, so as to obtain the license plate image with the highest definition.
By adopting the mode, in the process that the unmanned aerial vehicle tracks the target frame, the pod angle is adjusted according to the target frame, so that after the pod magnifies the optical zoom factor, the acquired image contains the license plate picture, the process only relates to the adjustment of the pod angle, the unmanned aerial vehicle does not need to be moved to the position near the license plate, and the license plate content recognition can be efficiently and simply realized.
In an embodiment of the present application, the image capturing device is a pod of an unmanned aerial vehicle, and the adjusting of the optical zoom factor of the image capturing device in the process of performing target tracking processing on the target frame includes: acquiring the current height of the unmanned aerial vehicle in the process of carrying out target tracking processing on the target frame; acquiring a preset zooming multiple corresponding to the height; the preset zoom multiple is the zoom multiple which is determined in advance that the acquired image completely contains the license plate and meets the definition requirement under the height; the optical zoom factor of the pod is adjusted to a preset zoom factor.
In the above embodiment, the flying height of the unmanned aerial vehicle may change. In this case, the adjusting the optical zoom factor of the pod during the target tracking process for the target frame includes: acquiring the current height of the unmanned aerial vehicle in the process of carrying out target tracking processing on the target frame; acquiring a preset zooming multiple corresponding to the height; the preset zoom multiple is a zoom multiple which is determined in advance that the acquired image completely contains the license plate and meets the definition requirement under the height; and adjusting the optical zoom multiple of the image acquisition equipment to a preset zoom multiple.
The height of the unmanned aerial vehicle is positively correlated with the optical zoom factor. The greater the height at which the drone is located, and thus the greater the optical zoom factor. At the moment, clear pictures containing license plates can be obtained according to the difference of the heights of the unmanned aerial vehicles. The corresponding relation between the height and the preset zooming multiple can be determined in advance under the condition that the image meets the definition requirement and contains the license plate picture, and the corresponding preset zooming multiple is obtained by combining the height of the unmanned aerial vehicle according to the corresponding relation.
Step S104: and identifying the license plate content according to the image acquired by the image acquisition equipment after the optical zoom multiple is increased.
The license plate content recognition mainly recognizes the license plate number.
In an embodiment of the application, in the process of license plate content identification according to an image acquired by an image acquisition device after the optical zoom multiple is increased, a license plate area picture in the acquired image is detected first, and license plate content identification is performed on the license plate area picture.
In a specific scenario, the process of recognizing license plate content is described with reference to fig. 2-5. Fig. 2 is a schematic diagram of an image to be recognized. Fig. 3 is a schematic diagram of the detection frame after moving downward in the image to be recognized. FIG. 4 is a schematic diagram of a reduced detection frame. Fig. 5 is a schematic diagram of an image acquired by the image acquisition device after the optical zoom factor is enlarged in the target tracking process.
Referring to fig. 2, the unmanned aerial vehicle pod tracks and shoots the head portion of the vehicle, detects an image to be recognized, and obtains a detection frame. In addition, the detection result indicates that the license plate is not successfully identified. At this time, the detection frame is moved downward at a certain ratio to obtain the position of the detection frame as shown in fig. 3. After moving downward, the detection frame is reduced to obtain the size of the detection frame as shown in fig. 4, and a final target frame is obtained. And tracking the target frame, and increasing the optical zoom multiple of the image acquisition equipment in the process of tracking the target frame. Acquiring an image acquired by the image acquisition device after the optical zoom multiple is increased, and performing license plate content recognition on the image to obtain a license plate number as follows as shown in fig. 5: jing XXXXXXX.
By adopting the mode, the image collected after the optical zoom multiple is increased is a clearer picture containing the license plate, so that the content of the license plate can be successfully identified.
In the following, referring to fig. 6, the technical scheme of the application is elaborated in detail by combining with a specific implementation mode of detecting the license plate number by the unmanned aerial vehicle in the scene of detecting the vehicle against the regulations.
Obtaining a detection frame: and detecting the violation vehicle through the shot image to obtain a detection frame of the violation vehicle, wherein the center coordinate of the detection frame is (1000,500), and the size of the detection frame is 100x500.
Detecting a license plate: whether the license plate exists or not can be identified through image classification. At the moment, the size of the license plate is small, and the detection may be successful or fail. And adjusting the detection frame according to a corresponding adjustment mode according to the success or failure of the detection.
Adjusting a detection frame:
if the license plate is successfully detected, the center coordinate of the license plate is detected to be (1000,700) and the size is 20x10, the position of the detection frame is adjusted by the center of the license plate, the position of the detection frame is adjusted to be (1000,700) according to the center coordinate of the detection frame, the height of the detection frame is reduced according to a certain proportion, namely the reduction proportion is 50%, and the size obtained after the reduction is 100x250. The width is unchanged and the height is halved.
Namely: cx =1000, cy =700, w =100, h =500x50% =250.cx is the abscissa, cy is the ordinate, w is the width and h is the height.
If the license plate detection fails, the license plate is generally positioned below the detection frame according to experience, and the height of the detection frame is reduced and moved downwards according to a certain proportion. Wherein, the detection frame is shifted down by 35% and reduced by 70%, and the adjusted center coordinate of the detection frame is (1000,675) and the size is 100x350. And enabling the license plate to be close to the center of the detection frame.
Namely: cx =1000, cy =500+500x35% =675, w =100, h =500x70% =375.
Starting tracking: and initializing target tracking by using the adjusted detection frame, namely the target frame.
Zooming the camera: the camera optics disposed in the drone pod is zoomed up to a number of times detectable by the license plate, such as 30 times.
Detecting a license plate: the coordinates of the center of the license plate frame are obtained to be (1000,500) and the size is 200x50.
And (3) identifying the license plate: obtaining a license plate number: "Jingxxxxxx".
By adopting the mode, the license plate is still in the picture after the optical zoom factor is amplified, so that the license plate content is successfully identified.
Fig. 7 illustrates a license plate recognition device according to an embodiment of the present application, the license plate recognition device including:
according to an aspect of an embodiment of the present application, there is disclosed a license plate recognition apparatus including:
the detection module 301 is configured to detect an image to be identified, which includes a vehicle and is acquired by an image acquisition device, to obtain a detection frame corresponding to the vehicle;
the adjusting module 302 is configured to adjust the position and size of the detection frame to obtain a target frame including a license plate;
the tracking module 303 is configured to perform target tracking processing on the target frame, and increase an optical zoom multiple of the image acquisition device in the process of performing the target tracking processing on the target frame;
and the recognition module 304 is used for recognizing the license plate content according to the image collected by the image collection equipment after the optical zoom multiple is increased.
In some embodiments of the present application, based on the above technical solutions, the adjusting module includes:
the target detection module is used for carrying out target detection on the detection frame to obtain a license plate position detection result;
the position adjusting module is used for adjusting the position of the detection frame according to the license plate position detection result;
and the size adjusting module is used for adjusting the size of the detection frame according to a preset proportion to obtain a target frame containing the license plate.
In some embodiments of the present application, based on the above technical solutions, the position adjustment module includes:
and the first moving module is used for acquiring the pixel coordinates of the license plate and moving the central point of the detection frame to the pixel coordinates of the license plate if the license plate in the detection frame is successfully detected as the license plate position detection result.
In some embodiments of the present application, based on the above technical solutions, the position adjustment module includes:
and the second moving module is used for moving the position of the detection frame according to the preset movement amount if the license plate position detection result indicates that the license plate in the detection frame is not detected.
In some embodiments of the present application, based on the above technical solutions, the second moving module includes:
the central point acquisition module is used for acquiring the central point of the detection frame;
and the longitudinal and transverse moving module is used for moving the central point of the detection frame to a position which is moved by a preset moving amount in the longitudinal direction and keeping the position of the detection frame in the transverse direction unchanged.
In some embodiments of the present application, based on the above technical solutions, the tracking module includes:
the current height acquisition module is used for acquiring the current height of the unmanned aerial vehicle in the process of carrying out target tracking processing on the target frame;
the zoom factor acquisition module is used for acquiring a preset zoom factor corresponding to the height; the preset zoom multiple is the zoom multiple which is determined in advance that the acquired image completely contains the license plate and meets the definition requirement under the height;
and the zooming multiple adjusting module is used for adjusting the optical zooming multiple of the pod to a preset zooming multiple.
In some embodiments of the present application, based on the above technical solutions, the detection module includes:
and the violation vehicle detection module is used for identifying the violation vehicle in the image to be identified, which contains the vehicle and is acquired by the image acquisition equipment, so as to obtain a detection frame corresponding to the violation vehicle.
The drone 40 according to an embodiment of the present application is described below with reference to fig. 8. The drone 40 shown in fig. 8 is merely an example, and should not bring any limitations to the functionality and scope of use of the embodiments of the present application.
As shown in fig. 8, the drone 40 is in the form of a general purpose computing device. The components of the drone 40 may include, but are not limited to: the at least one processing unit 410, the at least one memory unit 420, and a bus 430 that couples various system components including the memory unit 420 and the processing unit 410.
Wherein the storage unit stores a program code, which can be executed by the processing unit 410, to cause the processing unit 410 to perform the steps according to various exemplary embodiments of the present application described in the description part of the above exemplary methods of the present specification. For example, processing unit 410 may perform various steps as shown in fig. 1.
The storage unit 420 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM) 4201 and/or a cache memory unit 4202, and may further include a read only memory unit (ROM) 4203.
The storage unit 420 may also include a program/utility 4204 having a set (at least one) of program modules 4205, such program modules 4205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a drone to execute the method according to the embodiments of the present application.
In an exemplary embodiment of the present application, there is also provided a computer-readable storage medium having stored thereon computer-readable instructions which, when executed by a processor of a computer, cause the computer to perform the method described in the above method embodiment section.
According to an embodiment of the present application, there is also provided a program product for implementing the method in the above method embodiment, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a drone. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as JAVA, C + +, or the like, as well as conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Moreover, although the steps of the methods herein are depicted in the drawings in a particular order, this does not require or imply that the steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken into multiple step executions, etc.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a drone to execute the method according to the embodiments of the present application.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
Claims (10)
1. A license plate recognition method is characterized by comprising the following steps:
detecting an image to be identified containing a vehicle, which is acquired by an image acquisition device, to obtain a detection frame corresponding to the vehicle;
adjusting the position and the size of the detection frame to obtain a target frame containing a license plate;
carrying out target tracking processing on the target frame, and increasing the optical zoom multiple of the image acquisition equipment in the process of carrying out target tracking processing on the target frame;
and identifying the license plate content according to the image acquired by the image acquisition equipment after the optical zoom multiple is increased.
2. The method of claim 1, wherein adjusting the position and size of the detection frame to obtain a target frame including a license plate comprises:
carrying out target detection on the detection frame to obtain a license plate position detection result;
adjusting the position of the detection frame according to the license plate position detection result;
and adjusting the size of the detection frame according to a preset proportion to obtain a target frame containing the license plate.
3. The method of claim 2, wherein adjusting the position of the detection frame according to the license plate position detection result comprises:
and if the license plate position detection result is that the license plate in the detection frame is successfully detected, acquiring the pixel coordinate of the license plate, and moving the central point of the detection frame to the position of the pixel coordinate of the license plate.
4. The method of claim 2, wherein adjusting the position of the detection frame according to the license plate position detection result comprises:
and if the license plate position detection result indicates that the license plate in the detection frame is not detected, moving the position of the detection frame according to a preset movement amount.
5. The method of claim 4, wherein moving the position of the detection frame according to a preset amount of movement comprises:
acquiring a central point of the detection frame;
and moving the central point of the detection frame to a position which is moved by a preset movement amount in the longitudinal direction, and keeping the position of the detection frame in the transverse direction unchanged.
6. The method of claim 1, wherein the image capturing device is a pod of a drone, and wherein adjusting the optical zoom factor of the image capturing device during the target tracking process for the target frame comprises:
acquiring the current height of the unmanned aerial vehicle in the process of carrying out target tracking processing on the target frame;
acquiring a preset zooming multiple corresponding to the height; the preset zoom multiple is a preset zoom multiple which is determined in advance that the acquired image completely contains the license plate and meets the definition requirement under the height;
adjusting the optical zoom factor of the pod to the preset zoom factor.
7. The method according to any one of claims 1 to 6, wherein detecting the image to be identified including the vehicle, which is acquired by the image acquisition device, to obtain the detection frame corresponding to the vehicle comprises:
and identifying the violation vehicle in the image to be identified, which contains the vehicle and is acquired by the image acquisition equipment, and obtaining a detection frame corresponding to the violation vehicle.
8. A license plate recognition device, comprising:
the detection module is used for detecting the image to be identified which is acquired by the image acquisition equipment and contains the vehicle to obtain a detection frame corresponding to the vehicle;
the adjusting module is used for adjusting the position and the size of the detection frame to obtain a target frame containing a license plate;
the tracking module is used for carrying out target tracking processing on the target frame and increasing the optical zoom multiple of the image acquisition equipment in the process of carrying out target tracking processing on the target frame;
and the recognition module is used for recognizing the license plate content according to the image collected by the image collection equipment after the optical zoom multiple is increased.
9. An unmanned aerial vehicle, comprising:
one or more processors;
storage means for storing one or more programs that, when executed by the one or more processors, cause the drone to implement the method of any one of claims 1-7.
10. A computer-readable storage medium having stored thereon computer-readable instructions which, when executed by a processor of a computer, cause the computer to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211315086.8A CN115690767B (en) | 2022-10-26 | 2022-10-26 | License plate recognition method, license plate recognition device, unmanned aerial vehicle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211315086.8A CN115690767B (en) | 2022-10-26 | 2022-10-26 | License plate recognition method, license plate recognition device, unmanned aerial vehicle and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115690767A true CN115690767A (en) | 2023-02-03 |
| CN115690767B CN115690767B (en) | 2023-08-22 |
Family
ID=85098745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211315086.8A Active CN115690767B (en) | 2022-10-26 | 2022-10-26 | License plate recognition method, license plate recognition device, unmanned aerial vehicle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115690767B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102867416A (en) * | 2012-09-13 | 2013-01-09 | 中国科学院自动化研究所 | Vehicle part feature-based vehicle detection and tracking method |
| CN103426185A (en) * | 2013-08-09 | 2013-12-04 | 北京博思廷科技有限公司 | Method and device for adjusting target scale in pan-tilt-zoom (PTZ) tracking process |
| CN107145487A (en) * | 2016-03-01 | 2017-09-08 | 深圳中兴力维技术有限公司 | Image search method and device |
| KR20190020304A (en) * | 2019-02-13 | 2019-02-28 | 엘지이노텍 주식회사 | Apparatus and method for recognizing vehicle location information using number plate recognition |
| CN109949364A (en) * | 2019-04-01 | 2019-06-28 | 上海淞泓智能汽车科技有限公司 | A kind of vehicle attitude detection accuracy optimization method based on drive test monocular cam |
| CN111582263A (en) * | 2020-05-12 | 2020-08-25 | 上海眼控科技股份有限公司 | License plate recognition method and device, electronic equipment and storage medium |
| CN111709416A (en) * | 2020-05-15 | 2020-09-25 | 珠海亿智电子科技有限公司 | License plate positioning method, device and system and storage medium |
| CN112489077A (en) * | 2019-09-12 | 2021-03-12 | 阿里巴巴集团控股有限公司 | Target tracking method and device and computer system |
| WO2021169964A1 (en) * | 2020-02-24 | 2021-09-02 | 华为技术有限公司 | Target detection method and related device |
| CN114220087A (en) * | 2021-11-22 | 2022-03-22 | 浙江大华技术股份有限公司 | License plate detection method, license plate detector and related equipment |
| CN114373152A (en) * | 2022-01-07 | 2022-04-19 | 北京远度互联科技有限公司 | Method and device for identifying road violation, electronic equipment and storage medium |
| CN114387533A (en) * | 2022-01-07 | 2022-04-22 | 北京远度互联科技有限公司 | Method and device for identifying road violation, electronic equipment and storage medium |
-
2022
- 2022-10-26 CN CN202211315086.8A patent/CN115690767B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102867416A (en) * | 2012-09-13 | 2013-01-09 | 中国科学院自动化研究所 | Vehicle part feature-based vehicle detection and tracking method |
| CN103426185A (en) * | 2013-08-09 | 2013-12-04 | 北京博思廷科技有限公司 | Method and device for adjusting target scale in pan-tilt-zoom (PTZ) tracking process |
| CN107145487A (en) * | 2016-03-01 | 2017-09-08 | 深圳中兴力维技术有限公司 | Image search method and device |
| KR20190020304A (en) * | 2019-02-13 | 2019-02-28 | 엘지이노텍 주식회사 | Apparatus and method for recognizing vehicle location information using number plate recognition |
| CN109949364A (en) * | 2019-04-01 | 2019-06-28 | 上海淞泓智能汽车科技有限公司 | A kind of vehicle attitude detection accuracy optimization method based on drive test monocular cam |
| CN112489077A (en) * | 2019-09-12 | 2021-03-12 | 阿里巴巴集团控股有限公司 | Target tracking method and device and computer system |
| WO2021169964A1 (en) * | 2020-02-24 | 2021-09-02 | 华为技术有限公司 | Target detection method and related device |
| CN111582263A (en) * | 2020-05-12 | 2020-08-25 | 上海眼控科技股份有限公司 | License plate recognition method and device, electronic equipment and storage medium |
| CN111709416A (en) * | 2020-05-15 | 2020-09-25 | 珠海亿智电子科技有限公司 | License plate positioning method, device and system and storage medium |
| CN114220087A (en) * | 2021-11-22 | 2022-03-22 | 浙江大华技术股份有限公司 | License plate detection method, license plate detector and related equipment |
| CN114373152A (en) * | 2022-01-07 | 2022-04-19 | 北京远度互联科技有限公司 | Method and device for identifying road violation, electronic equipment and storage medium |
| CN114387533A (en) * | 2022-01-07 | 2022-04-22 | 北京远度互联科技有限公司 | Method and device for identifying road violation, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| CHINCHUAN HAN, ET AL: "License Plate Detection and Recognition Using a Dual-Camera Module in a Large Space", 《2007 41ST ANNUAL IEEE INTERNATIONAL CARNAHAN CONFERENCE ON SECURITY TECHNOLOGY》 * |
| 黄勇: "交通违法视频检测关键技术研究及系统实现", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》, vol. 2019, no. 05, pages 034 - 666 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115690767B (en) | 2023-08-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113486797B (en) | Unmanned vehicle position detection method, unmanned vehicle position detection device, unmanned vehicle position detection equipment, storage medium and vehicle | |
| CN110705405B (en) | Target labeling method and device | |
| KR101758684B1 (en) | Apparatus and method for tracking object | |
| EP2164027B1 (en) | Object detecting device, imaging apparatus, object detecting method, and program | |
| US9846823B2 (en) | Traffic lane boundary line extraction apparatus and method of extracting traffic lane boundary line | |
| KR101687530B1 (en) | Control method in image capture system, control apparatus and a computer-readable storage medium | |
| US20070041614A1 (en) | Road marking recognition apparatus and method | |
| US20200219281A1 (en) | Vehicle external recognition apparatus | |
| US10521915B2 (en) | Distance measurement device and distance measurement method | |
| JP2012063869A (en) | License plate reader | |
| CN112004732B (en) | Railway track recognition device | |
| CN109300322B (en) | Guideline drawing method, apparatus, device, and medium | |
| CN109740526B (en) | Signal lamp identification method, device, equipment and medium | |
| JP2013251005A (en) | Image correction method | |
| CN112489240B (en) | Commodity display inspection method, inspection robot and storage medium | |
| JP5442408B2 (en) | Image correction method | |
| CN110176000B (en) | Road quality detection method and device, storage medium and electronic equipment | |
| CN115690767B (en) | License plate recognition method, license plate recognition device, unmanned aerial vehicle and storage medium | |
| CN111951328A (en) | Object position detection method, device, equipment and storage medium | |
| CN113689493A (en) | Lens attachment detection method, lens attachment detection device, electronic equipment and storage medium | |
| JP3244850B2 (en) | License plate reader | |
| CN111626078A (en) | Method and device for identifying lane line | |
| CN113869163B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN113378735B (en) | Road marking line identification method and device, electronic equipment and storage medium | |
| JP2012118656A (en) | Parking frame detection device and parking frame detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |