CN111027534A - Compact double-license-plate detection method and device - Google Patents
Compact double-license-plate detection method and device Download PDFInfo
- Publication number
- CN111027534A CN111027534A CN201811174813.7A CN201811174813A CN111027534A CN 111027534 A CN111027534 A CN 111027534A CN 201811174813 A CN201811174813 A CN 201811174813A CN 111027534 A CN111027534 A CN 111027534A
- Authority
- CN
- China
- Prior art keywords
- license plate
- candidate frame
- candidate
- frame information
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The application provides a compact double-license plate detection method and device, which can acquire a plurality of license plate candidate frame information output by a license plate detection network; determining first license plate candidate frame information with the maximum confidence coefficient from a plurality of pieces of license plate candidate frame information, reducing the confidence coefficient in second license plate candidate frame information when second license plate candidate frame information to be deleted exists in the plurality of pieces of license plate candidate frame information, and inputting the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient of the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information. Therefore, the problem of missing detection of the license plate is avoided, and the detection rate of compact double-license plate detection is improved.
Description
Technical Field
The application relates to the technical field of intelligent traffic, in particular to a compact double-license plate detection method and device.
Background
The license plate is the 'ID card' of the vehicle and is an important information for distinguishing other motor vehicles. The license plate recognition technology is widely applied to scenes such as a gate, a parking lot, an electronic police and the like to acquire license plate information of vehicles in the scenes, and plays the power of an intelligent traffic algorithm in many aspects such as public security management and the like. In the existing deep learning license plate recognition algorithm framework, license plate numbers are firstly detected, and then character recognition is carried out on a license plate frame; thus, license plate detection is the basis for subsequent character recognition.
Most countries and regions around the world have one license plate hung on one vehicle, but some countries or regions (such as hong Kong and Australia) have two license plates hung on one vehicle, and the condition that the license plates are overlapped and the license plates are not overlapped when two license plates are hung on the vehicle is generally called as 'compact double license plate'. The existing deep learning license plate detection algorithm is based on license plate number recognition, but for a compact double license plate, the existing deep learning algorithm can only randomly detect the license plate number corresponding to one license plate, but can not recognize both license plates, so that the recognition result of the compact double license plate is incomplete.
Disclosure of Invention
In view of this, in order to solve the problem of incomplete recognition results of the compact double license plates in the prior art, the present application provides a method and an apparatus for detecting a compact double license plate, which enable a candidate frame of a license plate to be deleted to be input into a license plate detection network for continuous calculation by reducing the confidence of the candidate frame of the license plate to be deleted, and if the final confidence is not less than a preset threshold, the candidate frame of the license plate can be output as information of a target candidate frame of the license plate, thereby avoiding the problem of missing detection in the prior art.
Specifically, the method is realized through the following technical scheme:
according to a first aspect of embodiments of the present application, there is provided a compact two-license plate detection method, the method comprising:
acquiring a plurality of license plate candidate frame information output by a license plate detection network;
determining first license plate candidate frame information with the maximum confidence coefficient from the plurality of license plate candidate frame information; and when second license plate candidate frame information to be deleted exists in the plurality of license plate candidate frame information, reducing the confidence coefficient in the second license plate candidate frame information, and inputting the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient of the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information.
As one embodiment, the license plate detection network includes a first network and a second network that are cascaded;
the method for acquiring the information of the plurality of license plate candidate frames output by the license plate detection network comprises the following steps:
according to the vehicle image input into the first network, feature extraction is carried out by the first network to obtain a feature map, the feature map is input into a second network, and the second network determines a plurality of license plate candidate frame information.
As an embodiment, a method for determining that there is second license plate candidate frame information to be deleted in the plurality of pieces of license plate candidate frame information includes:
combining the plurality of license plate candidate frame information in pairs, and calculating the overlapping area proportion of each group of license plate candidate frame information;
and if the calculated overlapping area ratio is larger than the preset ratio, the license plate candidate frame information with lower confidence coefficient in the group of license plate candidate frame information is used as second license plate candidate frame information to be deleted.
As an embodiment, the license plate candidate frame information includes coordinate information of a license plate candidate frame;
the reducing the confidence level in the second card candidate box information specifically includes:
calculating a new confidence coefficient of the second license plate candidate frame according to the coordinate information of the first license plate candidate frame and the coordinate information of the second license plate candidate frame; the new confidence is less than the current confidence of the second card candidate box.
As one embodiment, a new confidence level S for a second card candidate frame is calculated based on coordinate information of a first card candidate frame and coordinate information of a second card candidate frameNewThe method comprises the following steps:
wherein S isNewNew confidence for the second card candidate box, SOld ageFor the current confidence of a second candidate box, iou (M, b) is the ratio of the overlapping area of the first candidate box and the second candidate box, where M is the coordinate of the first candidate box, b is the coordinate of the second candidate box, and σ is a gaussian parameter, which is adjustable.
As an embodiment, the license plate detection network further includes a third network, and the method further includes:
and inputting the feature map output by the first network and the target license plate candidate frame information output by the second network into a third network, and classifying license plate areas of the target license plate candidate frame by the third network to finally obtain a target license plate area.
According to a second aspect of the embodiments of the present application, there is provided a compact dual license plate detection device, comprising:
the license plate detection device comprises an acquisition unit, a storage unit and a display unit, wherein the acquisition unit is used for acquiring a plurality of license plate candidate frame information output by a license plate detection network;
the calculation unit is used for determining first license plate candidate frame information with the maximum confidence coefficient from the plurality of license plate candidate frame information; and when second license plate candidate frame information to be deleted exists in the plurality of license plate candidate frame information, reducing the confidence coefficient in the second license plate candidate frame information, and inputting the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient of the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information.
As one embodiment, the license plate detection network includes a first network and a second network that are cascaded;
the acquisition unit is specifically used for carrying out feature extraction on the first network according to the vehicle image input into the first network to obtain a feature map, inputting the feature map into a second network, and determining a plurality of license plate candidate frame information by the second network.
As an embodiment, the apparatus further comprises:
the determining unit is used for combining the plurality of license plate candidate frame information in pairs and calculating the overlapping area proportion of each group of license plate candidate frame information; and if the calculated overlapping area ratio is larger than the preset ratio, the license plate candidate frame information with lower confidence coefficient in the group of license plate candidate frame information is used as second license plate candidate frame information to be deleted.
As an embodiment, the license plate candidate frame information includes coordinate information of a license plate candidate frame;
the calculation unit is specifically configured to calculate a new confidence of the second candidate plate frame according to the coordinate information of the first candidate plate frame and the coordinate information of the second candidate plate frame; the new confidence is less than the current confidence of the second card candidate box.
As one embodiment, a new confidence level S for a second card candidate frame is calculated based on coordinate information of a first card candidate frame and coordinate information of a second card candidate frameNewThe method comprises the following steps:
wherein S isNewNew confidence for the second card candidate box, SOld ageFor the current confidence of a second candidate box, iou (M, b) is the ratio of the overlapping area of the first candidate box and the second candidate box, where M is the coordinate of the first candidate box, b is the coordinate of the second candidate box, and σ is a gaussian parameter, which is adjustable.
As an embodiment, the license plate detection network further comprises a third network, the apparatus further comprising:
and the input unit is used for inputting the feature map output by the first network and the target license plate candidate frame information output by the second network into a third network, and the third network classifies license plate areas of the target license plate candidate frame to finally obtain the target license plate area.
According to a third aspect of embodiments of the present application, there is provided an electronic device comprising a processor, a communication interface, a memory, and a communication bus;
the processor, the communication interface and the memory are communicated with each other through the communication bus;
the memory is used for storing a computer program;
the processor is used for executing the computer program stored in the memory, and when the processor executes the computer program, any step of the compact double-license plate detection method is realized.
According to the embodiment, the information of the plurality of license plate candidate frames output by the license plate detection network can be obtained; determining first license plate candidate frame information with the maximum confidence coefficient from a plurality of pieces of license plate candidate frame information, reducing the confidence coefficient in second license plate candidate frame information when the second license plate candidate frame information to be deleted exists in the plurality of pieces of license plate candidate frame information, and inputting the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient of the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information. Compared with the prior art, the confidence coefficient of the second license plate candidate frame to be deleted can be reduced, the second license plate candidate frame is input into the license plate detection network again to be calculated, and if the final confidence coefficient of the second license plate candidate frame is not smaller than the preset threshold value, the second license plate candidate frame can be output, so that the problem of license plate missing detection is avoided, and the detection rate of compact double-license plate detection is improved.
Drawings
FIG. 1 is a schematic diagram of an exemplary license plate candidate box of the present application;
FIG. 2 is a process flow diagram of an exemplary compact dual license plate detection method of the present application;
FIG. 3 is a schematic diagram of an exemplary license plate detection network according to the present application;
FIG. 4-1 is a schematic diagram of a license plate detection result in the prior art;
FIG. 4-2 is a schematic diagram of an exemplary license plate detection result of the present application;
FIG. 5 is a block diagram of one embodiment of a compact dual license plate detection apparatus of the present application;
FIG. 6 is a block diagram of an embodiment of an electronic device of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
The existing compact double-license plate detection method usually deeply learns a license plate detection network (such as fast RCNN) to identify the license plate position, and then uses an NMS (Non Maximum Suppression ) strategy to eliminate redundant (cross-repeat) candidate frames and find the optimal target position. The strategy comprises the following steps: and (3) sequencing the output license plate detection frames according to the confidence degrees, reserving the license plate detection frame with the highest confidence degree, and deleting other frames with the overlapping area proportion larger than a certain proportion with the frame, wherein as shown in fig. 1, a frame B and a frame A are detection results of the license plate position of the target detection network, and the output confidence degrees are 0.92 and 0.88 respectively. Because the confidence coefficient of the box B is higher than that of the box A, according to the strategy, if the overlapping area S of the box B and the box A exceeds a certain NMS threshold value, the box B and the box A are filtered; the threshold of the NMS is not easy to determine, if the threshold is set too large, the threshold cannot play a role in filtering, the same license plate is easy to be detected for many times, and even the risk of false detection is increased; if the threshold value is set too small, the compact overlapped license plates are easy to filter, and the problem of missing detection of the license plates is caused. Therefore, the method for detecting the compact double license plates has the problem of missing detection or false detection, and the detection rate is low.
In order to solve the problem that detection omission or false detection exists when a compact double license plate is detected in the prior art, the method and the device can acquire the information of a plurality of license plate candidate frames output by a license plate detection network; determining first license plate candidate frame information with the maximum confidence coefficient from a plurality of pieces of license plate candidate frame information, reducing the confidence coefficient in second license plate candidate frame information when the second license plate candidate frame information to be deleted exists in the plurality of pieces of license plate candidate frame information, and inputting the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient of the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information. Compared with the prior art, the confidence coefficient of the second license plate candidate frame to be deleted can be reduced, the second license plate candidate frame is input into the license plate detection network again to be calculated, and if the final confidence coefficient of the second license plate candidate frame is not smaller than the preset threshold value, the second license plate candidate frame can be output, so that the problem of license plate missing detection is avoided, and the detection rate of compact double-license plate detection is improved.
As follows, the following embodiments are shown to explain the image processing method provided by the present application.
The first embodiment is as follows:
referring to fig. 2, a flow chart of an exemplary embodiment of a compact double license plate detection method of the present application includes the following steps:
in this embodiment, the compact double-license-plate detection method is mainly applied to an NMS policy module in a license plate detection network model, and the NMS policy module can acquire information of a plurality of license plate candidate frames output by a license plate detection network, where the license plate candidate frame information includes coordinates and confidence of the license plate candidate frames.
As one embodiment, the license plate detection network includes a first network and a second network that are cascaded; specifically, when a license plate image is input to a first network in a license plate detection network, the first network performs feature extraction according to the input vehicle image to obtain a feature map, and then inputs the feature map to a second network, and the second network determines a plurality of license plate candidate frame information according to the feature map.
In this embodiment, the NMS policy module may determine, from the plurality of license plate candidate frames, first license plate candidate frame information with the highest confidence level, specifically, may rank the confidence levels of the plurality of license plate candidate frames, and the license plate candidate frame information with the highest confidence level is used as the first license plate candidate frame information.
The NMS strategy module can also judge whether the license plate candidate frame information to be deleted exists in the plurality of license plate candidate frame information. Specifically, combining the plurality of license plate candidate frame information pairwise, and calculating the overlapping area ratio of each group of license plate candidate frame information; and if the calculated overlapping area ratio is larger than the preset ratio, the license plate candidate frame information with lower confidence coefficient in the group of license plate candidate frame information is used as the second license plate candidate frame information to be deleted.
For example, as shown in fig. 1, it is assumed that a frame a and a frame B are vehicle candidate frames output to the NMS policy module by the license plate detection network, where the confidence of the frame a is 0.88 and the confidence of the frame B is 0.92, and the frame B is the first license plate candidate frame with the highest confidence according to the confidence ranking, and then the overlapping area ratio iou of the frame a and the frame B is calculated, specifically, the calculation method is as follows:
iou ═ formula (a ∩ B)/(a ∪ B) (-)
Where A ∩ B is the area where box A overlaps box B, and A ∪ B is the area where box A merges with box B.
The proportion iou of the overlapping area of the frame A and the frame B can be obtained through calculation of the first formula, and if the proportion iou of the overlapping area of the frame A and the frame B is larger than a preset proportion, the frame A and the frame B are likely to be candidate frames of the same target license plate, so that one candidate frame needs to be deleted. Then, the confidence degrees of the frame a and the frame B can be further compared, and the frame a is used as the license plate candidate frame to be deleted because the confidence degree of the frame a is lower than that of the frame B.
When the second license plate candidate frame information to be deleted exists in the plurality of pieces of license plate candidate frame information, the confidence coefficient in the second license plate candidate frame information can be reduced, and then the second license plate candidate frame information is input to a license plate detection network.
As an embodiment, the license plate candidate frame information may include coordinate information of a license plate candidate frame. The NMS strategy module can calculate the new confidence degree of the second license plate candidate frame according to the coordinate information of the first license plate candidate frame and the coordinate information of the second license plate candidate frame; the new confidence is less than the current confidence of the second card candidate box.
New confidence S for second card candidate boxNewThe formula of (c) is as follows:
wherein S isNewNew confidence for the second card candidate box, SOld ageFor the current confidence of a second candidate box, iou (M, b) is the ratio of the overlapping area of the first candidate box and the second candidate box, iou can be obtained by calculation according to a formula (I), wherein M is the coordinate of the first license plate candidate frame, b is the coordinate of the second license plate candidate frame, sigma is a Gaussian parameter, the Gaussian parameter sigma is an adjustable parameter, the attenuation degree of the confidence coefficient can be adjusted by adjusting the Gaussian parameter sigma, and the specific value is determined according to the actual situation.
After the confidence coefficient of the second license plate candidate frame is reduced, the first license plate candidate frame information and the second license plate candidate frame information can be input into the license plate detection network again for cycle iteration according to a preset algorithm, so that when the final confidence coefficient of the second license plate candidate frame information is not smaller than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information.
As an embodiment, the license plate detection network further includes a third network, where the feature map output by the first network and the target license plate candidate frame information output by the second network may be input to the third network, and the third network performs license plate region classification on the target license plate candidate frame to obtain a target license plate region finally.
Compared with the prior art that the license plate candidate frame information to be deleted is deleted when the license plate candidate frame information to be deleted exists, so that the missing detection problem is caused, the confidence coefficient of the license plate candidate frame information to be deleted can be reduced in the NMS strategy module processing process, the license plate candidate frame information can enter the license plate detection network again for calculation, the license plate candidate frame information is deleted until the final confidence coefficient is attenuated to be smaller than the preset threshold value, and if the final confidence coefficient is not smaller than the preset threshold value, the license plate candidate frame information is output as the target license plate candidate frame information, so that the missing detection problem can be avoided.
In order to make the present application more clear, the following examples illustrate the scheme of the present application in detail.
Example two:
please refer to fig. 3, which is a schematic diagram of an exemplary license plate detection network according to the present application; the network can be a fast RCNN license plate detection network, and the specific processing process of the network is as follows:
inputting a license plate sample image to a CNN network (equivalent to the first network); extracting the characteristics of the whole graph through a CNN network to obtain a characteristic graph; the CNN network inputs the feature map into an RPN (region protocol network) network (equivalent to the second network), and the RPR network traverses and regresses a plurality of areas based on the feature map to obtain a plurality of original license plate candidate frame information and inputs the information into an NMS strategy module; the NMS strategy module filters the original license plate candidate frame information by utilizing an NMS strategy to obtain target license plate candidate frame information; and inputting the target license plate candidate frame information and the characteristic diagram of the CNN network into the RCNN (equivalent to the third network), and classifying license plate areas of the target license plate candidate frame information by the RCNN to finally obtain a target license plate area.
The method mainly aims at optimizing an NMS strategy module in a license plate detection network, wherein when the NMS strategy module receives a plurality of pieces of license plate candidate frame information output by an RPR network in the license plate detection network, the output license plate candidate frame information is assumed to be shown in figure 1.
The confidence of the frame A is 0.88, the confidence of the frame B is 0.92, the frame B is the first license plate candidate frame with the highest confidence according to the confidence ranking, the overlapping area proportion iou of the frame A and the frame B is calculated, and the specific calculation method is shown as a formula (I). And if the overlapping area ratio iou of the frame A and the frame B is larger than or equal to a preset ratio, further comparing the confidence degrees of the frame A and the frame B, and taking the frame A as a second candidate frame of the card to be deleted because the confidence degree of the frame A is smaller than the confidence degree of the frame B.
Then, a new confidence coefficient S in the second card candidate frame information is calculated according to the formula (II)NewCalculating a new confidence level SNewAnd then, inputting the new confidence coefficient in the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient in the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network detects the license plate information according to the first license plate candidate frame information and the second license plate candidate frame information.
When the conventional NMS policy module processes, if there is a license plate candidate frame to be deleted, it is directly filtered at this stage, and only the license plate position with the maximum confidence is detected in the result of obtaining the final target license plate position, as shown in fig. 4-1, the license plate (white frame mark) above the vehicle is detected, and the license plate below is not detected, so that the detection result is incomplete.
And if the optimized NMS strategy module has the license plate candidate frame information to be deleted, the confidence coefficient of the optimized NMS strategy module is reduced instead of directly deleting the information, in the subsequent continuous updating iteration process, if the confidence coefficient in the final license plate candidate frame information to be deleted is not smaller than a preset threshold value, the license plate candidate frame information is output as a target license plate position, the final output license plate position is shown as a picture 4-2, and both the license plates above and below the vehicle (white frame marks) are detected. Therefore, the method and the device can output the real license plate frame to the maximum extent, improve the detection rate of the compact double license plates, and avoid considering the false detection or missing detection risk caused by the preset threshold value.
Corresponding to the embodiment of the image processing method, the application also provides an embodiment of the compact double-license plate detection device.
Referring to fig. 5, which is a block diagram of an embodiment of the compact dual license plate detection apparatus of the present application, the apparatus 50 may include:
an obtaining unit 51, configured to obtain information of a plurality of license plate candidate frames output by a license plate detection network;
a calculating unit 52, configured to determine, from the plurality of license plate candidate frame information, first license plate candidate frame information with the highest confidence; and when second license plate candidate frame information to be deleted exists in the plurality of license plate candidate frame information, reducing the confidence coefficient in the second license plate candidate frame information, and inputting the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient of the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information.
As one embodiment, the license plate detection network includes a first network and a second network that are cascaded;
the obtaining unit 51 is specifically configured to, according to a vehicle image input to a first network, perform feature extraction by the first network to obtain a feature map, input the feature map to a second network, and the second network determines information of a plurality of license plate candidate frames.
As an embodiment, the apparatus further comprises:
the determining unit 53 is configured to combine the plurality of license plate candidate frame information pairwise, and calculate an overlapping area ratio of each group of license plate candidate frame information; and if the calculated overlapping area ratio is larger than the preset ratio, the license plate candidate frame information with lower confidence coefficient in the group of license plate candidate frame information is used as second license plate candidate frame information to be deleted.
As an embodiment, the license plate candidate frame information includes coordinate information of a license plate candidate frame;
the calculating unit 52 is specifically configured to calculate a new confidence of the second candidate plate frame according to the coordinate information of the first candidate plate frame and the coordinate information of the second candidate plate frame; the new confidence is less than the current confidence of the second card candidate box.
As one embodiment, a new confidence level S for a second card candidate frame is calculated based on coordinate information of a first card candidate frame and coordinate information of a second card candidate frameNewThe method comprises the following steps:
wherein S isNewNew confidence for the second card candidate box, SOld ageFor the current confidence of a second candidate box, iou (M, b) is the ratio of the overlapping area of the first candidate box and the second candidate box, where M is the coordinate of the first candidate box, b is the coordinate of the second candidate box, and σ is a gaussian parameter, which is adjustable.
As an embodiment, the license plate detection network further comprises a third network, the apparatus further comprising:
and the input unit 54 is configured to input the feature map output by the first network and the target license plate candidate frame information output by the second network into a third network, and the third network performs license plate region classification on the target license plate candidate frame to obtain a target license plate region finally.
According to the embodiment, the information of the plurality of license plate candidate frames output by the license plate detection network can be acquired; determining first license plate candidate frame information with the maximum confidence coefficient from a plurality of pieces of license plate candidate frame information, reducing the confidence coefficient in second license plate candidate frame information when the second license plate candidate frame information to be deleted exists in the plurality of pieces of license plate candidate frame information, and inputting the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient of the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information. Compared with the prior art, the confidence coefficient of the second license plate candidate frame to be deleted can be reduced, the second license plate candidate frame is input into the license plate detection network again to be calculated, and if the final confidence coefficient of the second license plate candidate frame is not smaller than the preset threshold value, the second license plate candidate frame can be output, so that the problem of license plate missing detection is avoided, and the detection rate of compact double-license plate detection is improved.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the application. One of ordinary skill in the art can understand and implement it without inventive effort.
Corresponding to the embodiments of the image processing method described above, the present application also provides embodiments of an electronic device for performing the above-described compact two-license plate detection method.
Referring to fig. 6, an electronic device includes a processor 61, a communication interface 62, a memory 63, and a communication bus 64;
the processor 61, the communication interface 62 and the memory 63 are in communication with each other through the communication bus 64;
the memory 63 is used for storing computer programs;
the processor 61 is configured to execute the computer program stored in the memory 63, and when the processor 61 executes the computer program, any step of the compact two-vehicle license plate detection method is implemented.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiment of the computer device, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to part of the description of the method embodiment.
Corresponding to the embodiments of the aforementioned image processing method, the present application also provides embodiments of a computer-readable storage medium for executing the aforementioned image processing method.
As an embodiment, the present application further includes a computer-readable storage medium having stored therein a computer program that, when executed by a processor, performs any of the steps of the compact dual license plate detection method.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system embodiments and the computer-readable storage medium embodiments are substantially similar to the method embodiments, so that the description is simple, and reference may be made to some descriptions of the method embodiments for relevant points.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of protection of the present application.
Claims (13)
1. A compact dual license plate detection method, the method comprising:
acquiring a plurality of license plate candidate frame information output by a license plate detection network;
determining first license plate candidate frame information with the maximum confidence coefficient from the plurality of license plate candidate frame information; and when second license plate candidate frame information to be deleted exists in the plurality of license plate candidate frame information, reducing the confidence coefficient in the second license plate candidate frame information, and inputting the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient of the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information.
2. The method of claim 1, wherein the license plate detection network comprises a first network and a second network in cascade;
the method for acquiring the information of the plurality of license plate candidate frames output by the license plate detection network comprises the following steps:
according to the vehicle image input into the first network, feature extraction is carried out by the first network to obtain a feature map, the feature map is input into a second network, and the second network determines a plurality of license plate candidate frame information.
3. The method of claim 1, wherein determining that there is a second license plate candidate box information to be deleted from the plurality of license plate candidate box information comprises:
combining the plurality of license plate candidate frame information in pairs, and calculating the overlapping area proportion of each group of license plate candidate frame information;
and if the calculated overlapping area ratio is larger than the preset ratio, the license plate candidate frame information with lower confidence coefficient in the group of license plate candidate frame information is used as second license plate candidate frame information to be deleted.
4. The method of claim 1, wherein the license plate frame candidate information comprises coordinate information of a license plate frame candidate;
the reducing the confidence level in the second card candidate box information specifically includes:
calculating a new confidence coefficient of the second license plate candidate frame according to the coordinate information of the first license plate candidate frame and the coordinate information of the second license plate candidate frame; the new confidence is less than the current confidence of the second card candidate box.
5. The method of claim 4, wherein the new confidence level S for the second candidate card frame is calculated based on the coordinate information of the first candidate card frame and the coordinate information of the second candidate card frameNewThe method comprises the following steps:
wherein S isNewNew confidence for the second card candidate box, SOld ageFor the current confidence of a second candidate box, iou (M, b) is the ratio of the overlapping area of the first candidate box and the second candidate box, where M is the coordinate of the first candidate box, b is the coordinate of the second candidate box, and σ is a gaussian parameter, which is adjustable.
6. The method of claim 1, wherein the license plate detection network further comprises a third network, the method further comprising:
and inputting the feature map output by the first network and the target license plate candidate frame information output by the second network into a third network, and classifying license plate areas of the target license plate candidate frame by the third network to finally obtain a target license plate area.
7. A compact dual license plate detection device, the device comprising:
the license plate detection device comprises an acquisition unit, a storage unit and a display unit, wherein the acquisition unit is used for acquiring a plurality of license plate candidate frame information output by a license plate detection network;
the calculation unit is used for determining first license plate candidate frame information with the maximum confidence coefficient from the plurality of license plate candidate frame information; and when second license plate candidate frame information to be deleted exists in the plurality of license plate candidate frame information, reducing the confidence coefficient in the second license plate candidate frame information, and inputting the second license plate candidate frame information into the license plate detection network, so that when the final confidence coefficient of the second license plate candidate frame information is not less than a preset threshold value, the license plate detection network outputs the first license plate candidate frame information and the second license plate candidate frame information as target license plate candidate frame information.
8. The apparatus of claim 7, wherein the license plate detection network comprises a first network and a second network in cascade;
the acquisition unit is specifically used for carrying out feature extraction on the first network according to the vehicle image input into the first network to obtain a feature map, inputting the feature map into a second network, and determining a plurality of license plate candidate frame information by the second network.
9. The apparatus of claim 7, further comprising:
the determining unit is used for combining the plurality of license plate candidate frame information in pairs and calculating the overlapping area proportion of each group of license plate candidate frame information; and if the calculated overlapping area ratio is larger than the preset ratio, the license plate candidate frame information with lower confidence coefficient in the group of license plate candidate frame information is used as second license plate candidate frame information to be deleted.
10. The apparatus of claim 7, wherein the license plate frame candidate information includes coordinate information of a license plate frame candidate;
the calculation unit is specifically configured to calculate a new confidence of the second candidate plate frame according to the coordinate information of the first candidate plate frame and the coordinate information of the second candidate plate frame; the new confidence is less than the current confidence of the second card candidate box.
11. The apparatus of claim 10, wherein the new confidence level S for the second candidate plate frame is calculated based on the coordinate information of the first candidate plate frame and the coordinate information of the second candidate plate frameNewThe method comprises the following steps:
wherein S isNewNew confidence for the second card candidate box, SOld ageFor the current confidence of a second candidate box, iou (M, b) is the ratio of the overlapping area of the first candidate box and the second candidate box, where M is the coordinate of the first candidate box, b is the coordinate of the second candidate box, and σ is a gaussian parameter, which is adjustable.
12. The apparatus of claim 7, wherein the license plate detection network further comprises a third network, the apparatus further comprising:
and the input unit is used for inputting the feature map output by the first network and the target license plate candidate frame information output by the second network into a third network, and the third network classifies license plate areas of the target license plate candidate frame to finally obtain the target license plate area.
13. An electronic device comprising a processor, a communication interface, a memory, and a communication bus;
the processor, the communication interface and the memory are communicated with each other through the communication bus;
the memory is used for storing a computer program;
the processor is configured to execute the computer program stored in the memory, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811174813.7A CN111027534B (en) | 2018-10-09 | 2018-10-09 | Compact double license plate detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811174813.7A CN111027534B (en) | 2018-10-09 | 2018-10-09 | Compact double license plate detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111027534A true CN111027534A (en) | 2020-04-17 |
| CN111027534B CN111027534B (en) | 2023-05-26 |
Family
ID=70191582
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811174813.7A Active CN111027534B (en) | 2018-10-09 | 2018-10-09 | Compact double license plate detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111027534B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753826A (en) * | 2020-05-14 | 2020-10-09 | 北京迈格威科技有限公司 | Method and device for associating vehicle with license plate and electronic system |
| CN112329881A (en) * | 2020-11-20 | 2021-02-05 | 珠海大横琴科技发展有限公司 | License plate recognition model training method, license plate recognition method and device |
| CN113763338A (en) * | 2021-08-24 | 2021-12-07 | 合肥欣奕华智能机器有限公司 | Defect detection method and device of panel and storage medium |
| CN113781790A (en) * | 2021-08-31 | 2021-12-10 | 深圳市捷顺科技实业股份有限公司 | Method for recognizing license plate and parking lot terminal |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101630361A (en) * | 2008-12-30 | 2010-01-20 | 北京邮电大学 | Plate number, body color and mark identification-based equipment and plate number, body color and mark identification-based method for identifying fake plate vehicles |
| CN106845478A (en) * | 2016-12-30 | 2017-06-13 | 同观科技（深圳）有限公司 | The secondary licence plate recognition method and device of a kind of character confidence level |
| CN107679531A (en) * | 2017-06-23 | 2018-02-09 | 平安科技（深圳）有限公司 | Licence plate recognition method, device, equipment and storage medium based on deep learning |
| CN107862340A (en) * | 2017-11-16 | 2018-03-30 | 深圳市华尊科技股份有限公司 | A kind of model recognizing method and device |
| CN108009543A (en) * | 2017-11-29 | 2018-05-08 | 深圳市华尊科技股份有限公司 | A kind of licence plate recognition method and device |
| WO2018090771A1 (en) * | 2016-11-16 | 2018-05-24 | 杭州海康威视数字技术股份有限公司 | Vehicle license plate recognition method and apparatus |
| CN108073925A (en) * | 2016-11-17 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | A kind of licence plate recognition method and device |
| CN108073926A (en) * | 2016-11-17 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | A kind of licence plate recognition method and device |
| CN108446696A (en) * | 2018-02-09 | 2018-08-24 | 杭州雄迈集成电路技术有限公司 | A kind of end-to-end licence plate recognition method based on deep learning |
-
2018
- 2018-10-09 CN CN201811174813.7A patent/CN111027534B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101630361A (en) * | 2008-12-30 | 2010-01-20 | 北京邮电大学 | Plate number, body color and mark identification-based equipment and plate number, body color and mark identification-based method for identifying fake plate vehicles |
| WO2018090771A1 (en) * | 2016-11-16 | 2018-05-24 | 杭州海康威视数字技术股份有限公司 | Vehicle license plate recognition method and apparatus |
| CN108073925A (en) * | 2016-11-17 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | A kind of licence plate recognition method and device |
| CN108073926A (en) * | 2016-11-17 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | A kind of licence plate recognition method and device |
| CN106845478A (en) * | 2016-12-30 | 2017-06-13 | 同观科技（深圳）有限公司 | The secondary licence plate recognition method and device of a kind of character confidence level |
| CN107679531A (en) * | 2017-06-23 | 2018-02-09 | 平安科技（深圳）有限公司 | Licence plate recognition method, device, equipment and storage medium based on deep learning |
| CN107862340A (en) * | 2017-11-16 | 2018-03-30 | 深圳市华尊科技股份有限公司 | A kind of model recognizing method and device |
| CN108009543A (en) * | 2017-11-29 | 2018-05-08 | 深圳市华尊科技股份有限公司 | A kind of licence plate recognition method and device |
| CN108446696A (en) * | 2018-02-09 | 2018-08-24 | 杭州雄迈集成电路技术有限公司 | A kind of end-to-end licence plate recognition method based on deep learning |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753826A (en) * | 2020-05-14 | 2020-10-09 | 北京迈格威科技有限公司 | Method and device for associating vehicle with license plate and electronic system |
| CN111753826B (en) * | 2020-05-14 | 2024-02-20 | 北京迈格威科技有限公司 | Vehicle and license plate association method, device and electronic system |
| CN112329881A (en) * | 2020-11-20 | 2021-02-05 | 珠海大横琴科技发展有限公司 | License plate recognition model training method, license plate recognition method and device |
| CN112329881B (en) * | 2020-11-20 | 2022-02-15 | 珠海大横琴科技发展有限公司 | License plate recognition model training method, license plate recognition method and device |
| CN113763338A (en) * | 2021-08-24 | 2021-12-07 | 合肥欣奕华智能机器有限公司 | Defect detection method and device of panel and storage medium |
| CN113781790A (en) * | 2021-08-31 | 2021-12-10 | 深圳市捷顺科技实业股份有限公司 | Method for recognizing license plate and parking lot terminal |
| CN113781790B (en) * | 2021-08-31 | 2022-08-12 | 深圳市捷顺科技实业股份有限公司 | Method for recognizing license plate and parking lot terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111027534B (en) | 2023-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021003824A1 (en) | Image recognition-based illegal building identification method and device | |
| CN111027534A (en) | Compact double-license-plate detection method and device | |
| CN112381775B (en) | Image tampering detection method, terminal device and storage medium | |
| CN102419819B (en) | Method and system for recognizing human face image | |
| CN107944427B (en) | Dynamic face recognition method and computer readable storage medium | |
| CN108805016B (en) | Head and shoulder area detection method and device | |
| CN105426863B (en) | The method and apparatus for detecting lane line | |
| CN108960185A (en) | Vehicle target detection method and system based on YOLOv2 | |
| CN111914665A (en) | Face shielding detection method, device, equipment and storage medium | |
| WO2020181426A1 (en) | Lane line detection method and device, mobile platform, and storage medium | |
| CN111563468B (en) | Driver abnormal behavior detection method based on attention of neural network | |
| CN114067186A (en) | Pedestrian detection method and device, electronic equipment and storage medium | |
| CN104636748B (en) | A kind of method and device of number plate identification | |
| CN114266894A (en) | Image segmentation method and device, electronic equipment and storage medium | |
| CN113838087B (en) | Anti-occlusion target tracking method and system | |
| CN112580531B (en) | Identification detection method and system for true and false license plates | |
| US20220405527A1 (en) | Target Detection Methods, Apparatuses, Electronic Devices and Computer-Readable Storage Media | |
| CN116543189B (en) | Target detection method, device, equipment and storage medium | |
| CN113496215A (en) | Method and device for detecting human face of living body and electronic equipment | |
| CN110287786B (en) | Vehicle information identification method and device based on artificial intelligence anti-interference | |
| CN116740792A (en) | Face recognition method and system for sightseeing vehicle operators | |
| CN111163332A (en) | Video pornography detection method, terminal and medium | |
| CN113688810B (en) | Target capturing method and system of edge device and related device | |
| CN112597924B (en) | Electric bicycle track tracking method, camera device and server | |
| CN110634120A (en) | Vehicle damage judgment method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |