CN108073925B

CN108073925B - License plate recognition method and device

Info

Publication number: CN108073925B
Application number: CN201611024032.0A
Authority: CN
Inventors: 钱华
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2016-11-17
Filing date: 2016-11-17
Publication date: 2021-09-17
Anticipated expiration: 2036-11-17
Also published as: CN108073925A

Abstract

The embodiment of the application provides a license plate identification method and device. The method comprises the following steps: determining a first license plate area in a license plate image to be recognized, recognizing characters in the first license plate area, and obtaining a first character recognition result; judging whether the license plate number which is not successfully identified exists in the first license plate area or not according to the identification result; if yes, selecting a license plate template from the license plate template library as a target license plate template; determining a sub-region of the license plate number which is not successfully recognized from the first license plate region according to the template characteristics corresponding to the target license plate template; identifying characters in the sub-area to obtain a second character identification result; judging whether the character recognition aiming at the subarea is successful according to the recognition result; if the two recognition results are successful, obtaining the license plate number of the license plate image to be recognized according to the two recognition results; otherwise, reselecting an unselected license plate template as a target license plate template, and repeating the process. The license plate recognition method and device can improve the efficiency of the license plate recognition process.

Description

License plate recognition method and device

Technical Field

The application relates to the technical field of intelligent traffic, in particular to a license plate recognition method and device.

Background

The license plate is the 'ID card' of the vehicle and is important information which is different from other motor vehicles. The license plate recognition technology is widely applied to scenes such as a gate, a parking lot, an electronic police and the like to acquire license plate information of vehicles in the scenes, and plays the power of an intelligent traffic algorithm in many aspects such as public security management and the like.

Throughout the world, the characters in the license plate usually include Arabic numerals, English letters and local special characters. Fig. 1 is a diagram illustrating license plates of some countries and regions, where the license plate images include one or more of arabic numerals, english letters and local special characters, for example, license plates numbered 4-10 include local special characters korean and arabic numerals.

According to the character types, the characters in the license plate can be divided into two types, wherein the first type of characters comprise common characters such as Arabic numerals and English letters, and the second type of characters comprise special characters such as local characters. It can also be seen from fig. 1 that the second type of characters are not fixed in position in the license plate, their size is also inconsistent with the size of the first type of characters, and the spacing between the second type of characters is also inconsistent with the spacing between the first type of characters.

In the prior art, when a license plate number of a license plate in a license plate image containing local special characters is identified, one of the license plate images to be identified is matched with a plurality of license plate templates which are stored in advance, and then the license plate number is identified. The specific process is as follows: firstly, locating a license plate area in a license plate image to be recognized, secondly, performing character segmentation on a first type character and a second type character in the license plate image to be recognized according to a certain selected license plate template, and then performing character recognition on each segmented character area. And if the character recognition is successful, the license plate template is considered to be successfully matched, and the character recognition result is determined as the license plate number of the license plate image to be recognized. If the character recognition is unsuccessful, another license plate template is selected, and the process is repeated.

Under normal conditions, when the method is adopted for license plate recognition, the license plate number of the license plate in the license plate image to be recognized can be recognized. However, since a large number of license plate templates need to be matched, and the first type characters and the second type characters of the located license plate regions need to be completely subjected to the character segmentation and character recognition process once in each matching process, the efficiency of the license plate recognition process is not high.

Disclosure of Invention

The embodiment of the application aims to provide a license plate recognition method and a license plate recognition device, which can improve the efficiency of a license plate recognition process. The specific technical scheme is as follows.

In order to achieve the above object, the present application discloses a license plate recognition method, including:

obtaining a license plate image to be recognized, and determining a first license plate area in the license plate image to be recognized;

identifying characters in the first license plate area to obtain a first character identification result;

judging whether a license plate number which is not successfully recognized exists in the first license plate area or not according to the first character recognition result;

if yes, selecting a license plate template from a preset license plate template library as a target license plate template, wherein the license plate template library is used for storing each license plate template and corresponding template characteristics;

determining a sub-region of the license plate number which is not successfully recognized from the first license plate region according to the template characteristics corresponding to the target license plate template;

identifying characters in the sub-area to obtain a second character identification result;

judging whether the character recognition aiming at the subarea is successful or not according to the second character recognition result;

if the recognition is successful, obtaining the license plate number of the license plate image to be recognized according to the first character recognition result and the second character recognition result;

and if the license plate number is not successfully identified, selecting a license plate template from the unselected license plate templates in the license plate template library as a target license plate template, and returning to execute the step of determining a sub-region of the number of the license plate which is not successfully identified from the first license plate region according to the template characteristics corresponding to the target license plate template.

Optionally, the template features of the target license plate template include: the character recognition method comprises the following steps of obtaining a first character feature corresponding to a first character type, a second character feature corresponding to a second character type and a relative position relation between a character area of the first character type and a character area of the second character type;

the step of determining a sub-region of the number plate number which is not successfully recognized from the first license plate region according to the template characteristics corresponding to the target license plate template comprises the following steps:

determining the successfully recognized character in the first character recognition result as a target character;

determining the area where the target character is located as a target character area;

matching the characteristics of the target character with the first character characteristics and the second character characteristics respectively, and determining the character type corresponding to the successfully matched character characteristics as the target character type corresponding to the target character area;

and determining a sub-region of the license plate number which is not successfully recognized from the first license plate region according to the target character region, the target character type and the relative position relationship.

Optionally, the step of determining the successfully recognized character in the first character recognition result as the target character includes:

obtaining the successfully recognized characters in the first character recognition result;

and determining the character segment with the character continuous distribution and the maximum number of characters in the obtained characters as the target character.

Optionally, the step of recognizing the characters in the sub-region and obtaining a second character recognition result includes:

segmenting the sub-region to obtain a character region to be recognized;

and identifying the characters in the character area to be identified to obtain a second character identification result.

Optionally, the template features of the target license plate template include: the character size of the character of the first character type is larger than that of the character of the second character type;

the step of segmenting the sub-region to obtain a character region to be recognized comprises the following steps:

obtaining a first size according to the size of the target character;

matching the characteristics of the target character with the first character characteristics and the second character characteristics respectively to determine the character type of the target character;

determining a second size according to the first size, the character type of the target character and the relative size relationship;

and according to the second size, segmenting the sub-region to obtain a character region to be recognized.

Optionally, the obtaining of the license plate number of the license plate image to be recognized according to the first character recognition result and the second character recognition result includes:

and synthesizing the target character and a second character recognition result to obtain the license plate number of the license plate image to be recognized.

Optionally, the determining, according to the first character recognition result, whether a license plate number that is not successfully recognized exists in the first license plate area includes:

judging whether the license plate number which is not successfully identified exists in the first license plate area according to at least one of the following modes:

judging whether the first character recognition result contains an unrecognized recognition result, and if so, determining that the number plate number which is not successfully recognized exists in the first number plate area;

and judging whether the number of the successfully recognized characters in the first character recognition result is smaller than a preset number threshold, and if so, determining that the number of the license plate which is not successfully recognized exists in the first license plate area.

In order to achieve the above object, the present application discloses a license plate recognition device, the device including:

the license plate region determining module is used for obtaining a license plate image to be recognized and determining a first license plate region in the license plate image to be recognized;

the first character recognition module is used for recognizing characters in the first license plate area to obtain a first character recognition result;

the first recognition judging module is used for judging whether the license plate number which is not successfully recognized exists in the first license plate area according to the first character recognition result;

the target template selection module is used for selecting a license plate template from a preset license plate template library as a target license plate template when the license plate number which is not successfully identified exists in the first license plate area, wherein the license plate template library is used for storing each license plate template and corresponding template characteristics;

the sub-region determining module is used for determining a sub-region of the license plate number which is not successfully recognized from the first license plate region according to the template characteristics corresponding to the target license plate template;

the second character recognition module is used for recognizing the characters in the sub-area to obtain a second character recognition result;

the second recognition judging module is used for judging whether the character recognition aiming at the subarea is successful according to the second character recognition result;

the license plate number obtaining module is used for obtaining the license plate number of the license plate image to be recognized according to the first character recognition result and the second character recognition result when the character recognition aiming at the sub-region is successful;

and the target template replacing module is used for selecting one license plate template from the unselected license plate templates in the license plate template library as a target license plate template when the character recognition aiming at the sub-region is unsuccessful, and returning to execute the sub-region determining module.

the sub-region determination module comprises:

the target character determining submodule is used for determining the successfully recognized character in the first character recognition result as a target character;

the character area determining submodule is used for determining the area where the target character is located as a target character area;

the character type determining sub-module is used for respectively matching the characteristics of the target character with the first character characteristics and the second character characteristics, and determining the character type corresponding to the successfully matched character characteristics as the target character type corresponding to the target character area;

and the sub-region determining sub-module is used for determining a sub-region of the number plate number which is not successfully recognized from the first number plate region according to the target character region, the target character type and the relative position relationship.

Optionally, the target character determination sub-module is specifically configured to:

and obtaining the successfully recognized characters in the first character recognition result, and determining the character segment with the continuous character distribution and the maximum character number in the obtained characters as the target character.

Optionally, the second character recognition module includes:

the segmentation submodule is used for segmenting the sub-region to obtain a character region to be recognized;

and the recognition submodule is used for recognizing the characters in the character area to be recognized and obtaining a second character recognition result.

the partitioning submodule is specifically configured to:

obtaining a first size according to the size of the target character;

Optionally, the license plate number obtaining module includes:

and the license plate number obtaining submodule is used for synthesizing the target character and the second character recognition result to obtain the license plate number of the license plate image to be recognized.

Optionally, the first identification and judgment module is specifically configured to:

According to the technical scheme, the first license plate area in the license plate image to be recognized is determined, characters in the first license plate area are recognized, and a first character recognition result is obtained. And judging whether the first license plate area has the license plate number which is not successfully recognized or not according to the first character recognition result, if so, selecting a license plate template from a preset license plate template library as a target license plate template, wherein the license plate template library is used for storing each license plate template and corresponding template characteristics. And then, according to template features corresponding to the target license plate template, determining a sub-region of the license plate number which is not successfully recognized from the first license plate region, recognizing characters in the sub-region, obtaining a second character recognition result, and according to the second character recognition result, judging whether the character recognition aiming at the sub-region is successful. And if the recognition is successful, obtaining the license plate number of the license plate image to be recognized according to the first character recognition result and the second character recognition result. And if the number plate is not successfully identified, selecting one number plate template from the unselected number plate templates in the number plate template library as a target number plate template, and returning to the step of determining the sub-region of the number plate number which is not successfully identified from the first number plate region.

That is to say, in the embodiment of the application, a first license plate area is located for a license plate image to be recognized, characters in the first license plate area are recognized, license plate templates in a license plate template library are matched one by one, sub-areas in the first license plate area are located according to template features of the license plate templates in each matching process, and characters in the sub-areas are recognized. When the characters in the sub-region are successfully recognized, the matching of the template matching process is successful, and the license plate number of the license plate image to be recognized is obtained according to the two character recognition results. And when the character recognition in the sub-region is unsuccessful, replacing the license plate template, and repeating the process of positioning the sub-region until the matching is successful.

In the prior art, after the first license plate area is located, license plate templates in a license plate template library need to be matched one by one, but in each matching process, the processes of character segmentation and character recognition need to be repeatedly executed on the whole first license plate area. According to the embodiment of the application, after the first license plate area is located, and when the first license plate area is determined to have the number of the license plate which is not successfully identified according to the identification result, the license plate templates in the license plate template library are matched one by one, the character which is not successfully identified in the first license plate area is located and identified, and the information amount which needs to be processed in each matching process is reduced. Therefore, when the scheme provided by the embodiment of the application is applied to license plate recognition, the efficiency of the license plate recognition process can be improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.

FIG. 1 is an example diagram of a portion of a license plate;

fig. 2 is a schematic flow chart of a license plate recognition method according to an embodiment of the present disclosure;

fig. 3 is another schematic flow chart of a license plate recognition method according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a license plate recognition device according to an embodiment of the present disclosure;

fig. 5 is another schematic structural diagram of a license plate recognition device according to an embodiment of the present disclosure.

Detailed Description

The technical solution in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the described embodiments are merely a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The embodiment of the application provides a license plate identification method and device, which are applied to electronic equipment, wherein the electronic equipment can be a computer, a tablet personal computer, a smart phone, a vehicle event data recorder and the like. By applying the technical scheme provided by the embodiment of the application, the efficiency of the license plate recognition process can be improved.

The present application will be described in detail below with reference to specific examples.

Fig. 2 is a schematic flow chart of the license plate recognition method provided in the embodiment of the present application, and the license plate recognition method is applied to an electronic device. The method comprises the following steps:

step S201: and obtaining a license plate image to be recognized, and determining a first license plate area in the license plate image to be recognized.

The electronic device as the execution subject may or may not include an image capturing device therein.

Specifically, when the electronic device as the execution subject includes an image capture device inside, the electronic device may include, when obtaining the license plate image to be recognized: and receiving the license plate image to be recognized, which is acquired by the image acquisition equipment.

When the electronic device as the execution subject does not include an image capturing device inside, the electronic device may be connected to an external image capturing device, and when obtaining a license plate image to be recognized, the electronic device may include: and acquiring a license plate image to be recognized, which is acquired by image acquisition equipment.

The acquired license plate image to be recognized can be acquired by the image acquisition equipment in real time, or can be not acquired in real time, but is stored after being acquired in advance by the image acquisition equipment.

The license plate image to be recognized can be understood as follows: and (3) an image for license plate recognition. It is understood that the license plate is usually installed or placed on the vehicle, and therefore, the above-mentioned license plate image to be recognized can be understood as: an image containing a vehicle to be license plate recognized. Based on the above, the license plate image to be recognized may be an image including a vehicle captured on a road, an image including a vehicle captured in a parking lot, or the like. Of course, the license plate image to be recognized may also be obtained in other manners, and the obtaining manner of the license plate image to be recognized is not limited in the present application.

The first license plate region can be understood as a region located when the license plate image to be recognized is located, and can also be referred to as a locating layer.

After obtaining the license plate image to be recognized, the electronic device serving as the execution subject determines the first license plate region in the license plate image to be recognized by using a method for positioning the license plate region in the prior art, and the specific process is not repeated. The method also can be used for further positioning the positioned license plate region according to the preset region characteristics after positioning the license plate image to be recognized by adopting the prior art, so that the interference character region is removed, the positioning result of the license plate region is more accurate, and the finally obtained license plate region is the first license plate region. The preset features may include features such as a width-to-height ratio of the character region, a color of the character region, and the like.

Step S202: and identifying characters in the first license plate area to obtain a first character identification result.

Specifically, when the character in the first license plate region is recognized and the first character recognition result is obtained, the first license plate region may be first segmented by using a vertical projection method or a connected domain method, and the like, so as to obtain a character segmentation result, and then the character segmentation result is recognized by using a preset character recognizer, so as to obtain the first character recognition result.

The character recognizer may include N output units, and each output unit corresponds to one character. For example, the character recognizer includes 37 output units, which correspond to the following characters, respectively: 10 numbers, 26 letters and 1 "unknown".

The character segmentation result generally includes a plurality of character areas, and correspondingly, the first character recognition result generally includes a successfully recognized character and a corresponding character area, and an unsuccessfully recognized character and a corresponding character area.

Specifically, when performing character recognition on each character region, the character region may be input to the character recognizer, each output unit may output a confidence level, and a character corresponding to an output unit whose confidence level is greater than a preset threshold value is a character in the character region. At this time, the character area is considered to be successfully recognized, and the corresponding character is the character which is successfully recognized.

If the confidence of the unknown output unit is higher than the threshold value, and the confidence of other output units is lower than the threshold value, the character area is considered to be unsuccessfully recognized, and the unknown is the character which is not successfully recognized.

When the first license plate area is segmented, a characteristic image of pixels in the first license plate image area can be obtained according to a vertical projection method or a connected domain method, and a character segmentation point is determined from the characteristic image, so that a character segmentation result is obtained.

Therefore, for a license plate image with all characters in the license plate having obvious and consistent characteristics, for example, under the conditions that the distances among all characters in the license plate image are the same and the sizes of all character regions are consistent, when the license plate region in the license plate image is segmented, a better segmentation result can be obtained usually, so that all characters in the license plate image can be identified more easily and correctly. Wherein the size of the character area comprises the height and/or width of the character area.

Of course, there are license plate images in which the features of the individual characters in the license plate are inconsistent with those of other characters, for example, there may be a case where there are characters in which the distances between the individual characters and other characters are different and the sizes of the individual characters and other character regions are inconsistent in the license plate images. For the sake of clarity, this application refers to characters having the following characteristics as "first-type characters": the number of characters is large and the intervals between the characters are consistent with each other, the sizes of the characters are consistent with each other, and the like; characters having the following features are referred to as "second-class characters": the number of characters is small and the character pitch does not coincide with the pitch of the first type of characters, the size of the characters does not coincide with the size of the first type of characters, and so on.

When the license plate region in the license plate image is segmented, the first character part can be segmented correctly, and the second character part is difficult to segment correctly, so that the character recognition of the second character part is unsuccessful.

Because the first type of characters have more consistent characteristics, the characters can be correctly recognized when the characters of the license plate region are recognized. However, since the second type of characters are generally fewer and inconsistent with the first type of characters, the characters in the license plate region cannot be correctly recognized during character recognition.

The following description will be given of the characters in the license plate by taking the license plate shown in fig. 1 as an example. The license plate shown in FIG. 1 contains 1-2 local characters, and also contains 4-7 Arabic numerals and/or English letters. For the sake of expressing the situation, arabic numerals and english letters are collectively referred to as common characters. It can be found that in these license plates, the sizes of common characters are identical to each other, and the sizes of local characters are identical to each other, but the sizes of local characters are generally different from the sizes of common characters. The spacing between common characters is consistent with each other, and the spacing between local letters and common characters is inconsistent with the spacing between common characters. Common characters usually appear in a license plate continuously for 3-5 characters, and the appearance positions of local characters are not fixed. For a single-layer license plate, the local characters may appear at the head, middle, or tail of the license plate, and for a double-layer license plate, the local characters may appear at the upper or lower layer. In summary, the arabic numerals and the english alphabet characters in the license plate of the present embodiment can be classified as the first type of characters, and the local alphabet characters can be classified as the second type of characters.

Therefore, when the license plate is recognized, the common character part can be successfully recognized, and the local character part cannot be successfully recognized.

As a result of empirical conclusion, the second type of characters generally includes local characters such as korean, thai, chinese, bosch, etc. or symbolic characters such as vertical lines, etc. The first type of characters typically include arabic numerals and/or english letters.

Several cases involved in the unsuccessful character recognition are analyzed below. Since the character recognition for the first license plate region usually includes a character segmentation process and a character recognition process, the unsuccessful character recognition may include the following cases: one is that the character can be segmented, but the character recognizer cannot recognize the character because the segmentation is unsuccessful, for example, the confidence of the "unknown" output unit is the highest; in another aspect, the part of the character is not segmented, i.e., the part of the character region is not entered into the character recognizer.

Step S203: and judging whether the license plate number which is not successfully recognized exists in the first license plate area or not according to the first character recognition result, if so, indicating that characters to be recognized still exist in the first license plate area, and executing the step S204. If not, the first character recognition result is directly used as the license plate number of the license plate image to be recognized, which indicates that all characters in the first license plate area are successfully recognized.

As can be seen from the description of step S202, the first character recognition result may include characters that are successfully recognized and characters that are not successfully recognized. According to the first character recognition result, the recognition result of the first license plate area comprises the following conditions:

a. all characters in the first license plate area are searched (namely all characters are segmented), and all characters are successfully identified;

b. all characters in the first license plate area are searched, wherein part of characters are successfully identified, and part of characters are not successfully identified;

c. all characters in the first license plate area are searched, and all characters are not successfully identified;

d. searching partial characters in the first license plate area, wherein the partial characters are not searched, and when the searched characters are identified, all the characters are successfully identified;

e. searching partial characters in the first license plate area, wherein the partial characters are not searched, and when the searched characters are identified, the partial characters are successfully identified and the partial characters are not searched;

f. and searching partial characters in the first license plate area, wherein the partial characters are not searched, and when the searched characters are identified, all the characters are not successfully identified.

In the above case, a is a case where there is no license plate number that is not successfully recognized in the first license plate region, and b to f are both cases where there is a license plate number that is not successfully recognized in the first license plate region.

Note that the character portions that are not searched may be discarded as interfering components in the image, such as rivets, mud spots, and the like. When the character non-recognition success is caused by incorrect segmentation at the time of segmentation of the character region, it may be due to segmentation of two characters in one character region or segmentation of one character into two character regions.

Step S204: and selecting a license plate template from a preset license plate template library as a target license plate template, wherein the license plate template library is used for storing each license plate template and corresponding template characteristics.

The template features may include the total number of characters in the license plate region, the type of the character type, the number of characters of each character type, the relative position distribution of the characters of each character type, the color information of the license plate region, the special symbols contained in the license plate region, the relative positions of the special symbols, and the like. Of course, the template features may also include geographic location information, i.e., the country or region to which the license plate template belongs.

For example, 2 license plates numbered 6 and 10 in fig. 1 may belong to the same license plate template, and the template features of the license plate template may include the following:

the total number of characters in the license plate region is: 7;

types of characters in the license plate region, including: a number type and a korean type;

the number of characters of each character type in the license plate area comprises: the number of characters of the number type is 6, and the number of characters of the korean type is 1;

the relative position distribution of the characters of each character type includes: the korean type character is located between the 2 nd and 3 rd numeral type characters;

the special symbols contained in the license plate region and their relative positions: none.

Specifically, when one license plate template is selected from a preset license plate template library as a target license plate template, the license plate templates can be selected randomly or in sequence.

It should be noted that the license plate template and the corresponding template features may be obtained and stored in advance. Because the license plate characteristics of all regions are different greatly, the license plate template can be obtained for the license plates in the same region. For example, a license plate template library is uniformly created for countries or regions using license plates containing Bowen characters. Thus, the license plate images belonging to the countries or regions can be identified by the electronic equipment. Certainly, a license plate template library can be created for a certain country or region, so that the number of created license plate templates is small, and the number of license plate templates required to be matched is small.

Specifically, a license plate image sample of a designated area can be collected, characters of a first character type and characters of a second character type in the sample are marked, and a license plate template corresponding to the area and corresponding template features are extracted and stored according to the sample and the marks in the sample.

Step S205: and determining a sub-region of the license plate number which is not successfully recognized from the first license plate region according to the template characteristics corresponding to the target license plate template.

Specifically, when the sub-region of the license plate number which is not successfully recognized is determined from the first license plate region, the target region of the license plate number which is not successfully recognized in the first license plate region is determined according to the template features of the target license plate template and the character features of the successfully recognized characters in the first character recognition result, wherein the target region can be a continuous region block or a plurality of discontinuous region blocks, then the target region is determined as the sub-region, and the characters in the sub-region are continuously recognized.

For example, it is known that the license plate image area numbered 6 in fig. 1 is a first license plate area, character recognition is performed on the first license plate area, a first character recognition result is obtained, and sequentially recognized characters in the result are 228946, where a question mark "? The "part" indicates that the character was not recognized successfully. The template characteristics of the determined target license plate template are described in the above example of step S204. At this time, according to the template features of the target license plate template, the region between the second character 2 and the character 8 in the first license plate region is determined as a sub-region, and the characters in the sub-region are continuously recognized.

Step S206: and identifying the characters in the sub-area to obtain a second character identification result.

It should be noted that, when recognizing the characters in the sub-region, the same process as that in step S202 may be adopted, or a process different from that in step S202 may be adopted, and details of the process are not described again in this embodiment.

Step S207: and judging whether the character recognition aiming at the subarea is successful or not according to the second character recognition result. If the judgment result is yes, the license plate in the license plate image to be recognized is considered to be matched with the current target license plate template, and the step S208 is continuously executed to obtain the final license plate number. If the judgment result is negative, the license plate in the license plate image to be recognized is not matched with the current target license plate template, and the step S209 of replacing the license plate template is required to be executed.

Specifically, judging whether the character recognition for the sub-region is successful according to the second character recognition result may include the following various embodiments:

and judging whether the second character recognition result has successfully recognized characters or not, and if so, determining that the character recognition for the subarea is successful.

And judging whether the successfully recognized characters in the second character recognition result are matched with the template features of the target license plate template, and if so, determining that the character recognition aiming at the sub-region is successful.

The template features may include character type, number of characters, and the like.

Step S208: and acquiring the license plate number of the license plate image to be recognized according to the first character recognition result and the second character recognition result.

As a specific implementation manner, when obtaining the license plate number of the license plate image to be recognized according to the first character recognition result and the second character recognition result, the method may include:

and determining the successfully recognized characters in the first character recognition result as target characters, and synthesizing the target characters and a second character recognition result to obtain the license plate number of the license plate image to be recognized.

The second character recognition result may also include a character that is successfully recognized and a character that is not successfully recognized.

Therefore, in synthesizing the target character and the second character recognition result, it may also include:

and synthesizing the target character and the character which is successfully recognized in the second character recognition result to obtain the license plate number of the license plate image to be recognized.

As a specific embodiment, when characters are synthesized, the synthesis may be performed according to a relative positional relationship between the respective characters.

Step S209: and selecting a license plate template from the unselected license plate templates in the license plate template library as a target license plate template, and returning to the step S205, namely determining the sub-region of the license plate number which is not successfully recognized from the first license plate region according to the template characteristics corresponding to the target license plate template.

As can be seen from the above, in this embodiment, a first license plate region is located for a license plate image to be recognized, characters in the first license plate region are recognized, then license plate templates in a license plate template library are matched one by one, sub-regions in the first license plate region are located according to template features of the license plate templates in each matching process, and characters in the sub-regions are recognized. When the characters in the sub-region are successfully recognized, the matching of the template matching process is successful, and the license plate number of the license plate image to be recognized is obtained according to the two character recognition results. And when the character recognition in the sub-region is unsuccessful, replacing the license plate template, and repeating the process of positioning the sub-region until the matching is successful.

In addition, when the license plate number which is not successfully recognized does not exist in the first license plate area according to the recognition result, the license plate template does not need to be matched, and the license plate number of the license plate image to be recognized can be directly determined, so that the template matching times can be reduced on the whole, and the efficiency of the license plate recognition process is improved.

In another implementation manner based on the embodiment shown in fig. 2, in step S203, determining whether a license plate number that is not successfully recognized exists in the first license plate area according to the first character recognition result may specifically include:

judging whether the first character recognition result contains a recognition result which is not successfully recognized, and if so, determining that the number of the license plate which is not successfully recognized exists in the first license plate area;

in this embodiment, when the first character recognition result includes a recognition result that is not successfully recognized, it may be determined that a license plate number that is not successfully recognized exists in the first license plate area.

In this embodiment, when the number of successfully recognized characters in the first character recognition result is less than the preset number threshold, it indicates that there may be an unsearched character in the first license plate area, or a part of the searched characters is not successfully recognized, and at this time, it may also be considered that there is an unrecognized license plate number in the first license plate area.

The number threshold may be a preset value, and the preset value may be determined according to a statistical result of the total number of the license plate numbers.

Fig. 3 is another schematic flow chart of a license plate recognition method according to an embodiment of the present disclosure, which is an improvement of the embodiment shown in fig. 2. Wherein, the template characteristic of target license plate template includes: the character region of the first character type is corresponding to the character region of the second character type.

Specifically, the first character type may be a numeric or alphabetic type, and the second character type may be a local special character type. Correspondingly, the character characteristics may include character type, character number, and the like.

For example, referring to the license plate templates of the license plates numbered 6 and 10 in fig. 1, the template features of the license plate templates include: the character type is number, and the number of the characters is 6; the second character feature may include: the character type is Korean, and the number of the characters is 1; the relative positional relationship includes: korean is between the 2 nd and 3 rd digits.

Specifically, step S205 in the embodiment shown in fig. 2 is a step of determining a sub-region where a license plate number is not successfully recognized from the first license plate region according to a template feature corresponding to the target license plate template, and in the embodiment shown in fig. 3, the step may include:

step S205A: and determining the character successfully recognized in the first character recognition result as the target character.

Specifically, when the successfully recognized character in the first character recognition result is determined as the target character, all the successfully recognized characters in the first character recognition result may be determined as the target character, or a part of characters may be selected from all the successfully recognized characters in the first character recognition result as the target character.

The target character determined in the above case may be one character segment or may be a plurality of character segments.

As a specific implementation manner, in order to improve the accuracy of the determined sub-region and reduce the processing complexity, the step of determining the character successfully recognized in the first character recognition result as the target character may specifically include:

and acquiring the successfully recognized characters in the first character recognition result, and determining the character segment with the continuous character distribution and the maximum number of characters in the acquired characters as the target character.

The continuous distribution of characters means that there is no character which is not successfully recognized between characters.

In such an embodiment, the highest number of consecutively distributed character segments is more likely to be successfully recognized, and thus, the accuracy of the determined sub-region can be improved by using the same as the target character. Meanwhile, the character segments with the largest number are determined as the target characters, so that the number of the character segments in the target characters can be reduced, and the processing complexity is reduced.

Step S205B: and determining the area where the target character is located as a target character area.

Step S205C: and matching the characteristics of the target character with the first character characteristics and the second character characteristics respectively, and determining the character type corresponding to the successfully matched character characteristics as the target character type corresponding to the target character area.

The characteristics of the target character may include the character type, the character number, and the like. The character type may be a letter, number, or local letter type.

It should be noted that, in this step, a character type matched with the feature of the target character is taken as a character type corresponding to the target character region, and the character type is either a first character type or a second character type.

Step S205D: and determining a sub-region of the license plate number which is not successfully recognized from the first license plate region according to the target character region, the target character type and the relative position relationship.

Taking the license plate numbered 6 in fig. 1 as an example, the template features of the target license plate template have already been explained in the beginning of the embodiment shown in fig. 3, and are not described again here. The first character recognition result is known as "228946", where is the question mark "? "part indicates that the character is not recognized successfully, it can be detected that the successfully recognized character in the result constitutes 2 character fields: 22, 8946. The character segment with the largest number of characters is 8946, and this is used as the target character. According to the characteristic "number" of the target character, the character type matched with the "number" can be determined to be the first character type, and then the character type corresponding to the target character area can be determined to be the first character type, namely the target character type is the first character type. According to the relative position relation in the template features of the target license plate template, the preset range on the left side of the target character region can be determined as a sub-region.

It should be noted that, after the step of selecting one license plate template from the unselected license plate templates in the license plate template library in step S209 as the target license plate template, the step S205C should be executed again.

In summary, in this embodiment, according to the feature of the character successfully recognized in the first character recognition result and the template feature of the target license plate template, the sub-area in the first license plate area can be determined more accurately, and some interference symbols that do not need to be recognized are excluded.

In the embodiment shown in fig. 3, in step S206 in the embodiment shown in fig. 2, the step of recognizing the characters in the sub-area and obtaining a second character recognition result may specifically include:

step S206A: and segmenting the sub-region to obtain a character region to be recognized.

When dividing the second card area, the sub-areas may be divided according to a vertical projection method and/or a connected component method. The specific process belongs to the prior art and is not described herein again.

As a specific embodiment, when the sub-region is divided, it may also be determined whether the division process for the above-mentioned region is successful according to the division result, if the division is successful, step S206B is executed, and if the division is unsuccessful, the division is performed again.

Step S206B: and identifying characters in the character area to be identified to obtain a second character identification result.

When the characters in the character area to be recognized are recognized, the characters in the character area to be recognized can be recognized according to a preset character classifier. The specific process belongs to the prior art and is not described herein again.

In addition, in this embodiment, the template features of the target license plate template may further include: the character size of the character of the first character type is larger than the size of the character of the second character type. Correspondingly, the accuracy of the character segmentation process can be further improved according to the characteristics.

Therefore, in another implementation manner based on the embodiment shown in fig. 3, in step S206A, that is, the step of segmenting the sub-region to obtain the character region to be recognized may specifically include:

step 1: and determining the character successfully recognized in the first character recognition result as the target character.

Step 2: and obtaining a first size according to the size of the target character.

Wherein the dimension may be at least one of a width and a height. The size of the target character is the size of the area where the target character is located, namely the size of a rectangular frame tightly clamped around the target character in the license plate image.

It should be noted that, for the sake of brevity and clarity, in the embodiments of the present application, the size of the character refers to the size of the area where the character is located.

It will be appreciated that, in general, the size of the target characters is substantially the same. The first size is a size value that can represent the size of each character in the target character.

Specifically, when the first size is obtained according to the size of the target character, an average value of the sizes of the respective characters in the target character may be calculated, and the average value is used as the first size.

And step 3: and matching the characteristics of the target character with the first character characteristics and the second character characteristics respectively to determine the character type of the target character.

The character type of the target character is either the first character type or the second character type.

And 4, step 4: and determining a second size according to the first size, the character type of the target character and the relative size relationship.

When the character type of the target character is the first character type, the first size is the size of the first character type character. At this time, the size of the second character type character can be determined according to the size of the first character type character and the relative size relationship, and the size is the second size.

When the character type of the target character is the second character type, the first size is the size of the second character type character. At this time, the size of the first character type character can be determined according to the size of the second character type character and the relative size relationship, and the size is the second size.

And 5: and according to the second size, segmenting the sub-region to obtain a character region to be recognized.

Specifically, when the sub-region is divided, the sub-region may be firstly divided by using a vertical projection method and/or a connected domain method for the first time, and then the result of the first division may be corrected according to the second size on the basis of the first division.

In summary, in the embodiment, according to the size of the character successfully recognized in the first character recognition result and the relative size relationship, the size of the character in the character region to be recognized can be determined, and the character in the sub-region can be more accurately segmented according to the size, so that the accuracy of the character segmentation result can be improved.

Fig. 4 is a schematic flowchart of a license plate recognition apparatus provided in an embodiment of the present application, which corresponds to the embodiment shown in fig. 2 and is applied to an electronic device, where the apparatus includes:

the license plate region determining module 401 is configured to obtain a license plate image to be recognized, and determine a first license plate region in the license plate image to be recognized;

a first character recognition module 402, configured to recognize characters in the first license plate area, and obtain a first character recognition result;

a first recognition and judgment module 403, configured to judge whether a license plate number that is not successfully recognized exists in the first license plate area according to the first character recognition result;

a target template selection module 404, configured to select a license plate template from a preset license plate template library as a target license plate template when a license plate number that is not successfully identified exists in the first license plate region, where the license plate template library is used to store each license plate template and corresponding template features;

a sub-region determining module 405, configured to determine, according to the template features corresponding to the target license plate template, a sub-region where a license plate number is not successfully recognized from the first license plate region;

a second character recognition module 406, configured to recognize characters in the sub-region, and obtain a second character recognition result;

a second recognition and judgment module 407, configured to judge whether the character recognition for the sub-region is successful according to the second character recognition result;

a license plate number obtaining module 408, configured to, when the character recognition for the sub-region is successful, obtain a license plate number of the license plate image to be recognized according to the first character recognition result and the second character recognition result;

and the target template replacing module 409 is configured to, when the character recognition for the sub-region is unsuccessful, select a license plate template from the license plate templates not selected in the license plate template library as a target license plate template, and return to execute the sub-region determining module 405.

In a specific implementation manner based on the embodiment shown in fig. 4, the first identification and judgment module 403 may specifically be configured to:

In a specific implementation manner based on the embodiment shown in fig. 4, the license plate number obtaining module 408 may specifically include:

a target character determination sub-module (not shown in the figure) for determining the successfully recognized character in the first character recognition result as a target character;

and a license plate number obtaining sub-module (not shown in the figure) for synthesizing the target character and the second character recognition result to obtain the license plate number of the license plate image to be recognized.

Fig. 5 is another schematic structural diagram of a license plate recognition method according to an embodiment of the present application, where the embodiment is an improved embodiment based on the embodiment shown in fig. 4, and the unmodified portions are the same as those in the embodiment shown in fig. 4. This embodiment corresponds to the method embodiment shown in fig. 3. In this embodiment, the template features of the target license plate template include: the character recognition method comprises the following steps of obtaining a first character feature corresponding to a first character type, a second character feature corresponding to a second character type and a relative position relation between a character area of the first character type and a character area of the second character type;

the sub-region determining module 405 may include:

a target character determination submodule 501, configured to determine a character successfully recognized in the first character recognition result as a target character;

a character region determining submodule 502, configured to determine a region where the target character is located as a target character region;

the character type determining sub-module 503 is configured to match the features of the target character with the first character features and the second character features, and determine a character type corresponding to the successfully matched character feature as a target character type corresponding to the target character region;

and a sub-region determining sub-module 504, configured to determine, according to the target character region, the target character type, and the relative position relationship, a sub-region of the license plate number that is not successfully recognized from the first license plate region.

In the embodiment shown in fig. 5, the execution character type determination submodule 503 should be returned after the target template replacement module 409.

In a specific implementation manner based on the embodiment shown in fig. 5, the target character determining sub-module 501 may be specifically configured to:

Based on a specific implementation manner of the embodiment shown in fig. 5, the second character recognition module 406 may specifically include:

the segmentation submodule 505 is configured to segment the sub-region to obtain a character region to be recognized;

and the identifying submodule 506 is configured to identify characters in the character region to be identified, and obtain a second character identification result.

In a specific implementation manner based on the embodiment shown in fig. 5, the template features of the target license plate template include: the character size of the character of the first character type is larger than that of the character of the second character type;

the segmentation sub-module 505 may be specifically configured to:

obtaining a first size according to the size of the target character;

Since the device embodiment is obtained based on the method embodiment and has the same technical effect as the method, the technical effect of the device embodiment is not described herein again.

For the apparatus embodiment, since it is substantially similar to the method embodiment, it is described relatively simply, and reference may be made to some descriptions of the method embodiment for relevant points.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

It will be understood by those skilled in the art that all or part of the steps in the above embodiments can be implemented by hardware associated with program instructions, and the program can be stored in a computer readable storage medium. The storage medium referred to herein is a ROM/RAM, a magnetic disk, an optical disk, or the like.

The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A license plate recognition method is characterized by comprising the following steps:

judging whether a license plate number which is not successfully recognized exists in the first license plate area or not according to the first character recognition result, wherein the license plate number which is not successfully recognized is a part of the license plate number which is not successfully recognized in the first license plate area;

determining a sub-region of the number plate number which is not successfully recognized from the first number plate region according to the template characteristics corresponding to the target number plate template, wherein the sub-region of the number plate number which is not successfully recognized is a sub-region containing the number plate number which is not successfully recognized in the first number plate region;

2. The method of claim 1, wherein the template features of the target license plate template comprise: the character recognition method comprises the following steps of obtaining a first character feature corresponding to a first character type, a second character feature corresponding to a second character type and a relative position relation between a character area of the first character type and a character area of the second character type;

3. The method according to claim 2, wherein the step of determining the successfully recognized character in the first character recognition result as the target character comprises:

4. The method of claim 1, wherein the step of identifying the character in the sub-region and obtaining a second character identification result comprises:

segmenting the sub-region to obtain a character region to be recognized;

5. The method of claim 4, wherein the template features of the target license plate template comprise: the character size of the character of the first character type is larger than that of the character of the second character type;

obtaining a first size according to the size of the target character;

6. The method of claim 1, wherein obtaining the license plate number of the license plate image to be recognized according to the first character recognition result and the second character recognition result comprises:

7. The method of claim 1, wherein the determining whether the license plate number which is not successfully recognized exists in the first license plate area according to the first character recognition result comprises:

8. A license plate recognition device, the device comprising:

the first recognition and judgment module is used for judging whether the number plate number which is not successfully recognized exists in the first number plate area or not according to the first character recognition result, wherein the number plate number which is not successfully recognized is the part number which is not successfully recognized of the number plate number in the first number plate area;

a sub-region determining module, configured to determine, according to template features corresponding to the target license plate template, a sub-region where a successfully unidentified license plate number is not identified from the first license plate region, where the sub-region where the successfully unidentified license plate number is not identified is a sub-region where the successfully unidentified license plate number is included in the first license plate region;

9. The apparatus of claim 8, wherein the template features of the target license plate template comprise: the character recognition method comprises the following steps of obtaining a first character feature corresponding to a first character type, a second character feature corresponding to a second character type and a relative position relation between a character area of the first character type and a character area of the second character type;

the sub-region determination module comprises:

10. The apparatus of claim 9, wherein the target character determination submodule is specifically configured to:

11. The apparatus of claim 8, wherein the second character recognition module comprises:

12. The apparatus of claim 11, wherein the template features of the target license plate template comprise: the character size of the character of the first character type is larger than that of the character of the second character type;

the partitioning submodule is specifically configured to:

obtaining a first size according to the size of the target character;

13. The apparatus of claim 8, wherein the license plate number obtaining module comprises:

14. The apparatus of claim 8, wherein the first identification and determination module is specifically configured to: