WO2018219054A1

WO2018219054A1 - Method, device, and system for license plate recognition

Info

Publication number: WO2018219054A1
Application number: PCT/CN2018/083385
Authority: WO
Inventors: 韦立庆; 薛睿; 钮毅; 罗兵华
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2017-06-02
Filing date: 2018-04-17
Publication date: 2018-12-06
Also published as: CN108985137B; CN108985137A

Abstract

Provided in embodiments of the present application are a method, device, and system for license plate recognition. The method for license plate recognition comprises: detecting for characters in a captured license plate image by means of a preset image segmentation algorithm, determining the positions of the characters in the license plate image; determining a tilt angle range of the license plate image on the basis of the positions of the characters in the license plate image; rotationally correcting the license plate image according to the tilt angle range to produce a corrected license plate image; acquiring the apex coordinates of the corrected license plate image, transforming the apex coordinates via perspective transformation to produce a transformed license plate image in which the characters are equally tall; and recognizing the characters in the transformed license plate image to produce a recognized license plate. The present solution increases the computational efficiency and success rate of license plate recognition.

Description

License plate recognition method, device and system

This application claims the priority of the Chinese Patent Application entitled "A License Plate Recognition Method, Apparatus and System" by the Chinese Patent Office, filed on Jun. 2, 2017, the entire disclosure of which is hereby incorporated by reference. in.

Technical field

The present application relates to the field of intelligent transportation technologies, and in particular, to a license plate recognition method, device and system.

Background technique

License plate recognition technology is an application of computer video image recognition technology in vehicle license plate recognition. The license plate recognition technology requires that the license plate can be extracted and recognized from the complex background, and the vehicle license number can be identified through techniques such as license plate extraction, image preprocessing, feature extraction, and license plate character recognition. Therefore, the license plate recognition technology has been widely used in scenes such as bayonet, parking lot and electronic police to obtain the brand information of vehicles in the scene, and plays an important role in many aspects such as public security management.

When the camera is shooting an image, the captured license plate image may be tilted due to the shooting angle problem. For such problems, the corresponding license plate recognition method, by rotating the pre-processed image by n degrees in clockwise and counterclockwise directions, each rotation by 1 degree, obtains 2n rotated images, and statistically calculates the horizontal differential projection of each rotated image. The horizontal difference value is obtained, and then the horizontal difference mean value is calculated, and the non-tilted image in which the horizontal difference mean value is selected among the 2n+1 rotation images is determined, and the license plate in the non-tilted image is finally recognized.

However, for a license plate example similar to the illegal ball scene shown in FIG. 1, the illegal ball is a special ball type camera device for shooting illegal parking applied in the traffic management system, and the tilt of the photographed license plate image 101 is taken. The angle is often large. If the above method is used, the value of n needs to be set large, which increases the time consumption of the system operation and reduces the operation efficiency. Moreover, since the angle of the shooting is too large, the acquired license plate image will be deformed, resulting in a high probability of failure in license plate recognition.

Summary of the invention

The purpose of the embodiment of the present application is to provide a license plate recognition method, device and system to improve the operation efficiency and success rate of license plate recognition. The specific technical solutions are as follows:

In a first aspect, an embodiment of the present application provides a license plate recognition method, where the method includes:

The characters in the collected license plate image are detected by a preset image segmentation algorithm to determine the position of each character in the license plate image;

Determining a range of tilt angles of the license plate image according to a position of each character in the license plate image;

Performing rotation correction on the license plate image according to the tilt angle range to obtain a corrected license plate image;

Obtaining a vertex coordinate of the corrected license plate image, and transforming the vertex coordinates by a perspective transformation to obtain a converted license plate image with a character contour;

Each character in the converted license plate image is identified to obtain a recognized license plate.

Optionally, the preset image segmentation algorithm comprises: a maximum stable extreme value region MSER image segmentation algorithm;

The determining, by using a preset image segmentation algorithm, the characters in the collected license plate image to determine the position of each character in the license plate image, including:

The MSER image segmentation algorithm is used to detect characters in the collected license plate image to obtain the MSER frame position of each character;

The position of the MSER box of each character is determined as the position of the corresponding character.

Optionally, determining, according to a position of each character in the license plate image, a range of tilt angles of the license plate image, including:

Determining a center point coordinate of each character in the license plate image according to a position of each character in the license plate image;

Determining a slope of the license plate image compared to a horizontal direction by a least squares method according to a center point coordinate of each character;

Determining, according to the slope, an offset angle of the license plate image compared to a horizontal direction;

Determining a range of tilt angles of the license plate image according to the offset angle and a preset search angle range.

Optionally, the performing the rotation correction on the license plate image according to the tilt angle range to obtain a corrected license plate image includes:

Performing a Radon transform on the license plate image according to the tilt angle range to obtain a horizontal tilt angle of the license plate image;

Performing rotation correction on the license plate image according to the horizontal tilt angle to obtain a license plate image after horizontal tilt correction;

Performing a Radon transform on the license plate image after the horizontal tilt correction according to the tilt angle range, to obtain a vertical tilt angle of the license plate image after the horizontal tilt correction;

Correcting the horizontal tilt corrected license plate image according to the vertical tilt angle to obtain a corrected license plate image.

Optionally, the acquiring the coordinates of the vertex of the corrected license plate image, transforming the coordinates of the vertex by perspective transformation, and obtaining a converted license plate image with the same character height, including:

Obtaining vertex coordinates of the corrected license plate image according to the position of the MSER frame of the first character of the corrected license plate image and the position of the MSER frame of the last character;

Transforming the vertex coordinates to obtain vertex transformation coordinates of a character contour;

Calculating the vertex transformation coordinates to obtain transformation parameters;

Obtaining a mapping relationship between the vertex coordinates and the vertex transformation coordinates according to the perspective transformation general formula and the transformation parameters;

A converted license plate image is obtained based on the mapping relationship and the corrected license plate image.

Optionally, the identifying each character in the converted license plate image to obtain the recognized license plate includes:

Performing character position matching on the converted license plate image by using a plurality of preset license plate templates, and determining a position of a character in the converted license plate image that matches a preset license plate template;

The characters in each position are respectively input into a preset neural network for template matching, and the template matching confidence of each character is obtained;

Acquiring the MSER result confidence corresponding to the position of the MSER frame of each character in the converted license plate image;

The template matching confidence of each character is compared with the MSER result confidence of the MSER frame position corresponding to the character, and when the MSER result confidence is greater than the template matching confidence, the character of the location is updated to be the MSER result. The character corresponding to the position of the MSER box;

The updated converted license plate image is determined to be the recognized license plate.

In a second aspect, an embodiment of the present application provides a license plate recognition device, where the device includes:

a character position determining module, configured to detect, by using a preset image segmentation algorithm, characters in the collected license plate image to determine a position of each character in the license plate image;

a license plate tilt angle determining module, configured to determine a tilt angle range of the license plate image according to a position of each character in the license plate image;

a license plate rotation correction module, configured to perform rotation correction on the license plate image according to the tilt angle range to obtain a corrected license plate image;

a license plate distortion correction module, configured to acquire vertex coordinates of the corrected license plate image, and transform the vertex coordinates by perspective transformation to obtain a converted license plate image with a character contour;

The license plate character recognition module is configured to identify each character in the converted license plate image to obtain the recognized license plate.

The character position determining module is specifically configured to:

Optionally, the license plate tilt angle determining module is specifically configured to:

Optionally, the license plate rotation correction module is specifically configured to:

Optionally, the license plate distortion correction module is specifically configured to:

Optionally, the license plate character recognition module is specifically configured to:

Obtaining a confidence level of the MSER result corresponding to the position of the MSER frame of each character in the converted license plate image;

In a third aspect, an embodiment of the present application provides a license plate recognition system, where the system includes:

An image capture device for photographing a vehicle to obtain a license plate image;

a memory for storing a computer program;

The method steps provided by the first aspect of the embodiments of the present application are implemented when the processor is configured to execute the program stored in the memory.

In a fourth aspect, an embodiment of the present application provides a storage medium for storing executable code, where the executable code is used to execute at a runtime: the method steps provided by the first aspect of the embodiment of the present application.

In a fifth aspect, an embodiment of the present application provides an application program for performing, at runtime, the method steps provided by the first aspect of the embodiment of the present application.

A method, device and system for license plate recognition provided by an embodiment of the present invention obtain a position of each character in a license plate image by a preset image segmentation algorithm, and determine a range of inclination angles of the license plate image according to the position of each character, according to the The tilt angle range is used to rotate the license plate image, and the coordinate transformation is performed by perspective transformation to obtain the license plate image with the same character height. Finally, each character is identified, thereby reducing the number of images participating in the operation and improving the operation efficiency of the license plate recognition. And through the preset image segmentation algorithm, the accuracy of character segmentation is improved, and the influence of character distortion is effectively solved by perspective transformation, thereby improving the success rate of license plate recognition.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application and the technical solutions of the prior art, the following description of the embodiments and the drawings used in the prior art will be briefly introduced. Obviously, the drawings in the following description are only Some embodiments of the application may also be used to obtain other figures from those of ordinary skill in the art without departing from the scope of the invention.

1 is an example of a license plate in a prior art ball scene;

2 is a schematic flow chart of a license plate recognition method according to an embodiment of the present application;

3 is a schematic diagram of a character position detected by an MSER according to an embodiment of the present application;

4a is a license plate image after horizontal tilt correction obtained by rotating after X-direction Radon transform according to an embodiment of the present application;

4b is a corrected license plate image corrected by the Radon transform in the Y direction according to an embodiment of the present application;

FIG. 5a is a schematic diagram of obtaining key points based on character information according to an embodiment of the present application; FIG.

FIG. 5b is a converted license plate image of a character contour obtained by perspective transformation according to an embodiment of the present application; FIG.

6 is a schematic structural diagram of a license plate recognition device according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a license plate recognition system according to an embodiment of the present application.

detailed description

In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings. It is apparent that the described embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

In order to improve the computing efficiency and success rate of the license plate recognition, the embodiment of the present application provides a license plate recognition method, device and system.

First, a license plate recognition method provided by an embodiment of the present application is introduced.

The execution body of the license plate recognition method provided by the embodiment of the present application may be a processor equipped with a core processing chip, for example, a DSP (Digital Signal Processor) and an ARM (Advanced Reduced). Instruction Set Computer Machines, processors of core processing chips such as FPGAs (Field-Programmable Gate Arrays). A manner of implementing a license plate recognition method provided by an embodiment of the present application may be at least one of software, hardware circuits, and logic circuits disposed in an execution body.

As shown in FIG. 2, a license plate recognition method provided by an embodiment of the present application may include the following steps:

S201: The preset image segmentation algorithm is used to detect characters in the collected license plate image to determine the position of each character in the license plate image.

The image segmentation algorithm may be an edge-based image segmentation algorithm, a region-based image segmentation algorithm, or a texture-based image segmentation algorithm. An appropriate image segmentation algorithm can be selected by the characteristics of the image samples that need to be segmented. The preset image segmentation algorithm includes: an image segmentation algorithm of MSER (Maximally Stable Extremal Regions). Currently in the field of image processing, MSER is considered to be the best performing affine invariant region, and MSER is the most stable region obtained when binarizing images using different gray thresholds. The MSER has the following characteristics: it has invariance to the affine change of the image gray scale; stability, the support set of the region is stable with respect to the gray scale change; and the region of different fineness can be detected. Therefore, in order to make the image segmentation have higher stability and can detect different degrees of fineness, the embodiment of the present application adopts the MESR as a preset image segmentation algorithm. Specifically, the MSER extraction process includes: binarizing the image using a series of gray thresholds; obtaining a corresponding black region and a white region for each binary image obtained by the threshold; determining a wider gray threshold range Maintain a stable shape inside the area. The preset image segmentation algorithm is used to detect the characters in the license plate image. The actual process is to identify and segment the characters in the license plate image by using a preset image analysis algorithm, thereby identifying the region where each character in the license plate image is located. Depending on the vertex coordinates of the area in which each character is located, or based on the distance of the vertex of the area in which each character is located from the edge of the image, the position of the area in which each character is located can be determined to determine the position of each character. Other ways of determining the position by the coordinates, the distance, the angle, and the like are all in the protection scope of the embodiment of the present application, and are not described herein again.

Optionally, S201 can be specifically:

In the first step, the MSER image segmentation algorithm is used to detect the characters in the collected license plate image to obtain the position of the MSER frame of each character;

In the second step, the position of the MSER box of each character is determined as the position of the corresponding character.

Through the MSER algorithm, the MSER frame of each character is obtained, and each MSER box contains one character, and then the specific position of each character in the image is determined, which can be determined according to the vertex coordinates of the MSER frame of each character, or according to the MSER The distance from the vertex of the frame to the edge of the image is determined. Other ways of determining the position by the coordinates, the distance, the angle, and the like are all in the protection scope of the embodiment of the present application, and are not described herein again.

S202. Determine a range of tilt angles of the license plate image according to the position of each character in the license plate image.

After the character position is determined, the tilt angle of the license plate image may be determined according to the position of each character. For example, the tilt angle of the license plate image may be determined according to the coordinates of the center point of each character, or may be based on the area in which each character is located. The vertex coordinates determine the tilt angle of the license plate image, and the tilt angle of the license plate image can also be determined according to a certain corresponding position coordinate of the region in which each character is located, which is a protection range of the embodiment of the present application. Since there is interference during image capturing, there is a certain error between the tilt angle and the actual tilt angle by the above method. Therefore, in order to improve the detection accuracy, a certain deviation can be set based on the tilt angle obtained above. Range to determine the range of tilt angles of the license plate image. For example, if the calculated license plate tilt angle is 29 degrees, a deviation of 3 degrees may be allowed on the basis, and the tilt angle of the license plate image may be greater than or equal to 26 degrees and less than or equal to 32 degrees.

Optionally, S202 can be specifically:

In the first step, determining the coordinates of the center point of each character in the license plate image according to the position of each character in the license plate image;

In the second step, according to the coordinates of the center point of each character, the slope of the license plate image compared to the horizontal direction is determined by a least squares method;

The third step is to determine an offset angle of the license plate image compared to the horizontal direction according to the slope;

In the fourth step, the tilt angle range of the license plate image is determined according to the offset angle and the preset search angle range.

In order to ensure the accuracy of the tilt angle calculation, since the size of the area in which each character is located is uncertain, if the tilt angle is determined according to the vertex coordinates of the area, a certain error may be caused. Therefore, in this embodiment, each The center point coordinates of the character determine the tilt angle. After determining the coordinates of each center point, the slope of all the coordinates of the center point can be obtained, and the slope is calculated to obtain the inclination angle of the line connecting all the coordinates of the center point, that is, the inclination angle of the license plate image. Of course, other passing coordinates The manner of determining the tilt angle of the information also belongs to the protection scope of the embodiment of the present application, and details are not described herein again.

In order to improve the efficiency of the operation and ensure the minimum error between the actual data and the actual data, in the present embodiment, the slope of the connection is determined by the least squares method for the determined central point coordinates, and the least squares method is to find the square of the error by minimizing the error. The best function matching method for data. That is, the abscissa and the ordinate of the center point of all the characters are respectively subjected to the operation of averaging, and then brought into y=kx+b, where y is the average of the ordinates of the center points of all the characters, and k is the center The slope of the line connecting the point coordinates, x is the average of the abscissa of the center point of all characters, b is a constant, and then the slope of the line connecting all the coordinates of the center point is obtained by the inverse trigonometric function arctan(k). The angle of inclination of the license plate image. Moreover, in order to improve the accuracy of the detection, a preset search angle range is set based on the obtained tilt angle. For example, if the preset search angle range is set to θ, the tilt angle range of the license plate image is [arctan(k) -θ, arctan(k) + θ].

S203: Perform rotation correction on the license plate image according to the tilt angle range to obtain a corrected license plate image.

Since the acquired license plate image has a certain inclination angle compared to the horizontal direction, in order to achieve a better recognition effect, the license plate image needs to be rotated to the horizontal direction. Through the above method, the range of the tilt angle of the license plate image can be obtained, that is, the degree of tilt of the license plate image, for example, the license plate image is inclined by 35 degrees compared with the horizontal direction, and the license plate image needs to be rotated by 35 degrees to obtain the corrected license plate image. . Rotation correction can be understood as rotating the license plate image according to the range of the tilt angle. Of course, in order to achieve better results, the image can be rotated by, for example, Radon transform, which performs the horizontal axis direction and the vertical axis direction, respectively. Transform, get the angle that needs to be rotated, and then rotate the license plate image separately according to the horizontal axis direction and the vertical axis direction. The corrected license plate image thus obtained not only ensures that the whole is horizontal, but also ensures that each character is also horizontal. Certainly, other manners of performing rotation correction on the image also belong to the protection scope of the embodiment of the present application, and details are not described herein again.

Optionally, S203 can be specifically:

In the first step, the Radon transform is performed on the license plate image according to the tilt angle range to obtain the horizontal tilt angle of the license plate image;

In the second step, the license plate image is rotated and corrected according to the horizontal tilt angle to obtain the license plate image after the horizontal tilt correction;

In the third step, the Radon transform is performed on the license plate image after the horizontal tilt correction according to the tilt angle range, and the vertical tilt angle of the license plate image after the horizontal tilt correction is obtained;

In the fourth step, the license plate image corrected by the horizontal tilt is corrected according to the vertical tilt angle to obtain a corrected license plate image.

Since the Radon transform transforms and rotates the horizontal axis direction and the vertical axis direction respectively, the obtained corrected license plate image not only ensures that the whole is horizontal, but also ensures that each character is also horizontal. Therefore, this embodiment adopts the Radon transform method. The license plate image is rotated for correction. The specific implementation process of the Radon transform is based on the tilt angle range. First, the Radon transform is performed on the license plate image. The cumulative result of the first derivative is obtained from the Radon transform result. When the sum is maximum, the corresponding value is the license plate. The horizontal tilt angle of the image is corrected horizontally according to the horizontal tilt angle, and the Radon transform is performed on the horizontally corrected image according to the tilt angle range, and the absolute value of the first derivative is added to the Radon transform result. And, when accumulating and maximizing, the corresponding value is the vertical tilt angle of the image, and the horizontally corrected image is corrected in the vertical direction according to the vertical tilt angle to obtain a corrected license plate image, and the obtained corrected license plate image is a whole. Very positive license plate image.

S204: Acquire a vertex coordinate of the corrected license plate image, and transform the vertex coordinates by a perspective transformation to obtain a converted license plate image with the same character height.

If the angle of inclination of the acquired license plate image is too large, the characters in the corrected license plate image obtained after the rotation correction are closer to the image acquisition device side, and the characters away from the image acquisition device side are smaller because the character width and height are not Uniformity leads to a higher probability of failure for subsequent character recognition. Therefore, it is necessary to adjust the height and width of the characters of the corrected license plate image to adjust the image to a converted license plate image having the same character height. The height and width adjustment of the image can be realized by transforming the coordinates, and the process of the coordinate transformation is actually a process of projection. Therefore, the embodiment of the present application uses a perspective transformation method to transform the coordinates. The perspective transformation is to project the image to a new view plane, and each coordinate point has a corresponding relationship with the original image in the new view plane.

Optionally, S204 may be specifically:

In the first step, according to the position of the MSER frame of the first character of the license plate image and the position of the MSER frame of the last character, the vertex coordinates of the corrected license plate image are obtained;

In the second step, the vertex coordinates are transformed to obtain vertex transformation coordinates of the character contour;

In the third step, the coordinates of the vertex transformation are calculated to obtain a transformation parameter;

In the fourth step, according to the general formula of the perspective transformation and the transformation parameters, the mapping relationship between the vertex coordinates and the vertex transformation coordinates is obtained;

In the fifth step, the converted license plate image is obtained according to the mapping relationship and the correction of the license plate image.

Since there are a plurality of coordinate points in one image, if the transformation is performed for each coordinate point, the calculation amount is huge. Therefore, in this embodiment, only the coordinates of the four vertex of the corrected license plate image need to be transformed, and other coordinates are used. The corresponding license image can be obtained to obtain the converted license plate image. Specifically, the general transformation formula of perspective transformation is as shown in (1):

Where x is the abscissa of the vertex in the transformed license plate image, y is the ordinate of the vertex in the transformed license plate image, u is the abscissa of a vertex in the license plate image, and v is the ordinate of the vertex in the license plate image, a ₁₁ to a ₃₃ are transformation parameters. Therefore, it is known that the four vertices corresponding to the transformation can determine the vertex coordinates of the transformed license plate image. Specifically, the four vertex coordinates of the corrected license plate image can be obtained according to the vertex coordinates of the MSER frame position of the first and last characters. Let the coordinates of the four vertices of the corrected license plate image be (x0, y0) (x1, y1) (x2, y2) (x3, y3), in order to ensure the contour height of each character of the transformed image, the transformed point The coordinates should be (x0-b, y1)(x1, y1)(x2, y2)(x3-b, y2), where b is the correction value, that is, the adjusted point (xb, y1) and point The abscissas of (x3-b, y2) are equal, and thus the values of a ₁₁ to a ₃₃ are obtained by the general transformation formula (1), thereby obtaining a mapping relationship.

S205: Identify each character in the converted license plate image to obtain the recognized license plate.

After the converted license plate image is obtained, the characters of the license plate need to be identified, for example, by BP neural network recognition, according to convolutional neural network recognition, or self-learning recognition, characters in the license plate image can be recognized. Certainly, the manner of identifying the characters in the license plate image is within the scope of protection of the embodiment of the present application, and details are not described herein again. Taking BP network as an example, BP network is a multi-layer feedforward network trained by error inverse propagation algorithm. It is one of the most widely used neural network models. The BP network has the ability to learn and store a large number of input-output mode mapping relationships without the need to disclose the mathematical equations describing such mapping relationships in advance.

Optionally, S205 can be specifically:

In the first step, using a plurality of preset license plate templates to perform character position matching on the converted license plate image, and determining a position of the character in the converted license plate image that matches the preset license plate template;

In the second step, the characters in each position are respectively input into a preset neural network for template matching, and the template matching confidence of each character is obtained;

In the third step, obtaining the confidence of the MSER result corresponding to the position of the MSER frame of each character in the converted license plate image;

In the fourth step, the template matching confidence of each character is compared with the MSER result confidence of the MSER frame position corresponding to the character, and when the MSER result confidence is greater than the template matching confidence, the character of the position is updated as the MSER result corresponding to the MSER. The character of the box position;

In the fifth step, it is determined that the updated converted license plate image is the recognized license plate.

The method of template matching is to search for a target in a large image. It is known that there are targets to be found in the figure, and the target has the same size, direction and image as the template, and the target can be found in the figure by a certain algorithm. Determine the coordinate position, and determine the template matching confidence of the characters at each position by the neural network identification method. Due to image segmentation, selection, and character recognition, some characters of the license plate image may be distorted, so that the value of the template matching confidence may be lower, wherein the confidence may be a probability value or an evaluation value. The probability value is the probability of which standard character is a character in the license plate image. The evaluation value is a score of which standard character is similar to a character in the license plate image according to the definition, and the score may be a score within 10 minutes. It can be a score within 100 points. The MSER method can obtain the confidence of the MSER result relative to the position of the MSER frame when segmenting the image, that is, the degree to which the standard characters in a certain MSER box are. The result matching confidence of the template matching and the confidence of the MSER result are compared character by character, and the corresponding character position in the converted license plate image is updated by the larger result of the template matching result confidence and the MSER result confidence, in all characters After the identification is completed, it is possible to determine the updated license plate image. Of course, in order to improve the accuracy of character recognition, a threshold may be set, and the position of the character is updated when the confidence of the MSER result is greater than the threshold of the template.

Applying the embodiment, the position of each character in the license plate image is obtained by preset image segmentation algorithm, and the tilt angle range of the license plate image is determined according to the position of each character, and the license plate image is rotated and corrected according to the tilt angle range. Through the perspective transformation, the coordinate transformation is carried out to obtain the license plate image with the same character height. Finally, each character is identified, thereby reducing the number of images participating in the operation, improving the operation efficiency of the license plate recognition, and improving the characters by the preset image segmentation algorithm. The accuracy of segmentation and the effect of character distortion are effectively solved by perspective transformation, thereby improving the success rate of license plate recognition.

The license plate recognition method provided by the embodiment of the present application is introduced below in conjunction with a specific application example.

For the license plate image as shown in FIG. 1, the license plate image collected by the image acquisition device, the position of the character detected by the MSER is as shown by the square formed by each character in FIG. 3, and the straight line 301 in FIG. 3 is the license plate. The line connecting the coordinates of the center point of all the characters in the image is obtained by calculating the slope of the line 301 to obtain a tilt angle of the license plate image of 40 degrees, and setting the angle range to be searched to be 15 degrees, and the tilt angle range of the license plate image is 25 Degree to 55 degrees. Then, according to the tilt angle range of the license plate image, the tilted license plate is rotated by the X-direction Radon transform, and the rotated license plate image is shown in FIG. 4a, and then the Y-direction Radon transform is performed to correct the rotated license plate image. A corrected license plate image is shown in Figure 4b.

As shown in FIG. 4b, due to the tilt of the license plate image, the characters in the corrected license plate image obtained after the rotation correction are closer to the image acquisition device side, and the characters away from the image acquisition device side are smaller. For the situation existing in FIG. 4b, the figure is 4b corresponds to the image of the MSER frame position, as shown in Figure 5a, extract the coordinates of points a, b, c, d in the figure, points b, c can be obtained based on the MSER frame position of the last character, points a, d can be based on the first The position of the MSER frame of the character is obtained. The coordinates of the four points a, b, c, and d are respectively (x0, y0) (x1, y1) (x2, y2) (x3, y3), and the coordinates of the points obtained after the transformation are respectively For (x0-b, y1)(x1, y1)(x2, y2)(x3-b, y2), b is a correction value, which can be determined according to the effect of the corrected image that the image is expected to achieve, and The perspective transformation formula (1) obtains a mapping relationship, and Fig. 5a is corrected to obtain a converted license plate image of the character contour as shown in Fig. 5b. For the converted license plate image of the character contour as shown in FIG. 5b, each character in the converted license plate image of the same character height can be sent to the BP network for recognition, since characters may be distorted during the above transformation process, The way the template matches the results can be corrected by the MSER results.

Compared with the related art, in the solution, the position of each character in the license plate image is obtained by a preset image segmentation algorithm, and the tilt angle range of the license plate image is determined according to the position of each character, and the license plate is used according to the tilt angle range. The image is rotated and corrected, and the coordinate transformation is performed by perspective transformation to obtain the license plate image with the same character height. Finally, each character is recognized, thereby reducing the number of images participating in the operation, improving the operation efficiency of the license plate recognition, and adopting the preset image. The segmentation algorithm improves the accuracy of character segmentation and effectively solves the influence of character distortion through perspective transformation, thereby improving the success rate of license plate recognition.

Corresponding to the above embodiment, the embodiment of the present application provides a license plate recognition device. As shown in FIG. 6, the device may include:

a character position determining module 610, configured to detect, by using a preset image segmentation algorithm, characters in the collected license plate image to determine a position of each character in the license plate image;

a license plate tilt angle determining module 620, configured to determine a tilt angle range of the license plate image according to a position of each character in the license plate image;

a license plate rotation correction module 630, configured to perform rotation correction on the license plate image according to the tilt angle range to obtain a corrected license plate image;

a license plate distortion correction module 640, configured to acquire vertex coordinates of the corrected license plate image, and transform the vertex coordinates by perspective transformation to obtain a converted license plate image with a character contour;

The license plate character recognition module 650 is configured to identify each character in the converted license plate image to obtain the recognized license plate.

Optionally, the preset image segmentation algorithm may include: a maximum stable extreme value region MSER image segmentation algorithm;

The character position determining module 610 can be specifically configured to:

Optionally, the license plate tilt angle determining module 620 can be specifically configured to:

Optionally, the license plate rotation correction module 630 can be specifically configured to:

Correcting the horizontally corrected license plate image according to the vertical tilt angle to obtain a corrected license plate image.

Optionally, the license plate distortion correction module 640 can be specifically configured to:

Optionally, the license plate character recognition module 650 is specifically configured to:

The license plate recognition device of the embodiment of the present application is a device for applying the license plate recognition method, and all the embodiments of the license plate recognition method are applicable to the device, and both can achieve the same or similar beneficial effects.

Corresponding to the above embodiment, the embodiment of the present application provides a license plate recognition system. As shown in FIG. 7, the system may include:

An image capturing device 710 is configured to capture a vehicle to obtain a license plate image;

a memory 720, configured to store a computer program;

The processor 730 is configured to detect, by using a preset image segmentation algorithm, characters in the license plate image collected by the image capturing device 710, and determine a position of each character in the license plate image; according to each of the license plate images a position of the character, determining a range of the tilt angle of the license plate image; performing rotation correction on the license plate image according to the tilt angle range to obtain a corrected license plate image; acquiring vertex coordinates of the corrected license plate image, and performing perspective transformation on the The vertex coordinates are transformed to obtain a converted license plate image having the same character height; each character in the converted license plate image is identified to obtain the recognized license plate.

The processor 730 can be specifically configured to:

The processor 730 is specifically configured to:

Determining the slope of the license plate image relative to the horizontal direction by a least squares method according to the center point coordinates of each character;

The processor 730 is specifically configured to:

The license plate recognition system of the embodiment of the present application is a system for applying the license plate recognition method and device, and all the embodiments of the license plate recognition method and device are applicable to the system, and both can achieve the same or similar beneficial effects.

In addition, corresponding to the license plate recognition method provided by the foregoing embodiment, the embodiment of the present application provides a storage medium for storing executable code, which is used for execution at runtime: provided by the embodiment of the present application. a license plate recognition method; specifically, the license plate recognition method includes:

And detecting, by using a preset image segmentation algorithm, characters in the collected license plate image to determine a position of each character in the license plate image, including:

Obtaining vertex coordinates of the corrected license plate image according to the MSER frame position of the first character of the corrected license plate image and the MSER frame position of the last character;

In this embodiment, the storage medium stores an application program that executes the license plate recognition method provided by the embodiment of the present application at runtime, and thus can realize: obtaining a position of each character in the license plate image by using a preset image segmentation algorithm, and according to The position of each character determines the range of the tilt angle of the license plate image, according to the tilt angle range, the license plate image is rotated and corrected, and coordinate transformation is performed through perspective transformation to obtain a license plate image with the same character height, and finally each character is identified. Therefore, the number of images participating in the operation is reduced, the operation efficiency of the license plate recognition is improved, and the accuracy of the character segmentation is improved by the preset image segmentation algorithm, and the influence of the character distortion is effectively solved by the perspective transformation, thereby improving the success rate of the license plate recognition.

In addition, corresponding to the license plate recognition method provided by the foregoing embodiment, the embodiment of the present application provides an application program for performing the foregoing steps of the license plate recognition method provided by the embodiment of the present application.

In this embodiment, the application performs the license plate recognition method provided by the embodiment of the present application at runtime, so that the position of each character in the license plate image can be obtained by using a preset image segmentation algorithm, and according to the position of each character. Determining the range of the tilt angle of the license plate image, rotating the license plate image according to the tilt angle range, and performing coordinate transformation by perspective transformation to obtain a license plate image with the same character height, and finally identifying each character to reduce the participation operation The number of images improves the computational efficiency of license plate recognition, and improves the accuracy of character segmentation through preset image segmentation algorithm, and effectively solves the influence of character distortion through perspective transformation, thereby improving the success rate of license plate recognition.

For the application and the storage medium embodiment, since the method content involved is basically similar to the foregoing method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the term "comprises" or "comprises" or "comprises" or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

The various embodiments in the present specification are described in a related manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

The above is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., which are made within the spirit and principles of the present application, should be included in the present application. Within the scope of protection.

Claims

A license plate recognition method, characterized in that the method comprises:

The characters in the collected license plate image are detected by a preset image segmentation algorithm to determine the position of each character in the license plate image;

Determining a range of tilt angles of the license plate image according to a position of each character in the license plate image;

Performing rotation correction on the license plate image according to the tilt angle range to obtain a corrected license plate image;

Obtaining a vertex coordinate of the corrected license plate image, and transforming the vertex coordinates by a perspective transformation to obtain a converted license plate image with a character contour;

Each character in the converted license plate image is identified to obtain a recognized license plate.
The license plate recognition method according to claim 1, wherein the preset image segmentation algorithm comprises: a maximum stable extreme value region MSER image segmentation algorithm;

The determining, by using a preset image segmentation algorithm, the characters in the collected license plate image to determine the position of each character in the license plate image, including:

The MSER image segmentation algorithm is used to detect characters in the collected license plate image to obtain the MSER frame position of each character;

The position of the MSER box of each character is determined as the position of the corresponding character.
The license plate recognition method according to claim 1 or 2, wherein the determining the tilt angle range of the license plate image according to the position of each character in the license plate image comprises:

Determining a center point coordinate of each character in the license plate image according to a position of each character in the license plate image;

Determining a slope of the license plate image compared to a horizontal direction by a least squares method according to a center point coordinate of each character;

Determining, according to the slope, an offset angle of the license plate image compared to a horizontal direction;

And determining a range of the tilt angle of the license plate image according to the offset angle and the preset search angle range.
The license plate recognition method according to claim 1 or 2, wherein the correcting the license plate image according to the tilt angle range to obtain a corrected license plate image comprises:

Performing a Radon transform on the license plate image according to the tilt angle range to obtain a horizontal tilt angle of the license plate image;

Performing rotation correction on the license plate image according to the horizontal tilt angle to obtain a license plate image after horizontal tilt correction;

Performing a Radon transform on the license plate image after the horizontal tilt correction according to the tilt angle range, to obtain a vertical tilt angle of the license plate image after the horizontal tilt correction;

Correcting the horizontal tilt corrected license plate image according to the vertical tilt angle to obtain a corrected license plate image.
The license plate recognition method according to claim 2, wherein the acquiring the vertex coordinates of the corrected license plate image, and transforming the vertex coordinates by perspective transformation to obtain a converted license plate image having the same character height, comprising:

Obtaining vertex coordinates of the corrected license plate image according to the position of the MSER frame of the first character of the corrected license plate image and the position of the MSER frame of the last character;

Transforming the vertex coordinates to obtain vertex transformation coordinates of a character contour;

Calculating the vertex transformation coordinates to obtain transformation parameters;

Obtaining a mapping relationship between the vertex coordinates and the vertex transformation coordinates according to the perspective transformation general formula and the transformation parameters;

A converted license plate image is obtained based on the mapping relationship and the corrected license plate image.
The license plate recognition method according to claim 2, wherein the recognizing each character in the converted license plate image to obtain the recognized license plate comprises:

Performing character position matching on the converted license plate image by using a plurality of preset license plate templates, and determining a position of a character in the converted license plate image that matches a preset license plate template;

The characters in each position are respectively input into a preset neural network for template matching, and the template matching confidence of each character is obtained;

Obtaining a confidence level of the MSER result corresponding to the position of the MSER frame of each character in the converted license plate image;

The template matching confidence of each character is compared with the MSER result confidence of the MSER frame position corresponding to the character, and when the MSER result confidence is greater than the template matching confidence, the character of the location is updated to be the MSER result. The character corresponding to the position of the MSER box;

The updated converted license plate image is determined to be the recognized license plate.
A license plate recognition device, characterized in that the device comprises:

a character position determining module, configured to detect, by using a preset image segmentation algorithm, characters in the collected license plate image to determine a position of each character in the license plate image;

a license plate tilt angle determining module, configured to determine a tilt angle range of the license plate image according to a position of each character in the license plate image;

a license plate rotation correction module, configured to perform rotation correction on the license plate image according to the tilt angle range to obtain a corrected license plate image;

a license plate distortion correction module, configured to acquire vertex coordinates of the corrected license plate image, and transform the vertex coordinates by perspective transformation to obtain a converted license plate image with a character contour;

The license plate character recognition module is configured to identify each character in the converted license plate image to obtain the recognized license plate.
The license plate recognition device according to claim 7, wherein the preset image segmentation algorithm comprises: a maximum stable extreme value region MSER image segmentation algorithm;

The character position determining module is specifically configured to:

The MSER image segmentation algorithm is used to detect characters in the collected license plate image to obtain the MSER frame position of each character;

The position of the MSER box of each character is determined as the position of the corresponding character.
The license plate recognition device according to claim 7 or 8, wherein the license plate tilt angle determining module is specifically configured to:

Determining a center point coordinate of each character in the license plate image according to a position of each character in the license plate image;

Determining a slope of the license plate image compared to a horizontal direction by a least squares method according to a center point coordinate of each character;

Determining, according to the slope, an offset angle of the license plate image compared to a horizontal direction;

Determining a range of tilt angles of the license plate image according to the offset angle and a preset search angle range.
The license plate recognition device according to claim 7 or 8, wherein the license plate rotation correction module is specifically configured to:

Performing a Radon transform on the license plate image according to the tilt angle range to obtain a horizontal tilt angle of the license plate image;

Performing rotation correction on the license plate image according to the horizontal tilt angle to obtain a license plate image after horizontal tilt correction;

Performing a Radon transform on the license plate image after the horizontal tilt correction according to the tilt angle range, to obtain a vertical tilt angle of the license plate image after the horizontal tilt correction;

Correcting the horizontal tilt corrected license plate image according to the vertical tilt angle to obtain a corrected license plate image.
The license plate recognition device according to claim 8, wherein the license plate distortion correction module is specifically configured to:

Obtaining vertex coordinates of the corrected license plate image according to the position of the MSER frame of the first character of the corrected license plate image and the position of the MSER frame of the last character;

Transforming the vertex coordinates to obtain vertex transformation coordinates of a character contour;

Calculating the vertex transformation coordinates to obtain transformation parameters;

Obtaining a mapping relationship between the vertex coordinates and the vertex transformation coordinates according to the perspective transformation general formula and the transformation parameters;

A converted license plate image is obtained based on the mapping relationship and the corrected license plate image.
The license plate recognition device according to claim 8, wherein the license plate character recognition module is specifically configured to:

Performing character position matching on the converted license plate image by using a plurality of preset license plate templates, and determining a position of a character in the converted license plate image that matches a preset license plate template;

The characters in each position are respectively input into a preset neural network for template matching, and the template matching confidence of each character is obtained;

Obtaining a confidence level of the MSER result corresponding to the position of the MSER frame of each character in the converted license plate image;

The template matching confidence of each character is compared with the MSER result confidence of the MSER frame position corresponding to the character, and when the MSER result confidence is greater than the template matching confidence, the character of the location is updated to be the MSER result. The character corresponding to the position of the MSER box;

The updated converted license plate image is determined to be the recognized license plate.
A license plate recognition system, characterized in that the system comprises:

An image capture device for photographing a vehicle to obtain a license plate image;

a memory for storing a computer program;

The processor, when executed to execute the program stored on the memory, implements the method steps of any of claims 1-6.
A storage medium for storing executable code for performing at runtime: the method steps of any of claims 1-6.
An application, characterized in that it is executed at runtime: the method steps of any of claims 1-6.