CN102722707A - License plate character segmentation method based on connected region and gap model - Google Patents

Abstract

The invention belongs to the technical field of digital image processing and pattern recognition, in particular to a license plate character segmentation method based on a connected area and gap model. The present invention first performs binarization processing on the license plate gray image, then marks the connected area on the binarized license plate image, initially obtains the character area in the license plate, and then establishes a gap model according to the ratio between the standard license plate characters to determine the characters of the license plate , thus completing the segmentation of the character area in the license plate. The invention effectively utilizes the size of characters and gaps of the standard license plate, and avoids the influence of unclear characters on character segmentation caused by license plate stains, image illumination and the like. Compared with traditional license plate character segmentation methods based on connected regions and projection segmentation, the present invention effectively improves the accuracy of license plate character segmentation, and better solves the problem of license plate character segmentation caused by unclear images.

Description

License Plate Character Segmentation Method Based on Connected Region and Gap Model

technical field

The invention belongs to the technical field of digital image processing and pattern recognition, and in particular relates to a method for recognizing license plates in traffic scenes in intelligent traffic management.

technical background

Intelligent Transportation System (Intelligent Transportation System, referred to as ITS) is to integrate electronic technology, computer technology, information technology, sensor technology and system engineering technology into various traffic road monitoring and management systems, vehicle control systems and road traffic control systems to meet the actual needs of ground transportation. An integrated system of traffic safety systems. Vehicle License Plate Recognition (VLPR for short) is one of the important components of the intelligent transportation system, and its application is very extensive. License plate recognition is based on digital image processing, pattern recognition, computer vision and other technologies. It analyzes the vehicle images or video sequences captured by the camera to obtain the unique license plate number of each car, thus completing the recognition process. Through some follow-up processing methods, the license plate recognition technology can realize the functions of parking lot fee management, traffic flow control index measurement, vehicle positioning, car anti-theft, expressway overspeed automatic supervision, etc., to maintain traffic safety and urban security, prevent traffic jams, realize The automatic management of traffic has practical significance.

The license plate recognition technology mainly includes three steps: the location of the license plate image, the segmentation of the license plate characters and the recognition of the license plate characters. "License plate location" is to use image processing methods and pattern recognition technology to accurately locate the position of the license plate from a digital image of a vehicle with an uncertain background, and provide an accurate and reliable data source for subsequent license plate character segmentation and recognition processing. "Character segmentation" is to divide the license plate area into a single character area for the next step of character recognition. The characters of the license plate are composed of limited Chinese characters, English letters and numbers. "Recognition of license plate characters" is to classify the separated characters to be recognized to identify them.

Among them, "character segmentation" is a link between the past and the future of license plate recognition technology, and the correctness of character segmentation directly affects the result of license plate character recognition. However, "character segmentation" is a more difficult part, and its difficulty is mainly reflected in the following issues:

(1) Due to the weather conditions, the license plate image has different cleanliness. In the license plate binary image obtained after the license plate preprocessing, there may be character adhesion and character stroke breakage. Therefore, in the character segmentation method, it should be avoided. Characters are split into one character, and one character is split into multiple characters by mistake.

(2) Since the width of the character "1" is narrower than that of other characters, how to distinguish the character "1" from interfering strokes, especially the vertical borders left and right of the license plate, should be fully considered in the method.

(3) The preprocessed license plate binary image cannot accurately locate the upper and lower boundaries of the characters, and is also interfered by the frame and rivets. In addition, there will be interference on the left and right sides of the license plate, especially the left and right borders, and may Glue to the first or last character. Therefore the designed segmentation method cannot be affected by these disturbing regions.

(4) Due to the influence of light during shooting, all or part of the license plate area is particularly bright, resulting in little distinction between the characters and the background color. The segmentation method should consider a suitable grayscale and binarization method to accurately distinguish the background regions and character regions.

Several character segmentation methods commonly used at present are:

(1) Character segmentation method based on projection method. Use the vertical projection method to quickly find the optimal segmentation point between characters, and use the horizontal projection to remove interference such as license plate borders.

(2) A character segmentation method based on the geometric features of license plate characters. This method first uses mathematical morphology to perform a series of morphological operations on the binarized license plate image to remove some useless information, so that the characters and the left and right borders of the license plate and the gap between characters become larger, so that it is easy to draw the gap between characters. Vertical dividing line.

(3) A method for segmentation of license plate characters with maximum inter-class variance based on template matching. According to the structure and size characteristics of the string, a license plate string template is designed, which is classified by sliding matching in the license plate area, and combined with the maximum inter-class variance judgment criterion to determine the best matching position and segment the license plate characters.

(4) A character segmentation method based on scanning. This method utilizes the method of searching from the middle to both ends to determine the upper and lower boundaries of the character, and utilizes the one-dimensional cycle clearing method and the relevant rules of the license plate character to carry out vertical segmentation to obtain a single character.

(5) A method for segmenting license plate characters based on connected region analysis. That is, according to the principle that the pixels belonging to the same character form a connected area, combined with the relevant rules of the license plate characters, the character segmentation problem of the license plate under complex background conditions is better solved.

(6) License plate character segmentation method based on neural network and color features. In the color space, the tree-type decision structure is used to first identify the white and black pixels in the license plate area from the brightness information, and then use the network to identify the blue, red, yellow and other colors in the license plate area. According to the color feature of the license plate, after the type of the license plate is judged, the binarization process is performed on the license plate area. After removing the license plate frame and rivets, the license plate characters are segmented by comprehensively using the projection method and the character connectivity.

The above license plate character segmentation methods are gradually improved during the development of license plate character segmentation methods, but none of them can solve the problem of defaced or affected license plate character segmentation, especially if there are two or three character areas in the license plate that are blurred or blurred. In the case of confusion, these methods often cannot accurately segment, and in actual traffic scenarios, the accuracy of character segmentation will be greatly reduced.

Due to the complexity of the application scenarios in the traffic field, the problems to be solved by the license plate recognition technology are quite complicated. The current license plate recognition technology still has some shortcomings in practical applications, such as: low external brightness, bad lighting conditions, special weather conditions, complex non-license plate area interference, etc. will cause certain difficulties to the positioning of the license plate; the license plate itself The influence of cleanliness or lighting conditions will cause difficulty in character segmentation; there is also some difficulty in recognizing and distinguishing similar characters.

Contents of the invention

The object of the present invention is to provide a method for segmenting license plate characters based on a connected region and gap model in order to solve the problem that the license plate characters are difficult to distinguish due to the above-mentioned license plate defacement and uneven illumination.

The license plate character segmentation method based on the connected area and the gap model proposed by the present invention is to perform the following steps after positioning and obtaining the license plate:

(1) Grayscale and binarization processing

① Use the component method to grayscale the license plate image. The steps are: use the brightness of the RGB three-component in the color image as the grayscale value of the three grayscale images, and select one of the grayscale images according to the application needs;

Since the red component of the blue license plate is more obvious, and the green component of the yellow license plate is more obvious, in order to increase the distinction between the character area and the background, it is necessary to grayscale the blue and yellow license plates;

② After the grayscale processing in step (1)-①, the license plate image is binarized. The steps are: use the NiBlack dynamic threshold binarization algorithm, and calculate a certain pixel point (coordinate point) in the license plate image through the formula ) threshold, compare the threshold with the pixel value of the coordinate point, if it is less than the threshold, it is the background color, and if it is greater than the threshold, it is the character color, and the binary image of the license plate is obtained;

(2) Mark connected regions

The connected area is an eight-connected area, that is, when any pixel point has the same pixel value as the eight adjacent pixel points around it, it is regarded as connected between them; based on this standard, step (1)- ② The obtained binary image is marked with the connected region according to the recursive method; after the connected region is marked and determined, record the minimum and maximum horizontal and vertical coordinates of the connected region;

(3) Remove impurity noise

Calculate the geometric features of the connected region through the minimum and maximum horizontal and vertical coordinates of the connected region recorded in step (2), and judge whether the connected region is a character region; if the condition is met, keep it; otherwise, set it as the background color;

(4) Horizontal cutting

The license plate of the vehicle is fixed on the vehicle by rivets. In the binary image of the license plate, part of the character area is connected to the rivet, making it connected with the frame to form a connected area. Such an area will be blocked because it does not conform to the geometric characteristics of the character area. Delete; the upper and lower central coordinate points of the reserved character area are respectively stored in a set, and linear fitting is performed on the two coordinate point sets respectively, and a straight line is obtained from the upper and lower points respectively; the two straight lines connect the character area with the The rivet is separated, and the operation of step (2) and step (3) is repeated, and the character area connected with the rivet is reserved;

(5) The gap calculation restores the remaining characters.

By calculating the gap between adjacent connected regions, calculate how many characters and how many gaps exist between these two connected regions, and then restore the character region that was deleted by mistake according to the standard width of the known region and the calculated gap width; These continuous reserved character areas calculate the standard width and standard height of the characters, and then calculate the width of the gap through the standard width, and finally use the character frame determined by the standard width and standard height to expand left and right according to the gap, and finally determine the seven characters of the license plate region to complete the segmentation.

The positive effect of the present invention is:

(1) Utilizing the characteristics of binary image character connectivity and license plate character standard geometry, the irregular impurity noise can be removed through the geometric shape information of the connected area.

(2) Use the position information of the determined character area to perform upper and lower cutting, which can well avoid the influence of rivets and other impurities connected with characters on the segmentation.

(3) Using the ratio of the standard character size of the license plate and the character gap to calculate the position of the character, the key function is to be able to restore the character area that was deleted due to stains or lighting effects that do not conform to the geometric characteristics.

Description of drawings

Fig. 1 is a block flow diagram of the license plate character segmentation method based on the connected region and gap model of the present invention.

Figure 2 is a schematic diagram of the NiBlack binarization algorithm.

Fig. 3 is a flowchart of a method for marking a circulation area.

Fig. 4 is a flowchart of a method for removing impurities.

Fig. 5 is a type table of the reserved area of the license plate character area.

Figure 6 is a standard car license plate (the image comes from the GA36-2007 standard).

Fig. 7 is a table of license plate character gap types.

Fig. 8 is a type table after the gap expansion of the reserved area of the binary image of the license plate.

Fig. 9 is an effect diagram of each step of processing a license plate by the method of the present invention.

Detailed ways

The specific implementation of the license plate character segmentation method based on the connected region and gap model of the present invention is explained below in conjunction with the accompanying drawings, but it should be pointed out that the implementation of the present invention is not limited to the following embodiments.

A license plate character segmentation method based on the connected region and gap model. First, the license plate image is grayed and binarized, and the impurity noise is removed after marking the connected region, and then the two steps of marking the connected region and removing the impurity noise are repeated after the horizontal cutting step, and finally restore the remaining characters through gap calculation to obtain seven character regions.

The specific operation steps of the method of the present invention are as shown in accompanying drawing 1.

1. Grayscale and binarization processing

First, this is a blue license plate, and the red component is binarized; then NiBlack dynamic threshold binarization is performed.

The threshold of this point is determined by calculating the statistical properties (mean and variance) of each pixel in the image and its neighborhood. The schematic diagram of the method is shown in Figure 2. The calculation method is shown in the following formula:

                                                                              

         

的阈值计算公式，k是权重系数，

、

分别是以坐标点

为中心正方形的邻域内所有像素点的方差和均值。得到此点的阈值后，就将其与该坐标点的像素值做比较，小于阈值T为背景颜色，大于阈值T为字符颜色。如下公式所示： Formula (1) is the coordinate point in the image

threshold Calculation formula, k is the weight coefficient,

,

are coordinate points

is the variance and mean of all pixels in the neighborhood of the central square. After getting the threshold of this point, compare it with the pixel value of this coordinate point, if it is less than the threshold T, it is the background color, and if it is greater than the threshold T, it is the character color. As shown in the following formula:

                         

表示二值图像

中坐标点的像素值，

为灰度图像中坐标点的像素值，为上述步骤计算出来的阈值。 In formula (2)

Represents a binary image

middle the pixel value of the coordinate point,

is the coordinate point in the grayscale image the pixel value of Threshold calculated for the above steps.

It can be seen that due to the large amount of dirt on the license plate, there are many impurity areas in the binarized license plate image, and the noise has a great influence.

2. Mark connected regions

According to the result of binarization in the previous step, black is the character color and white is the background color. According to the idea of recursion, the connected regions are marked, and the flow chart is shown in Figure 3:

Step 1. Set a mark for each pixel, initially set to 0;

Step 2. Traverse each pixel. If the pixel value is black and marked as 0, add 1 to the counter. If all pixels have been traversed, it will end. Otherwise, go to step 3;

Step 3. Mark this point as the value of the current counter, and update the minimum and maximum horizontal and vertical coordinate values of this connected region;

Step 4. Determine in turn whether the pixel values of the eight points around this point are black and marked as 0. If the conditions are met, enter this point and continue to step 3; if none of the eight points meet the conditions, continue to search for the previous one recursively The connected area of the point, if the recursion ends, go to step 5;

Step 5. Calculate the characteristic values such as the height, width and aspect ratio of this connected region according to the minimum and maximum horizontal and vertical coordinates of the record, and judge whether this connected region is a character region according to these characteristic values, if so, then keep this region; if If not, mark this area as the background color and decrement the counter by 1. Go back to step 2.

After the connected region is marked and determined, record the minimum and maximum horizontal and vertical coordinates of the connected region.

3. Remove impurity and noise areas according to geometric features

Due to the illumination, license plate frame and stains on the license plate, the license plate image after binarization has more impurities and noise. The geometric characteristics of the impurity noise are calculated by the minimum and maximum horizontal and vertical coordinates of the connected area to judge the connected area marked in the previous step. Is it a character area. If the condition is met, keep it; otherwise, set it as the background color. The program flow for removing impurities is shown in Figure 4; respectively calculate the height of the connected region, the width of the connected region, the angle of the connected region, and the aspect ratio of the connected region, and judge whether these geometric features meet the conditions of the character region; retain the qualified connected area, if the condition is not satisfied, delete this connected area; then calculate the average width and average height of the connected area, and then judge whether the geometric characteristics of the connected area deviate greatly from the average value, if so, then Delete this connected region. After this step, you can see that only the "E" and "4" areas remain.

Four, up and down cutting

Calculate the height and position information of the character area through the minimum and maximum horizontal and vertical coordinates of the retained connected area, store the upper center coordinate points of the reserved character area in a set, and linearize this set of coordinate points through the least square method Fitting to get a fitted straight line; use the same method to linearly fit the lower center point of the reserved character area; the two straight lines cut the characters up and down, so that "B" and "2" and The upper and lower rivets or impurities are separated; and then the two steps of marking the connected area and removing impurities are performed again, so that the two character areas "B" and "2" also conform to the geometric characteristics of the character and are preserved.

5. Clearance Calculation

Through the calculation of the previous steps, due to the influence of stains and the interference of impurity noise such as the breakage of Chinese characters, not all character areas can be preserved. For the convenience of restoring the remaining characters, the Chinese character area is not used for the time being. If the remaining 6 letters and numbers can be successfully restored, the position of the Chinese characters can be determined by these letters and numbers.

The type table of the license plate character reserved area is shown in Figure 5. The solid black square or circle represents the reserved character area, while the hollow square or circle represents the character area that has been deleted by mistake. Restoring the hollow character area is the correct segmentation character. prerequisites.

，其中

是各省、直辖市的简称，

是大写英文字母，·是间隔符，后面的5个字符是英文字符或阿拉伯数字。车牌的原始尺寸中每个字符所占宽度一定，均为45mm，字符之间以及字符与间隔符之间的间隙为12mm，间隔符宽10mm，字符高度均为90mm”。所以标准间隙和标准字宽存在固定的比例，其中后五个字符以及前两个字符的间隙与标准字宽的比例约为0.267，称之为小间隙；由于间隔符在上述方法中会被认为是杂质而删去，于是第二个和第三个字符间的间隙与标准字宽的比例约为0.756，称之为大间隙。 As shown in Figure 6, since the standard license plates all follow a unified standard: "a sequence of 7 characters consisting of a province Chinese character followed by characters or Arabic numerals, the arrangement format of the standard license plate is:

,in

It is the abbreviation of provinces and municipalities directly under the Central Government.

is an uppercase English letter, · is a spacer, and the following 5 characters are English characters or Arabic numerals. The width of each character in the original size of the license plate is 45mm, the gap between characters and between characters and spacers is 12mm, the width of spacers is 10mm, and the height of characters is 90mm". Therefore, the standard gap and standard character The width has a fixed ratio, and the ratio of the gap between the last five characters and the first two characters to the standard character width is about 0.267, which is called a small gap; because the spacer will be considered as an impurity in the above method and deleted, Therefore, the ratio of the gap between the second and third characters to the standard word width is about 0.756, which is called a large gap.

The gap type table shown in Figure 7 can calculate the number of characters and several gaps between the two connected regions by calculating the gap between adjacent connected regions, and then calculate it based on the standard width of the known region The width of the size gap is used to restore the character regions that have been deleted by mistake, so that there are no missing character regions in these connected regions. The area between "E" and "4" satisfies the width of two small gaps for one character, so the character "1" in the middle is restored.

After the gap expansion shown in the gap type table, the reserved character area "B E142" is continuous, and only the character areas on both sides are missing to be restored, so the types of reserved areas will be greatly reduced, as shown in Figure 8. When there are three consecutive character areas remaining, how to expand the left and right remaining character areas can be uniquely determined by judging whether the first gap is a large gap; because "B" and "E" conform to the width of a large gap, so the left and right Extend one character. In addition, when there are 4, 3 or even 2 continuous connected regions without large gaps, it can be judged how many character regions can be expanded from the right side of the continuous connected region to the boundary, so as to expand left and right.

Then use these continuous reserved character areas to calculate the standard width and height of the characters to determine the character frame, and finally frame the seven character areas of the license plate to complete the segmentation; the rendering of a license plate processed by the above method is shown in Figure 9.

Claims (1)

1. A license plate character segmentation method based on connected regions and gap models, characterized in that, after the license plate is acquired at the location, the operation of the following steps is carried out: (1) Grayscale and binarization processing ① Use the component method to grayscale the license plate image. The steps are: use the brightness of the three components in the color image as the grayscale value of the three grayscale images, and select one of the grayscale images according to the application needs; Grayscale processing of blue and yellow license plates; ② After the grayscale processing in step (1)-①, the license plate image is binarized. The steps are: use the NiBlack dynamic threshold binarization algorithm to calculate the threshold of a pixel in the license plate image, and use the threshold Compare with the pixel value of the coordinate point, if it is less than the threshold, it is the background color, and if it is greater than the threshold, it is the character color, and obtain the binary image of the license plate; (2) Mark connected regions The connected area is an eight-connected area, that is, when any pixel point has the same pixel value as the eight adjacent pixel points around it, it is regarded as connected between them; based on this standard, step (1)- ② The obtained binary image is marked with the connected region according to the recursive method; after the connected region is marked and determined, record the minimum and maximum horizontal and vertical coordinates of the connected region; (3) Remove impurity noise Calculate the geometric features of the connected region through the minimum and maximum horizontal and vertical coordinates of the connected region recorded in step (2), and judge whether the connected region is a character region; if the condition is met, keep it; otherwise, set it as the background color; (4) Horizontal cutting The license plate of the vehicle is fixed on the vehicle by rivets. In the binary image of the license plate, part of the character area is connected to the rivet, making it connected with the frame to form a connected area. Such an area will be blocked because it does not conform to the geometric characteristics of the character area. Delete; the upper and lower central coordinate points of the reserved character area are respectively stored in a set, and linear fitting is performed on the two coordinate point sets respectively, and a straight line is obtained from the upper and lower points respectively; the two straight lines connect the character area with the The rivet is separated, and the operation of step (2) and step (3) is repeated, and the character area connected with the rivet is reserved; (5) The gap calculation restores the remaining characters; By calculating the gap between adjacent connected regions, it is judged how many characters and how many gaps exist between the two connected regions, and then restore the character region that has been deleted by mistake according to the standard width of the known region and the calculated gap width; These continuous reserved character areas calculate the standard width and standard height of the characters, and then calculate the width of the gap through the standard width, and finally use the character frame determined by the standard width and standard height to expand left and right according to the gap, and finally determine the seven characters of the license plate region to complete the segmentation.

Images