TWI522934B - Gyro sensor license plate recognition system for smart phone and method thereof - Google Patents

Description

Gyro sensor license plate recognition system for smart phone and method thereof

The invention relates to a gyroscope sensor license plate recognition system and a method thereof for a smart phone, in particular to an image recognition technology capable of randomly identifying a vehicle license plate character.

According to the license plate recognition system, it is a common image recognition technology in the field of image processing. Most of the conventional license plate recognition systems are built on fixed-type photographic equipment, and are connected with computers to transfer information to each other for calculation. As for its application level, it is such as e-car parking lot management system, traffic violation vehicle detection system, and brake identification system. When setting up these types of systems, it is possible to infer from the common sense that the vehicle will travel through the predictable route, and the license plate information falls within the system detection range. Therefore, the license plate recognition system only needs to be equipped with a fixed monitor to align the camera at a specific angle. It is possible to obtain an image of the approximate position of the license plate in advance and return it, and then calculate the result. Furthermore, the license plate recognition system has developed many mature algorithms and their thematic applications. The systems developed through various algorithms have their own performances, and the algorithms and processing effects are distinguished by the system flow that is generally seen. Can be divided into the following parts: image pre-processing, such as gray-scale conversion as shown in the attached reference [3], spatial filtering (smoothing as shown in the attached reference [4], and sharpening as attached reference [5] ] shown, binarized (global, or regional binarization as shown in the attached reference [6]), edge detection as shown in the attached reference [7], histogram homogenization as attached reference [8] As shown, and the license plate skew correction is shown in the attached reference [9], which is mainly used to connect the main algorithms of the system, obtain the information required by the system, and pre-process the input image. Processing, to achieve the best image, improve the overall performance of the system; license plate positioning, the main purpose is to search for the location of the license plate in the image, the correlation algorithm such as feature density determination such as the attached reference [10], gradient change statistics such as Attachment reference [11], etc.; license plate character cutting, license plate positioning results usually contain license plate characters, the more common process is to separate the characters by cutting, such as block marking method (or connected object method) Attached to the reference [12], the character boundary search method, etc., some people have proposed that there is no need to cut characters, such as character feature analysis as shown in the attached reference [13]; post-processing, assuming license plate characters The license plate characters obtained during the cutting stage are not ideal. The image will be post-processed and corrected before character recognition, such as character breakage detection and correction. License plate character recognition is used to identify the main calculation of characters. Methods such as template characterization, neural networks such as attachment references [14], nearest neighbors and hierarchical comparisons such as annex references such as annex references [15] Shown, wherein the direction of projection eight characters accessory Reference [16], the blur and (of Fuzzy) theory as shown in Annex reference [17]. In addition, for skewed license plate detection, as shown in the attached reference [18], the oblique character cut is shown in the attached reference [19].

Broadly speaking, in the field of image processing, many algorithmic processes include image pre-processing steps, in order to make the subsequent algorithms have better results, faster processing speed, or to get more consistent The image information required by the algorithm; among the various application algorithms, image pre-processing is especially common in the field of license plate recognition. The reason is that the content contained in the target image, in addition to the license plate itself, there are other such as The complex background information, the environmental factors of the target, and the state of the target itself and its surroundings are based on the premise that the main algorithm will apply the pre-processing results, and then select the appropriate pre-processing method to match the license plate identification. system. In the field of license plate recognition, the common type of image pre-processing is The target can roughly include the following: First, reduce the amount of image information, such as grayscale conversion, color filtering, limited detection range, etc., such algorithms are usually reduced by reducing the complexity of image information. Accelerate the processing speed of the subsequent steps of the system, or limit the detection range, while retaining the detection target, thereby improving the system operation efficiency of the subsequent steps; second, adjusting the image information intensity type, such as edge gradient enhancement, Histogram equalization, Gaussian blur, change of image contrast, etc. The main purpose of this type of algorithm is to highlight the detection target, remove noise, reduce the degree of image impact on the environment, etc. The third type belongs to the pre-operation type, in order to match the corresponding The various pre-processing proposed by the main algorithm, such as image environment brightness ratio calculation, as shown in the attached reference [22], image scale scaling, etc., this part of the algorithm links the main calculation process of the license plate recognition system, its main goal is to find The position of the license plate in the image, the accuracy of positioning, or the speed of calculation.

In addition, after searching by the applicant, it was found that the patent related case related to the case is, for example, the Republic of China Invention Disclosure No. 200943931 "Handheld Camera with Recognizable License Plate"; the Republic of China Invention No. I236638 "Multiple Identification System for Identifying License Plates and Method]; Republic of China Invention Disclosure No. 201234287 "License Plate Number Identification Method"; Republic of China Invention Disclosure No. 201220213 "Intelligent Image Type License Plate Automatic Identification and Monitoring System"; and Republic of China Invention No. 584811 "Image Structured License Plate Automatic According to the patents such as the identification system, it is found that although these patents can all achieve the effect of identifying the license plate; however, these patents do not have the function of setting the intuitive limited image area and image angle correction, so the subsequent processing The image size can not be effectively reduced, so that the image processing speed is slow, and the image angle skew caused by the hand-shake that must be faced by the smart phone cannot be solved. Moreover, the patents have no conditional iterative binarization calculus. The function is built, so the result of image binarization cannot be improved. The license plate position detection capture and the license plate character cutting and other processing can not accurately output, thus affecting the license plate character recognition success machine Rate, so these patents are indeed not perfect, there is still the need for further improvement.

In view of this, there is no patent or paper that combines the smart phone, intuitive limited image area, image angle correction and conditional iterative binary calculus to set the license plate recognition technology, and based on related industries. Under the urgent need, the present inventors have finally developed a set of inventions different from the above-mentioned prior art and patents through continuous efforts.

A first object of the present invention is to provide a gyro sensor license plate recognition system for a smart phone that can solve the above-mentioned conventional technology, which is mainly provided by an intuitively limited image area and an image angle correction function. In addition to the effective reduction of the image size of the subsequent processing to speed up the image processing speed, the image angle pre-processing and the license plate position caused by the hand-shake that must be faced by the correcting smart phone can be solved together. Detection is effectively connected. The technical means adopted to achieve the first object of the present invention is to provide a license plate recognition module for a smart phone. The license plate recognition module can perform image pre-processing, license plate position detection and capture processing, license plate area component segmentation processing, license plate area component feature extraction processing and license plate character recognition processing on the image of the license plate captured by the smart phone. And then outputting license plate character recognition result information; wherein, the license plate recognition module performs image pre-processing on the image of the region of interest; and extracts an angle signal generated by the gyroscope sensor as a corrected rotation region of interest According to the angle of the image, the user can use the convenience of the smart phone to achieve the purpose of randomly identifying the vehicle license plate image.

A second object of the present invention is to provide a smart phone license plate recognition system with conditional iterative binarization function, which is mainly constructed by a conditional iterative binarization calculation process. In order to effectively improve the result of image binarization, the treatment of license plate position detection and license plate character cutting can obtain more accurate output results, thereby improving the probability of recognition of license plate characters. The technical means adopted for achieving the second object of the present invention include a sphere, a motion sensing module and a power supply module. The first wireless transmission module is configured to sense a motion state of the sphere to generate motion sensing data. The power supply module is configured to supply power to the motion sensing module and the first wireless transmission module, so that the first wireless transmission module wirelessly transmits the motion sensing data to the outside for subsequent use. Wherein, when the license plate recognition module obtains an error result in the license plate position detection and capture processing, conditional iterative binarization processing is performed, and in the conditional iteration binarization processing, the license plate recognition module Then calculate the ratio of the license plate rectangle and determine whether the ratio is greater than the preset proportional threshold; when the judgment result is yes, the license plate position is captured and output; when the judgment result is no, it is determined whether the preset iteration is completed. The number of times; when the judgment result is yes, the license plate position is captured and output; when the judgment result is no, the threshold value is re-corrected, and the image is re-binarized and returned to the license plate position detection and capture. The steps of processing.

10‧‧‧Smart mobile phone

11‧‧‧ display screen

20‧‧‧ images

21‧‧‧Areas of interest

21a‧‧‧Rectangular selection area

30‧‧‧ License Plate Identification Module

1 is a schematic flow chart of a license plate recognition module system of the present invention.

2 is a schematic diagram of a conditional iterative binarization flow chart of the present invention.

Figure 3 is a schematic illustration of a region of interest in an image of the present invention.

Fig. 4 is a schematic view showing the image rotation correction of the present invention.

FIG. 5 is a schematic diagram of the implementation of the present invention for scanning license plate characters by the horizontal change amount marking method.

FIG. 6 is another schematic diagram of the present invention for scanning license plate characters by the horizontal change amount marking method.

Figure 7 is a schematic illustration of a candidate axis mark of the present invention.

Figure 8 is a schematic diagram of block scanning of the present invention.

Figure 9 is a schematic diagram showing the result of the license plate positioning of the present invention.

Figure 10 is a schematic illustration of the boundary of the components of the license plate area of the present invention.

Figure 11 is a schematic illustration of the feature binary code extraction of the elements of the present invention.

Attachments are references to the present invention.

壹. First embodiment of the present invention

Referring to FIG. 1 , FIG. 3 and FIG. 4 , in order to achieve the first embodiment of the present invention, a license plate recognition module 30 is disposed on the smart phone 10 . The license plate recognition module 30 is configured to sequentially perform image pre-processing, license plate position detection and capture processing on the image 20 containing the license plate captured by the image capturing device (not shown) built in the smart phone 10. The license plate area component segmentation process, the license plate area component feature extraction process, and the license plate character recognition process, thereby outputting the license plate character recognition result information; wherein the license plate recognition module 30 sets a region of interest for limiting the image range in the image 20 twenty one. The license plate recognition module 30 performs the image pre-processing only on the image of the region of interest 21 . In addition, the angle signal generated by the gyro sensor built in the smart phone 10 is taken as an angle basis for correcting the image of the rotation of the region of interest 21. The image pre-processing further includes grayscale conversion processing, Gaussian smoothing processing, and initial binarization processing on the image of the region of interest 21 in sequence.

Specifically, the region of interest 21 may be a rectangular frame selection area 21a selected by the user in the manner of the display screen 11 of the touch smart phone 10; or a rectangular frame selection area customized by the license plate recognition module 30. 21a, the rectangular frame selection area 21a has a width of 320 pixels wide and a height of 240 pixels. Further, the license plate recognition module 30 uses the Roller angle θ _roll converted by the gyro sensor to rotate the image of the region of interest 21 in the opposite direction with the image center of the region of interest 21 as the rotation axis. The θ _roll degree is used to complete the rotation correction of the image, and the license plate recognition module 30 determines whether the θ _roll is greater than 70 degrees or less than -70 degrees. When the judgment result is YES, it is not necessary to correct the image of the rotation of the region of interest 21 . The license plate position detection and capture processing system uses the horizontal change amount marking method to detect the license plate position and capture the license plate area image. The license plate area component division processing uses the boundary mark cutting method to cut the license plate area image into a plurality of components. The license plate area component feature extraction process uses a template feature comparison method to sequentially calculate a component feature value sufficient to describe the component feature, and the license plate recognition module 30 establishes a database, and the database is stored. There is a plurality of feature tables, each of which is written into a component class arranged in sequence, and each component class of the feature table corresponds to a feature value of the component, and the license plate character recognition process adopts a random forest method Comparing the component characteristic values of each component to each of the feature tables one by one, thereby obtaining the component class corresponding to each component, the component class containing characters and non-characters, the character including numbers and English letters .

贰. Second embodiment of the present invention

Referring to FIG. 1 to FIG. 4, in order to achieve the second embodiment of the present invention, a license plate recognition module 30 is disposed on the smart phone 10. The license plate recognition module 30 is configured to sequentially perform image pre-processing, license plate position detection and capture processing, license plate area component division processing on the image containing the license plate captured by the image capturing device built in the smart phone 10, The license plate area component feature extraction process and the license plate character recognition process, and then the license plate character recognition result information is outputted; wherein the license plate recognition module 30 is provided with an intuitive setting for the user to select or system settings and is used for limiting the image. The region of interest 21 of the image range. License plate recognition mode Group 30 performs this image pre-processing only on the image of region of interest 21. In addition, the angle signal generated by the gyro sensor built in the smart phone 10 is taken as an angle basis for correcting the image of the rotation of the region of interest 21. When the license plate recognition module 30 performs the license plate position detection and the capture processing to obtain an erroneous result, the conditional iterative binarization processing is performed, and in the conditional iterative binarization processing, the license plate recognition module 30 Then calculate the ratio of the license plate rectangle and determine whether the ratio is greater than the preset proportional threshold; when the judgment result is yes, the license plate position is captured and output; when the judgment result is no, it is determined whether the preset iteration is completed. The number of times; when the judgment result is yes, the license plate position is captured and output; when the judgment result is no, the threshold value is re-corrected, and the image is re-binarized and returned to the license plate position detection and capture. The steps of processing.

Participate in the specific implementation of the technical features of the present invention 3.1 Region of Interest Settings

Assume that the license plate recognition system established by the desktop computer and the monitor sets the image limiting range, which is not as applicable as the smart phone license plate recognition system of the present case. The reason is that the detection image obtained by the former is at a fixed viewing angle. There is no guarantee that the detection target will fall into the restricted area, and there are bound to be shortcomings such as the limitation of the system construction location, the effective calculation range is too small to detect the license plate, and the like, and when the case is built on the smart phone 10 The system is portable, and it detects the source of the license plate image from the perspective of continuous movement, from static targets, or moving vehicles. The region of interest (ROI) provided by the present invention can not only reduce the range of images to be processed, but also speed up the processing speed of the system, and on the other hand, the user can intuitively select the image range by touch. First, lock the license plate and its surroundings, as the target contains the area, improve the accuracy of finding the license plate, and indirectly affect the subsequent process accuracy.

Referring to FIG. 3, a schematic diagram of setting a region of interest 21 (ROI) on the display screen 11 is shown. In the figure, the center of the image 20 can be seen, and there is a region with a dotted line, which is the reduced area. The actual detection area, and the reduction calculation method, as shown in the following formulas (3.1) and (3.2):

The image shown in FIG. 3 is captured by the smart phone 10. The width and height are represented by W and H , respectively, and the unit is pixel (Pixel). The smart phone 10 used in the present invention has an image width of 800. The pixel is 480 pixels in height; the principle of setting the region of interest 21 (ROI) is based on user experience, and the constricted area is reduced to the center of the display screen, and the purpose is to make the user not interested in setting The usage habits before area 21 (ROI) are a bit like the feeling of aiming at the target; w 'and h ' are the unilateral reduction of the original width and height. The Ratio of Width used in the present invention, that is, the reduced width ratio is Ratio of Height, that is, the reduction height ratio is 1⁄4; the invention is based on the principle of the distance between the system and the license plate, and is reduced to a basic distance from the target without causing the target to be too large, if If the target is too close, it will increase the amount of calculation of the system, and the limitation on the ratio setting will be described in detail; after calculation, the actual width of the region of interest 21 (ROI) of the present invention is about 320 pixels wide and the height. It is about 240 pixels high.

3.2 Image Angle Acquisition and Correction

When the gyro sensor senses the rotation of the smart phone 10, the variable and the name in the three-axis direction, the X-axis is the rotation axis, which is called the Pitch rotation, and the Y-axis rotation. The axis is called the pitch rotation (Roll), and the Z axis is the rotation axis, which is called the yaw rotation (Yaw); when the smart phone 10 is laid flat, the three-axis rotation concept changes according to the above manner. In use, the smart phone 10 usually faces the screen horizontally, and the rotation variable of the three axes is different from that of the smart phone 10, which is called a system detection angle. Assume that the Roll, Pitch, and Yaw angles obtained when the smart phone 10 is laid flat are zero. When the smart phone 10 is turned into the system to detect the angle of view, as the front of the screen rotates the Pitch direction by 90° toward the user, the screen is displayed. Like the XZ plane; in order to avoid the user's non-horizontal image obtained by hand shaking or other reasons, the type of algorithm that can be used is limited, the source image will be rotated, and the correction angle can be returned by the gyroscope sensor. The Roll value is known. When the Pitch angle is between 0° and 90°, the Roll value varies from 90° to 179°, then from -179° to -90°, and when the Pitch angle is 90° to 0°. Between the time, the Roll value varies from 90° to 0°, and then from 0° to -90°. Since the system cannot pass the return value to determine which range the Pitch angle is in, it needs to be Set up a conversion type to convert the range of the Roll value so that it is not affected by the Pitch value. The conversion formula is as follows. By converting the formula (3.3), the system can obtain the Roll direction angle value that is not affected by the Pitch value, and then the system. Image correction can be performed through the converted angle returned by the G-Sensor.

3.3 Image pre-processing

The image pre-processing system of the present invention includes three steps of rotation correction, Gaussian smoothing, and initial binarization, and in order to ensure that the execution speed of the smart phone 10-terminal system is fast enough, many operations are directly using the OpenCV library. . The rotation correction of the image is to avoid the situation of hand-shake, and some algorithms cannot be used in the smart phone license plate recognition system. The concept of correction is to use the Roller angle θ _roll after the gyro sensor is converted. The ROI image center is the rotation axis, and the ROI image of the region of interest is rotated by θ _roll degrees in the opposite direction, that is, the image correction correction is completed. When the system performs rotation correction (ROI size is 320×240), it will judge whether θ _roll is greater than 70° or less than -70°. If it is true, it is regarded as unnecessary correction, that is, too large hand-held rotation angle The license plate recognition module 30 does not correct the ROI, so that the display error can be avoided. Next, the purpose of Gaussian smoothing of the region of interest 21 (ROI) is mainly due to the image smoothing process performed by the system in order to obtain better binarization results and improve the accuracy of license plate location. The method used for value is the Otsu method.

3.4 license plate position detection and capture processing

The invention adopts the horizontal change amount marking method to detect and capture the position of the license plate. In the field of license plate recognition, the color change amount of each horizontal line in the image is counted, and the licensee is marked, and the license plate in the search image is searched. Position is a relatively common practice. For a binarized image, the concept is to input the image and scan it line by line, calculate the number of black and white changes on each horizontal line, and use the license plate characters to arrange the least amount of change. In the case of the most variable amount, set a double threshold to mark the horizontal line that matches. Figures 5 and 6 show the combination of the least number of black and white changes (12 times) and the maximum number of times (36 times) in the license plate character arrangement, and the statistics and markings are as follows: peak _yn = peak _yn +1, if pixel _{yn , xn} ≠ pixel _{yn , xn +1} (3.4)

In formula (3.4), peak _yn represents the number of changes of the horizontal line. When scanning the horizontal axis of yn , the pixel value of each xn position on the axis is compared with the value of the next pixel. If the pixel value is different, the change is counted once, otherwise, Not counting; in formula (3.5), M indicates the state of the mark. When a certain yn- axis change is within the set interval, it is regarded as a candidate axis mark, and the upper and lower limits of the change contain ±4 fault tolerances. The information around the license plate interferes with the amount of calculation. In order to facilitate the observation, in general, the left image is complementary to the mark axis of the right image, and the license plate recognition module 30 obtains three candidate regions when searching for the license plate, which are respectively labeled as R _a , R _b , and R _{c .} , as shown in Figure 7. In order to facilitate the observation of the scanning schematic, the R _b region of FIGS. 7 to 9 is highlighted, and the dotted rectangular frame is used as the scanning block, and the total number of color changes in the block represented by each starting point xn is calculated from left to right. The number of times, and the starting point xn of the block with the most number of records, can be used to obtain preliminary positioning results. The above results are obtained under the condition that the pre-image processing effect is good. Assuming that the target area is affected by the surrounding environment, the system loses the ideal binarized license plate after preliminary binarization of the region of interest (ROI) image. Image, when the system calculates the amount of horizontal change, it will not mark the horizontal axis that belongs to the license plate, so that the calculation process finds the wrong upper and lower bounds of the license plate, and gets the wrong upper and lower bounds, and also calculates the wrong left and right bounds of the license plate. The rate of decline will drop. Since the built-in lens of the smart phone 10 automatically adjusts the internal aperture due to the ambient light, when the license plate recognition module 30 is in an extremely high brightness environment to capture images, the smart phone 10 adapts to make the overall display darker. The shadow part will be deepened; assuming there are some shadows and some of them are covered on the license plate characters, the image will be partially binarized and will cause some license plate characters to disappear, and the associated mark method can not find the correct upper and lower license plates. boundary. There are two directions for improving the environmental impact of binarization, one is oriented by the improved algorithm itself, such as the regional binarization method as shown in the attached reference [8] [23]. The other is to improve the problem of the environment itself, such as judging whether the brightness of the environment needs to be adjusted; the present invention proposes to use the conditional iterative binarization method, and the slight difference between the two directions is to modify the calculation process and join The method of judging uses the abnormal proportion of the output license plate to determine whether to re-binarize the image, and re-processes the corrected threshold value as the binarization threshold, replacing the Otsu binarization step.

3.5 conditional iterative binarization

Faced with environmental factors that produce less-than-ideal binarized images, the accuracy of the license plate recognition module 30 in searching for license plate positions is reduced, and most of the methods will explore from the algorithm itself whether there are still problems that are not enough to solve. At this stage, the present invention proposes to improve the degree of influence of deepening the shadow when the aperture of the smart phone 10 is affected by the ambient brightness through the modification process and adding a simple judgment, thereby making the accuracy of searching for the license plate position better. We call this the conditional iterative binarization process. The following is the operation flow chart: According to the control flow shown in Figure 2, first, the Gaussian smoothed region of interest 21 (ROI) image is used to perform the first image. Binarization (using the Otsu method), followed by the license plate position search, and the result of the output is abnormally determined. The criterion is the width ratio of the rectangular frame used to frame the license plate position, and is compared with the proportional threshold set by the system. If the former is smaller than the latter, it enters the iterative process. The setting of the proportional threshold is calculated based on the aspect ratio of the license plate; according to the type of license plate issued by the General Administration of Highways of the Ministry of Communications, there are old and new types. The old model has an aspect ratio of 2.1 (32 cm × 15 cm), new style. The license plate is about 2.4 (38 cm × 16 cm), plus the tolerance error based on the system test, the value is about 1.55, and finally the proportional threshold is 3.65. As for the ratio of the green rectangular frame of the license plate position, from left to right are 3.54, 3.31, and 1.66, respectively. There are still many positioning results due to the lack of the scale value due to the shadow, and the threshold value obtained by the Otsu method is not ideal. If you want to solve the problem of shadow influence positioning, you must use a lower value than the Otsu threshold to binarize. We can correct the threshold by formula (3.6): T _t = T _{t -1} ×Ratio of Reduction ( 3.6)

The threshold value calculated by the Otsu method is used as the initial T ₀ , and the threshold of the threshold is set to 15%. This value is not absolute, but if the setting is too large, the image information will disappear too quickly and must be noted. In the process of iteration, in order to prevent the license plate recognition module 30 from obtaining the license plate circle selection box that satisfies the condition, the upper limit of the number of iterations is set, and thus the entire process of iterative binarization is completed, and the license plate recognition module 30 sets the number of iterations to 3 times. The upper limit, that is to say, the threshold value is reduced by up to 40%. Because the threshold value is lowered, the license plate characters in the binarized image will not disappear, which increases the accuracy of finding the position of the license plate; this method is currently only applicable to the white background. Black-hand license plates, but for other types of license plates (such as different colors, or different number of characters), as long as other conditions are used to determine whether the license plate is abnormal, a similar process can be used to improve the shadow effect.

3.6 license plate area component cutting processing

The invention adopts the boundary mark cutting method to cut the license plate characters, and considers that there are various non-character elements in the license plate area, but it is possible to be cut out. In the subsequent license plate component identification stage, the license plate recognition module 30 can distinguish For all the cutting objects, some related methods, some people take advantage of the special license plate block feature cutting method, no need to cut out the complete character to identify, and some people use the connected object method to separate the independent objects in the license plate area Further identification; for the obtained license plate position, there are often bad elements such as characters connected to each other, or too many noises around the character, so that the effect of the cutting element is not satisfactory, however, the conditional expression proposed by the present invention By iterative binarization, the license plate recognition module 30 can obtain very good output results by using a simple boundary mark cutting method. The dotted arrow in Fig. 10 points to the left and right boundaries of the components in the license plate area. According to the experimental results, the system has been subjected to the aforementioned processing calculations, and the components can be easily cut out in this way.

3.7 license plate component identification processing

The method used in the identification stage of the present invention is similar to the template feature comparison method. The concept is to find an object to be identified, and through a special rule, calculate a value sufficient to describe the feature of the object, which is the component feature value; Many different sample objects exist between each other, and feature accumulation and calculus training are performed. It is calculated that they should belong to different categories. This method is the same as the cluster analysis. The license plate component set C , which contains the characters and non-characters output by the cutting component stage, the system must calculate the feature values of all components for comparison identification, and the calculation formula is as follows:

In formula (3.7), w and h respectively represent the width and height of the region of interest 21 (ROI), x and y are randomly generated coordinate positions, the range is limited to the region of interest 21 (ROI) image, and p is representative ( x , y ) the pixel value on the coordinates; in formula (3.8), all p belong to the pixel value set P , the subscript i is 0 to n , which represents the first pixel value in the set, and f is the feature binary code, The criterion j is 1 to n ; the meaning of this formula is that if the current pixel value is not equal to the next pixel value, the feature binary code is 1, otherwise it is 0; the formula (3.9) is the formula of the merged feature binary code, 11 is an example of this: It is assumed that the random coordinate points in Fig. 11 are arranged in order from top to bottom to assist the dotted line segment. The pixel value of the topmost coordinate point is p ₀ , and the lowermost layer is p ₆ because p ₀ It is not equal to p ₁ , so f ₁ is 1, and since p ₁ and p ₂ are equal, f ₂ is 0, and so on. Finally, the characteristic value of the element is 100111, and F is obtained by (3.9). 39; This example illustrates how to calculate a feature binary code, numerically describing the characteristics of the component, and on the other hand, if randomly generated The coordinate point, wherein the binary codes will be the more bits.

The license plate component identification is voted by referring to the feature table, and the person who gets the most votes is regarded as the category of the component. Assume that Table 1 extracts the positions from the three sets of eigenvalues randomly generated, and substitutes the feature table calculated by the template data. Each component has different index values corresponding to different feature tables. When the system calculates the characteristic values of the cutting components, According to the way of looking up the table, voting in the corresponding component category in the table, the category that gets the most votes is the identification result of the component. For example, inputting a component and calculating its characteristics, the feature value 42 is obtained from the first set of extraction positions, the second group is 80, and the third group is 22, then the corresponding component category 2 gets 2 votes, the component category Y Get 1 vote, the system judges this component to be the number 2.

The license plate component identification method proposed by the present invention is derived from Zdenek The research literature of Kalal et al., as shown in the attached reference [24], belongs to the deformation of Random Forest (RF). In the part describing the image features, the method generates multiple sets of random coordinates in the image. By comparing and combining, the feature binary code belonging to the target image is obtained, and the subsequent algorithm achieves the purpose of image tracking. The method of the invention adopts the method of using the method in part, and designs a classifier suitable for the identification of the license plate component; in the process of the license plate recognition system, we can obtain the image of the license plate component from the stage of cutting the license plate area, as the detective The target is established, and the pixel value of the point is obtained from the randomly generated coordinates, and the bit value is set by comparing the pixel values with each other, and all the obtained binary codes are used as the component feature values representing the image, and the component features. After the calculation is completed, the set of feature tables created by the template data is represented as the component's Ensemble Classifier, and finally the component identification is performed.

The random forest method used in the present invention is somewhat different from the original usage. When the forest is initially created, the decision tree is not built by random sampling, but all component templates are input, and the positions are extracted according to different feature values to establish Different decision trees form a "characteristic table forest". Finally, the ability of each weak classification is enhanced through training, so that the forest has a good component identification capability. Furthermore, a decision tree is based on a randomly generated set of coordinate points, which is built after all the templates are created. A decision tree model can be defined as the following formula (4.1):

The subscript P represents the set of coordinate points, and k represents the combination of the first feature extraction positions, when k The concept of 2nd forest is established, l represents the leaf node. When the component falls into a certain l , the ticket is voted for the component class represented by the leaf node; the more the template is input, the component represented in each leaf node. The category must be the highest as the final category. For example, there are 4 template components, and the license plate character represented by the category is 8. All the templates are calculated at the first feature extraction position, that is, k =1. The eigenvalue is directed to the first leaf node, which is l ₀ . Suppose there are 4 other license plate characters as the template of B. When k =1, 2 pieces fall into l ₀ and the other 2 pieces fall into l ₁ . In l ₀ , 8 got 4 votes, B only got 2 votes, so the last component class representing l ₀ is 8, if represented by a tree diagram, s represents the first component template, c represents the component class basis of each component. The characteristic value moves to the lower layer. When s =1, the leaf node l ₀ is 8, B, s = 2, the leaf node l ₀ is 8, and the leaf node l ₁ is B. In this way, the ticket is counted for l ₀ , we can it is apparent tree _leaf node of the decision tree l ₀ 4 8 had votes, B got 2 votes, so that the representative element category 8 l ₀ When the next input element unknown The fall l ₀ Shi 8 will get a vote, if the majority of the decision tree will vote for the component category 8, the identification result of this component will be recognized as the number 8.

Because the main algorithm for identification must rely on the template features to make it strong; the template data is obtained by the license plate component component cutting step. In the system design, the number of templates is small, and will not affect the system operation on the smart phone 10. Speed, so the template data needed to establish the feature table, the more the number, the better, can make the classifier more robust; on the other hand, the cutting components obtained from the system, if the same type (character or non-word Element), there are slight differences between different components, such as the difference in tilt angle, or with some noise, can also enhance the ability of the classifier; we encountered the under-sampling in the process of obtaining the sample components In other words, when the component data is cut for the collected experimental data, there is a serious shortage of certain types of character templates, such as the number 4. In order to facilitate the experiment, the premise of not affecting the balance of the feature table establishment process is premised. Manually re-copying a sample with too few components in a manual manner, so that the number of templates for each component category For different types of components, it is necessary to manually screen a more representative model from each side, such as slight left and right oblique, slight incomplete cutting, etc., if the character template is too different from the normal perceived situation. Large, classifier It may be necessary to have more information to distinguish between overly abnormal components, and it is foreseeable that the system will take a lot of time to train, which is why the slight abnormal components are selected; we also define several templates that represent non-characters and return them to In order to verify the performance of the license plate recognition system proposed by the present invention in the experimental stage, we also collected some new license plate characters to be added to the sample database, so that the system can trace the new license plate characters when it is detected. At this point, each category has 100 component template images, including numbers 0 to 9, letters A through Z (without the letter O), and dashes, noise, or component mis-segmentation (three are grouped into one). 36 types, 3600 pieces of component images.

In the process of establishing the feature table, the number of templates is regarded as a concept similar to the strength of solidarity. The component template information is piled up to the end by voting, so that the output of the final decision tree is a single component category, that is, The most similar components in the system, in Table 2, represent an index result in the feature table; the index refers to the characteristic value of the license plate component, which is in the form of a set of binary values, which can be regarded as a decimal when converted into decimal The index value is used. By using this conversion action, the system can complete the voting without spending too much searching time. If the number of bits used is too large, the memory usage will be too large, and even the parameter selection work cannot be started. The size of the feature table can be calculated by the following formula: FT _ SIZE = 2 ^bits

FT is the abbreviation list of features, feature values assuming the number of bits is 10 bits, the size compared with the profile table ^210, a total of 1024 addresses, when all the sample data calculations are completed, and this space is converted in Table 2 Expressed as follows:

After the feature space conversion is completed, the system uses the eigenvalues to perform table lookup voting. Assuming that an existing unknown component inputs the calculation feature, the feature value is 3 at the feature extraction position 1, the feature value is 1020 at the position 2, and the eigenvalue is obtained at the position 3. 1023, the table is corresponding to the component class M twice, and the component class N is once. If it is ignored that the subsequent different feature extraction positions are not counted, the system will determine that the component is the character M. The license plate component identification is voted by referring to the feature table. The person who gets the most votes is regarded as the category of the component; it is assumed that the table 1 is the feature table calculated by the three sets of eigenvalues randomly generated and substituted into the template data. Each component has a different index value corresponding to different feature tables. When the system calculates the feature value of the cutting component, it votes in the corresponding component category in the table according to the manner of looking up the table, and the category that gets the most votes is The identification result of the component.

肆.Experimental results

The license plate recognition system proposed by the invention obtains the feature bit, the number of feature tables, and the feature extraction position by means of parameter selection and training, instead of manually setting and randomly generating parameters, and transforming into feature space conversion steps. The last feature table to query. The following table is the feature table obtained by the system training, which is sorted according to the recognition rate from large to small, as shown in Table 3:

A single feature table (ie, a decision tree) can have a relatively high recognition rate through training, but has its limits. Therefore, each feature table needs to be assembled together to identify components and insufficient complementarities. The preliminary experiment of the system is carried out on the computer side. The test method is to give correct answers to each test data, compare it with the identification result, calculate the correct component quantity, and verify that the feature table is enough to be transplanted to the smart phone 10 for direct meter use. The following is the test data of the computer license plate recognition system. The horizontal axis is the component category, and the vertical axis is the number of times the component is identified incorrectly: from the upper right corner data, the overall recognition rate is 96.36%, for the non-character component category, and The number of errors in letters B and D is slightly more; in total, there are 67 license plate characters that have not been successfully cut, the total number of license plate characters is 1104, the cutting ratio of license plate characters is 99.93%, and the non-license characters of cut output are output. There are 215, only 6 of which are identified incorrectly; after we transplanted the system to the smart phone 10, the test screen shows Almost no identifiable components have been identified. Therefore, component identification errors existing on the computer side may be at some angles in the feature extraction position, and no better component resolution is obtained, but there is no smart device 10 end system. This concern, because the smart phone 10 end system for mobile viewing angle detection, the feature extraction position will be slightly different at each moment, so ignore the components that identify the error first; then we will carry out the smart phone 10 end system Actually, due to the detection screen of the smart phone 10 system, it is impossible to obtain a relatively fair graphical result like the computer end system. By outputting the identification result to the screen, the smart phone 10 is used to capture the image to display the identification result. Only some images are listed.

Wu. Conclusion

The invention provides a license plate recognition system for a smart phone. In the pre-image processing stage, the advantage area of the smart phone is used to set the region of interest, so that the image size of the system to be processed later is much simplified, and the built-in device is used, and the gyroscope is utilized. The sensor is used to correct the possible skew problem of the image, and also solves the hand-shake problem that the mobile device must face. In the connection between the pre-processing and the license plate position detection phase, a conditional iterative binarization is designed. The calculation process helps to improve the image binarization result, so that the license plate position and the character cutting rate in the license plate can obtain more accurate and better output results. On the other hand, the main algorithm for identifying the license plate components is applied from random. The concept of forest law, and according to the special design of the eigenvalue extraction method, the numerical value representing the feature is converted into a tree decision tree, that is, the feature table, thereby using the integrated classifier, through the experimental results, each feature table is With nearly 70% recognition capability, it can independently distinguish out more than 70% of the license plate components of the test data; The feature table is transplanted to the smart phone for final system measurement. According to the image displayed by the experimental results, it can be seen that there are good identification results in the front side, the side view, or the shadow cover.

The above is only a possible embodiment of the present invention, and is not intended to limit the scope of the patents of the present invention, and the equivalent implementations of other changes according to the contents, features and spirits of the following claims should be It is included in the patent of the present invention. The invention is specifically defined in the structural features of the request item, is not found in the same kind of articles, and has practicality and progress, has met the requirements of the invention patent, and has filed an application according to law, and invites the bureau to approve the patent according to law to maintain the present invention. The legal rights of the applicant.

30‧‧‧ License Plate Identification Module

Claims (9)

A gyro sensor license plate recognition system for a smart phone, which is provided with a license plate recognition module for the smart phone, and the license plate recognition module is used for capturing the image capture device of the smart phone. The image of the license plate sequentially performs image pre-processing, license plate position detection and capture processing, license plate area component segmentation processing, license plate area component feature extraction processing and license plate character recognition processing, and then outputs license plate character recognition result information; The license plate recognition module has a region of interest for limiting the image range, and the license plate recognition module performs the image pre-processing on the image of the region of interest; and captures the smart phone. An angle signal generated by the gyro sensor as an angle basis for correcting the image of the rotation of the region of interest; wherein, when the license plate recognition module detects and captures the license plate position, an error result is obtained When the conditional iterative binarization process is performed, the license plate recognition module calculates the frame selection license plate when the conditional iterative binarization process is performed. Shape ratio, and determine whether the ratio is greater than a preset proportional threshold; when the judgment result is yes, the license plate position is captured and output; when the judgment result is no, it is determined whether the preset number of iterations is completed; when the judgment result is If yes, the license plate position is captured and outputted; when the judgment result is no, the threshold value is re-corrected, and the image is re-binarized and returned to the step of detecting and capturing the license plate position. The gyroscope sensor license plate recognition system of the smart phone of claim 1, wherein the image pre-processing further comprises grayscale conversion processing, Gaussian smoothing processing, and first time on the image of the region of interest. Value processing. The gyro sensor license plate recognition system of the smart phone according to claim 1, wherein the license plate position detection and capture processing system uses a horizontal change amount marking method to detect the license plate position And capturing the license plate area image, the license plate area component segmentation processing uses the boundary mark cutting method to cut the license plate area image into a plurality of components, and the license plate area component feature extraction processing system uses the template feature comparison method to sequentially recite the plurality of components The component is configured to calculate a component feature value sufficient to describe the component feature, and the license plate recognition module establishes a database, wherein the database stores a plurality of feature tables, and each of the feature tables is written into the sequentially arranged component categories, each of which is Each component class of the feature table is correspondingly written with the component feature value, and the license plate character recognition process uses a random forest method to compare the component feature values of each component with each of the feature tables one by one, thereby obtaining The component class corresponding to each component, the component class containing characters and non-characters, the character containing numbers and English letters. The gyro sensor license plate recognition system of the smart phone of claim 3, wherein the license plate recognition module utilizes a Roll direction angle θ _roll converted by the gyro sensor to the region of interest The image center is a rotation axis, and the image of the region of interest is rotated by θ _roll degrees in the reverse direction to complete the rotation correction of the image. The license plate recognition module determines whether the θ _roll is greater than 70 degrees or less than -70 degrees. When the decision result is yes, there is no need to correct the image of the rotation of the region of interest. The gyro sensor license plate recognition system of the smart phone of claim 1, wherein the region of interest is selected by a user to touch a display screen of the smart phone. The area; or a rectangular frame selection area customized by the license plate recognition module, the rectangular frame selection area has a width of 320 pixels and a height of 240 pixels. A smart phone gyro sensor license plate recognition method, comprising: providing a smart phone, the smart phone includes an image capture device; and establishing a license plate recognition module on the smart phone; The license plate recognition module sequentially performs image pre-processing, license plate position detection and capture processing, license plate area component segmentation processing, license plate area component feature extraction processing on the image containing the license plate captured by the image capturing device. The license plate character recognition processing further outputs the license plate character recognition result information; wherein the license plate recognition module sets a region of interest for limiting the image reduction range in the image, the license plate recognition module is for the region of interest Performing the image pre-processing on the image; and capturing an angle signal generated by the gyro sensor of the smart phone as an angle basis for correcting the image of the rotation of the region of interest; wherein, when the license plate recognition mode When the error detection result is obtained by detecting and extracting the license plate position, conditional iterative binarization processing is performed. When the conditional iteration binarization processing is performed, the license plate recognition module calculates the rectangular ratio of the selected vehicle license plate. And determining whether the ratio is greater than a preset proportional threshold; when the judgment result is yes, the license plate position is captured and output; when the judgment result is no Then, it is judged whether the preset number of iterations is completed; when the judgment result is YES, the license plate position is extracted and output; when the judgment result is no, the threshold value is re-corrected, and the image is re-binarized and then returned. The steps to detect and retrieve the license plate position. The gyro sensor license plate recognition method for the smart phone according to claim 6, wherein the image pre-processing further comprises performing gray scale conversion processing, Gaussian smoothing processing, and first time on the image of the region of interest. Value processing. The gyro sensor license plate recognition method for the smart phone according to claim 6, wherein the license plate position detection and capture processing system uses a horizontal change amount marking method to detect a license plate position and capture a license plate area image, The license plate area component segmentation process uses the boundary mark cutting method to cut the license plate area image into a plurality of components, and the license plate area component feature extraction process uses the template feature comparison method to sequentially calculate the plurality of components to describe the The component feature value of the component feature, the license plate recognition module establishes a database, and the database stores the complex a number feature table, each of the feature tables is written into the component class arranged in sequence, and each component class of the feature table is correspondingly written with the component feature value, and the license plate character recognition process adopts a random forest method The component characteristic values of the respective elements are compared with each of the feature tables one by one, thereby obtaining the component class corresponding to each of the components. The method for identifying a gyro sensor license plate of a smart phone according to claim 6, wherein the license plate recognition module uses a Roll direction angle θ _roll converted by the gyro sensor to the region of interest The image center is a rotation axis, and the image of the region of interest is rotated by θ _roll degrees in the reverse direction to complete the rotation correction of the image. The license plate recognition module determines whether the θ _roll is greater than 70 degrees or less than -70 degrees. When the result of the determination is yes, the image of the region of interest is not required to be corrected, and the region of interest is selected by a user to touch the touch screen of the smart phone; Or a rectangular frame selection area customized by the license plate recognition module, the rectangular frame selection area has a width of 320 pixels and a height of 240 pixels.