CN112288372B

CN112288372B - Express bill identification method capable of simultaneously identifying one-dimensional bar code and three-segment code characters

Info

Publication number: CN112288372B
Application number: CN202011227735.XA
Authority: CN
Inventors: 赵楠楠; 邱林; 魏玉飞; 赵一帆; 张锋; 陈智博
Original assignee: Liaoning Heibeijian Technology Co ltd
Current assignee: Liaoning Heibeijian Technology Co ltd
Priority date: 2020-11-06
Filing date: 2020-11-06
Publication date: 2023-09-26
Anticipated expiration: 2040-11-06
Also published as: CN112288372A

Abstract

The invention provides an express bill identification method capable of simultaneously identifying one-dimensional bar codes and three-section code characters, which comprises the following steps: step one, acquiring an image of an express bill; the face list of each express item faces upwards and is arranged in the center of the trolley, and only one express item is placed on each trolley; the industrial camera is fixed at a certain height from the plane of the trolley; when the trolley carrying the express delivery piece moves to a designated position, an image is captured and transmitted to the industrial personal computer through the Ethernet; step two, coarse positioning of the express bill; step three, bar code rectangular positioning and inclination angle correction of the express bill; fourthly, positioning three sections of code characters under multiple scales; step five, three-segment code character recognition; according to the invention, the inclination angle of the express delivery face single picture with the complex background is corrected; positioning a one-dimensional bar code on the express delivery face sheet; positioning and identifying three-segment code characters (hereinafter referred to as three-segment codes) on the express delivery face list; all algorithms overall recognition speeds are less than 150ms.

Description

Express bill identification method capable of simultaneously identifying one-dimensional bar code and three-segment code characters

Technical Field

The invention relates to the technical field of sorting express, in particular to an express bill recognition method capable of recognizing one-dimensional bar codes and three-segment code characters simultaneously.

Background

With the development of electronic commerce, the consumption habits of people are greatly changed, and the demand for logistics service is gradually increased, so that the continuous improvement of the logistics automation level is also promoted. At present, the automatic sorting mode is only adopted in the sorting center of the domestic logistics industry, and the manual sorting mode is generally adopted in the small sorting center to manually sort logistics products one by one, so that the efficiency is low, the reliability is low, and the current increasing demand of the logistics flow cannot be met. In order to reduce the defects of low sorting efficiency, high employment cost, damage to express delivery and the like caused by traditional manual sorting express delivery, the demand on a small automatic express delivery sorting system is more and more urgent.

Typically, the basis for sorting the courier is to automatically or manually identify information on the courier sheet. Here, the express delivery bill information includes three codewords (generally consisting of printed digits and uppercase english letters) and a one-dimensional bar code, which each indicate codes of the place of sending and the place of receiving the express delivery, as shown in fig. 1. Generally, automatic sorting of domestic large sorting centers all relies on image recognition technology to recognize one-dimensional bar codes, however, three-segment code characters are also important information to be recognized, but are hardly used at present. In addition, the small sorting center is limited by the field area and the cost, and currently, three code characters on an express bill are mostly identified manually. Therefore, research on how to recognize one-dimensional bar codes and three-segment code characters simultaneously is an important development direction for improving automatic sorting efficiency of express delivery, both for a large sorting center and a small sorting station.

Disclosure of Invention

In order to solve the technical problems in the background technology, the invention provides an express bill identification method capable of identifying one-dimensional bar codes and three-section code characters simultaneously, which corrects the inclination angle of an express bill picture with a complex background; positioning a one-dimensional bar code on the express delivery face sheet; positioning and identifying three-segment code characters (hereinafter referred to as three-segment codes) on the express delivery face list; all algorithms overall recognition speeds are less than 150ms.

In order to achieve the above purpose, the invention is realized by adopting the following technical scheme:

a express bill identification method capable of identifying one-dimensional bar codes and three-segment code characters simultaneously comprises the following steps:

step one, acquiring an image of an express bill; the face list of each express item faces upwards and is arranged in the center of the trolley, and only one express item is placed on each trolley; the industrial camera is fixed at a height of 1.3-1.5 meters from the plane of the trolley; when the trolley carrying the express delivery piece moves to a designated position, an image is captured and transmitted to the industrial personal computer through the Ethernet;

step two, coarse positioning of the express bill;

firstly, initially positioning an express bill on an acquired image; finding out the maximum rectangle capable of sleeving the contour through binarization, morphological operation and contour calculation of the image; performing Sobel edge detection on an image contained in the maximum rectangle, binarizing, calculating the number of edge points, and if the number of the edge points is greater than 3000, considering that the maximum rectangle is complex enough, wherein the area is the rough position of the express bill;

then, correcting the inclination angle of the maximum rectangle by utilizing anti-radiation conversion to obtain a candidate image to be identified; the area to be identified is only the result of rough positioning, and further inclination angle correction and waybill information positioning are also required;

step three, bar code rectangular positioning and inclination angle correction of the express bill;

the processing process comprises Sobel edge detection, morphological operation, binarization and region growing algorithm;

1) Firstly, performing edge detection on an image in the horizontal direction and the vertical direction by using a Sobel operator, and adding two results to better highlight edge information in the image; combining the characteristics of bar codes that the bar codes form blocks and are densely distributed, firstly using morphological closing operation with larger structural elements (for example, 5 pixels) to change bar code parts on the image into a communicated area, so that the characteristics of the bar codes are more prominent, and then using morphological closing operation with smaller structural elements (for example, 3 pixels) to remove the interference of some simple single lines on the image;

2) The image is subjected to open-close operation, so that a plurality of tiny holes are formed, and the subsequent operation is interfered, so that the holes are filled;

3) Calculating the minimum outsourcing rectangle of each white area in the graph after filling the holes, and obtaining information of each outsourcing rectangle, wherein the information comprises four-vertex coordinates, inclination angles and the number of white pixels in the rectangular area; calculating the distribution of the rectangular inclination angles at 0-90 degrees according to the calculated rectangular inclination angles, and correcting the preliminary inclination angles by using the concentrated distribution angles;

4) In the image corrected by the initial inclination angle, the bar code is approximately positioned in the direction of 0 degrees or in the direction of 90 degrees, and the error is within the range of plus or minus 10 degrees; screening bar code candidate rectangles in the directions of 0 degrees and 90 degrees respectively, determining the final position and the final angle of the bar code, and rotating for the last time to obtain a bar code positioning image after accurate segmentation in the forward direction (0 degrees) or the reverse direction (180 degrees), and sending the segmented bar code image to a Zbar recognition function for recognition;

step four, three-segment codeword positioning under multi-scale

1) Firstly, the information distribution on the express delivery bill is analyzed to know that the three-section code character is possibly arranged at two positions above or below the bar code, so that the positions of the three-section code character are searched in the upward direction and the downward direction respectively by taking the position of the bar code as a reference, and the upper image and the lower image are intercepted respectively for self-adaptive binarization processing;

2) Then, calculating the minimum outsourcing rectangle, namely the outline of all white parts in the two images, solving the area of each outline, screening the outlines, and removing the outlines with the rectangular areas smaller than 10 pixels or larger than 1200 pixels;

3) After the rectangular outline in the step 2) is obtained, firstly, calculating in the horizontal direction, namely, carrying out probability distribution statistics on the heights of all the rectangles, wherein the heights of the rectangles correspond to the heights of three sections of code numbers; because the three-section code digital heights on the express bill image and other text heights such as addresses are concentrated, the concentrated heights are used as the searching scale; for example, on a certain image, the three-segment code number height is 20 pixels, the address information text height is 15 pixels, and after probability statistics is performed on all rectangle heights, two values of 20 and 15, namely two different scales, are found; then searching for a character rectangle on each scale respectively; each digital rectangle of the three-section code is rectangular and is distributed in one row or two rows in a concentrated manner, and according to the characteristic, the three-section code can be accurately positioned;

the relative angle between the bar code and the three-section code on some express bill is 90 degrees, in order to adapt to the situation, the algorithm is performed again in the vertical direction, namely probability distribution statistics is performed on the widths of all rectangles, wherein the widths of the rectangles correspond to the heights of the three-section code numbers;

4) Dividing the character according to the final position of the three-section code, and sending the character to a CNN neural network for recognition;

step five, three-segment codeword identification

The three-segment codeword identification adopts a Tiny-CNN deep learning framework, and comprises an input layer, a convolution layer, a pooling layer and an output layer; the training set is more than 2 ten thousand actually collected character gray-scale images which are segmented from the express delivery face sheets;

the number of input layer nodes is equal to the width (18) of the input image plus the height (18), together 324; the number of nodes of the convolution layer 1 is 6 x 16; the node number of the pooling layer 1 is 6 x 8; the number of nodes of the convolution layer 2 is 12 x 6; the node number of the pooling layer 2 is 12 x 3; the number of nodes of the full connection layer is 120, and the number of nodes of the output layer is the sum of 10 Arabic numerals, 26 capital English letters and one connector "-", and the total number of the nodes of the output layer is 37.

In the fifth step, the activation function adopted by each node is a Tanh function, the epoch value is 100 during training, and other parameters are default parameters.

In the fifth step, when the input images are fed into the neural network for learning, the input images are unified to the same size, and the size is 18 x 18, because the size of each character image has a certain difference due to different scales when the characters are segmented; and (5) taking the character value with the highest confidence as a result during recognition.

Further, in the fifth step, when the neural net learning is performed, there may be a kanji character or other non-character image in the divided character, so that it is not outputted as a recognition result at the time of recognition.

Further, in the fifth step, when the neural net is fed for learning, since the features of the numbers "1", "0" and the capital letters "I" and "O" are very close, it is necessary to consider "1" and "I" as one type, and "0" and "O" as one type.

Compared with the prior art, the invention has the beneficial effects that:

1) According to the express bill identification method capable of identifying the one-dimensional bar code and the three-section code characters at the same time, the inclination angle of the express bill picture with the complex background is corrected; positioning a one-dimensional bar code on the express delivery face sheet; positioning and identifying three-segment code characters (hereinafter referred to as three-segment codes) on the express delivery face list; all algorithms overall recognition speeds are less than 150ms.

2) In order to adapt to various complex backgrounds of express delivery face single images, the invention firstly provides a method for calculating the angle distribution of rectangular angles to calculate the angle of the express delivery face single, and other related technologies mostly adopt a Hough transformation method. The proposed method is faster in calculation speed than the Hough transform method in terms of algorithm basic principle.

3) The invention provides a three-section code positioning algorithm based on multiple scales for the first time, and can accurately position and identify three-section code numbers with any background images, any inclination angles and any sizes. Other techniques either use manual calibration of the three-code positions or require the image to be identified itself to be placed in a positive orientation.

Drawings

FIG. 1 is a schematic diagram of a bar code and a three-segment code;

FIG. 2-a is an original view of image one and coarse positioning results;

FIG. 2-b is a coarse localization contour binary map of image one;

FIG. 2-c is an original view of image two and coarse positioning results;

FIG. 2-d is a coarse positioning contour binary image of image two;

FIG. 3-a is a region to be identified taken on image one;

FIG. 3-b is a region to be identified taken on image two;

FIG. 4-a is a gray scale map to be identified after coarse positioning;

FIG. 4-b is a diagram with tiny holes;

FIG. 4-c is a graph of etch dryness after filling holes;

FIG. 5 is a waybill image after tilt angle correction;

FIG. 6 is a bar code positioning result diagram;

FIG. 7 is a binarization map over a bar code;

FIG. 8 is a binarization map under a bar code;

FIG. 9 is a rectangular screen above a bar code;

FIG. 10 is a rectangular screen under a bar code;

FIG. 11-a is a first set of statistics in the horizontal direction;

FIG. 11-b is a second set of statistics in the horizontal direction;

FIG. 11-c is a first set of statistics in the vertical direction;

FIG. 11-d is a second set of statistics in the vertical direction;

FIG. 12 is a three-segment codeword final positioning map;

FIG. 13 is a graph of a CNN neural network model;

FIG. 14 is a CNN neural network training set image;

fig. 15 is a result display diagram.

Description of the embodiments

The following detailed description of the embodiments of the invention is provided with reference to the accompanying drawings.

A method for identifying express bill capable of identifying one-dimensional bar code and three-segment code characters simultaneously is realized by the following steps:

step one, image acquisition of express bill

The face list of each express item faces upwards and is arranged in the center of the trolley, and only one express item is placed on each trolley. The industrial camera is fixed at a height of 1.3-1.5m from the plane of the trolley. When the trolley carrying the express delivery piece moves to a designated position, an image is captured and transmitted to the industrial personal computer through the Ethernet.

Step two, coarse positioning of express bill

1) Firstly, carrying out preliminary positioning on the express bill on the acquired image. The maximum rectangle capable of tightly fitting the outline is found through binarization and morphological operation on the image. And performing Sobel edge detection on the image contained in the maximum rectangle, binarizing, calculating the number of edge points, and if the number of the edge points is greater than 3000, considering that the maximum rectangle is sufficiently complex, wherein the region is the rough position of the express bill. As shown in fig. 2-a, 2-b, 2-c, 2-d. Wherein the coarse positioning effect of image one in fig. 2-a and fig. 2-b. FIG. 2-a is an original gray scale image, which is filtered by a preprocessing algorithm to obtain a white area of FIG. 2-b, and then calculating a minimum rectangle surrounding the white area to obtain a white rectangular frame on FIG. 2-a. Fig. 2-c and fig. 2-d are rough positioning effects of image two, unlike image one, the extracted image may be a parcel image with waybill information due to ambient brightness or self-material problems of the express parcel, but also consider that the rough location of the express waybill is found.

2) Then, the candidate image to be identified is obtained by affine transformation, as shown in fig. 3-a and 3-b. Fig. 3-a and 3-b are affine transformation results for image one and image two, respectively. It is noted here that the areas to be identified shown in fig. 3-a, 3-b are only the result of coarse positioning, and further tilt angle correction and waybill information positioning are required.

Step three, bar code rectangular positioning and inclination angle correcting of express bill

In the express waybill image under the complex background, people can rapidly judge the position of the bar code by eyes, and most of reasons are that the image can be primarily processed by utilizing the characteristic of the inter-black-white specificity of the bar code. The processing process comprises Sobel edge detection, morphological operation, binarization, region growth and other algorithms. The Sobel operator is used for detecting the edges of the image in the horizontal and vertical directions, and the two results are added to better highlight the edge information in the image. Combining the characteristics of bar codes which are formed into blocks and densely distributed, firstly using morphological closing operation with larger structural elements (for example, 5 pixels) to change a bar code part on an image into a communication area, so that the characteristics of the bar codes are more prominent, and then using morphological closing operation with smaller structural elements (for example, 3 pixels) to remove the interference of some simple single lines on the image; at this time, many tiny holes are formed in the image after the opening and closing operation, and a certain interference is generated to the subsequent calculation, so that the holes need to be filled. For example, an image obtained by performing rough positioning on one of the original images through the express delivery surface in the second step is shown in fig. 4-a. The bar code rectangle is processed to be more prominent to obtain the figure 4-b, and the figure 4-c is obtained after the small holes are filled in the figure 4-b.

And then calculating the minimum outsourcing rectangles according to each white area in the graph 4-c, and obtaining information of each outsourcing rectangle, wherein the information comprises four-vertex coordinates, inclination angles and the number of white pixels in the rectangular area. And calculating the distribution of the rectangles at 0-90 degrees according to the calculated inclination angles, correcting the inclined waybill information image by using the concentrated distribution angles, wherein the waybill information image after the inclination angle correction is shown in fig. 5.

In the image corrected by the initial inclination angle, the bar code is approximately positioned in the direction of 0 degrees or in the direction of 90 degrees, and the error is within the range of plus or minus 10 degrees; screening bar code candidate rectangles in the directions of 0 degrees and 90 degrees respectively, determining the final position and the final angle of the bar code, and rotating for the last time to obtain a bar code positioning image after accurate segmentation in the forward direction (0 degrees) or the reverse direction (180 degrees), and sending the segmented bar code image to a Zbar recognition function for recognition, wherein the final position of the bar code is shown in fig. 6.

Step four, three-segment codeword positioning under multi-scale

The three-segment codeword on the express waybill consists of 0-9 ten digits and 26 English letters. But can be seen from the photographed express waybill pictures, the characters in the images under different sizes are various, so that a large amount of interference is brought, and great difficulty is brought to recognition. We therefore employ a multi-scale approach to locating three codewords.

Firstly, by analyzing the distribution of the information on the express bill, we can know that the three-section code character may be at two positions above or below the bar code, so we need to search the positions of the three-section code character in the upward and downward directions respectively based on the position of the bar code, and intercept the upper and lower images respectively for self-adaptive binarization processing, the effect diagram is shown in fig. 7 and 8.

Then, for all white parts in the two images, calculating the minimum outsourcing rectangle, namely the outline, calculating the area of each outline, screening the outlines, and removing the outlines with the rectangular areas smaller than 10 pixels or larger than 1200 pixels, wherein the outline screening results are shown in fig. 9 and 10.

Secondly, calculating the obtained minimum tight rectangles in the horizontal direction, namely carrying out probability distribution statistics on the heights of all rectangles, wherein the heights of the rectangles correspond to the heights of three sections of code numbers; because the three-section code number height and other text heights such as addresses on the express bill image are different, the height values are concentrated, and the concentrated heights are used as the searching scale. For example, on a certain image, the three-segment code number height is 20 pixels, the address information text height is 15 pixels, and after probability statistics is performed on all rectangle heights, two values of 20 and 15, namely two different scales, are found. And then find the character rectangle on each scale separately. Each digital rectangle of the three-section code is rectangular and is distributed in one row or two rows in a concentrated mode, and according to the characteristic, the three-section code can be accurately positioned. In addition, the relative angle between the bar code and the three-section code on some express delivery face sheets is 90 degrees, in order to adapt to the situation, the algorithm performs probability statistics on the width of each rectangle again in the vertical direction, and three-section code digital positioning is performed after multi-scale data are obtained, wherein the width of each rectangle corresponds to the height of the three-section code digital. For example, on one of the images, two scales are found in the horizontal direction, as shown in fig. 11-a and 11-b, and two scales are found in the vertical direction, as shown in fig. 11-c and 11-d, and it is apparent that the three-segment code on this figure is in the horizontal direction.

And finally, determining the final position of the three-section code according to the calculation result in the horizontal or vertical direction, dividing the character, and sending the character to a CNN neural network for recognition. The three-segment code positioning result diagram is shown in fig. 12.

Step five, three-segment codeword identification

The three-segment codeword identification adopts a Tiny-CNN deep learning framework, and comprises an input layer, a convolution layer, a pooling layer and an output layer; the training set is a plurality of 2 ten thousand actually collected character gray-scale images which are segmented from the express delivery face sheets. The network model is shown in fig. 13, and the training set image is shown in fig. 14. The number of input layer nodes is equal to the width (18) of the input image plus the height (18), together 324; the number of nodes of the convolution layer 1 is 6 x 16; the node number of the pooling layer 1 is 6 x 8; the number of nodes of the convolution layer 2 is 12 x 6; the node number of the pooling layer 2 is 12 x 3; the number of nodes of the full connection layer is 120, and the number of nodes of the output layer is the sum of 10 Arabic numerals, 26 capital English letters and one connector "-", and the total number of the nodes of the output layer is 37. The activation function adopted by each node is a Tanh function, the value of epoch is 100 during training, and other parameters are default parameters.

Three points are notable when learning into the neural network. Firstly, when dividing characters, the sizes of images of each character have certain difference due to different scales, so that the input images are required to be unified to the same size, the size is 18 x 18, and the character value with the maximum confidence is taken as a result during recognition; second, there may be kanji and other non-character images in the divided characters, so that they are not outputted as recognition results at the time of recognition; third, since the features of the numbers "1", "0" and capital "I", "O" are very close, it is desirable to treat "1" and "I" as one class, and "0" and "O" as one class.

Fig. 15 is a result display diagram.

The above examples are implemented on the premise of the technical scheme of the present invention, and detailed implementation manners and specific operation processes are given, but the protection scope of the present invention is not limited to the above examples. The methods used in the above examples are conventional methods unless otherwise specified.

Claims

1. A express bill identification method capable of identifying one-dimensional bar codes and three-segment code characters simultaneously is characterized by comprising the following steps:

step two, coarse positioning of the express bill;

1) Firstly, performing edge detection on an image in the horizontal direction and the vertical direction by using a Sobel operator, and adding two results to better highlight edge information in the image; combining the characteristics of bar codes that the bar codes form blocks and are densely distributed, firstly using morphological closing operation with larger structural elements to change bar code parts on the images into a communication area, so that the characteristics of the bar codes are more prominent, and then using morphological closing operation with smaller structural elements to remove some simple single-line interference on the images;

4) In the image corrected by the initial inclination angle, the bar code is positioned in the direction of 0 degree or the direction of 90 degrees, and the error is within the range of plus or minus 10 degrees; screening bar code candidate rectangles in the directions of 0 degrees and 90 degrees respectively, determining the final position and the final angle of the bar code, and rotating for the last time to obtain a forward or reverse precisely segmented bar code positioning image, and sending segmented bar code pictures to a Zbar recognition function for recognition;

step four, three-segment codeword positioning under multi-scale

3) After the rectangular outline in the step 2) is obtained, firstly, calculating in the horizontal direction, namely, carrying out probability distribution statistics on the heights of all the rectangles, wherein the heights of the rectangles correspond to the heights of three sections of code numbers; the three-section code digital height on the express bill image is concentrated, other text heights of the address are concentrated, the concentrated heights are used as searching scales, and the three-section code is accurately positioned according to the characteristic;

step five, three-segment codeword identification

2. The method for identifying the express bill capable of simultaneously identifying the one-dimensional bar code and the three-segment code characters according to claim 1, wherein in the fifth step, the activation function adopted by each node is a Tanh function, the epoch value is 100 during training, and the other parameters are default parameters.

3. The method for identifying the express bill capable of identifying one-dimensional bar codes and three-segment code characters simultaneously according to claim 1, wherein in the fifth step, when the user sends the express bill into a neural network for learning, when the user segments characters, the size of each character image has a certain difference due to different scales, so that the input pictures are required to be unified to the same size, and the size is 18 x 18; and (5) taking the character value with the highest confidence as a result during recognition.

4. The method for identifying the express bill capable of simultaneously identifying the one-dimensional bar code and the three-segment code characters according to claim 1, wherein in the fifth step, when the user sends the express bill to the neural network for learning, chinese characters and other non-character images can exist in the segmented characters, so that the character images are not output as an identification result during identification.

5. The method for identifying the express bill capable of simultaneously identifying the one-dimensional bar code and the three-segment code characters according to claim 1, wherein in the fifth step, when the user sends the one-dimensional bar code and the three-segment code characters into a neural network for learning, the characters of the numbers "1", "0" and the capital letters "I" and "O" are very close, so that the "1" and the "I" are required to be regarded as one type, and the "0" and the "O" are required to be regarded as one type.