CN109977910B - Rapid bill positioning method and system based on color line segments - Google Patents
Rapid bill positioning method and system based on color line segments Download PDFInfo
- Publication number
- CN109977910B CN109977910B CN201910273966.5A CN201910273966A CN109977910B CN 109977910 B CN109977910 B CN 109977910B CN 201910273966 A CN201910273966 A CN 201910273966A CN 109977910 B CN109977910 B CN 109977910B
- Authority
- CN
- China
- Prior art keywords
- line segment
- line segments
- line
- gradient
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4023—Decimation- or insertion-based scaling, e.g. pixel or line decimation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/412—Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
Abstract
The invention relates to a method and a system for rapidly positioning a bill frame based on color line segment detection, wherein the method comprises the following steps: inputting a bill image, scaling the image according to a proportion, detecting color line segments, merging the line segments, screening the line segments, constructing a maximum quadrangle and outputting a vertex. The invention has the beneficial effects that: the method can identify the colorful bill, is not influenced by gray level conversion, accurately and quickly positions the bill vertex, and cuts out an effective bill area.
Description
Technical Field
The invention relates to a method and a system for rapidly positioning bills based on color line segments, belonging to the field of image recognition.
Background
Characters and numbers of the bill are converted into data of an electronic version through an image processing technical means, and data information is rapidly acquired through a computer program. In the traditional financial flow, a large number of bills need to be input manually, and the efficiency and accuracy of data acquisition are greatly improved by using an image recognition technology. Along with the development of image processing and OCR character recognition technology, the work intensity of financial bill reimbursement and reimbursement is greatly reduced through electronization of a financial flow.
In scenes such as automatic bill recognition and business card recognition, images obtained by photographing through a mobile terminal are not accurate bills, and even images obtained are distorted due to different photographing angles. In order to facilitate identification and processing, an automatic method is needed to cut out bills, correct distorted images and perform subsequent processing. The line segment detection algorithm in the prior art has several defects in bill detection: 1. the method is not suitable for a color image, and the color image needs to be converted into a gray image and then is subjected to line segment detection; 2. the bill content is provided with a large number of lines, the color gradient between the lines and the background is high, the step length is short, the bill edge is usually low in gradient and long in step length, and the bill edge is not easy to detect; 3. the detection speed is slow.
Disclosure of Invention
In order to solve the technical problem, the invention provides a method and a system for rapidly positioning a bill frame based on color line segment detection, which can rapidly position a color bill.
The technical scheme of the invention is as follows:
a method for rapidly positioning a bill frame based on color line segment detection comprises the following steps:
s1: a frame of images containing a ticket is acquired.
S2: and carrying out reduction processing on the one-frame image containing the bill according to the proportion by a linear interpolation method to obtain a reduced image, wherein the proportion value lv used by the linear interpolation method is 300/min (w, h), w and h are respectively the width and the length of the one-frame image containing the bill, and the width and the height of the reduced image are respectively w2＝w×lv，h2＝h×lv。
S3: identifying all line segments in the reduced image, which are represented by L ═ x1, y1, x2, y2, by a color LSD line segment detection algorithm, where (x1, y1) and (x2, y2) are two endpoints of the line segment L, and grouping all line segments into a set of line segments L _ set; the color LSD line segment detection algorithm performs Gaussian down sampling on a bill image once and reduces the image, performs gradient detection on RGB three channels of the image, determines the maximum gradient value of the three channels as a reference gradient, and outputs a detected line segment through region growing operation and rectangle estimation operation.
S4: and sorting all the line segments in the line segment set L _ set from large to small according to the length, merging the line segments by a line segment merging algorithm, and merging the line segments which are approximately on one straight line.
S5: and (4) dividing the line segments after the line segment combination in the step (S4) into 4 groups according to the coordinate difference of end points at two ends of the line segments, wherein the 4 groups correspond to the upper, lower, left and right sides of the quadrangle respectively.
S6: and respectively filtering the 4 groups of line segments to remove unqualified line segments including deviated and too short line segments.
S7: and sequentially taking a line segment from each group in 4 groups of filtered line segments according to the numbering sequence in the group by an exhaustion method, so that each line segment of each group is combined once, calculating the sum of the distances between the adjacent end points of two adjacent line segments in each combination, and outputting the four line segments when the sum of the distances is minimum.
S8: and (4) forming a quadrangle by the four line segments output in the step S7, calculating the intersection point of straight lines of two adjacent line segments to obtain four intersection points, and outputting the 4 intersection points as 4 vertexes of the bill.
In step S3, the color LSD line segment detection algorithm includes:
and performing Gaussian down-sampling on the image, and reducing the resolution of the image.
And respectively carrying out gradient calculation on each pixel point of the image with reduced resolution, wherein R, G, B three channels of the pixel respectively apply a gradient algorithm to carry out gradient calculation on the corresponding channel, and the maximum value of the three gradients is taken as the gradient value of the current pixel.
Screening each pixel according to the output gradient value, and removing the pixels smaller than a gradient threshold value, wherein the gradient threshold value is
And performing region growing operation on the screened pixels according to the gradient values of the pixels, and combining the pixels in the same gradient direction as the current pixel to obtain a corresponding planning region.
And performing rectangular approximate calculation on all the planning areas to obtain corresponding straight line segments.
And outputting the coordinates of the end points of all the straight line segments.
In step S4, the segment merging algorithm is:
firstly, judging included angles of line segments AC, AD, BC and BD and a horizontal line to be less than 1 degree, 2, 3 and 4 at a vertex A, B of a line segment L (i) and a vertex C, D of an L (j), and judging whether the distance between two vertexes closest to each other is less than 10 if the included angles are less than 1 degree, if so, successfully entering the next step, otherwise, failing to judge;
for the vertex A, B of the line segment L (i) and the vertex C, D of the line segment L (j), two vertices with the longest distance are taken as new line segments, and the new line segments are added into the set, and the original two line segments are deleted.
In step S5, the grouping method is that when dx is counted>dy and y1<y0 and y2<y0, then the line segment L is classified as group 1; when dx>dy and y1>y0 and y2>y0, grouping the line segments L into a group 2; when dx<dy and x1<x0 and x2<x0, grouping the line segment L into a group 3; when dx<dy and x1>x0 and x2>x0, grouping the line segment L into a group 4; otherwise, rejecting the line segments; wherein dx is | x2-x1|, dy is | y2-y1|, and x0 is w2/2,y0＝h2/2。
In the step S6, the filtering method is that, for any two line segments L1 and L2 in the current group, when ≈ (L1, L2) is less than 3 degrees and dst1< dst2, the deviated line segment L1 is deleted; when the angle (L1, L2) is less than 3 degrees and dst1 is greater than dst2, deleting the segment L2, wherein the angle (L1, L2) represents the included angle between L1 and L2, and the dst1 and the dst2 represent the distance between the straight line where L1 and L2 are located and a point (x0, y0) respectively;
and deleting too short line segments with the length less than len/3 in all current groups, wherein len is the value with the longest line segment length in the current groups.
The second technical scheme of the invention:
a rapid positioning system for bill borders based on color line segment detection comprises a memory and a processor, wherein the memory stores instructions, and the instructions are suitable for being loaded by the processor and executing the following steps:
a frame of images containing a ticket is acquired.
And carrying out reduction processing on the one-frame image containing the bill according to the proportion by a linear interpolation method to obtain a reduced image, wherein the proportion value lv used by the linear interpolation method is 300/min (w, h), w and h are respectively the width and the length of the one-frame image containing the bill, and the width and the height of the reduced image are respectively w2＝w×lv，h2＝h×lv。
Identifying all line segments in the reduced image, which are represented by L ═ x1, y1, x2, y2, by a color LSD line segment detection algorithm, where (x1, y1) and (x2, y2) are two endpoints of the line segment L, and grouping all line segments into a set of line segments L _ set; the color LSD line segment detection algorithm performs Gaussian down sampling on a bill image once and reduces the image, performs gradient detection on RGB three channels of the image, determines the maximum gradient value of the three channels as a reference gradient, and outputs a detected line segment through region growing operation and rectangle estimation operation.
And sorting all the line segments in the line segment set L _ set from large to small according to the length, merging the line segments by a line segment merging algorithm, and merging the line segments which are approximately on one straight line.
And (3) grouping the line segments after line segment combination, and dividing the line segments into 4 groups according to the coordinate difference of end points at two ends of the line segments, wherein the 4 groups correspond to the upper, lower, left and right sides of a quadrangle respectively.
And respectively filtering the 4 groups of line segments to remove unqualified line segments including deviated and too short line segments.
And sequentially taking a line segment from each group in 4 groups of filtered line segments according to the numbering sequence in the group by an exhaustion method, so that each line segment of each group is combined once, calculating the sum of the distances between the adjacent end points of two adjacent line segments in each combination, and outputting the four line segments when the sum of the distances is minimum.
And (4) forming the four output line segments into a quadrangle, calculating the intersection point of two adjacent line segment straight lines to obtain four intersection points, and outputting the 4 intersection points as 4 vertexes of the bill.
The line segment merging algorithm is as follows:
judging included angles of line segments AC, AD, BC and BD with a horizontal line of less than 1 degree, less than 2, less than 3 and less than 4 for vertexes A, B of the line segment L (i) and C, D of the line segment L (j), judging whether the distance between two vertexes closest to each other is less than 10 if the included angles are less than 1 degree, if yes, successfully entering the next step, and otherwise, failing to judge;
for the vertex A, B of the line segment L (i) and the vertex C, D of the line segment L (j), two vertices with the longest distance are taken as new line segments, and the new line segments are added into the set, and the original two line segments are deleted.
The grouping method is that when dx>dy and y1<y0 and y2<y0, then the line segment L is classified as group 1; when dx>dy and y1>y0 and y2>y0, grouping the line segments L into a group 2; when dx<dy and x1<x0 and x2<x0, grouping the line segment L into a group 3; when dx<dy and x1>x0 and x2>x0, grouping the line segment L into a group 4; otherwise, rejecting the line segments; wherein dx is | x2-x1|, dy is | y2-y1|, and x0 is w2/2,y0＝h2/2。
The color LSD line segment detection algorithm comprises the following steps:
and performing Gaussian down-sampling on the image, and reducing the resolution of the image.
And respectively carrying out gradient calculation on each pixel point of the image with reduced resolution, wherein R, G, B three channels of the pixel respectively apply a gradient algorithm to carry out gradient calculation on the corresponding channel, and the maximum value of the three gradients is taken as the gradient value of the current pixel.
Screening each pixel according to the output gradient value, and removing the pixels smaller than a gradient threshold value, wherein the gradient threshold value is
And performing region growing operation on the screened pixels according to the gradient values of the pixels, and combining the pixels in the same gradient direction as the current pixel to obtain a corresponding planning region.
And performing rectangular approximate calculation on all the planning areas to obtain corresponding straight line segments.
And outputting the coordinates of the end points of all the straight line segments.
The filtering processing method is that for any two line segments L1 and L2 in the current group, when the angle (L1 and L2) is less than 3 degrees and the dst1 is less than the dst2, the deviation L1 is deleted; when the angle (L1, L2) is less than 3 degrees and dst1 is greater than dst2, deleting the deviation L2, wherein the angle (L1, L2) represents the included angle of L1 and L2, and the dst1 and the dst2 represent the distance between the straight line where L1 and L2 are located and a point (x0, y0) respectively;
and deleting too short line segments with the length less than len/3 in all current groups, wherein len is the value with the longest line segment length in the current groups.
The invention has the beneficial effects that:
1. through reducing the image, can reduce the step length, the gradient is little etc. and leads to the problem that bill border can not detect, through later stage connection and screening, can effectual find out real bill border and summit.
2. The line segment is detected by using an improved color LSD line segment detection algorithm, the detection of the color line segment can be adapted, the gray level image conversion is not needed, and the color bill line segment can be quickly identified.
3. By filtering and combining line segments, a detection result of sub-pixel level precision can be obtained in a short time, and lines can be detected under the condition of low bill and background contrast.
Drawings
FIG. 1 is a flow chart of a method for rapidly positioning a bill frame based on color line segment detection according to the present invention;
FIG. 2 is a ticket artwork of one embodiment of the present invention;
FIG. 3 is a diagram illustrating results of finding color line segments by color line segment detection according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating four sets of line segment results grouped by line segment merger according to one embodiment of the present invention;
FIG. 5 is a schematic diagram of a document frame positioning line segment after line segment filtering according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a quadrilateral outline of a bill after being filtered by line segments according to an embodiment of the invention;
FIG. 7 is a flow chart of a color LSD line segment detection algorithm used in the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and the specific embodiments.
Example one
As shown in fig. 1, the method for rapidly positioning the bill border based on color line segment detection comprises the following steps:
s1: a frame of image containing a ticket is acquired, as shown in fig. 2, and the original image is a color image.
S2: and carrying out reduction processing on the one frame of image containing the bill according to the proportion by a linear interpolation method to obtain a reduced image. The linear interpolation method uses a scale value lv of 300/min (w, h), where w, h are the width and length of the frame including the image of the bill, respectively, and the width and height of the reduced image are w, respectively2＝w×lv，h2＝h×lv。
S3: identifying all line segments in the reduced image, which are represented by L ═ x1, y1, x2, y2, by a color LSD line segment detection algorithm, where (x1, y1) and (x2, y2) are two endpoints of the line segment L, and grouping all line segments into a set of line segments L _ set; the color LSD line segment detection algorithm performs Gaussian down sampling on a bill image once and reduces the image, performs gradient detection on RGB three channels of the image, determines the maximum gradient value of the three channels as a reference gradient, and outputs a detected line segment through region growing operation and rectangle estimation operation, as shown in FIG. 3.
S4: and sorting all the line segments in the line segment set L _ set from large to small according to the length, merging the line segments by a line segment merging algorithm, and merging the line segments which are approximately on one straight line.
S5: and (4) dividing the line segments after the line segment combination in the step (S4) into 4 groups according to the coordinate difference of end points at two ends of the line segments, wherein the 4 groups correspond to the upper, lower, left and right sides of the quadrangle respectively. As shown in fig. 4, they are respectively a schematic diagram of upper, lower, left, right line segments.
S6: and respectively filtering the 4 groups of line segments to remove unqualified line segments including deviated and too short line segments.
S7: sequentially taking a line segment from each group in sequence according to the numbering sequence in the group from 4 groups of line segments after filtering treatment by an exhaustion method, so that each line segment of each group is combined once, calculating the sum of the distances between the adjacent end points of two adjacent line segments in each combination, and outputting the four line segments when the sum of the distances is minimum, as shown in fig. 5.
The frame of the bill is a quadrangle with the largest composition range, theoretically, the end points of adjacent line segments are directly overlapped, the distance is 0, and the condition that the distance is the minimum is considered because the missing of partial line segments is not recognized probably due to the influence of factors such as fuzzy missing in image processing and the like.
S8: and (4) forming a quadrangle by the four line segments output in the step S7, solving intersection points of two adjacent line segments to obtain four intersection points, and outputting the 4 intersection points to be regarded as 4 vertexes of the bill. As shown in fig. 6, four line segments are used as reference line segments of four sides of the quadrangle, each line segment is extended until the line segment intersects with another line segment, a complete quadrangle is obtained, and four intersection point coordinates are output.
As shown in fig. 7, in step S3, the color LSD line segment detection algorithm includes the steps of:
and performing Gaussian down-sampling on the image, and reducing the resolution of the image.
And respectively carrying out gradient calculation on each pixel point of the image with reduced resolution, wherein R, G, B three channels of the pixel respectively apply a gradient algorithm to carry out gradient calculation on the corresponding channel, and the maximum value of the three gradients is taken as the gradient value of the current pixel.
Screening each pixel according to the output gradient value, and removing the pixels smaller than a gradient threshold value, wherein the gradient threshold value is
And performing region growing operation on the screened pixels according to the gradient values of the pixels, and combining the pixels in the same gradient direction as the current pixel to obtain a corresponding planning region.
And performing rectangular approximate calculation on all the planning areas to obtain corresponding straight line segments.
And outputting the coordinates of the end points of all the straight line segments.
The color LSD detection algorithm is optimized on the basis of the traditional LSD line segment detection algorithm, and comprises the following steps: performing Gaussian down-sampling on the picture once, reducing the picture, performing gradient detection on RGB three channels of the picture, determining the maximum gradient value of the three channels as a reference gradient, increasing the area, performing rectangular estimation, and outputting a detection result. When the sampled image is a color image, the gradient value is determined by performing gradient detection on three channels of the original image without gray processing, and the color edge line segment is quickly and accurately detected. In this embodiment, because the information of the bill includes the color line segment and the data, the color information is lost by using the conventional line segment detection algorithm, so that part of the key line segment cannot be read.
In step S4, in this embodiment, the line segment merging algorithm is:
wherein, analy _ merge (l (i), l (j)) is represented as, for vertex A, B of line segment l (i) and vertex C, D of l (j), judging included angles of line segments AC, AD, BC, BD and a horizontal line to be less than 1, less than 2, less than 3, less than 4, if the four included angles are less than 1 degree, judging whether the distance between two nearest vertices is less than 10, if yes, judging that the line segments successfully enter merge (l (i), and if not, judging that the line segments fail, and the threshold 10 is an optimal value obtained through multiple experiments.
merge (l (i), l (j)) means that two vertices with the longest distance are selected as new line segments for vertices A, B of line segment l (i) and vertex C, D of line segment l (j), and the new line segments are added to the set to delete the original two line segments.
Two line segments which are originally one line segment can be merged into one line segment through line segment merging, because in the line segment detection process, two line segments which are judged to be zigzag are broken due to the influence of original image distortion, image shooting, algorithm processing and other factors, and the two line segments can be restored through line segment merging.
In step S5, the grouping method is that when dx is counted>dy and y1<y0 and y2<y0, then the line segment L is classified as group 1; when dx>dy and y1>y0 and y2>y0, grouping the line segments L into a group 2; when dx<dy and x1<x0 and x2<x0, grouping the line segment L into a group 3; when dx<dy and x1>x0 and x2>x0, grouping the line segment L into a group 4; otherwise, rejecting the line segments; wherein dx is | x2-x1|, dy is | y2-y1|, and x0 is w2/2,y0＝h2/2。
Dividing the line segments into groups of an upper side, a lower side, a left side and a right side, wherein the difference value of horizontal coordinates of the upper side and the lower side is greater than the difference value of vertical coordinates, the upper side is positioned at the upper part of the image, the vertical coordinates are greater than the middle value of vertical coordinates of the image, and the lower side is positioned at the lower part of the image, the vertical coordinates are less than the middle value of the vertical coordinates of the image; and if the left side and the right side are smaller than the ordinate difference, the abscissa is smaller than the image abscissa middle value if the left side is positioned at the left part of the image, and the abscissa is larger than the image abscissa middle value if the right side is positioned at the right part of the image.
In the step S6, the filtering method is that, for any two line segments L1 and L2 in the current group, when ≈ (L1, L2) is less than 3 degrees and dst1< dst2, the deviated line segment L1 is deleted; when the angle (L1, L2) is less than 3 degrees and dst1 is greater than dst2, deleting a deviated line segment L2, wherein the angle (L1, L2) represents the included angle of L1 and L2, and the dst1 and the dst2 represent the distance between the straight line where L1 and L2 are located and a point (x0, y0) respectively; and if the longest line segment length in the current group is len, deleting all over-short line segments smaller than len/3 in the current group. The filtering process filters out short line segments closer to the center.
The method has the advantages that the problem that the bill edge cannot be detected due to long step length, small gradient and the like can be solved by reducing the image, and the real bill edge and the real bill vertex can be effectively found out through later-stage connection and screening. The line segment is detected by using an improved color LSD line segment detection algorithm, the detection of the color line segment can be adapted, the gray level image conversion is not needed, and the color bill line segment can be quickly identified. By filtering and combining line segments, a detection result of sub-pixel level precision can be obtained in a short time, and lines can be detected under the condition of low bill and background contrast.
Example two
A rapid positioning system for bill borders based on color line segment detection comprises a memory and a processor, wherein the memory stores instructions, and the instructions are suitable for being loaded by the processor and executing the following steps:
a frame of images containing a ticket is acquired.
And carrying out reduction processing on the one frame of image containing the bill according to the proportion by a linear interpolation method to obtain a reduced image. The linear interpolation method uses a scale value lv of 300/min (w, h), where w, h are the width and length of the frame including the image of the bill, respectively, and the width and height of the reduced image are w, respectively2＝w×lv，h2＝h×lv。
And finding all line segments in the reduced image by using a color LSD detection algorithm to form a line segment set L _ set, wherein the line segments are represented as L ═ { x1, y1, x2 and y2}, wherein (x1, y1) and (x2, y2) are two endpoints of the line segments L, the color LSD line segment detection algorithm performs Gaussian down-sampling on the bill image once and reduces the image, performs gradient detection on RGB three channels of the image, determines the maximum gradient value in the three channels as a reference gradient, and outputs the detected line segments through region growing operation and rectangle estimation operation.
And sorting all the line segments in the line segment set L _ set from large to small according to the length, merging the line segments by a line segment merging algorithm, and merging the line segments which are approximately on one straight line.
And (3) grouping the line segments after line segment combination, and dividing the line segments into 4 groups according to the coordinate difference of end points at two ends of the line segments, wherein the 4 groups correspond to the upper, lower, left and right sides of a quadrangle respectively.
And respectively filtering the 4 groups of line segments to remove unqualified line segments including deviated and too short line segments.
And sequentially taking a line segment from each group in 4 groups of filtered line segments according to the numbering sequence in the group by an exhaustion method, so that each line segment of each group is combined once, calculating the sum of the distances between the adjacent end points of two adjacent line segments in each combination, and outputting the four line segments when the sum of the distances is minimum.
And (3) forming the four output line segments into a quadrangle, solving intersection points of two adjacent line segments to obtain four intersection points, outputting the 4 intersection points, and considering the four intersection points as 4 vertexes of the bill.
In this embodiment, the line segment merging algorithm is:
wherein, analy _ merge (l (i), l (j)) is expressed as, for vertex A, B of segment l (i) and vertex C, D of l (j), judging the included angle of segment AC, AD, BC, BD and horizontal line is less than 1, less than 2, less than 3, less than 4, if four included angles are less than 1 degree, judging whether the distance between two nearest vertices is less than 10, if yes, judging to successfully enter merge (l (i), l (j), otherwise, judging to fail.
merge (l (i), l (j)) means that two vertices with the longest distance are selected as new line segments for vertices A, B of line segment l (i) and vertex C, D of line segment l (j), and the new line segments are added to the set to delete the original two line segments.
The grouping method is that when dx>dy and y1<y0 and y2<y0, then the line segment L is classified as group 1; when dx>dy and y1>y0 and y2>y0, grouping the line segments L into a group 2; when dx<dy and x1<x0 and x2<x0, grouping the line segment L into a group 3; when dx<dy and x1>x0 and x2>x0, grouping the line segment L into a group 4; otherwise, rejecting the line segments; wherein dx is | x2-x1|, dy is | y2-y1|, and x0 is w2/2,y0＝h2/2。
The color LSD line segment detection algorithm comprises the following steps:
and performing Gaussian down-sampling on the image, and reducing the resolution of the image.
And respectively carrying out gradient calculation on each pixel point of the image with reduced resolution, wherein R, G, B three channels of the pixel respectively apply a gradient algorithm to carry out gradient calculation on the corresponding channel, and the maximum value of the three gradients is taken as the gradient value of the current pixel.
Screening each pixel according to the output gradient value, and removing the pixels smaller than a gradient threshold value, wherein the gradient threshold value is
And performing region growing operation on the screened pixels according to the gradient values of the pixels, and combining the pixels in the same gradient direction as the current pixel to obtain a corresponding planning region.
And performing rectangular approximate calculation on all the planning areas to obtain corresponding straight line segments.
And outputting the coordinates of the end points of all the straight line segments.
The filtering processing method is that for any two line segments L1 and L2 in the current group, the angle (L1 and L2) represents the included angle of L1 and L2, and dst1 and dst2 represent the distance between the straight line where L1 and L2 are located and the point (x0 and y0), respectively. Deleting the deviated line segment L1 if ≈ (L1, L2) is less than 3 degrees and dst1< dst 2; deleting the deviated line segment L2 if ≈ (L1, L2) is less than 3 degrees and dst1> dst 2; and if the longest line segment length in the current group is len, deleting all over-short line segments smaller than len/3 in the current group.
The method has the advantages that the problem that the bill edge cannot be detected due to long step length, small gradient and the like can be solved by reducing the image, and the real bill edge and the real bill vertex can be effectively found out through later-stage connection and screening. The line segment is detected by using an improved color LSD line segment detection algorithm, the detection of the color line segment can be adapted, the gray level image conversion is not needed, and the color bill line segment can be quickly identified. By filtering and combining line segments, a detection result of sub-pixel level precision can be obtained in a short time, and lines can be detected under the condition of low bill and background contrast.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (6)
1. A method for rapidly positioning a bill frame based on color line segment detection is characterized by comprising the following steps:
s1: acquiring a frame of image containing a bill;
s2: and carrying out reduction processing on the one-frame image containing the bill according to the proportion by a linear interpolation method to obtain a reduced image, wherein the proportion value lv used by the linear interpolation method is 300/min (w, h), w and h are respectively the width and the length of the one-frame image containing the bill, and the width and the height of the reduced image are respectively w2＝w×lv，h2＝h×lv；
S3: identifying all line segments in the reduced image through a color LSD line segment detection algorithm, and outputting the line segments as L ═ { x1, y1, x2, y2}, wherein (x1, y1) and (x2, y2) are two endpoints of the line segments L, and all the line segments are combined into a line segment set L _ set; the color LSD line segment detection algorithm performs Gaussian down sampling on a bill image once and reduces the image, performs gradient detection on RGB three channels of the image, determines the maximum gradient value of the three channels as a reference gradient, and outputs a detected line segment through region growing operation and rectangle estimation operation;
in the step S3, the color LSD line segment detection algorithm specifically includes:
performing Gaussian down-sampling on the image, and reducing the resolution of the image;
respectively carrying out gradient calculation on each pixel point of the image with reduced resolution, wherein R, G, B three channels of the pixel respectively apply a gradient algorithm to carry out gradient calculation on corresponding channels, and the maximum value of the three gradients is taken as the gradient value of the current pixel;
screening each pixel according to the output gradient value, and removing the pixels smaller than a gradient threshold value, wherein the gradient threshold value is
Performing region growing operation on the screened pixels according to the gradient values of the pixels, and combining the pixels in the same gradient direction as the current pixel to obtain a corresponding planning region;
performing rectangular approximate calculation on all the planning areas to obtain corresponding straight line segments;
outputting the coordinates of the end points of all the straight line segments;
s4: sorting all line segments in the line segment set L _ set from large to small according to the length, and carrying out line segment combination through a line segment combination algorithm, wherein the line segment combination algorithm comprises the following steps:
wherein, analy _ merge (l (i), l (j)) is expressed as, for the vertex A, B of the line segment l (i) and the vertex C, D of l (j), judging the included angles of the line segments AC, AD, BC, BD and the horizontal line to be less than 1, less than 2, less than 3, less than 4, if the four included angles are less than 1 degree, judging whether the distance between the two nearest vertices is less than 10, if yes, judging that the line segment successfully enters the merge (l (i), l (j)), otherwise, judging that the line segment fails; merge (l (i), l (j)) means that two vertexes with the longest distance are selected as new line segments for vertexes A, B of line segment l (i) and vertex C, D of line segment l (j), and added to the set to delete the original two line segments;
s5: the line segments which are combined in the step S4 are grouped, and the line segments are divided into 4 groups according to the coordinate difference of end points at two ends of the line segments, wherein the 4 groups correspond to the upper, lower, left and right sides of a quadrangle respectively;
s6: respectively filtering the 4 groups of line segments to remove unqualified line segments including deviated and over-short line segments;
s7: sequentially taking a line segment from each group in sequence according to the numbering sequence in the group from 4 groups of line segments subjected to filtering treatment by an exhaustion method, so that each line segment of each group is combined once, calculating the sum of the distances between adjacent end points of two adjacent line segments in each combination, and outputting the four line segments when the sum of the distances is minimum;
s8: and (4) forming a quadrangle by the four line segments output in the step S7, calculating the intersection point of straight lines of two adjacent line segments to obtain four intersection points, and outputting the 4 intersection points as 4 vertexes of the bill.
2. The method of claim 1 based on color line segment detectionThe method for quickly positioning the bill borders is characterized in that in the step S5, the grouping method is that when dx is used>dy and y1<y0 and y2<y0, then the line segment L is classified as group 1; when dx>dy and y1>y0 and y2>y0, grouping the line segments L into a group 2; when dx<dy and x1<x0 and x2<x0, grouping the line segment L into a group 3; when dx<dy and x1>x0 and x2>x0, grouping the line segment L into a group 4; otherwise, rejecting the line segments; wherein dx is | x2-x1|, dy is | y2-y1|, and x0 is w2/2,y0＝h2/2。
3. The method for rapidly positioning bill borders based on color line segment detection according to claim 2, wherein in the step S6, the filtering processing method is to delete the deviated line segment L1 when ≈ (L1, L2) is less than 3 degrees and dst1< dst2 for any two line segments L1, L2 in the current group; when the angle (L1, L2) is less than 3 degrees and dst1 is greater than dst2, deleting a deviated line segment L2, wherein the angle (L1, L2) represents the included angle of L1 and L2, and the dst1 and the dst2 represent the distance between the straight line where L1 and L2 are located and a point (x0, y0) respectively;
and deleting too short line segments with the length less than len/3 in all the current groups, wherein len is the value with the longest line segment length in the current group.
4. The quick positioning system for the bill frame based on the color line segment detection is characterized by comprising a memory and a processor, wherein the memory stores instructions, and the instructions are suitable for being loaded by the processor and executing the following steps:
acquiring a frame of image containing a bill;
and carrying out reduction processing on the one-frame image containing the bill according to the proportion by a linear interpolation method to obtain a reduced image, wherein the proportion value lv used by the linear interpolation method is 300/min (w, h), w and h are respectively the width and the length of the one-frame image containing the bill, and the width and the height of the reduced image are respectively w2＝w×lv，h2＝h×lv；
Identifying all line segments in the reduced image, which are represented by L ═ x1, y1, x2, y2, by a color LSD line segment detection algorithm, where (x1, y1) and (x2, y2) are two endpoints of the line segment L, and grouping all line segments into a set of line segments L _ set; the color LSD line segment detection algorithm performs Gaussian down sampling on a bill image once and reduces the image, performs gradient detection on RGB three channels of the image, determines the maximum gradient value of the three channels as a reference gradient, and outputs a detected line segment through region growing operation and rectangle estimation operation;
the color LSD line segment detection algorithm specifically comprises the following steps:
performing Gaussian down-sampling on the image, and reducing the resolution of the image;
respectively carrying out gradient calculation on each pixel point of the image with reduced resolution, wherein R, G, B three channels of the pixel respectively apply a gradient algorithm to carry out gradient calculation on corresponding channels, and the maximum value of the three gradients is taken as the gradient value of the current pixel;
screening each pixel according to the output gradient value, and removing the pixels smaller than a gradient threshold value, wherein the gradient threshold value is
Performing region growing operation on the screened pixels according to the gradient values of the pixels, and combining the pixels in the same gradient direction as the current pixel to obtain a corresponding planning region;
performing rectangular approximate calculation on all the planning areas to obtain corresponding straight line segments;
outputting the coordinates of the end points of all the straight line segments;
sorting all line segments in the line segment set L _ set from large to small according to the length, and carrying out line segment combination through a line segment combination algorithm, wherein the line segment combination algorithm comprises the following steps:
wherein, analy _ merge (l (i), l (j)) is expressed as, for the vertex A, B of the line segment l (i) and the vertex C, D of l (j), judging the included angles of the line segments AC, AD, BC, BD and the horizontal line to be less than 1, less than 2, less than 3, less than 4, if the four included angles are less than 1 degree, judging whether the distance between the two nearest vertices is less than 10, if yes, judging that the line segment successfully enters the merge (l (i), l (j)), otherwise, judging that the line segment fails; merge (l (i), l (j)) means that two vertexes with the longest distance are selected as new line segments for vertexes A, B of line segment l (i) and vertex C, D of line segment l (j), and added to the set to delete the original two line segments;
grouping the line segments after line segment combination, and dividing the line segments into 4 groups according to the coordinate difference of end points at two ends of the line segments, wherein the 4 groups correspond to the upper, lower, left and right sides of a quadrangle respectively;
respectively filtering the 4 groups of line segments to remove unqualified line segments including deviated and over-short line segments;
sequentially taking a line segment from each group in sequence according to the numbering sequence in the group from 4 groups of line segments subjected to filtering treatment by an exhaustion method, so that each line segment of each group is combined once, calculating the sum of the distances between adjacent end points of two adjacent line segments in each combination, and outputting the four line segments when the sum of the distances is minimum;
and (4) forming the four output line segments into a quadrangle, calculating the intersection point of two adjacent line segment straight lines to obtain four intersection points, and outputting the 4 intersection points as 4 vertexes of the bill.
5. The system of claim 4 wherein the grouping is performed when dx is the color line segment detection based fast document frame location system>dy and y1<y0 and y2<y0, then the line segment L is classified as group 1; when dx>dy and y1>y0 and y2>y0, grouping the line segments L into a group 2; when dx<dy and x1<x0 and x2<x0, grouping the line segment L into a group 3; when dx<dy and x1>x0 and x2>x0, grouping the line segment L into a group 4; otherwise, rejecting the line segments; wherein dx is | x2-x1|, dy is | y2-y1|, and x0 is w2/2,y0＝h2/2。
6. The system for rapidly positioning bill borders based on color line segment detection according to claim 5, wherein the filtering processing method is that for any two line segments L1, L2 in the current set, when ≈ (L1, L2) is less than 3 degrees and dst1< dst2, the deviated line segment L1 is deleted; when the angle (L1, L2) is less than 3 degrees and dst1 is greater than dst2, deleting the segment L2, wherein the angle (L1, L2) represents the included angle between L1 and L2, and the dst1 and the dst2 represent the distance between the straight line where L1 and L2 are located and a point (x0, y0) respectively;
and deleting too short line segments with the length less than len/3 in all current groups, wherein len is the value with the longest line segment length in the current groups.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910273966.5A CN109977910B (en) | 2019-04-04 | 2019-04-04 | Rapid bill positioning method and system based on color line segments |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910273966.5A CN109977910B (en) | 2019-04-04 | 2019-04-04 | Rapid bill positioning method and system based on color line segments |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109977910A CN109977910A (en) | 2019-07-05 |
| CN109977910B true CN109977910B (en) | 2021-08-20 |
Family
ID=67083182
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910273966.5A Active CN109977910B (en) | 2019-04-04 | 2019-04-04 | Rapid bill positioning method and system based on color line segments |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109977910B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110414649B (en) * | 2019-07-29 | 2023-06-06 | 广州柔视智能科技有限公司 | DM code positioning method, device, terminal and storage medium |
| CN110533646B (en) * | 2019-08-27 | 2023-12-19 | 上海零眸智能科技有限公司 | Goods shelf detection method based on object detection |
| CN116258724A (en) * | 2023-05-15 | 2023-06-13 | 合肥联宝信息技术有限公司 | Image-based target positioning method and device, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102236784A (en) * | 2010-05-07 | 2011-11-09 | 株式会社理光 | Screen area detection method and system |
| CN103577817A (en) * | 2012-07-24 | 2014-02-12 | 阿里巴巴集团控股有限公司 | Method and device for identifying forms |
| CN105931239A (en) * | 2016-04-20 | 2016-09-07 | 北京小米移动软件有限公司 | Image processing method and device |
| CN106446881A (en) * | 2016-07-29 | 2017-02-22 | 北京交通大学 | Method for extracting lab test result from medical lab sheet image |
| CN106650608A (en) * | 2016-10-31 | 2017-05-10 | 广东工业大学 | Identification method for rectangle locating frame in test paper without locating points |
| CN107038721A (en) * | 2017-03-27 | 2017-08-11 | 西安交通大学 | A kind of line detection method based on LAPJV algorithms |
| CN107392141A (en) * | 2017-07-19 | 2017-11-24 | 武汉大学 | A kind of airport extracting method based on conspicuousness detection and LSD straight-line detections |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7171056B2 (en) * | 2003-02-22 | 2007-01-30 | Microsoft Corp. | System and method for converting whiteboard content into an electronic document |
| CN103871052A (en) * | 2014-02-21 | 2014-06-18 | 杭州奥视图像技术有限公司 | Straight line detection-based safety belt detecting algorithm |
| CN109300118B (en) * | 2018-09-11 | 2021-09-10 | 东北大学 | High-voltage power line unmanned aerial vehicle inspection method based on RGB image |
| CN109544577B (en) * | 2018-11-27 | 2022-10-14 | 辽宁工程技术大学 | Improved straight line extraction method based on edge point grouping |
-
2019
- 2019-04-04 CN CN201910273966.5A patent/CN109977910B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102236784A (en) * | 2010-05-07 | 2011-11-09 | 株式会社理光 | Screen area detection method and system |
| CN103577817A (en) * | 2012-07-24 | 2014-02-12 | 阿里巴巴集团控股有限公司 | Method and device for identifying forms |
| CN105931239A (en) * | 2016-04-20 | 2016-09-07 | 北京小米移动软件有限公司 | Image processing method and device |
| CN106446881A (en) * | 2016-07-29 | 2017-02-22 | 北京交通大学 | Method for extracting lab test result from medical lab sheet image |
| CN106650608A (en) * | 2016-10-31 | 2017-05-10 | 广东工业大学 | Identification method for rectangle locating frame in test paper without locating points |
| CN107038721A (en) * | 2017-03-27 | 2017-08-11 | 西安交通大学 | A kind of line detection method based on LAPJV algorithms |
| CN107392141A (en) * | 2017-07-19 | 2017-11-24 | 武汉大学 | A kind of airport extracting method based on conspicuousness detection and LSD straight-line detections |
Non-Patent Citations (1)
| Title |
|---|
| "基于拍照的银行卡卡号检测";杨芸芸;《中国优秀硕士学位论文全文数据库 信息科技辑》;20171115(第11期);第13-17页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109977910A (en) | 2019-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106960208B (en) | Method and system for automatically segmenting and identifying instrument liquid crystal number | |
| CN105913093B (en) | A kind of template matching method for Text region processing | |
| CN104112128B (en) | Digital image processing system and method applied to bill image character recognition | |
| CN109977910B (en) | Rapid bill positioning method and system based on color line segments | |
| CN109977907B (en) | Expense reimbursement method and system based on bill recognition | |
| CN110286126A (en) | A kind of wafer surface defects subregion area detecting method of view-based access control model image | |
| CN103034848B (en) | A kind of recognition methods of form types | |
| US20070253040A1 (en) | Color scanning to enhance bitonal image | |
| CN109948135B (en) | Method and equipment for normalizing image based on form characteristics | |
| CN108985305B (en) | Laser etching industrial detonator coded image positioning and correcting method | |
| CN110781877B (en) | Image recognition method, device and storage medium | |
| CN111259891B (en) | Method, device, equipment and medium for identifying identity card in natural scene | |
| US8744177B2 (en) | Image processing method and medium to extract a building region from an image | |
| CN110598566A (en) | Image processing method, device, terminal and computer readable storage medium | |
| CN113609984A (en) | Pointer instrument reading identification method and device and electronic equipment | |
| CN113673541A (en) | Image sample generation method for target detection and application | |
| CN116052152A (en) | License plate recognition system based on contour detection and deep neural network | |
| CN111626145A (en) | Simple and effective incomplete form identification and page-crossing splicing method | |
| CN108197624A (en) | The recognition methods of certificate image rectification and device, computer storage media | |
| CN113591507A (en) | Robust two-dimensional code DataMatrix positioning method and system | |
| Pan et al. | An efficient method for skew correction of license plate | |
| JP2000048120A (en) | Method for extracting character area of gray level image and recording medium having recorded the program thereon | |
| CN115690107A (en) | High-throughput counting method, system and device for pod fruit grains based on image processing | |
| CN115376149A (en) | Reimbursement invoice identification method | |
| CN112418210B (en) | Intelligent classification method for tower inspection information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |