CN114972309A - Image-based table detection method for broken connection of table lines - Google Patents
Image-based table detection method for broken connection of table lines Download PDFInfo
- Publication number
- CN114972309A CN114972309A CN202210702992.7A CN202210702992A CN114972309A CN 114972309 A CN114972309 A CN 114972309A CN 202210702992 A CN202210702992 A CN 202210702992A CN 114972309 A CN114972309 A CN 114972309A
- Authority
- CN
- China
- Prior art keywords
- image
- line
- vertical
- horizontal
- kernel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 30
- 230000009466 transformation Effects 0.000 claims description 21
- 230000000877 morphologic effect Effects 0.000 claims description 14
- 230000007797 corrosion Effects 0.000 claims description 8
- 238000005260 corrosion Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000012937 correction Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000003672 processing method Methods 0.000 claims description 3
- 238000007689 inspection Methods 0.000 claims 7
- 238000007405 data analysis Methods 0.000 abstract description 4
- 238000005286 illumination Methods 0.000 abstract description 4
- 230000008439 repair process Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 230000010339 dilation Effects 0.000 description 4
- 230000003628 erosive effect Effects 0.000 description 4
- 238000012015 optical character recognition Methods 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method for connecting broken grid lines in image-based table detection. Under the influence of illumination conditions, photographing angles and the like, the problems of shadows, low contrast, table image distortion and the like exist in the table image shot by the mobile terminal equipment under natural conditions, so that the table line fracture phenomenon is easy to occur in table detection, the acquired table structure information is wrong, and the accuracy and reliability of subsequent data analysis are influenced; by adopting the table line fracture connection method in the image-based table detection, automatic connection and repair of the table lines under the condition of the table line fracture in table identification can be basically realized.
Description
Technical Field
The invention relates to the field of image processing and Optical Character Recognition (OCR), in particular to a method for connecting broken grid lines in table detection based on an image.
Background
The table is used as a structured organization mode of information, has the characteristics of high refinement, conciseness, standardization and the like, and is commonly used for data record statistics, experimental result analysis and the like. At present, a plurality of form documents are provided in the form of pictures, and the form detection and identification task is to restore the form information in the form of pictures into digital data so as to provide a basis for further processing and data analysis.
In the task of detecting and identifying the table, the coordinate information of each table unit in the table image needs to be automatically acquired, and the structural relationship of the table is established accordingly. The device for acquiring the form image mainly comprises a scanner, a special high-speed shooting device and a handheld mobile terminal device. Due to the advantages of popularization and convenience of mobile terminal equipment, people tend to use the mobile terminal to acquire form images and identify form contents at any time. Under the influence of illumination conditions, photographing angles and the like, the table images photographed by the mobile terminal device under natural conditions have the problems of shadows, low contrast, table image distortion and the like, so that the fracture phenomenon of table lines is easy to occur in table detection, the acquired table structure information is wrong, and the accuracy and reliability of subsequent data analysis are influenced.
Disclosure of Invention
The invention aims to solve the defects in the prior art, provides a table line fracture connection method in image-based table detection, and aims to solve the problem of table line fracture when the table detection is carried out on images shot by mobile terminals such as mobile phones;
the breaking phenomenon of the grid line in the table test can be described as the following four cases:
1) the connection between the end point on the vertical form line in the form and the horizontal form line on the upper part of the vertical form line is broken;
2) the connection between the lower end point of the vertical form line in the form and the horizontal form line below the lower end point is broken;
3) the connection between the left end point of the horizontal form line in the table and the vertical form line on the left side of the horizontal form line is broken;
4) the connection between the right end point of the horizontal form line in the table and the horizontal form line to the right of it is broken.
In order to achieve the purpose, the invention adopts the following technical scheme: a method for connecting broken grid lines in image-based table detection comprises the following steps:
s1: converting the collected color form image into a gray level image, and performing image enhancement processing by adopting an image gamma conversion technology;
s2: carrying out distortion correction on the enhanced image by utilizing perspective transformation to obtain a corrected gray level image G (x, y);
s3: extracting a binary image B (x, y) only containing a table contour from the gray level image G (x, y) by using a table extraction technology;
s4-1: designing a horizontal kernel by using an image morphological algorithm, erasing a horizontal line segment of a table in a binary image B (x, y) by adopting an opening operation (firstly corroding and then expanding), and reserving a vertical table line segment;
s4-2: designing a vertical kernel by using an image morphological algorithm, erasing a vertical line segment of a table in a binary image B (x, y) by adopting an opening operation (firstly corroding and then expanding), and reserving a horizontal table line segment;
s5: respectively carrying out image logical negation operation on the vertical table line segment and the horizontal table line segment obtained in the step S4-1 and the step S4-2, then carrying out logical AND operation on the image logical negation operation and the operated binary image to obtain intersection point coordinates (x _ cross, y _ cross) of the vertical line segment and the horizontal line segment in the table;
s6: and judging the table line breakage problem and performing connection processing.
Preferably, in step S2, the image after the enhancement is subjected to distortion correction by perspective transformation to obtain a corrected gray-scale image G (x, y), and the specific steps are as follows: s21: the enhanced gray image is converted into a binary image, a minimum circumscribed rectangle minRectbox and a circumscribed rectangle Rectbox of the overall form area outline are obtained by utilizing a morphological processing method and a connected domain extraction algorithm, [ SrcPoint1, SrcPoint2, SrcPoint3, SrcPoint4] and [ DstPoint1, DstPoint2, DstPoint3 and DstPoint4] are respectively four corner points corresponding to the minimum circumscribed rectangle and the circumscribed rectangle, a transformation matrix M is calculated by utilizing a perspective transformation principle,
s22: performing perspective transformation on the gray-scale image by using a transformation matrix M to obtain a transformed gray-scale image G (x, y), wherein the image scale is (H _ image, W _ image); the mapping relation between the pixel coordinate point (X, Y) in the corrected image and the corresponding coordinate point (X, Y) in the original image is as follows:
preferably, in the step S4-1, a horizontal kernel is designed by using an image morphology algorithm, and an open operation (first erosion and then dilation) is adopted to erase a horizontal line segment of the table in the binary image B (x, y) and retain a vertical table line segment, and the specific steps are as follows:
s4-11: designing an image morphological operation as a rectangular Kernel, wherein the dimension of the rectangular Kernel is (H _ Kernel, W _ Kernel), and the H _ Kernel is usually 1 or 2;
w _ Kernel = W _ image/60, which is referred to herein as the rectangular Kernel;
s4-12: performing open operation (first corrosion and then expansion) on the binary image B (x, y) by using a horizontal core designed in S4-11, and erasing a horizontal line segment in the B (x, y) to obtain a vertical table line image V (x, y) of the table;
s4-13: two end points (x _ endpoint, y _ endpoint) of each vertical line segment contour are taken from the obtained vertical form line image V (x, y).
Preferably, in the step S4-2, a vertical kernel is designed by using an image morphology algorithm, and an open operation (first erosion and then dilation) is adopted to erase a vertical line segment of the table in the binary image B (x, y), and a horizontal table line segment is retained, specifically, the steps are as follows:
s4-21: designing an image morphological operation as a rectangular Kernel, wherein the dimension of the rectangular Kernel is (H _ Kernel, W _ Kernel), wherein H _ Kernel = W _ image/60, and W _ Kernel usually takes 1 or 2, and the rectangular Kernel is referred to as a vertical Kernel;
s4-22: performing open operation (first corrosion and later expansion) on the binary image B (x, y) by using a vertical core designed in S4-21, and erasing a vertical line segment in the B (x, y) to obtain a horizontal table line image H (x, y) of the table;
s4-23: two end points (x _ end, y _ end) of each horizontal line segment contour are taken from the horizontal table line image H (x, y).
Preferably, in step S6, it is determined whether there is a fracture problem in the connection between the end point on the vertical form line in the form and the horizontal form line above the end point, and if so, the vertical line segment is drawn upward in the grayscale image G (x, y) to complete the fracture connection of the form line.
Preferably, in step S6, it is determined whether there is a fracture problem in the connection between the lower end point of the vertical table line in the table and the horizontal table line below the lower end point, and if there is a fracture problem, drawing the vertical line segment downward in the grayscale image G (x, y) completes the fracture connection of the table line.
Preferably, in step S6, it is determined whether there is a fracture problem in the connection between the left end point of the horizontal table line and the vertical table line at the left end, and if so, the horizontal line segment is drawn to the left in the grayscale image G (x, y) to complete the fracture connection of the table line.
Preferably, in step S6, it is determined whether there is a fracture problem in the connection between the right end point of the horizontal table line and the vertical table line at the right end, and if so, drawing the horizontal line segment to the right in the grayscale image G (x, y) completes the fracture connection of the table line.
Compared with the prior art, the invention has the beneficial effects that:
under the influence of illumination conditions, photographing angles and the like, the problems of shadows, low contrast, table image distortion and the like exist in the table image shot by the mobile terminal equipment under natural conditions, so that the table line fracture phenomenon is easy to occur in table detection, the acquired table structure information is wrong, and the accuracy and reliability of subsequent data analysis are influenced; by adopting the table line fracture connection method in the image-based table detection, the table line connection repair under the condition of table line fracture in table identification can be basically realized.
Drawings
FIG. 1 is a flow chart of a method of break-and-connect of gridlines in image-based table detection in accordance with the present invention.
Fig. 2 is an original form image to be subjected to form recognition in the form line fracture connection method in the image-based form detection of the present invention.
Fig. 3 is a schematic diagram of four corner points of a minimum bounding rectangle and a bounding rectangle of the overall table region outline of the method for connecting broken ruled lines in image-based table detection of the present invention, where point "∘" is the four corner points of the minimum bounding rectangle, and point "□" is the four corner points of the bounding rectangle.
Fig. 4 is a table contour image after extraction of the table structure of the method for break-and-connect of the grid lines in the image-based table detection according to the present invention.
FIG. 5 is a vertical table line graph of a table image after erasing horizontal table lines in the method for table line break connection in image-based table detection of the present invention.
FIG. 6 is a horizontal table line graph of a table image after erasing vertical table lines in the method for break-and-connect of table lines in image-based table detection of the present invention.
Fig. 7 is an intersection of a horizontal table line graph and a vertical table line graph of the method for table line fracture connection in image-based table detection according to the present invention.
Fig. 8 is a schematic diagram of the table line fracture at the end point of the vertical table line in the table line fracture connection method in the image-based table detection of the present invention, where the end point "□" indicates that there is a first type of table line fracture at this point, and the end point "o" indicates that there is a second type of table line fracture at this point.
Fig. 9 is a schematic diagram of table line breakage bonds when a vertical table line is broken in the table line breakage connection method in the image-based table detection of the present invention, where a scribe line extends upward at the end point "□" and a scribe line extends downward at the end point "o".
Fig. 10 is a schematic diagram of the table line fracture at the end point of the horizontal table line in the table line fracture connection method in the image-based table detection of the present invention, where the end point "□" indicates that the third type of table line fracture exists at the point, and the end point "o" indicates that the fourth type of table line fracture exists at the point.
Fig. 11 is a schematic diagram of table line breakage bonds when a horizontal table line is broken in the table line breakage connection method in the image-based table detection of the present invention, where a scribe line extends leftward at the end point "□" and a scribe line extends rightward at the end point "o".
Fig. 12 is a table image after performing table line fracture connection using the table line fracture connection method in image-based table detection of the present invention.
Detailed Description
In order to further understand the objects, structures, features and functions of the present invention, the following embodiments are described in detail.
Referring to fig. 1 to 12, the present invention provides a method for table line fracture connection in table detection based on images, which comprises the following steps:
s1: converting the collected color form image into a gray level image, and performing image enhancement processing by adopting an image gamma conversion technology; the image is enhanced to eliminate the influence of low image contrast, uneven illumination and the like on the table detection performance.
S2: and carrying out distortion correction on the enhanced image by utilizing perspective transformation to obtain a corrected gray level image G (x, y).
S3: extracting a binary image B (x, y) only containing a table contour from the gray level image G (x, y) by using a table extraction technology; as shown in fig. 4, the table lines in the image B (x, y) are broken, which may affect the detection of the subsequent table cells and the recognition of the characters in each cell.
In addition, through perspective transformation in step S2, the original form lines having a certain inclination angle are transformed into vertical and horizontal form lines.
S4-1: designing a horizontal kernel by using an image morphological algorithm, erasing a horizontal line segment of a table in a binary image B (x, y) by adopting an opening operation (firstly corroding and then expanding), and reserving a vertical table line segment;
s4-2: and (3) designing a vertical kernel by using an image morphology algorithm, erasing a vertical line segment of a table in the binary image B (x, y) by adopting an opening operation (firstly corroding and then expanding), and reserving a horizontal table line segment.
S5: and performing image non-operation on the vertical table line segment and the horizontal table line segment obtained in the step S4-1 and the step S4-2, performing logical AND operation on the image non-operation and the operated binary image, and obtaining intersection point coordinates (x _ cross, y _ cross) of the vertical line segment and the horizontal line segment in the table.
S6: and judging the table line breakage problem and performing connection processing.
As shown in fig. 3, preferably, in step S2, the image is subjected to distortion correction by perspective transformation on the enhanced image to obtain a corrected grayscale image G (x, y), and the specific steps are as follows:
s21: the enhanced gray level image is converted into a binary image, and the minimum circumscribed rectangle minRectbox and the circumscribed rectangle Rectbox of the overall form area outline, SrcPoint1, SrcPoint2, SrcPoint3 and SrcPoint4 are obtained by utilizing a morphological processing method and a connected domain extraction algorithm]And [ DstPoint1, DstPoint2, DstPoint3, DstPoint4]Four corner points corresponding to the minimum circumscribed rectangle and the circumscribed rectangle respectively, points "∘" in fig. 3 are the four corner points of the minimum circumscribed rectangle, points "□" are the four corner points of the circumscribed rectangle, and the transformation matrix is calculated by using the perspective transformation principleM，
Transformation matrixMIs composed of a minimum circumscribed rectangle minRectbox and a circumscribed rectangle Rectbox, [ SrcPoint1, SrcPoint2, SrcPoint3 and SrcPoint4]And [ DstPoint1，DstPoint2，DstPoint3，DstPoint4]Obtained through perspective transformation, the matrixMIs a 3 × 3 matrix with elements of m;
s22: using transformation matricesMPerforming perspective transformation on the gray level image to obtain a transformed gray level image G (x, y) with an image scale of (H _ image, W _ image); the mapping relation between the pixel coordinate point (X, Y) in the corrected image and the corresponding coordinate point (X, Y) in the original image is as follows:
an image scale (H _ image, W _ image), which is the pixel size of the image, H _ image being the image height, W _ image being the image width;
preferably, in the step S4-1, a horizontal kernel is designed by using an image morphology algorithm, and an open operation (first erosion and then dilation) is adopted to erase a horizontal line segment of the table in the binary image B (x, y) and retain a vertical table line segment, and the specific steps are as follows:
s4-11: designing an image morphological operation as a rectangular Kernel, wherein the dimension of the rectangular Kernel is (H _ Kernel, W _ Kernel), and the H _ Kernel is usually 1 or 2;
w _ Kernel = W _ image/60, and the rectangular Kernel is referred to herein as the horizontal Kernel.
S4-12: performing open operation (first corrosion and then expansion) on the binary image B (x, y) by using a horizontal core designed in S4-11, and erasing a horizontal line segment in the B (x, y) to obtain a vertical table line image V (x, y) of the table;
s4-13: two end points (x _ end, y _ end) of each vertical line segment contour are taken in the vertical table line image V (x, y).
Preferably, in the step S4-2, a vertical kernel is designed by using an image morphology algorithm, and an open operation (first erosion and then dilation) is adopted to erase a vertical line segment of the table in the binary image B (x, y), and a horizontal table line segment is retained, specifically, the steps are as follows:
s4-21: designing an image morphological operation as a rectangular Kernel, wherein the dimension of the rectangular Kernel is (H _ Kernel, W _ Kernel), wherein H _ Kernel = W _ image/60, and W _ Kernel usually takes 1 or 2, and the rectangular Kernel is referred to as a vertical Kernel;
s4-22: performing open operation (first corrosion and later expansion) on the binary image B (x, y) by using a vertical core designed in S4-21, and erasing a vertical line segment in the B (x, y) to obtain a horizontal table line image H (x, y) of the table;
s4-23: two end points (x _ end, y _ end) of each horizontal line segment contour are taken from the horizontal table line image H (x, y).
Preferably, in step S6, it is determined whether there is a fracture problem in the connection between the end point on the vertical form line in the form and the horizontal form line above the end point, and if so, the vertical line segment is drawn upward in the grayscale image G (x, y) to complete the fracture connection of the form line.
Respectively calculating the coordinate difference (x _ endpoint-x _ crosspoint, y _ endpoint-y _ crosspoint) of the endpoint (x _ endpoint, y _ endpoint) and each intersection point (x _ crosspoint, y _ crosspoint) on each vertical line segment;
if there is an intersection point where the distance between the end point (x _ end, y _ end) and the intersection point (x _ cross point, y _ cross point) is smaller than the threshold value D _ samepoint, the end point and the intersection point are considered to be coincident, and the table line breakage problem does not exist; here, D _ samepoint takes a distance of 3 pixels, otherwise, there is a table line fracture problem, as shown in fig. 8, there is a table line fracture problem in the first case at the end point "□".
If there is a table line breakage problem, a binary image Draw (x, y) of all zero values is generated, on which line segments are drawn with (x _ end, y _ end) and (x _ end, y _ leftup) as end points.
Performing logical AND operation on the binary images Draw (x, y) and H (x, y), obtaining intersection points (x _ cross point, y _ cross point) of the vertical line segment and the horizontal table line, and taking the point with the minimum value of y _ cross point in the intersection points as a target point (x _ target, y _ target)
When the gray image G (x, y) connects the end point (x _ end, y _ end) and the target point (x _ target, y _ target), the form line connection in the first case at the time of the form line break is completed, as shown in fig. 9 as an extension line of the form upward at the end point "□".
Preferably, in step S6, it is determined whether there is a fracture problem in the connection between the lower end point of the vertical table line in the table and the horizontal table line below the lower end point, and if so, drawing the vertical line segment downward in the grayscale image G (x, y) completes the fracture connection of the table line.
Respectively calculating the coordinate difference (x _ endpoint-x _ crosspoint, y _ endpoint-y _ crosspoint) of the lower endpoint (x _ endpoint, y _ endpoint) and each intersection point (x _ crosspoint, y _ crosspoint) of each vertical line segment;
if there is an intersection point where the distance between the end point (x _ endpoint, y _ endpoint) and the intersection point (x _ cross point, y _ cross point) is smaller than the threshold value D _ samepoint, it is considered that the end point and the intersection point coincide, and there is no table line fracture problem, where D _ samepoint takes a distance of 3 pixels, otherwise there is a table line fracture problem in the second case, as shown in fig. 8, there is a table line fracture problem in the second case at the end point "∘".
If there is a table line breakage problem, a binary image Draw (x, y) with all zero values is generated, and line segments are drawn on the image with (x _ end, y _ end) and (x _ end, y _ right bottom) as end points.
Performing logical AND operation on the binary images Draw (x, y) and H (x, y), obtaining intersection points (x _ cross point, y _ cross point) of the vertical line segment and the horizontal table line, and taking the point with the maximum value of y _ cross point in the intersection points as a target point (x _ target, y _ target)
When the gray image G (x, y) connects the end point (x _ end, y _ end) and the target point (x _ target, y _ target), the table line connection in the second case when the table line is broken is completed, as shown in fig. 9 as the table extension line downward at the end point "∘".
Preferably, in step S6, it is determined whether there is a fracture problem in the connection between the left end point of the horizontal table line and the vertical table line at the left end, and if so, the horizontal line segment is drawn to the left in the grayscale image G (x, y) to complete the fracture connection of the table line.
Respectively calculating the coordinate difference (x _ end-x _ cross point, y _ end-y _ cross point) of the left end point (x _ end, y _ end) and each intersection point (x _ cross point, y _ cross point) of each horizontal line segment;
if there is an intersection point where the distance between the end point (x _ end, y _ end) and the intersection point (x _ cross point, y _ cross point) is smaller than the threshold value D _ samepoint, it is considered that the end point and the intersection point are coincident, and there is no table line fracture problem, where D _ samepoint takes a distance of 3 pixels, otherwise there is a table line fracture problem in the third case, as shown in fig. 10, there is a table line fracture problem in the third case at the end point "□".
If there is a table line breakage problem, a binary image Draw (x, y) of all zero values is generated, on which line segments are drawn with (x _ endpoint, y _ endpoint) and (x _ leftup, y _ endpoint) as end points.
Performing logical AND operation on the binary images Draw (x, y) and V (x, y), obtaining intersection points (x _ cross point, y _ cross point) of the horizontal line segment and the vertical table line, and taking the point with the maximum value of x _ cross point in the intersection points as a target point (x _ target, y _ target)
When the gray image G (x, y) connects the end point (x _ end, y _ end) and the target point (x _ target, y _ target), the table line connection in the third case when the table line is broken is completed, as shown in fig. 11 as the table extension line to the left at the end point "□".
Preferably, in step S6, it is determined whether there is a fracture problem in the connection between the right end point of the horizontal table line and the vertical table line at the right end, and if so, drawing the horizontal line segment to the right in the grayscale image G (x, y) completes the fracture connection of the table line.
Respectively calculating the coordinate difference (x _ end-x _ cross point, y _ end-y _ cross point) of the right end point (x _ end, y _ end) and each intersection point (x _ cross point, y _ cross point) of each horizontal line segment;
if there is an intersection point where the distance between the end point (x _ endpoint, y _ endpoint) and the intersection point (x _ cross point, y _ cross point) is smaller than the threshold value D _ samepoint, it is considered that the end point and the intersection point coincide, and there is no table line fracture problem, where D _ samepoint takes a distance of 3 pixels, otherwise there is the table line fracture problem in the first case, as shown in fig. 10, there is the table line fracture problem in the fourth case at the end point "∘".
If there is a table line breakage problem, a binary image Draw (x, y) of all zero values is generated, and line segments are drawn on the image with (x _ end, y _ end) and (x _ end, y _ leftup) as end points.
Performing logical AND operation on the binary images Draw (x, y) and H (x, y), finding the intersection points (x _ cross point, y _ cross point) of the horizontal line segment and the vertical table line, and taking the point with the minimum value of x _ cross point as the target point (x _ target, y _ target)
When the gray image G (x, y) connects the end point (x _ end, y _ end) and the target point (x _ target, y _ target), the table line connection in the fourth case when the table line is broken is completed, and the table extension line to the right at the end point "∘" is shown in fig. 11.
Fig. 12 is a table image in which table line fracture is connected by the table line fracture connection method in the image-based table detection according to the present invention, and it can be seen from the table image that table line fracture repair in the case of table line fracture can be basically realized by the table line fracture connection method in the image-based table detection according to the present invention.
The present invention has been described in relation to the above embodiments, which are only exemplary of the implementation of the present invention. It should be noted that the disclosed embodiments do not limit the scope of the invention. Rather, it is intended that all such modifications and variations be included within the spirit and scope of this invention.
Claims (8)
1. A method for connecting broken grid lines in table detection based on images is characterized in that: the method comprises the following steps:
s1: converting the collected color form image into a gray level image, and performing image enhancement processing by adopting an image gamma conversion technology;
s2: carrying out distortion correction on the enhanced image by utilizing perspective transformation to obtain a corrected gray level image G (x, y);
s3: extracting a binary image B (x, y) only containing a table contour from the gray level image G (x, y) by using a table extraction technology;
s4-1: designing a horizontal kernel by using an image morphological algorithm, erasing a horizontal line segment of a table in a binary image B (x, y) by adopting an opening operation (firstly corroding and then expanding), and reserving a vertical table line segment;
s4-2: designing a vertical kernel by using an image morphological algorithm, erasing a vertical line segment of a table in a binary image B (x, y) by adopting an opening operation (firstly corroding and then expanding), and reserving a horizontal table line segment;
s5: respectively carrying out image logical negation operation on the vertical table line segment and the horizontal table line segment obtained in the step S4-1 and the step S4-2, then carrying out logical AND operation on the image logical negation operation and the operated binary image to obtain intersection point coordinates (x _ cross, y _ cross) of the vertical line segment and the horizontal line segment in the table;
s6: and judging the table line breakage problem and performing connection processing.
2. The method of claim 1 for table-line fracture connection in image-based table inspection, wherein: in step S2, the enhanced image is subjected to distortion correction by perspective transformation to obtain a corrected grayscale image G (x, y), and the specific steps are as follows: s21: the enhanced gray image is converted into a binary image, a minimum circumscribed rectangle minRectbox and a circumscribed rectangle Rectbox of the overall form area outline are obtained by utilizing a morphological processing method and a connected domain extraction algorithm, [ SrcPoint1, SrcPoint2, SrcPoint3, SrcPoint4] and [ DstPoint1, DstPoint2, DstPoint3 and DstPoint4] are respectively four corner points corresponding to the minimum circumscribed rectangle and the circumscribed rectangle, a transformation matrix M is calculated by utilizing a perspective transformation principle,
s22: performing perspective transformation on the gray-scale image by using a transformation matrix M to obtain a transformed gray-scale image G (x, y), wherein the image scale is (H _ image, W _ image); the mapping relation between the pixel coordinate point (X, Y) in the corrected image and the corresponding coordinate point (X, Y) in the original image is as follows:
3. the method of claim 2 for table-line fracture connection in image-based table inspection, wherein: in the step S4-1, a horizontal kernel is designed by using an image morphology algorithm, an open operation (first corrosion and then expansion) is adopted to erase a horizontal line segment of a table in a binary image B (x, y), and a vertical table line segment is reserved, and the specific steps are as follows:
s4-11: designing an image morphological operation as a rectangular Kernel, wherein the dimension of the rectangular Kernel is (H _ Kernel, W _ Kernel), and the H _ Kernel is usually 1 or 2;
w _ Kernel = W _ image/60, which is referred to herein as the rectangular Kernel;
s4-12: performing open operation (first corrosion and then expansion) on the binary image B (x, y) by using a horizontal core designed in S4-11, and erasing a horizontal line segment in the B (x, y) to obtain a vertical table line image V (x, y) of the table;
s4-13: two end points (x _ endpoint, y _ endpoint) of each vertical line segment contour are taken from the obtained vertical form line image V (x, y).
4. The method of claim 3 for table-line fracture connection in image-based table inspection, wherein: in the step S4-2, a vertical kernel is designed by using an image morphology algorithm, an open operation (first corrosion and then expansion) is adopted to erase a vertical line segment of a table in the binary image B (x, y), and a horizontal table line segment is reserved, and the specific steps are as follows:
s4-21: designing an image morphological operation as a rectangular Kernel, wherein the dimension of the rectangular Kernel is (H _ Kernel, W _ Kernel), wherein H _ Kernel = W _ image/60, and W _ Kernel usually takes 1 or 2, and the rectangular Kernel is referred to as a vertical Kernel;
s4-22: performing open operation (first corrosion and later expansion) on the binary image B (x, y) by using a vertical core designed in S4-21, and erasing a vertical line segment in the B (x, y) to obtain a horizontal table line image H (x, y) of the table;
s4-23: two end points (x _ end, y _ end) of each horizontal line segment contour are taken from the horizontal table line image H (x, y).
5. The method of claim 4 for table-line fracture connection in image-based table inspection, wherein: in step S6, it is determined whether there is a problem of breaking the connection between the end point on the vertical form line in the form and the horizontal form line above it, and if so, the vertical line segment is drawn upward in the grayscale image G (x, y) to complete the broken connection of the form line.
6. The method of claim 4 for table-line fracture connection in image-based table inspection, wherein: in step S6, it is determined whether there is a fracture problem in the connection between the lower end point of the vertical table line in the table and the horizontal table line below the lower end point, and if so, drawing the vertical line segment downward in the grayscale image G (x, y) completes the fracture connection of the table line.
7. The method of claim 4 for table-line fracture connection in image-based table inspection, wherein: in step S6, it is determined whether there is a problem of fracture in the connection between the left end point of the horizontal form line and the vertical form line at the left end thereof, and if so, a horizontal line segment is drawn to the left in the grayscale image G (x, y) to complete the fracture connection of the form line.
8. The method of claim 4 for table-line fracture connection in image-based table inspection, wherein: in step S6, it is determined whether there is a problem of fracture in the connection between the right end point of the horizontal form line and the vertical form line at the right end, and if so, a horizontal line segment is drawn rightward in the grayscale image G (x, y) to complete the fracture connection of the form line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210702992.7A CN114972309A (en) | 2022-06-21 | 2022-06-21 | Image-based table detection method for broken connection of table lines |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210702992.7A CN114972309A (en) | 2022-06-21 | 2022-06-21 | Image-based table detection method for broken connection of table lines |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114972309A true CN114972309A (en) | 2022-08-30 |
Family
ID=82964951
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210702992.7A Pending CN114972309A (en) | 2022-06-21 | 2022-06-21 | Image-based table detection method for broken connection of table lines |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114972309A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112016557A (en) * | 2020-08-26 | 2020-12-01 | 上海致宇信息技术有限公司 | Form interference line removing algorithm |
| CN112036294A (en) * | 2020-08-28 | 2020-12-04 | 山谷网安科技股份有限公司 | Method and device for automatically identifying paper table structure |
| CN112800731A (en) * | 2021-02-23 | 2021-05-14 | 浪潮云信息技术股份公司 | Table repairing method for dealing with distorted graphs in image table extraction |
| CN114120303A (en) * | 2021-11-25 | 2022-03-01 | 南京华苏科技有限公司 | Detection method of image table based on MSER under natural photographing condition |
-
2022
- 2022-06-21 CN CN202210702992.7A patent/CN114972309A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112016557A (en) * | 2020-08-26 | 2020-12-01 | 上海致宇信息技术有限公司 | Form interference line removing algorithm |
| CN112036294A (en) * | 2020-08-28 | 2020-12-04 | 山谷网安科技股份有限公司 | Method and device for automatically identifying paper table structure |
| CN112800731A (en) * | 2021-02-23 | 2021-05-14 | 浪潮云信息技术股份公司 | Table repairing method for dealing with distorted graphs in image table extraction |
| CN114120303A (en) * | 2021-11-25 | 2022-03-01 | 南京华苏科技有限公司 | Detection method of image table based on MSER under natural photographing condition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106960208B (en) | Method and system for automatically segmenting and identifying instrument liquid crystal number | |
| CN104751142B (en) | A kind of natural scene Method for text detection based on stroke feature | |
| US8611662B2 (en) | Text detection using multi-layer connected components with histograms | |
| CN109344820B (en) | Digital ammeter reading identification method based on computer vision and deep learning | |
| CN108133216B (en) | Nixie tube reading identification method capable of realizing decimal point reading based on machine vision | |
| CN110647882A (en) | Image correction method, device, equipment and storage medium | |
| CN115690823B (en) | Table information extraction method and device with burr characteristics in electrical drawing | |
| CN110738030A (en) | Table reconstruction method and device, electronic equipment and storage medium | |
| CN112364834A (en) | Form identification restoration method based on deep learning and image processing | |
| CN115497109B (en) | Character and image preprocessing method based on intelligent translation | |
| CN112419207A (en) | Image correction method, device and system | |
| CN111626145A (en) | Simple and effective incomplete form identification and page-crossing splicing method | |
| CN113221778B (en) | Method and device for detecting and identifying handwritten form | |
| CN109271882B (en) | Method for extracting color-distinguished handwritten Chinese characters | |
| CN114463770A (en) | Intelligent question-cutting method for general test paper questions | |
| CN112800731A (en) | Table repairing method for dealing with distorted graphs in image table extraction | |
| CN110610163B (en) | Table extraction method and system based on ellipse fitting in natural scene | |
| CN112200053A (en) | Table identification method fusing local features | |
| CN112036294A (en) | Method and device for automatically identifying paper table structure | |
| CN114972309A (en) | Image-based table detection method for broken connection of table lines | |
| CN111611783A (en) | Positioning and dividing method and device for graphic table | |
| CN111539312A (en) | Method for extracting table from image | |
| CN115588208A (en) | Full-line table structure identification method based on digital image processing technology | |
| CN115171133A (en) | Table structure detection method for leveling irregular table image | |
| CN111814780A (en) | Bill image processing method, device and equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 224051 innovation center, 42 environmental protection Avenue, environmental protection science and Technology City, Tinghu District, Yancheng City, Jiangsu Province Applicant after: Jiangsu Kunlun Internet Technology Co.,Ltd. Address before: 224051 innovation center, 42 environmental protection Avenue, environmental protection science and Technology City, Tinghu District, Yancheng City, Jiangsu Province Applicant before: Kunyue Internet Environmental Technology (Jiangsu) Co.,Ltd. |
|
| CB02 | Change of applicant information |