CN114417792A - Processing method and device of form image, electronic equipment and medium - Google Patents

Abstract

The present invention provides a form image processing method, device, electronic device and medium. The method includes: determining a target form image as a borderless form, and processing the target form image through a form recognition structure to obtain a first border frame , adding the content in the target table image to the first border frame. The table image processing method provided by the present invention can quickly and effectively obtain the first border frame, accurately realize the conversion of the table image into an editable table, obtain more comprehensive and clear table information based on the table image itself, improve the processing speed of the table image, and improve the user experience. experience.

Description

Form image processing method, device, electronic device and medium

technical field

The present invention relates to the technical field of data processing, and in particular, to a method, device, electronic device and medium for processing table images.

Background technique

With the continuous development of office software, people's requirements for office software processing are getting higher and higher, and it is hoped that different application scenarios can be met while meeting normal office requirements.

At present, in practical applications, users may only get table images, such as table images searched on the Internet, but cannot get editable tables. deal with.

In the prior art, when a table image is converted into an editable table document, the table line is divided by image segmentation to obtain the horizontal and vertical lines of the table, and an editable table is generated. However, this processing method, There are defects such as poor segmentation effect of table images and incomplete lines, resulting in poor processing results and affecting user experience.

SUMMARY OF THE INVENTION

Based on the problems in the prior art, the present invention proposes a table line generation method, device, electronic device and medium for a borderless table, which can meet the situation of any form of table image, and has the advantages of improving table image processing effect and user experience. The advantages.

In a first aspect, the present invention provides a method for processing table images, including:

Determine the target table image as a borderless table;

The target table image is processed through the table recognition structure to obtain a first border frame;

Adding the content in the target table image to the first border frame.

Further, according to the method for processing table images provided by the present invention, the method further includes:

The table recognition structure is a line, a sliding window or a graph convolutional neural network model.

Further, according to the method for processing table images provided by the present invention, the method further includes:

When the table recognition structure is the line or the sliding window, the target table image is processed through the table recognition structure to obtain a first frame frame, including:

moving the table identification structure along the first direction of the target table image to record the first area not covering the content;

And/or, moving the form identification structure along the second direction of the target form image, and recording the second area that does not cover the content;

A table line is generated in at least one of the first area and the second area to obtain a first border frame.

Further, according to the method for processing table images provided by the present invention, the method further includes:

In the case where the table identification structure is the line, the table identification structure includes a first direction line and a second direction line;

The target table image is processed through the table recognition structure to obtain a first frame frame, including:

Based on the content distribution in the target form image, translating the first direction line and the second direction line in the target form image to determine a non-intersection area in the target form image;

A target point is determined in the non-intersection area, and a first frame frame is obtained by connecting the target points.

Further, according to the method for processing a table image provided by the present invention, the first direction line and the second direction line are translated in the target table image based on the content distribution in the target table image , determine the non-intersection area in the target table image, including:

Perform binarization processing on the target table image to obtain a first binary image; wherein, the first value in the first binary image is a pixel corresponding to the content distribution area in the target table image, and the The second value in the first binary image is the pixel corresponding to the non-content distribution area in the target table image;

Process the pixels in the first binary image according to the first direction to obtain lines in the first direction;

Process the pixels in the first binary image according to the second direction to obtain lines in the second direction;

Cross the lines in the first direction with the lines in the second direction, and take the intersected area not covered by the lines in the first direction or the lines in the second direction as the non-intersection in the target table image area.

Further, according to the processing method of the table image provided by the present invention, the determining the target point in the non-intersecting area includes:

Determine the contour of the non-intersecting area by contour searching;

According to the outline of the non-intersection area, a coordinate point of the non-intersection area is used as the target point.

Further, according to the method for processing table images provided by the present invention, the method further includes:

When the table identification structure is the sliding window, the table identification structure includes a first-direction sliding window and a second-direction sliding window;

The target table image is processed through the table recognition structure to obtain a first frame frame, including:

Based on the content distribution in the target table image, translate the sliding window in the first direction in the target table image to obtain a frame body in the first direction; translate the second direction in the target table image Sliding the window to get the second direction frame;

A first direction table line is set based on the first direction frame body, and a second direction table line is set based on the second direction frame body to obtain a first frame frame.

Further, according to the method for processing table images provided by the present invention, the method further includes:

When the table recognition structure is the graph convolutional neural network model, the first border frame is determined according to the positional relationship between the first text box and the second text box in the target table image.

Further, according to the form image processing method provided by the present invention, before the target form image is processed through the form recognition structure to obtain the first border frame, the method further includes:

Perform binarization processing on the target table image to obtain a second binary image; wherein, the second value in the second binary image is a pixel corresponding to a text character in the target table image, and the second value in the second binary image is a pixel corresponding to a text character in the target table image. The first value in the binary image is a pixel other than text characters in the target table image;

A table identification structure is set on the second binary image.

Further, according to the method for processing a table image provided by the present invention, the adding the content in the target table image into the first border frame includes:

Text detection is performed on the target table image, and a text box in the target table image is obtained according to the text detection result;

setting a text box in the first frame;

Fill in the content in the text box in the target table image into the text box in the first border frame.

In a second aspect, the present invention also provides a table image processing device, comprising:

A determination module for determining the target table image as a borderless table;

a processing module, configured to process the target table image through the table recognition structure to obtain a first border frame;

The adding module is used for adding the content in the target table image into the first border frame.

In a third aspect, the present invention also provides an electronic device, comprising: a processor, a memory and a bus, wherein,

The processor and the memory communicate with each other through the bus;

The memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the steps of the method for processing a table image as described in any one of the above.

In a fourth aspect, the present invention also provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the method for processing a table image according to any one of the above. step.

The present invention provides a form image processing method, device, electronic device and medium. The method includes: determining a target form image as a borderless form; processing the target form image through a form recognition structure to obtain a first border frame ; Add the content in the target table image to the first border frame. The table image processing method provided by the present invention can quickly and effectively obtain the first border frame, accurately realize the conversion of the table image into an editable table, obtain more comprehensive and clear table information based on the table image itself, and improve the processing speed of the table image. Improve user experience.

Description of drawings

In order to explain the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are the For some embodiments of the invention, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is the schematic flow chart of the table line generation method of the borderless table provided by the present invention;

2 is an exemplary diagram of a borderless table provided by the present invention;

3 is an exemplary diagram of a first binary image provided by the present invention;

4 is an exemplary diagram of a vertical table line area provided by the present invention;

5 is an exemplary diagram of a horizontal table line area provided by the present invention;

FIG. 6 is an exemplary diagram after the horizontal table line area and the vertical table line area provided by the present invention are superimposed;

7 is an exemplary diagram of the present invention determining the center point of the non-intersection area;

8 is an exemplary diagram obtained after the vertical center point is connected in the present invention;

9 is an exemplary diagram obtained after the horizontal upper center point is connected in the present invention;

FIG. 10 is an exemplary diagram of the present invention in which both the horizontal and vertical center points are connected;

FIG. 11 is an exemplary diagram of a text detection result provided by the present invention;

12 is an exemplary diagram of a second binary image provided by the present invention;

Figure 13 is an example diagram provided by the present invention after morphological processing;

14 is an example diagram after the text box provided by the present invention is split;

15 is a schematic structural diagram of a table line generation device for a borderless table provided by the present invention;

FIG. 16 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention. , not all examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

1 is a schematic flowchart of a method for processing a table image provided by an embodiment of the present invention. As shown in FIG. 1 , the method for processing a table image provided by the present invention includes the following steps:

Step 101: Determine that the target table image is a borderless table.

In this embodiment, the target table image needs to be determined and processed. When it is determined that the target table image is a borderless table, the table image processing method provided by the present invention is used to process the table lines in the borderless table, so as to realize the borderless table. Table line generation for the table.

It should be noted that the target form image refers to the form image to be processed by the method of the present invention. As the name suggests, a table image is a table with an image format, for example, a table is saved in an image format such as jpg, png, etc. A borderless table refers to a table with no table lines in all cells. For the table shown in Figure 2, all cells have no table lines and table borders, so it belongs to a borderless table. It should be noted that the borderless table may be a separate table image, or may be a table image formed by converting a table in a PDF document, which is not specifically limited here.

It should be noted that, in the borderless table shown in FIG. 2 , each cell is filled with text content, but in other embodiments, some cells in the borderless table may be blank. For such borderless tables, the method of the present invention can also be used to generate table lines.

Step 102: Process the target table image through the table recognition structure to obtain a first border frame.

In this embodiment, the first border frame refers to a frame formed by table lines obtained by restoring a borderless table. In this embodiment, the target table image needs to be processed through the table recognition structure to obtain the first border frame. The specific processing flow may be: using the table recognition result to process the target table image, and then determining the table line corresponding to the borderless table in the target table image according to the processing result to obtain the first border frame. The specific processing flow can be seen in the following Examples are not described in detail here.

It should be noted that there is no text data in the first border frame, but a frame composed of simple table lines, and the first border frame is identified and determined on the target table image, which can realize the table information in the target table image. Precise restoration.

Step 103: Add the content in the target table image to the first border frame.

In this embodiment, the content in the target table image needs to be added to the first border frame to realize the conversion between the target table image and the editable table. The first border frame is a table frame with table lines. Text recognition is performed on each cell in the target table image, and the text recognition result is filled into the corresponding cells of the first border frame to obtain an editable table.

It should be noted that, after the conversion of the target table image to an editable table with table lines is successfully implemented, the text results recognized by each cell in the target table image need to be filled into the cells corresponding to the first border frame to achieve Restoration of data within the target table image.

According to the form image processing method provided by the present invention, it is determined that the target form image is a borderless form, the target form image is processed through the form recognition structure to obtain a first border frame, and then the content in the target form image is added to the first border within the framework. The table image processing method provided by the present invention can quickly and effectively obtain the first border frame, accurately realize the conversion of the table image to an editable table, and obtain more comprehensive and clear table information based on the table image itself, which improves the processing of the table image. speed and improve user experience.

Based on any of the foregoing embodiments, in this embodiment, the method further includes: the table identification structure is a line, a sliding window, or a graph convolutional neural network model.

In this embodiment, the table recognition structure is a line, a sliding window, or a graph convolutional neural network model. The line refers to the use of horizontal or vertical lines to process the target table image, and the sliding window refers to the use of closed graphics of different shapes. The target table image is processed, and the graph convolutional neural network model recognizes and processes the target table image through a processing model obtained after training based on sample data.

It should be noted that the input of the graph convolutional neural network model is the position information of each text box (such as the coordinate information of the four points of the text box) and the target table image, and the output is a text box and its adjacent multiple texts. Whether the box is the information of the same row or the same column, please refer to the detailed introduction in the following embodiments for details.

According to the table image processing method provided by the present invention, the table recognition structure can be set as a line, a sliding window or a graph convolutional neural network model, which can realize various processing on the target table image, ensure the processing effect of the table image, and improve the user experience.

Based on any of the foregoing embodiments, in this embodiment, the method further includes: in the case that the table recognition structure is a line or a sliding window, processing the target table image through the table recognition structure to obtain a first border frame, including:

Move the table recognition structure along the first direction of the target table image, and record the first area of the uncovered content;

And/or, moving the form identification structure along the second direction of the target form image to record the second area of the uncovered content;

A table line is generated in at least one of the first area and the second area to obtain a first border frame.

In this embodiment, the table recognition structure needs to be moved along the first direction of the target table image to record the first area of the uncovered content; it is also necessary to move the table recognition structure along the second direction of the target table image, and then Record the second area of the uncovered content, determine an area from the first area or the second area to generate a table line, and obtain a first border frame. It should be noted that the first direction refers to the vertical direction of moving along the target table image, that is, moving downward from the top of the target table image, or moving from bottom to top; the second direction refers to the horizontal direction moving along the target table image , that is, move from the left side of the target table image to the right, or move from right to left; correspondingly, the first area is the area obtained by moving in the vertical direction, and the second area is the area obtained by moving in the horizontal direction.

It should be noted that, in other embodiments, the first direction and the second direction may also be an arc direction or a diameter direction in a fan shape, or a direction at an angle of 45 degrees, etc., and are not limited to the horizontal and vertical directions. Movement in any direction can be implemented according to the actual needs of the user, which is not specifically limited here.

It should be noted that, in this embodiment, the table identification structure that moves along the vertical or horizontal direction of the target table image may be a line or a sliding window, and the area to be recorded refers to the area that does not cover the content. The area refers to a blank area where no text data exists.

According to the processing method of the form image provided by the present invention, by moving the form identification structure along the first direction of the target form image, the first area of the uncovered content is recorded, and/or, the form identification structure is moved along the second direction of the target form image. Move in the direction, record the second area that does not cover the content, and then generate table lines in at least one area of the first area and the second area to obtain a first border frame, which can realize the generation of table lines in areas in different directions of the target table image. , to get the purpose of the first border frame and improve the speed of table image processing.

Based on any of the foregoing embodiments, in this embodiment, the method further includes: in the case that the table identification structure is a line, the table identification structure includes a first direction line and a second direction line;

The target table image is processed through the table recognition structure to obtain the first border frame, including:

Based on the content distribution in the target table image, the first direction line and the second direction line are translated in the target table image to determine the non-intersecting area in the target table image;

Target points are determined in each non-intersection area, and the target points are connected to obtain a first frame.

In this embodiment, the lines in the first direction are horizontal lines, and the lines in the second direction are vertical lines. In this embodiment, it is necessary to perform translation processing on the obtained horizontal lines and vertical lines based on the content distribution in the target table image to determine the non-intersecting areas in the target table image, and then determine the target points in each non-intersecting area, Connect all the obtained target points to get the first frame frame. The target point may be the center point, or other two-thirds points, etc., and the center point is preferred in this embodiment. It should be noted that the non-intersecting area refers to a blank area, an area that is neither distributed with text content nor covered by horizontal lines or vertical lines.

It should be noted that when the table recognition structure is a line, the text content in the target table image needs to be detected and processed to obtain multiple text boxes, and then the text boxes are processed accordingly to obtain the target point, and the first border frame is determined. .

It should be noted that, in this embodiment, by performing text detection processing on the target table image, multiple text boxes are obtained, and then the vertical table line area and the horizontal table line area in the target table image need to be determined according to the multiple text boxes, Then, perform translation processing on the obtained vertical table line area and horizontal table line area to obtain the table line area after the translation, determine multiple non-intersecting areas from the table line area, and then determine the center points of the multiple non-intersecting areas. , and generate a first frame frame corresponding to the target table image according to the multiple center points.

It should be noted that, in this embodiment, the vertical table line area refers to the area formed by vertically moving the table recognition structure along the target table image, that is, the second area in the above embodiment, and the horizontal table line area refers to the The area formed by the horizontal movement of the table recognition structure along the target table image, that is, the first area in the above-mentioned embodiment; the multiple non-intersecting areas refer to the area not covered by the horizontal table line area and the vertical table line area, and there is no superimposed area.

According to the form image processing method provided by the present invention, based on the text distribution in the target form image, the first direction lines and the second direction lines obtained by translation in the target form image are determined to determine the non-intersecting area in the target form image, and then The target points are determined in each non-intersection area, and the first frame frame is obtained by connecting the target points, the operation is simple, and the accuracy of the restoration processing of the table image can be ensured.

Based on any of the above embodiments, in this embodiment, based on the content distribution in the target form image, the lines in the first direction and the lines in the second direction are translated in the target form image to determine the non-intersecting area in the target form image, including :

Perform binarization processing on the target table image to obtain a first binary image; wherein, the first value in the first binary image is the pixel corresponding to the content distribution area in the target table image, and the first value in the first binary image is the pixel corresponding to the content distribution area in the target table image. The binary value is the pixel corresponding to the non-content distribution area in the target table image;

Process the pixels in the first binary image according to the first direction to obtain lines in the first direction;

Process the pixels in the first binary image according to the second direction to obtain lines in the second direction;

Cross the lines in the first direction with the lines in the second direction, and take the intersected area not covered by the lines in the first direction or the lines in the second direction as a non-intersection area in the target table image.

In this embodiment, the target table image after the text box is determined needs to be binarized to obtain a first binary image in which the pixels corresponding to the text box are the first value and the pixels other than the text box are the second value , the first binary image shown in Figure 3. The first value refers to a value with a pixel of 0, the second value refers to a value with a pixel of 255, and the first binary image refers to a binary image generated by binarizing the text box area.

In this embodiment, the first direction may refer to the vertical direction of the target table image. For the determination of vertical lines, a pixel column with a certain width needs to be selected, and the pixel column and the text box are subjected to a closing operation, and the text in the vertical direction needs to be closed. The pixels corresponding to the boxes are connected to form vertical lines, and a vertical table line area is obtained, as shown in FIG. 4 , where the pixel value corresponding to the vertical table line is 0. It should be noted that the closing operation is a morphological calculation method, specifically the calculation method of dilation first and then erosion, wherein dilation is to merge all the background points in contact with the object into the object, so that the boundary expands to the outside. The process used to fill holes in objects. Erosion is a process of eliminating boundary points and shrinking the boundary inward, which can be used to eliminate small and meaningless objects.

In this embodiment, the second direction may refer to the horizontal direction of the target table image. For the determination of horizontal lines, select pixel rows of a certain width and perform a closing operation on them with the text box to obtain horizontal lines. All horizontal lines form horizontal lines. Table area, such as the horizontal table line area shown in Figure 5.

It should be noted that, in this embodiment, it is necessary to perform translation processing on the obtained horizontal table line area and vertical table line area to obtain the translated table line area. The translated table line area can be shown in Figure 6. From Figure 6, it can be determined that the area that is not superimposed, not covered by the vertical table line area or the horizontal table line area is determined as a non-intersecting area, that is, Figure 6 The blank area shown in , that is, the blank area is a non-intersecting area.

According to the table image processing method provided by the present invention, the target table image is subjected to binarization to obtain a first binary image, and then the pixels in the first binary image are processed according to the first direction to obtain the first direction. Lines, process the pixels in the first binary image according to the second direction to obtain lines in the second direction, translate the lines in the first direction and the lines in the second direction, and convert the lines that are not intersected, not in the first direction or in the first direction. The area covered by the lines in the second direction serves as a non-intersecting area in the target form image, which improves the processing speed of the form image.

Based on any of the foregoing embodiments, in this embodiment, determining target points in each non-intersecting area includes:

Determine the contour of each non-intersecting area by contour search;

According to the outline of each non-intersecting area, the target point in each non-intersecting area is determined.

In this embodiment, the coordinates of the target points of each non-intersection area need to be determined by means of contour search. First determine the contours of the multiple non-intersecting areas, then determine the coordinates of the respective contour target points of the multiple non-intersecting areas according to the contours of the multiple non-intersecting areas, and obtain the first frame frame according to the coordinate information of the target points. It should be noted that the target point in this embodiment is set as the center point. Since the text box is generally a quadrilateral, the center point of the text box can be determined by determining the coordinates of the four vertices, as shown in FIG. The circle is the center point of each text box determined.

For example, to obtain a non-intersection area, find the coordinates of the four vertices of the non-intersection area by contour search, which are (2, 6) and (2, 9), (8, 6) and (8, 9), calculate and obtain (5, 7.5) as the coordinates of the center point, and then connect the determined multiple center points through the above search method to obtain the table line corresponding to the target table image.

It should be noted that, connecting the center points in the longitudinal direction can obtain the area formed by each vertical line as shown in FIG. In the area formed by the line, the obtained vertical connecting line area and horizontal connecting line area are translated to obtain the table line shown in FIG. 10 , that is, the first border frame of the borderless table is obtained.

It should be noted that, in this embodiment, the first border frame is generated on the original target table image. In other embodiments, according to the determined coordinates of the center point, the mapping points of a plurality of center points may be connected vertically and/or horizontally in a preset area, and all the mapping points may be connected to generate the corresponding mapping points of the target table image. form line.

According to the table image processing method provided by the present invention, the contour of each non-intersecting area is determined by contour search, and then the target point in each non-intersecting area is determined according to the contour of each non-intersecting area. The first border frame of the target table image can be generated by means of the center point, thereby improving the accuracy of generating the first border frame.

Based on any of the above embodiments, in this embodiment, in the case that the table identification structure is a sliding window, the table identification structure includes a first-direction sliding window and a second-direction sliding window;

The target table image is processed through the table recognition structure to obtain the first border frame, including:

Based on the content distribution in the target table image, translate the sliding window in the first direction in the target table image to obtain the frame body in the first direction; translate the sliding window in the second direction in the target table image to obtain the frame body in the second direction;

A first-direction table line is set based on the first-direction frame body, and a second-direction table line is set based on the second-direction frame body to obtain a first frame frame.

In this embodiment, it is necessary to translate the first sliding window and the second sliding window based on the distribution of the text content in the target table image to obtain the first direction frame body and the second direction frame body, and then set the first direction frame body based on the first direction frame body. A table line in one direction, a table line in a second direction is set based on the frame body in the second direction, and a first border frame is determined according to the table line obtained after translation.

It should be noted that, in this embodiment, the frame body in the first direction may be a frame body along the longitudinal direction of the target table image, and may be a frame body of any shape, such as a rectangle, a square, etc.; the frame body in the second direction is a frame body along the The frame body in the horizontal direction of the target form image can also be a frame body of any shape; in other embodiments, the frame body in the first direction and the frame body in the second direction can also be frames in any direction on the target form image, For example, the frame body in the 45-degree direction can be set according to the actual needs of the user, which is not specifically limited here.

It should be noted that the frame body in the first direction is a frame body obtained by sliding the sliding window along the first direction of the target table image according to the first direction; the frame body in the second direction is a frame body obtained by sliding the window along the target table image according to the second sliding window. The frame body obtained by sliding in the second direction; the frame body in the first direction and the frame body in the second direction refer to a blank area without text data content in this embodiment, that is, the frame body in the first direction and the frame body in the second direction The first border frame obtained after the direction frame is translated is an area without text data content.

It should be noted that the first direction frame body and the second direction frame body obtained in this embodiment can generate table lines, the first direction table line is set based on the first direction frame body, and the first direction frame body is set based on the second direction frame body. The two-direction table line can accurately obtain the first border frame after completing the generation of the table lines of the first-direction frame body and the second-direction frame body.

According to the form image processing method provided by the present invention, based on the content distribution in the target form image, the sliding window in the first direction is translated in the target form image to obtain the frame body in the first direction, and the sliding window in the second direction is translated and slid in the target form image window, obtain the second direction frame body, set the first direction table line based on the first direction frame body, set the second direction table line based on the second direction frame body, obtain the first frame frame, can accurately obtain the first frame frame, improve Processing speed of table images.

Based on any of the above embodiments, in this embodiment, based on the content distribution in the target form image, the sliding window in the first direction is translated in the target form image to obtain the frame body in the first direction; Sliding the window in the second direction to obtain the frame in the second direction, including:

In the process of sliding the sliding window in the first direction along the direction perpendicular to the first direction, according to the content distribution in the target table image, determine the sliding range in which the sliding window in the first direction does not intersect with the content, and slide the window according to the first direction. The shape of the first-direction sliding window and the sliding range where the first-direction sliding window does not intersect with the content determine the first-direction frame;

In the process of sliding the sliding window in the second direction along the direction perpendicular to the second direction, according to the text distribution in the target table image, determine the sliding range of the sliding window in the second direction that does not intersect with the content, and slide the window according to the second direction. The shape of the second direction and the sliding range of the second direction sliding window that does not intersect with the content determine the second direction frame.

In this embodiment, a first-direction sliding window needs to be set on the target table image, and when the first-direction sliding window slides in a direction perpendicular to the first direction, according to the distribution of text content in the target table image, determine The first-direction sliding window is a sliding range that does not intersect with the text content, and then the first-direction frame is determined according to the shape of the first-direction sliding window and the sliding range. The sliding window in the first direction refers to a sliding window set in the longitudinal direction of the target table image, which may be a rectangular sliding window or a square sliding window, which is not specifically limited herein.

It should be noted that in this embodiment, the first direction is the longitudinal direction of the target table image. When the sliding window in the first direction is set to slide along the direction perpendicular to the first direction, the determined sliding range that does not intersect with the text is is the range area in the lateral direction, that is, the obtained first-direction frame body is the frame body in the lateral direction. In other embodiments, the sliding window in the first direction can also be slid along the first direction, and the obtained first-direction frame is a vertical frame, which can be set according to the actual needs of the user, here There is no specific limitation.

In this embodiment, a second-direction sliding window needs to be set on the target table image. When the second-direction sliding window slides in a direction perpendicular to the second direction, according to the distribution of text content in the target table image, determine The second-direction frame is determined according to the shape of the second-direction sliding window and the sliding range where the second-direction sliding window does not intersect with the text content. The sliding window in the second direction refers to a sliding window set in the horizontal direction of the target table image, which may be a rectangular sliding window or a square sliding window, which is not specifically limited herein.

It should be noted that in this embodiment, the second direction is the horizontal direction of the target table image. When the sliding window in the second direction is set to slide along the direction perpendicular to the second direction, the determined sliding range that does not intersect with the text content Then it is the range area in the longitudinal direction, that is, the obtained frame body in the second direction is the frame body in the longitudinal direction. In other embodiments, the sliding window in the second direction can also be slid along the second direction, and the obtained frame body in the second direction is the frame body in the horizontal direction, which can be set according to the actual needs of the user, here There is no specific limitation.

It should be noted that the settings of the sliding window in the first direction and the sliding window in the second direction can be calculated and designed by using a relatively mature sliding window algorithm in the prior art, and the specific setting method will not be described in detail here.

According to the table image processing method provided by the present invention, by setting a sliding window on the target table image, and then determining the sliding range that does not intersect with the text content during the sliding process of the sliding window, and determining the sliding window according to the shape and sliding range of the sliding window. The first-direction frame body and the second-direction frame body are simple to operate and improve the processing speed of table images.

Based on any of the above embodiments, in this embodiment, when the table recognition structure is a graph convolutional neural network model, the first border frame is determined according to the positional relationship between the text box in the target table image and the text box .

In this embodiment, when the table recognition structure is a graph convolutional neural network model, it is necessary to determine the first frame frame according to the positional relationship between each text box in the target table image. It should be noted that each text box The positional relationship between them can be confirmed according to the coordinate information of the center point of the text box.

It should be noted that, according to the output of the graph convolutional neural network model provided in this embodiment, the text box in the same row and the text box in the same column can be determined, and according to the determined height value of the text box in the same row can be determined. The height of the cell is determined, and the width value of the text box in the same column determines the width of the cell, that is, the table line of the borderless table is drawn, and the detailed description can be found in the following embodiments.

According to the method for processing a table image provided by the present invention, in the case that the table recognition structure is a graph convolutional neural network model, the first border frame is determined according to the positional relationship between the text box and the text box in the target table image, and the The trained graph convolutional neural network model can accurately and quickly determine the positional relationship of each text box in the target table image, and obtain the first frame frame.

Based on any of the above embodiments, in this embodiment, when the table recognition structure is a graph convolutional neural network model, the first border frame is determined according to the positional relationship between the text box in the target table image and the text box ,include:

Input the pre-acquired position information of multiple text boxes in the target table image and the target table image into the pre-trained graph convolutional neural network model, and obtain the relationship between any text box in the multiple text boxes and its adjacent text boxes. Location relationship; wherein, the graph convolutional neural network model is trained based on the location information of multiple text boxes in the sample table, the image of the sample table, and the label information of the sample table;

Determine the text box of each row and the text box of each column in the target table image according to the positional relationship between any text box in the plurality of text boxes and its adjacent text boxes;

A first border frame of the target table image is generated according to the text boxes of each row and the text boxes of each column in the target table image.

In this embodiment, it is necessary to input the obtained position information of multiple text boxes in the target table image and the target table image into the pre-trained graph convolutional neural network model, and obtain the relationship between any text box and its adjacent text boxes. Then, according to the obtained position relationship, the text boxes of each row and the text boxes of each column in the target table image are determined, and the first border frame corresponding to the target table image is generated. The graph convolutional neural network model is an artificial intelligence model, which is obtained by training based on the position information of multiple text boxes in the sample table, the image of the sample table, and the label information of the sample table.

In this embodiment, the input of the graph convolutional neural network model is the position information of each text box (such as the coordinate information of the four points of the text box) and the target table image, and the output is a certain text box and its adjacent multiple Whether the textbox is the same row or the same column of information.

For example, if the determined position information of the text box a is (1, 1), the position information of the adjacent text box is calculated and analyzed as: a1(0,1), a2(0,1), a3( 1,0), a4(1,0). Among them, the first number in the position information indicates whether it is in the same row as the text box a. If 1 indicates the same row, and 0 indicates a different row, then the adjacent text boxes a1 and a2 are in the same column as the text box a; the second number indicates the same column as the text box a. Whether box a is in the same column, if 1 means the same column, 0 means different columns, adjacent text boxes a1, a2 are in the same column as text box a, and adjacent text boxes a3, a4 are in the same row as text box a. Specifically, it can be set according to the actual needs of the user, which is not specifically limited here.

It should be noted that, according to the output of the graph convolutional neural network model provided in this embodiment, the text box in the same row and the text box in the same column can be determined, and according to the determined height value of the text box in the same row can be determined. The height of the cell is determined, and the width value of the text box in the same column determines the width of the cell, that is, the table line of the borderless table is drawn.

According to the method for processing table images provided by the present invention, by inputting the position information of multiple text boxes and target table images into a pre-trained rare graph convolutional neural network model, the relationship between any text box and its adjacent text boxes is obtained. and then determine the text row of each row and the text box of each column according to the position relationship, and generate the first border frame corresponding to the target table image according to the determined text box of each row and column. The first border frame corresponding to the target table image is generated by the positional relationship of the multiple text boxes, which can improve the accuracy of table line generation.

Based on any of the above embodiments, in this embodiment, according to the text boxes of each row and the text boxes of each column in the target table image, the first border frame of the target table image is generated, including:

According to the height target value of each text box in the target row of the target table image, the height value of the target row is set; wherein, the target row is any row of the target table image;

According to the width target value of each text box in the target column of the target table image, the width value of the target column is set; wherein, the target column is any column of the target table image;

Draw the table line of the target row according to the height value of the target row and the position of the target row in the target table image; and/or, according to the width value of the target column and the position of the target column in the target table image, draw the table of the target column line to get the first border frame of the target table image.

In this embodiment, it is necessary to set the height value of the target row according to the height target value of each text box in the target row, set the width value of the target column according to the width target value of each text box in the target column, and then according to the target row The height value of the target row and the position information of the target row in the target table image, draw the table line of the row corresponding to the target row, and draw the target row according to the width value of the target column and the position information of the target column in the target table image. For the table line of the column corresponding to the column, the first border frame is obtained according to the table line generated by each row and each column.

It should be noted that, in this embodiment, the maximum height value of each text box in the target row is determined as the height value of the target row, or the maximum width corresponding to each text box in the target column is set as the width of the target column. In other embodiments, the average height value of each text box in the target row can also be selected as the height value of the target row, and the average width value of each text box in the target column can be selected as the width value of the target column. Specific users can also set according to actual needs, which is not specifically limited here.

It should be noted that the position information of the target row or target column may be its coordinate information, or may refer to the positional relationship with any other adjacent row or column, which can be set according to actual needs, which is not specifically limited here. .

For example, if the height values of multiple text boxes in the target row are 4cm, 3cm, and 5cm, respectively, the maximum height value of the multiple text boxes in the target row is determined as the height value of the target row, that is, 5cm is determined as the height value of the target row. The height value of the target row; similarly, the maximum width value in the target column is determined as the width value of the target column, and the above method is used to confirm the height value or width value for other rows or columns, which will not be described in detail here. .

It should be noted that the pixels of each text box can also be used as the unit of measurement to determine the height value of the target row or the width value of the target column. For example, the maximum pixel of the target text box in a target row is 1024. If the value corresponding to the pixel is 8.67cm, then the 8.67cm corresponding to the pixel is determined as the height value of the target row. In other embodiments, other numerical determination methods may also be adopted. Different determination methods can be selected according to the specific needs of the user, which is not specifically limited here.

According to the table image processing method provided by the present invention, the height target value in each text box in the target row is determined as the height value of the target row, and the width target value of each text box in the target column is determined as the width value of the target column, Then according to the height value of the target row and the position of the target row in the target table image, the width value of the target column, and the position of the target column in the target table image, draw the table line corresponding to the target row or target column, and get the first The border frame can ensure the accuracy of table line generation, so that the generated table line is complete and clear.

Based on any of the foregoing embodiments, in this embodiment, before the target table image is processed through the table recognition structure to obtain the first border frame, the method further includes:

The target table image is binarized to obtain a second binary image; wherein, the second value in the second binary image is the pixel corresponding to the text character in the target table image, and the second value in the second binary image is the pixel corresponding to the text character in the target table image. One value is the pixels other than text characters in the target table image.

Set the table recognition structure for the second binary image.

In this embodiment, the target table image containing the text detection result needs to be binarized to obtain a second binary image in which the pixels corresponding to the text characters are of the second value and the pixels other than the text characters are of the first value, Wherein, the first value refers to the value where the pixel is 0, and the second value refers to the value where the pixel is 255. Then, perform morphological processing on the obtained second binary image to obtain a morphological processing result. It is determined that there is a text box with an interval in the processing result, and the text box is split and processed to obtain multiple texts in the target table image. frame.

It should be noted that, in this embodiment, a table recognition structure also needs to be set for the obtained second binary graph, wherein the table recognition structure may be a line, a frame or a graph convolutional neural network model, which may be specifically performed according to the needs of the user. The setting is not specifically limited here.

According to the form image processing method provided by the present invention, by performing binarization processing on the target form image containing the text detection result, the pixels corresponding to the text characters are obtained as the first value, and the pixels other than the text characters in the target form image are obtained. A table identification structure is set for the second binary image of the second value and then for the second binary image, which provides support for the subsequent determination of the first border frame according to the table identification structure.

Based on any of the above embodiments, in this embodiment, adding the content in the target table image to the first border frame includes:

Text detection is performed on the target table image, and a text box in the target table image is obtained according to the text detection result;

setting a text box in the first frame;

Fill in the content in the text box in the target table image into the text box in the first border frame.

In this embodiment, it is necessary to perform text detection on the target table image, and then obtain a text box in the target table image according to the text detection result, set a plurality of text boxes in the first border frame, and combine the plurality of text boxes in the target table image The data content existing in the box is filled in the corresponding text box in the first border frame, so as to realize the restoration of the data in the target table image.

In this embodiment, as shown in FIG. 11 , a text detection unit is used to perform text detection processing on a borderless table, multiple text contents are identified, and multiple text boxes in the target table image are determined according to the file detection result, wherein, A rectangular box represents a text box, and there are multiple text boxes in Figure 11.

Specifically, after detecting the text in the target table image, it is necessary to perform binarization and morphological processing on the text detection result, so as to obtain the text box in the target table image. Among them, image binarization, that is, ImageBinarization, is to set the gray value of the pixel on the image to 0 or 255, which is actually the process of presenting the entire image with an obvious black and white effect. Figure 12 shows the above text detection results. Schematic diagram after binarization of multiple text boxes. Morphology, mathematical morphology, or mathematical Morphology, is one of the most widely used techniques in image processing. It is mainly used to extract image components that are meaningful for expressing and describing the shape of an area from an image, so that subsequent recognition work can capture The most essential shape feature of the target object. Among them, the most essential refers to the most discriminative ability, that is, the most discriminative, such as boundaries and connected areas, as shown in Figure 13. Figure 13 shows the morphology of the table image after binarization processing. Process the resulting schematic. The specific processing process is shown in the following embodiments, which will not be described in detail here.

It should be noted that in this embodiment, after multiple text boxes in the target table image are determined, the text boxes with gaps in the middle are split, as shown in FIG. 14 after splitting the text boxes with gaps in the middle. For example, the single text box "General Manager Zhang San 123456937" is divided into three text boxes according to the existing interval, namely "Zhang San", "General Manager" and "123456937".

In this embodiment, it is necessary to perform binarization processing on the plurality of text boxes obtained after splitting, confirm processing of the horizontal table line area and the vertical table line area, and then determine each non-intersecting area according to the determined table line area. The center point is connected to each center point to determine the first frame frame corresponding to the target table image. The details can be found in the following embodiments, which will not be described in detail in this embodiment.

It should be noted that, in this embodiment, the horizontal table area and the vertical table area are determined, and then the center point of the translated table line area is determined, and the table line corresponding to the target table image is generated by connecting the center points. In other embodiments, other generating manners may be employed, such as generating table lines based on the positional relationship of text boxes, or other generating manners, which are not specifically limited herein.

It should be noted that, in this embodiment, multiple text boxes need to be set in the obtained first border frame, and then the text content existing in the multiple text boxes in the target table image is added to the corresponding text boxes in the first border frame. In the text box, wherein the more mature text recognition method in the prior art is used to recognize the text content in the multiple text boxes in the target table image, and then the recognition results are filled into the corresponding text boxes in the first border frame , the specific process of text recognition processing will not be described in detail here.

According to the form image processing method provided by the present invention, by performing text detection on the target form image, a text frame in the target form image is obtained according to the text detection result, and then a text frame is set in the first border frame, and the text frame in the target form image is set. The content in the text box is filled into the text box of the first border frame. The accurate restoration of the text data in the table image can be realized, and the processing speed of the table image can be improved.

Based on any of the foregoing embodiments, in this embodiment, determining that the target table image is a borderless table includes:

Input the target table image into the pre-trained table classification model;

According to the output result of the table classification model, determine that the target table image is a borderless table;

Among them, the table classification model is trained based on the sample table and the category label of the sample table.

In this embodiment, the target table image needs to be input into the pre-trained table classification model, and then the target table image is determined to be a borderless table according to the output result of the table classification model, wherein the table classification model is based on the sample table, the sample table There are two output results obtained from category label training, one is a borderless table, and the other is a bordered table. Only when there is no table line in each cell in the target table image, can the target table image be determined as Borderless table.

It should be noted that the table classification model may be a VGG model or a Resnet model, and a table classification model is obtained by training the selected model, which is used to identify the type of the target table image. Among them, the VGG model is a model that can be recognized and confirmed in multiple transfer learning tasks. To extract CNN features from images, the VGG model is the preferred algorithm. Resnet model refers to the deep residual network model, which realizes the ability of image recognition and speech recognition with very high accuracy through a very deep network.

For example, if the obtained target table image 1 and target table image 2 are input into the pre-trained table classification model, the model output result is that each cell in the target table image 1 does not have border lines, and the target table image 2 does not have border lines. There is a table frame line in the middle part, and the target table image 1 is determined as a borderless table, and the target table image 2 is determined as a half-border table.

According to the table image processing method provided by the present invention, the type of the target table image can be determined through a pre-trained table classification model, and the target table image is determined as a borderless table according to the output result of the table classification model, so that the target can be accurately identified. The type of table image, which improves the processing speed of subsequent borderless tables.

Based on any of the foregoing embodiments, in this embodiment, the content in the first border frame is in an editable state.

In this embodiment, the content in the first border frame obtained by converting the target table image is in an editable state. According to the table image processing method provided above, the target table image whose type is a borderless table is converted to obtain a first border frame with table lines, and then text recognition needs to be performed on each cell in the target table image, and The results are filled into the corresponding cells of the first border frame, resulting in an editable table.

According to the table image processing method provided by the present invention, the obtained content in the first frame frame is in an editable state, which satisfies the user's editing requirements and improves the user experience.

Based on any of the above embodiments, in this embodiment, the table classification model is used to first confirm that the target table image is a borderless table, and then the borderless table is subjected to text detection processing according to the text detection unit to obtain a plurality of text boxes. The target table image with multiple text boxes is subjected to binarization processing and morphological processing, and then according to the processing results, the text boxes with intermediate intervals are split and processed, and the split target table with multiple text boxes is obtained. image.

In this embodiment, it is also necessary to perform table line generation processing according to multiple text boxes of the target table image, first perform binarization processing to obtain a first binary image, and extract from the first binary image in the vertical direction. For the vertical table line area, the horizontal table line area is extracted according to the horizontal direction, wherein the vertical table line and the horizontal table line are given templates. Then, the two obtained regions are translated, and the white region on the translated region is set as a non-intersecting region.

In this embodiment, the contour of each non-intersecting area is determined from the translated area by means of contour searching, the center point of the contour is calculated, and the coordinate information of the non-intersecting area is determined, according to the coordinate values of the vertices in each direction. The average determines the coordinates of the center point. Connect the center points in the horizontal and vertical directions to generate the table line of the target table image, and obtain the editable first border frame.

It should be noted that, in this embodiment, the table line may also be determined by using the text box and the positional relationship between the text boxes. For example, according to the positional relationship between the text boxes, a graph convolution network model is constructed, and the trained graph convolution network model is used to determine the text boxes in the same column and the text boxes in the same row. The position relationship is determined, and the table line that completely surrounds all the text boxes in the target table image is determined.

Figure 15 is a table image processing device provided by the present invention. As shown in Figure 15, the table image processing device provided by the present invention includes:

A determination module 1501, configured to determine that the target table image is a borderless table;

a processing module 1502, configured to process the target table image through the table recognition structure to obtain a first border frame;

The adding module 1503 is configured to add the content in the target table image into the first border frame.

According to the table image processing device provided by the present invention, it is determined that the target table image is a borderless table, the target table image is processed through the table recognition structure to obtain a first border frame, and then the content in the target table image is added to the first border within the framework. The table image processing method provided by the present invention can quickly and effectively obtain the first border frame, accurately realize the conversion of the table image to an editable table, and obtain more comprehensive and clear table information based on the table image itself, which improves the processing of the table image. speed and improve user experience.

Further, the table recognition structure is a line, a sliding window or a graph convolutional neural network model.

According to the table image processing device provided by the present invention, the table recognition structure can be set as a line, a sliding window or a graph convolutional neural network model, which can realize various processing of the target table image, ensure the processing effect of the table image, and improve the user experience.

Further, the processing module 1502 is also used for:

moving the table identification structure along the first direction of the target table image to record the first area of the uncovered content;

And/or, moving the form identification structure along the second direction of the target form image to record the second area of the uncovered content;

A table line is generated in at least one of the first area and the second area to obtain a first border frame.

According to the form image processing device provided by the present invention, by moving the form identification structure along the first direction of the target form image, the first area of the uncovered content is recorded, and/or, the form identification structure is moved along the second direction of the target form image. Move in the direction, record the second area that does not cover the content, and then generate table lines in at least one area of the first area and the second area to obtain a first border frame, which can realize the generation of table lines in areas in different directions of the target table image. , to get the purpose of the first border frame and improve the speed of table image processing.

Further, in the case where the table identification structure is a line, the table identification structure includes horizontal lines and vertical lines, and the processing module 1502 is further configured to:

Based on the content distribution in the target table image, the first direction line and the second direction line are translated in the target table image to determine the non-intersecting area in the target table image;

Determine the target points in the non-intersection area, and connect the target points to obtain the first frame.

According to the table image processing device provided by the present invention, based on the text distribution in the target table image, the first direction lines and the second direction lines obtained by superimposing in the target table image are determined to determine the non-intersecting area in the target table image, and then The target points are determined in each non-intersection area, and the first frame frame is obtained by connecting the target points, the operation is simple, and the accuracy of the restoration processing of the table image can be ensured.

Further, the processing module 1502 is also used for:

Perform binarization processing on the target table image to obtain a first binary image; wherein, the first value in the first binary image is the pixel corresponding to the content distribution area in the target table image, and the first value in the first binary image is the pixel corresponding to the content distribution area in the target table image. The second value in the binary image is the pixel corresponding to the non-content distribution area in the target table image;

Process the pixels in the first binary image according to the first direction to obtain lines in the first direction;

Process the pixels in the first binary image according to the second direction to obtain lines in the second direction;

Cross the lines in the first direction with the lines in the second direction, and take the intersected area not covered by the lines in the first direction or the lines in the second direction as the non-intersection in the target table image area.

According to the table image processing device provided by the present invention, the target table image is binarized to obtain a first binary image, and then the pixels in the first binary image are processed according to the first direction to obtain the first direction. Lines, process the pixels in the first binary image according to the second direction to obtain lines in the second direction, cross the lines in the first direction with the lines in the second direction, and cross the lines that are not in the first direction or the lines in the second direction. The area covered by the two-direction lines is regarded as the non-intersecting area in the target table image, which improves the processing speed of the table image.

Further, the processing module 1502 is also used for:

Determine the contour of the non-intersecting area by contour searching;

According to the outline of the non-intersection area, a coordinate point of the non-intersection area is used as the target point.

According to the table image processing device provided by the present invention, the contour of each non-overlapping area is determined by contour search, and then the target point in each non-intersecting area is determined according to the contour of each non-intersecting area. The first border frame of the target table image can be generated by means of the center point, thereby improving the accuracy of generating the first border frame.

Further, in the case that the table identification structure is a frame body, the table identification structure includes a first direction frame body and a second direction frame body;

Correspondingly, the processing module 1502 is also used for:

Based on the content distribution in the target table image, translate the sliding window in the first direction in the target table image to obtain the frame body in the first direction; translate the sliding window in the second direction in the target table image to obtain the frame body in the second direction;

A first-direction table line is set based on the first-direction frame body, and a second-direction table line is set based on the second-direction frame body to obtain a first frame frame.

According to the table image processing device provided by the present invention, based on the content distribution in the target table image, the sliding window in the first direction is translated in the target table image to obtain the frame body in the first direction; the sliding window in the second direction is translated and slid in the target table image window to obtain the second direction frame body; set the first direction table line based on the first direction frame body, set the second direction table line based on the second direction frame body, obtain the first frame frame, can accurately obtain the first frame frame, improve the Processing speed of table images.

Further, the processing module 1502 is also used for:

In the process of sliding the sliding window in the first direction along the direction perpendicular to the first direction, according to the content distribution in the target table image, determine the sliding range in which the sliding window in the first direction does not intersect with the text, according to the first direction sliding window The shape of the sliding window in one direction and the sliding range in which the first sliding window does not intersect with the text determine the frame body in the first direction;

In the process of sliding the sliding window in the second direction along the direction perpendicular to the second direction, according to the text distribution in the target table image, determine the sliding range of the sliding window in the second direction that does not intersect with the text, according to the second direction The shape of the sliding window and the sliding range in which the second sliding window does not intersect with the text determine the frame body of the second direction.

According to the table image processing device provided by the present invention, by setting a sliding window on the target table image, and then determining the sliding range that does not intersect with the text content during the sliding process of the sliding window, and determining the sliding window according to the shape and sliding range of the sliding window. The first-direction frame body and the second-direction frame body are simple to operate and improve the processing speed of table images.

Further, when the table recognition structure is a graph convolutional neural network model, the first frame frame is determined according to the positional relationship between the text box and the text box in the target table image.

According to the table image processing device provided by the present invention, when the table recognition structure is a graph convolutional neural network model, the first border frame is determined according to the positional relationship between the text box and the text box in the target table image, and the The trained graph convolutional neural network model can accurately and quickly determine the positional relationship of each text box in the target table image, and obtain the first frame frame.

Further, the processing module 1502 is also used for:

Inputting the pre-acquired position information of multiple text boxes in the target table image and the target table image into a pre-trained graph convolutional neural network model to obtain any text box and its adjacent text in the multiple text boxes The positional relationship between the frames; wherein, the graph convolutional neural network model is obtained by training based on the position information of a plurality of text frames in the sample table, the sample table image and the label information of the sample table;

Determine the text box of each row and the text box of each column in the target table image according to the positional relationship between any text box in the plurality of text boxes and its adjacent text boxes;

A first border frame of the target table image is generated according to the text boxes of each row and the text boxes of each column in the target table image.

According to the table image processing device provided by the present invention, by inputting the position information of a plurality of text boxes and target table images into a pre-trained rare graph convolutional neural network model, the relationship between any text box and its adjacent text boxes is obtained. and then determine the text row of each row and the text box of each column according to the position relationship, and generate the first border frame corresponding to the target table image according to the determined text box of each row and column. The first border frame corresponding to the target table image is generated by the positional relationship of the multiple text boxes, which can improve the accuracy of table line generation.

Further, the processing module 1502 is also used for:

According to the height target value of each text box in the target row of the target table image, the height value of the target row is set; wherein, the target row is any row of the target table image;

According to the width target value of each text box in the target column of the target table image, the width value of the target column is set; wherein, the target column is any column of the target table image;

Drawing the table line of the target row according to the height value of the target row and the position of the target row in the target table image; and/or, according to the width value of the target column and the target column in the target table image position, draw the table line of the target column, and obtain the first border frame of the target table image.

According to the table image processing device provided by the present invention, the height target value of each text box in the target row is determined as the height value of the target row, and the width target value of each text box in the target column is determined as the width value of the target column, Then according to the height value of the target row and the position of the target row in the target table image, the width value of the target column, and the position of the target column in the target table image, draw the table line corresponding to the target row or target column, and get the first The border frame can ensure the accuracy of table line generation, so that the generated table line is complete and clear.

Further, the processing device of the table image is also used for:

Perform binarization processing on the target table image to obtain a second binary image; wherein, the second value in the second binary image is the pixel corresponding to the text character in the target table image, and the second binary value is the pixel corresponding to the text character in the target table image. The first value in the figure is a pixel other than text characters in the target table image.

A table identification structure is set on the second binary image.

According to the table image processing device provided by the present invention, by performing binarization processing on the target table image containing the text detection result, the pixels corresponding to the text characters are obtained as the first value, and the pixels other than the text characters in the target table image are obtained. For the second binary image of the second value, and then set the table recognition structure for the second binary image, which provides support for the subsequent determination of the first border frame according to the table recognition structure

Further, the adding module 1503 is also used for:

Text detection is performed on the target table image, and a text box in the target table image is obtained according to the text detection result;

setting a text box in the first frame;

Fill in the content in the text box in the target table image into the text box in the first border frame.

According to the table image processing device provided by the present invention, by performing text detection on the target table image, multiple text boxes in the target table image are obtained according to the text detection results, a plurality of text boxes are arranged in the first frame frame, and the target table The contents of the multiple text boxes in the image are filled into the multiple text boxes of the first border frame. The accurate restoration of the text data in the table image can be realized, and the processing speed of the table image can be improved.

Further, the determining module 1501 is also used for:

Input the target table image into the pre-trained table classification model;

According to the output result of the table classification model, determine that the target table image is a borderless table;

Wherein, the table classification model is obtained by training based on the sample table and the category label of the sample table.

According to the table image processing device provided by the present invention, the type of the target table image can be determined through the pre-trained table classification model, and the target table image can be determined as a borderless table according to the output result of the table classification model, so that the target can be accurately identified. The type of table image, which improves the processing speed of subsequent borderless tables.

Further, the content in the first border frame is in an editable state.

According to the table image processing device provided by the present invention, the obtained content in the first frame frame is in an editable state, which satisfies the user's editing requirements and improves the user experience.

Since the principle of the apparatus described in the embodiment of the present invention is the same as that of the method described in the foregoing embodiment, more detailed explanations are not repeated here.

FIG. 16 is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. As shown in FIG. 16 , the present invention provides an electronic device, including: a processor (processor) 1601, a memory (memory) 1602, and a bus 1603;

The processor 1601 and the memory 1602 communicate with each other through the bus 1603;

The processor 1601 is configured to call program instructions in the memory 1602 to execute the methods provided in the above method embodiments, for example, including: determining that the target table image is a borderless table; processing the target table image through a table recognition structure , to obtain a first border frame; and adding the content in the target table image to the first border frame.

An embodiment of the present invention provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the methods provided in the foregoing method embodiments, for example, including: Determining that the target table image is a borderless table; processing the target table image through the table recognition structure to obtain a first border frame; adding the content in the target table image to the first border frame.

The present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer program The method provided by the above methods can be implemented, and the method includes: determining a target table image as a borderless table; processing the target table image through a table recognition structure to obtain a first border frame; Content is added to the first border frame.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions related to hardware, the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, execute It includes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be Modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present invention .

Claims (13)

1. A method of processing a form image, comprising:

determining the target table image as a borderless table;

processing the target form image through a form identification structure to obtain a first frame;

and adding the content in the target form image into the first frame.

2. A method of processing form images according to claim 1, the method further comprising:

the table identification structure is a line, a sliding window or a graph convolution neural network model.

3. A method of processing form images according to claim 2, the method further comprising:

in a case that the table identification structure is the line or the sliding window, the processing the target table image through the table identification structure to obtain a first frame includes:

moving the table recognition structure along a first direction of the target table image, and recording a first area which does not cover the content;

and/or moving the table identification structure along a second direction of the target table image, and recording a second area which does not cover the content;

and generating a table line in at least one of the first area and the second area to obtain a first frame.

4. A method of processing form images according to claim 2, the method further comprising:

in the case that the table identification structure is the line, the table identification structure comprises a first direction line and a second direction line;

the processing the target form image through the form recognition structure to obtain a first frame, including:

translating the first direction line and the second direction line in the target form image based on the content distribution condition in the target form image, and determining a non-intersection area in the target form image;

and determining a target point in the non-intersection area, and connecting the target point to obtain a first frame.

5. The method of processing a form image according to claim 4, wherein the determining a non-intersection region in the target form image by translating the first direction line and the second direction line in the target form image based on the content distribution in the target form image comprises:

performing binarization processing on the target table image to obtain a first binary image; wherein, a first value in the first binary image is a pixel corresponding to a content distribution area in the target form image, and a second value in the first binary image is a pixel corresponding to a non-content distribution area in the target form image;

processing the pixels in the first binary image according to a first direction to obtain a first direction line;

processing the pixels in the first binary image according to a second direction to obtain a second direction line;

and intersecting the first direction lines and the second direction lines, and taking the intersected area which is not covered by the first direction lines or the second direction lines as a non-intersecting area in the target form image.

6. The method of processing a form image of claim 4, wherein the determining target points in non-intersecting regions comprises:

determining the contour of the non-intersection region through contour searching;

and taking a coordinate point of the non-intersection area as the target point according to the contour of the non-intersection area.

7. A method of processing form images according to claim 2, the method further comprising:

in the case that the table identification structure is the sliding window, the table identification structure comprises a first direction sliding window and a second direction sliding window;

the processing the target form image through the form recognition structure to obtain a first frame, including:

translating the first-direction sliding window in the target form image based on the content distribution condition in the target form image to obtain a first-direction frame body; translating the second-direction sliding window in the target form image to obtain a second-direction frame body;

and setting a first direction table line based on the first direction frame body, and setting a second direction table line based on the second direction frame body to obtain a first side frame.

8. A method of processing form images according to claim 2, the method further comprising:

and under the condition that the table identification structure is the graph convolution neural network model, determining a first frame according to the position relation between a first text box and a second text box in the target table image.

9. The method of processing a form image of claim 1, wherein prior to said processing the target form image by the form recognition structure resulting in a first frame, the method further comprises:

carrying out binarization processing on the target table image to obtain a second binary image; wherein, the second value in the second two-value image is the pixel corresponding to the text character in the target form image, and the first value in the second two-value image is the pixel except the text character in the target form image;

and setting a table identification structure for the second binary image.

10. The method of processing a form image of claim 1, wherein the adding content in the target form image into the first bounding box frame comprises:

performing text detection on the target form image, and obtaining a text box in the target form image according to a text detection result;

setting a text box in the first frame;

filling the content in the text box in the target form image into the text box of the first frame.

11. A form image processing apparatus, comprising:

the determining module is used for determining the target form image as a borderless form;

the processing module is used for processing the target form image through the form identification structure to obtain a first frame;

and the adding module is used for adding the content in the target form image into the first frame.

12. An electronic device, comprising: a processor, a memory, and a bus, wherein,

the processor and the memory are communicated with each other through the bus;

the memory stores program instructions executable by the processor for invoking steps of a method of processing a form image capable of performing any of claims 1-10.

13. A computer-readable storage medium, characterized in that it stores computer instructions that cause a computer to perform the steps of the method of processing a form image according to any one of claims 1 to 10.

Images