WO2022134771A1

WO2022134771A1 - Table processing method and apparatus, and electronic device and storage medium

Info

Publication number: WO2022134771A1
Application number: PCT/CN2021/124416
Authority: WO
Inventors: 高超; 徐国强
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2020-12-23
Filing date: 2021-10-18
Publication date: 2022-06-30
Also published as: CN112668566A

Abstract

The present application relates to image processing technology, and in particular to a table processing method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring a table image of a first table, performing identification on the table image, so as to obtain an identification result, and performing feature extraction on the identification result, so as to obtain a first feature vector; performing line segment detection on the table image, so as to obtain a line segment detection result, and performing feature extraction on the line segment detection result, so as to obtain a second feature vector; splicing the first feature vector and the second feature vector, so as to obtain a third feature vector; inputting the third feature vector into a preset model, so as to obtain a vertex feature, and determining a relationship adjacency matrix on the basis of the vertex feature; and performing restoration processing according to the relationship adjacency matrix, so as to obtain a second table, wherein the second table has the table structure of the first table. By using the embodiments of the present application, table restoration precision can be improved.

Description

Form processing method, device, electronic device and storage medium

This application claims the priority of the Chinese patent application filed on December 23, 2020 with the application number 202011538336.5 and the invention title is "Form Processing Method, Apparatus, Electronic Device and Storage Medium", the entire contents of which are incorporated by reference in this application.

technical field

The present application relates to the technical field of image processing, and in particular, to a form processing method, apparatus, electronic device, and storage medium.

Background technique

Optical Character Recognition (OCR) technology can convert the printed text in the image into a text format that can be processed by the computer. key aspects of the application. With the continuous development of big data and deep learning technology, OCR technology has made breakthroughs. In the recognition of scanned documents of printed documents, the character recognition accuracy rate of more than 99% can usually be achieved.

In addition to solving the problem of text position detection and content recognition, the OCR system usually needs to parse and restore the layout structure of the document. As the most common and important layout structure, table parsing technology has become a key part of the electronicization of paper documents.

The inventor realizes that the current table detection and structure recognition in the industry is usually based on table frame line detection, and the text row and column positions are obtained from the horizontal and vertical lines in the image, and then the structure information of the entire table is obtained. This method has a better recognition effect on the tables with complete frame lines of conventional tables, but cannot accurately restore the structure of tables without frame lines, three-line tables, and four-line tables with invisible table lines. Therefore, the problem of how to improve the accuracy of table restoration needs to be solved urgently.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a table processing method, apparatus, electronic device, and storage medium, which can improve table restoration accuracy.

In a first aspect, an embodiment of the present application provides a form processing method, the method comprising:

Obtain a table image of the first table, identify the table image, obtain a recognition result, and perform feature extraction on the recognition result to obtain a first feature vector;

Perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector;

Splicing the first feature vector and the second feature vector to obtain the third feature vector;

Inputting the third feature vector into a preset model, obtaining vertex features, and determining a relationship adjacency matrix based on the vertex features;

The restoration process is performed according to the relationship adjacency matrix to obtain a second table, where the second table is the table structure of the first table.

In a second aspect, an embodiment of the present application provides a table processing device, the device includes: an acquisition unit, a detection unit, a splicing unit, an input unit, and a restoration unit, wherein,

The acquiring unit is configured to acquire a table image of the first table, identify the table image, obtain a recognition result, and perform feature extraction on the recognition result to obtain a first feature vector;

The detection unit is configured to perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector;

The splicing unit is used for splicing the first eigenvector and the second eigenvector to obtain a third eigenvector;

The input unit is used to input the third feature vector into a preset model, obtain vertex features, and determine a relationship adjacency matrix based on the vertex features;

The restoration unit is configured to perform restoration processing according to the relationship adjacency matrix to obtain a second table, where the second table is the table structure of the first table.

In a third aspect, embodiments of the present application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be processed by the above-mentioned processing to implement the above table processing method, the method includes:

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute to implement the table processing method. , the method includes:

Implementing the embodiment of the present application, firstly, the overall recognition of the table image can be performed, and the overall recognition result can be obtained, for example, the content of the text box and the content information, and secondly, the line detection can also be performed, which is equivalent to partial image recognition. The features are fused, and then the global vertex information is effectively used, and the number of layers is deeper, and the feature extraction ability is stronger. In this way, the table can be deeply restored and the accuracy of the table restoration can be improved.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic flowchart of a form processing method provided by an embodiment of the present application;

2 is a schematic flowchart of another form processing method provided by an embodiment of the present application;

3 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

FIG. 4 is a block diagram of functional units of a table processing apparatus provided by an embodiment of the present application.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The terms "first", "second" and the like in the description and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

The present application may relate to artificial intelligence technology, for example, relevant data may be acquired and processed based on artificial intelligence technology, for example, vertex features may be determined based on artificial intelligence technology. Optionally, the technical solution of the present application can be applied to form processing scenarios in various fields, such as form processing in digital medical scenarios, or form processing in financial technology scenarios, etc., to improve the accuracy of form restoration, thereby Promote the construction of smart cities.

The electronic devices involved in the embodiments of this application may include various handheld devices with image processing functions (such as mobile phones, tablet computers, POS machines, etc.), scanners, micro-printers, desktop computers, vehicle-mounted devices, wearable devices ( smart watches, smart bracelets, wireless headsets, augmented reality/virtual reality devices, smart glasses), AI robots, computing devices or other processing devices connected to wireless modems, and various forms of user equipment (UE), Mobile station (mobile station, MS), terminal device (terminal device) and so on. For convenience of description, the devices mentioned above are collectively referred to as electronic devices.

The embodiments of the present application will be described in detail below.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a form processing method provided by an embodiment of the present application. As shown in the figure, applied to an electronic device, the form processing method includes:

101. Acquire a table image of a first table, identify the table image, obtain a recognition result, and perform feature extraction on the recognition result to obtain a first feature vector.

Wherein, in this embodiment of the present application, the first table may be any table, for example, an excel table, or a table hand-drawn by a user, or a table in a word file. In a specific implementation, the electronic device can photograph the first form, obtain a form image of the first form, and identify the form image, mainly by using OCR technology, to obtain a recognition result, and the recognition result can include the text box content and content information , and further, perform feature extraction on the recognition result to obtain a first feature vector, and the feature obtained by feature extraction can be at least one of the following: vertex coordinates, center coordinates, width, height, color, text character type statistics of the text box, Text word vectors, sentence vectors, front and background colors, fonts, textures, etc., are not limited here. The algorithm corresponding to feature extraction can be at least one of the following: Harris corner detection, scale-invariant feature transformation algorithm, neural network algorithm, wavelet transformation, etc., which are not limited here, and each feature can be represented in the form of a vector , to obtain the first feature vector, that is, the first feature vector may include the feature of the text box dimension of the table, and may also include the feature of the content dimension of the table.

Specifically, the electronic device can perform OCR recognition on the entire document, obtain text box position and content information, and extract features to obtain a N _t ×D feature vector, where N _t is the number of text boxes, D is the feature dimension, t is a positive integer. Features may include at least one of the following: location features (eg, vertex coordinates, center coordinates, width, height of text boxes), language features (eg, text character type statistics, text word vectors, sentence vectors, etc.), image features (eg, front Background color, font, texture, etc.), etc., are not limited here.

Optionally, in the above step 101, obtaining the table image of the first table may include the following steps:

11. Obtain the target environment parameters;

12. According to the mapping relationship between the preset environmental parameters and the shooting parameters, determine the target shooting parameters corresponding to the target environmental parameters;

13. Shoot the first table according to the target shooting parameters to obtain the table image.

The environmental parameter may be at least one of the following: ambient light brightness, ambient color temperature, jitter parameter, temperature, humidity, weather, etc., which are not limited herein. The electronic device can obtain the environmental parameters through the sensor, and the sensor can be at least one of the following: ambient light sensor, temperature sensor, humidity sensor, color temperature sensor, jitter detection sensor, weather sensor, etc., which are not limited here, the electronic device can focus on the above Sensors, through which each sensor is used to detect environmental parameters. The shooting parameters may be at least one of the following: ISO, exposure time, flash brightness, flash operating frequency, flash color, anti-shake parameters, white balance parameters, etc., which are not limited herein. The mapping relationship between the preset environmental parameters and the shooting parameters may be pre-stored in the electronic device.

In a specific implementation, the electronic device can obtain the target environment parameters, and then, according to the preset mapping relationship between the environment parameters and the shooting parameters, determine the target shooting parameters corresponding to the target environment parameters, and shoot the first table according to the target shooting parameters, The table image is obtained, in this way, shooting parameters suitable for the environment can be obtained, which helps to obtain the best table image and helps to improve the efficiency of subsequent table restoration.

102. Perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector.

The electronic device may perform line segment detection on the table image, and the specific algorithm may be at least one of the following: Hough transform, neural network algorithm, ecological algorithm, etc., which are not limited here. The algorithm corresponding to the feature extraction may be at least one of the following: Harris corner detection, scale-invariant feature transformation algorithm, neural network algorithm, wavelet transformation, etc., which are not limited herein.

In specific implementation, for example, the electronic device can perform line segment detection on the table image, obtain all line segment coordinate information in the table image, extract features, and obtain a feature vector of N _l ×D, where N _l is the number of line segments, and D is the feature dimension . The features may include at least one of the following: position features (eg, line segment start point, end point, midpoint coordinates, length, angle, etc.), image features (eg, foreground and background color, line shape, thickness, etc.), which are not limited here. Furthermore, the electronic device can process the feature into a second feature vector based on the feature, for example, convert different features into the same dimension, and then perform splicing. The line segment detection algorithm may be at least one of the following: Hough transform, a Line Segment Detector (LSD: aLine Segment Detector), a neural network algorithm, etc., which are not limited herein.

Optionally, in the above step 102, performing line segment detection on the table image to obtain a line segment detection result, which may include the following steps:

21. Determine the boundary outline of the table image;

22. Perform texture extraction on the image in the interface outline to obtain a P-striped road, where P is an integer greater than 1;

23. Screening the P-striped road to obtain a Q-striped road, where Q is an integer greater than 1 and less than the P;

24. Perform line segment detection on the Q-striped road to obtain the line segment detection result.

In a specific implementation, the electronic device can determine the boundary contour of the form image, and the interface contour can be set by the user, or can perform rough identification on the form image (for example, identify from the periphery to the inside, recognize only the peripheral contour, recognize the peripheral contour After that, the recognition operation is stopped), and the outermost contour is taken as the boundary contour. Further, the electronic device can perform texture extraction on the image in the interface outline to obtain the P-striped route, and then the electronic device can screen the P-striped route to obtain the Q-striped route. For example, it can detect whether the length of each texture in the P-striped route is It is within the preset length range, or it can be detected whether the width of each line in the P-striped road is within the preset width range, or it can be detected whether the bending angle of each line in the P-striped road is within the preset angle range, etc. The length range, the preset width range, and the preset angle range can be set by the user or the system defaults. Finally, the line segment detection is performed on the Q-stripe road to obtain the line segment detection result. In this way, the probability of misrecognition can be reduced.

103. Splicing the first feature vector and the second feature vector to obtain a third feature vector.

In a specific implementation, the electronic device may process the dimensions of the first feature vector and the second feature vector into the same dimension in at least one dimension, and then splicing the two to obtain the third feature vector.

In specific implementations, eg. The electronic device may splicing the first feature vector and the second feature vector to obtain a third feature vector, such as feature splicing, to obtain an N×D feature vector E, where N=N _t +N _l .

104. Input the third feature vector into a preset model to obtain vertex features, and determine a relationship adjacency matrix based on the vertex features.

The preset model may be set by the user or the system defaults, and the preset model may be at least one of the following: a Transformer model, a neural network model, etc., which are not limited here. The vertex feature may be at least one of the following: vertex position, vertex number, vertex vector, etc., which are not limited herein.

In a specific implementation, the electronic device may input the feature vector E into the Transformer model, and obtain the feature representation of the vertex X=Transformer(E), where the shape of X is N×D. Based on vertex feature X. Further, the relational adjacency matrix A= ^XWXT can also be obtained by bilinear multiplication, the shape of W can be R×D×D, and the shape of A can be R×N×N, where R is the number of relations. The relationship predicted here can include three types (R=3): two vertices are in the same table, the same row, and the same column, which can be represented by A0, A1, and A2 respectively. It is also possible to manually mark the target relation adjacency matrix Agt, input the vertex feature vector E, and obtain the predicted relation adjacency matrix A. The prediction result A is compared with the labeled answer Agt, and the model parameters can also be optimized using gradient descent. Iterative optimization continues until the model training is complete.

Optionally, in the above step 104, the third feature vector is input into a preset model to obtain vertex features, and based on the vertex features, a relationship adjacency matrix is determined, including the following steps:

41. Input the third feature vector into a preset model to obtain vertex features;

42. Based on the vertex feature, perform an operation through bilinear multiplication to obtain an initial relationship adjacency matrix;

43. Obtain a preset relationship adjacency matrix predicted based on the vertex feature;

44. Optimizing the model parameters of the preset model through the comparison result between the initial relationship adjacency matrix and the preset relationship adjacency matrix;

45. When the model parameters satisfy a preset condition, output a relation adjacency matrix based on the optimized preset model.

The preset condition may be set by the user or the system defaults, for example, the preset condition is that the preset model converges, or the preset condition is that the recognition accuracy of the preset model reaches a specified threshold, and the specified threshold can be set by the user or system default. The electronic device can input the third feature vector into the preset model to obtain vertex features, perform operations through bilinear multiplication based on the vertex features, obtain an initial relationship adjacency matrix, and obtain a preset relationship adjacency matrix predicted based on the vertex features, the The preset relationship adjacency matrix can be drawn by the user, or obtained by connecting each vertex according to a preset rule. The preset rule can be preset or the system defaults, for example, a straight line connection or a certain curved arc connection, etc., It is not limited here. Further, the electronic device can optimize the model parameters of the preset model through the comparison result between the initial relationship adjacency matrix and the preset relationship adjacency matrix, and specifically can compare the difference between the initial relationship adjacency matrix and the preset relationship adjacency matrix to obtain a comparison As a result, the comparison result is fed back into the preset model to optimize the model parameters. The model parameters can be control parameters of each module of the preset model, and the control parameters can be at least one of the following: convolution kernel size, offset, Threshold batch-size, etc., are not limited here. When the model parameters meet the preset conditions, based on the optimized preset model, the relational adjacency matrix is output. Specifically, the third eigenvector can be input into the optimized preset model to obtain the final relational adjacency matrix.

105. Perform restoration processing according to the relationship adjacency matrix to obtain a second table, where the second table is a table structure of the first table.

Wherein, after obtaining the relational adjacency matrix A, the electronic device can restore the table structure through a post-processing algorithm, and the specific implementation can be as follows: (1) solve the connected components for A0, and each connected component corresponds to a table; (2) according to The vertices with heavy connected components construct subgraphs from A1 and A2, and solve the maximal cliques for the subgraphs, and each clique corresponds to a row or column of the table; (3) Sort the rows and columns in the same table in order of position; ( 4) Restore cells according to the sorted rows and columns. If multiple vertices belong to the same row and column, it means they belong to the same cell. The electronic device may perform restoration processing on the relational adjacency matrix through a table restoration algorithm to obtain a second table, where the second table is the table structure of the first table. The table restoration algorithm may be at least one of the following: a neural network algorithm, an inverse transformation algorithm corresponding to a relational adjacency matrix, etc., which are not limited herein.

In the specific implementation, in this embodiment of the present application, the text boxes and line segments in the document can be input into the graph model as vertices, and the relationship between the vertices can be predicted to obtain whether the two text boxes are in the same table, row, and column. relationship, and then restore the table structure, and further, can effectively use information such as the positional relationship between text boxes, text logical relationship, etc., and can support table recognition with incomplete frame lines, and achieve better table detection and structure recognition. effect. In addition, the Transformer structure can also be used for vertex feature representation, which can effectively utilize global vertex information, and has deeper layers and stronger feature extraction capabilities, which can not only improve the efficiency of document entry and electronic paper text, but also better promote The development of informatization and digitalization in various industries.

Optionally, between the above steps 101 to 102, the following steps may also be included:

A1. Determine the target image quality evaluation value of the table image;

A2. When the target image quality evaluation value is greater than the preset image quality evaluation value, perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second Eigenvector steps;

or,

A3. When the target image quality evaluation value is less than or equal to the preset image quality evaluation value, perform image enhancement processing on the table image to obtain a target table image;

Then, in the above step 102, line segment detection is performed on the table image to obtain a line segment detection result, and feature extraction is performed on the line segment detection result to obtain a second feature vector, which can be implemented as follows:

Perform line segment detection on the target table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector.

Wherein, in this embodiment of the present application, the preset image quality evaluation value may be preset or system default. The electronic device can perform image quality evaluation on the form image, and further, can obtain a target image quality evaluation value of the form image. Specifically, the electronic device can use at least one image quality evaluation value to perform image quality evaluation on the table image, and the image quality evaluation index can be at least one of the following: information entropy, sharpness, mean square error, mean gradient, edge retention, mean gray and so on, which are not limited here, and further, the target image quality evaluation value can be obtained, and when the target image quality evaluation value is greater than the preset image quality evaluation value, step 102 is performed; otherwise, the target image quality evaluation value can be less than or equal to the preset image quality evaluation value, perform image enhancement processing on the table image to obtain the target table image, and then perform step 102 according to the target table image, wherein, the image enhancement algorithm corresponding to the image enhancement processing can be at least one of the following : grayscale stretching, histogram equalization, wavelet transform, neural network algorithm, etc., which are not limited here, and further, can ensure the accuracy of line detection.

Further, optionally, in the above step A3, image enhancement processing is performed on the table image to obtain the target table image, which may include the following steps:

A31. Obtain the text box content in the recognition result;

A32. Determine the target attribute parameter of the text box content;

A33. Obtain the reference attribute parameter of the first table;

A34. Determine the target deviation degree between the target attribute parameter and the reference attribute parameter;

A35. Determine the target image enhancement parameter corresponding to the target deviation degree according to the mapping relationship between the preset deviation degree and the image enhancement parameter;

A36. Perform image enhancement processing on the table image according to the target image enhancement parameter to obtain the target table image.

Among them, it can be known from the above that the recognition result can include the text box content and content information, and further, the electronic device can obtain the text box content in the recognition result, and can also determine the target attribute parameter of the text box content. In the embodiment of the present application, the target The attribute information may be at least one of the following: average width of lines of the text box, number of lines of the text box, vertex positions of the text box, number of vertices of the text box, area of the text box, etc., which are not limited herein. The reference attribute parameter can also be at least one of the following: the average line width of the text box, the number of lines of the text box, the vertex position of the text box, the number of vertices of the text box, the area of the text box, etc., which are not limited here. The reference attribute information may be a fixed attribute of the form. Since the first form exists in advance, for example, the form attribute of a printed form is also inherently set. Further, the electronic device can directly obtain the reference attribute parameter of the first form. Of course, the reference attribute parameters can also be set by the user based on experience. The electronic device can also pre-store the mapping relationship between the preset deviation degree and the image enhancement parameter, and the image enhancement parameter can be at least one of the following: an image enhancement algorithm and a control parameter of the image enhancement algorithm, which are not limited here. The image enhancement algorithm has different control parameters of the corresponding image enhancement algorithm. The control parameters of the image enhancement algorithm are used to control the image enhancement parameters. Usually, the control parameters of the image enhancement algorithm are adjusted reasonably, so as to avoid excessive image enhancement or The image is under-enhanced.

Further, the electronic device can obtain the reference attribute parameter of the first table, and can also determine the target deviation degree between the target attribute parameter and the reference attribute parameter, and the target deviation degree can be obtained as follows:

Target deviation = (reference attribute parameter - target attribute parameter) / reference attribute parameter

Further, the electronic device determines the target image enhancement parameter corresponding to the target deviation degree according to the mapping relationship between the preset deviation degree and the image enhancement parameter, and finally, can perform image enhancement processing on the table image according to the target image enhancement parameter to obtain the target image enhancement parameter. The table image, and further, the image enhancement parameters can be adjusted according to the deviation between the table image and the real table, so as to enhance the image in a targeted manner, thereby preventing the table image from causing the image to be over-enhanced or under-enhanced.

Further, optionally, in the above step A1, determining the target image quality evaluation value of the table image may include the following steps:

A11. Perform a background removal operation on the table image to obtain a target foreground image;

A12. Determine the target area area of the target foreground image;

A13. Perform feature point extraction on the target foreground image to obtain a target feature point set, and determine the number of target feature points according to the target feature point set;

A14. According to the number of the target feature points and the area of the target area, determine the distribution density of the target feature feature points;

A15. Determine the target image quality evaluation value corresponding to the target feature point distribution density according to the preset mapping relationship between the distribution density of feature points and the image quality evaluation value.

In a specific implementation, a preset mapping relationship between the distribution density of feature points and the image quality evaluation value may be pre-stored in the electronic device. The higher the distribution density of feature points, the better the image quality.

In a specific implementation, the electronic device can perform a background removal operation on the table image to obtain the target foreground image. Since the background is a blank part, or a pre-selected background for the table (for example, a watermark, a background image, etc.), it can be determined that The area of the target area of the target foreground image, and extracting feature points from the target foreground image to obtain a target feature point set, and the feature point extraction algorithm can be at least one of the following: Harris corner detection algorithm, scale-invariant feature extraction algorithm, neural network Algorithms, etc., are not limited here. Furthermore, the electronic device can determine the number of target feature points according to the target feature point set, and can also determine the distribution density of target feature points according to the number of target feature points and the area of the target area, that is:

Distribution density of target feature points = number of target feature points/target area area

Furthermore, the electronic device can determine the target image quality evaluation value corresponding to the target feature point distribution density according to the preset mapping relationship between the feature point distribution density and the image quality evaluation value. Since the background is removed, and only the foreground part of the image is used When performing image quality evaluation, specifically combined with the characteristics of the table, there are many blank contents in the table, and the image quality evaluation can be performed according to the corresponding feature points in the contour, and then accurate image quality evaluation can be performed for the table image.

It can be seen that the table processing method described in the embodiment of the present application obtains the table image of the first table, identifies the table image, obtains the recognition result, and performs feature extraction on the recognition result to obtain the first feature vector, which is Perform line segment detection on the table image to obtain the line segment detection result, and perform feature extraction on the line segment detection result to obtain the second feature vector, splicing the first feature vector and the second feature vector to obtain the third feature vector, and the third feature vector Input into the preset model to obtain vertex features, and based on the vertex features, determine the relationship adjacency matrix, and perform restoration processing according to the relationship adjacency matrix to obtain a second table, and the second table is the table structure of the first table, and then, one of them can be The overall recognition of the table image can obtain the overall recognition result, for example, the text box content and content information, and secondly, it can also detect lines, which is equivalent to local image recognition, fuse the features of the two, and then effectively use the global vertex In addition, the number of layers is deeper, and the feature extraction ability is stronger. In this way, the table can be deeply restored and the accuracy of the table restoration can be improved.

Please refer to FIG. 2. FIG. 2 is a schematic flowchart of a form processing method provided by an embodiment of the present application, which is applied to an electronic device. As shown in the figure, the form processing method includes:

201. Acquire a table image of a first table, identify the table image, obtain a recognition result, and perform feature extraction on the recognition result to obtain a first feature vector.

202. Determine a target image quality evaluation value of the table image.

203. When the target image quality evaluation value is greater than a preset image quality evaluation value, perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector.

204. Splicing the first feature vector and the second feature vector to obtain a third feature vector.

205. Input the third feature vector into a preset model to obtain vertex features, and determine a relationship adjacency matrix based on the vertex features.

206. Perform restoration processing according to the relationship adjacency matrix to obtain a second table, where the second table is a table structure of the first table.

For the specific description of the above steps 201 to 206, reference may be made to the corresponding steps described in the above FIG. 1 , which will not be repeated here.

It can be seen that the table processing method described in the embodiment of the present application obtains the table image of the first table, identifies the table image, obtains the recognition result, performs feature extraction on the recognition result, obtains the first feature vector, and determines The target image quality evaluation value of the table image, when the target image quality evaluation value is greater than the preset image quality evaluation value, perform line segment detection on the table image to obtain the line segment detection result, and perform feature extraction on the line segment detection result to obtain the second feature vector , splicing the first eigenvector and the second eigenvector to obtain the third eigenvector, inputting the third eigenvector into the preset model to obtain the vertex feature, and based on the vertex feature, determine the relational adjacency matrix, and carry out according to the relational adjacency matrix The restoration process is performed to obtain a second table, the second table is the table structure of the first table, and then, firstly, the overall recognition of the table image can be performed, and the overall recognition result can be obtained, for example, the text box content and content information, and secondly, It can also detect lines, which is equivalent to local image recognition, fuse the features of the two, and then effectively use the global vertex information. The number of layers is deeper, and the feature extraction ability is stronger. In this way, the table can be deeply restored and the accuracy of table restoration can be improved. .

Consistent with the above-mentioned embodiment, please refer to FIG. 3 , which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in the figure, the electronic device includes a processor, a memory, a communication interface, and one or more A program, the above-mentioned one or more programs are stored in the above-mentioned memory, and are configured to be executed by the above-mentioned processor. In the embodiment of the present application, the above-mentioned program includes instructions for executing the following steps:

It can be seen that the electronic device described in the embodiment of the present application obtains the table image of the first table, recognizes the table image, obtains the recognition result, and performs feature extraction on the recognition result to obtain the first feature vector, and the table Perform line segment detection on the image to obtain the line segment detection result, and perform feature extraction on the line segment detection result to obtain the second feature vector, splicing the first feature vector and the second feature vector to obtain the third feature vector, and input the third feature vector Go to the preset model, obtain the vertex features, and determine the relationship adjacency matrix based on the vertex features, perform restoration processing according to the relationship adjacency matrix, and obtain a second table, and the second table is the table structure of the first table. The overall recognition of the table image can obtain the overall recognition results, such as the text box content and content information, and secondly, it can also detect lines, which is equivalent to local image recognition, fuse the features of the two, and then effectively use the global vertex information. , and the number of layers is deeper, and the feature extraction ability is stronger. In this way, the table can be deeply restored and the accuracy of the table restoration can be improved.

Optionally, in the aspect of performing line segment detection on the table image to obtain a line segment detection result, the above program includes instructions for performing the following steps:

determining the boundary contour of the table image;

Extracting the texture of the image in the interface outline to obtain P striped texture, where P is an integer greater than 1;

Screening the P-striped roads to obtain Q-striped roads, where Q is an integer greater than 1 and less than the P;

Perform line segment detection on the Q-striped road to obtain the line segment detection result.

Optionally, in the aspect of inputting the third feature vector into a preset model, obtaining vertex features, and determining a relationship adjacency matrix based on the vertex features, the above program includes instructions for performing the following steps:

Inputting the third feature vector into a preset model to obtain vertex features;

Based on the vertex features, the operation is performed by bilinear multiplication to obtain an initial relationship adjacency matrix;

obtaining a preset relationship adjacency matrix predicted based on the vertex feature;

Optimizing the model parameters of the preset model through the comparison result between the initial relationship adjacency matrix and the preset relationship adjacency matrix;

When the model parameters satisfy a preset condition, a relation adjacency matrix is output based on the optimized preset model.

Optionally, in terms of obtaining the table image of the first table, the above program includes instructions for performing the following steps:

Get the target environment parameters;

According to the mapping relationship between the preset environmental parameters and the shooting parameters, determine the target shooting parameters corresponding to the target environmental parameters;

The first table is photographed according to the target photographing parameters to obtain the table image.

Optionally, after obtaining the table image of the first table, identifying the table image, obtaining a recognition result, and performing feature extraction on the recognition result to obtain the first feature vector, Perform line segment detection on the table image, obtain a line segment detection result, and perform feature extraction on the line segment detection result, before obtaining the second feature vector, the above program also includes an instruction for performing the following steps:

determining the target image quality evaluation value of the table image;

When the target image quality evaluation value is greater than the preset image quality evaluation value, perform the line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector A step of;

or,

When the target image quality evaluation value is less than or equal to the preset image quality evaluation value, image enhancement processing is performed on the table image to obtain a target table image, and line segment detection is performed on the table image to obtain line segment detection As a result, feature extraction is performed on the line segment detection result to obtain a second feature vector, including:

Optionally, in the aspect of performing image enhancement processing on the form image to obtain the target form image, the above program includes instructions for performing the following steps:

Obtain the text box content in the recognition result;

determining the target attribute parameter of the text box content;

obtaining the reference attribute parameter of the first table;

determining the target deviation degree between the target attribute parameter and the reference attribute parameter;

According to the mapping relationship between the preset deviation degree and the image enhancement parameter, determine the target image enhancement parameter corresponding to the target deviation degree;

Perform image enhancement processing on the table image according to the target image enhancement parameter to obtain the target table image.

Optionally, in the aspect of determining the target image quality evaluation value of the table image, the above program includes instructions for performing the following steps:

performing a background removal operation on the table image to obtain a target foreground image;

determining the target area area of the target foreground image;

Perform feature point extraction on the target foreground image to obtain a target feature point set, and determine the number of target feature points according to the target feature point set;

Determine the distribution density of target feature points according to the number of target feature points and the area of the target area;

The target image quality evaluation value corresponding to the distribution density of the target feature points is determined according to the preset mapping relationship between the distribution density of feature points and the image quality evaluation value.

The foregoing mainly introduces the solutions of the embodiments of the present application from the perspective of the method-side execution process. It can be understood that, in order to realize the above-mentioned functions, the electronic device includes corresponding hardware structures and/or software modules for executing each function. Those skilled in the art should easily realize that the present application can be implemented in hardware or in the form of a combination of hardware and computer software, in combination with the units and algorithm steps of each example described in the embodiments provided herein. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In this embodiment of the present application, the electronic device may be divided into functional units according to the foregoing method examples. For example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical function division, and other division methods may be used in actual implementation.

FIG. 4 is a block diagram of functional units of the table processing apparatus 400 involved in the embodiment of the present application. The table processing apparatus 400, the apparatus 400 includes: an acquisition unit 401, a detection unit 402, a splicing unit 403, an input unit 404 and a restoration unit 405, wherein,

The obtaining unit 401 is configured to obtain a table image of a first table, identify the table image, obtain a recognition result, and perform feature extraction on the recognition result to obtain a first feature vector;

The detection unit 402 is configured to perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector;

The splicing unit 403 is used for splicing the first eigenvector and the second eigenvector to obtain the third eigenvector;

The input unit 404 is configured to input the third feature vector into a preset model, obtain vertex features, and determine a relationship adjacency matrix based on the vertex features;

The restoration unit 405 is configured to perform restoration processing according to the relationship adjacency matrix to obtain a second table, where the second table is the table structure of the first table.

It can be seen that the table processing device described in the embodiments of the present application acquires the table image of the first table, recognizes the table image, obtains the recognition result, and performs feature extraction on the recognition result to obtain the first feature vector, which is Perform line segment detection on the table image to obtain the line segment detection result, and perform feature extraction on the line segment detection result to obtain the second feature vector, splicing the first feature vector and the second feature vector to obtain the third feature vector, and the third feature vector Input into the preset model to obtain vertex features, and based on the vertex features, determine the relationship adjacency matrix, and perform restoration processing according to the relationship adjacency matrix to obtain a second table, and the second table is the table structure of the first table, and then, one of them can be The overall recognition of the table image can obtain the overall recognition result, for example, the text box content and content information, and secondly, it can also detect lines, which is equivalent to local image recognition, fuse the features of the two, and then effectively use the global vertex In addition, the number of layers is deeper, and the feature extraction ability is stronger. In this way, the table can be deeply restored and the accuracy of the table restoration can be improved.

In a possible example, in terms of performing line segment detection on the table image to obtain a line segment detection result, the detection unit 402 is specifically configured to:

determining the boundary contour of the table image;

Optionally, in the aspect of inputting the third feature vector into a preset model, obtaining vertex features, and determining a relationship adjacency matrix based on the vertex features, the input unit 404 is specifically used for:

Optionally, in terms of acquiring the table image of the first table, the acquiring unit 401 is specifically configured to:

Get the target environment parameters;

Optionally, after obtaining the table image of the first table, identifying the table image, obtaining a recognition result, and performing feature extraction on the recognition result to obtain the first feature vector, Perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result, before obtaining the second feature vector, the device 400 is also specifically used for:

determining the target image quality evaluation value of the table image;

or,

Optionally, in the aspect of performing image enhancement processing on the form image to obtain the target form image, the device 400 is specifically used for:

Obtain the text box content in the recognition result;

determining the target attribute parameter of the text box content;

obtaining the reference attribute parameter of the first table;

Optionally, in the aspect of determining the target image quality evaluation value of the table image, the apparatus 400 is specifically configured to:

determining the target area area of the target foreground image;

It can be understood that the functions of each program module of the table processing apparatus in this embodiment can be specifically implemented according to the methods in the above method embodiments, and the specific implementation process can refer to the relevant descriptions of the above method embodiments, which will not be repeated here.

Embodiments of the present application further provide a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes the computer to execute part or all of the steps of any method described in the above method embodiments , the above computer includes electronic equipment.

Optionally, the storage medium involved in the present application may be a computer-readable storage medium, such as a computer-readable storage medium, which may be non-volatile or volatile.

Embodiments of the present application further provide a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of the method embodiments described above. some or all of the steps of the method. The computer program product may be a software installation package, and the computer includes an electronic device.

It should be noted that, for the sake of simple description, the foregoing method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence. Because in accordance with the present application, certain steps may be performed in other orders or concurrently. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical or other forms.

The above-mentioned units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The above-mentioned integrated units, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art, or all or part of the technical solution, and the computer software product is stored in a memory, Several instructions are included to cause a computer device (which may be a personal computer, an electronic device, or a network device, etc.) to execute all or part of the steps of the above-mentioned methods of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash disk , Read-only memory (English: Read-Only Memory, referred to as: ROM), random access device (English: Random Access Memory, referred to as: RAM), magnetic disk or optical disk, etc.

The embodiments of the present application have been introduced in detail above, and the principles and implementations of the present application are described in this paper by using specific examples. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; at the same time, for Persons of ordinary skill in the art, based on the idea of the present application, will have changes in the specific implementation manner and application scope. In summary, the contents of this specification should not be construed as limitations on the present application.

Claims

A form processing method, wherein the method comprises:

Obtain a table image of the first table, identify the table image, obtain a recognition result, and perform feature extraction on the recognition result to obtain a first feature vector;

Perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector;

Splicing the first feature vector and the second feature vector to obtain the third feature vector;

Inputting the third feature vector into a preset model, obtaining vertex features, and determining a relationship adjacency matrix based on the vertex features;

The restoration process is performed according to the relationship adjacency matrix to obtain a second table, where the second table is the table structure of the first table.
The method according to claim 1, wherein the performing line segment detection on the table image to obtain a line segment detection result comprises:

determining the boundary contour of the table image;

Extracting the texture of the image in the interface outline to obtain P striped texture, where P is an integer greater than 1;

Screening the P-striped roads to obtain Q-striped roads, where Q is an integer greater than 1 and less than the P;

Perform line segment detection on the Q-striped road to obtain the line segment detection result.
The method according to claim 1 or 2, wherein, inputting the third feature vector into a preset model to obtain vertex features, and determining a relationship adjacency matrix based on the vertex features, comprising:

Inputting the third feature vector into a preset model to obtain vertex features;

Based on the vertex features, the operation is performed by bilinear multiplication to obtain an initial relationship adjacency matrix;

obtaining a preset relationship adjacency matrix predicted based on the vertex feature;

Optimizing the model parameters of the preset model through the comparison result between the initial relationship adjacency matrix and the preset relationship adjacency matrix;

When the model parameters satisfy a preset condition, a relation adjacency matrix is output based on the optimized preset model.
The method according to claim 1 or 2, wherein the obtaining the table image of the first table comprises:

Get the target environment parameters;

According to the mapping relationship between the preset environmental parameters and the shooting parameters, determine the target shooting parameters corresponding to the target environmental parameters;

The first table is photographed according to the target photographing parameters to obtain the table image.
The method according to claim 1 or 2, wherein, in the obtaining of the form image of the first form, and recognizing the form image, a recognition result is obtained, and feature extraction is performed on the recognition result to obtain the first form. After the feature vector, and before performing line segment detection on the table image to obtain a line segment detection result, and performing feature extraction on the line segment detection result to obtain a second feature vector, the method further includes:

determining the target image quality evaluation value of the table image;

When the target image quality evaluation value is greater than the preset image quality evaluation value, perform the line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector A step of;

or,

When the target image quality evaluation value is less than or equal to the preset image quality evaluation value, image enhancement processing is performed on the table image to obtain a target table image, and line segment detection is performed on the table image to obtain line segment detection As a result, feature extraction is performed on the line segment detection result to obtain a second feature vector, including:

Perform line segment detection on the target table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector.
The method according to claim 5, wherein the performing image enhancement processing on the form image to obtain the target form image comprises:

Obtain the text box content in the recognition result;

determining the target attribute parameter of the text box content;

obtaining the reference attribute parameter of the first table;

determining the target deviation degree between the target attribute parameter and the reference attribute parameter;

According to the mapping relationship between the preset deviation degree and the image enhancement parameter, determine the target image enhancement parameter corresponding to the target deviation degree;

Perform image enhancement processing on the table image according to the target image enhancement parameter to obtain the target table image.
The method according to claim 5, wherein the determining the target image quality evaluation value of the table image comprises:

performing a background removal operation on the table image to obtain a target foreground image;

determining the target area area of the target foreground image;

Perform feature point extraction on the target foreground image to obtain a target feature point set, and determine the number of target feature points according to the target feature point set;

Determine the distribution density of target feature points according to the number of target feature points and the area of the target area;

The target image quality evaluation value corresponding to the distribution density of the target feature points is determined according to the preset mapping relationship between the distribution density of feature points and the image quality evaluation value.
A table processing device, wherein the device comprises: an acquisition unit, a detection unit, a splicing unit, an input unit and a restoration unit, wherein,

The acquiring unit is configured to acquire a table image of the first table, identify the table image, obtain a recognition result, and perform feature extraction on the recognition result to obtain a first feature vector;

The detection unit is configured to perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector;

The splicing unit is used for splicing the first eigenvector and the second eigenvector to obtain a third eigenvector;

The input unit is configured to input the third feature vector into a preset model, obtain vertex features, and determine a relationship adjacency matrix based on the vertex features;

The restoration unit is configured to perform restoration processing according to the relationship adjacency matrix to obtain a second table, where the second table is a table structure of the first table.
An electronic device, comprising a processor and a memory for storing one or more programs and configured to be executed by the processor to implement a table processing method, the method comprising:

obtaining a table image of the first table, and identifying the table image to obtain a recognition result, and performing feature extraction on the recognition result to obtain a first feature vector;

Perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector;

Splicing the first feature vector and the second feature vector to obtain the third feature vector;

Inputting the third feature vector into a preset model, obtaining vertex features, and determining a relationship adjacency matrix based on the vertex features;

The restoration process is performed according to the relationship adjacency matrix to obtain a second table, where the second table is the table structure of the first table.
The electronic device according to claim 9, wherein performing the line segment detection on the table image to obtain a line segment detection result comprises:

determining the boundary contour of the table image;

Extracting the texture of the image in the interface outline to obtain P striped texture, where P is an integer greater than 1;

Screening the P-striped roads to obtain Q-striped roads, where Q is an integer greater than 1 and less than the P;

Perform line segment detection on the Q-striped road to obtain the line segment detection result.
The electronic device according to claim 9 or 10, wherein performing the inputting the third feature vector into a preset model to obtain vertex features, and based on the vertex features, determining a relationship adjacency matrix, comprising:

Inputting the third feature vector into a preset model to obtain vertex features;

Based on the vertex features, the operation is performed by bilinear multiplication to obtain an initial relationship adjacency matrix;

obtaining a preset relationship adjacency matrix predicted based on the vertex feature;

Optimizing the model parameters of the preset model through the comparison result between the initial relationship adjacency matrix and the preset relationship adjacency matrix;

When the model parameters satisfy a preset condition, a relation adjacency matrix is output based on the optimized preset model.
The electronic device according to claim 9 or 10, wherein, in the process of acquiring a form image of the first form, recognizing the form image, obtaining a recognition result, and performing feature extraction on the recognition result, obtaining the first form image After a feature vector, and before performing line segment detection on the table image to obtain a line segment detection result, and performing feature extraction on the line segment detection result to obtain a second feature vector, the method further includes:

determining the target image quality evaluation value of the table image;

When the target image quality evaluation value is greater than the preset image quality evaluation value, perform the line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector A step of;

or,

When the target image quality evaluation value is less than or equal to the preset image quality evaluation value, image enhancement processing is performed on the table image to obtain a target table image, and line segment detection is performed on the table image to obtain line segment detection As a result, feature extraction is performed on the line segment detection result to obtain a second feature vector, including:

Perform line segment detection on the target table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector.
The electronic device according to claim 12, wherein performing the image enhancement processing on the form image to obtain the target form image comprises:

Obtain the text box content in the recognition result;

determining the target attribute parameter of the text box content;

obtaining the reference attribute parameter of the first table;

determining the target deviation degree between the target attribute parameter and the reference attribute parameter;

According to the mapping relationship between the preset deviation degree and the image enhancement parameter, determine the target image enhancement parameter corresponding to the target deviation degree;

Perform image enhancement processing on the table image according to the target image enhancement parameter to obtain the target table image.
The electronic device according to claim 12, wherein performing the determining of the target image quality evaluation value of the table image comprises:

performing a background removal operation on the table image to obtain a target foreground image;

determining the target area area of the target foreground image;

Perform feature point extraction on the target foreground image to obtain a target feature point set, and determine the number of target feature points according to the target feature point set;

Determine the distribution density of target feature points according to the number of target feature points and the area of the target area;

According to the mapping relationship between the preset feature point distribution density and the image quality evaluation value, the target image quality evaluation value corresponding to the target feature point distribution density is determined.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, the computer program includes program instructions that, when executed by a processor, cause the processor to perform a table processing method, The method includes:

obtaining a table image of the first table, and identifying the table image to obtain a recognition result, and performing feature extraction on the recognition result to obtain a first feature vector;

Perform line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector;

Splicing the first feature vector and the second feature vector to obtain the third feature vector;

Inputting the third feature vector into a preset model, obtaining vertex features, and determining a relationship adjacency matrix based on the vertex features;

The restoration process is performed according to the relationship adjacency matrix to obtain a second table, where the second table is the table structure of the first table.
The computer-readable storage medium according to claim 15, wherein performing the line segment detection on the table image to obtain a line segment detection result comprises:

determining the boundary contour of the table image;

Extracting the texture of the image in the interface outline to obtain P striped texture, where P is an integer greater than 1;

Screening the P-striped roads to obtain Q-striped roads, where Q is an integer greater than 1 and less than the P;

Perform line segment detection on the Q-striped road to obtain the line segment detection result.
The computer-readable storage medium of claim 15 or 16, wherein performing the inputting the third feature vector into a preset model to obtain vertex features, and based on the vertex features, determining a relationship adjacency matrix, comprising: :

Inputting the third feature vector into a preset model to obtain vertex features;

Based on the vertex features, the operation is performed by bilinear multiplication to obtain an initial relationship adjacency matrix;

obtaining a preset relationship adjacency matrix predicted based on the vertex feature;

Optimizing the model parameters of the preset model through the comparison result between the initial relationship adjacency matrix and the preset relationship adjacency matrix;

When the model parameters satisfy a preset condition, a relation adjacency matrix is output based on the optimized preset model.
The computer-readable storage medium according to claim 15 or 16, wherein, in the obtaining of the table image of the first table, the table image is recognized, the recognition result is obtained, and the feature extraction is performed on the recognition result , after obtaining the first feature vector, and before performing line segment detection on the table image to obtain a line segment detection result, and performing feature extraction on the line segment detection result to obtain the second feature vector, further comprising:

determining the target image quality evaluation value of the table image;

When the target image quality evaluation value is greater than the preset image quality evaluation value, perform the line segment detection on the table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector A step of;

or,

When the target image quality evaluation value is less than or equal to the preset image quality evaluation value, image enhancement processing is performed on the table image to obtain a target table image, and line segment detection is performed on the table image to obtain line segment detection As a result, feature extraction is performed on the line segment detection result to obtain a second feature vector, including:

Perform line segment detection on the target table image to obtain a line segment detection result, and perform feature extraction on the line segment detection result to obtain a second feature vector.
The computer-readable storage medium according to claim 18, wherein performing the image enhancement processing on the form image to obtain the target form image comprises:

Obtain the text box content in the recognition result;

determining the target attribute parameter of the text box content;

obtaining the reference attribute parameter of the first table;

determining the target deviation degree between the target attribute parameter and the reference attribute parameter;

According to the mapping relationship between the preset deviation degree and the image enhancement parameter, determine the target image enhancement parameter corresponding to the target deviation degree;

Perform image enhancement processing on the table image according to the target image enhancement parameter to obtain the target table image.
The computer-readable storage medium of claim 18, wherein performing the determining of the target image quality evaluation value of the form image comprises:

performing a background removal operation on the table image to obtain a target foreground image;

determining the target area area of the target foreground image;

Perform feature point extraction on the target foreground image to obtain a target feature point set, and determine the number of target feature points according to the target feature point set;

Determine the distribution density of target feature points according to the number of target feature points and the area of the target area;

The target image quality evaluation value corresponding to the distribution density of the target feature points is determined according to the preset mapping relationship between the distribution density of feature points and the image quality evaluation value.