CN114418869A - Method, system, device and medium for geometric correction of document image - Google Patents

Abstract

The invention discloses a method, a system, a device and a medium for geometric correction of a document image, wherein the method comprises the following steps: acquiring a first document image, classifying pixels in the first document image, distinguishing a foreground document area and an environment boundary area in the document image, and acquiring a mask image of the foreground document area; extracting control points on the mask image, primarily correcting the first document image according to the control points, deleting the environment boundary, and obtaining a second document image with primarily corrected and deleted environment boundary; and acquiring a first coordinate offset matrix of the second document image, and offsetting the second document image according to the first coordinate offset matrix to obtain a corrected third document image. The present invention is capable of handling captured document images having different environmental boundary regions, including situations with smaller environmental boundary regions, with larger environmental boundary regions, or without environmental boundary regions. The invention can be widely applied to the technical field of pattern recognition and artificial intelligence.

Description

A document image geometric correction method, system, device and medium

technical field

The invention relates to the technical field of pattern recognition and artificial intelligence, and in particular, to a method, system, device and medium for geometric correction of a document image.

Background technique

With the development of semiconductor technology, the built-in cameras of mobile devices are becoming more and more advanced, and the image quality is getting higher and higher. Using the built-in camera to take pictures to digitize document images has become a very convenient way to digitize documents. However, due to the perspective deformation caused by the inappropriate position and angle of the camera when taking pictures, as well as the bending, folding and wrinkling of the document itself, the captured document image will have geometric deformation. These deformations can affect the performance of optical character recognition systems, as well as the aesthetics and readability of document images. Deep learning-based methods for geometric correction of document images have achieved great progress in both correction performance and robustness to different document layouts. However, the existing deep learning correction methods only focus on correcting the cropped document image, that is, the document image with a small environmental boundary area, and requires a complete document boundary. However, there are various environmental boundary conditions in actual situations. Some document images have a large environmental boundary area, and the foreground document area only occupies a small part, while some document images do not have an environmental boundary area, so there is no complete document boundary. The aforementioned deep learning correction methods do not work well for such images.

SUMMARY OF THE INVENTION

In order to solve one of the technical problems existing in the prior art at least to a certain extent, the purpose of the present invention is to provide a method, system, device and medium for geometric correction of a document image.

The technical scheme adopted in the present invention is:

A document image geometric correction method, comprising the following steps:

obtaining a first document image, classifying the pixels in the first document image, distinguishing a foreground document area and an environmental boundary area in the document image, and obtaining a mask map of the foreground document area;

Extracting control points on the mask map, performing preliminary correction on the first document image according to the control points, deleting the environmental boundary, and obtaining a second document image with preliminary correction and deleting the environmental boundary;

A first coordinate offset matrix of the second document image is acquired, and after the second document image is offset according to the first coordinate offset matrix, a corrected third document image is obtained.

Further, the obtaining the corrected third document image includes:

It is judged according to the first coordinate offset matrix whether to perform the iterative step, if the iterative step is not required to be performed, the third document image is used as the output image; otherwise, the iterative step is performed;

The iterative steps include:

Acquiring a second coordinate offset matrix of the third document image, after offsetting the third document image according to the second coordinate offset matrix, updating the corrected image to the third document image, and recording the second coordinate offset matrix in the iteration step;

Determine whether to continue the iterative step according to the second coordinate offset matrix, and if so, return to the previous step; The second document image is offset, and the corrected image is obtained as the output image.

Further, the classifying the pixels in the first document image includes:

Using the first deep convolutional neural network to obtain the classification confidence of each pixel position in the first document image, and classify according to the classification confidence;

The acquiring the first coordinate offset matrix of the second document image includes:

A first coordinate offset matrix of the second document image is obtained by using a second deep convolutional neural network.

Further, the control points are extracted on the mask map, the first document image is preliminarily corrected according to the control points, the environmental boundary is deleted, and the second document image that is preliminarily corrected and the environmental boundary is deleted, including :

Extract the four corners of the document on the mask map of the foreground document area using a polygon fitting algorithm;

According to the preset equal division ratio, draw a vertical bisecting line on the line segment formed by the adjacent corner points as endpoints, and take the intersection of the bisecting line and the mask image boundary of the foreground document area as the bisecting point of the document boundary;

Draw a quadrilateral mask map with four corners, and calculate the intersection ratio according to the quadrilateral mask map and the mask map of the foreground document area;

If the intersection ratio is smaller than the first preset threshold, no correction is performed, and the first document image is used as the second document image;

If the intersection ratio is greater than the first preset threshold, take the four corner points and several boundary points as control points, use the thin-plate spline interpolation algorithm to perform preliminary correction on the first document image, delete the environmental boundary, and obtain the preliminary correction and Delete the second document image of the environment boundary.

Further, after the second document image is offset according to the first coordinate offset matrix, a corrected third document image is obtained, including:

The first coordinate offset matrix specifies a two-dimensional offset vector for each pixel position in the second document image, and each pixel is offset according to the corresponding offset vector to obtain a corrected third document image;

Among them, the offset vector is used to represent the offset direction and distance on the two-dimensional plane.

Further, judging whether to continue to perform the iterative step according to the second coordinate offset matrix includes:

calculating the standard deviation of the second coordinate offset matrix;

If the standard deviation of the second coordinate offset matrix is greater than the second preset threshold, continue to perform the iterative step;

If the standard deviation of the second coordinate offset matrix is smaller than the second preset threshold, the iterative step is stopped.

Further, the offsetting of the second document image according to the first coordinate offset matrix and all the second coordinate offset matrices in the record, to obtain a corrected image as an output image, includes:

和迭代步骤中记录的若干个坐标偏移矩阵

进行求和，获得最终的坐标偏移矩阵

根据最终坐标偏移矩阵

对所述第二文档图像进行偏移，获得矫正后的图像作为输出图像。Offset the first coordinate to the matrix

and several coordinate offset matrices recorded in the iteration step

Do the summation to get the final coordinate offset matrix

Offset matrix according to final coordinates

Offset the second document image to obtain a corrected image as an output image.

Another technical scheme adopted by the present invention is:

A document image geometric correction system, comprising:

a pixel classification module, configured to obtain a first document image, classify the pixels in the first document image, distinguish a foreground document area and an environmental boundary area in the document image, and obtain a mask map of the foreground document area;

A preliminary correction module for extracting control points on the mask map, performing preliminary correction on the first document image according to the control points, deleting the environmental boundary, and obtaining a second document image with preliminary correction and deleting the environmental boundary;

an offset correction module, configured to obtain a first coordinate offset matrix of the second document image, and after offsetting the second document image according to the first coordinate offset matrix, obtain a corrected third document image.

Another technical scheme adopted by the present invention is:

A document image geometric correction device, comprising:

at least one processor;

at least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

Another technical scheme adopted by the present invention is:

A computer-readable storage medium in which a processor-executable program is stored, the processor-executable program, when executed by the processor, is used to perform the method as described above.

The beneficial effects of the present invention are that the present invention can process photographed document images with different environmental boundary areas, including situations with smaller environmental boundary areas, larger environmental boundary areas, or no environmental boundary areas.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following descriptions are given to the accompanying drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings in the following introduction are only In order to facilitate and clearly express some embodiments of the technical solutions of the present invention, for those skilled in the art, other drawings can also be obtained from these drawings without creative work.

1 is an overall flow chart of a method for geometric correction of photographed document images suitable for multiple environmental boundary conditions in an embodiment of the present invention;

2 is a schematic diagram of a control point extraction and preliminary correction to obtain a document image with an environmental boundary area removed in an embodiment of the present invention;

3 is a schematic diagram of iterative correction in an embodiment of the present invention;

FIG. 4 is a correction effect for document images with different environmental boundary conditions in an embodiment of the present invention.

FIG. 5 is a flow chart of steps of a method for geometric correction of a document image in an embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, only used to explain the present invention, and should not be construed as a limitation of the present invention. The numbers of the steps in the following embodiments are only set for the convenience of description, and the sequence between the steps is not limited in any way, and the execution sequence of each step in the embodiments can be adapted according to the understanding of those skilled in the art Sexual adjustment.

In the description of the present invention, it should be understood that the azimuth description, such as the azimuth or position relationship indicated by up, down, front, rear, left, right, etc., is based on the azimuth or position relationship shown in the drawings, only In order to facilitate the description of the present invention and simplify the description, it is not indicated or implied that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the present invention.

In the description of the present invention, the meaning of several is one or more, the meaning of multiple is two or more, greater than, less than, exceeding, etc. are understood as not including this number, above, below, within, etc. are understood as including this number. If it is described that the first and the second are only for the purpose of distinguishing technical features, it cannot be understood as indicating or implying relative importance, or indicating the number of the indicated technical features or the order of the indicated technical features. relation.

In the description of the present invention, unless otherwise clearly defined, words such as setting, installation, connection should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

As shown in FIG. 5 , this embodiment provides a method for geometric correction of a document image, including the following steps:

S101, obtaining a first document image, classifying the pixels in the first document image, distinguishing a foreground document area and an environmental boundary area in the document image, and obtaining a mask map of the foreground document area;

S102, extracting control points on the mask map, performing preliminary correction on the first document image according to the control points, deleting the environmental boundary, and obtaining a second document image with preliminary correction and deleting the environmental boundary;

S103: Acquire a first coordinate offset matrix of the second document image, and obtain a corrected third document image after offsetting the second document image according to the first coordinate offset matrix.

In this embodiment, the first document image may be a document image obtained by photographing by a photographing device (such as a smart terminal, a camera, etc.), or may be a document image obtained by scanning, or the like.

In some embodiments, the step of obtaining the corrected third document image in step S103 specifically includes:

According to the first coordinate offset matrix, it is judged whether to perform the iterative step, if the iterative step is not required to be performed, the third document image is used as the output image; otherwise, the iterative step is performed;

Among them, the iterative steps include A1-A2:

A1. Obtain the second coordinate offset matrix of the third document image, and after offsetting the third document image according to the second coordinate offset matrix, update the corrected image to the third document image, and record the The second coordinate offset matrix;

A2. Determine whether to continue the iterative step according to the second coordinate offset matrix, if so, return to the previous step; otherwise, perform the second document image according to the first coordinate offset matrix and all the second coordinate offset matrices in the record. Offset to obtain the corrected image as the output image.

In this embodiment, the flatness of the input image is reflected by the statistical information of the coordinate offset matrix; a low response means that the input image is flat. When the first coordinate offset matrix is smaller than the preset threshold, the third document image is directly used as the output image. When the first coordinate offset matrix is greater than the preset threshold, the iterative step is performed. As the number of iterations increases, the response of the coordinate offset matrix becomes lower and lower until the coordinate offset matrix of the document image is smaller than the preset value. Threshold value; add all the obtained coordinate offset matrices, offset the second document image, and obtain the corrected image as the output image.

The above method will be explained in detail below with reference to specific embodiments and accompanying drawings.

As shown in FIG. 1 , this embodiment provides a geometric correction method for photographed document images suitable for various environmental boundary conditions, which is used to solve the geometric correction problem of photographed document images with different environmental boundary conditions in an actual scene. The method specifically includes the following steps:

S1. Classify each pixel of the input photographed document image (ie, the first document image), distinguish the foreground document area and the environmental boundary area on the image, and obtain a mask map of the foreground document area. This allows for precise separation of foreground document regions from the document image.

S2, extracting control points on the mask map, using the control points to perform preliminary correction on the input document image, and removing the environmental boundary at the same time, to obtain a document image (ie, the second document image) with preliminary correction and removal of the environmental boundary as the input of the next step .

Extracting control points on the mask map, because it is a binary map, is much easier than extracting directly on the input document image. Use the control points to perform preliminary correction on the input document image, while removing the environmental boundary to obtain preliminary correction. And remove the document image of the environment boundary as the input of the next step.

Specifically, as shown in FIG. 2 , this step S2 includes steps S21-S24:

S21, using the Douglas-Peucker polygon fitting algorithm to extract four corner points of the document on the mask map of the foreground document area;

S22. Distinguish the upper left, upper right, lower right and lower left corner points according to the relative positional relationship of the four corner points;

S23, draw a vertical bisector line on the line segment formed by the adjacent corner points according to the preset equal division ratio, and use the intersection of the bisector line and the border of the mask image of the foreground document area as the equal division point of the document boundary;

S24, draw a quadrilateral mask map with four corner points, and calculate the intersection ratio with the mask map of the foreground document area, if the intersection ratio is less than a preset threshold (according to the small intersection ratio, it can be judged that the input shot document image does not contain complete document boundary), then do not correct, and use the input document image as the input of the next step; if the intersection ratio is greater than the preset threshold (according to the large intersection ratio, it can be judged that the input captured document image contains a complete document boundary), then use The four corner points and several boundary points are used as control points. The thin-plate spline interpolation algorithm is used to initially correct the input document image, and the environmental boundary is removed at the same time, and the document image after preliminary correction and removal of the environmental boundary is obtained as the input of the next step. . Because the initial correction is realized by using the boundary of the document, if the input document image does not have a complete document boundary, it is unreasonable to perform the above thin plate spline correction. The input document image of the boundary is culled, and the thin plate spline correction is not performed, but is directly sent to the next step.

S3. Predict the coordinate offset matrix for the document image. After the document image is offset according to the coordinate offset matrix, a corrected document image is obtained. The corrected document image can be used again as the input of step S3 for iterative correction. Whether to perform iterative correction depends on the coordinates The statistics of the offset matrix are determined adaptively. When the iteration stops, the final corrected document image is obtained according to the obtained several coordinate offset matrices.

In some optional embodiments, the coordinate offset matrix in step S3 specifies a two-dimensional offset vector for each pixel position in the input image, and the offset vector indicates the offset direction and distance on the two-dimensional plane, After the pixels are offset according to the corresponding offset vector, the corrected document image is obtained, and the offset process adopts linear interpolation.

In some optional embodiments, the standard deviation is used as the statistical information of the coordinate offset matrix in step S3, because as the number of iterations increases, the input image becomes more and more flat, and the corresponding predicted coordinate offset matrix is It will become lower and lower, and its standard deviation will also become smaller and smaller. When the standard deviation is less than a certain threshold, we can consider that the corresponding input document image is flat enough, so the iteration can be stopped, otherwise, the iteration will continue. In this way, a balance between correction performance and correction efficiency can be achieved, making the system more efficient.

求和得到最终的坐标偏移矩阵

根据最终坐标偏移矩阵

对步骤S2的输出进行坐标偏移得到最终矫正后的文档图像。这里不直接采用

矫正得到文档图像作为最终输出的原因是：此时文档图像已经经过多次采样，会导致出现模糊的问题。而采用

在步骤S2的输出上获得的矫正文档图像则只经过了一次采样，可以有效避免出现模糊的问题。In some optional embodiments, as shown in FIG. 3 , performing correction based on several coordinate offset matrices in step S3 includes: firstly aligning the obtained several coordinate offset matrices

Sum up to get the final coordinate offset matrix

Offset matrix according to final coordinates

Coordinate offset is performed on the output of step S2 to obtain the final corrected document image. not directly used here

The reason for correcting the document image as the final output is that the document image has been sampled multiple times at this time, which will cause blurring. using

The corrected document image obtained on the output of step S2 has only been sampled once, which can effectively avoid the problem of blurring.

In some optional embodiments, in step S1, a deep convolutional neural network is used to obtain the classification confidence of each pixel position, the parameters of the network are pre-trained and optimized with synthetic data, and binarization is performed by a threshold of 0.5 during prediction, Get the final foreground document area mask map. The network parameter optimization specifically includes:

(1) Data acquisition: use 100,000 data samples in the Doc3D public synthetic data set (one data sample includes an input captured document image and its corresponding foreground document area mask map) as training (90,000 data samples) and verification data (10000 data samples);

(2) Network training:

(2-1) Build a deep neural network: Use the DeepLabv3+ segmentation model as the network structure, and set the number of output categories to 1, that is, the number of channels of the output result of the last layer of the network is 1.

(2-2) Training method: The training uses the gradient descent algorithm, and the purpose of training the network is achieved by calculating the gradient from the last layer, passing it layer by layer, and updating all parameters. The loss function during training is binary cross entropy loss.

(2-3) Setting of training parameters:

Iterations: 50epoch

Optimizer: Adam

Learning rate: 0.0001 (learning rate update strategy: after every 5 iterations, the learning rate decays to 1/2 of the original)

Weight decay: 0.0005

(2-4) Start training the deep neural network with random initialization parameters.

In some optional embodiments, in step S3, a deep convolutional neural network is used to obtain a coordinate offset matrix, and the parameters of the network are pre-trained and optimized through synthetic data, specifically including:

(1) Data acquisition: use 100,000 data samples in the Doc3D public synthetic data set (one data sample includes an input photographed document image and its corresponding left offset matrix) as training (90,000 data samples) and verification data (10,000 data samples), before the training, the samples are processed in the aforementioned steps S1 and S2 to remove the environmental boundary;

(2) Network training:

(2-1) Constructing a deep neural network: The network adopts an encoder-decoder structure of downsampling and then upsampling, and skip connections are used to retain detailed features and facilitate gradient return, as shown in Table 1 below:

Table 1

Network layer specific operation Feature map size input layer - 3*448*448 convolutional layer The number of kernels is 32, the convolution kernel is 3*3, the step size is 1*1, and the edge is supplemented 32*448*448 nonlinear layer - 32*448*448 pooling layer Pooling kernel 2*2, step size 2*2 32*224*224 convolutional layer The number of kernels is 64, the convolution kernel is 3*3, the step size is 1*1, and the edge is supplemented 64*224*224 nonlinear layer - 64*224*224 pooling layer Pooling kernel 2*2, step size 2*2 64*112*112 convolutional layer The number of kernels is 128, the convolution kernel is 3*3, the step size is 1*1, and the edge is supplemented 128*112*112 nonlinear layer - 128*112*112 pooling layer Pooling kernel 2*2, step size 2*2 128*56*56 convolutional layer The number of kernels is 256, the convolution kernel is 3*3, the step size is 1*1, and the edge is supplemented 256*56*56 nonlinear layer - 256*56*56 pooling layer Pooling kernel 2*2, step size 2*2 256*28*28 convolutional layer The number of kernels is 128, the convolution kernel is 3*3, the step size is 1*1, and the edge is supplemented 512*28*28 nonlinear layer - 512*28*28 transposed convolutional layer The number of kernels is 256, the convolution kernel is 4*4, the step size is 2*2, and the edge is supplemented 256*56*56 nonlinear layer - 256*56*56 skip connection layer Splicing the corresponding feature maps in the downsampling path on the channel 512*56*56 convolutional layer The number of kernels is 256, the convolution kernel is 3*3, the step size is 1*1, and the edge is supplemented 256*56*56 nonlinear layer - 256*56*56 transposed convolutional layer The number of kernels is 128, the convolution kernel is 4*4, the step size is 2*2, and the edge is supplemented 128*112*112 nonlinear layer - 128*112*112 skip connection layer Splicing the corresponding feature maps in the downsampling path on the channel 256*112*112 convolutional layer The number of kernels is 128, the convolution kernel is 3*3, the step size is 1*1, and the edge is supplemented 128*112*112 nonlinear layer - 128*112*112 transposed convolutional layer The number of kernels is 64, the convolution kernel is 4*4, the step size is 2*2, and the edge is supplemented 64*224*224 nonlinear layer - 64*224*224 skip connection layer Splicing the corresponding feature maps in the downsampling path on the channel 128*224*224 convolutional layer The number of kernels is 64, the convolution kernel is 3*3, the step size is 1*1, and the edge is supplemented 64*224*224 nonlinear layer - 64*224*224 transposed convolutional layer The number of kernels is 32, the convolution kernel is 4*4, the step size is 2*2, and the edge is supplemented 32*448*448 nonlinear layer - 32*448*448 skip connection layer Splicing the corresponding feature maps in the downsampling path on the channel 64*448*448 convolutional layer The number of kernels is 32, the convolution kernel is 3*3, the step size is 1*1, and the edge is supplemented 32*448*448 nonlinear layer - 32*448*448 convolutional layer The number of kernels is 2, the convolution kernel is 3*3, the stride size is 1*1, and the edge is supplemented 2*448*448 nonlinear layer - 2*448*448

(2-2) Training method: The training uses the gradient descent algorithm to achieve the purpose of training the network by calculating the gradient from the last layer, passing it layer by layer, and updating all parameters. The loss function during training is the mean squared loss.

(2-3) Setting of training parameters:

Iterations: 50epoch

Optimizer: Adam

Learning rate: 0.0001 (learning rate update strategy: after every 5 iterations, the learning rate decays to 1/2 of the original)

Weight decay: 0.0005

(2-4) Start training the deep neural network with random initialization parameters.

As shown in FIG. 4 , the method provided in this embodiment can process captured document images in various environmental boundary conditions, and can achieve better correction effects. To sum up, the method proposed in this embodiment can process captured document images with different environmental boundary areas, including situations with smaller environmental boundary areas, larger environmental boundary areas, and no environmental boundary areas. At the same time, for document images with different degrees of geometric deformation, the method proposed in this embodiment can adaptively determine the number of iterations to achieve better correction effects.

This embodiment also provides a document image geometric correction system, including:

a pixel classification module, configured to obtain a first document image, classify the pixels in the first document image, distinguish a foreground document area and an environmental boundary area in the document image, and obtain a mask map of the foreground document area;

A preliminary correction module for extracting control points on the mask map, performing preliminary correction on the first document image according to the control points, deleting the environmental boundary, and obtaining a second document image with preliminary correction and deleting the environmental boundary;

an offset correction module, configured to obtain a first coordinate offset matrix of the second document image, and after offsetting the second document image according to the first coordinate offset matrix, obtain a corrected third document image.

A document image geometric correction system in this embodiment can perform a document image geometric correction method provided by the method embodiment of the present invention, and can perform any combination of implementation steps of the method embodiment, and has corresponding functions and beneficial effects of the method. .

This embodiment also provides a document image geometric correction device, including:

at least one processor;

at least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the method shown in FIG. 5 .

A document image geometric correction system in this embodiment can perform a document image geometric correction method provided by the method embodiment of the present invention, and can perform any combination of implementation steps of the method embodiment, and has corresponding functions and beneficial effects of the method. .

Embodiments of the present application further disclose a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method shown in FIG. 5 .

This embodiment also provides a storage medium storing an instruction or a program for executing a method for geometric correction of a document image provided by the method embodiment of the present invention. When the instruction or program is executed, any of the method embodiments can be executed. The combined implementation steps have corresponding functions and beneficial effects of the method.

In some alternative implementations, the functions/operations noted in the block diagrams may occur out of the order noted in the operational diagrams. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/operations involved. Furthermore, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of the various operations are altered and in which sub-operations described as part of larger operations are performed independently.

Furthermore, while the invention is described in the context of functional modules, it is to be understood that, unless stated to the contrary, one or more of the described functions and/or features may be integrated in a single physical device and/or or software modules, or one or more functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to understand the present invention. Rather, given the attributes, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the modules will be within the routine skill of the engineer. Accordingly, those skilled in the art, using ordinary skill, can implement the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are illustrative only and are not intended to limit the scope of the invention, which is to be determined by the appended claims along with their full scope of equivalents.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus.

More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

In the above description of the present specification, reference to the description of the terms "one embodiment/example", "another embodiment/example" or "certain embodiments/examples" etc. means the description in conjunction with the embodiment or example. Particular features, structures, materials, or characteristics are included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, The scope of the invention is defined by the claims and their equivalents.

The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned embodiments, and those skilled in the art can also make various equivalent deformations or replacements on the premise of not violating the spirit of the present invention. Equivalent modifications or substitutions are included within the scope defined by the claims of the present application.

Claims (10)

1. a document image geometric correction method, is characterized in that, comprises the following steps: obtaining a first document image, classifying the pixels in the first document image, distinguishing a foreground document area and an environmental boundary area in the document image, and obtaining a mask map of the foreground document area; Extracting control points on the mask map, performing preliminary correction on the first document image according to the control points, deleting the environmental boundary, and obtaining a second document image with preliminary correction and deleting the environmental boundary; A first coordinate offset matrix of the second document image is acquired, and after the second document image is offset according to the first coordinate offset matrix, a corrected third document image is obtained. 2. A document image geometric correction method according to claim 1, wherein the obtaining the corrected third document image comprises: It is judged according to the first coordinate offset matrix whether to perform the iterative step, if the iterative step is not required to be performed, the third document image is used as the output image; otherwise, the iterative step is performed; The iterative steps include: Acquiring a second coordinate offset matrix of the third document image, after offsetting the third document image according to the second coordinate offset matrix, updating the corrected image to the third document image, and recording the second coordinate offset matrix in the iteration step; Determine whether to continue the iterative step according to the second coordinate offset matrix, and if so, return to the previous step; The second document image is offset, and the corrected image is obtained as the output image. 3. A document image geometric correction method according to claim 1, wherein the classifying the pixels in the first document image comprises: Using the first deep convolutional neural network to obtain the classification confidence of each pixel position in the first document image, and classify according to the classification confidence; The acquiring the first coordinate offset matrix of the second document image includes: A first coordinate offset matrix of the second document image is obtained by using a second deep convolutional neural network. 4. The method for geometric correction of a document image according to claim 1, wherein the control points are extracted on the mask map, and the first document image is preliminarily corrected according to the control points, Remove the environmental boundary, obtain a second document image with preliminary rectification and remove the environmental boundary, including: Extract the four corners of the document on the mask map of the foreground document area using a polygon fitting algorithm; Draw a vertical bisector line on the line segment formed by the adjacent corner points according to the preset bisector ratio, and take the intersection of the bisector line and the mask image boundary of the foreground document area as the bisector point of the document boundary; Draw a quadrilateral mask map with four corners, and calculate the intersection ratio according to the quadrilateral mask map and the mask map of the foreground document area; If the intersection ratio is smaller than the first preset threshold, no correction is performed, and the first document image is used as the second document image; If the intersection-union ratio is greater than the first preset threshold, the four corner points and several boundary points are used as control points, and the thin-plate spline interpolation algorithm is used to perform preliminary correction on the first document image, delete the environmental boundary, and obtain a preliminary correction and Delete the second document image of the environment boundary. 5 . The method for geometric correction of a document image according to claim 1 , wherein after the second document image is offset according to the first coordinate offset matrix, a corrected third image is obtained. 6 . Documentation images, including: The first coordinate offset matrix specifies a two-dimensional offset vector for each pixel position in the second document image, and each pixel is offset according to the corresponding offset vector to obtain a corrected third document image; Among them, the offset vector is used to represent the offset direction and distance on the two-dimensional plane. 6. The method for geometric correction of document images according to claim 2, wherein the judging whether to continue to perform the iterative step according to the second coordinate offset matrix comprises: calculating the standard deviation of the second coordinate offset matrix; If the standard deviation of the second coordinate offset matrix is greater than the second preset threshold, continue to perform the iterative step; If the standard deviation of the second coordinate offset matrix is smaller than the second preset threshold, the iterative step is stopped. 7 . The method for geometric correction of a document image according to claim 2 , wherein the second coordinate offset matrix is adjusted according to the first coordinate offset matrix and all second coordinate offset matrices in the record. 8 . The document image is offset, and the corrected image is obtained as the output image, including: Offset the first coordinate to the matrix

and several coordinate offset matrices recorded in the iteration step

Do the summation to get the final coordinate offset matrix

Offset matrix according to final coordinates

Offset the second document image to obtain a corrected image as an output image. 8. A document image geometric correction system, characterized in that, comprising: a pixel classification module, configured to obtain a first document image, classify the pixels in the first document image, distinguish a foreground document area and an environmental boundary area in the document image, and obtain a mask map of the foreground document area; A preliminary correction module for extracting control points on the mask map, performing preliminary correction on the first document image according to the control points, deleting the environmental boundary, and obtaining a second document image with preliminary correction and deleting the environmental boundary; an offset correction module, configured to obtain a first coordinate offset matrix of the second document image, and after offsetting the second document image according to the first coordinate offset matrix, obtain a corrected third document image. 9. A document image geometric correction device, characterized in that, comprising: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1-7. 10. A computer-readable storage medium in which a processor-executable program is stored, wherein the processor-executable program is used to execute any one of claims 1-7 when executed by the processor the method.

Images