CN118334670A - Seal content identification method, system and medium - Google Patents

Abstract

The invention discloses a seal content identification method, a seal content identification system and a seal content identification medium, wherein the method comprises the following steps: acquiring a picture with a seal, detecting and positioning the seal in the picture, and obtaining the position coordinates and the category of the seal in the image; cutting out an image area of the seal according to the position coordinates of the seal to obtain a seal content area represented by a polygon coordinate list; ordering the polygonal coordinates, executing a seal content area correction algorithm based on an arc curve on seal content areas with circular and elliptical seal types, and correcting the circular or elliptical seal content areas into rectangles to obtain corrected seal content areas; and executing a seal content identification algorithm on the seal content area, and outputting the text content of the seal. The invention can effectively solve the problem of seal content identification, complete the comparison and verification of the names of main objects such as contracts or invoices and the seal content, save a great deal of labor cost and improve the office efficiency of enterprises.

Description

Seal content identification method, system and medium

Technical Field

The invention relates to the technical field of computer vision target detection and text recognition, in particular to a seal content recognition method, a seal content recognition system and a seal content recognition medium based on arc curve correction.

Background

With the advancement of science and technology, computer-trained algorithms have been developed that can scan, read, understand digital and paper documents like humans, and intelligent document processing. Intelligent document processing (INTELLIGENT DOCUMENT PROCESSING) IDP refers to the automatic processing of documents by a computer based on the understanding of the documents. Common typical forms include automatic or semi-automatic generation of documents, automatic understanding of documents, integration of documents with business processes, automatic evaluation of documents, auditing of documents, and the like. This technology is becoming increasingly popular in many areas. Intelligent Document Processing (IDP) can help enterprises to automate daily document processing operations, and provide assistance to enterprise staff in various aspects of document identification, classification, information extraction, and comparison.

Stamp identification is a very important identification capability in intelligent document processing. With the development of image processing technology, technologies such as optical character recognition (Optical Character Recognition, OCR) have been greatly developed, and with the popularization of paperless office work and electronic office work, seal electronic work has also become a trend. The seal is an important mark letter used by national authorities, social groups and enterprises and public institutions at all levels as a mark and evidence with legal significance, plays an important role in modern social life, and is widely applied to the scenes of contract comparison, input system, warehouse in and out auditing, invoice reimbursement and the like. In the past working links, the seal images need to be checked manually, and the process is tedious. The seal text content can be extracted efficiently by using the artificial intelligence technology to automatically detect and identify the seal, so that the problems of timeliness and accuracy of tasks such as contract management work and the like are solved, and meanwhile, the method can effectively save great labor cost for enterprises and has great practical application value.

Compared with the character recognition problem in the common scene, the seal recognition has some unique characteristics, and is also a difficult point. Firstly, the seal has strong diversity, and is classified according to the types, such as official seal, contract special seal, financial special seal, legal representative seal and the like; there are squares, diamonds, circles, ovals, etc. according to shape. The characters in the square seal are horizontally distributed, the detection and recognition method aiming at the horizontal text is mature, and after the square seal is rotated to the direction of the content level, the content in the square seal can be well recognized by using some common OCR recognition models. However, the common seal contents are arranged in a circular ring shape, and recognition of curved texts like this still faces great challenges at present. Secondly, the seal is often accompanied with various complex backgrounds, in an actual scene, the seal is often stamped on various documents, the background where a seal area is located is complex, interference can be shielded by factors such as characters of the documents, handwritten signatures and the like, the quality of ink paste used during stamping, the stamping force and the like are influenced, the seal image is easy to have uneven colors, the characters are fuzzy, and even human eyes are hard to identify. Therefore, a method for identifying seal content with complex background in various documents is urgently required to be provided to solve the practical problem.

On one hand, under a complex document background, an accurate seal content area is more difficult to detect, the precision of a text detection algorithm is higher, and the detection effect of the conventional text detection algorithm under the scene is not ideal; on the other hand, the identification mode of the curved seal content area also needs to adapt to seal characteristics, and the existing seal image content identification based on Bezier curve correction uses the Bezier curve to fit the seal content area frame, and although the curve can fit any curve in a two-dimensional space, the curve is fit to the Bezier curve from a point set through a calculation method, so that the problem that the fitted curve is not completely matched with an actual text frame exists, and the conditions of cutting the seal content by the curve, excessive distortion of corrected characters and the like can be caused; in addition, there is a seal image content recognition method based on polar coordinate conversion, which requires a large amount of post-processing to acquire and correct seal content areas, and has a problem that successive text areas after correction are divided, and post-processing needs to be continued to be recombined. Still other seal identification methods based on image feature matching do not really identify seal content, and are very poor in universality.

Disclosure of Invention

The invention mainly aims to provide a seal content identification method, a seal content identification system and a seal content identification medium, aiming at solving the problems of seal detection, classification and identification required in various scenes, wherein a pipelining flow from text detection to correction to identification is adopted in a seal text identification part, compared with an end-to-end identification method, the seal content identification method has better adaptability to different seal detection and identification application scenes, wherein a seal content area expansion algorithm based on a deep neural network model is added in a text detection part, a more accurate seal content area boundary can be obtained, and a better effect can be realized under the condition that the training data amount of the whole frame is smaller.

In order to achieve the above object, the present invention provides a method for identifying content of a seal, comprising the steps of:

And detecting the seal position: acquiring a picture with a seal, and executing a seal detection and classification algorithm to detect and position the seal in the picture to obtain the position coordinates and the category of the seal in the image, wherein the seal category comprises a circle, an ellipse, a diamond and a square;

And detecting seal content: cutting out an image area of the seal according to the position coordinates of the seal, and executing a seal content detection algorithm on the image area of the seal to obtain a seal content area represented by a polygon coordinate list;

Correcting the seal content area: ordering the polygonal coordinates, executing a seal content area correction algorithm based on an arc curve on seal content areas with circular and elliptical seal types, and correcting the circular or elliptical seal content areas into rectangles to obtain corrected seal content areas;

and (3) identifying the seal content: and executing a seal content identification algorithm on the diamond or square seal content area or the corrected seal content area, and outputting the text content of the seal.

The further technical scheme of the invention is that the seal position detection step comprises the following steps:

acquiring a feature map of an input image, and constructing a plurality of candidate frames possibly with seals;

Performing binary classification and regression on the candidate frames, wherein a filtering part is judged as a candidate frame of the background;

And performing candidate frame-feature map alignment operation on the rest candidate frames to obtain feature representations of corresponding areas of the candidate frames on the image feature map, and enabling the feature representations to perform multi-classification and frame position offset regression on the rest candidate frames to obtain seal position coordinates and classification.

The further technical scheme of the invention is that the seal content detection step comprises the following steps:

Using all seal images and corresponding category information obtained in the seal detection step as input information of the seal content detection step, and detecting the content of each seal by using a corresponding detection method according to the category;

under the condition that the seal is square or diamond, a horizontal text detection method is used for executing a horizontal text detection algorithm on the seal image, so that the seal content area is obtained;

and under the condition that the seal type is round or elliptical, executing a text detection algorithm with any shape on the seal image by using a text detection method with any shape to obtain the seal content area.

According to a further technical scheme of the invention, under the condition that the seal type is circular or elliptical, executing a text detection algorithm with any shape on a seal image by using a text detection method with any shape, and obtaining the seal content area comprises the following steps:

Executing a text detection algorithm based on an example segmentation idea on a seal image with a circular or elliptical category to obtain a pixel classification probability map and a seal image feature map on the seal image, selecting a larger probability threshold value to ensure that the initial seal content area is in the real seal content area, executing a binarization method on the pixel classification probability map to obtain a binarized image, and executing a seal content area frame generation algorithm on the binarized image to obtain a plurality of initial seal content area frame coordinate sequences;

Extracting feature vectors corresponding to the coordinates from a seal image feature map by using a plurality of initial frame coordinate sequences of seal content areas to obtain feature representations corresponding to a plurality of text examples, inputting the feature representations corresponding to the text examples into a seal content area expansion algorithm based on a depth neural network model to obtain predicted displacement distances and displacement directions of all coordinate points, and applying the displacement distances and the displacement directions of all coordinate points to the corresponding coordinate points to obtain the frame coordinate sequence of the seal content area, wherein the frame coordinate sequence is more accurate than the frame coordinate sequence of the seal content area which is directly based on preset threshold binarization.

The method for correcting the seal content area comprises the following steps of:

when the stamp content area obtained by the horizontal file detection method is rectangular, a rectangular rotation leveling algorithm is executed, and the stamp content area is corrected to the horizontal direction, so that a horizontal stamp content image is obtained;

Executing a polygon shape judging method on the seal content area when judging that the seal content area coordinate sequence forms a polygon to be classified, obtaining the area of the area through the seal content area coordinate sequence, generating a minimum generated rectangle comprising the seal content area, calculating the minimum generated rectangle area, calculating the ratio of the area of the seal content area to the minimum generated rectangle area corresponding to the minimum generated rectangle area, comparing with a set threshold value, judging that the shape of the seal content area is rectangular when the ratio is larger than the threshold value, and judging that the shape of the seal content area is circular arc when the ratio is smaller than the threshold value;

Executing a rectangular rotation leveling algorithm on the rectangular seal content area, and adjusting the rectangular seal content area to the horizontal direction to obtain a horizontal seal content image;

And executing a seal content area correction algorithm based on an arc curve on the arc seal content area, and carrying out straightening correction by utilizing the particularity of the shape similar to a circle of the seal bending text area to obtain a horizontal seal content image.

The method for correcting the seal content area further comprises the following steps:

Executing a post-processing method, rotating the seal image to the forward direction according to the seal content region coordinate sequence, and rearranging and properly expanding the coordinate sequence to obtain a seal content region coordinate sequence which starts from the upper left corner of seal content in the semantic sense and rotates clockwise;

Executing an inner arc coordinate sequence and outer arc coordinate sequence calculation algorithm on the coordinate sequence of the seal content area to obtain two coordinate sequences respectively representing an inner arc and an outer arc, and then executing a circle fitting calculation method to obtain circle center coordinates and radius sizes corresponding to the inner arc and the outer arc and an equation of a circle fitted by the inner arc and the outer arc;

Presetting the width and height of an output image, sequentially selecting pixel points in the output image according to a row-column sequence, calculating the proportion of the horizontal position and the vertical position of the pixel points, using the proportion, determining the point matched with the pixel point in the output image in a seal content area based on the equation of a circle obtained by fitting, the starting point and the end point coordinate information of an inner arc and an outer arc, using an interpolation algorithm, and calculating the pixel value corresponding to the pixel point in the output image based on the pixel value information of the seal content area;

and calculating the pixel value of each pixel point in the output image to obtain a horizontal seal content image.

The invention further adopts the technical scheme that the seal content identification step comprises the following steps:

And executing a character recognition algorithm on the horizontal seal content image to obtain character content in the seal content image.

In order to achieve the above object, the present invention also proposes a seal content recognition system comprising a memory, a processor and a seal content recognition program stored on the processor, which when executed by the processor performs the steps of the method as described above.

To achieve the above object, the present invention also proposes a computer-readable storage medium having stored thereon a stamp content identification program which, when executed by a processor, performs the steps of the method as described in the above embodiments.

Compared with the prior art, the invention has the beneficial effects that: the process of detecting, extracting, correcting and identifying the seal in the assembly line mode makes the invention have smaller requirement on the marking data of the real seal, but can achieve higher accuracy of detecting, classifying, extracting the content and identifying the content, meanwhile, the invention considers that the seal often has more complex background and noise interference under the real scene, so the generated data used in the model training is close to the reality as much as possible, and the adaptability and the effectiveness of the invention under various scenes are ensured. The invention can effectively solve the problem of identifying the seal content in the scenes of contract input systems, warehouse entry auditing, invoice reimbursement and the like, complete the comparison and verification of the names of main objects such as contracts or invoices and the seal content, save a great deal of labor cost and improve the office efficiency of enterprises.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a preferred embodiment of the stamp content identification method of the present invention.

Fig. 2 is a flowchart of a preferred embodiment of step S200 of fig. 1 according to the present invention.

Fig. 3 is a flowchart of stamp content detection when a square or diamond stamp is input in step S300 of fig. 1 according to the present invention.

Fig. 4 is a flowchart of stamp content detection when a circular or oval stamp is input in step S300 of fig. 1 according to the present invention.

Fig. 5 is a correction flowchart when it is determined that the stamp content area is rectangular in step S400 in fig. 1 according to the present invention.

FIG. 6 is a correction flow chart when it is determined that the seal content area is circular arc in step S400 in FIG. 1 according to the present invention;

Fig. 7 is a schematic diagram of a real sample.

The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The invention provides a seal content identification method, as shown in fig. 1, a preferred embodiment of the seal content identification method of the invention comprises the following steps:

And detecting the seal position: and acquiring a picture with a seal, and executing a seal detection and classification algorithm to detect and position the seal in the picture to obtain the position coordinates and the category of the seal in the image, wherein the seal category comprises but is not limited to a circle, an ellipse, a diamond or a square.

And detecting seal content: cutting out an image area of the seal according to the position coordinates of the seal, and executing a seal content detection algorithm on the image area of the seal to obtain a seal content area represented by a polygon coordinate list.

Correcting the seal content area: and ordering the polygonal coordinates, executing a seal content area correction algorithm based on the arc curve on seal content areas with circular and elliptical seal types, correcting the circular or elliptical seal content areas into rectangles, and obtaining corrected seal content areas.

And (3) identifying the seal content: and executing a seal content identification algorithm on the diamond or square seal content area or the corrected seal content area, and outputting the text content of the seal.

Specifically, the seal position detection step includes:

And acquiring a feature map of the input image, and constructing a plurality of candidate frames possibly with seals.

The candidate frames are subjected to binary classification and regression, and the filtering section is determined as a candidate frame of the background.

And performing candidate frame-feature map alignment operation on the rest candidate frames to obtain feature representations of the corresponding areas of the candidate frames on the image feature map, and enabling the feature representations to perform multi-classification and frame position offset regression on the rest candidate frames to obtain seal position coordinates and classification.

The seal content detection step comprises the following steps:

using all seal images and corresponding category information obtained in the seal detection step as input information of the seal content detection step, and detecting the content of each seal by using a corresponding detection method according to the category.

Under the condition that the seal is square or diamond, a horizontal text detection method is used for executing a horizontal text detection algorithm on the seal image, and the seal image is input into a horizontal text detection model to obtain a seal content area. Wherein the horizontal text detection model is trained based on sample data of text detection in a common document.

Under the condition that the seal type is round or elliptical, executing any shape text detection algorithm on the seal image by using any shape text detection method, and inputting the seal image into any shape text detection model to obtain a seal content area. The text detection model with any shape is obtained by pretraining based on a large number of content detection labeling samples of generated seals and fine-tuning training of content detection labeling data of a small number of various real seals.

Under the condition that the seal type is round or elliptical, executing a text detection algorithm with any shape on the seal image by using a text detection method with any shape, and obtaining a seal content area comprises the following steps:

And executing a text detection algorithm based on an example segmentation idea on a seal image with a circular or elliptical category to obtain a pixel classification probability map and a seal image feature map on the seal image, selecting a larger probability threshold value to ensure that the initial seal content area is in the real seal content area, executing a binarization method on the pixel classification probability map to obtain a binarized image, and executing a seal content area frame generation algorithm on the binarized image to obtain a plurality of initial seal content area frame coordinate sequences.

Extracting feature vectors corresponding to the coordinates from a seal image feature map by using a plurality of initial frame coordinate sequences of seal content areas to obtain feature representations corresponding to a plurality of text examples, inputting the feature representations corresponding to the text examples into a seal content area expansion algorithm based on a depth neural network model to obtain predicted displacement distances and displacement directions of all coordinate points, and applying the displacement distances and the displacement directions of all coordinate points to the corresponding coordinate points to obtain the frame coordinate sequence of the seal content area, wherein the frame coordinate sequence is more accurate than the frame coordinate sequence of the seal content area which is directly based on preset threshold binarization.

The seal content area correction step comprises the following steps:

and when the stamp content area obtained by the horizontal file detection method is rectangular, executing a rectangular rotation leveling algorithm, and correcting the stamp content area to the horizontal direction to obtain a horizontal stamp content image.

And when the coordinate sequence of the seal content area is judged to form a polygon to be classified, executing a polygon shape judging method on the seal content area, obtaining the area of the area through the coordinate sequence of the seal content area, generating a minimum generated rectangle comprising the seal content area, calculating the minimum generated rectangle area, calculating the ratio of the area of the seal content area to the corresponding minimum generated rectangle area, comparing with a set threshold value, judging that the shape of the seal content area is rectangular when the ratio is larger than the threshold value, and judging that the shape of the seal content area is circular arc when the ratio is smaller than the threshold value.

And executing a rectangular rotation leveling algorithm on the rectangular seal content area, and adjusting the rectangular seal content area to the horizontal direction to obtain a horizontal seal content image.

And executing a seal content area correction algorithm based on an arc curve on the arc seal content area, and carrying out straightening correction by utilizing the particularity of the shape similar to a circle of the seal bending text area to obtain a horizontal seal content image.

The seal content areas are classified to obtain rectangular seal content areas and circular arc seal content areas, the rectangular seal content areas are corrected to the horizontal direction by using the rectangular rotating leveling method, a circular arc text box is subjected to a post-processing method, seal images are corrected, coordinates are reordered, a small amount of external expansion is performed on the seal content areas, and the correction effect is improved.

The seal content area correction step further comprises:

And executing a post-processing method, rotating the seal image to the forward direction according to the seal content region coordinate sequence, and rearranging and properly expanding the coordinate sequence to obtain the seal content region coordinate sequence which starts from the upper left corner of the seal content in the semantic sense and rotates clockwise.

And executing an inner and outer arc coordinate sequence calculation method on the coordinate sequence of the seal content area according to the reordered coordinate sequence to obtain two coordinate sequences respectively representing the inner arc and the outer arc, and executing a circle fitting calculation method to obtain the circle center coordinates and the radius corresponding to the inner and outer arcs and the equation of the circle fitted by the inner and outer arcs.

The post-treatment method specifically comprises the following steps:

dividing an arc-shaped seal content area frame into an upper curve and a lower curve, obtaining points at the right middle positions of the two curves, respectively calculating the included angle between the circle center and the connecting line of the two points in the horizontal direction and the vertical direction to obtain the average value of the angles in the horizontal direction and the vertical direction, rotating the whole seal to the positive direction according to the two angles, obtaining a rotated coordinate sequence through a coordinate rotation calculation method, classifying each point into an inner circular arc or an outer circular arc coordinate list through the distance relation between each point and the current circle center, and finally obtaining the respective coordinate sequence of the inner circular arc and the outer circular arc;

The forward direction refers to the direction of the seal, which is almost right-left symmetric, and the characters are forward direction, and are not reversed, and the direction is considered to be forward direction.

The coordinate sequence of the inner arc and the outer arc, the circle center coordinate and the radius obtained by fitting and the original seal image are used as inputs of a seal content correction method module, the size of an output rectangular image frame of the seal content correction method module is set, each pixel position in the output image corresponds to the position in the seal content area through an equation of the circle obtained by fitting, and the value of each pixel point in the output rectangular image frame is calculated by using an interpolation method based on the corresponding relation between the output rectangular image and the pixel position in the seal content area.

The specific process for correcting the arc curve comprises the following steps:

The width and the height of an output image are preset, pixel points are sequentially selected in the output image according to a row-column sequence, the proportion of the horizontal position and the vertical position of the pixel points is calculated, the proportion is used, the point matched with the pixel points in the output image is determined in a seal content area based on the equation of a circle obtained by fitting, the starting point and the end point coordinate information of an inner arc and an outer arc, and the pixel value corresponding to the pixel points in the output image is calculated and obtained by using an interpolation algorithm and pixel value information based on the seal content area.

And calculating the pixel value of each pixel point in the output image, and finally obtaining a corrected seal content horizontal rectangular text line image to obtain a horizontal seal content image.

The seal content identification step comprises the following steps:

and executing a character recognition algorithm on the horizontal seal content image to obtain character content in the seal content image.

Specifically, the stamp content recognition algorithm is sequentially executed on all the corrected stamp images, and the stamp content recognition algorithm is output as text content in the horizontal stamp content images.

The method for identifying the content of the seal according to the invention is described in further detail below with reference to fig. 1 to 7.

Referring to fig. 1, fig. 1 is a flowchart of a seal content identification method according to the present invention. As shown in fig. 1, the seal content identification method according to the embodiment of the invention includes the following steps:

S100, acquiring an input file, and converting the original input file into standard input image data.

The original file can be in various formats such as PNG, JPEG, TIFF and PDF, and is subjected to format standardization processing, if the original file is a file with a picture type, the original file is directly read, if the original file is a file with a PDF and the like, and the original file also comprises a plurality of pages of files, and the original file needs to be split and converted into images and then read, so that the universality of the method for various input files is realized.

And S200, executing a seal detection and classification algorithm on the input picture, detecting the seal existing in the picture, and obtaining the position coordinates and the category of the seal.

Specifically, the seal detection and classification algorithm is used for detecting and classifying the seal of the input image, supported seal types include square, diamond, round and oval, and the position information refers to coordinates of four points of a rectangular area containing the seal. The main stream target detection algorithms used by the seal detection algorithm mainly can be divided into Two types, namely One-Stage and Two-Stage types of algorithms, wherein the former is YOLO (You OnlyLookOnce) series, and the position and the type of a target are predicted by the algorithms directly through image features extracted by a network, so that the speed is high, but the accuracy is difficult to ensure; the latter is represented by R-CNN series algorithm, which uses image features to generate candidate frames of some detected objects, then classifies and regresses the candidate frames to obtain final detection results, which have slower speed but higher accuracy.

And S300, extracting a seal image area from the input image by using seal position coordinates, and executing a seal content detection algorithm on the seal image area to obtain a seal content area.

Specifically, a seal area is cut out from an original image by using position coordinates of a seal, a seal content detection algorithm is executed on the area to obtain the seal content area, different seal content detection algorithms are used for seals of different types, the seal content detection algorithms are divided into a horizontal text detection algorithm and an arbitrary shape text detection algorithm, one or more coordinate sequences representing the seal content area are finally obtained, and each coordinate sequence represents one content area of the seal.

S400, using different correction methods for seal content areas with different shapes to obtain horizontal seal content images.

S500, performing character recognition on the horizontal seal content image to obtain the character content of the seal.

Specifically, the text recognition method uses a horizontal text line image recognition algorithm CRNN, uses generated seal text image annotation data to perform fine adjustment on a general text recognition model trained by a large number of scene texts to obtain a text recognition model which is more suitable for the real application scene of the invention, and inputs a text image into the text recognition model to obtain a recognition result.

As an embodiment of the present invention, as shown in fig. 2, step S200 specifically includes:

S210, acquiring an input image, extracting image features by using a detection model backbone network, and constructing a candidate frame set.

Specifically, the processed image data is input to a seal detection module, image features are extracted by using a detection model backbone network, a feature map is obtained, and adaptive multi-scale and multi-scale candidate frames are set for each point in the feature map based on the graph proportion and the size of square, diamond, elliptic and circular seals, so that a plurality of candidate frames are obtained.

S220, performing binary classification and regression on the candidate frames, filtering part of the candidate frames, performing candidate frame-feature map alignment operation on the rest of the candidate frames, extracting the features of each candidate frame, and finally performing multi-classification and frame coordinate regression on the candidate frames to obtain the detected position information and the detected category information of the seal.

Specifically, binary classification and regression are performed on the candidate frames by using an RPN network, a part of the candidate frames are filtered, other candidate frames are updated according to regression results, candidate frame-feature map alignment operation is performed on the remaining candidate frames, features of each candidate frame are extracted, and finally multi-classification and frame coordinate regression are performed on the candidate frames, so that detected position information and category information of the seal are obtained.

S230, performing multi-classification and boundary coordinate regression on the filtered candidate frames to obtain target position coordinates and classes determined to be the seal.

In the seal content detection step, the input seal images can be square or diamond, the text contents of the input seal images are considered to be distributed in the horizontal direction, and the horizontal seal content areas are required to be detected; the input seal images can also be round or oval, the text content of the input seal images is mainly considered to be curved circular arc-shaped text areas, and the curved seal content areas need to be detected. As one embodiment of the present invention, the content detection of the stamp includes the steps of:

s310, cutting out an image area mainly containing the seal from the original image according to the position coordinates of the seal.

S310, judging the type of the seal corresponding to the position coordinate, and obtaining the result of the seal detection and classification algorithm.

As shown in fig. 3, if the stamp category is square or diamond, the following steps S330 to S331 are performed:

s330, executing a horizontal text detection algorithm, and inputting the seal image area into a horizontal text detection model.

S331, obtaining a coordinate sequence of a rectangular text box representing the seal content area in the seal image area.

Specifically, the horizontal text detection model is obtained by training based on sample data of text detection in a normal document, and the position coordinates of the rectangular text box are the coordinates of four vertexes of the rectangular text box.

As shown in fig. 4, if the stamp category is circular or elliptical, the following steps S340 to S341 are performed:

s340, executing a text detection algorithm with any shape, and inputting the seal image area into a text detection model with any shape.

S341, obtaining a polygonal coordinate sequence representing the seal content area in the seal image area.

Specifically, the text detection model with any shape is obtained by pretraining a large number of content detection labeling samples of generated seals and performing fine tuning training on content detection labeling data of a small number of various real seals, and the polygonal coordinate sequence is a coordinate sequence of n points on the boundary of a seal content area, wherein n is a preset constant.

In the seal content area correction step, the input seal content area may be rectangular, and the rectangular seal content area needs to be rotationally corrected to the horizontal direction to obtain a horizontal seal content image; the input text box image may also be circular arc, and the circular arc seal content area needs to be straightened into a horizontal seal content image. As one embodiment of the present invention, the stamp content area correction includes the steps of:

S410, acquiring an input seal content area coordinate sequence and original image data.

S420, judging the shape of the input seal content area. The ratio of the polygonal area formed by the coordinate sequences to the area of the smallest generated rectangle is compared with a set threshold value to judge, so that the input seal content area can be classified as a rectangle or an arc.

As shown in fig. 5, if the stamp content area is rectangular in shape, the following steps S430 to S431 are performed:

s430, executing a rectangular text box correction method, and calculating to obtain the angle required by the input rectangular text box to rotate to the horizontal direction.

S431, rotating the inclined rectangular text box to the horizontal direction using the rotation angle and affine transformation method.

As shown in fig. 6, if the text box type is circular arc, the following steps S440 to S444 are performed:

s440, executing a seal content area correction algorithm based on the circular arc curve according to the characteristics of the circular and elliptical seal content areas.

S441, a post-processing algorithm aiming at the circular arc seal content area is executed, all coordinate points are rearranged, and all coordinate points are subjected to outward expansion operation, so that the area included by the coordinate point sequence can cover the needed seal content part, and the sequence of the finally obtained coordinate points starts from the left upper corner of the first character in the forward reading sequence of the seal content area and rotates clockwise to the left lower corner of the first character in the forward reading sequence.

S442, dividing the coordinate sequence into two sequences by using the processed coordinate sequence and the calculation method, respectively corresponding to the inner arc and the outer arc of the circular arc curve, and calculating an equation of a circle corresponding to the inner arc and the outer arc curve by using the two coordinate sequences based on the least square method, wherein the equation comprises the center coordinate and the radius of the circle.

S443, correcting and straightening are completed by using an equation of a fitting circle, a coordinate sequence and an original seal content area image, wherein the specific correction process is that the width and the height of an output image are preset, pixel points are sequentially selected in the output image according to a row-column sequence, the proportion of the horizontal position and the vertical position of the pixel points is calculated, the point matched with the pixel points in the output image in the seal content area is found by using the proportion based on an equation of an inner arc curve and an outer arc curve, the pixel value of the pixel points in the output image is calculated by using an interpolation method and pixel value information of the seal content area, and finally the corrected horizontal seal content image is obtained.

In the embodiment of seal content detection and correction, the real scene application flow is shown in fig. 7, and specifically includes:

Detecting to obtain a seal content area by using a text detection method with any shape;

Using a seal content area correction method based on an arc curve for the arc seal content area, using a calculation method for the seal content area judged to be the arc to obtain coordinate sequences respectively belonging to inner and outer arcs, using the coordinate sequences to perform inner and outer arc curve fitting to obtain an equation of a circle corresponding to the inner and outer arc curves, wherein the visualized fitting inner and outer arc curves are shown as blue lines and green lines in fig. 5, and the red coordinate points are coordinate points output by a text detection algorithm with any shape;

sequentially selecting pixel points in an output image with a preset size according to a row-column sequence, obtaining the corresponding relation between the pixel points and the pixel points in the seal content area through an inner arc curve equation and an outer arc curve equation, calculating to obtain the pixel values of the pixel points in the output image through an interpolation method, and finally obtaining the seal content image with corrected level.

And (3) using a rectangular text box correction method for the rectangular seal content area, using a calculation method to obtain an angle required to be rotated, using the angle and an affine transformation method to rotate the inclined rectangular text area back to the horizontal direction, and finally obtaining a corrected result.

Specifically, in the true example of fig. 7, a seal content area correction algorithm based on the arc curve is performed on the arc-shaped seal content area "1234 futures limited", and rectangular text box correction algorithms are performed on the rectangular seal content areas "futures brokerage contract opening special seal" and "1".

The invention provides a seal content identification method in an image, which realizes seal detection, classification, content detection and content identification in the image by using related methods such as target detection, text detection and text identification, designs a circular arc curve correction method which is suitable for correcting the curved text of the circular arc shape based on the text morphological characteristics of circular and elliptic seals, obtains better identification accuracy under the condition of less data resource investment, greatly reduces the investment of manual resources in seal identification verification in real scenes, improves office efficiency of related departments and effectively solves the practical problem.

Compared with the prior art, the invention has the beneficial effects that: the process of detecting, extracting, correcting and identifying the seal in the assembly line mode makes the invention have smaller requirement on the marking data of the real seal, but can achieve higher accuracy of detecting, classifying, extracting the content and identifying the content, meanwhile, the invention considers that the seal often has more complex background and noise interference under the real scene, so the generated data used in the model training is close to the reality as much as possible, and the adaptability and the effectiveness of the invention under various scenes are ensured. The invention can effectively solve the problem of identifying the seal content in the scenes of contract input systems, warehouse entry auditing, invoice reimbursement and the like, complete the comparison and verification of the names of main objects such as contracts or invoices and the seal content, save a great deal of labor cost and improve the office efficiency of enterprises.

In order to achieve the above objective, the present invention further provides a seal content identification system, where the system includes a memory, a processor, and a seal content identification program stored on the processor, and the seal content identification program executes steps of the method described in the above embodiment when executed by the processor, which is not described herein again.

To achieve the above object, a computer readable storage medium of the present invention stores a stamp content identification program thereon, and the stamp content identification program is executed by a processor to perform the steps of the method described in the above embodiments, which will not be described herein.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the invention, and all equivalent structural changes made by the specification and drawings of the present invention or direct/indirect application in other related technical fields are included in the scope of the present invention.

Claims (9)

1. A method for identifying content of a stamp, the method comprising the steps of:

And detecting the seal position: acquiring a picture with a seal, and executing a seal detection and classification algorithm to detect and position the seal in the picture to obtain the position coordinates and the category of the seal in the image, wherein the seal category comprises a circle, an ellipse, a diamond and a square;

And detecting seal content: cutting out an image area of the seal according to the position coordinates of the seal, and executing a seal content detection algorithm on the image area of the seal to obtain a seal content area represented by a polygon coordinate list;

Correcting the seal content area: ordering the polygonal coordinates, executing a seal content area correction algorithm based on an arc curve on seal content areas with circular and elliptical seal types, and correcting the circular or elliptical seal content areas into rectangles to obtain corrected seal content areas;

and (3) identifying the seal content: and executing a seal content identification algorithm on the diamond or square seal content area or the corrected seal content area, and outputting the text content of the seal.

2. The seal content identification method according to claim 1, wherein the seal position detection step includes:

acquiring a feature map of an input image, and constructing a plurality of candidate frames possibly with seals;

Performing binary classification and regression on the candidate frames, wherein a filtering part is judged as a candidate frame of the background;

And performing candidate frame-feature map alignment operation on the rest candidate frames to obtain feature representations of corresponding areas of the candidate frames on the image feature map, and enabling the feature representations to perform multi-classification and frame position offset regression on the rest candidate frames to obtain seal position coordinates and classification.

3. The seal content identification method according to claim 1, wherein the seal content detection step includes:

Using all seal images and corresponding category information obtained in the seal detection step as input information of the seal content detection step, and carrying out content detection on each seal according to categories by using a corresponding detection method:

under the condition that the seal is square or diamond, a horizontal text detection method is used for executing a horizontal text detection algorithm on the seal image, so that the seal content area is obtained;

and under the condition that the seal type is round or elliptical, executing a text detection algorithm with any shape on the seal image by using a text detection method with any shape to obtain the seal content area.

4. A seal content recognition method according to claim 3, wherein, in the case where the seal type is circular or elliptical, the step of performing an arbitrary shape text detection algorithm on the seal image using an arbitrary shape text detection method, to obtain the seal content area comprises:

Executing a text detection algorithm based on an example segmentation idea on a seal image with a circular or elliptical category to obtain a pixel classification probability map and a seal image feature map on the seal image, selecting a larger probability threshold value to ensure that the initial seal content area is in the real seal content area, executing a binarization method on the pixel classification probability map to obtain a binarized image, and executing a seal content area frame generation algorithm on the binarized image to obtain a plurality of initial seal content area frame coordinate sequences;

Extracting feature vectors corresponding to the coordinates from a seal image feature map by using a plurality of initial frame coordinate sequences of seal content areas to obtain feature representations corresponding to a plurality of text examples, inputting the feature representations corresponding to the text examples into a seal content area expansion algorithm based on a depth neural network model to obtain predicted displacement distances and displacement directions of all coordinate points, and applying the displacement distances and the displacement directions of all coordinate points to the corresponding coordinate points to obtain the frame coordinate sequence of the seal content area, wherein the frame coordinate sequence is more accurate than the frame coordinate sequence of the seal content area which is directly based on preset threshold binarization.

5. A seal content recognition method according to claim 3, wherein the seal content area correction step includes:

when the stamp content area obtained by the horizontal file detection method is rectangular, a rectangular rotation leveling algorithm is executed, and the stamp content area is corrected to the horizontal direction, so that a horizontal stamp content image is obtained;

Executing a polygon shape judging method on the seal content area when judging that the seal content area coordinate sequence forms a polygon to be classified, obtaining the area of the area through the seal content area coordinate sequence, generating a minimum generated rectangle comprising the seal content area, calculating the minimum generated rectangle area, calculating the ratio of the area of the seal content area to the minimum generated rectangle area corresponding to the minimum generated rectangle area, comparing with a set threshold value, judging that the shape of the seal content area is rectangular when the ratio is larger than the threshold value, and judging that the shape of the seal content area is circular arc when the ratio is smaller than the threshold value;

Executing a rectangular rotation leveling algorithm on the rectangular seal content area, and adjusting the rectangular seal content area to the horizontal direction to obtain a horizontal seal content image;

And executing a seal content area correction algorithm based on an arc curve on the arc seal content area, and carrying out straightening correction by utilizing the particularity of the shape similar to a circle of the seal bending text area to obtain a horizontal seal content image.

6. The seal content identification method according to claim 5, wherein the seal content area correction step further comprises:

Executing a post-processing method, rotating the seal image to the forward direction according to the seal content region coordinate sequence, and rearranging and properly expanding the coordinate sequence to obtain a seal content region coordinate sequence which starts from the upper left corner of seal content in the semantic sense and rotates clockwise;

Executing an inner arc coordinate sequence and outer arc coordinate sequence calculation algorithm on the coordinate sequence of the seal content area to obtain two coordinate sequences respectively representing an inner arc and an outer arc, and then executing a circle fitting calculation method to obtain circle center coordinates and radius sizes corresponding to the inner arc and the outer arc and an equation of a circle fitted by the inner arc and the outer arc;

Presetting the width and height of an output image, sequentially selecting pixel points in the output image according to a row-column sequence, calculating the proportion of the horizontal position and the vertical position of the pixel points, using the proportion, determining the point matched with the pixel point in the output image in a seal content area based on the equation of a circle obtained by fitting, the starting point and the end point coordinate information of an inner arc and an outer arc, using an interpolation algorithm, and calculating the pixel value corresponding to the pixel point in the output image based on the pixel value information of the seal content area;

and calculating the pixel value of each pixel point in the output image to obtain a horizontal seal content image.

7. The seal content identification method according to claim 6, wherein the seal content identification step includes:

And executing a character recognition algorithm on the horizontal seal content image to obtain character content in the seal content image.

8. A stamp content identification system, characterized in that the system comprises a memory, a processor and a stamp content identification program stored on the processor, which stamp content identification program, when being executed by the processor, performs the steps of the method according to any of claims 1 to 7.

9. A computer-readable storage medium, on which a stamp content identification program is stored, which when executed by a processor performs the steps of the method according to any one of claims 1 to 7.