WO2022183907A1

WO2022183907A1 - Image processing method and apparatus, intelligent invoice recognition device, and storage medium

Info

Publication number: WO2022183907A1
Application number: PCT/CN2022/076400
Authority: WO
Inventors: 徐青松; 李青
Original assignee: 杭州睿胜软件有限公司
Priority date: 2021-03-04
Filing date: 2022-02-16
Publication date: 2022-09-09
Also published as: CN113033325A

Abstract

At least one embodiment of the present disclosure provides an image processing method, an image processing apparatus, an intelligent invoice recognition device, and a storage medium. The image processing method comprises: obtaining an input image, wherein the input image comprises an input seal; recognizing the input seal in the input image to obtain a seal image, wherein the seal image comprises an intermediate seal corresponding to the input seal; performing feature extraction processing on the seal image to obtain a feature point image; processing the seal image and the feature point image to obtain a first unfolding point, a second unfolding point, and a unfolding line; by using the connecting line between the first unfolding point and the second unfolding point as an unfolding reference line and the first unfolding point as an unfolding starting point, unfolding the input seal horizontally along the unfolding line to obtain an unfolded seal image; performing area recognition processing on the unfolded seal image to determine a first intermediate object area; and performing object recognition processing on the first intermediate object area to obtain a first recognition result.

Description

Image processing method and device, intelligent invoice recognition device and storage medium

technical field

Embodiments of the present disclosure relate to an image processing method, an image processing apparatus, an intelligent invoice recognition device, and a non-transitory computer-readable storage medium.

Background technique

Since the irregularly arranged characters are arcs, curved surfaces or have a perspective effect, the recognition of irregularly arranged characters (for example, the recognition of arcuate characters in images of seals such as official seals or invoice seals or other types of arc character recognition, etc. ) is not accurate. Irregularly arranged text recognition has always been a technical difficulty in the field of text recognition.

SUMMARY OF THE INVENTION

At least one embodiment of the present disclosure provides an image processing method, including: acquiring an input image, wherein the input image includes an input seal, and the input seal includes a first object; identifying the input seal in the input image, To obtain a seal image, wherein, the seal image includes an intermediate seal corresponding to the input seal; feature extraction processing is performed on the seal image to obtain a feature point image; the seal image and the feature point image are processed. processing to obtain a first unfolding point, a second unfolding point and a unfolding line; taking the connecting line between the first unfolding point and the second unfolding point as the unfolding reference line and the first unfolding point as the unfolding At the starting point, the input seal is laterally expanded along the expansion line to obtain an expanded seal image; region recognition processing is performed on the expanded seal image to determine the first intermediate object region in the expanded seal image, wherein, The region corresponding to the first intermediate object region in the input image is the first object region, and the first object is located in the first object region; object recognition processing is performed on the first intermediate object region to obtain The identification obtains the first identification result.

Optionally, in the image processing method provided by an embodiment of the present disclosure, the pixels corresponding to the middle seal in the seal image have a first pixel value, and the seal image except for the pixels corresponding to the middle seal has a first pixel value. The pixels of have a second pixel value, and the first pixel value and the second pixel value are different.

Optionally, in the image processing method provided by an embodiment of the present disclosure, identifying the input seal in the input image to obtain a seal image includes: using an image segmentation model to identify the input image to obtain The initial seal pixels corresponding to the input seal; the initial seal pixels are blurred to obtain a seal pixel mask area; according to the seal pixel mask area, determine the corresponding input seal in the input image. pixel; set the pixel value of the pixel corresponding to the input seal in the input image to the first pixel value and set the pixel value of the pixel other than the pixel corresponding to the input seal in the input image to the desired value. the second pixel value to obtain the stamp image.

Optionally, in the image processing method provided by an embodiment of the present disclosure, the unfolding line includes a first annular unfolding line, and the first annular unfolding line is an edge line of the input stamp. The feature point image is processed to obtain the first unfolding point, the second unfolding point and the unfolding line, including: processing the seal image and the feature point image based on the algorithm of OpenCV to obtain the initial second unfolding point and an initial first annular unfolding line, wherein the initial second unfolding point and the initial first annular unfolding line are located in the seal image; processing to determine a characteristic object area corresponding to the first object in the seal image, determine an opening area in the seal image based on the characteristic object area, obtain any point in the opening area, and based on the The arbitrary point and the initial first annular expansion line are used to determine the initial first expansion point, wherein the initial first expansion point is located in the seal image, and the arbitrary point, the initial first expansion point and all the The line segment between any two points in the initial second expansion point does not overlap with the feature object area, and the initial first expansion point, the initial second expansion point and the initial first expansion point A circular unfolding line is mapped from the stamp image to the input image to obtain the first unfolding point, the second unfolding point and the first circular unfolding line.

Optionally, in the image processing method provided by an embodiment of the present disclosure, determining the initial first unfolding point based on the any point and the initial first annular unfolding line includes: obtaining the initial first unfolding point based on the any point A point corresponding to the any point on the initial first annular unfolding line is used as the initial first unfolding point.

Optionally, in the image processing method provided by an embodiment of the present disclosure, the unfolding line includes a first annular unfolding line and a second annular unfolding line, and the first annular unfolding line is an edge line of the input stamp, In the input image, the second annular development line is located in an area surrounded by the first annular development line, and the first object area is located in the area surrounded by the first annular development line and the second annular development line In the annular area formed, the seal image and the feature point image are processed to obtain the first unfolding point, the second unfolding point and the unfolding line, including: using an algorithm based on OpenCV to analyze the seal image and the feature point image. point images are processed to obtain an initial first annular development line and an initial second annular development line, wherein the initial first annular development line and the initial second annular development line are located in the stamp image; through the development point The extraction model processes the seal image and the feature point image to determine a feature object area corresponding to the first object in the seal image, and determines an opening area in the seal image based on the feature object area , obtain any point in the opening area, and determine the initial first expansion point and the initial second expansion point based on the arbitrary point, the initial first annular expansion line and the initial second annular expansion line, wherein , the initial first unfolding point and the initial second unfolding point are located in the stamp image, and any two points among the any point, the initial first unfolding point and the initial second unfolding point are located The connecting line segment between is not overlapped with the feature object area; the initial first unfolding point, the initial second unfolding point, the initial first circular unfolding line and the initial second circular unfolding line are changed from The stamp image is mapped to the input image to obtain the first expansion point, the second expansion point, the first annular expansion line and the second annular expansion line.

Optionally, in the image processing method provided by an embodiment of the present disclosure, based on the arbitrary point, the initial first annular development line and the initial second annular development line, an initial first development point and an initial first development point are determined. Two expansion points, including: based on the arbitrary point, acquiring a point on the initial first annular expansion line corresponding to the arbitrary point as the initial first expansion point; based on the arbitrary point, acquiring the initial first expansion point A point corresponding to the any point on the two-ring unfolding line is used as the initial second unfolding point.

Optionally, in the image processing method provided by an embodiment of the present disclosure, the initial first unfolding point, the initial second unfolding point, and the any point are located on the same straight line.

Optionally, in the image processing method provided by an embodiment of the present disclosure, the any point is a center point of the opening area.

Optionally, in the image processing method provided by an embodiment of the present disclosure, the first annular expansion line is expanded into a straight line in the expanded seal image.

Optionally, in the image processing method provided by an embodiment of the present disclosure, the first object is text, and the shape of the first object area is an arc.

Optionally, in the image processing method provided by an embodiment of the present disclosure, the shape of the input stamp is a circle, and the second expansion point is the center of the circle; or, the shape of the input stamp is an ellipse, and the second expansion point is the midpoint of the line connecting the two foci of the ellipse.

Optionally, the image processing method provided by an embodiment of the present disclosure further includes: determining a center point of the first intermediate object area; an intermediate object region is mapped back to the input image to determine the first object region, and a center point of the first intermediate object region is mapped back to the input image to determine a center point of the first object region; determining The center point of the input seal; the correction angle used to correct the input image is determined by the center point of the first object area and the center point of the input seal; the input image is corrected based on the correction angle Correction is performed to obtain a corrected input image.

Optionally, in the image processing method provided by an embodiment of the present disclosure, the input stamp further includes a second object, and the image processing method further includes: performing region recognition processing on the corrected input image to determine The second intermediate object area, wherein the area corresponding to the second intermediate object area in the input image is the second object area, and the second object is located in the second object area; The object area is subjected to object recognition processing to obtain a second recognition result.

Optionally, in the image processing method provided by an embodiment of the present disclosure, acquiring an input image includes: acquiring an original image, wherein the original image includes an original seal; Determining a seal area, marking the seal area through a seal labeling frame, and slicing the seal labeling frame to obtain an intermediate input image, wherein the original seal is located in the seal area, and the seal labeling frame includes In the seal area, the intermediate input image includes the original seal, and the intermediate input image is processed to remove interference pixels in the intermediate input image to obtain the input image, wherein the interference pixels include Pixels of interfering objects in the intermediate input image that do not belong to the original seal, and the input seal corresponds to the original seal.

At least one embodiment of the present disclosure further provides an image processing apparatus, including: a memory for non-transitory storage of computer-readable instructions; and a processor for executing the computer-readable instructions, the computer-readable instructions being executed by The processor executes the image processing method according to any one of the above embodiments when running.

At least one embodiment of the present disclosure further provides an intelligent invoice recognition device, including: an image acquisition component for acquiring an invoice image of a paper invoice; a memory for storing the invoice image and computer-readable instructions; a processor for using upon reading the invoice image and determining the input image based on the invoice image, and executing the computer readable instructions, the computer readable instructions being executed by the processor to execute the above described embodiments image processing method.

At least one embodiment of the present disclosure further provides a non-transitory computer-readable storage medium for non-transitory storage of computer-readable instructions, which, when executed by a computer, can execute any of the foregoing embodiments. image processing method.

Description of drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the accompanying drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limit the present disclosure. .

FIG. 1 is a schematic flowchart of an image processing method provided by some embodiments of the present disclosure;

2A is a schematic diagram of an original image provided by some embodiments of the present disclosure;

2B is a schematic diagram of an intermediate input image determined based on the original image shown in FIG. 2A;

2C is a schematic diagram of an input image obtained by identifying the intermediate input image shown in FIG. 2B;

2D is a schematic diagram of a seal image obtained by recognizing the input image shown in FIG. 2C;

2E is a schematic diagram of a feature point image obtained by performing feature extraction processing on the seal image shown in FIG. 2D;

2F is a schematic diagram of another input image provided by some embodiments of the present disclosure;

2G is another schematic diagram of a seal image obtained by recognizing the input image shown in FIG. 2C;

Fig. 2H is the schematic diagram of the expanded seal image obtained by expanding the input seal in Fig. 2C;

Fig. 2I is the schematic diagram of the first intermediate object region obtained by region recognition processing to the unfolded seal image in Fig. 2H;

2J is a schematic diagram of a first recognition result obtained by performing object recognition processing on the first intermediate object region in FIG. 2I;

3A is a schematic diagram of another original image provided by some embodiments of the present disclosure;

3B is a schematic diagram of an intermediate input image determined based on the original image shown in FIG. 3A;

3C is a schematic diagram of an input image obtained by identifying the intermediate input image shown in FIG. 3B;

3D is a schematic diagram of a seal image obtained by recognizing the input image shown in FIG. 3C;

3E is a schematic diagram of a feature point image obtained by performing feature extraction processing on the seal image shown in FIG. 3D;

3F is a schematic diagram of an expanded seal image obtained by expanding the input seal in FIG. 3C;

3G is a schematic diagram of a first intermediate object region obtained by performing region identification processing on the expanded seal image in FIG. 3F;

3H is a schematic diagram of a first recognition result obtained by performing object recognition processing on the first intermediate object region in FIG. 3G;

FIG. 4 is a schematic block diagram of an image processing apparatus according to some embodiments of the present disclosure;

FIG. 5 is a schematic block diagram of an intelligent invoice recognition device according to some embodiments of the present disclosure;

FIG. 6 is a schematic diagram of a storage medium provided by some embodiments of the present disclosure.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, embodiments of the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present disclosure.

Unless otherwise defined, technical or scientific terms used in this disclosure shall have the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure belongs. As used in this disclosure, "first," "second," and similar terms do not denote any order, quantity, or importance, but are merely used to distinguish the various components. "Comprises" or "comprising" and similar words mean that the elements or things appearing before the word encompass the elements or things recited after the word and their equivalents, but do not exclude other elements or things. Words like "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "Down", "Left", "Right", etc. are only used to indicate the relative positional relationship. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits a detailed description of some well-known functions and well-known components.

The images of seals such as official seals or invoice seals have irregularly arranged characters, for example, arc characters. At present, the recognition of these arc characters is not accurate. In addition, if the seal is tilted when stamping, it will also cause the regularly arranged characters in the image corresponding to the seal, such as horizontally arranged characters or vertically arranged characters, will also be slanted or reversed, making it impossible to judge the seal. forward direction, resulting in inaccurate recognition.

At least one embodiment of the present disclosure provides an image processing method, an image processing apparatus, an intelligent invoice recognition device, and a non-transitory computer-readable storage medium. The image processing method includes: acquiring an input image, wherein the input image includes an input seal, and the input seal includes a first object; identifying the input seal in the input image to obtain a seal image, wherein the seal image includes an intermediate seal corresponding to the input seal; Perform feature extraction processing on the seal image to obtain the feature point image; process the seal image and the feature point image to obtain the first unfolding point, the second unfolding point and the unfolding line; take the difference between the first unfolding point and the second unfolding point. The connecting line between them is used as the unfolding reference line and the first unfolding point is the unfolding starting point, and the input stamp is horizontally unfolded along the unfolding line to obtain the unfolding seal image; The first intermediate object area, wherein the area corresponding to the first intermediate object area in the input image is the first object area, and the first object is located in the first object area; object recognition processing is performed on the first intermediate object area, so as to obtain The first recognition result.

The image processing method can well realize the recognition of irregularly arranged objects (eg, characters, etc.) in the input image, improve the accuracy of identifying the irregularly arranged objects, and obtain accurate recognition results.

It should be noted that, in the embodiments of the present disclosure, “irregularly arranged objects” may mean that multiple objects (eg, characters) are not arranged in a row or column, that is, multiple objects are not arranged along the same straight line , for example, the centers of multiple objects are arranged along a curve (eg, a wavy line) or a polyline, etc.

The image processing method provided by the embodiment of the present disclosure can be applied to the image processing apparatus provided by the embodiment of the present disclosure, and the image processing apparatus can be configured on an electronic device. The electronic device may be a personal computer, a mobile terminal, etc., and the mobile terminal may be a hardware device such as a mobile phone and a tablet computer.

The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

FIG. 1 is a schematic flowchart of an image processing method provided by some embodiments of the present disclosure, FIG. 2A is a schematic diagram of an original image provided by some embodiments of the present disclosure, and FIG. 2B is a determination based on the original image shown in FIG. 2A . The intermediate input image shown in FIG. 2C is an input image obtained by recognizing the intermediate input image shown in FIG. 2B . 3A is a schematic diagram of another original image provided by some embodiments of the present disclosure, FIG. 3B is an intermediate input image determined based on the original image shown in FIG. 3A , and FIG. 3C is obtained by identifying the intermediate input image shown in FIG. 3B the input image.

As shown in FIG. 1 , first, in step S10 of the image processing method provided by the embodiment of the present disclosure, an input image is acquired.

For example, in step S10, the input image includes an input seal, for example, the input seal may be various types of seals such as contract-specific seals, invoice-specific seals, and the like. The input image can be any image that includes a seal, for example, as shown in FIG. 2C, in some embodiments, the input image can be an image including a company seal, as shown in FIG. 3C, in other embodiments, the input image can be For including the image of the special stamp for the invoice. The input stamp may be a regular shape stamp such as a circular stamp, an oval stamp, a polygon stamp (for example, a rectangular stamp), or an irregular shape stamp. The input image shown in FIG. 2C includes a circular seal, and the input image shown in FIG. 3C includes an oval seal.

It should be noted that the present disclosure is not limited to this, and the input image may also be a document image or the like.

For example, the input seal includes a first object, the first object may be a character, and the character may be a number, a Chinese character (Chinese characters, Chinese words, etc.), foreign characters (eg, foreign letters, foreign words, etc., such as English, Japanese, Korean, German, etc.), special characters (eg, percent sign "%"), punctuation marks, etc. In addition, the characters may also include graphics (eg, circles, rectangles, etc.), and the like. For example, in some embodiments, the first object may be text. As shown in FIG. 2C and FIG. 3C , the first object may include a plurality of characters arranged irregularly, and the centers of the plurality of characters are arranged in a curve, for example , arranged in an arc.

For example, as shown in Fig. 2C, the first object includes "Hangzhou Ruisheng Software Co., Ltd.", and the centers of the characters in "Hangzhou Ruisheng Software Co., Ltd." are arranged on an arc line; as shown in Fig. 3C, the first object An object includes "Hangzhou Ruisheng Software Co., Ltd.", and the centers of the characters in "Hangzhou Ruisheng Software Co., Ltd." are arranged on an elliptical arc.

For example, in some embodiments, step S10 includes: acquiring an original image; processing the original image through a seal area recognition model to determine the seal area, marking the seal area through the seal annotation frame, and slicing the seal annotation frame to Obtain an intermediate input image; process the intermediate input image to remove interfering pixels in the intermediate input image to obtain an input image.

For example, both the original image and the intermediate input image include the original stamp, the original stamp is located within the stamp area, and the stamp callout box includes the stamp area. The area of the original seal can be marked in the original image, and then the area corresponding to the original seal can be cut from the original image to obtain a separate intermediate input image, so that in subsequent operations, the cut intermediate input image can be directly obtained. to be processed.

For example, as shown in FIG. 2A , in some embodiments, the original image may be an image including a company seal, and the intermediate input image shown in FIG. 2B can be obtained by processing the original image shown in FIG. 2A . As shown in FIG. 3A , in other embodiments, the original image may be an image including a special seal for invoices, and the intermediate input image shown in FIG. 3B can be obtained by processing the original image shown in FIG. 3A .

For example, the stamp callout box can be a rectangular box, so that the intermediate input image can have a rectangular shape. For example, the dimensions of the stamp callout box can be the same as the dimensions of the intermediate input image. However, the embodiments of the present disclosure are not limited to this, and the size of the seal annotation frame and the size of the intermediate input image may also be different. For example, the size of the seal annotation frame is larger than the size of the intermediate input image, that is, the intermediate input image is located in the seal annotation frame. inside the box.

It should be noted that the seal marking frame may also be a diamond frame, an oval frame, a circular frame, and the like.

For example, the seal area recognition model can be implemented using machine learning technology, and the seal area recognition model is a pre-trained model. The seal region recognition model can be implemented by neural networks such as deep convolutional neural network (CNN) or deep residual network (Resnet).

For example, the size of the intermediate input image can be set by the user according to the actual situation.

For example, the original image may be an image captured by a digital camera or a mobile phone, and the original image may be a grayscale image or a color image.

For example, the original image may be an image directly collected by an image collection device, or may be an image obtained after preprocessing the directly collected image. For example, in order to avoid the influence of the data quality and data imbalance of the original image on the recognition of the input image, before processing the original image, the image processing method provided by the embodiments of the present disclosure may further include an operation of preprocessing the original image. Preprocessing can eliminate irrelevant information or noise information in the original image, so as to better process the original image. The preprocessing may include, for example, scaling, cropping, gamma correction, image enhancement, or noise reduction filtering on the original image.

It should be noted that, in other examples, acquiring the input image includes: acquiring the original image, processing the original image through the seal region recognition model to determine the intermediate input image; processing the intermediate input image to remove interference in the intermediate input image pixels to get the input image. The area of the stamp annotation frame can be marked in the original image, and this area is the intermediate input image, so that in subsequent operations, the marked area can be directly processed, for example, to remove interference pixels. That is, the stamp callout box in the original image can not be cut.

For example, the input stamp in the input image corresponds to the original stamp in the intermediate input image. It should be noted that the size, shape, etc. of the input stamp and the original stamp are the same, except that the input stamp is located in the input image, and the original stamp is located in the original image and the intermediate input image. Furthermore, as shown in Figures 2B and 2C, in some embodiments, during the process of removing interfering pixels on the intermediate input image, some pixels of the original stamp may be removed, resulting in the original stamp and the input stamp being merged with each other. Not exactly the same, however, it is worth noting that the objects included in the original seal and the input image are the same, e.g. the original seal included the text "Hangzhou Ruisheng Software Co., Ltd." and the input image also included the text "Hangzhou Ruisheng Software Co., Ltd.".

For example, interfering pixels include pixels of interfering objects in the intermediate input image that do not belong to the original stamp. For example, the interfering objects may include horizontal lines covered by the original seal or characters or numbers on the date of the seal, and may also include other characters, numbers or graphics that do not overlap with the original seal.

For example, in some embodiments, the interfering objects may be printed words, symbols, graphics, etc. As shown in FIG. 2B , the intermediate input image includes horizontal lines and commas that do not belong to the original seal, and the horizontal lines and commas are Interfering objects, the pixel corresponding to the horizontal line and comma is the interference pixel. For example, in other embodiments, the interfering objects can be handwritten words, symbols, graphics, etc. As shown in FIG. 3B, the input image includes numbers and points that do not belong to the original seal (ie, handwritten 2021.1.7), the numbers and The point is the interference object, and the pixel corresponding to the number and the point is the interference pixel.

For example, in step S10, processing the intermediate input image to remove interfering pixels in the intermediate input image to obtain the input image may include: using an image segmentation model (such as U-Net model, Mask-RCNN model, etc.) The input image is identified to obtain the initial interference pixels of the interference object; the initial interference pixels are blurred to obtain the interference pixel mask area; the interference pixels corresponding to the interference object are determined according to the interference pixel mask area; the interference pixels in the intermediate input image are removed. Interfering pixels corresponding to interfering objects are obtained to obtain the input image.

For example, the process of identifying and removing the interfering pixels may be performed based on the difference between the pixel value of the pixel corresponding to the interfering object and the pixel value of the pixel corresponding to the original seal.

For example, Gaussian blurring can be performed on the initial interference pixels through the GaussianBlur function of Gaussian filtering based on OpenCV to expand the area corresponding to the initial interference pixels, thereby obtaining the interference pixel mask area. According to the mask area of the interference pixel, the interference pixel corresponding to the interference object can be determined. Next, the interfering pixels corresponding to the interfering objects in the intermediate input image can be removed by the inpaint function based on OpenCV, so as to obtain an image from which the interfering objects are removed, that is, the input image is obtained.

For example, in the input image shown in FIG. 2C, the horizontal lines and commas in the intermediate input image shown in FIG. 2B have been removed; in the input image shown in FIG. 3C, the intermediate input image shown in FIG. 3B has been removed. Handwritten numerals and dots etc. have been removed from .

FIG. 2D is a schematic diagram of a seal image obtained by recognizing the input image shown in FIG. 2C , and FIG. 3D is a schematic diagram of a seal image obtained by recognizing the input image shown in FIG. 3C .

Next, as shown in FIG. 1, in step S11, the input seal in the input image is recognized to obtain a seal image.

For example, the seal image includes an intermediate seal corresponding to the input seal. It should be noted that the size and shape of the input stamp and the intermediate stamp are the same. In addition, the objects included in the input stamp and the objects included in the intermediate stamp and their relative positional relationships are also the same. The difference between the input stamp and the intermediate stamp is the same. where: the input stamp is located in the input image, and the intermediate stamp is located in the stamp image.

For example, in some embodiments, step S11 includes: using an image segmentation model (such as U-Net model, Mask-RCNN model, etc.) to identify the input image to obtain initial seal pixels corresponding to the input seal; Blur processing to obtain the seal pixel mask area; determine the pixel corresponding to the input seal in the input image according to the seal pixel mask area; set the pixel value of the pixel corresponding to the input seal in the input image to the first pixel value and set the input The pixel values of the pixels other than the pixels corresponding to the input seal in the image are the second pixel values, so as to obtain the seal image.

For example, as shown in Figure 2D and Figure 3D, the seal image can be a black and white image with obvious black and white contrast, and the black and white image has less noise interference, which can effectively improve the recognition of the content in the seal image. The pixels corresponding to the middle seal in the seal image have the first pixel value, the pixels in the seal image except the pixels corresponding to the middle seal have the second pixel value, and the first pixel value and the second pixel value are different. For example, both the first pixel value and the second pixel value may be grayscale values, and the first pixel value may be 255, and the second pixel value may be 0.

It should be noted that both the image segmentation model for recognizing input images and the image segmentation model for recognizing intermediate input images can be implemented using machine learning technology (eg, deep learning technology), and both are pre-trained models. The image segmentation model for recognizing the input image and the image segmentation model for recognizing the intermediate input image can be two different models, but both adopt the U-Net model structure.

FIG. 2E is a feature point image obtained by performing feature extraction processing on the seal image shown in FIG. 2D , and FIG. 3E is a feature point image obtained by performing feature extraction processing on the seal image shown in FIG. 3D .

Next, as shown in FIG. 1, in step S12, feature extraction processing is performed on the seal image to obtain feature point images.

For example, in step S12, feature extraction processing may be performed on the seal image through a pre-trained feature extraction model to obtain feature point images. Feature extraction models can also be implemented based on machine learning techniques.

For example, feature extraction processing is performed on the seal image shown in FIG. 2D to obtain the feature point image shown in FIG. 2E , and feature extraction processing is performed on the seal image shown in FIG. 3D to obtain the feature point image shown in FIG. 3E . Taking the feature point image shown in FIG. 2E as an example, as shown in FIG. 2C , the first object includes "Hangzhou Ruisheng Software Co., Ltd.", and in the feature point image shown in FIG. 2E , the feature point image includes 11 features. The 11 feature points correspond to each character in "Hangzhou Ruisheng Software Co., Ltd." and the center point of the middle seal. For each character, the feature point corresponding to the character is located in the center of the region corresponding to the character.

It should be noted that, as shown in FIGS. 2A-2D , each of the original image, the intermediate input image, the input image and the seal image includes the first object "Hangzhou Ruisheng Software Co., Ltd.".

For example, the image segmentation model is established by processing the input image or intermediate input image into a black and white image and labeling the sample, and then putting it into the U-net model for training; the feature extraction model is also by using the seal image as a sample. After labeling, it is established by training the neural network model.

FIG. 2F is a schematic diagram of another input image provided by an embodiment of the present disclosure.

Next, as shown in FIG. 1, in step S13, the stamp image and the feature point image are processed to obtain the first development point, the second development point and the development line.

For example, in some embodiments, the unfolding line includes a first annular unfolding line, and the first annular unfolding line is the edge line of the input stamp. For example, in some embodiments, the edge line of the input stamp may be the circle shown in FIG. 2C . ; In other embodiments, the edge line of the input stamp may be an elliptical circle as shown in FIG. 3C .

For example, step S13 includes: processing the seal image and the feature point image based on the algorithm of OpenCV to obtain the initial second expansion point and the initial first annular expansion line; processing the seal image and the feature point image through the expansion point extraction model, Determine the characteristic object area corresponding to the first object in the seal image, determine the opening area in the seal image based on the characteristic object area, and obtain any point in the opening area; an expansion point; the initial first expansion point, the initial second expansion point and the initial first circular expansion line are mapped from the stamp image to the input image to obtain the first expansion point, the second expansion point and the first circular expansion line.

For example, the initial first unfolding point, the initial second unfolding point, and the initial first annular unfolding line are all located in the stamp image. The connecting line segment between any two points among any point, the initial first unfolding point and the initial second unfolding point does not overlap with the feature object area, so that it can be ensured that when the input stamp is horizontally unfolded, the first The object is split into two parts.

For example, the area 100 shown in FIG. 2D is the characteristic object area. In the seal image shown in FIG. 2D , the first object "Hangzhou Ruisheng Software Co., Ltd." is located in the characteristic object area. As shown in FIG. 2F , the region corresponding to the characteristic object region 100 in the input image is the first object region 200 , and in the input image, the first object is located in the first object region 200 . The shape of the characteristic object area 100 may be an arc, and the shape of the first object area 200 may also be an arc.

For example, the initial first annular development line may be the edge line of the middle seal shown in FIG. 2D , and the edge line of the middle seal may be the white circle shown in FIG. 2D .

For example, as shown in FIG. 2D , an annular area may be determined based on the characteristic object area 100 , for example, an annular area, the annular area includes the characteristic object area 100 , and the part of the annular area that does not belong to the characteristic object area 100 is is the opening area 110 , and the arbitrary point B is a point in the opening area 110 .

For example, the seal image and the feature point image may be processed based on the Hough gradient circle finding algorithm using OpenCV to obtain the initial second expansion point and the initial first annular expansion line. For the specific implementation process of the Hough gradient circle finding algorithm of OpenCV, reference may be made to relevant descriptions in the prior art, and details are not described here. It should be noted that, in the embodiments of the present disclosure, other methods may also be used to obtain the initial second deployment point and the initial first annular deployment line. No restrictions apply.

For example, the expansion point extraction model may be implemented based on machine learning, and the expansion point extraction model may be a neural network model.

For example, in step S13, determining the initial first expansion point based on any point and the initial first annular expansion line includes: based on any point, acquiring a point corresponding to any point on the initial first annular expansion line as the initial first expansion point.

For example, the initial second unfolding point may be the center point of the middle seal. For example, in some embodiments, as shown in FIG. 2D , the shape of the middle seal is a circle, and the initial second unfolding point A1 is the center of the circle (ie The center point of the middle seal), at this time, the initial first unfolding point C1 may be the intersection between the extension line connecting the initial second unfolding point A1 and any point B1 and the initial first annular unfolding line.

For example, after mapping the initial first unfolding point, the initial second unfolding point, and the initial first circular unfolding line from the stamp image to the input image, the second unfolding point may be the center point of the input stamp, eg, in some embodiments 2F, the initial first expansion point C1 is mapped to the first expansion point C2, the initial second expansion point A1 is mapped to the second expansion point A2, and any point B1 is mapped to the point B2. For example, the shape of the input stamp is a circle, and the second expansion point A2 is the center of the circle (ie, the center point of the input stamp). At this time, the first expansion point C2 can be the connection between the second expansion point A2 and the point B2. The intersection between the extension line of the line and the first annular expansion line.

For example, in other embodiments, as shown in FIG. 3D , the shape of the middle seal is an ellipse, and the initial second expansion point may be the midpoint (not shown) of the line connecting the two focal points of the ellipse. At this time, The shape of the input stamp is also an ellipse, and after mapping, the second expansion point is also the midpoint of the line connecting the two focal points of the ellipse.

FIG. 2G is another schematic diagram of a seal image obtained by recognizing the input image shown in FIG. 2C .

For example, in other embodiments, the unfolding line includes a first annular unfolding line and a second annular unfolding line, the first annular unfolding line is an edge line of the input stamp, and in the input image, the second annular unfolding line is located in the first annular unfolding line. In the area enclosed by the expansion line, the first object area is located in the annular area enclosed by the first annular expansion line and the second annular expansion line.

For example, step S13 includes: processing the seal image and the feature point image based on the algorithm of OpenCV to obtain the initial first annular expansion line and the initial second annular expansion line; processing the seal image and the feature point image through the expansion point extraction model , to determine the characteristic object area corresponding to the first object in the seal image, determine the opening area in the seal image based on the characteristic object area, obtain any point in the opening area, and based on any point, the initial first annular expansion line and the initial first Two circular expansion lines, determine the initial first expansion point and the initial second expansion point; map the initial first expansion point, the initial second expansion point, the initial first circular expansion line and the initial second circular expansion line from the stamp image to An image is input to obtain a first unfolding point, a second unfolding point, a first circular unfolding line, and a second circular unfolding line.

For example, the initial first annular development line, the initial second annular development line, the initial first development point, and the initial second development point are all located in the stamp image. The connecting line segment between any two points among any point, the initial first unfolding point and the initial second unfolding point does not overlap with the feature object area.

For example, as shown in FIG. 2G , the white circle 300 may be the initial second annular expansion line, the area 100 shown in FIG. 2G is the feature object area, and the area 110 shown in FIG. 2G is the opening area.

For example, in step S13, determining the initial first expansion point and the initial second expansion point based on any point, the initial first annular expansion line and the initial second annular expansion line, including: obtaining the initial first annular expansion based on any point A point on the line corresponding to any point is used as an initial first expansion point; based on any point, a point corresponding to any point on the initial second annular expansion line is obtained as an initial second expansion point.

For example, as shown in FIG. 2G, the any point B1 is a point in the opening area 110, the shape of the middle seal is a circle, and the initial first expansion point C1 can be the radius of the circle including any point B1 and the initial first For the intersection between the annular development lines, the initial second development point A1 may be the intersection between the radius of the circle including any point B1 and the initial second annular development line 300 .

For example, any point B1 may be the center point of the opening area.

For example, the initial first unfolding point C1, the initial second unfolding point A1 and any point B1 are located on the same straight line, as shown in FIG. 2F, after the mapping, the first unfolding point C2, the second unfolding point A2 and the point B2 are also on the same straight line. For example, as shown in FIGS. 2D and 2G , the initial first unfolding point C1 , the initial second unfolding point A1 and the arbitrary point B1 are located on a radius of the middle seal of the circle. For example, in the example shown in FIG. 2D , the distance between the initial first unfolding point C1 and the initial second unfolding point A1 is the radius of the middle seal of the circle. Similarly, as shown in Figure 2F, after mapping, the first unfolding point C2, the second unfolding point A2 and the point B2 are located on a radius of the circular input stamp, the first unfolding point C2 and the second unfolding point A2 The distance between is the radius of the input stamp for that circle.

It should be noted that, although in the present disclosure, the first annular development line and the second annular development line are described as circles or elliptical circles as an example, the present disclosure is not limited to this, the first annular development line and/or the second annular development line are The annular unfolding line may also be an unclosed arc or curve, and its specific shape is related to the shape of the first object area including the first object. For example, if the shape of the first object area is wavy, the first annular The unfolding line and/or the second annular unfolding line may also be a wavy line.

In addition, the first annular development line can also be a concentric ring line of the edge line of the input seal. In the input image, the edge line of the input seal is located in the area surrounded by the first annular development line. For example, when the shape of the input seal is a circle , the shape of the first annular development line and the shape of the edge line of the input stamp may be the same, for example, both are circular, but the radius corresponding to the first annular development line is larger than the radius corresponding to the edge line of the input stamp.

2H is a schematic diagram of an expanded seal image obtained by expanding the input seal in FIG. 2C ; FIG. 3F is a schematic diagram of an expanded seal image obtained by expanding the input seal in FIG. 3C .

Next, as shown in FIG. 1, in step S14, take the connecting line between the first development point and the second development point as the development reference line and the first development point as the development starting point, and place the input stamp along the development line Expand horizontally for expanded stamp image.

For example, the unfolded seal image shown in FIG. 2H is based on the first unfolding point obtained by mapping the initial first unfolding point shown in FIG. 2G and the second unfolding point obtained by mapping the initial second unfolding point shown in FIG. 2G . The connecting line is used as the unfolding reference line and the first unfolding point obtained by mapping the initial first unfolding point shown in FIG. 2G as the unfolding starting point, and is obtained by horizontally unfolding along the first annular unfolding line.

For example, the first annular expansion line is expanded into a straight line in the expanded stamp image. As shown in Figure 2H, the straight line above the text is the expanded first circular expansion line; as shown in Figure 3F, the line above the text The straight line is the first annular expansion line after expansion. For the expanded seal image shown in FIG. 3F, during the expansion process, the second expansion point is also a point corresponding to any point on the second annular expansion line.

For example, as shown in Fig. 2H, the shape of the expanded stamp image is a rectangle, the length of the rectangle is equal to the length of the first annular expansion line, and the width of the rectangle is the same as the first expansion point and the point obtained based on the mapping of any point (that is, the initial first The distance between the expansion point and any point) is equal. For the example shown in Fig. 2D, the shape of the middle seal is a circle, the length of the rectangle is equal to the circumference of the circle, and the width of the rectangle is equal to the radius of the circle; for the example shown in Fig. 2G, the length of the middle seal is equal to the radius of the circle. The shape is a circle, the length of the rectangle is equal to the circumference of the circle, and the width of the rectangle is less than the radius of the circle.

It should be noted that, in other embodiments, the connecting line between the initial first unfolding point and the initial second unfolding point may also be used as the unfolding reference line and the initial first unfolding point may be used as the unfolding starting point, and the middle seal Expand horizontally along the expansion line to obtain the expanded stamp image, that is, at this time, the initial first expansion point is the first expansion point, the initial second expansion point is the second expansion point, and the initial first circular expansion line is the first expansion point. For the annular expansion line, the initial second annular expansion line is the second annular expansion line.

Fig. 2I is the schematic diagram of the first intermediate object region obtained by carrying out region recognition processing to the expanded seal image in Fig. 2H; Fig. 3G is the schematic diagram of the first intermediate object region obtained by performing region identification processing on the expanded seal image in Fig. 3F.

Next, as shown in FIG. 1 , in step S15 , an area identification process is performed on the developed seal image to determine the first intermediate object area in the expanded seal image. For example, an area corresponding to the first intermediate object area in the input image is the first object area, and the first object is located in the first object area.

For example, the shape of the first object area is an arc.

Expand the circular or oval stamp into a rectangle according to the first expansion point, the second expansion point and the horizontal line of the expansion line, so that the original arc-shaped text area is expanded into a long text area with a certain deformation, that is, the first intermediate object area, as shown in FIG. 2I and FIG. 3G , the shape of the first intermediate object area may be a rectangle. It should be noted that the specific unfolding methods of the circular seal and the oval seal may refer to the prior art, which will not be repeated here.

For example, in step S15, the first intermediate object region in the expanded stamp image can be identified by the region identification model. The region recognition model can be implemented using machine learning technology, and the region recognition model is a pre-trained model. The region recognition model can be implemented by neural networks such as deep convolutional neural network (CNN) or deep residual network (Resnet).

2J is a schematic diagram of a first recognition result obtained by performing object recognition processing on the first intermediate object area in FIG. 2I ; FIG. 3H is a schematic diagram of a first recognition result obtained by performing object recognition processing on the first intermediate object area in FIG. 3G . Schematic.

Finally, as shown in FIG. 1 , in step S16 , an object recognition process is performed on the first intermediate object region to recognize and obtain a first recognition result. For example, as shown in FIG. 2J and FIG. 3H , the first recognition result is “Hangzhou Ruisheng Software Co., Ltd.”, that is, the first object.

For example, the first object includes text, and character recognition processing may be performed on the first intermediate object region through the first character recognition model to obtain the first recognition result, that is, the first object. The accuracy of character recognition based on the first character recognition model is high. For example, the first character recognition model may be implemented based on technologies such as optical character recognition (Optical Character Recognition, OCR). For example, the first character recognition model may also be a pre-trained model.

For example, performing object recognition processing on the first intermediate object area to recognize and obtain the first recognition result may include: performing object recognition processing on the first intermediate object area to recognize and obtain the first intermediate recognition result; A check is performed to obtain the first identification result.

For example, the first intermediate recognition result may have semantic errors, logical errors, etc. Therefore, it is necessary to verify the first intermediate recognition result, and correct the semantic errors and logical errors in the first intermediate recognition result, so as to obtain an accurate first intermediate recognition result. Identify the results. For example, for the example shown in Figure 2I, the first intermediate recognition result may include "Hangzhou Ruisheng Software Co., Ltd.", wherein the character "zhou" does not correspond to the text in the seal, and the word "Hangzhou" is in The semantics is wrong. After verification, "Hangzhou" can be corrected to "Hangzhou", so the first recognition result after verification is "Hangzhou Ruisheng Software Co., Ltd.", thus obtaining an accurate recognition result .

For example, as shown in FIG. 2J and FIG. 3H , the first identification result obtained by identification is "Hangzhou Ruisheng Software Co., Ltd.", which is the first object in the input seal.

In addition, the image processing method can also determine the forward direction of the input image based on the regions corresponding to the irregularly arranged objects, so as to correct the input image and improve the recognition accuracy of the regularly arranged objects in the input image.

For example, in some embodiments, the image processing method further includes: determining the center point of the first intermediate object region; mapping the first intermediate object region back to the input image through the mapping relationship between the first intermediate object region and the input image to determine the first intermediate object region an object area, and mapping the center point of the first intermediate object area back to the input image to determine the center point of the first object area; determining the center point of the input seal; determining by the center point of the first object area and the center point of the input seal Correction angle for correcting the input image; correcting the input image based on the correction angle to obtain the corrected input image.

For example, the first intermediate object region in the expanded seal image can be identified by the region recognition model, and the center point of the first intermediate object region can be determined, and the An intermediate object area is mapped into the input image (or the seal image), thereby determining the arc-shaped character area in the input image (or the seal image), that is, the first object area 200, and at the same time, the center point of the first intermediate object area is mapped to The center point of the first object area is determined in the input image (or seal image), and the forward direction corresponding to the input image can be obtained through the center point of the first object area and the center point of the input seal (for example, the center of the circle of the input seal). (From the center point of the input stamp to the center point of the first object area is the forward direction), the angle between the forward direction and the reference direction (for example, the horizontal direction or the vertical direction) is the correction angle, Then, the input image can be corrected based on the correction direction, so that the forward direction and the reference direction overlap, so as to obtain the corrected input image, thus, it is convenient for the user to check and compare whether the first recognition result obtained by the recognition is correct, etc. , that is, it is determined whether the first recognition result obtained by the recognition is the same as the first object.

For example, in other embodiments, the first intermediate object region may also be mapped back to the intermediate input image, and a correction angle for correcting the intermediate input image is determined; and the intermediate input image is corrected based on the correction angle to obtain the correction After the intermediate input image. The user can check whether the first recognition result obtained by the comparison and recognition based on the corrected intermediate input image is correct, etc.

For example, in some embodiments, the input stamp further includes a second object. At this time, the image processing method further includes: performing region identification processing on the corrected input image to determine the second intermediate object region, wherein the region corresponding to the second intermediate object region in the input image is the second object region, and the second intermediate object region is the second object region. The object is located in the second object area; the object recognition processing is performed on the second intermediate object area to obtain a second recognition result.

For example, the shape of the second object area may be a rectangle.

For example, the second object may include a plurality of characters arranged regularly, and a line connecting the center points of the plurality of characters is located on the same straight line. As shown in FIG. 3D, the second object may include numbers and letters "91330108MA2CDKJ756", and the second object may also include the text "Invoice Special Seal". The center points of each character (numbers and letters) in "91330108MA2CDKJ756" are located on the same line (such as a horizontal line), and the center points of each character in the "Special Invoice Seal" are also located on the same line (such as a horizontal line).

It should be noted that "91330108MA2CDKJ756" and "Invoice Special Seal" can be located in the two second object areas respectively.

For example, character recognition processing can be performed on the second intermediate object area through the second character recognition model to obtain the second intermediate recognition result; the second intermediate recognition result is verified to obtain the second recognition result, and the second recognition result is for the second object. Thereby, both the object in the arc area and the object in the rectangular area in the input stamp are recognized.

For example, the second character recognition model may be implemented based on technologies such as optical character recognition. For example, the second character recognition model may also be a pre-trained model.

It should be noted that the first character recognition model and the second character recognition model may be the same model, or may be different models.

For example, in some embodiments, the image processing method may further include: outputting the first recognition result and the second recognition result. For example, the first recognition result and the second recognition result may be displayed on the display panel to achieve output.

For example, the image processing method may further include: outputting the corrected input image and/or the corrected intermediate input image, so that the user can judge whether the outputted first recognition result and the second recognition result are correct. For example, the corrected input image and/or the corrected intermediate input image may also be displayed on the display panel for output.

It should be understood that, in the embodiment of the present disclosure, before acquiring the input image, the image processing method further includes: a training phase. The training phase includes the process of training the models (image segmentation model, region recognition model, expansion point extraction model, seal region recognition model, character recognition model, etc.).

FIG. 4 is a schematic block diagram of an image processing apparatus according to some embodiments of the present disclosure.

At least one embodiment of the present disclosure further provides an image processing apparatus. As shown in FIG. 4 , the image processing apparatus 400 includes a processor 402 and a memory 401 . It should be noted that the components of the image processing apparatus 400 shown in FIG. 4 are only exemplary and not restrictive, and the image processing apparatus 400 may also have other components according to actual application requirements.

For example, the memory 401 is used for non-transitory storage of computer-readable instructions; the processor 402 is used for executing computer-readable instructions, and the computer-readable instructions are executed by the processor 402 when running the image processing method according to any of the above embodiments. one or more steps.

For example, components such as processor 402 and memory 401 may communicate through a network connection. The network may include a wireless network, a wired network, and/or any combination of wireless and wired networks. The network may include a local area network, the Internet, a telecommunication network, the Internet of Things (Internet of Things) based on the Internet and/or a telecommunication network, and/or any combination of the above networks, etc. For example, the wired network may use twisted pair, coaxial cable or optical fiber transmission for communication, and the wireless network may use, for example, 3G/4G/5G mobile communication network, Bluetooth, Zigbee or WiFi and other communication methods. The present disclosure does not limit the type and function of the network.

For example, processor 402 may control other components in image processing apparatus 400 to perform desired functions. The processor 402 may be a device with data processing capability and/or program execution capability, such as a central processing unit (CPU), a tensor processing unit (TPU), or a graphics processing unit (GPU). The central processing unit (CPU) can be an X86 or an ARM architecture or the like. The GPU can be individually integrated directly onto the motherboard, or built into the motherboard's Northbridge chip. GPUs can also be built into central processing units (CPUs).

For example, memory 401 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and/or cache memory, among others. Non-volatile memory may include, for example, read only memory (ROM), hard disk, erasable programmable read only memory (EPROM), portable compact disk read only memory (CD-ROM), USB memory, flash memory, and the like. One or more computer-readable instructions may be stored on the computer-readable storage medium, and the processor 402 may execute the computer-readable instructions to implement various functions of the image processing apparatus 400 . Various application programs, various data and the like can also be stored in the storage medium.

For example, for a detailed description of the process of image processing performed by the image processing apparatus 400, reference may be made to the relevant descriptions in the embodiments of the image processing method, and repeated descriptions will not be repeated.

FIG. 5 is a schematic block diagram of an intelligent invoice recognition device provided by some embodiments of the present disclosure.

At least one embodiment of the present disclosure further provides an intelligent invoice recognition device. As shown in FIG. 5 , the intelligent invoice recognition device 500 may include a memory 501 , a processor 502 and an image acquisition component 503 . It should be noted that the components of the smart invoice recognition device 500 shown in FIG. 5 are only exemplary, not limiting, and the smart invoice recognition device 500 may also have other components according to actual application requirements.

For example, the image acquisition part 503 is used to acquire an invoice image of a paper invoice. Memory 501 is used to store invoice images and computer readable instructions. Processor 502 operates to read the invoice image and determine an input image based on the invoice image and execute computer readable instructions. The computer readable instructions are executed by the processor 502 to perform one or more steps in the image processing method according to any of the above embodiments. For example, the invoice image may be the original image described in the embodiment of the image processing method.

For example, the image acquisition component 503 is the image acquisition device described in the embodiments of the above image processing method. For example, the image acquisition component 503 may be a camera of a smartphone, a camera of a tablet computer, a camera of a personal computer, a lens of a digital camera, Or even a webcam.

For example, the image of the invoice may be the image of the original invoice directly collected by the image acquisition component 503, or may be the image obtained after preprocessing the image of the original invoice. Preprocessing can remove irrelevant information or noise information in the original invoice image to facilitate better processing of the invoice image. The preprocessing may include, for example, performing image augmentation (Data Augment), image scaling, gamma (Gamma) correction, image enhancement or noise reduction filtering on the original invoice image.

For example, processor 502 may control other components in intelligent invoice recognition device 500 to perform desired functions. The processor 502 may be a device with data processing capability and/or program execution capability, such as a central processing unit (CPU), a tensor processing unit (TPU), or a graphics processing unit (GPU). The central processing unit (CPU) can be an X86 or an ARM architecture or the like. The GPU can be individually integrated directly onto the motherboard, or built into the motherboard's Northbridge chip. GPUs can also be built into central processing units (CPUs).

For example, memory 501 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and/or cache memory, among others. Non-volatile memory may include, for example, read only memory (ROM), hard disk, erasable programmable read only memory (EPROM), portable compact disk read only memory (CD-ROM), USB memory, flash memory, and the like. One or more computer-readable instructions may be stored on the computer-readable storage medium, and the processor 502 may execute the computer-readable instructions to implement various functions of the intelligent invoice recognition device 500.

For example, for a detailed description of the process of image processing performed by the intelligent invoice recognition device 500, reference may be made to the relevant descriptions in the embodiments of the image processing method, and repeated descriptions will not be repeated.

FIG. 6 is a schematic diagram of a storage medium provided by some embodiments of the present disclosure. For example, as shown in FIG. 6 , one or more computer-readable instructions 601 may be non-transitory stored on storage medium 600 . For example, when the computer readable instructions 601 are executed by a computer, one or more steps in the image processing method according to the above description may be performed.

For example, storage medium 600 is a non-transitory computer-readable storage medium.

For example, the storage medium 600 can be applied to the above-mentioned image processing apparatus 400 and/or the smart invoice recognition apparatus 500 , for example, it can be the memory 401 in the image processing apparatus 400 and/or the memory 501 in the smart invoice recognition apparatus 500 .

For example, for the description of the storage medium 600, reference may be made to the description of the memory in the embodiments of the image processing apparatus 400 and/or the smart invoice recognition device 500, and the repetition will not be repeated.

For the present disclosure, the following points need to be noted:

(1) The accompanying drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure, and other structures may refer to general designs.

(2) In the drawings for describing the embodiments of the present invention, the thickness and size of layers or structures are exaggerated for clarity. It will be understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, it can be "directly on" or "under" the other element, Or intermediate elements may be present.

(3) The embodiments of the present disclosure and the features in the embodiments may be combined with each other to obtain new embodiments without conflict.

The above descriptions are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

An image processing method, comprising:

acquiring an input image, wherein the input image includes an input seal, and the input seal includes a first object;

Identifying the input seal in the input image to obtain a seal image, wherein the seal image includes an intermediate seal corresponding to the input seal;

Perform feature extraction processing on the seal image to obtain feature point images;

Process the seal image and the feature point image to obtain a first unfolding point, a second unfolding point and an unfolding line;

Taking the connecting line between the first unfolding point and the second unfolding point as the unfolding reference line and the first unfolding point as the unfolding starting point, unfold the input stamp laterally along the unfolding line to get the expanded stamp image;

Performing region identification processing on the expanded seal image to determine the first intermediate object region in the expanded seal image, wherein the region corresponding to the first intermediate object region in the input image is the first object region, the first object is located within the first object area;

Perform object recognition processing on the first intermediate object area to recognize and obtain a first recognition result.
The image processing method according to claim 1, wherein the pixels corresponding to the middle seal in the seal image have a first pixel value, and the pixels in the seal image other than the pixels corresponding to the middle seal have a first pixel value. Having a second pixel value, the first pixel value and the second pixel value are not the same.
The image processing method according to claim 2, wherein identifying the input seal in the input image to obtain a seal image comprises:

Use an image segmentation model to identify the input image to obtain initial seal pixels corresponding to the input seal;

Blur the initial seal pixels to obtain the seal pixel mask area;

According to the seal pixel mask area, determine the pixel corresponding to the input seal in the input image;

Setting the pixel value of the pixel corresponding to the original seal in the input image to the first pixel value and setting the pixel value of the pixel other than the pixel corresponding to the input seal in the input image to the first pixel value Two pixel values to obtain the stamp image.
The image processing method according to claim 1, wherein the unfolding line comprises a first annular unfolding line, and the first annular unfolding line is an edge line of the input stamp,

The seal image and the feature point image are processed to obtain the first unfolding point, the second unfolding point and the unfolding line, including:

The seal image and the feature point image are processed by the algorithm based on OpenCV to obtain the initial second expansion point and the initial first circular expansion line, wherein the initial second expansion point and the initial first circular expansion the line is located in the stamp image;

The seal image and the feature point image are processed through the expansion point extraction model to determine the feature object area corresponding to the first object in the seal image, and the seal image is determined based on the feature object area. The opening area is obtained, and any point in the opening area is obtained, and based on the arbitrary point and the initial first annular expansion line, the initial first expansion point is determined, wherein the initial first expansion point is located at the seal In the image, the line segment between any two points in the any point, the initial first unfolding point and the initial second unfolding point does not overlap with the feature object area,

Mapping the initial first unfolding point, the initial second unfolding point and the initial first circular unfolding line from the stamp image to the input image to obtain the first unfolding point, the first unfolding point Two deployment points and the first annular deployment line.
The image processing method according to claim 4, wherein determining the initial first expansion point based on the arbitrary point and the initial first annular expansion line comprises:

Based on the arbitrary point, a point on the initial first annular expansion line corresponding to the arbitrary point is acquired as the initial first expansion point.
The image processing method according to claim 1, wherein the unfolding line comprises a first annular unfolding line and a second annular unfolding line, and the first annular unfolding line is an edge line of the input stamp. In the input image, the second annular development line is located in the area enclosed by the first annular development line, and the first object area is located in the area enclosed by the first annular development line and the second annular development line. in the annular area,

Described seal image and described feature point image are processed, to obtain the first unfolding point, the second unfolding point and unfolding line, including:

An algorithm based on OpenCV processes the seal image and the feature point image to obtain an initial first annular development line and an initial second annular development line, wherein the initial first annular development line and the initial second annular development line a circular spread line is located in the stamp image;

The seal image and the feature point image are processed through the expansion point extraction model to determine the feature object area corresponding to the first object in the seal image, and the seal image is determined based on the feature object area. the opening area of point, wherein the initial first unfolding point and the initial second unfolding point are located in the stamp image, any two of the any point, the initial first unfolding point and the initial second unfolding point The line segments between the points do not overlap with the feature object area;

The initial first unfolding point, the initial second unfolding point, the initial first circular unfolding line and the initial second circular unfolding line are mapped from the stamp image to the input image to obtain the The first deployment point, the second deployment point, the first annular deployment line and the second annular deployment line.
The image processing method according to claim 6, wherein an initial first unfolding point and an initial second unfolding are determined based on the arbitrary point, the initial first circular unfolding line and the initial second circular unfolding line points, including:

Based on the arbitrary point, acquiring a point corresponding to the arbitrary point on the initial first annular expansion line as the initial first expansion point;

Based on the arbitrary point, a point corresponding to the arbitrary point on the initial second annular expansion line is acquired as the initial second expansion point.
The image processing method according to any one of claims 4-7, wherein the initial first unfolding point, the initial second unfolding point and the any point are located on the same straight line.
The image processing method according to any one of claims 4-7, wherein the any point is a center point of the opening area.
The image processing method according to any one of claims 4-7, wherein the first annular development line is expanded into a straight line in the expanded seal image.
The image processing method according to any one of claims 1-7, wherein the first object is text, and the shape of the first object area is an arc.
The image processing method according to any one of claims 1-5, wherein the shape of the input stamp is a circle, and the second expansion point is the center of the circle; or,

The shape of the input stamp is an ellipse, and the second expansion point is the midpoint of a line connecting two focal points of the ellipse.
The image processing method according to any one of claims 1-7, further comprising:

determining the center point of the first intermediate object area;

According to the mapping relationship between the first intermediate object area and the input image, the first intermediate object area is mapped back to the input image to determine the first object area, and the first intermediate object area is mapped mapping the center point back to the input image to determine the center point of the first object region;

determining the center point of the input stamp;

Determine a correction angle for correcting the input image by the center point of the first object area and the center point of the input stamp;

The input image is corrected based on the correction angle to obtain a corrected input image.
The image processing method according to claim 13, wherein the input seal further comprises a second object,

The image processing method further includes:

Performing region identification processing on the corrected input image to determine a second intermediate object region, wherein the region corresponding to the second intermediate object region in the input image is the second object region, and the second object region is located within the second object area;

Perform object recognition processing on the second intermediate object area to obtain a second recognition result.
The image processing method according to any one of claims 1-7, wherein acquiring the input image comprises:

obtaining an original image, wherein the original image includes an original seal;

The original image is processed by a seal area recognition model to determine the seal area, the seal area is marked by a seal labeling frame, and the seal labeling frame is sliced to obtain an intermediate input image, wherein the original The seal is located in the seal area, the seal labeling frame includes the seal area, and the intermediate input image includes the original seal,

The intermediate input image is processed to remove interfering pixels in the intermediate input image to obtain the input image, wherein the interfering pixels include interfering objects in the intermediate input image that do not belong to the original seal; pixel, the input stamp corresponds to the original stamp.
An image processing device, comprising:

memory for non-transitory storage of computer readable instructions; and

a processor, configured to execute the computer-readable instructions, and when the computer-readable instructions are executed by the processor, the image processing method according to any one of claims 1-15 is executed.
An intelligent invoice identification device, characterized in that it includes:

Image acquisition component for acquiring invoice images of paper invoices;

memory for storing the invoice image and computer readable instructions;

a processor for reading the invoice image and determining the input image based on the invoice image, and executing the computer readable instructions, the computer readable instructions being executed by the processor according to claim 1 - The image processing method of any one of 15.
A non-transitory computer-readable storage medium storing non-transitory computer-readable instructions, when the computer-readable instructions are executed by a computer, the image processing method according to any one of claims 1-15 can be performed.