CN115830604A

CN115830604A - Surface single image correction method, device, electronic apparatus, and readable storage medium

Info

Publication number: CN115830604A
Application number: CN202111085527.5A
Authority: CN
Inventors: 刘文龙
Original assignee: SF Technology Co Ltd
Current assignee: SF Technology Co Ltd
Priority date: 2021-09-16
Filing date: 2021-09-16
Publication date: 2023-03-21

Abstract

The application provides a method and device for correcting a single-face image, an electronic device and a computer-readable storage medium. The single-face image correction method comprises the following steps: acquiring the coordinates of the surface single center point of the target surface single in the surface single image to be corrected; acquiring target corner mark offsets of four top points of the target surface sheet and the central point of the surface sheet; determining the coordinates of the corner mark of four vertexes of the target surface sheet according to the offset of the target corner mark and the coordinates of the center point of the surface sheet; and correcting the single image of the surface to be corrected based on the coordinates of the corner point marks of the four vertexes of the target surface single to obtain a corrected single image of the surface to be corrected. Need not additionally to increase categorised branch in this application and carry out the judgement of information such as orientation, inclination, improved the location efficiency of bill of face, and then improved the recognition efficiency of bill of face information.

Description

Surface single image correction method, device, electronic apparatus, and readable storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for correcting a surface simplex image, an electronic device, and a computer-readable storage medium.

Background

The bill is a bill for recording various information, for example, the express bill is a bill used by the express industry to record relevant information such as a sender, a receiver, product weight, price and the like in the process of transporting goods. In order to automatically recognize various information recorded in a bill by a computer, it is generally necessary to perform bill detection and correction based on a bill image.

The method comprises the steps of firstly carrying out positioning detection on the bill of the face, then judging the inclination angle of the positioned bill of the face through a classification branch, and then correcting the bill of the face based on the inclination angle of the bill of the face so as to facilitate subsequent information identification.

However, the increased classification branches seriously reduce the positioning efficiency of the bill of dough, and further affect the efficiency of bill of dough information identification.

Disclosure of Invention

The application provides a method and a device for correcting a facial sheet image, electronic equipment and a computer readable storage medium, and aims to solve the problem that the facial sheet positioning efficiency is reduced and the facial sheet information identification efficiency is reduced due to the fact that the inclination angle of a classification branch judgment facial sheet needs to be increased in the prior art.

In a first aspect, the present application provides a method for correcting a planar single image, the method comprising:

acquiring a single image of a surface to be corrected;

acquiring the coordinates of the surface single center point of the target surface single in the surface single image to be corrected;

acquiring target corner mark offsets of four top points of the target surface sheet and the central point of the surface sheet;

determining corner mark coordinates of four vertexes of the target surface sheet according to the target corner mark offset and the coordinates of the center point of the surface sheet, wherein the corner mark coordinates are used for indicating an upper left corner point, a lower right corner point and an upper right corner point of the target surface sheet;

and correcting the single image of the surface to be corrected based on the coordinates of the corner point marks of the four vertexes of the target surface single to obtain a corrected single image of the surface to be corrected.

In a second aspect, the present application provides a single-sided image rectification device comprising:

the acquisition unit is used for acquiring a single image of a surface to be corrected;

the coordinate prediction unit is used for acquiring the coordinates of the surface single center point of the target surface single in the surface single image to be corrected;

the coordinate prediction unit is further configured to obtain target corner mark offsets between four vertices of the target surface sheet and a center point of the surface sheet;

the coordinate prediction unit is further configured to determine corner mark coordinates of four vertices of the target surface sheet according to the target corner mark offset and the coordinates of the center point of the surface sheet, where the corner mark coordinates are used to indicate an upper left corner point, a lower right corner point, and an upper right corner point of the target surface sheet;

and the correcting unit is used for correcting the single image of the surface to be corrected based on the corner point mark coordinates of the four vertexes of the target surface single to obtain a corrected single image of the surface to be corrected.

In a third aspect, the present application further provides an electronic device, where the electronic device includes a processor and a memory, where the memory stores a computer program, and the processor executes the steps in any one of the methods for correcting a simplex image provided in the present application when calling the computer program in the memory.

In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, where the computer program is loaded by a processor to execute the steps in the method for correcting a single face image.

According to the method and the device, the corner mark coordinates of the four vertexes of the target surface sheet are determined according to the coordinates of the surface single center point of the target surface sheet in the surface single image to be corrected and the target corner mark offset of the four vertexes of the target surface sheet and the surface single center point, and the surface single image to be corrected is corrected to obtain the corrected back single image. On one hand, because the corner mark coordinates carry corner mark information, the positions of an upper left corner point, a lower right corner point and an upper right corner point of the target surface sheet can be determined, and information such as the orientation, the inclination angle and the like of the target surface sheet can be further reflected; therefore, the single image of the surface to be corrected is corrected based on the angular point mark coordinates, and the target surface can be ensured to restore to the correct orientation, so that the single image of the surface to be corrected is accurately corrected. On the other hand, because the information such as the orientation, the inclination angle and the like of the target surface sheet can be reflected only by predicting the coordinates of the corner point marks of the four vertexes of the target surface sheet, the information such as the orientation, the inclination angle and the like of the target surface sheet can be obtained while the target surface sheet is detected, and the information such as the orientation, the inclination angle and the like can be judged without additionally increasing a classification branch, so that the classified data processing amount is reduced; therefore, compared with the algorithm for detecting the added classification branches in the prior art, the method and the device for identifying the bill of surface information improve the locating efficiency of the bill of surface information and further improve the identification efficiency of the bill of surface information.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic view of a scene of a single-sided image rectification detection system according to an embodiment of the present application;

FIG. 2 is a schematic flow chart illustrating a method for correcting a single face image according to an embodiment of the present disclosure;

FIG. 3 is an illustrative diagram of the offset provided in the embodiments of the present application;

FIG. 4 is an illustrative schematic diagram of a minimum bounding rectangle provided in embodiments of the present application;

FIG. 5 is a flow chart illustrating a process for training a predictive model provided in an embodiment of the present application;

FIG. 6 is a schematic diagram of a network architecture of a predictive model provided in an embodiment of the present application;

FIG. 7 is a schematic structural diagram of an embodiment of a facial unit image correction apparatus provided in an embodiment of the present application;

fig. 8 is a schematic structural diagram of an embodiment of an electronic device provided in the embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the description of the embodiments of the present application, it should be understood that the terms "first", "second", and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the embodiments of the present application, "a plurality" means two or more unless specifically defined otherwise.

The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known processes have not been described in detail so as not to obscure the description of the embodiments of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed in the embodiments herein.

An execution main body of the single-sided image correction method according to the embodiment of the present application may be the single-sided image correction apparatus provided in the embodiment of the present application, or different types of electronic devices such as a server device, a physical host, or a User Equipment (UE) integrated with the single-sided image correction apparatus, where the single-sided image correction apparatus may be implemented in a hardware or software manner, and the UE may specifically be a terminal device such as a smart phone, a tablet computer, a notebook computer, a palm computer, a desktop computer, or a Personal Digital Assistant (PDA).

This electronic equipment can adopt the working method of independent operation, perhaps also can adopt the working method of equipment cluster, through using the face list image correction method that this application embodiment provided, need not additionally to increase categorised branch and carry out the judgement of information such as orientation, inclination, has improved the location efficiency of face list, and then has improved the recognition efficiency of face list information.

Referring to fig. 1, fig. 1 is a schematic view of a scene of a single-sided image rectification system according to an embodiment of the present disclosure. The single-plane image correction system may include an electronic device 100, and the single-plane image correction apparatus is integrated in the electronic device 100. For example, the electronic device may acquire coordinates of a surface single center point of a target surface single in the surface single image to be corrected; acquiring target corner mark offsets of four top points of the target surface sheet and the central point of the surface sheet; determining the coordinates of the corner mark of four vertexes of the target surface sheet according to the offset of the target corner mark and the coordinates of the center point of the surface sheet; and correcting the single image of the surface to be corrected based on the coordinates of the corner point marks of the four vertexes of the target surface single to obtain a corrected single image of the surface to be corrected.

In addition, as shown in fig. 1, the facial single image rectification system may further include a memory 200 for storing data, such as image data and video data.

It should be noted that the scene schematic diagram of the single-sided image rectification system shown in fig. 1 is merely an example, and the single-sided image rectification system and the scene described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application, and it is obvious to a person of ordinary skill in the art that the technical solution provided in the embodiment of the present invention is also applicable to similar technical problems with the evolution of the single-sided image rectification system and the appearance of new service scenes.

Next, a description is started on a face simplex image correction method provided in an embodiment of the present application, in which an electronic device is used as an execution subject, and the execution subject will be omitted in subsequent method embodiments for simplicity and convenience of description.

Referring to fig. 2, fig. 2 is a schematic flowchart of a method for correcting a single-sided image according to an embodiment of the present disclosure. It should be noted that, although a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different than that shown or described herein. The single-face image correction method comprises steps 201 to 205, wherein:

201. and acquiring a single image of the surface to be corrected.

The bill is a receipt for recording various information, such as an express bill and a delivery bill.

The image of the bill of face to be corrected is an image containing the bill of face, such as an image containing an express bill of face.

In step 201, there are various ways to obtain a single image of a surface to be corrected, which illustratively includes:

(1) In practical application, the electronic equipment can integrate a camera on hardware, and a video frame or an image of the bill is shot in real time through the camera to serve as a bill image to be corrected.

(2) The video frames or images of the menu can also be obtained by shooting in real time through cameras of other terminals, for example, the video frames or images of the menu are obtained by shooting when the cameras of the express sorting equipment perform express sorting, and the electronic equipment establishes network connection with the cameras of other terminals. And according to the network connection, acquiring video frames or images of the bill shot by the cameras of other terminals on line from the cameras of other terminals to serve as the bill image to be corrected.

(3) The electronic device may also read out the facial sheet image captured by the camera from a storage medium associated with the facial sheet image captured by the camera (including a camera integrated with the electronic device or a camera of another terminal) as the facial sheet image to be corrected.

(4) And reading a surface single video frame or image which is collected in advance and stored in the electronic equipment as a surface single image to be corrected.

The method for acquiring the single image of the surface to be corrected is only an example, and is not limited thereto.

202. And acquiring the coordinates of the single center point of the target surface in the single image of the surface to be corrected.

The target surface sheet is the surface sheet in the surface sheet image to be corrected.

In some embodiments, the coordinates of the surface sheet center point of the target surface sheet in the prediction can be obtained based on the surface sheet image to be corrected by using a pre-trained prediction model provided in the embodiments of the present application. The "coordinates of the surface single center point of the target surface single in prediction based on the surface single image to be corrected through the trained prediction model" will be described in detail later when the prediction model is described, and will not be described again here.

In some embodiments, a central point thermodynamic diagram of the surface single image to be corrected may be predicted based on the deep convolutional neural network, and a pixel point with a confidence greater than a threshold confidence threshold is obtained based on the central point thermodynamic diagram, and is used as a surface single central point of a target surface single in the surface single image to be corrected, so as to obtain coordinates of the surface single central point.

203. And acquiring the mark offsets of the four top points of the target surface sheet and the target corner point of the central point of the surface sheet.

The four vertexes of the target surface list refer to an upper left corner point, a lower right corner point and an upper right corner point when the target surface list is placed in the forward direction.

For simplifying the description, hereinafter, the upper left corner point, the lower right corner point, and the upper right corner point when the target surface sheet is placed in the forward direction are respectively referred to as the upper left corner point, the lower right corner point, and the upper right corner point of the target surface sheet.

Wherein, the surface sheet central point refers to the central point of the target surface sheet.

The offset is the position offset of the vertex of the target surface sheet relative to the center point of the surface sheet. The position offset may be represented by an offset in the coordinate axis direction, for example, the position offset of the upper left corner point when the target surface sheet is placed in the forward direction with respect to the center point of the surface sheet is: and the offset of the upper left corner point of the target surface sheet relative to the center point of the surface sheet in the x-axis direction and the offset in the y-axis direction.

The target corner mark offset is the offset of the vertex of the target surface sheet relative to the center point of the surface sheet, wherein the offset is provided with corner mark information.

Here, the corner mark information is indication information of the upper left corner, the lower right corner, and the upper right corner of the target sheet. The corner mark information may be used to indicate which of the upper left corner, the lower right corner, and the upper right corner of the target sheet the target corner mark offset is with respect to the center point of the sheet. For example, in step 203, it is determined that the offsets of the four vertices of the target surface sheet and the target corner mark of the center point of the surface sheet are: offset with text labels "upper left corner", "lower left corner", "upper right corner", "lower right corner".

For convenience of understanding, please refer to fig. 3, fig. 3 is an illustrative diagram of the offset provided in the embodiment of the present application.

For example, a rectangular coordinate system is established by taking a lower left corner point of a single image of a surface to be corrected as a coordinate origin, taking a direction in which the lower left corner point points to the lower right corner point as an x-axis direction, and taking a direction in which the lower left corner point points to the upper left corner point as a y-axis direction. The coordinates of the surface single center point of the target surface single are (x 0, y 0), the upper left corner point of the target surface single is a1, the lower left corner point is a2, the lower right corner point is a3, the upper right corner point is a4, the coordinates of the upper left corner point a1 of the target surface single are (x 1, y 1), the coordinates of the lower left corner point a2 are (x 2, y 2), the coordinates of the lower right corner point a3 are (x 3, y 3), and the coordinates of the upper right corner point a4 are (x 4, y 4).

The position offset of the upper left corner point a1 of the target surface sheet relative to the center point of the surface sheet is the offset (x 0-x 1) of the upper left corner point a1 relative to the center point O of the surface sheet in the x-axis direction and the offset (y 0-y 1) in the y-axis direction.

The position offset of the lower left corner point a2 of the target surface sheet relative to the center point of the surface sheet is the offset (x 0-x 2) of the lower left corner point a2 relative to the center point O of the surface sheet in the x-axis direction and the offset (y 0-y 2) in the y-axis direction.

The position offset of the lower right corner point a3 of the target surface sheet relative to the center point of the surface sheet is the offset (x 0-x 3) of the lower right corner point a3 relative to the center point O of the surface sheet in the x-axis direction and the offset (y 0-y 3) in the y-axis direction.

The position offset of the upper right corner point a4 of the target surface sheet relative to the center point of the surface sheet is the offset (x 0-x 4) of the upper right corner point a4 relative to the center point O of the surface sheet in the x-axis direction and the offset (y 0-y 4) in the y-axis direction.

In some embodiments, the target corner mark offsets of the four vertexes of the target surface sheet and the central point of the surface sheet can be predicted through a pre-trained prediction model based on the surface sheet image to be corrected. "predicting the target corner mark offset between the four vertexes of the target surface sheet and the center point of the surface sheet based on the image of the surface sheet to be corrected through a pre-trained prediction model" will be described in detail in the following description of the prediction model, and will not be described herein again.

204. And determining the coordinates of the corner mark of the four vertexes of the target surface sheet according to the offset of the target corner mark and the coordinates of the center point of the surface sheet.

The coordinates of the corner marks of the four vertexes of the target surface sheet refer to coordinates with corner mark information. The corner mark information may be a character, a number, or a symbol, for example, the coordinates of the corner marks of the four vertexes of the target surface sheet determined in step 204 are coordinates with text marks of "upper left corner", "lower left corner", "upper right corner", and "lower right corner", respectively.

For example, if the target corner mark offset is "upper left corner" (. DELTA.x 1,. DELTA.y 1), "lower left corner" (. DELTA.x 2,. DELTA.y 2), "upper right corner" (. DELTA.x 3,. DELTA.y 3), "lower right corner" (. DELTA.x 4,. DELTA.y 4), and the coordinates of the center point of the surface sheet are (x 0, y 0), it can be determined that the coordinates of the corner mark at the four vertices of the target surface sheet are: "upper left corner" (. DELTA.x 1+ x0,. DELTA.y 1+ y 0), "lower left corner" (. DELTA.x 2+ x0,. DELTA.y 2+ y 0), "upper right corner" (. DELTA.x 3+ x0,. DELTA.y 3+ y 0), "lower right corner" (. DELTA.x 4+ x0,. DELTA.y 4+ y 0).

In step 204, after the coordinates of the corner point marks of the four vertexes of the target surface sheet are determined, the four vertexes of the target surface sheet are marked at the corresponding positions in the image of the surface sheet to be corrected, and the four vertexes are connected to form a quadrangle, so that the detection result of the target surface sheet is obtained. The detection result of the target surface sheet is used for indicating the area where the target surface sheet is located and coordinates of an upper left corner point, a lower right corner point and an upper right corner point of the target surface sheet, and the area where the quadrangle is located is the area where the target surface sheet is located.

205. And correcting the single image of the surface to be corrected based on the coordinates of the corner point marks of the four vertexes of the target surface single to obtain a corrected single image of the surface to be corrected.

The coordinates of the corner point marks of the four vertexes of the target surface sheet can reflect important information such as the inclination angle and the orientation of the target surface sheet.

In some embodiments, the single image of the surface to be corrected may be rotated according to the corner point mark coordinates of the four vertices of the target surface sheet, so that the upper left corner point, the lower right corner point, and the upper right corner point of the target surface sheet are restored to the state when placed in the forward direction, thereby obtaining the corrected rear single image.

Further, based on the detection result of the target surface sheet in step 204, the area image of the target surface sheet is captured from the single image of the surface to be corrected, and then the area image of the target surface sheet is rotated, so that the upper left corner point, the lower right corner point, and the upper right corner point of the target surface sheet are restored to the state when the target surface sheet is placed in the forward direction, and the corrected rear single image is obtained.

In some embodiments, based on the detection result of the target sheet in step 204, the region image of the target sheet is subjected to a perspective transformation operation to restore the correct orientation of the target sheet and restore the most original real element linear relationship in the image, such as restoring parallel relationship of opposite sides of the target sheet.

Due to the influence of the shooting angle, two originally parallel straight lines of the target surface sheet may be unparallel, that is, a quadrangle formed by the coordinates of the corner marks of the four vertices of the target surface sheet detected in step 204 is non-rectangular. The surface sheet is generally rectangular, the surface sheet image after perspective transformation and correction is also rectangular, and if the non-rectangular covered surface sheet area is directly converted into the corrected surface sheet image through perspective transformation, the bar code and characters in the surface sheet can be obviously distorted, which is not beneficial to subsequent identification. Therefore, the perspective transformation matrix of the image to be corrected is determined to perform perspective change by acquiring the minimum bounding rectangle of the four vertexes of the target surface sheet and based on the coordinates of the four vertexes of the minimum bounding rectangle and the coordinates of the corner mark of the four vertexes of the target surface sheet. Therefore, in some embodiments, step 205 may specifically include steps 2051 to 2053:

2051. and acquiring the minimum enclosing rectangle of the four vertexes of the target surface sheet and the coordinates of the four rectangle vertexes of the minimum enclosing rectangle based on the coordinates of the corner mark of the four vertexes of the target surface sheet.

The minimum bounding rectangle refers to a minimum bounding rectangle of four vertexes of the target surface sheet. As shown in fig. 4, fig. 4 is an explanatory schematic view of a minimum bounding rectangle provided in the embodiment of the present application, and in some cases, due to the influence of the shooting angle, two originally parallel straight lines of the target surface sheet may be non-parallel, and therefore perspective change is required to be performed, so that the two originally parallel straight lines in the corrected image are restored to be parallel, so as to facilitate subsequent recognition. In fig. 4, a rectangle dotted line frame indicates a minimum bounding rectangle.

And the four vertexes of the minimum bounding rectangle of the four vertexes of the target surface list are the four vertexes of the minimum bounding rectangle.

For example, after determining the coordinates of the corner point marks of the four vertices of the target surface sheet, the quadrangle of the corrected front single area of the target surface sheet may be determined according to the coordinates of the corner point marks of the four vertices of the target surface sheet, as shown by the quadrangle area surrounded by b1, b2, b3, and b4 in fig. 4. And solving and correcting the minimum circumscribed rectangle of the quadrangle of the front single area according to a minimum circumscribed rectangle algorithm of some polygons, such as a rotation method and a best-fit straight line algorithm, so as to obtain the minimum enclosing rectangles of four vertexes of the target surface list.

And obtaining the coordinates of the four vertexes of the minimum bounding rectangle after determining the minimum bounding rectangle of the four vertexes of the target surface list, and obtaining the coordinates of the four vertexes of the minimum bounding rectangle.

Here, the "polygonal minimum bounding rectangle algorithm" is merely an example, and actually, the "polygonal minimum bounding rectangle algorithm" may be the rotation method and the best-fit straight line algorithm mentioned herein, or may be another "polygonal minimum bounding rectangle algorithm", or may be a "polygonal minimum bounding rectangle algorithm" appearing in the future, where the "polygonal minimum bounding rectangle algorithm" is not limited herein.

2052. And determining a perspective transformation matrix of the image to be corrected based on the coordinates of the four rectangular vertexes and the coordinates of the corner mark of the four vertexes of the target surface sheet.

The essence of the Perspective Transformation (Perspective Transformation) is to project an image to a new viewing plane, and the general Transformation formula is:

where, (u, v) is the original image pixel coordinates, (x = x '/w', y = y '/w') is the image pixel coordinates after transformation. The perspective transformation matrix is as follows:

wherein the content of the first and second substances,

representing a linear transformation of the image; t2= [ a = ₁₃ a ₂₃ ] ^T For generating a linear transformation of the image; t3= [ a ] ₃₁ a ₃₂ ]And represents image translation.

Affine Transformation (Affine Transformation) can be understood as a special form of perspective Transformation. The mathematical expression of the perspective transformation is:

therefore, the coordinates of four pairs of pixel points corresponding to the perspective transformation are given, and the perspective transformation matrix can be obtained; otherwise, the perspective transformation matrix is given, and then the perspective transformation can be completed on the image or pixel point coordinates.

In the embodiment of the application, the coordinates of four pairs of pixel points before and after conversion are obtained based on the coordinates of four rectangular vertexes and the coordinates of corner point marks of four vertexes of a target surface list, and a perspective transformation matrix is obtained.

2053. And performing perspective transformation on the area of the minimum bounding rectangle in the single image of the surface to be corrected based on the perspective transformation matrix to obtain a corrected single image of the surface to be corrected.

Specifically, when perspective transformation is performed, the pixel point coordinate matrix of the region of the minimum bounding rectangle can be transformed based on the perspective transformation matrix to obtain the pixel point coordinate matrix after the perspective transformation of the region of the minimum bounding rectangle, and the pixel point coordinate matrix is used as a pixel point where the single region behind the correction is located, so that an image after the perspective transformation of the region of the minimum bounding rectangle is obtained.

In some embodiments, an area image of a minimum bounding rectangle may be cut from the single image of the surface to be corrected to perform perspective transformation, so as to obtain a corrected single image of the surface to be corrected.

The position corresponding relation between the four vertexes of the minimum enclosing rectangle and the four vertexes of the target surface list can be determined by a minimum point distance matching method, so that the corner point marking information of the four vertexes of the rectangle can be obtained. For example, a point of the four rectangular vertices that is the smallest distance from the top left corner point of (the four vertices of) the target surface sheet is calculated and marked as the top left corner point of the target surface sheet. And calculating a point with the minimum distance from the lower left corner point of (the four vertexes of) the target surface sheet in the four rectangular vertexes based on the coordinates of the four rectangular vertexes and the coordinates of the four vertexes of the target surface sheet, and marking the point as the lower left corner point of the target surface sheet. And calculating the point with the minimum distance from the upper right corner point of (the four vertexes of) the target surface sheet in the four rectangular vertexes, and marking the point as the upper right corner point of the target surface sheet. And calculating the point with the minimum distance from the lower right corner point of (the four vertexes of) the target surface sheet in the four rectangular vertexes, and marking the point as the lower right corner point of the target surface sheet.

By calculating the minimum enclosing rectangle of the four vertexes of the target surface sheet and carrying out perspective transformation based on the area of the minimum enclosing rectangle, on the first hand, the problem that when a quadrangle formed by the corner point mark coordinates of the four vertexes of the target surface sheet is a non-rectangle, the area of the surface sheet covered by the non-rectangle is directly transformed into a corrected rectangular surface single image by means of perspective transformation, so that bar codes and characters in the surface sheet are obviously distorted and are not easy to identify subsequently is solved. In the second aspect, because the perspective transformation is performed based on the region of the minimum bounding rectangle, the number of pixels in the region of the minimum bounding rectangle is relatively small, and the data processing amount during the perspective transformation is reduced under the condition of ensuring that the bar codes and characters in the menu are not obviously distorted. In the third aspect, the rotation operation is carried out on the surface sheet image to be corrected in the process of carrying out perspective transformation on the area based on the minimum bounding rectangle, so that the orientation information of the target surface sheet can be determined after the perspective transformation.

From the above, it can be seen that the corner mark coordinates of the four vertexes of the target surface sheet are determined by obtaining and according to the coordinates of the surface single center point of the target surface sheet and the target corner mark offset between the four vertexes of the target surface sheet and the surface single center point in the surface single image to be corrected, and the surface single image to be corrected is corrected to obtain the corrected back single image. On one hand, because the corner mark coordinates carry corner mark information, the positions of an upper left corner point, a lower right corner point and an upper right corner point of the target surface sheet can be determined, and information such as the orientation, the inclination angle and the like of the target surface sheet can be further reflected; therefore, the single image of the surface to be corrected is corrected based on the coordinates of the angular point marks, and the target surface can be ensured to restore to the correct orientation, so that the single image of the surface to be corrected is accurately corrected. On the other hand, because the information such as the orientation, the inclination angle and the like of the target surface sheet can be reflected only by predicting the coordinates of the corner point marks of the four vertexes of the target surface sheet, the information such as the orientation, the inclination angle and the like of the target surface sheet can be obtained while the target surface sheet is detected, and the information such as the orientation, the inclination angle and the like can be judged without additionally increasing a classification branch, so that the classified data processing amount is reduced; therefore, compared with the algorithm for detecting the added classification branches in the prior art, the method and the device for identifying the bill of surface information improve the locating efficiency of the bill of surface information and further improve the identification efficiency of the bill of surface information.

The following describes a training process of the prediction model provided in the embodiment of the present application. As shown in fig. 5, fig. 5 is a schematic flowchart of a training process of a predictive model provided in an embodiment of the present application, and the training process of the predictive model includes the following steps 501 to 506.

For ease of understanding, the network structure of the prediction model in the embodiment of the present application is described first. As shown in fig. 6, fig. 6 is a schematic network structure diagram of a prediction model provided in an embodiment of the present application, where the prediction model includes a feature extraction module, a first prediction module, and a second prediction module.

The device comprises a feature extraction module.

And the feature extraction module is used for extracting features of the single face image to obtain a feature map of the single face image. The feature extraction module takes the single image as an input, and performs one or more operations including but not limited to convolution, pooling and the like on the single image so as to realize feature extraction on the single image to obtain a feature map of the single image (i.e. image features of the single image). For example, the feature extraction module takes the single image of the surface to be corrected as an input and outputs a target feature map of the single image of the surface to be corrected.

Illustratively, the feature extraction module may include a backbone Network backbone part such as a ResNet Network (Residual Network), a mobile Network, and the like, for extracting information in the picture.

Further, the feature extraction module may further include a neck part for integrating information extracted by the backbone network, where the neck part is used for integrating features in the backbone network at different scales, so as to better utilize the information extracted by the backbone network, thereby ensuring that information prediction can be performed accurately subsequently. Illustratively, the hack part is composed of a Feature Pyramid Network (FPN) or a bidirectional feature pyramid network (bipfn), etc.

And (II) a first prediction module.

And the first prediction module is used for predicting the coordinates of the single surface center point of the single surface in the single surface image according to the feature graph output by the feature extraction model. For example, the first prediction module takes the target feature map of the surface single image to be corrected as an input and outputs the coordinates of the surface single center point of the target surface single in the surface single image to be corrected.

And (III) a second prediction module.

And the second prediction module is used for predicting the offset of four vertexes corresponding to the single surface and the single central point of the surface respectively according to the feature graph output by the feature extraction model and carrying corner mark information (referred to as corner mark offset for short). For example, the second prediction module takes the target feature map of the surface sheet image to be corrected as an input, and outputs the target corner mark offsets between four vertexes of the target surface sheet and the center point of the surface sheet.

Further, the second prediction module is further configured to determine coordinates of the four vertices of the surface sheet according to offsets of the four vertices of the surface sheet relative to the center point of the surface sheet and the coordinates of the center point of the surface sheet.

After the first prediction module determines the coordinates of the center point of the sheet, the second prediction module determines the offset of the corner mark, based on the offset of the corner mark and the coordinates of the center point of the sheet, the coordinates of each vertex of the sheet (including an upper left corner point, a lower right corner point and an upper right corner point) can be calculated, and right corner mark information is marked, so that the coordinates of the corner mark of the sheet are obtained, as shown in fig. 6, (1), (2), (3) and (4) in fig. 6 are the corner mark information respectively and are used for indicating the upper left corner point, the lower right corner point and the upper right corner point of the sheet respectively.

501. And acquiring a sample surface single image.

The sample bill image is an image containing a sample bill, such as an image containing an express bill. The manner of obtaining the sample surface single image is similar to the manner of obtaining the surface single image to be corrected in step 201, and reference may be specifically made to the description of step 201, which is not described herein again.

The actual offsets of the four top points of the sample surface sheet and the central point of the sample surface sheet and the actual central point coordinate of the sample surface sheet are marked on the sample surface sheet image.

The actual offset is the offset of the vertex of the sample surface sheet with the corner mark information relative to the center point of the sample surface sheet.

The actual center point coordinates are the coordinates of the center point of the sample face single label.

502. And performing feature extraction on the sample surface single image through a feature extraction module in the prediction model to be trained to obtain a sample feature map of the sample surface single image.

The sample feature map refers to spatial feature information of a sample surface single image.

Specifically, the sample plane single image is input into the prediction model to be trained, so that the feature extraction module in the prediction model to be trained performs one or more operations including but not limited to convolution, pooling and the like on the sample plane single image based on the backbone network part to obtain the image information of the sample plane single image, thereby realizing feature extraction on the sample plane single image to obtain the sample feature image of the sample plane single image.

Further, after the backbone network performs one or more of operations such as convolution and pooling on the sample surface single image, the extracted image information is integrated based on the neck part, so as to obtain a sample characteristic map of the sample surface single image.

503. And predicting based on the sample characteristic diagram through a first prediction module in a prediction model to be trained to obtain the coordinate of the prediction central point of the sample surface list.

The predicted center point coordinate is the coordinate of the center point of the sample surface sheet obtained by prediction.

Specifically, prediction is carried out according to the sample characteristic diagram through a first prediction module in a prediction model to be trained, and the prediction central point coordinate of the sample surface is obtained.

504. And predicting based on the sample characteristic diagram through a second prediction module in the prediction model to be trained to obtain the prediction offset of the four top points of the sample surface sheet and the center point of the sample surface sheet.

The predicted offset is the offset of each vertex of the sample surface sheet relative to the center point of the sample surface sheet, which is obtained through prediction and has corner mark information.

Specifically, prediction is performed according to the sample characteristic diagram through a second prediction module in the prediction model to be trained, and prediction offsets of four top points of the sample surface unit and the center point of the sample surface unit are obtained.

505. And determining the training loss of the prediction model to be trained based on the actual offset, the prediction offset, the actual central point coordinate and the prediction central point coordinate.

In some embodiments, a training loss of the predictive model to be trained may be determined in conjunction with the vertex offset loss of the second prediction module and the centerpoint prediction loss of the first prediction module. At this time, step 505 may specifically include steps 5051A to 5053A:

5051A, based on the actual central point coordinate and the predicted central point coordinate, determining the central point prediction loss of the first prediction module.

Illustratively, the prediction model is correspondingly provided with a second loss function, so that the first prediction module can learn the position relation of four top points of the surface sheet and the center point of the surface sheet in the image. The second loss function is set corresponding to the predicted center point coordinates of the prediction model output. In the training process, the value of the second loss function is the central point prediction loss, and the central point prediction loss of the first prediction module can be obtained through calculation by substituting the actual central point coordinate and the prediction central point coordinate into the second loss function. In the embodiment of the present application, a specific function type of the second loss function is not limited, and the second loss function may be, for example, a focal loss function.

5052A, determining a vertex offset penalty for the second prediction module based on the actual offset and the prediction offset.

Illustratively, the prediction model is correspondingly provided with a first loss function so that the second prediction module can learn the single-face vertex information in the image. The first penalty function is set in correspondence with a prediction offset output by the prediction model. In the training process, the value of the first loss function is the vertex offset loss, and the vertex offset loss of the second prediction module can be calculated by substituting the prediction offset and the actual offset into the first loss function. In the embodiment of the present application, a specific function type of the first loss function is not limited, and the first loss function may be a wing loss function, for example.

5053A, determining the training loss of the prediction model to be trained according to the vertex offset loss and the central point prediction loss.

Illustratively, the vertex offset penalty and the center point prediction penalty may be directly summed as the training penalty for the predictive model to be trained.

Alternatively, the vertex offset loss and the central point prediction loss may be added according to a certain weight to be used as the training loss of the prediction model to be trained.

In some embodiments, in addition to the vertex offset loss of the second prediction module and the central point prediction loss of the first prediction module, the training loss of the prediction model to be trained may be further determined by combining the circumscribed rectangle prediction loss of the single minimum circumscribed rectangle of the sample plane. In this case, step 505 is preceded by: determining coordinates of four vertices of the sample mask based on the predicted offsets; determining the mark coordinates of the predicted vertexes of the maximum circumscribed rectangle of the sample surface list based on the four vertexes of the sample surface list; and the sample surface single image is also marked with the actual vertex mark coordinates of the maximum circumscribed rectangle of the sample surface single. Step 505 may specifically include steps 5051B to 5054B:

the step "determining the coordinates of the four vertices of the sample surface sheet based on the prediction offset" is similar to the step 204 of determining the coordinates of the corner marks of the four vertices of the target surface sheet, and the step "determining the coordinates of the predicted vertex marks of the maximum circumscribed rectangle of the sample surface sheet based on the four vertices of the sample surface sheet" is similar to the step 2051, which may refer to the descriptions of the step 204 and the step 2051, respectively, and is not repeated here.

5051B, based on the actual central point coordinate and the predicted central point coordinate, determining the central point prediction loss of the first prediction module.

Step 5051B is implemented identically to step 5051A described above and is not described in further detail herein.

5052B, determining a vertex offset penalty for the second prediction module based on the actual offset and the prediction offset.

Step 5052B is implemented identically to step 5052A described above and is not described in further detail herein.

5053B, and determining the external rectangle prediction loss of the sample surface sheet based on the actual vertex mark coordinates and the prediction vertex mark coordinates.

The actual vertex marking coordinates refer to the offset of the vertex of the sample surface single maximum circumscribed rectangle with the corner point marking information, which is obtained by marking, relative to the surface single center point. Here, the corner mark information is indicative of the upper left corner, lower right corner, and upper right corner of the sample plane single maximum circumscribed rectangle.

The predicted vertex mark coordinates refer to the offset of the vertex of the sample surface single maximum circumscribed rectangle with corner mark information relative to the surface single center point. Here, the corner mark information is indicative of the upper left corner, the lower right corner, and the upper right corner of the single maximum bounding rectangle of the sample plane.

Illustratively, the prediction model is correspondingly provided with a third loss function, so that the second prediction module can learn the position information of the single four vertexes in the image. The third loss function is set corresponding to the predicted vertex marker coordinates of the prediction model output. In the training process, the value of the third loss function is the prediction loss of the circumscribed rectangle, and the actual vertex mark coordinate and the prediction vertex mark coordinate are substituted into the third loss function, so that the prediction loss of the circumscribed rectangle of the sample surface list can be calculated and obtained and used as the prediction loss of the circumscribed rectangle of the second prediction module. In the embodiment of the present application, a specific function type of the third loss function is not limited, and the third loss function may be, for example, a GiouLoss loss function.

5054B, determining the training loss based on the vertex offset loss, the centroid prediction loss, and the bounding rectangle prediction loss.

Illustratively, the vertex offset loss, the center point prediction loss, and the circumscribed rectangle prediction loss may be directly added as the training loss of the prediction model to be trained.

Or, the vertex offset loss, the central point prediction loss and the circumscribed rectangle prediction loss may be added according to a certain weight to be used as the training loss of the prediction model to be trained.

In some embodiments, the training loss of the prediction model to be trained may be determined in combination with the face single vertex prediction loss of the second prediction module and the center point prediction loss of the first prediction module. In this case, step 505 is preceded by: determining prediction corner coordinates of four vertices of the sample sheet based on the prediction offsets. Step 505 may specifically include steps 5051C to 5053C:

the step of "determining the coordinates of the predicted corner points of the four vertices of the sample surface sheet based on the prediction offset" is similar to the step of determining the coordinates of the corner point marks of the four vertices of the target surface sheet in the step 204, and reference may be made to the description of the step 204, which is not repeated here.

5051C, based on the actual central point coordinate and the predicted central point coordinate, determining the central point prediction loss of the first prediction module.

Step 5052C is implemented identically to step 5051A described above and is not described in further detail herein.

5052C, and determining the face single-vertex prediction loss of the second prediction module based on the actual corner coordinates and the prediction corner coordinates.

And the sample surface single image is also marked with the actual corner point coordinates of four vertexes of the sample surface single.

The actual corner coordinates refer to the coordinates of each vertex of the sample face single label.

Illustratively, the prediction model is correspondingly provided with a fourth loss function, so that the second prediction module can learn the position information of the single four vertexes in the image. The fourth loss function is set in correspondence with the coordinates of the prediction corner outputted by the prediction model. In the training process, the value of the fourth loss function is the surface single-vertex prediction loss, and the surface single-vertex prediction loss of the second prediction module can be obtained through calculation by substituting the actual corner point coordinates and the prediction corner point coordinates into the fourth loss function. In the embodiment of the present application, a specific function type of the fourth loss function is not limited, and the fourth loss function may be a wing loss function, for example.

5053C, determining the training loss based on the face single-vertex prediction loss and the center point prediction loss.

Illustratively, the face single vertex prediction loss and the center point prediction loss may be directly added as a training loss of the prediction model to be trained.

Or, the face single vertex prediction loss and the center point prediction loss may be added according to a certain weight to be used as the training loss of the prediction model to be trained.

In some embodiments, the training loss of the predictive model to be trained may be determined in combination with the face single-vertex prediction loss of the second predictive module, the center point prediction loss of the first predictive module, and the bounding rectangle prediction loss. In this case, step 505 further includes: determining the coordinates of the prediction corner points of four vertexes of the sample surface sheet based on the prediction offset; and determining the mark coordinates of the predicted vertexes of the maximum bounding rectangle of the sample surface list based on the four vertexes of the sample surface list. Step 505 may specifically include steps 5051D to 5054D:

5051D, determining the center point prediction loss of the first prediction module based on the actual center point coordinate and the prediction center point coordinate.

Step 5051D is implemented identically to step 5051A described above and is not described in further detail herein.

5052D, and determining the face single-vertex prediction loss of the second prediction module based on the actual corner coordinates and the prediction corner coordinates.

Step 5052D is implemented identically to step 5052C described above and is not described in further detail herein.

5053D, determining the circumscribed rectangle prediction loss of the sample sheet based on the actual vertex mark coordinates and the predicted vertex mark coordinates.

Step 5053D is implemented the same as step 5053B described above and is not described here in detail.

5054D, determining the training loss based on the face single-vertex prediction loss, the center point prediction loss and the circumscribed rectangle prediction loss.

Illustratively, the face single-vertex prediction loss, the center point prediction loss and the circumscribed rectangle prediction loss may be directly added as the training loss of the prediction model to be trained.

Or, the surface single-vertex prediction loss, the central point prediction loss and the circumscribed rectangle prediction loss may be added according to a certain weight to be used as the training loss of the prediction model to be trained.

506. And adjusting parameters of the prediction model to be trained based on the training loss until the parameters meet the preset training stopping condition, so as to obtain the trained prediction model.

Wherein, the preset training stopping condition can be set according to the actual requirement. For example, when the training loss is smaller than a preset value, or the training loss is not substantially changed, that is, the difference between the training losses corresponding to adjacent training times is smaller than the preset value; or when the iteration number of the prediction model to be trained reaches the maximum iteration number.

In particular, the model parameters of the second prediction module may be adjusted for back propagation based on the face single vertex prediction penalty. Alternatively, back propagation based on the center point prediction loss adjusts the model parameters of the first prediction module. Or performing back propagation based on the prediction loss of the circumscribed rectangle to adjust the model parameters of the second prediction module. Alternatively, the model parameters of the second prediction module are adjusted by back-propagation based on the vertex offset penalty.

After the trained prediction model is obtained, acquiring the target corner mark offsets of the four top points of the target surface sheet and the central point of the surface sheet based on the image of the surface sheet to be corrected through a second prediction module in the trained prediction model; and acquiring the coordinates of the surface single center point of the target surface single in the surface single image to be corrected based on the surface single image to be corrected through a first prediction module in a trained prediction model.

Specifically, the surface single image to be corrected is input into a trained prediction model, and the surface single image to be corrected is subjected to one or more operations including but not limited to convolution, pooling and the like through a feature extraction module, so that the target feature map of the surface single image to be corrected (namely, the image features of the surface single image) is obtained by performing feature extraction on the surface single image to be corrected.

And then, predicting according to the target characteristic image of the single image of the surface to be corrected through a second prediction module to obtain the target corner mark offsets of four vertexes of the target single surface and the central point of the single surface. And predicting according to the target characteristic graph of the single image of the surface to be corrected through a first prediction module to obtain the coordinates of the single center point of the surface of the target surface in the single image of the surface to be corrected.

Further, in an actual application scene, after the corrected single image is obtained, the single image can be identified based on the corrected single image to obtain the single information of the target single.

The menu information is recorded information of the target surface, such as characters, bar codes, two-dimensional codes, and the like.

Illustratively, the to-be-corrected surface single image is an express surface single image in an express sorting scene, and the express surface single image in the express surface image can be detected and corrected through the above steps 201 to 205, so as to obtain a corrected express surface single image. And identifying based on the express bill image to obtain the bill information of the express bill, such as the sender, the receiver, the product weight, the price and other related information.

Further, in an actual application scenario, a surface single image of the express to be sorted may be acquired in step 201 to serve as a surface single image to be corrected. And detecting and correcting the single-surface image of the express to be sorted according to the mode of the steps 202 to 205 to obtain a corrected single-surface image of the express. And identifying based on the corrected express bill image to obtain the bill information of the express bill, such as the sender, the receiver, the product weight, the price, the destination, the departure place and other related information. And finally, sorting the express to be sorted based on the order information of the express order.

The method comprises the steps 201 to 205, and the method comprises the steps of detecting and correcting the image of the surface sheet to be sorted, determining and correcting the inclined angle points of the image based on the angle point mark coordinates of the predicted surface sheet directly, and judging the inclined angle points of the image without adding a classification branch, so that the positioning efficiency of the surface sheet is improved, and the information identification efficiency of the surface sheet is further improved.

In order to better implement the method for correcting a single planar image in the embodiment of the present application, based on the method for correcting a single planar image, an embodiment of the present application further provides a single planar image correction apparatus, as shown in fig. 7, which is a schematic structural diagram of an embodiment of the single planar image correction apparatus in the embodiment of the present application, and the single planar image correction apparatus 700 includes:

an obtaining unit 701, configured to obtain a single image of a surface to be corrected;

a coordinate prediction unit 702, configured to obtain coordinates of a surface single center point of a target surface single in the surface single image to be corrected;

the coordinate prediction unit 702 is further configured to obtain offsets of the four vertices of the target surface sheet and the target corner mark of the center point of the surface sheet;

the coordinate prediction unit 702 is further configured to determine corner mark coordinates of four vertices of the target surface sheet according to the target corner mark offset and the coordinates of the center point of the surface sheet, where the corner mark coordinates are used to indicate an upper left corner point, a lower right corner point, and an upper right corner point of the target surface sheet;

the correcting unit 703 is configured to correct the single image of the surface to be corrected based on the coordinates of the corner marks of the four vertices of the target surface single, so as to obtain a corrected single image of the surface to be corrected.

In some embodiments, the coordinate prediction unit 702 is specifically configured to:

acquiring minimum enclosing rectangles of the four vertexes of the target surface sheet and coordinates of the four rectangular vertexes of the minimum enclosing rectangles based on the corner mark coordinates of the four vertexes of the target surface sheet;

determining a perspective transformation matrix of the image to be corrected based on the coordinates of the four rectangular vertexes and the coordinates of the corner mark of the four vertexes of the target surface sheet;

and performing perspective transformation on the area of the minimum bounding rectangle in the single image of the surface to be corrected based on the perspective transformation matrix to obtain a corrected single image of the surface to be corrected.

In some embodiments, the correction unit 703 is specifically configured to:

based on the perspective transformation matrix, carrying out perspective transformation on the area of the minimum bounding rectangle in the single image of the surface to be corrected to obtain an image after the perspective transformation;

determining corner mark information of the four rectangular vertexes based on the coordinates of the four rectangular vertexes and the coordinates of the four vertexes of the target surface sheet;

and rotating the image after perspective transformation based on the angular point mark information to obtain a corrected single image of the image to be corrected.

and acquiring the coordinates of the single center point of the target surface in the single image of the surface to be corrected based on the single image of the surface to be corrected through a first prediction module in a trained prediction model.

and acquiring the mark offsets of the four top points of the target surface sheet and the target corner point of the central point of the surface sheet based on the image of the surface sheet to be corrected through a second prediction module in a trained prediction model.

In some embodiments, the facial single image rectification apparatus further includes a training unit (not shown in the figures), which is specifically configured to:

acquiring a sample surface single image, wherein the sample surface single image is marked with actual offsets of four top points of the sample surface single and a central point of the sample surface single and actual central point coordinates of the sample surface single;

performing feature extraction on the sample surface single image through a feature extraction module in a prediction model to be trained to obtain a sample feature map of the sample surface single image;

predicting based on the sample characteristic diagram through a first prediction module in a prediction model to be trained to obtain a prediction central point coordinate of the sample surface sheet;

predicting based on the sample characteristic diagram through a second prediction module in a prediction model to be trained to obtain prediction offsets of four top points of the sample surface sheet and a central point of the sample surface sheet;

determining the training loss of the prediction model to be trained based on the actual offset, the prediction offset, the actual center point coordinate and the prediction center point coordinate;

and adjusting parameters of the prediction model to be trained based on the training loss until the parameters meet the preset training stopping condition, so as to obtain the trained prediction model.

In some embodiments, the sample face sheet image is further labeled with actual vertex label coordinates of a maximum bounding rectangle of the sample face sheet, and the training unit is specifically configured to:

determining coordinates of four vertices of the sample mask based on the predicted offsets;

determining the mark coordinates of the predicted vertexes of the maximum circumscribed rectangle of the sample surface list based on the coordinates of the four vertexes of the sample surface list;

determining a center point prediction loss of the first prediction module based on the actual center point coordinates and the predicted center point coordinates;

determining a vertex offset penalty for the second prediction module based on the actual offset and the predicted offset;

determining the prediction loss of the circumscribed rectangle of the sample surface sheet based on the actual vertex mark coordinates and the predicted vertex mark coordinates;

determining the training loss based on the vertex offset loss, the center point prediction loss, and the bounding rectangle prediction loss.

In some embodiments, the facial single image rectification apparatus further includes an identification unit (not shown in the figures), the identification unit being specifically configured to:

and identifying based on the corrected single image to obtain the single information of the target single.

In some embodiments, the obtaining unit 701 is specifically configured to:

acquiring a surface single image of an express to be sorted to serve as a surface single image to be corrected;

in some embodiments, the facial single image rectification apparatus further comprises a sorting unit (not shown in the figures), which is specifically configured to:

the identification is carried out based on the corrected single image to obtain the single information of the target single, and then the method further comprises the following steps:

and sorting the express to be sorted based on the bill information.

In a specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and the specific implementation of the above units may refer to the foregoing method embodiments, which are not described herein again.

Since the facial single image correction apparatus can execute the steps in the facial single image correction method according to any embodiment of the present application corresponding to fig. 1 to 6, the beneficial effects that can be realized by the facial single image correction method according to any embodiment of the present application corresponding to fig. 1 to 6 can be realized, which are detailed in the foregoing description and will not be repeated herein.

In addition, in order to better implement the method for correcting the single-sided image in the embodiment of the present application, based on the method for correcting the single-sided image, an electronic device is further provided in the embodiment of the present application, referring to fig. 8, fig. 8 shows a schematic structural diagram of the electronic device in the embodiment of the present application, specifically, the electronic device in the embodiment of the present application includes a processor 801, and when the processor 801 is used to execute a computer program stored in a memory 802, the processor 801 is configured to implement steps of the method for correcting the single-sided image in any embodiment corresponding to fig. 1 to 6; alternatively, the processor 801 is configured to implement the functions of the units in the corresponding embodiment of fig. 7 when executing the computer program stored in the memory 802.

Illustratively, a computer program may be partitioned into one or more modules/units, which are stored in the memory 802 and executed by the processor 801 to implement the embodiments of the present application. One or more modules/units may be a series of computer program instruction segments capable of performing certain functions, the instruction segments being used to describe the execution of a computer program in a computer device.

The electronic device may include, but is not limited to, a processor 801, a memory 802. Those skilled in the art will appreciate that the illustration is merely an example of an electronic device and does not constitute a limitation of an electronic device, and may include more or less components than those illustrated, or combine some components, or different components, for example, an electronic device may further include an input output device, a network access device, a bus, etc., and the processor 801, the memory 802, the input output device, the network access device, etc., are connected via the bus.

The Processor 801 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being the control center for the electronic device and the various interfaces and lines connecting the various parts of the overall electronic device.

The memory 802 may be used to store computer programs and/or modules, and the processor 801 may implement various functions of the computer device by running or executing the computer programs and/or modules stored in the memory 802 and invoking data stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, video data, etc.) created according to the use of the electronic device, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described single-sided image correction apparatus, the electronic device and the corresponding units thereof may refer to the description of the single-sided image correction method in any embodiment corresponding to fig. 1 to 6, and are not repeated herein.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute steps in the method for correcting a single-sided image in any embodiment corresponding to fig. 1 to 6 in the present application, and specific operations can refer to descriptions of the method for correcting a single-sided image in any embodiment corresponding to fig. 1 to 6, which are not repeated herein.

Wherein the computer-readable storage medium may include: read Only Memory (ROM), random Access Memory (RAM), magnetic or optical disks, and the like.

Since the instructions stored in the computer-readable storage medium can execute the steps in the method for correcting a single-sided image in any embodiment corresponding to fig. 1 to 6 of the present application, the beneficial effects that can be achieved by the method for correcting a single-sided image in any embodiment corresponding to fig. 1 to 6 of the present application can be achieved, which are detailed in the foregoing description and will not be repeated herein.

The method, the apparatus, the electronic device, and the computer-readable storage medium for correcting a facial single image provided by the embodiments of the present application are described in detail above, and specific examples are applied herein to explain the principles and embodiments of the present application, and the description of the embodiments is only used to help understand the method and the core concept of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A method of facial single image correction, the method comprising:

acquiring a single image of a surface to be corrected;

2. The method for correcting a facial single image according to claim 1, wherein the determining a corrected single image of the facial single image to be corrected based on the coordinates of the corner mark of the four vertices of the target facial single image comprises:

determining a perspective transformation matrix of the single image of the surface to be corrected based on the coordinates of the four rectangular vertexes and the coordinates of the corner mark of the four vertexes of the target surface single;

3. The method for correcting the facial single image according to claim 1, wherein the obtaining coordinates of a facial single center point of a target facial single in the facial single image to be corrected comprises:

acquiring the coordinates of the surface single center point of a target surface single in the surface single image to be corrected based on the surface single image to be corrected through a first prediction module in a trained prediction model;

the obtaining of the target corner mark offsets between the four vertices of the target surface sheet and the center point of the surface sheet includes:

4. The method of facial single image correction according to claim 3, further comprising:

predicting based on the sample characteristic diagram through a first prediction module in a prediction model to be trained to obtain a prediction central point coordinate of the sample surface list;

determining the training loss of the prediction model to be trained based on the actual offset, the prediction offset, the actual central point coordinate and the prediction central point coordinate;

and adjusting parameters of the prediction model to be trained based on the training loss until the parameters meet the preset training stopping condition, and obtaining the trained prediction model.

5. The method according to claim 4, wherein the sample surface sheet image is further labeled with actual vertex labeling coordinates of a maximum circumscribed rectangle of the sample surface sheet, and prediction is performed by a second prediction module in the prediction model to be trained based on the sample feature map to obtain prediction offsets between four vertices of the sample surface sheet and a center point of the sample surface sheet, and then the method further comprises:

determining coordinates of four vertices of the sample sheet based on the predicted offsets;

the determining a training loss of the prediction model to be trained based on the actual offset, the predicted offset, the actual centroid coordinate and the predicted centroid coordinate includes:

6. The method of facial single image correction according to any one of claims 1-5, further comprising:

and identifying based on the corrected single image to obtain the single information of the target surface single.

7. The method for correcting the facial single image according to claim 6, wherein the acquiring the facial single image to be corrected comprises:

and sorting the express to be sorted based on the bill information.

8. A facial single image correction apparatus, comprising:

the coordinate prediction unit is further used for obtaining target corner mark offsets of four top points of the target surface sheet and a central point of the surface sheet;

and the correcting unit is used for correcting the single image of the surface to be corrected based on the corner mark coordinates of the four vertexes of the target surface single to obtain a corrected single image of the surface to be corrected.

9. An electronic device comprising a processor and a memory, the memory having a computer program stored therein, the processor executing the method for facial single image correction according to any one of claims 1 to 7 when calling the computer program in the memory.

10. A computer-readable storage medium, having stored thereon a computer program which is loaded by a processor to perform the steps of the method for facial single image correction according to any one of claims 1 to 7.