CN112950528A - Certificate posture determining method, model training method, device, server and medium - Google Patents

Certificate posture determining method, model training method, device, server and medium Download PDF

Info

Publication number
CN112950528A
CN112950528A CN201911249237.2A CN201911249237A CN112950528A CN 112950528 A CN112950528 A CN 112950528A CN 201911249237 A CN201911249237 A CN 201911249237A CN 112950528 A CN112950528 A CN 112950528A
Authority
CN
China
Prior art keywords
certificate
determining
image
angle
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911249237.2A
Other languages
Chinese (zh)
Inventor
陈志军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alipay Labs Singapore Pte Ltd
Original Assignee
Alipay Labs Singapore Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alipay Labs Singapore Pte Ltd filed Critical Alipay Labs Singapore Pte Ltd
Priority to CN201911249237.2A priority Critical patent/CN112950528A/en
Publication of CN112950528A publication Critical patent/CN112950528A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/243—Aligning, centring, orientation detection or correction of the image by compensating for image skew or non-uniform image deformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20164—Salient point detection; Corner detection

Abstract

The embodiment of the specification discloses a certificate attitude determination method, a model training method, a device, a server and a medium. The scheme provides an effective method for determining the posture deviation of the certificate, and the method is simple to implement and easy to operate.

Description

Certificate posture determining method, model training method, device, server and medium
Technical Field
The embodiment of the specification relates to the technical field of computers, in particular to a certificate posture determining method, a model training method, a device, a server and a medium.
Background
With the continuous development of scientific technology, electronic technology has also been developed rapidly, and the functions that can be realized by electronic equipment are also more and more abundant. In the prior art, certificate detection is applied to many scenarios, such as verifying the identity of a user by detecting certificate information of the user, automatically filling in bank card information of the user by detecting a bank card, and the like. When the certificate is detected, a user is generally required to scan or photograph the certificate by using electronic equipment, and then the obtained image is subjected to certificate detection, and the certificate in the image is generally inclined, so that subsequent certificate detection is inaccurate. Therefore, how to accurately determine the pose of the certificate in the certificate image is crucial to certificate detection.
Disclosure of Invention
The embodiment of the specification provides a certificate posture determining method, a model training method, a device, a server and a medium.
In a first aspect, an embodiment of the present specification provides a method for determining a document pose, including:
carrying out corner point detection on a certificate image, and determining the corner point position of each corner point of a certificate area in the certificate image, wherein the certificate area is an area where a target certificate is located;
determining a multidimensional offset parameter according to the corner point position of each corner point of the certificate area, wherein the multidimensional offset parameter is used for representing the offset degree of the target certificate along the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the space coordinate system in which the target certificate is located;
and determining the space posture of the target certificate based on the multi-dimensional shift parameters.
In a second aspect, an embodiment of the present specification provides a method for training a certificate gesture recognition model, including:
acquiring a plurality of certificate images;
according to the certificate attitude determination method provided by the first aspect, a certificate space attitude corresponding to each certificate image in the plurality of certificate images is determined;
marking the certificate attitude of each certificate image based on the certificate space attitude corresponding to each certificate image;
training an initial certificate posture recognition model based on each certificate image after certificate posture marking to obtain a trained certificate posture recognition model, wherein the trained certificate posture recognition model is used for carrying out certificate posture recognition on an input certificate image and outputting a corresponding certificate space posture.
In a third aspect, embodiments of the present specification provide a document pose determination apparatus, the apparatus comprising:
the angle point determining module is used for detecting the angle points of the certificate image and determining the position of each angle point of a certificate area in the certificate image, wherein the certificate area is an area where a target certificate is located;
the shift parameter determining module is used for determining multidimensional shift parameters according to the corner point position of each corner point of the certificate area, and the multidimensional shift parameters are used for representing the shift degrees of the target certificate in the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the space coordinate system where the target certificate is located;
and the gesture determining module is used for determining the spatial gesture of the target certificate based on the multi-dimensional shift parameters.
In a fourth aspect, an embodiment of the present specification provides a method for training a certificate gesture recognition model, including:
the acquisition module is used for acquiring a plurality of certificate images;
an angle determining module, configured to determine, according to the certificate posture determining method provided in the first aspect, a corresponding certificate image in each of the plurality of certificate images
A spatial attitude;
the angle marking module is used for marking the certificate posture of each certificate image based on the corresponding space posture of each certificate image;
and the model training module is used for training the initial certificate posture recognition model based on each certificate image marked with the certificate posture to obtain a trained certificate posture recognition model, and the trained certificate posture recognition model is used for carrying out certificate posture recognition on the input certificate image and outputting a corresponding space posture.
In a fifth aspect, embodiments of the present specification provide a server, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor performing the steps of any one of the above methods.
In a sixth aspect, the present specification provides a computer readable storage medium, on which a computer program is stored, and the computer program is used for implementing the steps of any one of the above methods when executed by a processor.
The embodiment of the specification has the following beneficial effects:
in the method for determining the document pose provided in the embodiment of the present specification, the corner point position of each corner point of a document region included in a document image is obtained, and according to the corner point position of each corner point, a multidimensional offset parameter for representing the offset degree of a target document along the horizontal axis direction, the longitudinal axis direction and the vertical direction is determined, and then according to the multidimensional offset parameter, the spatial pose of the target document is determined. In addition, the angle point position of the target certificate in the certificate image is easy to obtain, so that the certificate posture is determined through the angle point position, and the method is convenient to operate and easy to realize.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the specification. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart of a method for determining a document pose provided by a first aspect of embodiments of the present description;
FIG. 2 is a schematic view of corner points of a document region in a document image shown in an embodiment of the present description;
FIG. 3 is a schematic view of a spatial coordinate system of a target document shown in an embodiment of the present description;
FIG. 4 is a flowchart of a method for training a credential pose recognition model provided by a second aspect of embodiments of the present description;
FIG. 5 is a schematic diagram of a document pose determination apparatus provided by a third aspect of embodiments of the present description;
FIG. 6 is a schematic diagram of a credential pose recognition model training apparatus provided in a fourth aspect of embodiments of the present description;
fig. 7 is a schematic diagram of a server provided in a fifth aspect of an embodiment of the present disclosure.
Detailed Description
In order to better understand the technical solutions, the technical solutions of the embodiments of the present specification are described in detail below with reference to the drawings and specific embodiments, and it should be understood that the specific features of the embodiments and embodiments of the present specification are detailed descriptions of the technical solutions of the embodiments of the present specification, and are not limitations of the technical solutions of the present specification, and the technical features of the embodiments and embodiments of the present specification may be combined with each other without conflict.
In a first aspect, an embodiment of the present specification provides a method for determining a document pose, and as shown in fig. 1, is a flowchart of the method for determining a document pose provided by the embodiment of the present specification, and the method includes the following steps:
step S11: carrying out corner point detection on a certificate image, and determining the corner point position of each corner point of a certificate area in the certificate image, wherein the certificate area is an area where a target certificate is located;
step S12: determining a multidimensional offset parameter according to the corner point position of each corner point of the certificate area, wherein the multidimensional offset parameter is used for representing the offset degree of the target certificate along the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the space coordinate system in which the target certificate is located;
step S13: and determining the space posture of the target certificate based on the multi-dimensional shift parameters.
The certificate posture determining method provided by the embodiment of the specification can be applied to an electronic equipment terminal and can also be applied to a server. For example, the method can be applied to a mobile phone, a computer, and a certificate capture device capable of capturing certificate images, and can also scan or photograph certificates through an electronic device terminal, and send the obtained images to a corresponding server, so that the server processes the received images to obtain the certificate postures, which is not limited herein.
The target certificate can be an identity card, a bank card and the like of a user, and the certificate image can be obtained by scanning or shooting the target certificate through electronic equipment. The certificate image comprises a certificate area, the certificate area is an area where image information of the target certificate is located, namely the certificate area does not comprise a background image, and the edge of the certificate area is overlapped with the edge of the image information of the target certificate. When the certificate is scanned or shot, in order to ensure that the certificate area in the obtained certificate image is convenient for subsequent posture determination and certificate identification, the position and the size of the certificate area in the certificate image can be specified by displaying prompt information on a scanning or shooting interface. For example, a rectangular frame can be displayed on a scanning or shooting interface, and prompt information is displayed to prompt a user to adjust the position of the certificate, so that the acquired certificate is located in the rectangular frame; or for the identity card and other documents, a rectangular frame can be displayed in the scanning or shooting interface, and a head portrait area is displayed in the rectangular frame to prompt the user to adjust the position of the document, so that the acquired identity card is located in the rectangular frame, and the head portrait on the identity card is located in the head portrait area. Of course, the prompting message may also include other information, such as information for prompting the front or back of the certificate collection document, which is not necessarily listed here.
After the certificate image is obtained, the corner position of each corner point of the certificate area in the certificate image is determined through step S11. The corner position can be determined in various ways, for example, by Harris corner detection, CNN (Convolutional Neural Network) corner positioning detection, and the like. As shown in fig. 2, the four corner points of the certificate area are A, B, C, D, and corner point information of the four corner points of the certificate area is obtained by performing corner point detection on the certificate image, specifically, the corner point position may be position information such as a corner point coordinate.
When scanning or shooting the target certificate, under the ideal condition, the target certificate is placed on a horizontal plane, and the lens of the camera for scanning or shooting is parallel to the target certificate, so that the certificate area in the obtained certificate image is rectangular, namely the upper side and the lower side of the certificate image are the same in length, and the left side and the lower side of the certificate image are the same in length. However, due to the fact that the target document is placed at a different position or the lens and the target document form a certain angle, the document area in the document image is shifted or inclined relative to the document area which is ideally shot, and thus the document area in the obtained document image is deformed and is not in a standard rectangle shape. In the embodiment of the present specification, the offset parameter of the offset of the target document is determined through step S12.
The deviation of the target certificate is relative to a spatial coordinate system in which the target certificate is located, the spatial coordinate system includes a horizontal axis direction (X-axis direction), a vertical axis direction (Y-axis direction) and a vertical direction (Z-axis direction), and since the target certificate may have inclination or deviation in each direction, in this embodiment of the present description, in order to accurately obtain the posture of the target certificate, it is necessary to determine multidimensional deviation parameters of the target certificate in each direction. The multidimensional offset parameter can be parameters such as offset proportion and offset displacement for representing the offset degree of the target certificate relative to each direction of the space coordinate system.
After the multi-dimensional shift parameters are acquired, the spatial pose of the target document is further determined. The spatial pose of the target certificate may be an included angle between the target certificate and each direction of the spatial coordinate system, or an offset angle along each direction of the spatial coordinate system, which is not limited herein. Determining the spatial pose of the target document may be selected according to actual needs, for example, by looking up from a pre-constructed mapping between offset parameters and angles, or by simulation calculations, etc. After the spatial posture of the target certificate is obtained, image correction or certificate anti-counterfeiting operation and the like can be carried out on the certificate image according to the posture angle.
Therefore, the method for determining the posture of the certificate provided by the embodiment of the specification can determine the multidimensional offset parameters of the target certificate along each direction through the angular point position of the target certificate in the certificate image, and further determine the space posture of the target certificate, and an effective method for determining the posture of the certificate is provided.
In the embodiment of the present specification, when determining the multidimensional offset parameter, the following manner may be adopted: determining the length of each edge of the certificate area according to the position of each corner point of the certificate area; determining a length ratio between any two sides of the document area based on the length of each side of the document area, and determining the multi-dimensional offset parameter based on the length ratio between any two sides
In a specific implementation process, as shown in fig. 2, after the corner positions of the four corners A, B, C, D of the certificate area are determined, lengths of the four sides AB, CD, AC, and BD may be calculated according to the corner positions of the four corners, and taking the corner positions as corner coordinates as an example, the side length may be calculated in an euclidean distance or other manners, which is not limited herein. It will be appreciated that the lengths of the edges of the document region may reflect the spatially offset condition of the target document. As shown in fig. 3, a spatial coordinate system is established for the target document, and if the target document is rotated relative to the X-axis, the lengths of the AB edge and the CD edge of the document area in the document image are different; if the target certificate rotates relative to the Y axis, the lengths of the AC side and the BD side of the certificate area in the certificate image are different; if the target document is rotated about the Z axis, the ratio of the AC side to the BD side of the document area in the document image will be different from the actual ratio of the two sides of the target document.
Therefore, in the embodiment of the present specification, when determining the multidimensional offset parameter based on the length of each edge, the following manner may be adopted: the certificate area comprises a first edge and a second edge which are opposite to each other, and a third edge and a fourth edge which are opposite to each other; determining a first offset parameter of the target document along the transverse axis based on a first ratio between a first length of the first edge and a second length of the second edge; determining a second offset parameter of the target document along the longitudinal axis based on a second ratio between a third length of the third side and a fourth length of the fourth side; determining a third offset parameter of the target document along the vertical axis based on a third ratio between the first length of the first edge and a third length of the third edge.
In a specific implementation process, the first side and the second side may correspond to the AB side and the CD side in fig. 2, respectively, and the third side and the fourth side may correspond to the AC side and the BD side in fig. 2, respectively, or the first side and the second side may correspond to the AC side and the BD side in fig. 2, respectively, and the third side and the fourth side may correspond to the AB side and the CD side in fig. 2, respectively, which is not limited herein. For convenience of description, the first side corresponds to the side AB, the second side corresponds to the side CD, the third side corresponds to the side AC, and the fourth side corresponds to the side BD. The first length of the first side is LABThe second length of the second side is LCDAnd a third side having a third length LACThe fourth side has a fourth length LBD. Then, the first ratio is LAB/LCDThe second ratio is LAC/LBDThe third ratio is LAB/LAC。
In this embodiment of the present description, the first ratio obtained by calculation may be directly used as a first offset parameter, the second ratio may be used as a second offset parameter, and the third ratio may be used as a third offset parameter, and each ratio may be further processed to obtain a corresponding offset parameter. In a specific implementation, the following methods can be adopted to process each ratio: determining a first side length ratio, a second side length ratio and an adjacent side length ratio corresponding to the target certificate in a preset corresponding relation between the certificate and the side length ratio; normalizing the first ratio based on the first side-to-side length ratio to determine the first offset parameter; normalizing the second ratio based on the second length-to-edge ratio to determine a second offset parameter; and normalizing the third ratio based on the side length ratios of the two adjacent sides to determine the third offset parameter.
It should be noted that the preset correspondence between the certificate and the side length ratio refers to a correspondence between the certificate and the actual side length ratio of the two sides of the certificate. For example, the preset correspondence between the certificate and the side length ratio may be a correspondence between a certificate name, a certificate type, a first side length ratio of the certificate, a second side length ratio of the certificate, and a side length ratio of two adjacent sides of the certificate. Based on the certificate name or the certificate type, the corresponding side length ratio can be determined.
Generally speaking, the shape of the document is a rectangle, and therefore, of two sets of opposite sides included in the document, the two opposite sides included in each set of opposite sides are equal in length, that is, the first opposite side length ratio and the second opposite side length ratio of the rectangular document are both 1. The side length ratio of two adjacent sides of the certificate may be different according to the sizes of various certificates, for example, the side length ratio of two adjacent sides of the identity card is 1.58. Of course, if the certificate is not rectangular, the first pair of side length ratios, the second pair of side length ratios, and the adjacent two side length ratios of the certificate may be sequentially obtained, and the side length ratios may be associated with the certificate. Further, if the certificate is in other polygonal shapes, the side length ratio of each two adjacent sides can be determined according to needs and associated with the certificate, which is not limited herein.
The preset corresponding relation between the certificate and the side length ratio can be stored in the electronic equipment or the server, so that when the target certificate is scanned or shot, the first side length ratio, the second side length ratio and the adjacent side length ratio of the target certificate can be determined in the corresponding relation according to the name or the type of the target certificate. Further, normalization processing is carried out on the first ratio, the second ratio and the third ratio according to the first pair of side length ratio, the second pair of side length ratio and the adjacent side length ratio of the target certificate, and the result after normalization processing is used as a corresponding offset parameter. Taking the first pair edge length ratio, the second pair edge length ratio and the adjacent two edge length ratios of the target certificate as 1,1 and 1.58 as examples, the obtained first offset parameter is LAB/LCD-1, the second offset parameter being LAC/LBD-1, third offset LAB/LAC-1. In the embodiment of the present specification, the first side length ratio, the second side length ratio, and the adjacent side length ratio of the target document can reflect a state of the target document when there is no offset, that is, there is no offset included angle between the target document and the X axis, the Y axis, and the Z axis. Due to the fact thatTherefore, the offset parameters obtained after the normalization processing can represent the offset of the target certificate relative to the non-offset state in three axial directions. It should be understood that, for documents with equal edges, the normalized offset parameter has a value range of [ -1,1 [ -1 [ ]]If the range is exceeded, the obtained certificate image is considered to be unusable, and the user may be prompted to rescan or shoot.
Further, after obtaining the offset parameter, determining the spatial pose of the target document based on the offset parameter, in an embodiment of the present specification, the following may be adopted to determine the spatial pose of the target document: and determining the attitude angle corresponding to the target certificate based on the preset mapping relation between the offset parameter and the attitude angle, wherein the attitude angle comprises a pitch angle, a yaw angle and a roll angle of the target certificate.
In a specific implementation process, the preset mapping relationship between the offset parameter and the attitude angle may be set according to actual needs, for example, the mapping relationship in the embodiment of the present specification may include a mapping relationship between a first offset parameter and a pitch angle, a mapping relationship between a second offset parameter and a yaw angle, and a mapping relationship between a third offset parameter and a roll angle. The pitch angle (pitch) is an angle formed by rotation around the X axis, the yaw angle (yaw) is an angle formed by rotation around the Y axis, and the roll angle (roll) is an angle formed by rotation around the Z axis.
The preset mapping relationship between the offset parameter and the posture angle can be obtained through statistics, for example, the offset angle of the current posture of the target certificate relative to each direction is known, then each side length of a certificate area in a certificate image corresponding to the current posture is measured, the corresponding offset parameter is determined according to the ratio of the two sides, and the offset parameter is associated with the posture angle (namely, the offset angle), so that the preset mapping relationship between the offset parameter and the posture angle is obtained through traversing the offset angle.
In addition, the preset mapping relation between the offset parameter and the attitude angle is a mapping relation constructed by performing three-dimensional affine transformation on a reference image of the target certificate, wherein the reference image is an image of the target certificate without offset relative to the space coordinate system. In this embodiment of the present specification, the reference image may be a document image acquired in an ideal state, that is, a document region in the reference image has no offset with respect to each direction. The three-dimensional affine transformation comprises the transformation of the certificate image such as translation, rotation, scaling and the like in space. In a specific implementation process, affine transformation can be performed on a reference image to obtain an image with any angle, angles transformed in all directions along a spatial coordinate system are recorded, side length ratios between two adjacent sides and opposite sides of a certificate area in the transformed image are obtained, corresponding offset parameters are calculated, and the offset parameters are associated with the recorded angles to obtain a preset mapping relation between the offset parameters and the attitude angles.
Certainly, the spatial pose of the target certificate may be determined in other manners besides the above manners, for example, after the corner point position of each corner point of the certificate region is obtained, affine transformation may be performed according to the corner point position of each corner point, the certificate region in the certificate image is mapped into a standard image, the standard image is fixed in size and is an image that has no offset relative to each direction of a spatial coordinate system, then the standard image is rotated based on the rotation matrix to obtain the same pose as the target certificate, and the rotation angle is recorded, and the rotation angle may be used as the certificate pose of the target certificate.
Further, according to the scheme in the embodiment of the specification, after the posture angle is obtained, the certificate area in the certificate image can be corrected according to the posture angle, so that subsequent certificate identification or certificate anti-counterfeiting detection is facilitated. Taking certificate anti-counterfeiting detection as an example, some certificates have anti-counterfeiting marks, and the anti-counterfeiting marks can generate different colors under different angles, for example, a first color is presented at a first angle, and a second color is presented at a second angle, so that certificate anti-counterfeiting can be performed according to the corresponding relation between the posture angle of the certificate and the color of the anti-counterfeiting mark.
In addition, in the embodiment of the specification, the certificate image can be marked through the obtained posture angle, so that the certificate posture recognition model can be trained. That is, after determining the pose angle of the target document, the method further comprises: performing angle marking on the target image based on the attitude angle; and constructing a training sample set based on the angle-labeled target image, training an initial certificate posture recognition model based on the training sample set, and obtaining a trained certificate posture recognition model, wherein the trained certificate posture recognition model is used for performing certificate posture recognition on an input image and outputting a corresponding certificate posture angle.
In the specific implementation process, the posture angle of the target certificate corresponding to the certificate image is obtained through the method, the angle marking is carried out on the certificate image through the posture angle, namely, marking operation is carried out, and the certificate image after the angle marking is used as a training sample. It should be understood that the constructed initial document pose recognition model can be trained by performing angle labeling on a large number of document images by the method to obtain a certain number of training sample sets. The initial certificate gesture recognition model can be selected according to actual needs, for example, a CNN model, an RNN (Recurrent Neural Network) model, or the like is used. The input of the initial certificate gesture recognition model can be a certificate image to be gesture recognized, and the output is the gesture angle of the certificate corresponding to the input image. After the trained certificate posture angle model is obtained, the trained certificate posture recognition model can be applied to various scenes, such as certificate posture detection, certificate posture correction and the like.
In a second aspect, based on the same inventive concept, an embodiment of the present specification provides a method for training a certificate gesture recognition model, please refer to fig. 4, where the method includes:
step S41: acquiring a plurality of certificate images;
step S42: according to the certificate attitude determination method provided by the first aspect, a certificate space attitude corresponding to each certificate image in the plurality of certificate images is determined;
step S43: marking the certificate posture of each certificate image based on the corresponding space posture of each certificate image;
step S44: training an initial certificate posture recognition model based on each certificate image after certificate posture marking to obtain a trained certificate posture recognition model, wherein the trained certificate posture recognition model is used for carrying out certificate posture recognition on an input certificate image and outputting a corresponding space posture.
In the embodiment of the specification, the plurality of certificate images can be images obtained by scanning or shooting various certificates, and can also be images in sources, networks or various databases. By the method for determining the certificate attitude provided by the first aspect of the embodiment of the present specification, each certificate image is processed to obtain the certificate spatial attitude of each certificate image, and the certificate spatial attitude is marked on the corresponding certificate image to form a model training sample. Taking the space posture of the certificate as an example of the posture angle of the certificate, the pitch angle, the yaw angle and the roll angle of the certificate which deviates along the space coordinate system can be marked in the corresponding certificate image.
The initial certificate gesture recognition model may be selected according to actual needs, for example, a CNN model, an RNN model, or the like is adopted, and the number of layers of the network model and the number of neurons in each layer may be set according to actual needs, which is not limited herein. For the initial certificate posture recognition model, the input is a certificate image, and the output is a space posture corresponding to the certificate image, such as a pitch angle, a yaw angle and a roll angle. In the training process, model parameters can be continuously adjusted according to the space posture marked by the certificate image until the accuracy of the output of the model reaches a preset value, or the training times of the model reach preset times, so that a trained certificate posture angle model is obtained. The trained certificate gesture recognition model is applied to various scenes, such as certificate gesture detection, certificate gesture correction, certificate anti-counterfeiting and the like.
With regard to the above method, specific implementation of each step can refer to the detailed description of the document pose determination method provided in the first aspect of the embodiments of the present specification, and will not be elaborated herein.
In a third aspect, based on the same inventive concept, an embodiment of the present specification provides a document posture determining apparatus, please refer to fig. 5, the apparatus includes:
the corner point determining module 51 is configured to perform corner point detection on a certificate image, and determine a corner point position of each corner point of a certificate area in the certificate image, where the certificate area is an area where a target certificate is located;
the offset parameter determining module 52 is configured to determine a multidimensional offset parameter according to the corner point position of each corner point of the certificate region, where the multidimensional offset parameter is used to represent offset degrees of the target certificate along a horizontal axis direction, a vertical axis direction, and a vertical axis direction of a spatial coordinate system where the target certificate is located;
and the posture determining module 53 is configured to determine a spatial posture of the target document based on the multi-dimensional shift parameters.
In an alternative implementation, the offset parameter determination module 52 is configured to:
determining the length of each edge of the certificate area according to the position of each corner point of the certificate area;
and determining a length ratio between any two sides of the certificate area based on the length of each side of the certificate area, and determining the multi-dimensional offset parameter based on the length ratio between any two sides.
In an optional implementation manner, the certificate region includes a first edge and a second edge that are opposite to each other, and a third edge and a fourth edge that are opposite to each other, and the offset parameter determining module 52 is configured to:
determining a first offset parameter of the target document along the transverse axis based on a first ratio between a first length of the first edge and a second length of the second edge;
determining a second offset parameter of the target document along the longitudinal axis based on a second ratio between a third length of the third side and a fourth length of the fourth side;
determining a third offset parameter of the target document along the vertical axis based on a third ratio between the first length of the first edge and a third length of the third edge.
In an alternative implementation, the offset parameter determination module 52 is configured to:
determining a first side length ratio, a second side length ratio and an adjacent side length ratio corresponding to the target certificate in a preset corresponding relation between the certificate and the side length ratio;
normalizing the first ratio based on the first side-to-side length ratio to determine the first offset parameter;
normalizing the second ratio based on the second length-to-edge ratio to determine a second offset parameter;
and normalizing the third ratio based on the side length ratios of the two adjacent sides to determine the third offset parameter.
In an alternative implementation, the pose determination module 53 is configured to:
and determining the attitude angle corresponding to the target certificate based on the preset mapping relation between the offset parameter and the attitude angle, wherein the attitude angle comprises a pitch angle, a yaw angle and a roll angle of the target certificate.
In an optional implementation manner, the preset mapping relationship between the offset parameter and the pose angle is a mapping relationship constructed by performing three-dimensional affine transformation on a reference image of the target certificate, where the reference image is an image of the target certificate without offset with respect to the spatial coordinate system.
In an alternative implementation, the apparatus further includes:
the angle marking module is used for marking the angle of the certificate image based on the attitude angle;
and the model training module is used for constructing a training sample set based on the angle-labeled certificate image, training a preset certificate posture recognition model based on the training sample set to obtain a trained certificate posture recognition model, wherein the trained certificate posture recognition model is used for carrying out certificate posture recognition on an input image and outputting a corresponding certificate posture angle.
With regard to the above-mentioned apparatus, the specific functions of the respective modules have been described in detail in the embodiment of the document posture determination method provided in the embodiment of the present specification, and will not be elaborated here.
In a fourth aspect, based on the same inventive concept, an embodiment of the present specification provides a device for training a credential pose recognition model, please refer to fig. 6, where the device includes:
the acquisition module 61 is used for acquiring a plurality of certificate images;
an angle determining module 62, configured to determine a certificate space pose corresponding to each certificate image in the plurality of certificate images according to the certificate pose determining method provided in the first aspect;
the angle labeling module 63 is configured to label the certificate posture of each certificate image based on the certificate space posture corresponding to each certificate image;
and the model training module 64 is used for training the initial certificate posture recognition model based on each certificate image marked with the certificate posture to obtain a trained certificate posture recognition model, and the trained certificate posture recognition model is used for carrying out certificate posture recognition on the input certificate image and outputting the corresponding certificate space posture.
With regard to the above-mentioned apparatus, the specific functions of the respective modules have been described in detail in the embodiments of the document pose determination method and the document pose recognition model training method provided in the embodiments of the present specification, and will not be described in detail here.
In a fifth aspect, based on the same inventive concept as the method for determining a document pose and the method for training a document pose recognition model in the foregoing embodiments, an embodiment of the present specification further provides a server, as shown in fig. 7, including a memory 404, a processor 402, and a computer program stored in the memory 404 and executable on the processor 402, wherein the processor 402 implements the steps of any one of the method for determining a document pose and the method for training a document pose recognition model when executing the program.
Where in fig. 7 a bus architecture (represented by bus 400) is shown, bus 400 may include any number of interconnected buses and bridges, with bus 400 linking together various circuits including one or more processors, represented by processor 402, and memory, represented by memory 404. The bus 400 may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface 406 provides an interface between the bus 400 and the receiver 401 and transmitter 403. The receiver 401 and the transmitter 403 may be the same element, i.e., a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 402 is responsible for managing the bus 400 and general processing, while the memory 404 may be used for storing data used by the processor 402 in performing operations.
In a sixth aspect, based on the inventive concepts of the method for determining a document pose and the method for training a document pose recognition model in the foregoing embodiments, the present specification further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of any one of the methods for determining a document pose and the method for training a document pose recognition model in the foregoing embodiments.
The description has been presented with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the description. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present specification have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all changes and modifications that fall within the scope of the specification.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present specification without departing from the spirit and scope of the specification. Thus, if such modifications and variations of the present specification fall within the scope of the claims of the present specification and their equivalents, the specification is intended to include such modifications and variations.

Claims (18)

1. A method of document pose determination, the method comprising:
carrying out corner point detection on a certificate image, and determining the corner point position of each corner point of a certificate area in the certificate image, wherein the certificate area is an area where a target certificate is located;
determining a multidimensional offset parameter according to the corner point position of each corner point of the certificate area, wherein the multidimensional offset parameter is used for representing the offset degree of the target certificate along the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the space coordinate system in which the target certificate is located;
and determining the space posture of the target certificate based on the multi-dimensional shift parameters.
2. The method of claim 1, wherein determining a multi-dimensional offset parameter according to the corner position of each corner point of the certificate region comprises:
determining the length of each edge of the certificate area according to the position of each corner point of the certificate area;
and determining a length ratio between any two sides of the certificate area based on the length of each side of the certificate area, and determining the multi-dimensional offset parameter based on the length ratio between any two sides.
3. The method of claim 2, wherein the document area includes a first edge and a second edge that are opposite each other, and a third edge and a fourth edge that are opposite each other, and wherein determining the length ratio between any two edges of the document area based on the length of each edge of the document area and determining the multi-dimensional offset parameter based on the length ratio between any two edges comprises:
determining a first offset parameter of the target document along the transverse axis based on a first ratio between a first length of the first edge and a second length of the second edge;
determining a second offset parameter of the target document along the longitudinal axis based on a second ratio between a third length of the third side and a fourth length of the fourth side;
determining a third offset parameter of the target document along the vertical axis based on a third ratio between the first length of the first edge and a third length of the third edge.
4. The method of claim 3, wherein determining the multi-dimensional offset parameter based on a ratio of lengths between any two edges comprises:
determining a first side length ratio, a second side length ratio and an adjacent side length ratio corresponding to the target certificate in a preset corresponding relation between the certificate and the side length ratio;
normalizing the first ratio based on the first side-to-side length ratio to determine the first offset parameter;
normalizing the second ratio based on the second length-to-edge ratio to determine a second offset parameter;
and normalizing the third ratio based on the side length ratios of the two adjacent sides to determine the third offset parameter.
5. The method of claim 1, the determining a spatial pose of the target document based on the multi-dimensional shift parameters, comprising:
and determining the attitude angle corresponding to the target certificate based on the preset mapping relation between the offset parameter and the attitude angle, wherein the attitude angle comprises a pitch angle, a yaw angle and a roll angle of the target certificate.
6. The method of claim 5, wherein the preset mapping relationship between the offset parameter and the pose angle is a mapping relationship constructed by performing three-dimensional affine transformation on a reference image of the target document, wherein the reference image is an image of the target document without offset relative to the spatial coordinate system.
7. The method of claim 5, after determining the spatial pose of the target document, the method further comprising:
performing angle marking on the certificate image based on the attitude angle;
and constructing a training sample set based on the angle-labeled certificate image, training an initial certificate posture recognition model based on the training sample set to obtain a trained certificate posture recognition model, wherein the trained certificate posture recognition model is used for carrying out certificate posture recognition on an input image and outputting a corresponding certificate posture angle.
8. A method for training a certificate gesture recognition model, the method comprising:
acquiring a plurality of certificate images;
the document pose determination method of any of claims 1-6, determining a document spatial pose corresponding to each document image of the plurality of document images;
marking the certificate attitude of each certificate image based on the certificate space attitude corresponding to each certificate image;
training an initial certificate posture recognition model based on each certificate image after certificate posture marking to obtain a trained certificate posture recognition model, wherein the trained certificate posture recognition model is used for carrying out certificate posture recognition on an input certificate image and outputting a corresponding certificate space posture.
9. A document pose determination apparatus, the apparatus comprising:
the angle point determining module is used for detecting the angle points of the certificate image and determining the position of each angle point of a certificate area in the certificate image, wherein the certificate area is an area where a target certificate is located;
the shift parameter determining module is used for determining multidimensional shift parameters according to the corner point position of each corner point of the certificate area, and the multidimensional shift parameters are used for representing the shift degrees of the target certificate in the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the space coordinate system where the target certificate is located;
and the gesture determining module is used for determining the spatial gesture of the target certificate based on the multi-dimensional shift parameters.
10. The apparatus of claim 9, the offset parameter determination module to:
determining the length of each edge of the certificate area according to the position of each corner point of the certificate area;
and determining a length ratio between any two sides of the certificate area based on the length of each side of the certificate area, and determining the multi-dimensional offset parameter based on the length ratio between any two sides.
11. The apparatus of claim 10, the document area including first and second sides opposite one another and third and fourth sides opposite one another, the offset parameter determination module to:
determining a first offset parameter of the target document along the transverse axis based on a first ratio between a first length of the first edge and a second length of the second edge;
determining a second offset parameter of the target document along the longitudinal axis based on a second ratio between a third length of the third side and a fourth length of the fourth side;
determining a third offset parameter of the target document along the vertical axis based on a third ratio between the first length of the first edge and a third length of the third edge.
12. The apparatus of claim 11, the offset parameter determination module to:
determining a first side length ratio, a second side length ratio and an adjacent side length ratio corresponding to the target certificate in a preset corresponding relation between the certificate and the side length ratio;
normalizing the first ratio based on the first side-to-side length ratio to determine the first offset parameter;
normalizing the second ratio based on the second length-to-edge ratio to determine a second offset parameter;
and normalizing the third ratio based on the side length ratios of the two adjacent sides to determine the third offset parameter.
13. The apparatus of claim 9, the pose determination module to:
and determining the attitude angle corresponding to the target certificate based on the preset mapping relation between the offset parameter and the attitude angle, wherein the attitude angle comprises a pitch angle, a yaw angle and a roll angle of the target certificate.
14. The apparatus of claim 13, wherein the preset mapping relationship between the offset parameter and the pose angle is a mapping relationship constructed by performing three-dimensional affine transformation on a reference image of the target document, wherein the reference image is an image of the target document without offset relative to the spatial coordinate system.
15. The apparatus of claim 13, the apparatus further comprising:
the angle marking module is used for marking the angle of the certificate image based on the attitude angle;
and the model training module is used for constructing a training sample set based on the angle-labeled certificate image, training a preset certificate posture recognition model based on the training sample set to obtain a trained certificate posture recognition model, wherein the trained certificate posture recognition model is used for carrying out certificate posture recognition on an input image and outputting a corresponding certificate posture angle.
16. A document pose recognition model training apparatus, the apparatus comprising:
the acquisition module is used for acquiring a plurality of certificate images;
an angle determination module for determining a document spatial pose corresponding to each document image of the plurality of document images according to the document pose determination method of any one of claims 1-6;
the angle marking module is used for marking the certificate attitude of each certificate image based on the certificate space attitude corresponding to each certificate image;
and the model training module is used for training the initial certificate posture recognition model based on each certificate image marked with the certificate posture to obtain a trained certificate posture recognition model, and the trained certificate posture recognition model is used for carrying out certificate posture recognition on the input certificate image and outputting the corresponding certificate space posture.
17. A server comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method of any one of claims 1 to 8 when executing the program.
18. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.
CN201911249237.2A 2019-12-09 2019-12-09 Certificate posture determining method, model training method, device, server and medium Pending CN112950528A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911249237.2A CN112950528A (en) 2019-12-09 2019-12-09 Certificate posture determining method, model training method, device, server and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911249237.2A CN112950528A (en) 2019-12-09 2019-12-09 Certificate posture determining method, model training method, device, server and medium

Publications (1)

Publication Number Publication Date
CN112950528A true CN112950528A (en) 2021-06-11

Family

ID=76225142

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911249237.2A Pending CN112950528A (en) 2019-12-09 2019-12-09 Certificate posture determining method, model training method, device, server and medium

Country Status (1)

Country Link
CN (1) CN112950528A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113743396A (en) * 2021-08-31 2021-12-03 支付宝(杭州)信息技术有限公司 Method and device for identifying injection attack in certificate identification process
CN114897999A (en) * 2022-04-29 2022-08-12 美的集团(上海)有限公司 Object pose recognition method, electronic device, storage medium, and program product
CN117237682A (en) * 2023-10-17 2023-12-15 支付宝(杭州)信息技术有限公司 Certificate verification method and device, storage medium and electronic equipment

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113743396A (en) * 2021-08-31 2021-12-03 支付宝(杭州)信息技术有限公司 Method and device for identifying injection attack in certificate identification process
CN113743396B (en) * 2021-08-31 2023-11-10 支付宝(杭州)信息技术有限公司 Method and device for identifying injection attack in certificate identification process
CN114897999A (en) * 2022-04-29 2022-08-12 美的集团(上海)有限公司 Object pose recognition method, electronic device, storage medium, and program product
CN114897999B (en) * 2022-04-29 2023-12-08 美的集团(上海)有限公司 Object pose recognition method, electronic device, storage medium, and program product
CN117237682A (en) * 2023-10-17 2023-12-15 支付宝(杭州)信息技术有限公司 Certificate verification method and device, storage medium and electronic equipment

Similar Documents

Publication Publication Date Title
CN109754426B (en) Method, system and device for verifying camera calibration parameters
US10726580B2 (en) Method and device for calibration
CN108701234A (en) Licence plate recognition method and cloud system
CN109685913B (en) Augmented reality implementation method based on computer vision positioning
WO2022170844A1 (en) Video annotation method, apparatus and device, and computer readable storage medium
JP6688277B2 (en) Program, learning processing method, learning model, data structure, learning device, and object recognition device
CN112950528A (en) Certificate posture determining method, model training method, device, server and medium
CN111145271B (en) Method and device for determining accuracy of camera parameters, storage medium and terminal
CN106570907B (en) Camera calibration method and device
CN111860527A (en) Image correction method, image correction device, computer device, and storage medium
CN113329179B (en) Shooting alignment method, device, equipment and storage medium
CN111667398A (en) Image processing method, apparatus and computer-readable storage medium
CN110443245A (en) Localization method, device and the equipment of a kind of license plate area under unrestricted scene
CN111681186A (en) Image processing method and device, electronic equipment and readable storage medium
CN110084743A (en) Image mosaic and localization method based on more air strips starting track constraint
CN109949249B (en) Cylindrical image correction method and system
US20220084291A1 (en) Augmented reality display method and device for chip structure, and readable storage medium
CN115187612A (en) Plane area measuring method, device and system based on machine vision
CN114998447A (en) Multi-view vision calibration method and system
CN111260574B (en) Seal photo correction method, terminal and computer readable storage medium
CN115937003A (en) Image processing method, image processing device, terminal equipment and readable storage medium
CN112365399B (en) Deep learning-based panoramic stitching method and system for fan blade images
JP2016038790A (en) Image processor and image feature detection method thereof, program and device
CN114037822A (en) Method and system for detecting driving license
JP5975484B2 (en) Image processing device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination